Category: WebXPRT

Interesting questions

on September 26, 2013

We’ve had a couple of interesting questions about WebXPRT this week.

The first question was about the Face detect test in WebXPRT. One person, having noticed that changing the version of Firefox affected the WebXPRT score on a particular device, asked whether the test used the JavaScript Canvas element. The answer is yes, the Face detection test does use the Canvas element. It is based on the JavaScript library by Dr. Liu Liu.

As we have discussed in the past, the software stack on a device affects the benchmark scores. WebXPRT is a HTML5 benchmark and uses elements in the HTML5 specification, such as Canvas. Browsers implement HTML in their JavaScript engines, whose performance depends on the OS and the underlying platform. So, WebXPRT scores are influenced by the browser and OS, as well as the platform.

The second question was whether it is possible to run WebXPRT without an Internet connection. Generally speaking, the answer to that is no. WebXPRT is a hosted application, and to run the official version, you must be able to connect to the WebXPRT servers.

However, community members can download the WebXPRT source and configure local servers that will run WebXPRT, if they desire. Note: As we discussed in Sources, any published results must be from the version hosted at webxprt.com.

Thanks for the questions and keep experimenting!

Eric

Comment on this post in the forums

Posted in HTML5, WebXPRT |

Sources

By Eric Hale

on September 19, 2013

If you’ve checked out the MobileXPRT and WebXPRT pages recently, you’ve probably noticed that the number of results has started to grow. The results are coming from three sources:

Internal testing at PT.
Results submitted by the public.
Results published on the Web. We link back to the source from these results. Results published on-line include results in reports PT publishes for clients and reviews by other parties.

While we are excited about the growing number of results, we do sanity check them. We compare the results with other runs for the same device when available, or with similar devices if not.

The source code for the benchmarks is available, and we encourage experimentation. However, it should go without saying that valid runs must come from the builds of the benchmarks the development community has published. We can’t compare two results generated by different builds.

That being said, if you change the code and get an interesting result, by all means do contact us. You may have discovered something that we’ll want to include in a future version

Keep the results coming and keep experimenting!

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT, BenchmarkXPRT development community, MobileXPRT, TouchXPRT, WebXPRT |

Endurance

By Bill Catchings

on September 5, 2013

In There’s always something new to measure, we proposed several possible tests for the next version of WebXPRT. Of those, battery life testing generated the most interest.

Battery life testing poses a number of challenges. It’s not as simple as making WebXPRT loop. The biggest challenge is that different devices take different measures when the battery runs low. These measures range from dimming the screen, to stopping the hard disk, to totally shutting down the device. While these are perfectly reasonable, they are out of the benchmark’s control. Worse, most current browsers offer no way of knowing that these measures even happen nor do they offer good ways of querying the device to find out the state of its battery. We want to make sure that our approach does not unfairly advantage one device over another and gives a fair and accurate measure.

Because WebXPRT is a hosted application, we are looking at one of the other XPRT benchmarks for our first attempt at adding battery life to an existing benchmark. MobileXPRT seems to be the best fit. It runs on Android, which has a functional API for monitoring and managing power events, and the diversity of the Android ecosystem forces the benchmark to deal with a greater range of devices and OS configurations than TouchXPRT.

We are trying a number of approaches, and we have made some progress. We will discuss what we have learned in the next few weeks.

Our hope is that what we learn from MobileXPRT will better equip us to add battery life testing to WebXPRT.

Have any thoughts or comments? Post to the forums or e-mail benchmarkxprtsupport@principledtechnologies.com to let us know.

-Bill

Comment on this post in the forums

Posted in TouchXPRT, WebXPRT |

Fragments

By Eric Hale

on August 1, 2013

The folks over at OpenSignal have created some pretty impressive visualizations of how fragmented the Android marketplace is. Pictures like that are a little daunting as we head to the public release of MobileXPRT 2013. However, the benchmark is looking pretty stable. Since we released MobileXPRT CP1.1 back in June, we have not encountered any problem configurations.

That includes our first encounter with the recently released Android 4.3. We tested MobileXPRT 2013 CP 1.1 on a Nexus 7 running Android 4.3. The benchmark ran with no problem. Here are the scores, along with scores from the same device running Android 4.2.2.

	Android 4.2.2	Android 4.3
MobileXPRT performance	116	120
MobileXPRT user experience	97	98

Of course, WebXPRT 2013 ran on Android 4.3 as well. If you’re curious, here are the scores.

	Android 4.2.2	Android 4.3
WebXPRT	183	185

While the upgrade did not have a big effect on the score in the case of the Nexus 7, we will need to test on more devices before we can make a definitive statement about the effect of Android 4.3.3 on performance.

Eric

Comment on this post in the forums

Posted in MobileXPRT, WebXPRT |

There’s always something new to measure

By Eric Hale

on July 25, 2013

Scientific Reports published an article this month showing that electrically charged insects, such as honey bees, can cause some spider webs to deform. The effect, as pictures in the report show, is that the webs can literally reach out a short distance to snag the passing insect. It’s a cool optimization for the web, and one no one had measured before.

Like the scientists in the report, we are always looking for interesting things to measure. Even as we head toward the public release of MobileXPRT, we’re already thinking ahead to the next versions of all the XPRT benchmarks. This week, I’d like to share a few of the things we have been thinking about, based on conversations with folks (both within and outside of the Development Community) for WebXPRT.

Enhancing the benchmark UI to provide better feedback about the test progress
Extending coverage to other Web technologies such as Web Workers and CSS 3
Extending the workload categories to include other areas, such as productivity
Enabling WebXPRT to be used as a platform-independent battery life test

There are just some of the ideas we’ve been talking about. Which of these excite you? Even better, do you have any cool ideas of your own? Either way, post to the forums or e-mail BenchmarkXPRT@principledtechnologies.com

Eric

Comment on this post in the forums

Posted in Benchmarking, BenchmarkXPRT, Collaborative benchmark development, WebXPRT |

The show is in previews

By Eric Hale

on July 11, 2013

I love theater, both as an actor and as an audience member. Seeing a show in preview means you have the chance to see what might be the next big thing before the critics do. It also means you may be watching all the things they need to fix before opening night.

Microsoft released the Windows 8.1 preview at the end of June. Of course we had to give it a look. After installing it, we ran the compatible XPRT benchmarks: HDXPRT, TouchXPRT, and WebXPRT. (Because MobileXPRT is Android based, it does not run on Windows.)

WebXPRT ran without any problems. To date, WebXPRT has run on everything we’ve thrown at it.

We took a Windows 8 system with TouchXPRT already installed and upgraded it to Windows 8.1. The previously installed version of TouchXPRT had some problems. However, when we did a clean install of Windows 8.1 and then installed TouchXPRT, it ran just fine.

HDXPRT failed while configuring the applications. We are looking into this and hope to have a solution soon.

I’m not going to talk about the results, because this is a preview of Windows. By the time of the official release, the results from the benchmarks certainly may change.

The Windows 8.1 RTM is currently set for August. I’m looking forward to seeing how it performs!

Eric

Comment on this post in the forums

Posted in HDXPRT 2012, TouchXPRT, WebXPRT, Windows 8.1 |

Category: WebXPRT

Interesting questions

Sources

Endurance

Fragments

There’s always something new to measure

The show is in previews

Check out the other XPRTs: