Category: WebXPRT

Winding down the year

on December 19, 2013

We have a few things we need to touch on before the year ends.

In TouchXPRT 2014, we described some of the changes to TouchXPRT. We had hoped to have the Consumer Preview out in December, but we need to do a little more work on it. However, the TouchXPRT 2014 CP is coming after first of the year, and it’s worth the wait!

For the first time ever, we found a situation that caused WebXPRT to fail. We got a report of an iPhone 5s failing. We found that the user was using the private browsing mode. The WebXPRT Offline Notes test uses HTML5 local storage to sync and store notes locally. Safari and iOS Safari don’t support setting the HTML 5 localStorage property when using private browsing mode. Not using private browsing fixes the problem.

Remember that we are in the comment period for the new battery test. Community members should definitely check out the battery test RFC.

As we mentioned before, Bill and Mark are going to CES. If you’d like to chat, send an e-mail to benchmarkxprtsupport@principledtechnologies.com.

This is the last blog of 2013. The next post will be on January 2.

It’s been a great year. Thank you all.

Happy holidays!

Eric

Comment on this post in the forums

Posted in Battery life, Collaborative benchmark development, TouchXPRT, WebXPRT |

It’s the same phone

By Eric Hale

on November 14, 2013

A friend of mine thought there was something wrong with her phone. The reason? Her WebXPRT score was a lot lower than the score we published on the WebXPRT Web site.

From the frequency with which she posts on Facebook, I guessed she had the Facebook app running on her phone. So I asked if she had installed any apps on her phone since she bought it. She said “a few,” and I suggested we try turning some of them off.

Sometimes “a few” means “a lot.”

Anyway, after we turned a bunch of the apps off, her score jumped to over 90 percent of the published score. That made her happy, but she wanted to know why the apps would affect her score because “it’s the same phone.” I asked if she would expect her car to get the same gas mileage if it were carrying a bunch of cinderblocks. “Of course not,” she said, quickly understanding the metaphor. Asking your phone to do other things during the test is going to put a drag on the system.

It’s important to remember that benchmarking can be a pretty specialized area. As the popularity of the XPRTs grows, they are being run more often by bright, well educated people who don’t understand benchmarking basics. For this reason, we’re putting a lot of effort into automating best practices where we can, and guiding people when necessary.

Have the XPRTs ever confused you? If so, e-mail us at benchmarkxprtsupport@principledtechnologies.com. We can use your input.

Eric

Comment on this post in the forums

Posted in WebXPRT |

The Microsoft Surface 2

By Eric Hale

on October 24, 2013

As soon as the Microsoft Surface 2 became available, we got one and have been putting it through its paces. Of course, we ran WebXPRT and TouchXPRT. The results are on the TouchXPRT and WebXPRT sites, but I’ll repeat them here along with the results for its predecessor, the Microsoft Surface RT.

	TouchXPRT	WebXPRT
Surface RT	98	167
Surface 2	284	324

TouchXPRT shows the Surface 2 to be almost three times faster than the Microsoft Surface RT, while WebXPRT shows it to be almost twice as fast.

Why the difference? The most obvious explanation is that WebXPRT depends on the browser and its implementations of JavaScript and HTML5. TouchXPRT relies less on additional software and seems to take better advantage of the underlying hardware.

While we have yet to test the Intel Core i5-based Microsoft Surface Pro 2 ourselves, others have been doing so. Interestingly, Anandtech’s review of the Surface Pro 2 included WebXPRT results from both Chrome and IE. The Chrome result was over 30 percent higher than the IE result: 1,260 vs. 960. Unfortunately, Google has not made Chrome available for the ARM-based Surface 2, so we were not able to make that comparison.

As always, please let us know any results you get on any new hardware so we can get as many results as possible in our result databases. There are lots of new products coming out in the next few weeks and we’d love your help in getting results for as many of them as possible. Thanks!

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Performance testing on tablets, Touch-based computing, Touch-based devices, TouchXPRT, TouchXPRT results, WebXPRT, Windows 8.1 |

Look how we’ve grown!

By Eric Hale

on October 10, 2013

We talk a lot about the community model, but perhaps not enough about the community itself. I was looking at our growth over the past few months, and struck by how far we have come this year. We don’t reveal our members’ personal information, but I would like to share some statistics with you.

The community has grown by over 40 percent in 2013. Members are affiliated with 49 organizations, representing chip makers, OEMs, software vendors, and the press. Some members work outside technology, in fields such as engineering, finance, and communications.

Back in April, we blogged about setting up a second WebXPRT host to help with download speeds in China, so it’s no surprise that the community has also spread across the world. Because we don’t require detailed information from community members, I don’t know how many countries are represented. However, the members with whom I’ve had personal contact come from at least four continents.

Thanks to all our community members for making it a great year so far. If you’ve not joined yet, now is a great time. Remember that members see the previews of the benchmarks, and the 2014 versions are coming.

Here’s looking forward to greater growth in the future.

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, WebXPRT |

Interesting questions

By Eric Hale

on September 26, 2013

We’ve had a couple of interesting questions about WebXPRT this week.

The first question was about the Face detect test in WebXPRT. One person, having noticed that changing the version of Firefox affected the WebXPRT score on a particular device, asked whether the test used the JavaScript Canvas element. The answer is yes, the Face detection test does use the Canvas element. It is based on the JavaScript library by Dr. Liu Liu.

As we have discussed in the past, the software stack on a device affects the benchmark scores. WebXPRT is a HTML5 benchmark and uses elements in the HTML5 specification, such as Canvas. Browsers implement HTML in their JavaScript engines, whose performance depends on the OS and the underlying platform. So, WebXPRT scores are influenced by the browser and OS, as well as the platform.

The second question was whether it is possible to run WebXPRT without an Internet connection. Generally speaking, the answer to that is no. WebXPRT is a hosted application, and to run the official version, you must be able to connect to the WebXPRT servers.

However, community members can download the WebXPRT source and configure local servers that will run WebXPRT, if they desire. Note: As we discussed in Sources, any published results must be from the version hosted at webxprt.com.

Thanks for the questions and keep experimenting!

Eric

Comment on this post in the forums

Posted in HTML5, WebXPRT |

Sources

By Eric Hale

on September 19, 2013

If you’ve checked out the MobileXPRT and WebXPRT pages recently, you’ve probably noticed that the number of results has started to grow. The results are coming from three sources:

Internal testing at PT.
Results submitted by the public.
Results published on the Web. We link back to the source from these results. Results published on-line include results in reports PT publishes for clients and reviews by other parties.

While we are excited about the growing number of results, we do sanity check them. We compare the results with other runs for the same device when available, or with similar devices if not.

The source code for the benchmarks is available, and we encourage experimentation. However, it should go without saying that valid runs must come from the builds of the benchmarks the development community has published. We can’t compare two results generated by different builds.

That being said, if you change the code and get an interesting result, by all means do contact us. You may have discovered something that we’ll want to include in a future version

Keep the results coming and keep experimenting!

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT, BenchmarkXPRT development community, MobileXPRT, TouchXPRT, WebXPRT |

Category: WebXPRT

Winding down the year

It’s the same phone

The Microsoft Surface 2

Look how we’ve grown!

Interesting questions

Sources

Check out the other XPRTs: