Category: WebXPRT

It’s always worth asking

on March 6, 2014

Last week, one of our community members asked for a couple of enhancements to WebXPRT. They wanted WebXPRT to be easier to automate, and they made two specific requests:

Add debug/result logs
Add the ability to start the test without UI interactions, by using a specific URL or a command line

This is a great example of why we put so much emphasis on the community. We have tried to make the BenchmarkXPRT benchmarks easy to use, but we don’t always face the same testing demands you do. If there’s anything we can do to make these tools more valuable, please let us know by posting on the forums or e-mailing us at benchmarkxprtsupport@principledtechnologies.com.

We are adding those abilities to the upcoming WebXPRT 2014 community preview. Speaking of the community preview, we have been working hard on it, and in the next few weeks, we’ll be talking about what will be in it.

Keep those requests coming!

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, WebXPRT, What makes a good benchmark? |

Happy New Year everyone!

By Eric Hale

on January 2, 2014

We hope you had a great holiday.

Bill and Mark are preparing to go to CES this year. We’re really excited about the chance to talk with folks because there’s so much to talk about.

It’s been a great year for the XPRTs. At the start of 2013, HDXPRT 2012 was the only XPRT benchmark available to the public. Now four XPRT benchmarks are available through our Web site. Take a look at some of the milestones from this banner year:

We released MobileXPRT 2013, TouchXPRT 2013 and WebXPRT 2013 to the public.
To reflect the expanded offerings, the HDXPRT Development Community evolved into the BenchmarkXPRT Development Community.
We radically redesigned the Web site.
The greater XPRT offerings let us test more varied devices, including the Amazon Kindle Fire HDX, Barnes & Noble Nook HD+, NVIDIA Shield, XBOX ONE, and PS4.
We released four videos.

And 2014 is going to be an even bigger year! We are very close to releasing the TouchXPRT 2014 community preview and are in the comment period for a new battery life test—and we are only two days into the year!

If you will be at CES, this is a great time make yourself heard. E-mail benchmarkxprtsupport@principledtechnologies.com if you’d like to chat with Bill and Mark about what you’d like to see in the coming year.

Comment on this post in the forums

Posted in BenchmarkXPRT development community, CES, MobileXPRT, TouchXPRT, WebXPRT |

Winding down the year

By Eric Hale

on December 19, 2013

We have a few things we need to touch on before the year ends.

In TouchXPRT 2014, we described some of the changes to TouchXPRT. We had hoped to have the Consumer Preview out in December, but we need to do a little more work on it. However, the TouchXPRT 2014 CP is coming after first of the year, and it’s worth the wait!

For the first time ever, we found a situation that caused WebXPRT to fail. We got a report of an iPhone 5s failing. We found that the user was using the private browsing mode. The WebXPRT Offline Notes test uses HTML5 local storage to sync and store notes locally. Safari and iOS Safari don’t support setting the HTML 5 localStorage property when using private browsing mode. Not using private browsing fixes the problem.

Remember that we are in the comment period for the new battery test. Community members should definitely check out the battery test RFC.

As we mentioned before, Bill and Mark are going to CES. If you’d like to chat, send an e-mail to benchmarkxprtsupport@principledtechnologies.com.

This is the last blog of 2013. The next post will be on January 2.

It’s been a great year. Thank you all.

Happy holidays!

Eric

Comment on this post in the forums

Posted in Battery life, Collaborative benchmark development, TouchXPRT, WebXPRT |

It’s the same phone

By Eric Hale

on November 14, 2013

A friend of mine thought there was something wrong with her phone. The reason? Her WebXPRT score was a lot lower than the score we published on the WebXPRT Web site.

From the frequency with which she posts on Facebook, I guessed she had the Facebook app running on her phone. So I asked if she had installed any apps on her phone since she bought it. She said “a few,” and I suggested we try turning some of them off.

Sometimes “a few” means “a lot.”

Anyway, after we turned a bunch of the apps off, her score jumped to over 90 percent of the published score. That made her happy, but she wanted to know why the apps would affect her score because “it’s the same phone.” I asked if she would expect her car to get the same gas mileage if it were carrying a bunch of cinderblocks. “Of course not,” she said, quickly understanding the metaphor. Asking your phone to do other things during the test is going to put a drag on the system.

It’s important to remember that benchmarking can be a pretty specialized area. As the popularity of the XPRTs grows, they are being run more often by bright, well educated people who don’t understand benchmarking basics. For this reason, we’re putting a lot of effort into automating best practices where we can, and guiding people when necessary.

Have the XPRTs ever confused you? If so, e-mail us at benchmarkxprtsupport@principledtechnologies.com. We can use your input.

Eric

Comment on this post in the forums

Posted in WebXPRT |

The Microsoft Surface 2

By Eric Hale

on October 24, 2013

As soon as the Microsoft Surface 2 became available, we got one and have been putting it through its paces. Of course, we ran WebXPRT and TouchXPRT. The results are on the TouchXPRT and WebXPRT sites, but I’ll repeat them here along with the results for its predecessor, the Microsoft Surface RT.

	TouchXPRT	WebXPRT
Surface RT	98	167
Surface 2	284	324

TouchXPRT shows the Surface 2 to be almost three times faster than the Microsoft Surface RT, while WebXPRT shows it to be almost twice as fast.

Why the difference? The most obvious explanation is that WebXPRT depends on the browser and its implementations of JavaScript and HTML5. TouchXPRT relies less on additional software and seems to take better advantage of the underlying hardware.

While we have yet to test the Intel Core i5-based Microsoft Surface Pro 2 ourselves, others have been doing so. Interestingly, Anandtech’s review of the Surface Pro 2 included WebXPRT results from both Chrome and IE. The Chrome result was over 30 percent higher than the IE result: 1,260 vs. 960. Unfortunately, Google has not made Chrome available for the ARM-based Surface 2, so we were not able to make that comparison.

As always, please let us know any results you get on any new hardware so we can get as many results as possible in our result databases. There are lots of new products coming out in the next few weeks and we’d love your help in getting results for as many of them as possible. Thanks!

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Performance testing on tablets, Touch-based computing, Touch-based devices, TouchXPRT, TouchXPRT results, WebXPRT, Windows 8.1 |

Look how we’ve grown!

By Eric Hale

on October 10, 2013

We talk a lot about the community model, but perhaps not enough about the community itself. I was looking at our growth over the past few months, and struck by how far we have come this year. We don’t reveal our members’ personal information, but I would like to share some statistics with you.

The community has grown by over 40 percent in 2013. Members are affiliated with 49 organizations, representing chip makers, OEMs, software vendors, and the press. Some members work outside technology, in fields such as engineering, finance, and communications.

Back in April, we blogged about setting up a second WebXPRT host to help with download speeds in China, so it’s no surprise that the community has also spread across the world. Because we don’t require detailed information from community members, I don’t know how many countries are represented. However, the members with whom I’ve had personal contact come from at least four continents.

Thanks to all our community members for making it a great year so far. If you’ve not joined yet, now is a great time. Remember that members see the previews of the benchmarks, and the 2014 versions are coming.

Here’s looking forward to greater growth in the future.

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, WebXPRT |

Category: WebXPRT

It’s always worth asking

Happy New Year everyone!

Winding down the year

It’s the same phone

The Microsoft Surface 2

Look how we’ve grown!

Check out the other XPRTs: