BenchmarkXPRT Blog banner

Category: White papers

Get the 411 on WebXPRT 2015

We’re publishing the WebXPRT 2015 white paper tomorrow. It contains lots of information about WebXPRT 2015 that you won’t find anywhere else. In addition to explaining what the tests do, the test descriptions detail the particular aspects of Web browsing performance that the test stresses.

The white paper also contains a list of the third-party libraries that WebXPRT uses, what those libraries do, and where you can find them.  It also lists the HTML 5 capabilities that WebXPRT 2015 requires. And, as in past XPRT white papers, we include a spreadsheet showing how the benchmark calculates its results. There’s a lot of information, so check it out!

Meanwhile, we’re hard at work on MobileXPRT 2015.  The Android M Developer Preview 2 hasn’t presented any new problems, so things are looking good. I expect to have more details about the upcoming MobileXPRT 2015 community preview soon!

Eric

Lots of things are happening!

The WebXPRT 2015 community preview hasn’t even been out two weeks yet, but there are already hundreds of runs in the database. If you haven’t checked it out yet, now is a good time! (login required)

As I mentioned last week, Bill and Justin are at Intel Developer Forum 2015 – Shenzhen. Here’s their home away from home this week.

booth

Bill and Justin have been having a lot of good conversations and have found a lot of interest in an open development community. We’re looking forward to having even more members in Asia soon!

On Monday we released the CrXPRT white paper. If you want to know more about the concepts behind CrXPRT 2015, how it was developed, how the results are calculated, or anything else about CrXPRT, the white paper is a great place to start.
Finally, the MobileXPRT 2015 design document is coming in the next couple of weeks. What would you like to see in the next version of MobileXPRT? 64-bit support? New types of tests? Improvements to the UI? Everything is on the table. This is the time to make your voice heard!
Eric

TouchXPRT 2014 arrives Friday!

Friday is the big day for TouchXPRT 2014. It will be available for download at TouchXPRT.com and through the Windows App Store no later than 5:00 PM EDT on Friday May 16.

As we mentioned last week, in addition to the new look and more-demanding component tests, TouchXPRT has a number of new features. One feature we are especially excited about is that users will be able to automate their TouchXPRT tests, something they could not do in TouchXPRT 2013.

You can run TouchXPRT 2014 from a command prompt or using scripting mechanisms such as batch files or PowerShell. The scripting interface allows you to set several parameters, including:

  • Start running the test automatically on launch. You can choose to run all of the scenarios to generate an overall score, or run any single scenario for a component test score.
  • Change the number of iterations the test should run.
  • Change the delay before the test starts running after launch.
  • Change the output filename.

You can read the automation details and find specific commands in the Exploring TouchXPRT 2014 white paper.

Also, we’re in the last steps of preparing the HDXPRT 2014 Community Preview. It’s coming in the next couple of weeks, so stay tuned for more information.

Eric

Comment on this post in the forums

TouchXPRT 2014 is almost here

We’re excited to say that TouchXPRT 2014 will launch within the next few days. TouchXPRT 2014 includes a completely new UI, updated component tests, improved disclosure information, improved automation, and the ability to submit results directly from the app.

TouchXPRT 2014 also addresses the number-one request we got about TouchXPRT 2013—to make it more obvious how to run all the tests. In TouchXPRT 2013, users had to choose the Run All button from the hidden Charms menu to run all of the tests and produce an overall score. In TouchXPRT 2014, the Start Test button on the benchmark’s main page makes it obvious how to run all five tests at once.

After the launch, TouchXPRT will be available for download at TouchXPRT.com and through the Windows App Store.

TouchXPRT 2013 will continue to be available for a while, as we understand that labs may have tests in progress. Likewise, the TouchXPRT 2013 results database will continue to be available, but separate from the TouchXPRT 2014 results. You should never compare results for the two versions of TouchXPRT.

For a more in-depth discussion of the new benchmark, please read the Exploring TouchXPRT 2014 white paper.

We’ve been busy with the BatteryXPRT and TouchXPRT launches, but more is on the way. HDXPRT 2014 is next in line!

Eric

Comment on this post in the forums

It’s show time!

Tomorrow is the big day for BatteryXPRT 2014 for Android. After months of development, it’s finally time to release it to the world. We’ve learned quite a bit throughout the process, and we expect BatteryXPRT to become a trusted tool for Android users around the world.

BatteryXPRT will be available for download at BatteryXPRT.com, or through the Google Play Store. It will be available to the public no later than 5:00 PM ET on Friday May 2.

The BatteryXPRT results page allows you to submit test results directly from the app. We hope you’ll use this feature to submit your test runs for publication in the database.

There’s a huge diversity of Android devices out there, so BatteryXPRT.com will link to a page of tips, tricks, and known issues that we compiled during development testing. As with any of the XPRTs, if you encounter any issues please let us know.

Along with the new app, we also invite you to check out the Exploring BatteryXPRT 2014 for Android white paper for a more in-depth discussion of how it all works.

It’s definitely BatteryXPRT’s time to shine, but don’t forget that the new and improved TouchXPRT is coming soon!

Eric

Comment on this post in the forums

Lies, damned lies, and statistics

No one knows who first said “lies, damned lies, and statistics,” but it’s easy to understand why they said it. It’s no surprise that the bestselling statistics book in history is titled How to Lie with Statistics. While the title is facetious, it is certainly true that statistics can be confusing—consider the word “average,” which can refer to the mean, median, or mode. “Mean average,” in turn, can refer to the arithmetic mean, the geometric mean, or the harmonic mean. It’s enough to make a non-statistician’s head spin.

In fact, a number of people have been confused by the confidence interval WebXPRT reports. We believe that the best way to stand behind your results is to be completely open about how you crunch the numbers. To this end, we released the white paper WebXPRT 2013 results calculation and confidence interval this past Monday.

This white paper, which does not require a background in mathematics, explains what the WebXPRT confidence interval is and how it differs from the benchmark variability we sometimes talk about. The paper also gives an overview of the statistical and mathematical techniques WebXPRT uses to translate the raw timing numbers into results.

Because sometimes the devil is in the details, we wanted to augment our overview by showing exactly how WebXPRT calculates results. The white paper is accompanied by a spreadsheet that reproduces the calculations WebXPRT uses. If you are mathematically inclined and would like to suggest improvements to the process, by all means let us know!

Eric

Comment on this post in the forums

Check out the other XPRTs:

Forgot your password?