Author Archives: Eric Hale

An update on TouchXPRT 2016

on October 8, 2015

We’ll be releasing the MobileXPRT 2015 white paper tomorrow. It contains lots of information about MobileXPRT 2015 that you won’t find anywhere else. We hope you’ll find it very informative.

A couple of weeks ago, we released the design document for TouchXPRT 2016 (login required). This week, we put the first build of TouchXPRT 2016 into testing. It’s a Universal Windows app that runs on Windows 10 tablets, PCs, and phones. This means that TouchXPRT can now run on a wider variety of devices. However, it also means that TouchXPRT 2016 will not be backward compatible with Windows 8 and 8.1.

Given the current state of the SDKs, installing the test builds on phones is more complicated than we would like. We’re looking into ways to simplify the install before releasing the community preview. Testing on phones is particularly important because we made many of the UI changes to enable TouchXPRT to work acceptably on a small display.

We’ll keep you informed as testing proceeds. We’re hoping to release the community preview in the next couple of weeks.

Eric

Posted in Android, MobileXPRT, MobileXPRT 2015, TouchXPRT, TouchXPRT 2016, TouchXPRT release cycle, White papers, Windows 10, Windows 8, Windows 8.1 |

Question we get a lot

By Eric Hale

on October 2, 2015

“How come your benchmark ranks devices differently than [insert other benchmark here]?” It’s a fair question, and the reason is that each benchmark has its own emphasis and tests different things. When you think about it, it would be unusual if all benchmarks did agree.

To illustrate the phenomenon, consider this excerpt from a recent browser shootout in VentureBeat:

While this looks very confusing, the simple explanation is that the different benchmarks are testing different things. To begin with, SunSpider, Octane, JetStream, PeaceKeeper, and Kraken all measure JavaScript performance. Oort Online measures WebGL performance. WebXPRT measures both JavaScript and HTML 5 performance. HTML5Test measures HTML5 compliance.

Even with benchmarks that test the same aspect of browser performance, the tests differ. Kraken and SunSpider both test the speed of JavaScript math, string, and graphics operations in isolation, but run different sets of tests to do so. PeaceKeeper profiles the JavaScript from sites such as YouTube and FaceBook.

WebXPRT, like the other XPRTs, uses scenarios that model the types of work people do with their devices.

It’s no surprise that the order changes depending on which aspect of the Web experience you emphasize, in much the same way that the most fuel-efficient cars might not be the ones with the best acceleration.

This is a bigger topic than we can deal with in a single blog post, and we’ll examine it more in the future.

Eric

Posted in Benchmark metrics, Benchmarking, Benchmarking computing devices, BenchmarkXPRT development community, Collaborative benchmark development, Performance benchmarking, Performance of computing devices, WebXPRT, What makes a good benchmark? |

MobileXPRT 2015 source code and TouchXPRT 2016 Design Overview are available

By Eric Hale

on September 25, 2015

We’re excited to announce that the MobileXPRT 2015 source code and TouchXPRT 2016 Design Overview are now available to community members!

Download the MobileXPRT 2015 source here (login required).

Download the MobileXPRT 2015 build instructions here (login required).

Download the TouchXPRT 2016 Design Overview here (login required).

We also posted links to all three items on the MobileXPRT and TouchXPRT tabs in the Members’ Area.

If you want more information, please contact BenchmarkXPRTsupport@principledtechnologies.com.

We look forward to your feedback!

Posted in BenchmarkXPRT, BenchmarkXPRT development community, MobileXPRT, MobileXPRT 2015, Source code, TouchXPRT, TouchXPRT 2016, TouchXPRT development process |

The benefits of membership

By Eric Hale

on September 24, 2015

We have a couple of goodies for community members coming tomorrow.

The TouchXPRT 2016 design overview will tell you what we’re planning for the upcoming community preview. Thanks to everyone who’s contributed ideas. Let us know if the design overview omits anything you’d like to see in the benchmark.

The MobileXPRT 2015 source code and the instructions for building MobileXPRT 2015 will be available as well. Community members have access to the source for all the XPRT benchmarks. Making the source available is a pillar of the community model.

Look for the design overview and source code in the members’ area.

If you aren’t yet a member, this is a great time to join!

Eric

Posted in Collaborative benchmark development, MobileXPRT, MobileXPRT 2015, TouchXPRT, TouchXPRT 2016, TouchXPRT development process, TouchXPRT release cycle |

Impressed and Excited

By Eric Hale

on September 17, 2015

A couple of weeks ago we talked about some initiatives we’ve been exploring. This week, we’re happy to be able to talk about a new project. We’re sponsoring a senior project with the computer science department at North Carolina State University (NCSU).

As part of their education, small teams of seniors work with local companies on various programming projects. The students are expected to put significant time into these projects, and the tasks aren’t easy. You can get a sense of the range and complexity of these projects by looking at past project proposals.

We submitted a proposal for creating an experimental benchmark test to NCSU and they accepted it and assigned us a team of students. I’ve met a couple of times now with them. I’m impressed and very excited about what they’re going to do. We’re not ready to talk about the project yet, but if they’re successful, we’ll make the new test available on the BenchmarkXPRT site. We might even include it in a future XPRT!

Our hope is that if this project is successful, we can replicate it at other schools and help train the next generation of benchmark developers. As a bonus, the BenchmarkXPRT community will get some fresh perspectives and some new experimental test tools.

Eric

Posted in BenchmarkXPRT development community, Collaborative benchmark development, Future of performance evaluation |

BenchmarkXPRT in China

By Eric Hale

on September 10, 2015

Last week, we talked about some of the changes we’re making to the BenchmarkXPRT site to make it easier to use. This week, we’d like to talk a bit about improvements we’ve been making to support our users in China.

As you may remember, the first of the XPRTs to have a Chinese UI was BatteryXPRT. We’ve since released WebXPRT 2015 and MobileXPRT 2015, both of which have also have Chinese UIs. We’re also in the process of getting MobileXPRT 2015 listed in several major Chinese app stores. (MobileXPRT 2013 is currently available from Xiaomi and Zhushou 360.)

In other words, we’re always thinking of ways to enhance the XPRT experience for our users in China. To improve download speeds, we’ve long hosted WebXPRT on a mirror site in Singapore. Recently, based on feedback from our users and our own analysis, we’ve changed the way that the privacy notice is displayed on that site. The change allows you to run WebXPRT without loading any Google analytics, which means faster load times for all users.

We will continue to work to improve our localization. This is an area where we can use the help of the community. If you have translation skills and want to contribute the strings for a UI in your language, let us know.

Eric

Last week in the XPRTs

We added a new TouchXPRT result
We added a new HDXPRT result

Posted in BatteryXPRT 2014 for Android, BenchmarkXPRT, BenchmarkXPRT development community, Chinese, Collaborative benchmark development, MobileXPRT, MobileXPRT 2013 source, MobileXPRT 2015, WebXPRT 2015 |

Author Archives: Eric Hale

An update on TouchXPRT 2016

Question we get a lot

MobileXPRT 2015 source code and TouchXPRT 2016 Design Overview are available

The benefits of membership

Impressed and Excited

BenchmarkXPRT in China

Last week in the XPRTs

Check out the other XPRTs: