BenchmarkXPRT Blog banner

Category: MobileXPRT

Chaos and opportunity

With both E3 and Apple’s WWDC happening this week, there’s been a lot of news. There’s also been a lot of hyperbolic commentary. I am not about to get into the arguments about the PS4 vs. the Xbox One or iOS 7 vs. Android.

It was Tim Cook’s presentation at WWDC that really got my attention. It’s unusual in an executive presentation to focus so much attention on a particular competitor, but Android was clearly on his mind. At one point, he focused harsh attention on fragmentation in the Android market, calling it “terrible” for developers. You can see the video here, at about 74 minutes.

As we saw in the 90s, chaos can breed innovation. At that time, the paradigm was that Macs always worked, but if you wanted the most advanced hardware, you should get a PC. I remember the editors at MacWorld, who deeply, truly loved the Mac, lusting over the (by the standards of the time) small, light, cheap notebooks PC users could get.

That being said, we understand the challenges of developing in the Android market. As I said in It’s finally here!, the Android ecosystem is sufficiently diverse that we know the benchmark will encounter configurations we’ve not seen before. If you have any problems with the MobileXPRT CP, please let us know at benchmarkxprtsupport@principledtechnologies.com. We want the benchmark to be the best it can be.

Eric

Comment on this post in the forums

Kick the tires. Look under the hood.

Today, we released the source code for the MobileXPRT 2013 CP to the community (community login required). If you already have an Android development environment set up, building the benchmark is very straightforward. If you do not have a build environment, it’s not difficult to create one. All the necessary software is free, and the build documentation includes detailed instructions for creating the build environment on Windows 8.

Of course, you can also build Android applications using Linux or Mac OS X. The instructions include links to the information you need to set up those environments as well.

If you have suggestions about how we could make the application better, or you write some code you would like to submit for inclusion in the benchmark, please contact us at benchmarkxprtsupport@principledtechologies.com.

Bill wrote about a presentation he gave to Mobile PC Extended Battery Life Working Group (EBL WG) in the post Presentation XPRT. We’ve put the slides from that presentation up on Slideshare. Check them out!

Eric

Comment on this post in the forums

It’s finally here!

As the community members know, we released the MobileXPRT 2013 community preview yesterday. As I said in last week’s blog A look into the future, you are free to publish results from this community preview.

We hope you’ll submit results to be published in the MobileXPRT database. You’ll find the file containing your results in the MobileXPRT/results directory on the SD card of your device. The results directory will contain a time-stamped directory for each run, for example ALL28May2013112522. You will find a file named mobilexprt_results.xml in the appropriate time-stamped directory. To submit a result, just e-mail the mobilexprt_results.xml file from your run to benchmarkxprtsupport@principledtechnologies.com.

The benchmark includes a user manual in the UI. In addition to the benchmark itself, you can also find installation instructions and a design overview in the member’s area.

We will make the source code for the community preview available in the next few days.

Given the diversity and changing nature of Android devices, we expect that you may run into some issues. As with any of our community previews, if you encounter any issues, please let us know so we can continue to improve the benchmark as we prepare it for release.

We’re very excited about the release of the MobileXPRT 2013 community preview. If you’re not a member, join and find out what all the excitement is about!

Eric

Comment on this post in the forums

The MobileXPRT 2013 community preview is available!

If you’re a member, you can download it here. If you’re not a member, join and see what the excitement’s about.

Comment on this post in the forums

A look into the future

This week, we’ve been testing builds of the upcoming MobileXPRT benchmark. So far, things have been looking really good.

MobileXPRT is designed for testing Android-based devices, from smart phones to tablets. Like TouchXPRT, MobileXPRT uses native code to implement real-world use cases. Mobile XPRT offers two categories of tests: Performance tests and User Experience (UX) tests.

The five Performance tests are Apply Photo Effects, Create Photo Collages, Create Slideshow, Encrypt Personal Content, and Detect Faces to Organize Photos.

The five UX tests are List Scroll, Grid Scroll, Gallery Scroll, Browser Scroll, and Zoom and Pinch.

The benchmark provides a score for each individual test, as well as a rollup score for each category.

Sometime next week, we plan to release the MobileXPRT 2013 Community Preview. The CP, as its name makes clear, is not the final MobileXPRT release. It is, though, a useful tool for beginning to measure Android device performance. It is also a great way for everyone in the community to see the current state of our thinking and to provide feedback—rather than reading a design spec, you can actually run this version of the tool and see what you think!

As with TouchXPRT, we will allow publication of scores from the community preview. We will also be releasing the source code  to all community members. If you’re curious about not just what we’re doing but how we’re doing it, you can find out. If you have suggestions about how to do it better, we’re all ears.

As a reminder, the community previews are available only to members of the BenchmarkXPRT Development Community. If you haven’t joined yet, come join us!

Eric

Comment on this post in the forums

Our baby has a new name!

The upcoming Android benchmark will be called MobileXPRT. Thanks to everyone who sent in suggestions. We are testing development builds now and look forward to having a community preview available in the next few weeks.

In other news, the developer license used to build TouchXPRT expired this week. We have created a new version to fix this problem. If you are a TouchXPRT user, you’ll need to download the new version for any future testing. You can find the details here.

As we mentioned in the post Three names, two hosts, we set up a second WebXPRT hosting site to see if that would improve the slow downloads reported in China. To help us better understand the situation, we are going to start logging IP addresses for the host at http://54.251.252.204/webxprt/. We are doing this to see if there are regional differences in the download time. This change does not affect users of WebXPRT at http://www.principledtechnologies.com/benchmarkxprt/webxprt/. Neither host will gather personally identifying information.

Eric

Comment on this post in the forums

Check out the other XPRTs:

Forgot your password?