Author Archives: Eric Hale

The show is in previews

on July 11, 2013

I love theater, both as an actor and as an audience member. Seeing a show in preview means you have the chance to see what might be the next big thing before the critics do. It also means you may be watching all the things they need to fix before opening night.

Microsoft released the Windows 8.1 preview at the end of June. Of course we had to give it a look. After installing it, we ran the compatible XPRT benchmarks: HDXPRT, TouchXPRT, and WebXPRT. (Because MobileXPRT is Android based, it does not run on Windows.)

WebXPRT ran without any problems. To date, WebXPRT has run on everything we’ve thrown at it.

We took a Windows 8 system with TouchXPRT already installed and upgraded it to Windows 8.1. The previously installed version of TouchXPRT had some problems. However, when we did a clean install of Windows 8.1 and then installed TouchXPRT, it ran just fine.

HDXPRT failed while configuring the applications. We are looking into this and hope to have a solution soon.

I’m not going to talk about the results, because this is a preview of Windows. By the time of the official release, the results from the benchmarks certainly may change.

The Windows 8.1 RTM is currently set for August. I’m looking forward to seeing how it performs!

Eric

Comment on this post in the forums

Posted in HDXPRT 2012, TouchXPRT, WebXPRT, Windows 8.1 |

Back to the source

By Eric Hale

on July 4, 2013

Last week, we released MobileXPRT CP1.1.This week, we released the source code for MobileXPRT CP1.1. You can download it here (login required). The procedure for building it is the same as for the previous CP. As we discussed in Kick the tires. Look under the hood, it’s easy to set up the environment and all the necessary software is free.

We believe that one of the most important things we can do is make the source code available. We believe that increasing the transparency of the benchmarking process and stimulating the participation of industry players and the public in the definition, development, understanding, and auditing of the benchmarks will lead to better benchmarks.

You may be thinking “Then why not open source the benchmark?” The short answer is that we need to make sure that the results from any version of our benchmarks are ones you can trust. You can watch Bill discuss this in BenchmarkXPRT: It’s not a benchmark.

We believe that the community model—which gives you total visibility into the benchmark, allows you to run your own experiments and contribute to future versions of the benchmark, and still protects the integrity of the results—strikes the right balance.

If you’re not a member, please consider joining. It’s easy.

If you are a member, check out the code and tell us how it can be better!

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT development community, Collaborative benchmark development, MobileXPRT 2013 source, Video |

A new version and a new video

By Eric Hale

on June 27, 2013

As we said in It’s finally here!, the Android ecosystem is very diverse and we knew that MobileXPRT might have problems on some devices. So far, we have encountered a couple of issues:

The Slideshow test failed during post validation check on the Onda tablet.
The Zoom-n-pinch test was unstable, failing part of the time, on the Zopo phone

On Friday, we will be releasing MobileXPRT 2013 community preview 1.1 (CP1.1), which fixes these problems. The results from version CP1.1 are comparable to the current community preview. If you have any problems, or questions, please e-mail us at benchmarkxprtsupport@principledtechnologies.com

We will release the source for CP1.1 next week.

In other news, we released a new video this week, BenchmarkXPRT: It’s not a benchmark. In this video, Bill answers some common questions about the BenchmarkXPRT Development Community. If you’ve ever wondered what exactly BenchmarkXPRT is, or why the world needs more benchmarks, this is the video for you.

Eric

Comment on this post in the forums

Posted in Android, BenchmarkXPRT, BenchmarkXPRT development community, MobileXPRT, MobileXPRT 2013 source, Video |

The truth hurts

By Eric Hale

on June 20, 2013

The truth hurts sometimes. Bill loves to mention how long he, Mark, and I have been involved in benchmark software. I am, of course, grateful to him for reminding me of just how old we all are.

Another truth that hurts is that sometimes things don’t happen when you want them to. Case in point: The preview of Windows 8.1 is supposed to be available on June 26. We released TouchXPRT, which is based on Windows 8, to the public in February.

The problem of timing is not limited to benchmarks. All software has timing issues. While the advance descriptions of the changes in Windows 8.1 don’t mention anything that should break the benchmark, we need to see what, if any, effect, the new version of the OS may have on the benchmark and its results.

As soon as the preview is available, we’ll start testing with it. As soon as we know whether TouchXPRT runs on Windows 8.1 and gives results comparable to Windows 8, we will let you know. We would appreciate help from any of you in trying out TouchXPRT with Windows 8.1. Let us know what you find out!

Eric

Comment on this post in the forums

Posted in Benchmarking, TouchXPRT, Windows 8.1 |

Chaos and opportunity

By Eric Hale

on June 13, 2013

With both E3 and Apple’s WWDC happening this week, there’s been a lot of news. There’s also been a lot of hyperbolic commentary. I am not about to get into the arguments about the PS4 vs. the Xbox One or iOS 7 vs. Android.

It was Tim Cook’s presentation at WWDC that really got my attention. It’s unusual in an executive presentation to focus so much attention on a particular competitor, but Android was clearly on his mind. At one point, he focused harsh attention on fragmentation in the Android market, calling it “terrible” for developers. You can see the video here, at about 74 minutes.

As we saw in the 90s, chaos can breed innovation. At that time, the paradigm was that Macs always worked, but if you wanted the most advanced hardware, you should get a PC. I remember the editors at MacWorld, who deeply, truly loved the Mac, lusting over the (by the standards of the time) small, light, cheap notebooks PC users could get.

That being said, we understand the challenges of developing in the Android market. As I said in It’s finally here!, the Android ecosystem is sufficiently diverse that we know the benchmark will encounter configurations we’ve not seen before. If you have any problems with the MobileXPRT CP, please let us know at benchmarkxprtsupport@principledtechnologies.com. We want the benchmark to be the best it can be.

Eric

Comment on this post in the forums

Posted in Android, BenchmarkXPRT development community, History of benchmarking, History of computers, MobileXPRT |

Kick the tires. Look under the hood.

By Eric Hale

on June 6, 2013

Today, we released the source code for the MobileXPRT 2013 CP to the community (community login required). If you already have an Android development environment set up, building the benchmark is very straightforward. If you do not have a build environment, it’s not difficult to create one. All the necessary software is free, and the build documentation includes detailed instructions for creating the build environment on Windows 8.

Of course, you can also build Android applications using Linux or Mac OS X. The instructions include links to the information you need to set up those environments as well.

If you have suggestions about how we could make the application better, or you write some code you would like to submit for inclusion in the benchmark, please contact us at benchmarkxprtsupport@principledtechologies.com.

Bill wrote about a presentation he gave to Mobile PC Extended Battery Life Working Group (EBL WG) in the post Presentation XPRT. We’ve put the slides from that presentation up on Slideshare. Check them out!

Eric

Comment on this post in the forums

Posted in Android, BenchmarkXPRT development community, MobileXPRT, MobileXPRT 2013 source |

Author Archives: Eric Hale

The show is in previews

Back to the source

A new version and a new video

The truth hurts

Chaos and opportunity

Kick the tires. Look under the hood.

Check out the other XPRTs: