Author Archives: Bill Catchings

MobileXPRT 2013 lives!

on August 12, 2013

On behalf of the BenchmarkXPRT Development Community, I’m proud to announce that today we are releasing MobileXPRT 2013. MobileXPRT is our benchmark for measuring the performance of Android-based devices. Like the other benchmarks (or XPRTs as we sometimes call them), MobileXPRT attempts to measure performance by using activities that real users do on their devices. For more information, check out our press release http://www.prweb.com/releases/2013/8/prweb11016543.htm.

We still have more to do over the next few days. We need to release the source, which we expect to do later this week. We need to make MobileXPRT 2013 available on Google Play. We’ll also work to add more results to our MobileXPRT 2013 results database. Watch this space for more details!

I’d also like to thank those of you that have helped us by developing, testing, or commenting on the Community Preview release. Your efforts have been an important part of making this benchmark available to a larger audience. Please continue to let us know your thoughts, send us your results, and tell us about any problems you find.

Benchmark development is never done!

Bill

Comment on this post in the forums

Posted in MobileXPRT |

Presentation XPRT

By Bill Catchings

on May 9, 2013

A few weeks ago, the Mobile PC Extended Battery Life Working Group (EBL WG) asked me to give a presentation about the BenchmarkXPRT benchmarks during one of their meetings. The group started a little over 10 years ago and is an industry organization focusing on achieving all-day battery life in mobile PC platforms.

There was a catch, of course. The meeting was this week, in Taipei. Luckily, we were able to grant the request without racking up any frequent flier miles—thanks to the wonders of technology, I was able to give the presentation remotely.

Taipei is 12 hours ahead of us here on the East Coast, so I made arrangements to give the presentation last night at 9:30 pm, making it 9:30 am the next day there—the first thing on the agenda for today. (Oh, the vagaries of time zones!)

During the one-hour session we talked about HDXPRT, TouchXPRT, WebXPRT and MobileXPRT (which, for the moment, is how we internally refer to the XPRT formerly known as PhoneXPRT). And of course, we discussed the BenchmarkXPRT Development Community. It went well, and we had some great questions from the group.

It went so well, in fact, that I’d love to do it again. I like talking about the development community and the XPRTs, and would welcome the chance to present to other groups. If you are interested or know an organization that is, you can email us at benchmarkxprtsupport@principledtechnologies.com.

Bill

Comment on this post in the forums

A happy coincidence

By Bill Catchings

on May 2, 2013

I love new gadgets and even the promise of new ones. Samsung just announced the specs for their upcoming Galaxy Tab 3. Initial reactions to the specs have been somewhat muted to say the least. Basically, they’ve looked at this as only an incremental improvement to the current model. The early rumors of a larger screen and bigger improvements, which turned out to be false, surely contributed to the disappointed reactions.

That being said, some sites claim that the performance of the Galaxy Tab 3 is much higher than the Galaxy Tab 2, particularly regarding graphics. We look forward to verifying these claims ourselves.

Coincidentally, this week we have been playing with an early version of PhoneXPRT (or whatever we end up calling it). So far, things are looking good. We ran it on several devices, including a Samsung Galaxy Tab 2. Like all the XPRT benchmarks, it uses real- world scenarios, which we think result in more useful and accurate results. We’ll talk more about the scenarios in the next few weeks.

It’s a very exciting time in the Android phone and tablet market. I can’t wait to try out subsequent versions of the new benchmark on the latest and greatest Android devices!

Bill

Comment on this post in the forums

Posted in MobileXPRT, Performance testing on tablets, PhoneXPRT, Touch-based devices |

The name game

By Bill Catchings

on March 29, 2013

There’s a lot going on in the world of the XPRTs. We’re working on HDXPRT 2013, writing white papers, building up our results database, and thinking ahead to the next versions of TouchXPRT and WebXPRT. At the same time, we are giving quite a bit of attention to PhoneXPRT.

As we said in the PhoneXPRT press release last month, PhoneXPRT will use the same kinds of realistic scenarios the other XPRT benchmarks do. These scenarios include tasks that people perform on phones as well as other mobile devices.

We’ve gone back and forth on the seemingly simple question of how to define what a phone is. At one point, defining a phone as a mobile device that you put to your ear seemed to suffice. As phones grow larger and tablets smaller, that doesn’t really hold up.

We also have a decent bit of interest in using the benchmark on Android-based tablets as well as phones. That seems like a good idea to us, as we are all for getting the most out of any benchmark.

One issue, however, is that PhoneXPRT is not a great name for a benchmark that may be commonly used on devices other than phones. So we’ve started thinking about what else we could call it.

As always, we look to the community. Do you think calling the benchmark PhoneXPRT would limit its usefulness for benchmarking tablets? Do you have any ideas for more inclusive names? We really need your feedback here and look forward to getting it. Please send suggestions to benchmarkxprtsupport@principledtechnologies.com or post your comments in the forum. Thanks!

Bill

Comment on this post in the forums

Posted in Android, BenchmarkXPRT, HDXPRT 2013, PhoneXPRT |

Going Live

By Bill Catchings

on February 21, 2013

As you may know, the Mobile World Congress (MWC) starts Monday in Barcelona. I really hate that I can’t go to this show and not only because I love Barcelona!

However, MWC seems like a good time to release TouchXPRT 2013 and WebXPRT 2013. We’re testing the final builds now, and plan to make them available to the public next week. If you’re not a community member and want to see what all the fuss is about before then, please join the community and check them out.

As we discussed in the Webinar back in January, we have been able to verify that the results from both benchmarks are in line with those of the community previews. That means that we don’t anticipate any problems comparing results from the CP1 versions with those from the release versions.

While TouchXPRT 2013 CP1 did not have a total score, based on your feedback, we added one in final version of TouchXPRT 2013.

I’ve included some comparison results from WebXPRT 2013 at the end of this post. As always, we would love to have any results you can share for either benchmark.

We’ve also been thinking about where to go next. We’re not ready to talk yet, but expect big news next week!

Bill

Device	WebXPRT 2013 Overall score
Nexus 7	201
Kindle Fire	95
iPad 2	52
iPad 4	181
Samsung Galaxy SIII	168
iPhone 5	167
iPad Mini	110

Comment on this post in the forums

Posted in TouchXPRT 2013 release, Trade Shows, WebXPRT 2013 release, Windows 8 |

Unchecked ambition

By Bill Catchings

on January 17, 2013

With 10 applications and five scenarios, HDXPRT 2012 is the most ambitious version of HDXPRT ever. However, as we said in the blog post of the same name, There is such a thing as too much. We heard that message from you as well: HDXPRT 2012 is too big. At 11 GB, we can’t make it available for download, and it won’t fit on a single DVD.

It also takes longer to use than many of you would like. The installation takes a couple of hours, and each iteration takes 2 to 3 hours. Because a valid run of HDXPRT 2012 comprises three iterations, getting a single result takes all day.

As we work to reduce the size of HDXPRT 2013, we’re being careful. While we need to shrink the benchmark and its running time, we want to be sure that we don’t compromise its essential value. As part of this process, we’re looking at both the selection of applications and the ways we can reengineer the scenarios.

If you have ideas about HDXPRT 2012 that you haven’t sent yet, now is a good time to do that. We’ll be sending out the HDXPRT 2013 RFC in the next couple of weeks. Once it’s out, we will look forward to hearing your comments.

I also wanted to mention that as we had to cancel CES at the last minute, we’re planning to do a Webinar next week (Tuesday, January 22) to cover the material we planned to present there. We’ll send out an email later this week with more details.

Bill

Comment on this post in the forums

Posted in BenchmarkXPRT, CES, HDXPRT, HDXPRT 2013, HDXPRT Development Community benefits, Let us know your thoughts |

Author Archives: Bill Catchings

MobileXPRT 2013 lives!

Presentation XPRT

A happy coincidence

The name game

Going Live

Unchecked ambition

Check out the other XPRTs: