Category: Collaborative benchmark development

Something new

on April 10, 2014

We have seen significant interest from China in the XPRTs over the last year, with particular interest given to MobileXPRT. Given how important Android is in the Asian market, this not a surprise.

Our newest benchmark for Android devices is BatteryXPRT. While it’s still in Community Preview, we’re experimenting a bit with internationalization. On Monday, we’re going to release a second Community Preview with a UI in Simplified Chinese. Here’s what it looks like.

BatteryXPRT uses Android’s global language setting to determine whether to display in English or Simplified Chinese. Assuming your device is currently set to English, this is the setting you change by navigating to Settings → Language & input → Language. For those who are familiar with Android development, the new resources are in the res/drawable-zh-rCN and res/values-zh-rCN directories.

If your device is currently set up to use Simplified Chinese, and if everything is working correctly, BatteryXPRT will open with a Simplified Chinese UI. For any language other than Simplified Chinese, the application will default to English.

Because this is an experiment, the localization is not complete. For example, the User Manual is still in English.

We’re close to the general release, so this will be a really quick Community Preview. We’ll release CP2 to the community on Monday, April 14. We’ll take comments for a week, through Monday, April 21. Assuming the community does not find any significant issues, we hope to release the final version shortly thereafter. Please let us know if you see any problems.

We are very aware that this is a tiny first step on the road to internationalization. We’re currently researching what it would take to localize our various assets such as the documentation and Web sites. If you have any comments or ideas, please contact us at BenchmarkXPRTsupport@principledtechnologies.com.

And if you’d like to try your hand at translating the user manual, let us know!

Eric

Comment on this post in the forums

Posted in Android, BatteryXPRT 2014 for Android, BenchmarkXPRT development community, Chinese, Collaborative benchmark development, Community Preview, MobileXPRT |

What is truth?

By Eric Hale

on April 3, 2014

Last fall, we discussed AnandTech’s report on benchmark cheating, and why open and honest benchmark development is so important. This week, benchmark optimization is back in the news, as AnandTech says that the HTC One M8 boosts its performance in whitelisted benchmarks. CNET has quoted HTC admitting that they not only boosted performance, but promoted the boost as a feature.

However, HTC has gone a step further, giving users the option to manually set the phone to high-performance mode. Some of us at PT have been involved in developing benchmarks for over 20 years, and it’s always been true that one person’s cheat can be another person’s valid optimization. Whatever their motivation, HTC’s position – that some people will choose higher performance over longer battery life – is not necessarily wrong.

BatteryXPRT recognizes that there’s a tradeoff between performance and battery life, and that you shouldn’t penalize a fast system the same way you would a system that simply has poor battery life. That’s why it reports a performance score along with the estimated battery life.

Do you have thoughts on optimizations, cheating, or ways to make the benchmarks better? Please drop us a line at BenchmarkXPRTSupport@principledtechnologies.com.

Eric

Comment on this post in the forums

Posted in BatteryXPRT 2014 for Android, Benchmarking, Collaborative benchmark development, Performance benchmarking, What makes a good benchmark? |

More grease, more tracks

By Eric Hale

on March 27, 2014

We’re always looking for ways to make it easier for people to use the benchmarks and interact with the community. I wrote about a new form for suggesting new benchmarks last week. However, we are not stopping there.

Although the community previews for TouchXPRT 2014 and BatteryXPRT 2014 for Android have been out for a few weeks, we’ve continued to improve them before their general releases. For example, we mentioned in TouchXPRT CP1 that we would continue making changes to the UI.

Taking a cue from WebXPRT 2013, the upcoming general releases of TouchXPRT 2014 and BatteryXPRT 2014 for Android will have a button in the UI for submitting results. This will make it easier for you to submit your results to be approved for publication on the PT Web site.

Although there’s not much time left before the general releases of these benchmarks, it’s not too late to send in your ideas. Even if we can’t use your suggestions in this release, we may be able to use them in the future. Let us know what you think by posting to the forum or emailing us at BenchmarkXPRTsupport@principledtechnologies.com.

Eric

Comment on this post in the forums

Posted in BatteryXPRT 2014 for Android, Collaborative benchmark development, Community Preview, Let us know your thoughts, TouchXPRT, TouchXPRT 2014, WebXPRT |

Greasing the track

By Eric Hale

on March 20, 2014

While BatteryXPRT 2014 for Android may still be in its community preview stage, we’re already looking for the next great XPRT. As we’ve said in the past, there’s always something new to measure. That’s where you come in − we want to hear about the benchmarks you want to see!

To make it easier for Community members to send us their ideas, we’ve made a form available in the member’s area. All members have to do is click Have an idea for a new XPRT? and fill in a description. This form saves you the trouble of composing an e-mail and makes it easier for us to track community interest.

However, even though we’re making life simpler for members of the community, anyone can suggest a new benchmark. If you have a good idea or any other comments you want us to hear, just send an e-mail to BenchmarkXPRTsupport@principledtechnologies.com. We also hope you’ll consider joining the community.

There’s a lot of new technology out there, and we’re very excited about upcoming possibilities. Speaking of new technology, check out Bill’s new blog, Tech Everywhere. He’s already written about electric cars, smart watches and a Bluetooth-enabled electric toothbrush. It’s definitely worth a read.

Eric

Comment on this post in the forums

Posted in Battery life, BatteryXPRT 2014 for Android, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview |

It’s always worth asking

By Eric Hale

on March 6, 2014

Last week, one of our community members asked for a couple of enhancements to WebXPRT. They wanted WebXPRT to be easier to automate, and they made two specific requests:

Add debug/result logs
Add the ability to start the test without UI interactions, by using a specific URL or a command line

This is a great example of why we put so much emphasis on the community. We have tried to make the BenchmarkXPRT benchmarks easy to use, but we don’t always face the same testing demands you do. If there’s anything we can do to make these tools more valuable, please let us know by posting on the forums or e-mailing us at benchmarkxprtsupport@principledtechnologies.com.

We are adding those abilities to the upcoming WebXPRT 2014 community preview. Speaking of the community preview, we have been working hard on it, and in the next few weeks, we’ll be talking about what will be in it.

Keep those requests coming!

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, WebXPRT, What makes a good benchmark? |

Golden tickets go global

By Bill Catchings

on February 27, 2014

I’ve spent the last week at Mobile World Congress (MWC) in Barcelona. The show is somewhat reminiscent of CES in January, but with a much more European/global feel. There are seven large halls of booths with phones, tablets, accessories, and gadgets of every sort.

I have felt Fujitsu’s haptic technology on a tablet, seen phones operating under water such as Kyocera’s and others, and even talked with someone at Oral-B about their Bluetooth-enabled electric toothbrush.

As at CES , my primary goal has not been to play with cool gadgets, see the sights, and eat good food. (Though, admittedly, all of that has been fun!) Instead, I was here to spread the word about the XPRT benchmarks. I handed out more of the special golden tickets that invite folks to be heard by joining the BenchmarkXPRT community and to take advantage of the opportunity to have PT test devices for free with all the applicable XPRT benchmarks.

If you were not able to attend CES or MWC, or I missed you and did not give you a golden ticket (or one of our cool shirts), please contact us. We don’t want any members of the community to feel left out! And, we’d love any of the rest of you readers to join the community and be part of creating the benchmarks for the next wave of cool phones, tablets, and devices.

-Bill Catchings

Comment on this post in the forums

Posted in BenchmarkXPRT development community, CES, Collaborative benchmark development, Golden Tickets, Mobile World Congress, Uncategorized |

Category: Collaborative benchmark development

Something new

What is truth?

More grease, more tracks

Greasing the track

It’s always worth asking

Golden tickets go global

Check out the other XPRTs: