Category: Collaborative benchmark development

In the works

on June 26, 2014

A couple of weeks ago, we mentioned stability issues with BatteryXPRT 2014 for Android when running on systems with only 1 GB of RAM. After some restructuring, we now have a build that’s stable on those devices. We’re finishing testing, and it’s looking good. We’ll be making the new build available at BatteryXPRT.com and in the Google Play store early next week. Battery life and performance results from the new build are equivalent to those from the previous build, so your existing results are still valid and comparable.

Speaking of BatteryXPRT, we have something special coming soon. I can’t say what it is yet, but we haven’t done anything like it before.

We’re also getting closer to a general release for HDXPRT 2014. As we’ve explained before, the new HDXPRT runs three iterations in less than 2 hours, and will be downloadable, so the entire installation and testing process is much quicker than before. We’re currently working out a few kinks, but we expect the general release very soon.

Also, we’re still accepting ideas for naming the Chrome-related XPRT now in development. Share your ideas at BenchmarkXPRTsupport@principledtechnologies.com.

Eric

Comment on this post in the forums

Posted in BatteryXPRT 2014 for Android, Chrome OS, Collaborative benchmark development, HDXPRT 2014 |

Testing the waters

By Eric Hale

on June 19, 2014

A couple of weeks ago, we talked about some of our ideas for a new XPRT designed for Google’s Chrome OS. We’ve been working with some of these ideas and, while we’re still in the experimental stage, things look promising so far.

As we mentioned in the earlier blog, we’re trying WebXPRT as a base for the performance part of the test. So far, the performance component is working well. In addition to modified WebXPRT tests, we’re also trying some things that are not part of the WebXPRT 2013 workload.

We’ve been able to get battery life, but it’s been challenging and we haven’t found a way to avoid using Chrome’s Developer Mode. Accessing Developer Mode in Chrome can be tricky and requires different steps for each hardware manufacturer. We’re hoping to find ways to make battery life testing easier.

I’ve been vague about the tests because they’re likely to change over the next few weeks. We’re experimenting with both browser-based and Native Client-based performance tasks. As they firm up, I’ll be able to share more information.

Challenges aside, we’re excited about this new benchmark, and committed to making it as effective as possible. We’d still love feedback on a name, so feel free to contact us at BenchmarkXPRTsupport@principledtechnologies.com with your ideas.

Eric

Comment on this post in the forums

Posted in Battery life, Chrome OS, Collaborative benchmark development, Performance benchmarking, What makes a good benchmark? |

The name game

By Eric Hale

on June 5, 2014

In Something shiny, we discussed the leading contender in our search for new benchmark ideas, a benchmark tailored especially for the Chrome OS, and we’ve been looking at what workloads would make sense.

As we said, the ability to measure battery life would be useful. That’s not easy in the Chrome environment. We think we may be able to do it, but the Chromebook may have to be in developer mode. Even so, we can leverage what we’ve learned from BatteryXPRT to get a reliable estimate of battery life in less than a working day.

Measuring performance, however, is a must. We’ve been looking at the existing WebXPRT workloads as well as other applications, such as education apps, online games, HD video playback, music playback, and more. We’re also looking for areas where using native client execution makes sense, such as higher-resolution photo editing.

In addition, we’re thinking about what we might call this benchmark. ChromeXPRT would be obvious, but probably wouldn’t pass Google’s naming restrictions.

Do you have ideas for the benchmark’s name? Are there Chrome-based benchmark workloads you’d love to see? Let us know at BenchmarkXPRTsupport@principledtechnologies.com!

Eric

Comment on this post in the forums

Posted in Chrome OS, Collaborative benchmark development, Future of performance evaluation, WebXPRT |

Stronger, faster, and definitely better

By Eric Hale

on May 29, 2014

This week we released HDXPRT 2014 Community Preview 1 (CP1) to BenchmarkXPRT Development Community members. For those community members who’ve been around since the beginning, you’ll notice how much HDXPRT has changed. We’ve trimmed down HDXPRT without reducing the value of the test. While HDXPRT 2012 required multiple installation DVDs, HDXPRT 2014 CP1 is available for download in the member’s area. In addition, it took HDXPRT 2012 at least five hours to complete the recommended three test iterations. HDXPRT 2014 is much faster – the total time required for install and three test iterations is less than two hours!

Like all versions of HDXPRT, CP1 uses real-world media applications such as Adobe® Photoshop® Elements 12, Apple® iTunes®11.1, CyberLink MediaEspresso 6.7, and others to perform common consumer tasks like editing photos, converting videos, and editing music files.

If you’re a community member, give CP1 a try. If you have questions about HDXPRT 2014 or any of the XPRTs, please contact us at BenchmarkXPRTsupport@principledtechnologies.com. If you’d like to be a part of future XPRT development and community previews, now’s a great time to join!

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, Future of performance evaluation, HDXPRT 2012, HDXPRT 2014 |

BatteryXPRT 2014 for Android and TouchXPRT 2014 source codes are now available.

By Eric Hale

on May 23, 2014

As we mentioned in the BatteryXPRT 2014 and TouchXPRT 2014 release announcements, we are making the sources available to community members.

Download the BatteryXPRT 2014 source here.

Download the TouchXPRT 2014 source here.

We’re also posting BatteryXPRT build instructions and TouchXPRT build instructions in the Members Area. If you want more information, please contact BenchmarkXPRTsupport@principledtechnologies.com.

We look forward to your feedback!

Eric

Comment on this post in the forums

Posted in BatteryXPRT 2014 for Android, Collaborative benchmark development, Source code, TouchXPRT 2014 |

Something shiny

By Eric Hale

on May 22, 2014

As I mentioned in Looking for the next big thing, we’ve received some ideas for benchmarks recently. While there were several good ideas, the one that rose to the top of the list was a benchmark for Chrome-based devices.

For the past year or so, Chromebooks have been the fastest-growing segment of the PC market, so there’s a lot of interest. Given the rising profile of Chromebooks, a Chrome OS benchmark seems like a good fit with the existing XPRTs.

We’ve just started batting around ideas about what a Chrome OS benchmark might look like. Because the browser is the interface, WebXPRT is the natural place to start. However, WebXPRT runs on all platforms. We would like a Chrome benchmark to consider Chrome‑specific technologies like Google Native Client. If possible, it would also be great to get the battery life as well as performance.

Those are a few of our ideas. Do you have ideas about what you would like to see in a Chrome OS benchmark? If so, what would you consider most important for it to include? Let us know at benchmarkxprtsupport@principledtechnologies.com!

And remember, we’re always looking for new ideas. If there’s a benchmark you’d love to see, let us know.

Eric

Comment on this post in the forums

Posted in Collaborative benchmark development, Future of performance evaluation, Google Chrome, WebXPRT, What makes a good benchmark? |

Category: Collaborative benchmark development

In the works

Testing the waters

The name game

Stronger, faster, and definitely better

BatteryXPRT 2014 for Android and TouchXPRT 2014 source codes are now available.

Something shiny

Check out the other XPRTs: