Category: Let us know your thoughts

More grease, more tracks

on March 27, 2014

We’re always looking for ways to make it easier for people to use the benchmarks and interact with the community. I wrote about a new form for suggesting new benchmarks last week. However, we are not stopping there.

Although the community previews for TouchXPRT 2014 and BatteryXPRT 2014 for Android have been out for a few weeks, we’ve continued to improve them before their general releases. For example, we mentioned in TouchXPRT CP1 that we would continue making changes to the UI.

Taking a cue from WebXPRT 2013, the upcoming general releases of TouchXPRT 2014 and BatteryXPRT 2014 for Android will have a button in the UI for submitting results. This will make it easier for you to submit your results to be approved for publication on the PT Web site.

Although there’s not much time left before the general releases of these benchmarks, it’s not too late to send in your ideas. Even if we can’t use your suggestions in this release, we may be able to use them in the future. Let us know what you think by posting to the forum or emailing us at BenchmarkXPRTsupport@principledtechnologies.com.

Eric

Comment on this post in the forums

Posted in BatteryXPRT 2014 for Android, Collaborative benchmark development, Community Preview, Let us know your thoughts, TouchXPRT, TouchXPRT 2014, WebXPRT |

Broadening our appeal

By Eric Hale

on October 31, 2013

As I mentioned last week, we’ve asked the PT design team to help improve the XPRT benchmarks. I’m learning a lot working with them. As someone who’s been involved with benchmarking a long time, it can be a shock to realize that there are people who think “ms” is a magazine, “geomean” has something to do with the environment, and “UX” sounds like it would be a great name for a band. But the fact is that most consumers don’t need to know any of these terms, so our benchmarks shouldn’t rely on them, either.

This collaboration already paying off. The PT design team rewrote the MobileXPRT FAQ, making it much more extensive, accessible, and fun to read. We think the new FAQ is greatly improved, and it’s certainly more informative. We’ll be upgrading the FAQs for the other benchmarks in the near future.

Our efforts are going far beyond FAQs. Data presentation, graphics, the basic UI design philosophy—everything is on the table. Let us know what you think by emailing benchmarkxprtsupport@principledtechnolgies.com or by posting on the forums.

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Let us know your thoughts, MobileXPRT |

Unchecked ambition

By Bill Catchings

on January 17, 2013

With 10 applications and five scenarios, HDXPRT 2012 is the most ambitious version of HDXPRT ever. However, as we said in the blog post of the same name, There is such a thing as too much. We heard that message from you as well: HDXPRT 2012 is too big. At 11 GB, we can’t make it available for download, and it won’t fit on a single DVD.

It also takes longer to use than many of you would like. The installation takes a couple of hours, and each iteration takes 2 to 3 hours. Because a valid run of HDXPRT 2012 comprises three iterations, getting a single result takes all day.

As we work to reduce the size of HDXPRT 2013, we’re being careful. While we need to shrink the benchmark and its running time, we want to be sure that we don’t compromise its essential value. As part of this process, we’re looking at both the selection of applications and the ways we can reengineer the scenarios.

If you have ideas about HDXPRT 2012 that you haven’t sent yet, now is a good time to do that. We’ll be sending out the HDXPRT 2013 RFC in the next couple of weeks. Once it’s out, we will look forward to hearing your comments.

I also wanted to mention that as we had to cancel CES at the last minute, we’re planning to do a Webinar next week (Tuesday, January 22) to cover the material we planned to present there. We’ll send out an email later this week with more details.

Bill

Comment on this post in the forums

Posted in BenchmarkXPRT, CES, HDXPRT, HDXPRT 2013, HDXPRT Development Community benefits, Let us know your thoughts |

The umbrella is open

By Eric Hale

on January 11, 2013

Last week, we told you that we were going to be reorganizing the XPRT benchmarks under the BenchmarkXPRT umbrella. We’ve now taken a big step forward. The BenchmarkXPRT.com Web site is live.

BenchmarkXPRT.com is now the one place to go for news about all the benchmarks. When you visit the HDXPRT site, you’ll notice that the TouchXPRT and WebXPRT links are gone. The HDXPRT.com site is now specific to HDXPRT, and so matches the TouchXPRT.com and WebXPRT.com sites.

As part of this effort, we have also created a new Facebook page and Twitter feed for BenchmarkXPRT and they are already active. However, so that no one misses any important information, we will redundantly post the news to the old HDXPRT page and tweet to the old HDXPRT account until April 1.

The next step in the process will be converting the existing HDXPRT Development Community memberships to be BenchmarkXPRT Development community memberships. The new memberships will work like the ones you have now. However, you’ll also have the ability to follow specific benchmarks. That means that any time there’s news related to a specific benchmark, you’ll be notified. In the next couple of days, we will send an email to the members explaining this in more detail and giving the specific date of the cut over.

We will also be renaming the HDXPRT forums to be BenchmarkXPRT forums, and reorganizing the forums themselves.

As we’ve said before, this new organization better fits with the reality now, and gives us flexibility for the future. Who knows how when the family might grow again?

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT, HDXPRT, Let us know your thoughts, Principled Technologies |

Ending the year with a bang!

By Eric Hale

on January 2, 2013

As we promised in the blog post The newest member of the family, we made the WebXPRT 2013 community preview available this week. It has already been used in a review! The AnandTech review of the Acer Iconia W510 includes results from the WebXPRT 2013 community preview for that device and for the Microsoft Surface RT and the Apple iPad 4. The review has results from the TouchXPRT 2013 community preview for the Acer Iconia W510, Microsoft Surface RT, and the ASUS VivoTab RT as well.

Obviously, we’ve been doing some testing ourselves. Here’s a sampling of the devices on which we’ve successfully run WebXPRT:

Device	Processor	Operating system	Browser	Score	Confidence interval
HP Envy 2	1.8 GHz Intel Atom Z2760	Windows 8	Internet Explorer 10.0.92	201	+/- 6
Asus VivoTab RT	1.2 GHz Tegra 3 T30L	Windows RT	Internet Explorer 10.0.92	160	+/- 5
Kindle Fire	1.2 GHz ARM Cortex-A9	Android OS 2.3 (customized: 6.3.1_user_4107720)	Safari 5	92	+/- 2
ASUS-made Google Nexus 7	1.2 GHz Tegra 3 T30L	Android 4.2	Chrome 18	201	+/- 4
Motorola DroidX phone	1 GHz TI OMAP3630-1000	Android 4.5.621	Browser version 2.3.4	26	+/- 1
iPhone 5	1.3 GHz Apple A6	iOS 6.0.2	Safari 6	168	+/- 2
iPad mini	1GHz Apple A5	iOS 6.0.2	Safari 6	110	+/- 1
iPad 4	1.4 GHz Apple A6X	iOS 6.0.1	Safari 6	180	+/- 2

As the results above show, WebXPRT can run on a wide range of devices. We are working to get results on lots of different devices and would like your help. We’ll set up a forum thread for results that starts with these. We’ll then add additional ones we produce. Please respond in the thread with results you get.

In addition to performance results, the WebXPRT 2013 community preview also provides a report on the HTML 5 capabilities of your device. For those who want to know more about the capabilities of HTML 5, there’s more good news. The W3C community released the feature-complete spec for HTML 5 and Canvas 2D this week.

You can find an explanation of scenarios in the WebXPRT 2013 community preview, and an explanation of how it calculates its results in the WebXPRT 2013 CP1 Overview. Let us know what you think. There’s still time to help us shape the final version of both WebXPRT 2013 and TouchXPRT 2013.

Eric

Comment on this post in the forums

Posted in Benchmarking, HTML5, Let us know your thoughts, Performance testing on tablets, TouchXPRT, Web-based testing, WebXPRT |

The newest member of the family

By Eric Hale

on December 14, 2012

In his blog post TouchXPRT Web test update, Bill mentioned that we would be releasing Web-based workloads for the community to try out. Although we developed them as part of TouchXPRT, the cross-platform nature of these tests suggested to us that they should stand on their own.

After seeking community input, we have decided to make them a separate benchmark. So, we are proud to announce WebXPRT. The first community preview will be available mid-next week. WebXPRT CP1 contains four workloads: Photo Effects, Face Detect, Stocks Dashboard, and Offline Notes. Because the workloads are all HMTL5 based, they run on a wide variety of devices and operating systems—from iPad tablets to Android phones to Windows computers.

As with all community previews, we are very interested in your opinion. Tell us what you like or don’t like about the workloads. Are there other use cases you’d like to see?

Now that we have three benchmarks, the old HDXPRT-centric model of the community needs updating. Earlier this week, a message went out to the community announcing that we will be reorganizing the benchmarks under the umbrella of BenchmarkXPRT. This reorganization will touch all aspects of the community, from the Web site to Facebook, Twitter, and even the memberships themselves. We’ll be rolling out these changes over the next few weeks, and we’ll keep you informed every step of the way.

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT, HTML5, Let us know your thoughts, Web-based testing, WebXPRT |

Category: Let us know your thoughts

More grease, more tracks

Broadening our appeal

Unchecked ambition

The umbrella is open

Ending the year with a bang!

The newest member of the family

Check out the other XPRTs: