Category: Benchmarking

Golden tickets

on January 9, 2014

We (Bill and Mark) are on our way home from CES. There were lots of cool things to see, from electric cars to health and fitness wearables to all manner of mobile devices. And, more, a whole lot more.

We enjoyed seeing many of those products, but that was not our primary mission at the show. Our main goal was to spread the word about the XPRT benchmarks. We did that by visiting multiple mobile-device members and giving out to many of them a very special golden ticket. Yes, we’re talking a physical, Willy Wonka-style golden ticket. The two-sided tickets look really cool.

One side invites folks to be heard by joining the BenchmarkXPRT community. The other offers them the opportunity to have PT test devices for free with all the applicable XPRT benchmarks. All a vendor has to do to get this free testing is send the device to PT. We hope to get many devices in-house and to provide a great many results on our Web sites.

We wore the new BenchmarkXPRT shirts as we walked the floor.

We will soon be sending one shirt—and one golden ticket—to each member of the community. Please make sure we have your latest mailing address so we can ship those to you.

-Bill & Mark

Comment on this post in the forums

Posted in BenchmarkXPRT development community, CES, Future of performance evaluation, Performance benchmarking, Principled Technologies |

Winding down the year

By Eric Hale

on December 19, 2013

We have a few things we need to touch on before the year ends.

In TouchXPRT 2014, we described some of the changes to TouchXPRT. We had hoped to have the Consumer Preview out in December, but we need to do a little more work on it. However, the TouchXPRT 2014 CP is coming after first of the year, and it’s worth the wait!

For the first time ever, we found a situation that caused WebXPRT to fail. We got a report of an iPhone 5s failing. We found that the user was using the private browsing mode. The WebXPRT Offline Notes test uses HTML5 local storage to sync and store notes locally. Safari and iOS Safari don’t support setting the HTML 5 localStorage property when using private browsing mode. Not using private browsing fixes the problem.

Remember that we are in the comment period for the new battery test. Community members should definitely check out the battery test RFC.

As we mentioned before, Bill and Mark are going to CES. If you’d like to chat, send an e-mail to benchmarkxprtsupport@principledtechnologies.com.

This is the last blog of 2013. The next post will be on January 2.

It’s been a great year. Thank you all.

Happy holidays!

Eric

Comment on this post in the forums

Posted in Battery life, Collaborative benchmark development, TouchXPRT, WebXPRT |

The battery life test RFC is here

By Eric Hale

on December 12, 2013

As we’ve mentioned in recent blog posts, we’ve been investigating a battery life test. We’ve now done enough research that we feel comfortable making a proposal for the test. We released the RFC, or request for comments, for upcoming battery life test yesterday. Our major objective with the RFC is to get your feedback.

Given the amount of time and effort it took to develop the methodology, we had to lay a lot of code. While nothing is written in stone, things are pretty far along.

For the next four weeks, until January 8, we want to hear what you what you think about the approach, the results calculations, and anything else you think is worth discussing. Notably, we are still debating if this should be part of MobileXPRT or be its own benchmark.

Community members can download the RFC from here. If you’re not yet a member of the community, now’s a great time to join. Not only can you comment on the RFC, you’ll be able to see the community preview when it becomes available.

Since the benchmark does not have a name yet, we haven’t dedicated a forum to it yet. Please post your comments in the BenchmarkXPRT forum, or mail them to benchmarkxprtsupport@principledtechnologies.com.

Let us know what you think! We probably won’t be able to get everything done, but now is the time to dream.

Eric

Comment on this post in the forums

Posted in Battery life, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development |

Sounds easy, but…

By Eric Hale

on December 5, 2013

Sounds easy, but…

In Endurance, Bill said that we were going to be investigating battery life testing. He also discussed some of the issues that make battery testing difficult to do well. Finally, he explained why we were looking at MobileXPRT as the basis for the first version of the battery life test.

Over the last couple of months, we have been experimenting with a number of different approaches to battery testing. We now think that we have enough empirical data that we can make a proposal. We are working on that now. It should be available to community members in the next couple of weeks.

We hope you’ll look at the proposal and let us know what you think. Your input is an essential part of developing a really great test. If you’re not a member of the community, it’s easy to join.

In other news, we’re going to CES and would love to talk with you. If you’d like to chat, send an e-mail to benchmarkxprtsupport@principledtechnologies.com.

Eric

Comment on this post in the forums

Posted in Benchmarking, Collaborative benchmark development, MobileXPRT, What makes a good benchmark? |

Diversity

By Eric Hale

on November 7, 2013

Fall is beautiful in North Carolina. The temperature is dropping. The leaves are changing color, making the hills scarlet and orange. And, of course, the stores have been decorated for Christmas since Halloween.

As we head into the biggest shopping season of the year, it’s a great time to be getting XPRT results from the hottest devices. In the last few weeks, we’ve published results from

tablets such as the Apple iPad Air, Google Nexus 7 2, and both the Microsoft Surface 2 and Microsoft Surface Pro 2
phones such as the Apple iPhone 5c, Apple iPhone 5s, and LG G2
devices you might not have expected, such as the Amazon Kindle Fire HDX, Barnes and Noble Nook HD+, and NVIDIA Shield

The diversity of devices is nice to see. The results come from PT testing, the press, and benchmark users. Note that you don’t have to be a community member to submit results. The person who submitted the MobileXPRT Nook HD+ results was not a member. If you’ve tested something interesting, send the results on!

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT, BenchmarkXPRT development community, MobileXPRT, Performance testing on tablets, Touch-based computing, Touch-based devices, TouchXPRT, TouchXPRT results |

Broadening our appeal

By Eric Hale

on October 31, 2013

As I mentioned last week, we’ve asked the PT design team to help improve the XPRT benchmarks. I’m learning a lot working with them. As someone who’s been involved with benchmarking a long time, it can be a shock to realize that there are people who think “ms” is a magazine, “geomean” has something to do with the environment, and “UX” sounds like it would be a great name for a band. But the fact is that most consumers don’t need to know any of these terms, so our benchmarks shouldn’t rely on them, either.

This collaboration already paying off. The PT design team rewrote the MobileXPRT FAQ, making it much more extensive, accessible, and fun to read. We think the new FAQ is greatly improved, and it’s certainly more informative. We’ll be upgrading the FAQs for the other benchmarks in the near future.

Our efforts are going far beyond FAQs. Data presentation, graphics, the basic UI design philosophy—everything is on the table. Let us know what you think by emailing benchmarkxprtsupport@principledtechnolgies.com or by posting on the forums.

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Let us know your thoughts, MobileXPRT |

Category: Benchmarking

Golden tickets

Winding down the year

The battery life test RFC is here

Sounds easy, but…

Diversity

Broadening our appeal

Check out the other XPRTs: