Author Archives: Bill Catchings

TouchXPRT Web test update

on November 29, 2012

On October 22, we released TouchXPRT CP1 to the community. We took the unprecedented step of releasing CP1 without any restrictions on publishing results, and since then reviews of the Microsoft Slate and the Sony Duo 11 Convertible Laptop have used TouchXPRT.

The five scenarios in CP1 focus on media manipulation. While this is an important activity on touch devices, we know this is not all people do.

Next week, we plan to release Web-based scenarios. They use HTML 5 for a variety of activities. Unlike the original scenarios in TouchXPRT CP1, there will be nothing to download. You simply browse to a URL and run the tests online. There’s nothing to set up, just browse and run.

That means that there is nothing preventing you from running these tests on pretty much any system-browser combination that supports HTML5, not just on touch-based, Windows 8 devices like the rest of TouchXPRT. That started us wondering whether these Web-based activities should be thought of as a different benchmark entirely. When these tests are available, please try them out and let us know what you think. Do you think they are worthwhile for a broader range of devices? Do you think their scenario-based emphasis is a good alternative to existing lower-level Web-based tests?

Please keep in mind that it’s not too late to give feedback on TouchXPRT CP1. Let us know how you like the scenarios on CP1 as well as what other activities you would like to see.

-Bill Catchings

Comment on this post in the forums

Posted in Benchmarking, HTML5, Performance testing on tablets, Touch-based benchmarking, Touch-based devices, TouchXPRT, Web-based testing |

Touch device first impressions

By Bill Catchings

on November 1, 2012

One of the nice benefits of working on TouchXPRT has been the opportunity to play with the latest touch devices. The latest two are Windows 8/RT tablets, the ASUS Vivo Tab RT and the Microsoft Surface. I’ve been using the Vivo Tab without a keyboard and relying solely on touch while on the Surface, I’ve been using the cool keyboard/screen cover as well as touch. Overall, I’ve found the devices and Windows 8 very interesting, but still rough around the edges. One example of interesting on the interface formerly known as Metro (IFKaM) is the elongated landscape screens. They take a bit of getting used to, but may prove to be a good choice. On the rough-around-the-edges front, the first thing I currently do on any new device is to install Dropbox so I can get to some files to play around with. Dropbox is not yet available, though it certainly will be—hopefully soon!

IFKaM has a lot to it. On the one hand, it does not seem to me as intuitive as IOS. On the other hand, I have found that if I look hard enough, I can find a way to do something I’m used to doing in Windows. For example, even though I couldn’t use Dropbox, it dawned on me that I could just mount my file server and get to some files that way. As I spend more time with these devices, I’ll have to see if the tradeoff of a longer learning curve pays off in terms of more power and flexibility.

While I ponder that, I want to put in a mention about TouchXPRT 2013 CP1. The source is now available to members. Let us know if you see areas to improve TouchXPRT or the code itself as we work the final version over the next few months. Thanks!

Bill

Comment on this post in the forums

Posted in Touch-based benchmarking, Touch-based computing, Touch-based devices, TouchXPRT |

TouchXPRT in the fast lane

By Bill Catchings

on October 25, 2012

I titled last week’s blog “Putting the TouchXPRT pedal to the metal.” The metaphor still applies. On Monday, we released TouchXPRT 2013 Community Preview 1 (CP1). Members can download it here.

CP1 contains five scenarios based on our research and community feedback. The scenarios are Beautify Photo Album, Prepare Photos for Sharing, Convert Videos for Sharing, Export Podcast to MP3, and Create Slideshow from Photos.

Each scenario gives two types of results. There’s a rate, which allows for simple “bigger is better” comparisons. CP1 also gives the elapsed time for each scenario, which is easier to grasp intuitively. Each approach has its advantages. We’d like to get your feedback on whether you’d like us to pick one of those metrics for the final version of TouchXPRT 2013 or whether it makes more sense to include both. You’ll find a fuller description of the scenarios and the results in the TouchXPRT 2013 Community Preview 1 Design overview.

While you’re looking at CP1, we’re getting the source ready to release. To check out the source, you’ll need a system running Windows 8, with Visual Studio 2012 installed. We hope to release it on Friday. Keep your eye the TouchXPRT forums for more details.

Post your feedback to the TouchXPRT forum, or e-mail it to TouchXPRTSupport@principledtechnologies.com. Do you want more scenarios? Different metrics? A new UI feature? Let us know! Make TouchXPRT the benchmark you want it to be.

As I explained last week, we released CP1 without any restrictions on publishing results. It seems that AnandTech was the first to take advantage of that. Read AnandTech’s Microsoft Surface Review to see TouchXPRT in action.

We are hoping that other folks take advantage of CP1’s capability to act as a cross-platform benchmark on the new class of Windows 8 devices. Come join us in the fast lane!

Bill

Comment on this post in the forums

Posted in Benchmark metrics, Collaborative benchmark development, Touch-based benchmarking, Touch-based devices, TouchXPRT development process, TouchXPRT release cycle, TouchXPRT results |

Putting the TouchXPRT pedal to the metal

By Bill Catchings

on October 19, 2012

Since we announced TouchXPRT early this year, we’ve been following a typical benchmark development path. We started with the most important question—“What are people likely to do with a touch-based Windows 8 device?”—and built from there. We looked at what people are doing now with iOS- and Android-based devices. We worked with early Windows 8 units. We studied app stores. We spoke with members of the development community. And so on. When we were done studying, we moved to coding.

We’re making great progress, but something has been nagging at us: When Windows 8 tablets and other devices ship next week, there just won’t be much in the way of tools for measuring their performance when running Windows 8 apps. Sure, you may be able use standard benchmarks to assess the performance of typical desktop applications, but that won’t tell you how the devices will perform with tablet apps.

So, we’ve decided to put the pedal to the metal and provide everyone in our development community with a special treat. Sometime next week, before Windows 8 ships, we plan to release a sneak preview of TouchXPRT, the TouchXPRT 2013 Community Preview 1 (CP1).

CP1, as its name makes clear, is not the final TouchXPRT release. It is, though, a useful tool for beginning to measure Windows 8 device performance. It is also a great way for everyone in the community to see the current state of our thinking and to provide us feedback—rather than read a design spec, you can actually run this version of the tool and see what you think! (If you would like to read the informal design spec, check out http://www.hdxprt.com/forum/touchxprt2013cp1.php .)

To make the tool easier to evaluate and more useful to all of us, we’re also taking two more unusual steps:

1. We’re not putting any publication restrictions on this preview release. Test at will, and publish your findings.

2. We’re releasing the source code to all community members. If you’re curious about not just what we’re doing but how we’re doing it, you can find out.

We hope these steps will speed acceptance of TouchXPRT 2013 and foster more and faster feedback. Releasing a preview version is more work, because we have to do much of the work of a software release and on less-than-final code, but we believe the value to our community justifies the effort.

Next week, when we release CP1, I’ll go over more details, the known limitations, and how you can get us your feedback—feedback we very much want.

Between now and then, we’ll be readying CP1 for your use.

Bill

Comment on this post in the forums

Posted in Performance testing on tablets, Touch-based computing, TouchXPRT, TouchXPRT development process, TouchXPRT results |

TouchXPRT update

By Bill Catchings

on September 13, 2012

We have been spending a lot of our energy (and blog space) on HDXPRT 2012 over the last few weeks. We have not, however, forgotten about TouchXPRT.

As we discussed in the past, TouchXPRT will be a new benchmark designed for touch-based devices like tablets and touch-capable Ultrabooks. It will incorporate apps and scenarios tailored to touch-based environments and devices. The initial version will run on the Windows 8 environment formerly known as Metro.

Our current plan is to release an informal design overview later this month. We then hope to have a preview version of TouchXPRT available to community members in October. Depending upon feedback, we will work on improvements and fixes through January with a release in February. We will have more details in the forums and in blog entries over the next few weeks.

As always, we need feedback and input from the community. If you are not currently a member, now is the time to join so you can participate in specifying and developing TouchXPRT. We are excited about both the benchmark and this emerging product space. We hope you are, too!

I do have one thing to mention about HDXPRT 2012. Please remember that we are going to do a Webinar next week (Tuesday, 9/18 at 2:00pm ET). We plan to discuss HDXPRT 2012 and have plenty of time for questions from attendees. We’d love for you to join, learn more about the benchmark, and give us your thoughts and feedback. We will send out more details to community members in an email later this week.

Bill

Comment on this post in the forums

Posted in HDXPRT Webinar, TouchXPRT, TouchXPRT development process, TouchXPRT release cycle |

Growing the community

By Bill Catchings

on August 23, 2012

Growing the community

It’s probably stating the obvious to say that what makes the XPRT Development Community work is the people. The more people there are in the community, the better the resulting benchmarks will be. Naturally, that means we want more people in the community.

Part of PT’s role in building the community is to get more people involved. Over the last couple of years, that has meant doing a number of different things—from conducting Webinars (we’ll have the next one in September) to meeting at trade shows (we expect to be at CES again next year) to contacting individuals in person and via email to using social media like Twitter (follow us at @HDXPRT) and Facebook (like HDXPRT Development Community). As I mentioned last week, we are now working on some videos to help get out the message. Basically, we’ve been telling everyone we can about the benchmarks and the community that creates them.

One thing we haven’t done, however, is to actively enlist the community itself in building the community. Here are some ways you can help. Please like, follow, or whatever us on the different social media platforms. Invite others to join the community. Probably most importantly, use the benchmarks and publish the results. Now that HDXPRT 2012 is out, we want to see its results popping up in as many places as possible.

You can also help by telling us your ideas for how to grow the community. What else do you think we should be doing?

Bill

Comment on this post in the forums

Posted in Collaborative benchmark development, HDXPRT 2012 |

Author Archives: Bill Catchings

TouchXPRT Web test update

Touch device first impressions

TouchXPRT in the fast lane

Putting the TouchXPRT pedal to the metal

TouchXPRT update

Growing the community

Check out the other XPRTs: