Category: TouchXPRT

Ending the year with a bang!

on January 2, 2013

As we promised in the blog post The newest member of the family, we made the WebXPRT 2013 community preview available this week. It has already been used in a review! The AnandTech review of the Acer Iconia W510 includes results from the WebXPRT 2013 community preview for that device and for the Microsoft Surface RT and the Apple iPad 4. The review has results from the TouchXPRT 2013 community preview for the Acer Iconia W510, Microsoft Surface RT, and the ASUS VivoTab RT as well.

Obviously, we’ve been doing some testing ourselves. Here’s a sampling of the devices on which we’ve successfully run WebXPRT:

Device	Processor	Operating system	Browser	Score	Confidence interval
HP Envy 2	1.8 GHz Intel Atom Z2760	Windows 8	Internet Explorer 10.0.92	201	+/- 6
Asus VivoTab RT	1.2 GHz Tegra 3 T30L	Windows RT	Internet Explorer 10.0.92	160	+/- 5
Kindle Fire	1.2 GHz ARM Cortex-A9	Android OS 2.3 (customized: 6.3.1_user_4107720)	Safari 5	92	+/- 2
ASUS-made Google Nexus 7	1.2 GHz Tegra 3 T30L	Android 4.2	Chrome 18	201	+/- 4
Motorola DroidX phone	1 GHz TI OMAP3630-1000	Android 4.5.621	Browser version 2.3.4	26	+/- 1
iPhone 5	1.3 GHz Apple A6	iOS 6.0.2	Safari 6	168	+/- 2
iPad mini	1GHz Apple A5	iOS 6.0.2	Safari 6	110	+/- 1
iPad 4	1.4 GHz Apple A6X	iOS 6.0.1	Safari 6	180	+/- 2

As the results above show, WebXPRT can run on a wide range of devices. We are working to get results on lots of different devices and would like your help. We’ll set up a forum thread for results that starts with these. We’ll then add additional ones we produce. Please respond in the thread with results you get.

In addition to performance results, the WebXPRT 2013 community preview also provides a report on the HTML 5 capabilities of your device. For those who want to know more about the capabilities of HTML 5, there’s more good news. The W3C community released the feature-complete spec for HTML 5 and Canvas 2D this week.

You can find an explanation of scenarios in the WebXPRT 2013 community preview, and an explanation of how it calculates its results in the WebXPRT 2013 CP1 Overview. Let us know what you think. There’s still time to help us shape the final version of both WebXPRT 2013 and TouchXPRT 2013.

Eric

Comment on this post in the forums

Posted in Benchmarking, HTML5, Let us know your thoughts, Performance testing on tablets, TouchXPRT, Web-based testing, WebXPRT |

TouchXPRT Web test update

By Bill Catchings

on November 29, 2012

On October 22, we released TouchXPRT CP1 to the community. We took the unprecedented step of releasing CP1 without any restrictions on publishing results, and since then reviews of the Microsoft Slate and the Sony Duo 11 Convertible Laptop have used TouchXPRT.

The five scenarios in CP1 focus on media manipulation. While this is an important activity on touch devices, we know this is not all people do.

Next week, we plan to release Web-based scenarios. They use HTML 5 for a variety of activities. Unlike the original scenarios in TouchXPRT CP1, there will be nothing to download. You simply browse to a URL and run the tests online. There’s nothing to set up, just browse and run.

That means that there is nothing preventing you from running these tests on pretty much any system-browser combination that supports HTML5, not just on touch-based, Windows 8 devices like the rest of TouchXPRT. That started us wondering whether these Web-based activities should be thought of as a different benchmark entirely. When these tests are available, please try them out and let us know what you think. Do you think they are worthwhile for a broader range of devices? Do you think their scenario-based emphasis is a good alternative to existing lower-level Web-based tests?

Please keep in mind that it’s not too late to give feedback on TouchXPRT CP1. Let us know how you like the scenarios on CP1 as well as what other activities you would like to see.

-Bill Catchings

Comment on this post in the forums

Posted in Benchmarking, HTML5, Performance testing on tablets, Touch-based benchmarking, Touch-based devices, TouchXPRT, Web-based testing |

Lots of things happening!

By Eric Hale

on November 15, 2012

If you’re interested in the interface formerly known as Metro (IFKaM), it’s an exciting time. A lot has happened in the last couple of weeks.

Perhaps the most exciting news is that the first Windows 8 tablets using the Intel Atom (Clover Trail) processor have started hitting the shelves, with many more coming in the next few weeks.

Nor are things are boring in the Windows RT space.

The first Windows RT tablet with LTE capabilities will be available this week from AT&T. You’ll find the press release here.
Microsoft released new firmware for Windows RT devices. Anecdotal evidence suggests that the firmware update improves performance. This is something we will be investigating in the near future.
Microsoft added Windows RT to its Compatibility Center. You can go there to find a comprehensive list of devices that are compatible with Windows RT. Because TouchXPRT’s scenarios focus on media creation and manipulation, we were particularly interested in cameras, media players, and TV devices.

With so much news, TouchXPRT needs some news of its own. When we released the community preview, we gave permission to publish results – the first time that’s happened for a benchmark that was not final. Now we are going to be creating an online database of results from the community preview. Look for it to go live next week.

With so many new devices and updated old ones, there’s a lot out there to test. We hope that you’ll share the results of your testing, and submit them to the database. The more results we get, the more valuable the database will be.

One final note: Microsoft released its first security patches for Windows 8 on Tuesday. If you are running Windows 8, whether on a touch device or not, check them out.

Eric

Comment on this post in the forums

Posted in Touch-based devices, TouchXPRT, TouchXPRT results, Windows 8 |

Touch device first impressions

By Bill Catchings

on November 1, 2012

One of the nice benefits of working on TouchXPRT has been the opportunity to play with the latest touch devices. The latest two are Windows 8/RT tablets, the ASUS Vivo Tab RT and the Microsoft Surface. I’ve been using the Vivo Tab without a keyboard and relying solely on touch while on the Surface, I’ve been using the cool keyboard/screen cover as well as touch. Overall, I’ve found the devices and Windows 8 very interesting, but still rough around the edges. One example of interesting on the interface formerly known as Metro (IFKaM) is the elongated landscape screens. They take a bit of getting used to, but may prove to be a good choice. On the rough-around-the-edges front, the first thing I currently do on any new device is to install Dropbox so I can get to some files to play around with. Dropbox is not yet available, though it certainly will be—hopefully soon!

IFKaM has a lot to it. On the one hand, it does not seem to me as intuitive as IOS. On the other hand, I have found that if I look hard enough, I can find a way to do something I’m used to doing in Windows. For example, even though I couldn’t use Dropbox, it dawned on me that I could just mount my file server and get to some files that way. As I spend more time with these devices, I’ll have to see if the tradeoff of a longer learning curve pays off in terms of more power and flexibility.

While I ponder that, I want to put in a mention about TouchXPRT 2013 CP1. The source is now available to members. Let us know if you see areas to improve TouchXPRT or the code itself as we work the final version over the next few months. Thanks!

Bill

Comment on this post in the forums

Posted in Touch-based benchmarking, Touch-based computing, Touch-based devices, TouchXPRT |

TouchXPRT in the fast lane

By Bill Catchings

on October 25, 2012

I titled last week’s blog “Putting the TouchXPRT pedal to the metal.” The metaphor still applies. On Monday, we released TouchXPRT 2013 Community Preview 1 (CP1). Members can download it here.

CP1 contains five scenarios based on our research and community feedback. The scenarios are Beautify Photo Album, Prepare Photos for Sharing, Convert Videos for Sharing, Export Podcast to MP3, and Create Slideshow from Photos.

Each scenario gives two types of results. There’s a rate, which allows for simple “bigger is better” comparisons. CP1 also gives the elapsed time for each scenario, which is easier to grasp intuitively. Each approach has its advantages. We’d like to get your feedback on whether you’d like us to pick one of those metrics for the final version of TouchXPRT 2013 or whether it makes more sense to include both. You’ll find a fuller description of the scenarios and the results in the TouchXPRT 2013 Community Preview 1 Design overview.

While you’re looking at CP1, we’re getting the source ready to release. To check out the source, you’ll need a system running Windows 8, with Visual Studio 2012 installed. We hope to release it on Friday. Keep your eye the TouchXPRT forums for more details.

Post your feedback to the TouchXPRT forum, or e-mail it to TouchXPRTSupport@principledtechnologies.com. Do you want more scenarios? Different metrics? A new UI feature? Let us know! Make TouchXPRT the benchmark you want it to be.

As I explained last week, we released CP1 without any restrictions on publishing results. It seems that AnandTech was the first to take advantage of that. Read AnandTech’s Microsoft Surface Review to see TouchXPRT in action.

We are hoping that other folks take advantage of CP1’s capability to act as a cross-platform benchmark on the new class of Windows 8 devices. Come join us in the fast lane!

Bill

Comment on this post in the forums

Posted in Benchmark metrics, Collaborative benchmark development, Touch-based benchmarking, Touch-based devices, TouchXPRT development process, TouchXPRT release cycle, TouchXPRT results |

Putting the TouchXPRT pedal to the metal

By Bill Catchings

on October 19, 2012

Since we announced TouchXPRT early this year, we’ve been following a typical benchmark development path. We started with the most important question—“What are people likely to do with a touch-based Windows 8 device?”—and built from there. We looked at what people are doing now with iOS- and Android-based devices. We worked with early Windows 8 units. We studied app stores. We spoke with members of the development community. And so on. When we were done studying, we moved to coding.

We’re making great progress, but something has been nagging at us: When Windows 8 tablets and other devices ship next week, there just won’t be much in the way of tools for measuring their performance when running Windows 8 apps. Sure, you may be able use standard benchmarks to assess the performance of typical desktop applications, but that won’t tell you how the devices will perform with tablet apps.

So, we’ve decided to put the pedal to the metal and provide everyone in our development community with a special treat. Sometime next week, before Windows 8 ships, we plan to release a sneak preview of TouchXPRT, the TouchXPRT 2013 Community Preview 1 (CP1).

CP1, as its name makes clear, is not the final TouchXPRT release. It is, though, a useful tool for beginning to measure Windows 8 device performance. It is also a great way for everyone in the community to see the current state of our thinking and to provide us feedback—rather than read a design spec, you can actually run this version of the tool and see what you think! (If you would like to read the informal design spec, check out http://www.hdxprt.com/forum/touchxprt2013cp1.php .)

To make the tool easier to evaluate and more useful to all of us, we’re also taking two more unusual steps:

1. We’re not putting any publication restrictions on this preview release. Test at will, and publish your findings.

2. We’re releasing the source code to all community members. If you’re curious about not just what we’re doing but how we’re doing it, you can find out.

We hope these steps will speed acceptance of TouchXPRT 2013 and foster more and faster feedback. Releasing a preview version is more work, because we have to do much of the work of a software release and on less-than-final code, but we believe the value to our community justifies the effort.

Next week, when we release CP1, I’ll go over more details, the known limitations, and how you can get us your feedback—feedback we very much want.

Between now and then, we’ll be readying CP1 for your use.

Bill

Comment on this post in the forums

Posted in Performance testing on tablets, Touch-based computing, TouchXPRT, TouchXPRT development process, TouchXPRT results |

Category: TouchXPRT

Ending the year with a bang!

TouchXPRT Web test update

Lots of things happening!

Touch device first impressions

TouchXPRT in the fast lane

Putting the TouchXPRT pedal to the metal

Check out the other XPRTs: