Category: TouchXPRT

One now, one later

on July 9, 2015

Windows 10 has been on our mind this week.

Last week, we explained why the Notes test in WebXPRT would not complete when running in Edge on Windows 10. We’ve implemented the fix we discussed and have finished testing the updated versions of WebXPRT 2013 and WebXPRT 2015. We’ll release them by the end of the week. Results from the new versions are comparable with results from the existing versions.

In the current Windows 10 Mobile Beta, WebXPRT 2015 does not scroll correctly in portrait mode. It does scroll correctly in landscape mode, so, as a workaround, one can run it that way on the Windows 10 Mobile Beta.

Speaking of Windows 10 Mobile, we’ve talked before about TouchXPRT 2016 and how its purpose is to compare Windows 10 across different device types. However, Microsoft has said that Windows 10 Mobile won’t be available until after the release of Windows 10 on PCs. More importantly, the APIs and development tools won’t be final until July 29. Once Microsoft releases those tools, we’ll do our builds and tests and release a community preview.

That being said, TouchXPRT 2014 is the tool to use for comparing Windows 8.1 and Windows 10. By the time mobile devices running Windows 10 are available, TouchXPRT 2016 will be available.

Eric

Posted in TouchXPRT, TouchXPRT 2014, TouchXPRT 2016, TouchXPRT development process, WebXPRT, WebXPRT 2015, Windows 10, Windows 8.1 |

Something old, something new

By Eric Hale

on June 25, 2015

Last week, we talked about porting TouchXPRT 2014 to be a Windows 10 universal app. This will let it run on devices running Windows 10 and those running Windows 10 mobile.

We won’t be retiring TouchXPRT 2014 when we release the Windows 10 universal app version. Windows 8 doesn’t support Windows 10 universal apps, but Windows 10 will be able to run Windows 8 applications. This means you’ll also be continue to be able to use TouchXPRT 2014 to test Windows 8 based systems, as well as to compare Windows 8 and Windows 10 performance.

The results from TouchXPRT 2014 and the universal app version of the benchmark won’t be compatible. Even though the test scenarios will be the same, the porting process means that we have to change the APIs the benchmark is using and rebuild the benchmark with different tools.

We’re currently debating changing the way we version the benchmarks. As the number of versions of each benchmark increases, it may make sense to move away from year-based versioning. This will obviously affect what we call the new Windows 10 version of TouchXPRT. If you have any thoughts on this, please let us know!

Eric

Posted in BenchmarkXPRT development community, Collaborative benchmark development, Future of performance evaluation, TouchXPRT, TouchXPRT 2014, TouchXPRT release cycle, Windows 10, Windows 8 |

Hoping for a perfect 10

By Eric Hale

on June 18, 2015

As many of you know by now, the release date for Windows 10 is July 29. As we’ve said before, we are hard at work getting TouchXPRT ready for Windows 10. We’ve succeeded in building TouchXPRT as a universal app, and it’s now running on Windows 10. We haven’t successfully run it on Windows 10 Phone yet, but we’re working on that.

Unfortunately, I can’t share any performance data. The EULA for the current build of Windows 10 (build 10143 as I’m writing this) forbids publishing benchmark results without prior written approval from Microsoft.

We’ll continue testing and refining the porting of TouchXPRT to Windows 10. Our goal is to release it as a universal app to the community in July.

What are your experiences testing Windows 10? We’d love to hear about them!

Eric

Posted in Performance benchmarking, TouchXPRT, TouchXPRT development process, TouchXPRT release cycle, Windows 10 |

What are the implications?

By Eric Hale

on May 14, 2015

It’s been a couple of weeks since the Microsoft Build 2015 conference. There was a lot of interesting news and we are still digesting what it means for the XPRTS, especially TouchXPRT.

Rebuilding TouchXPRT as a universal app has the potential to let it run on a much wider range of devices: PCs, tablets, phones, even the Xbox. This would give TouchXPRT the kind of versatility that we enjoy in the Android space with MobileXPRT and BatteryXPRT.

It’s a lot more complicated to sort out the implications of Microsoft Continuum, which allows you to use your phone as a computer by connecting it to a docking station. The features of your device and the way the apps behave can change based on the display available. Connect the phone to a docking station and it behaves like a desktop. It also means that the hardware and features available on a device could potentially change while you are testing the device. TouchXPRT would need to detect any such changes and respond appropriately.

That’s a lot to think about, and we’ve been experimenting. If you have any thoughts about Windows 10 and benchmarking, please let us know.

Eric

Posted in BatteryXPRT 2014 for Android, MobileXPRT, TouchXPRT, TouchXPRT release cycle, Windows 10 |

Back to Barcelona

By Eric Hale

on February 26, 2015

It’s time for one of Bill’s favorite shows of the year. Mobile World Congress (MWC) starts Monday in Barcelona. Talking about technology in Barcelona, now that’s a plum assignment!

Bill’s got a lot to talk about. Since last year’s MWC, here’s some of what’s happened:

We released two new benchmarks to the public, BatteryXPRT 2014 for Android and CrXPRT 2015.
We released updates to two benchmarks, TouchXPRT 2014 and HDXPRT 2014.
We created our first training course, for BatteryXPRT.
We released two videos Mine is better: TouchXPRT 2014 and CrXPRT in action.

(more…)

Posted in BatteryXPRT 2014 for Android, BenchmarkXPRT development community, CrXPRT, HDXPRT 2014, Mobile World Congress, TouchXPRT 2014, Trade Shows |

More than Chromebooks

By Eric Hale

on January 22, 2015

Recently, we got a question from AnandTech asking how hard it would be to get CrXPRT to run on Chrome on Windows.

The short answer is that getting it to run isn’t difficult. However, as we have written about many times in the past, it’s not enough for a benchmark to simply run on a device. The results it produces must be comparable. Even if the benchmark appears to run identically, small differences in timers or how the platform reports its state can have a big impact.

To date, we have been dealing only with Chromebooks of various flavors. However, we’re now testing CrXPRT on a much wider range of devices. The results are generally looking reasonable, although we’re finding some minor issues. For example, the battery information isn’t as granular on some devices as it is on Chromebooks.

As soon as we are sure that CrXPRT is returning reasonable results on the new classes of devices, you’ll be the first to know!

In other news, we’re planning to remove TouchXPRT 2013 from the Windows Store on February 16. We wanted to have a period of overlap with TouchXPRT 2014 to allow labs time to transition. It’s been over 6 months, and we feel this is a good time. TouchXPRT 2013 will remain available in the members’ area of the BenchmarkXPRT.com Web site.

Eric

Posted in BenchmarkXPRT development community, Chrome OS, Collaborative benchmark development, CrXPRT, TouchXPRT, What makes a good benchmark?, Windows 8, Windows 8.1 |

Category: TouchXPRT

One now, one later

Something old, something new

Hoping for a perfect 10

What are the implications?

Back to Barcelona

More than Chromebooks

Check out the other XPRTs: