Category: Benchmarking

MobileXPRT 2015 Community Preview is here!

on August 14, 2015

Today we are releasing the MobileXPRT 2015 Community Preview (CP1). MobileXPRT 2015 is compatible with systems running Android 4.4 and above. It is a 64-bit application, but will work on both 32-bit and 64-bit hardware.

MobileXPRT 2015 includes the same performance workloads as MobileXPRT 2013, but not the UX Tests. Should you need the UX tests, MobileXPRT 2013 will continue to be available.

Because this is a community preview, you have to be a community member to download it. Members may download the preview from the MobileXPRT tab in the Members’ Area. There are two options for downloading:

MobileXPRT 2015 CP1 application and test data, 249 MB
MobileXPRT 2015 CP1 application only, 18 MB (downloads test data the first time it’s run)

After trying out CP1, please send in your comments. Either post them to the forum or mail them to BenchmarkXPRTsupport@principledtechnologies.com. If you send us information that’s relevant to the entire community, we may post an anonymous version of your comments to the forum.

Thanks for your participation in the community!

It’s almost here!

By Eric Hale

on August 13, 2015

This Friday, we will be releasing the community preview of MobileXPRT 2015. BenchmarkXPRT Development Community members will be able to download the installation APK from the MobileXPRT tab in the Members’ Area. We are happy to say that, although we thought that MobileXPRT 2015 would only run on Android 5.0 and above, we have been able to extend its compatibility back to Android 4.4 (KitKat).

As we’ve said, the new MobileXPRT is a 64-bit application. Although it will work on both 32-bit and 64-bit hardware, the code will automatically select 64-bit when available. If you want to compare 32-bit performance to 64-bit on a single device, MobileXPRT 2013 will still be available. It’s not a problem to have both versions of MobileXPRT installed at the same time, and results from the two versions of the benchmark are comparable. On the same note, remember that the UX tests are no longer available in MobileXPRT 2015, but they are available as part of MobileXPRT 2013 if you need them.

As always, the results from the community preview will be comparable to the general release.

We hope you enjoy the new MobileXPRT, and we look forward to seeing your results!

Eric

Posted in BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, MobileXPRT, MobileXPRT 2015 |

Explaining the BenchmarkXPRT Development Community

By Bill Catchings

on August 6, 2015

Over the last year, I’ve spoken about the XPRT benchmarks with people across America, in China (at IDF Shenzhen), and in Europe (at Mobile World Congress). I regularly found myself having to explain how the BenchmarkXPRT Development Community works. While I was glad to do so, I found myself wishing that the many people I wasn’t able to talk with could also learn about how the community works.

To help make that happen, we’ve developed a simple and engaging video. My (admittedly prejudiced) opinion is that it does a great job of explaining how the community works in less than two minutes. That’s a lot faster than I was able to explain it to folks!

We hope you enjoy the video. And we hope you’ll pass it along to other folks who aren’t already part of the community so they can learn how it works and hopefully be persuaded to join us. Thanks!

Bill

Posted in Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, IDF, What makes a good benchmark? |

What’s in a name?

By Eric Hale

on July 30, 2015

A couple of weeks ago, the Notebookcheck German site published a review of the Huawei P8lite. We were pleased to see they used WebXPRT 2015, and the P8 Lite got an overall score of 47. This week, AnandTech published their review of the Huawei P8lite. In their review, the P8lite got an overall score of 59!

Those scores are very different, but it was not difficult to figure out why. The P8lite comes in two versions, depending on your market. The version Notebookcheck used is based on HiSilicon’s Kirin 620, while the version AnandTech used was Qualcomm’s Snapdragon 615 SoC. It’s also worth noting that the phone Notebookcheck tested was running Android 5.0, while the phone AnandTech tested was running Android 4.4. With different hardware and different operating systems, it’s no surprise that the results were different.

One consequence of the XPRTs being used across the world is that is that it is not uncommon to see results from devices in different markets. As we’ve said before, many things can influence benchmark results, so don’t assume that two devices with the same name are identical.

Kudos to both AnandTech and Notebookcheck for their care in presenting the system information for the devices in their reviews. The AnandTech review even included a brief description of the two models of the P8lite. This type of information is essential for helping people make informed decisions.

In other news, Windows 10 launched yesterday. We’re looking forward to seeing the TouchXPRT and WebXPRT results!

Eric

Posted in Benchmark metrics, Benchmarking computing devices, WebXPRT 2015, Windows 10 |

An update on MobileXPRT 2015

By Eric Hale

on July 23, 2015

As I mentioned last week, we’ve been testing MobileXPRT 2015.

We’ve rebuilt the current MobileXPRT as a 64-bit application. This means that MobileXPRT 2015 will only run on Android 5.0 and above. For this reason, we’ll make MobileXPRT 2013 available for testing older versions of the operating system. Because the workloads haven’t changed, you’ll be able to compare results from MobileXPRT 2013 to those from MobileXPRT 2015.

Another change to MobileXPRT 2015 will be the removal of the UX tests. While these tests were useful on lower-end devices when MobileXPRT 2013 came out, they don’t distinguish modern devices. However, should you need them, these tests will continue to be available as part of MobileXPRT 2013.

There are a few other features that we’ll be talking about over the next few weeks. We’ll be releasing the MobileXPRT 2015 community preview before Android M is released. We’re looking forward to see how Android M performs!

Eric

Posted in Android, Android L, Android M, Collaborative benchmark development, Community Preview, MobileXPRT 2015 |

Mystery solved

By Eric Hale

on July 2, 2015

As we mentioned a few weeks ago, the WebXPRT Local Notes test would not complete on recent builds of Windows 10 when using the Edge browser. Other browsers complete WebXPRT in recent Windows 10 builds without any problems.

We now know what is causing this behavior. The Local Notes test stores encrypted content in LocalStorage as UTF-16 character encoded Unicode strings. The encrypted content included values that are not considered valid characters in certain use cases. The current Edge implementation treats these characters as undefined and cannot store them. Other browsers may not have had an issue with the characters because of differences in the way they implement LocalStorage.

We’ve been able to work around this by using escape sequences for unsupported Unicode code points. Testing so far has not shown any perceptible change in results, so we believe that we will able to make this change to WebXPRT without compromising the comparability of the results.

Because this issue affects both WebXPRT 2013 and WebXPRT 2015, we’re planning to update both versions. We’ll let you know as soon as they are available.

If you’d like more details about this issue and the fix, please let us know.

Eric

Posted in Performance benchmarking, WebXPRT, WebXPRT 2015, What makes a good benchmark?, Windows 10 |

Category: Benchmarking

MobileXPRT 2015 Community Preview is here!

It’s almost here!

Explaining the BenchmarkXPRT Development Community

What’s in a name?

An update on MobileXPRT 2015

Mystery solved

Check out the other XPRTs: