Category: Benchmarking

Please let us know

on November 19, 2015

Todd Reifsteck from the Web Platform Team at Microsoft was kind enough to let me share a conversation we had last week:

Todd reported he was having problems running WebXPRT on the Edge browser. This was a surprise to us, as we’d already released a WebXPRT update to resolve Edge browser issues.

We were not seeing this problem, and as we talked with Todd we verified there was no issue in WebXPRT itself. The fix we released was working; however, we found a path through the web site that launched the previous version of WebXPRT. Once we fixed that URL to point to the latest version of WebXPRT, Todd reported that WebXPRT was working with Edge, just as we expected.

This problem would not have affected results on other browsers. The results from the previous version of WebXPRT are comparable to the current version. Compatibility with the Edge browser is the only difference between the versions.

Thanks to Todd for his help. As always, we encourage you to contact us if you have any issues or questions. We’ll do our best to resolve them as quickly as possible.

Eric

Posted in Benchmarking, BenchmarkXPRT development community, Collaborative benchmark development, Let us know your thoughts, Microsoft Edge, WebXPRT, WebXPRT 2015 |

The TouchXPRT 2016 Community Preview is here!

By Eric Hale

on November 13, 2015

Today we are releasing the TouchXPRT 2016 Community Preview (CP). TouchXPRT 2016 includes the same performance workloads as TouchXPRT 2014, but we have rebuilt it as a Universal Windows app. This makes TouchXPRT 2016 compatible with systems running Windows 10 and Windows 10 Mobile.

Because this is a community preview, it is available only to community members. Members may download the preview from the TouchXPRT tab in the Members’ Area.

The results viewer in the TouchXPRT 2016 CP is not fully functional. Please consult the release notes for further details (login required).

After you try out the CP, please send your comments. Either post them to the forum or mail them to BenchmarkXPRTsupport@principledtechnologies.com. If you send us information that’s relevant to the entire community, we may post an anonymous version of your comments to the forum.

Thanks for your participation!

Posted in BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, TouchXPRT, TouchXPRT 2016, TouchXPRT development process, TouchXPRT release cycle, Windows 10 |

The TouchXPRT 2016 CP arrives tomorrow!

By Eric Hale

on November 12, 2015

As we said a couple of weeks ago, we wanted to test on Windows 10 Threshold 2 before releasing the TouchXPRT 2016 community preview. Well, Threshold 2 is out and the testing has been going very well.

We’ll release the TouchXPRT 2016 to the community tomorrow. Because community previews are not available to the general public, members will need to download it from our site.

The installation procedure is fairly straightforward. First, you put the device in developer mode. Then, for a tablet or PC, run a PowerShell script, as you did for TouchXPRT 2014. For a mobile device, once it’s in developer mode, copy the bundle to the device and install it.

If you have any problems, please let us know.

We’re looking forward to seeing the results from phones and, as we have done with MobileXPRT, to comparing results across different-sized devices.

Enjoy!

Eric

Posted in Benchmark metrics, Benchmarking, Collaborative benchmark development, Community Preview, TouchXPRT, TouchXPRT 2016, TouchXPRT development process, Windows 10 |

Things are looking good!

By Eric Hale

on October 29, 2015

It’s been a couple of weeks since we talked about TouchXPRT 2016. I’m happy to say that we have made great progress since then.

The UI is adapting to the different form factors very well. We’ve tested the resizing on phones and on tablets with the screen snapped to half and quarter screen. While a couple of UI elements still need work, we’ve had no problems running the tests. Here’s how TouchXPRT looks on a phone:

We have also greatly simplified installing the application on phones, so we’re no longer concerned about that.

All in all, things are looking good for releasing the community preview soon. However, we’re going to wait until we can test on the Windows 10 Fall Update (Threshold Build 2). That’s supposed to be released on November 2. We’re not expecting any problems with the Fall Update, but it’s always to wise to check.

November can’t come quickly enough!

Eric

Posted in Benchmarking, BenchmarkXPRT, Collaborative benchmark development, Community Preview, TouchXPRT, TouchXPRT 2016, TouchXPRT development process, TouchXPRT release cycle |

More than the sum of its parts

By Eric Hale

on October 22, 2015

There was a recent article in Bloomberg about phone maker ZTE’s increasing market share in the US. The article singled out one phone, the ZTE Maven, which costs about $60 (US).

This phrase jumped out at me: “a processor with capabilities somewhere between the iPhone 5 and 6.” The iPhone 5S could also fit that description. The ZTE Maven uses the ARM Cortex-A53, 64-bit processor running at 1.2 GHz. The Apple iPhone 5s uses the Apple Cyclone-A7 Cortex-A7 Harvard Superscalar processor running at 1.3 GHz.

We decided to put that statement to the test. We ran WebXPRT 2015 on the ZTE Maven and its score was 47. The iPhone 5s scored 100. The Maven was not even close.

As we’ve said before, the performance of a device depends on more than the GHz of its processor. For example, the ZTE Maven uses the Snapdragon 410 SoC, which was aimed at mid-level devices. The iPhone 5s uses the Apple A7, which was intended for higher-end devices. You can find side by side specs here.

Be wary when you see unsupported performance claims. As this example shows, specs can appear comparable even when the actual performance of the devices differs considerably. A good benchmark can provide insights into performance that specs alone can’t.

Eric

Posted in Android, Benchmark metrics, Benchmarking, WebXPRT, WebXPRT 2015 |

We haven’t mentioned this in a while

By Eric Hale

on October 15, 2015

I had a conversation with a community member yesterday who wanted to know whether we would test his device with one of the XPRTs. The short answer is “Absolutely!” The somewhat longer answer follows.

If you send us a device you want us to test, we will do so, with the appropriate set of XPRTs, free of charge. You will know that an impartial, third-party lab has tested your device using the best benchmarking practices. After we share the results with you, you will have three options: (1) to keep the results private, (2) to have us make the results public immediately in the appropriate XPRT results databases, or (3) to delay releasing the results until a future date. Regardless of your choice, we will keep the device so that we can use it as part of our testbed for developing and testing future versions of the XPRTs.

When we add the results to our online databases, we will cite Principled Technologies as the source, indicating that we stand behind the results.

The free testing includes no collateral beyond publishing the results. If you would like to publicize them through a report, an infographic, or any of the other materials PT can provide, just let us know and the appropriate person will contact you to discuss the how much those services would cost.

If you’re interested in getting your device tested for free, contact us at BenchmarkXPRTSupport@principledtechnologies.com.

Eric

Posted in Benchmarking, Benchmarking computing devices, BenchmarkXPRT, BenchmarkXPRT development community, Principled Technologies |

Category: Benchmarking

Please let us know

The TouchXPRT 2016 Community Preview is here!

The TouchXPRT 2016 CP arrives tomorrow!

Things are looking good!

More than the sum of its parts

We haven’t mentioned this in a while

Check out the other XPRTs: