Category: WebXPRT 2015

An anniversary update

on July 21, 2016

The Windows 10 Anniversary Update release is scheduled for August 2, and we’ve been running the XPRTs on the Windows Insider preview builds. While we can’t publish performance data from developer builds, we’re happy to say that WebXPRT and TouchXPRT run well on the Anniversary Update.

The story for HDXPRT 2014 is more complicated. Back in May, we reported that it would not run on more recent versions of Windows. However, we’ve identified steps that enable HDXPRT to run on the current stable Windows 10 build, as well as the latest Anniversary Update preview. It’s running well, but it’s possible that testers will encounter other issues as Microsoft releases new builds.

We have included the steps below. We’re considering an update to HDXPRT 2014 that will incorporate these changes. If you have any comments or suggestions related to HDXPRT, please let us know.

Justin

Summary
In addition to the normal system configuration requirements for HDXPRT, testers must also overwrite HDXPRT’s CPU-Z files with newer versions and change the default browser from Microsoft Edge to Internet Explorer. After configuring the system for HDXPRT testing, testers may encounter errors related to administrative privileges when attempting to launch Microsoft Edge. Returning User Account Control settings to their default pre-configuration state resolves the problem.

Process
1. Install the latest version of CPU-Z.
      a. Open any browser and download the latest version of CPU-Z for Windows
          (currently CPU-Z 1.76).
      b. Install CPU-Z on the system, using the default settings and installation path.
2. Install the HDXPRT 2014 benchmark using the default installation process. Reboot the system
    after installation.
3. Copy all the files from the C:\Program Files\CPUID\CPU-Z\ directory to the C:\Program Files
    (x86)\HDXPRT\bin, and overwrite the existing CPU-Z files.
4. Change the default browser from Microsoft Edge to Internet Explorer:
      a. Open the Windows Settings app and select System/Default apps.
      b. Under Web browser, click the Edge icon, and select Internet Explorer from the list.
      c. At the Before you switch window, click Switch anyway.
      d. Close the Settings app.
5. Adjust SmartScreen and security settings:
      a. Open Internet Explorer.
      b. Go to Settings/Internet options/Security, and make the following changes for the Internet
           and Trusted Sites zones:
            i. Select Custom Level.
            ii. Disable SmartScreen Filter.
            iii. Under Launching applications and unsafe files, click Enable (not Secure).
            iv. Click OK, and click Apply. If a warning message appears, click Yes.
6. Restart the system.
7. Open HDXPRT and run the benchmark normally.

If, after installing HDXPRT, you encounter an error related to administrative permissions when trying to open Microsoft Edge, return User Account Controls to the default setting, and restart the system. The default User Account Control setting is the third notch from the bottom: “Notify me only when apps try to makes changes to my computer.”

Posted in BenchmarkXPRT development community, Collaborative benchmark development, HDXPRT, HDXPRT 2014, Performance benchmarking, TouchXPRT, Uncategorized, WebXPRT 2015, Windows 10, Windows 10 Anniversary Update |

Focusing the spotlight

By Justin Greene

on March 24, 2016

As you may have heard, the Samsung Galaxy S7 is the XPRT Weekly Tech Spotlight this week. As we were testing it, we noticed that our WebXPRT scores were about 8 percent lower than those reported by AnandTech.

The folks at AnandTech do a good job on their reviews, so we wanted to understand the discrepancy in scores. The S7 comes in a couple of models, so we started by verifying that our model was the same as theirs. It was.

The next step was to check their configuration against ours, and this is where we found the difference. Both phones were running the same version of Android, but the S7 AnandTech tested used Chrome 48 while the S7 we tested came preloaded with Chrome 49. In our testing, we’ve noticed that upgrading from Chrome 48 to Chrome 49 has a noticeable performance impact on certain devices. On the Samsung Galaxy S6, the scores went down about 10 percent. In all cases we’ve seen, the decrease is driven largely by the Stock Option Pricing workload.

This isn’t the first time we’ve written about browser versions affecting results. WebXPRT is a browsing benchmark, and the browser has a legitimate impact on performance. When you’re comparing results, it’s always important to look at all the factors involved.

Justin

Last week in the XPRTs

We published the XPRT Weekly Tech Spotlight on the Samsung Galaxy S7.
We added two new BatteryXPRT ’14 results.
We added one new MobileXPRT ’15 result.
We added four new WebXPRT ’15 results.

Posted in AnandTech, Android, Android M, Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Google Chrome, WebXPRT, WebXPRT 2015 |

The big weekend is almost here!

By Eric Hale

on March 10, 2016

A few weeks ago, I talked about the XPRT Women Code-a-Thon. Well, all the work that we and our friends at ChickTech Seattle have done is about to pay off! On Saturday, March 12, dozens of women will go to the Sole Repair Shop in Seattle for two days of coding, good food, and networking opportunities.

The goal of each team at the code-a-thon is to create a new workload that might be included in a future version of WebXPRT or MobileXPRT. Judges will award prizes to the top three workloads: $2,500 for first place, $1,500 for second place, and $1,000 for third place. I can’t wait to see the winning workloads!

We’re very fortunate to have Kristin Toth Smith as the keynote speaker. She is an avid supporter of women in tech, current COO of Dolly, and former CEO of Code Fellows.

It should be a great time for all. If you or anyone you know can get to Seattle this weekend, registration is still open.

Eric

Last week in the XPRTs
We published the XPRT Weekly Tech Spotlight on the ASUS ZenFone 2.
We added one new BatteryXPRT ’14 result.
We added five new WebXPRT ’15 results.

Posted in ChickTech Seattle, Collaborative benchmark development, MobileXPRT 2015, WebXPRT 2015, XPRT Women Code-a-Thon |

Auf Deutsche

By Eric Hale

on December 17, 2015

Early next week, we will update WebXPRT by adding a German UI. This brings the number of available languages to three. WebXPRT has had a Simplified Chinese UI for a while, but you had to click a link on the WebXPRT page to get it. The new version removes that limitation, and lets you select Simplified Chinese, English, or German from the UI.

We’re working on getting WebXPRT to automatically detect the language of your device, but for now, the UI defaults to English.

We would like to expand the range of languages the XPRTs support over time. This is an area where you can help. If you’d like to see your language represented and are willing to help with translation, please let us know.

I know it’s the holiday season, but remember that CES will be here before we know it. I’m really looking forward to seeing the show, and I may have some big news to talk about while I’m there! If you’re planning to be at CES, send a message and let’s find a time to meet!

We will not have a blog post next week. Happy holidays!

Eric

Last week in the XPRTs
We published the December 2015 BenchmarkXPRT Development Community newsletter.
We added one new BatteryXPRT ’14 result.
We added nine new MobileXPRT ’13 results.
We added one new MobileXPRT ’15 result.
We added four new WebXPRT ’15 results.

Posted in BenchmarkXPRT development community, CES, Collaborative benchmark development, German, Simplified Chinese, WebXPRT, WebXPRT 2015, What makes a good benchmark? |

Please let us know

By Eric Hale

on November 19, 2015

Todd Reifsteck from the Web Platform Team at Microsoft was kind enough to let me share a conversation we had last week:

Todd reported he was having problems running WebXPRT on the Edge browser. This was a surprise to us, as we’d already released a WebXPRT update to resolve Edge browser issues.

We were not seeing this problem, and as we talked with Todd we verified there was no issue in WebXPRT itself. The fix we released was working; however, we found a path through the web site that launched the previous version of WebXPRT. Once we fixed that URL to point to the latest version of WebXPRT, Todd reported that WebXPRT was working with Edge, just as we expected.

This problem would not have affected results on other browsers. The results from the previous version of WebXPRT are comparable to the current version. Compatibility with the Edge browser is the only difference between the versions.

Thanks to Todd for his help. As always, we encourage you to contact us if you have any issues or questions. We’ll do our best to resolve them as quickly as possible.

Eric

Posted in Benchmarking, BenchmarkXPRT development community, Collaborative benchmark development, Let us know your thoughts, Microsoft Edge, WebXPRT, WebXPRT 2015 |

More than the sum of its parts

By Eric Hale

on October 22, 2015

There was a recent article in Bloomberg about phone maker ZTE’s increasing market share in the US. The article singled out one phone, the ZTE Maven, which costs about $60 (US).

This phrase jumped out at me: “a processor with capabilities somewhere between the iPhone 5 and 6.” The iPhone 5S could also fit that description. The ZTE Maven uses the ARM Cortex-A53, 64-bit processor running at 1.2 GHz. The Apple iPhone 5s uses the Apple Cyclone-A7 Cortex-A7 Harvard Superscalar processor running at 1.3 GHz.

We decided to put that statement to the test. We ran WebXPRT 2015 on the ZTE Maven and its score was 47. The iPhone 5s scored 100. The Maven was not even close.

As we’ve said before, the performance of a device depends on more than the GHz of its processor. For example, the ZTE Maven uses the Snapdragon 410 SoC, which was aimed at mid-level devices. The iPhone 5s uses the Apple A7, which was intended for higher-end devices. You can find side by side specs here.

Be wary when you see unsupported performance claims. As this example shows, specs can appear comparable even when the actual performance of the devices differs considerably. A good benchmark can provide insights into performance that specs alone can’t.

Eric

Posted in Android, Benchmark metrics, Benchmarking, WebXPRT, WebXPRT 2015 |

Category: WebXPRT 2015

An anniversary update

Focusing the spotlight

The big weekend is almost here!

Auf Deutsche

Please let us know

More than the sum of its parts

Check out the other XPRTs: