Author Archives: Justin Greene

XPRT Spotlight: making it happen

on September 22, 2016

This week’s XPRT Weekly Tech Spotlight features the Apple iPhone 7, one of the bigger launches in a year of relatively few big phone releases. We like to feature a wide array of devices in Spotlight, but events like the iPhone 7 launch are good opportunities for us to provide quick data for buyers who are considering taking the plunge.

As those of you who need to have the new hot device on the first day know, getting that device can be a trial. Even though we preordered our iPhone 7, last Friday I found myself standing in line at the Apple store for almost two hours. However, that was much shorter than the half-day-plus wait for those who hadn’t ordered ahead.

We also ordered an iPhone 7 Plus. We’ll feature it in Spotlight as soon as it arrives, but we don’t expect it to ship until October.

Have you waited in line for a popular device this year? We’d love to hear your story. As always, if there are any devices that you’d like to see in Spotlight, let us know!

Justin

An anniversary update

By Justin Greene

on July 21, 2016

The Windows 10 Anniversary Update release is scheduled for August 2, and we’ve been running the XPRTs on the Windows Insider preview builds. While we can’t publish performance data from developer builds, we’re happy to say that WebXPRT and TouchXPRT run well on the Anniversary Update.

The story for HDXPRT 2014 is more complicated. Back in May, we reported that it would not run on more recent versions of Windows. However, we’ve identified steps that enable HDXPRT to run on the current stable Windows 10 build, as well as the latest Anniversary Update preview. It’s running well, but it’s possible that testers will encounter other issues as Microsoft releases new builds.

We have included the steps below. We’re considering an update to HDXPRT 2014 that will incorporate these changes. If you have any comments or suggestions related to HDXPRT, please let us know.

Justin

Summary
In addition to the normal system configuration requirements for HDXPRT, testers must also overwrite HDXPRT’s CPU-Z files with newer versions and change the default browser from Microsoft Edge to Internet Explorer. After configuring the system for HDXPRT testing, testers may encounter errors related to administrative privileges when attempting to launch Microsoft Edge. Returning User Account Control settings to their default pre-configuration state resolves the problem.

Process
1. Install the latest version of CPU-Z.
      a. Open any browser and download the latest version of CPU-Z for Windows
          (currently CPU-Z 1.76).
      b. Install CPU-Z on the system, using the default settings and installation path.
2. Install the HDXPRT 2014 benchmark using the default installation process. Reboot the system
    after installation.
3. Copy all the files from the C:\Program Files\CPUID\CPU-Z\ directory to the C:\Program Files
    (x86)\HDXPRT\bin, and overwrite the existing CPU-Z files.
4. Change the default browser from Microsoft Edge to Internet Explorer:
      a. Open the Windows Settings app and select System/Default apps.
      b. Under Web browser, click the Edge icon, and select Internet Explorer from the list.
      c. At the Before you switch window, click Switch anyway.
      d. Close the Settings app.
5. Adjust SmartScreen and security settings:
      a. Open Internet Explorer.
      b. Go to Settings/Internet options/Security, and make the following changes for the Internet
           and Trusted Sites zones:
            i. Select Custom Level.
            ii. Disable SmartScreen Filter.
            iii. Under Launching applications and unsafe files, click Enable (not Secure).
            iv. Click OK, and click Apply. If a warning message appears, click Yes.
6. Restart the system.
7. Open HDXPRT and run the benchmark normally.

If, after installing HDXPRT, you encounter an error related to administrative permissions when trying to open Microsoft Edge, return User Account Controls to the default setting, and restart the system. The default User Account Control setting is the third notch from the bottom: “Notify me only when apps try to makes changes to my computer.”

Sleep studies

By Justin Greene

on May 12, 2016

Last week, we discussed the increasing complexity of power options in Android 6.0. Features such as Doze and App Standby have changed the way that the operating system manages app activity, and the wide array of UI skins used by many vendors ensures that the steps needed for pre-test configuration differ considerably from device to device.

Managing Android’s Doze feature is critical to getting a good BatteryXPRT score. To show how involved this process can be, we thought it might be helpful to present the steps for one device. Below my sig are the configuration steps we used for the Huawei Mate 8, which we recently featured in the XPRT Weekly Tech Spotlight. For other phones we’ve tested, the steps have been quite different. We’re working on distilling our experience for our tips and tricks document, and the updated version of the document will be available soon. If you have any useful tips, please let us know.

Justin

Whitelist BatteryXPRT (there are two ways to do this)
1) Access Battery manager from Settings/Advanced settings or from the Phone Manager app on the home screen.
2) Select Protected apps.
3) Use the toggles beside BatteryXPRT and BatteryXPRT Tests to allow them to keep running after the screen turns off.

Configure sleep settings
1) Open Settings from the home screen.
2) Select Display.
3) Select Sleep.
4) Select Never. This may reset to a default setting on its own. In our case, it reset to 10 minutes.

Configure screen lock settings
1) Open Settings from the home screen.
2) Select Advanced settings.
3) Select Security.
4) Scroll to the bottom of the list and use the toggle to turn off Screen lock. This keeps the device screen from locking after standby periods during the test.

Posted in Android M, Android N, Battery life, BatteryXPRT 2014 for Android, Benchmarking, Mobile devices, Power consumption |

Focusing the spotlight

By Justin Greene

on March 24, 2016

As you may have heard, the Samsung Galaxy S7 is the XPRT Weekly Tech Spotlight this week. As we were testing it, we noticed that our WebXPRT scores were about 8 percent lower than those reported by AnandTech.

The folks at AnandTech do a good job on their reviews, so we wanted to understand the discrepancy in scores. The S7 comes in a couple of models, so we started by verifying that our model was the same as theirs. It was.

The next step was to check their configuration against ours, and this is where we found the difference. Both phones were running the same version of Android, but the S7 AnandTech tested used Chrome 48 while the S7 we tested came preloaded with Chrome 49. In our testing, we’ve noticed that upgrading from Chrome 48 to Chrome 49 has a noticeable performance impact on certain devices. On the Samsung Galaxy S6, the scores went down about 10 percent. In all cases we’ve seen, the decrease is driven largely by the Stock Option Pricing workload.

This isn’t the first time we’ve written about browser versions affecting results. WebXPRT is a browsing benchmark, and the browser has a legitimate impact on performance. When you’re comparing results, it’s always important to look at all the factors involved.

Justin

Last week in the XPRTs

We published the XPRT Weekly Tech Spotlight on the Samsung Galaxy S7.
We added two new BatteryXPRT ’14 results.
We added one new MobileXPRT ’15 result.
We added four new WebXPRT ’15 results.

Posted in AnandTech, Android, Android M, Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Google Chrome, WebXPRT, WebXPRT 2015 |

Detective work

By Justin Greene

on January 21, 2016

A few weeks ago, we described an issue with TouchXPRT’s Create Slideshow test. On some systems, this test would take longer than expected to complete due to the output video not rendering correctly. Since then, we released TouchXPRT 2016 CP2, which includes a check that verifies the output and reports an error if there’s a problem.

It took a lot of detective work, but I’m happy to say that we’ve been able to resolve this issue. We had to make a couple of changes. First, we changed the pixel format in which the photo content is read from GUID_WICPixelFormat32bppPBGRA to GUID_WICPixelFormat32bppBGR. Second, we changed the media format of the input into the video converter from MFVideoFormat_ARGB32 to MFVideoFormat_RGB32.

After making these changes, the Create Slideshow test ran in the expected amount of time, and rendered the output video correctly.

In our testing so far, results have not changed noticeably. However, we’re continuing to test. We want to be sure that the updated build is stable and that results are comparable to earlier TouchXPRT CPs. We’ll then release it to the community.

Here’s looking forward to the release of TouchXPRT 2016 to the world at large!

Posted in BenchmarkXPRT development community, TouchXPRT, TouchXPRT 2016, TouchXPRT development process, TouchXPRT release cycle, TouchXPRT results, Windows 10 |

The XPRTs at IDF15 Shenzhen

By Justin Greene

on April 23, 2015

We recently traveled to Shenzhen, China to anchor Principled Technologies’ booth in the IDF15 Shenzhen technology showcase. Over 60 companies set up displays to interact with approximately 2,000 attendees, and it was a great opportunity for us to connect with innovators in one of the fastest-growing technology hubs in the world.

We spent most of our time talking with people about the benefits of the BenchmarkXPRT Development Community, demonstrating the XPRTs on our display systems, and describing the scope and abilities of each XPRT. Many of the people we talked with showed great interest in the XPRTs’ commitment to developing easy-to-use benchmarks that measure performance and battery life while doing everyday tasks.

Of course, we also spent a good deal of free time exploring Shenzhen and nearby Hong Kong. If you ever get the opportunity to visit either of these dynamic cities, you won’t be disappointed!

The IDF15 Shenzhen technology showcase floor. Our booth is just to the right of the gray-and-white balloon. The booming Shenzhen skyline, including the soon-to-be-second-tallest building in the world.