Author Archives: Justin Greene

Introducing the WebXPRT 2015 Processor Comparison Chart

on October 5, 2017

Today, we’re excited to announce the WebXPRT 2015 Processor Comparison Chart, a new tool that makes it easier to access hundreds of PT-curated, real-world performance scores from a wide range of devices covering everything from TVs to phones to tablets to PCs.

The chart offers a quick way to browse and compare WebXPRT 2015 results grouped by processor. Unlike benchmark-score charts that may contain results from unknown sources, PT hand-selected each of the results from internal lab testing and reliable tech media sources. If we published multiple scores for an individual processor, the score presented in the chart will be an average of those scores. Users can hover over and click individual score bars for additional information about the test results and test sources for each processor.

We think the WebXPRT Processor Comparison Chart will be a valuable resource for folks interested in performance testing and product evaluation, but the current iteration is only the beginning. We plan to add additional capabilities on a regular basis, such a detailed filtering and enhanced viewing and navigational options. It’s also possible that we may integrate other XPRT benchmarks down the road.

Most importantly, we want the chart to be a great asset for its users, and that’s where you come in. We’d love to hear your feedback on the features and types of data you’d like to see. If you have suggestions, please let us know!

Justin

Posted in Benchmark metrics, Browser-based benchmarks, Cross-platform benchmarks, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2015 |

A new HDXPRT 2014 build is available

By Justin Greene

on September 21, 2017

Today, we’re releasing a new build of HDXPRT 2014 (v1.3) that eliminates the need for workarounds when testing with the Windows 10 Creators Update. The new build is available for download at HDXPRT.com. Please note that the app package is 5.09 GB, so allow time and space for the download process.

This build contains the following changes:

– Updates the Microsoft Visual C++ Redistributable (x86) to version 14.10.25017
– Updates the version of CPU-Z to v1.8.01
– Deletes Bin\power_settings.exe, which was prompting errors in certain configurations

The changes do not affect results, so v1.3 scores are comparable to v1.2 scores.

The new build ran well during testing in our labs, but issues could emerge as Microsoft releases new Windows updates this fall. If you have any questions about HDXPRT or encounter any issues during testing, we encourage you to let us know.

We look forward to seeing your test results!

Justin

Posted in HDXPRT, HDXPRT 2014, HDXPRT development process, Windows 10, Windows 10 Creators Update |

Decisions, decisions

By Justin Greene

on September 14, 2017

Back in April, we shared some of our initial ideas for a new version of WebXPRT, and work on the new benchmark is underway. Any time we begin the process of updating one of the XPRT benchmarks, one of the first decisions we face is how to improve workload content so it better reflects the types of technology average consumers use every day. Since benchmarks typically have a life cycle of two to four years, we want the benchmark to be relevant for at least the next couple of years.

For example, WebXPRT contains two photo-related workloads, Photo Effects and Organize Album. Photo Effects applies a series of effects to a set of photos, and Organize Album uses facial recognition technology to analyze a set of photos. In both cases, we want to use photos that represent the most relevant combination of image size, resolution, and data footprint possible. Ideally, the resulting image sizes and resolutions should differentiate processing speed on the latest systems, but not at the expense of being able to run reasonably on most current devices. We also have to confirm that the photos aren’t so large as to impact page load times unnecessarily.

The way this strategy works in practice is that we spend time researching hardware and operating system market share. Given that phones are the cameras that most people use, we look at them to help define photo characteristics. In 2017, the most widespread mobile OS is Android, and while reports vary depending on the metric used, the Samsung Galaxy S5 and Galaxy S7 are at or near the top of global mobile market share. For our purposes, the data tells us that choosing photo sizes and resolutions that mirror those of the Galaxy line is a good start, and a good chunk of Android users are either already using S7-generation technology, or will be shifting to new phones with that technology in the coming year. So, for the next version of WebXPRT, we’ll likely use photos that represent the real-life environment of an S7 user.

I hope that provides a brief glimpse into the strategies we use to evaluate workload content in the XPRT benchmarks. Of course, since the BenchmarkXPRT Development Community is an open development community, we’d love to hear your comments or suggestions!

Justin

Posted in Android, Benchmark metrics, Benchmarking, BenchmarkXPRT development community, Collaborative benchmark development, Performance benchmarking, WebXPRT, WebXPRT 2017, What makes a good benchmark? |

Keeping up with the latest Android news

By Justin Greene

on September 7, 2017

Ars Technica recently published a deep-dive review of Android 8.0 (Oreo) that contains several interesting tidbits about what the author called “Android’s biggest re-architecture, ever.” After reading the details, it’s hard to argue with that assessment.

The article’s thorough analysis includes a list of the changes Oreo is bringing to the UI, notification settings, locations service settings, and more. In addition to the types of updates that we usually see, a few key points stand out.

Project Treble, a complete reworking of Android’s foundational structure intended to increase the speed and efficiency of update delivery
A serious commitment to eliminating silent background services, giving users more control over their phone’s resources, and potentially enabling significant gains in battery life
Increased machine learning/neural network integration for text selection and recognition
A potential neural network API that allows third-party plugins
Android Go, a scaled-down version of Android tuned for budget phones in developing markets

There’s too much information about each of the points to discuss here, but I encourage anyone interested in Android development to check out the article. Just be warned that when they say “thorough,” they mean it, so it’s not exactly a quick read.

Right now, Oreo is available on only the Google Pixel and Pixel XL phones, and will not likely be available to most users until sometime next year. Even though widespread adoption is a way off, the sheer scale of the expected changes requires us to adopt a long-term development perspective.

We’ll continue to track developments in the Android world and keep the community informed about any impact that those changes may have on MobileXPRT and BatteryXPRT. If you have any questions or suggestions for future XPRT/Android applications, let us know!

Justin

Posted in Android, Android O, Battery life, Google, Machine learning, Mobile devices, MobileXPRT, Open Source, Phones | Tagged Android, BatteryXPRT, MobileXPRT, Oreo |

WebXPRT and user-agent strings

By Justin Greene

on August 31, 2017

After running WebXPRT in Microsoft Edge, a tester recently asked why the browser information field on the results page displayed “Chrome 52 – Edge 15.15063.” It’s a good question; why would the benchmark report Chrome 52 when Microsoft Edge is the browser under test? The answer lies in understanding user-agent strings and the way that WebXPRT gathers specific bits of information.

When browsers request a web page from a hosting server, they send an array of basic header information that allows the server to determine the client’s capabilities and the best way to provide the requested content. One of these headers, the user-agent, presents a string of tokens that provide information about the application making the request, the operating system and version, rendering engine compatibility, and browser platform details. In effect, the user-agent string is a way for a browser to tell the hosting server the full extent of its capabilities.

When WebXPRT attempts to identify a browser, it references the browser token in the user-agent string.

The process is generally straightforward, but in some cases, browsers spoof information from other browsers in their user-agent strings, which makes accurate browser detection difficult. The reasons for this are complex, but they involve web development practices and the fact that some web pages are not designed to recognize and work well with new or less-popular browsers. When we released WebXPRT 2015, Microsoft Edge was new. The Edge team wanted to make sure that as much advanced web content as possible would be available to Edge users, so they created a user-agent string that declared itself to be several different browsers at once.

I can see this in action if I check Edge’s user-agent string on my system. Currently, it reports “Mozilla/5.0 (Windows NT 10.0; Win64; x64; ServiceUI 9) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063.” So, because of the way Edge’s user-agent string is constructed, and the way WebXPRT parses that information, the browser information field on WebXPRT’s results page will report “Chrome 52 – Edge 15.15063” on my system.

To try this on your system, in Edge, select the ellipses icon in the top right-hand corner, then F12 Developer Tools. Next, select the Console tab, and run “javascript:alert(navigator.userAgent).” A popup window will display the UA string.

You can find instructions for finding the user-agent string in other browsers here: http://techdows.com/2016/07/edge-ie-chrome-firefox-user-agent-strings.html.

In the next version of WebXPRT, we’ll work to refine the way that the test parses user-agent strings, and provide more accurate system information for testers. If you have any questions or suggestions regarding this topic, let us know!

Justin

Posted in Browser-based benchmarks, Google Chrome, Microsoft Edge, WebXPRT, WebXPRT 2015 | Tagged browsers, Chrome, Microsoft Edge, WebXPRT |

Introducing the XPRT Selector

By Justin Greene

on August 24, 2017

We’re proud of all the XPRT tools, each of which serves a different purpose for the people who rely on them. But for those new to the XPRTs, we wanted a way to make it easy to tell which tool will best suit each person’s specific requirements. To that end, today we’re excited to announce the XPRT Selector, an interactive web tool that helps consumers, developers, manufacturers, and reviewers zero in on exactly which XPRT tool is the right match for their needs.

Using the XPRT Selector is easy. Simply spin the dials on the wheel to choose the categories that best describe yourself, the devices and operating systems you’re working with, and the topic that interests you. Once you’ve aligned the dials, click Get results, and the Selector will present all the free XPRT tools and resources that are available to you. Along with choosing the best tools for you, the XPRT Selector also explains the purpose and capabilities of each tool.

To see the Selector in action, check out the short video below. You can take the XPRT Selector for a spin at http://www.principledtechnologies.com/benchmarkxprt/the-xprt-selector/.

All the XPRT tools have one thing in common: They help take the guesswork out of device evaluation and comparison, making them invaluable for anyone using, making, or writing about tech products. We think the XPRT Selector is a great addition to the fold!

Justin

Posted in Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Interactive tools, The XPRT Selector |

Author Archives: Justin Greene

Introducing the WebXPRT 2015 Processor Comparison Chart

A new HDXPRT 2014 build is available

Decisions, decisions

Keeping up with the latest Android news

WebXPRT and user-agent strings

Introducing the XPRT Selector

Check out the other XPRTs: