Author Archives: Eric Hale

A Chrome-plated example

on September 1, 2016

A couple of weeks ago, we talked about how benchmarks have to evolve to keep up with the changing ways people use their devices. One area where we are expecting a lot of change in the next few months is Chromebooks.

These web-based devices have become very popular, even outselling Macs for the first time in Q1 of this year. Chromebooks run Google Apps and a variety of third-party Chrome apps that also run on Windows, Mac, and Linux systems.

Back in May, Google announced that Android apps would be coming to Chromebooks. This exciting development will bring a lot more applications to the platform. Now, Google has announced that they will be “moving away” from the Chrome apps platform and will be phasing out Chrome app support on other platforms within the next two years.

Clearly, the uses of Chromebooks are likely to change a lot in coming months. Interestingly, part of the rationale Google gives for this decision is the development of powerful new Web APIs, which will have implications for WebXPRT as well.

As we’ve said before, we’ll be watching and adapting as the applications change.

Eric

Posted in Application-based benchmarks, Benchmarking, Chrome OS, Future of performance evaluation, Google, WebXPRT |

Apples to apples?

By Eric Hale

on August 25, 2016

PCMag published a great review of the Opera browser this week. In addition to looking at the many features Opera offers, the review included performance data from multiple benchmarks, which look at areas such as hardware graphics acceleration, WebGL performance, memory consumption, and battery life.

Three of the benchmarks have a significant, though not exclusive, focus on JavaScript performance: Google Octane 2.0, JetStream 1.1, and WebXPRT 2015. The three benchmarks did not rank the browsers the same way, and in the past, we‘ve discussed some of the reasons why this happens. In addition to the difference in tests, there are also sometimes differences in approaches that are worth considering.

For example, consider the test descriptions for JetStream 1.1. You’ll immediately notice that the tests are much lower-level tests than the ones in WebXPRT. However, consider these phrases from a few of the test descriptions:

code-first-load “…This test attempts to defeat the browser’s caching capabilities…”
splay-latency “Tests the worst-case performance…”
zlib “…modified to restrict code caching opportunities…”

While the XPRTs test typical performance for higher level applications, the tests in JetStream are tweaked to stress devices in very specific ways, some of which are not typical. The information these tests provide can be very useful for engineers and developers, but may not be as meaningful to the typical user.

I have to stress that both approaches are valid, but they are doing somewhat different things. There’s a cliché about comparing apples to apples, but not all apples are the same. If you’re making a pie, a Granny Smith would be a good choice, but for snacking, you might be better off with a Red Delicious. Knowing a benchmark’s purpose will help you find the results that are most meaningful to you.

Eric

Posted in Benchmark metrics, Browser-based benchmarks, JavaScript, Performance benchmarking, WebGL, WebXPRT, What makes a good benchmark? |

The things we do now

By Eric Hale

on August 18, 2016

We mentioned a couple of weeks ago that the Microsoft Store added an option to indicate holographic support, which we selected for TouchXPRT. So, it was no surprise to see Microsoft announce that next year, they will release an update to Windows 10 that enables mainstream PCs to run the Windows Holographic shell. They also announced that they‘re working with Intel to develop a reference architecture for mixed-reality-ready PCs. Mixed-reality applications, which combine the real world with a virtual reality, demand sophisticated computer vision, and applications that can learn about the world around them.

As we’ve said before, we are constantly watching how people use their devices. One of the most basic principles of the XPRT benchmarks is to test devices using the same kinds of work that people do in the real world. As people find new ways to use their devices, the workloads in the benchmarks should evolve as well. Virtual reality, computer vision, and machine learning are among the technologies we are looking at.

What sorts of things are you doing today that you weren’t a year ago? (Other than Pokémon GO – we know about that one.) Would you like to see those sorts of workloads in the XPRTs? Let us know!

Eric

Posted in Benchmarking, computer vision, Future of performance evaluation, Holograph, Machine learning, TouchXPRT, TouchXPRT 2016, Virtual reality |

New device shopping tool helps consumers make smarter buying choices

By Eric Hale

on August 16, 2016

Durham, NC—The Benchmark XPRT Development Community has launched a shopping tool that includes side-by-side comparisons of today’s leading devices. The tool is part of the XPRT Weekly Tech Spotlight’s Back to School Roundup, which highlights leading devices and helps families and students choose the right tools for their educational needs. The tool lets buyers compare all‑in‑one PCs, Chromebooks, convertibles, laptops, and tablets running Chrome OS, OS X, and Windows.

“Our new tool lets buyers make comparisons using results from the XPRT benchmarks,” said Bill Catchings, co-founder of Principled Technologies, Inc., which administers the BenchmarkXPRT Development Community. “Heading back to school often means tech purchases, so we launched the tool with our Back to School Roundup.”

Principled Technologies tests the devices in the Spotlight using the industry standard BenchmarkXPRT tools: WebXPRT, MobileXPRT, TouchXPRT, CrXPRT, BatteryXPRT, and HDXPRT. In addition to the benchmark results, the side-by-side comparisons include photographs, device specs, and prices.

The XPRT Weekly Tech Spotlight debuted February 16, making it easier for consumers to shop for a new laptop, smartphone, tablet, or PC. Recent devices in the spotlight include the Intel Skull Canyon NUC, LG Chromebase, Acer Aspire Z3-715, Dell Inspiron 24 3459, HP Chromebook 14, and Apple MacBook Air 11. More devices come on line weekly; see the full list on the XPRT Weekly Tech Spotlight page.

Vendors interested in having their devices featured in the XPRT Weekly Tech Spotlight can visit the website for more details.

About Principled Technologies, Inc.
Principled Technologies, Inc. is a leading provider of technology marketing and learning & development services. It administers the BenchmarkXPRT Development Community.

Principled Technologies, Inc. is located in Durham, North Carolina, USA. For more information, please visit www.PrincipledTechnologies.com.

Company Contact
Eric Hale
Principled Technologies, Inc.
1007 Slater Road, Suite #300
Durham, NC 27703
XPRTSpotlight@PrincipledTechnologies.com

Posted in Press Releases |

Side by side

By Eric Hale

on August 11, 2016

We’re making a big upgrade to the Spotlight: It will now let you make side-by-side comparisons of any of the devices we’ve featured. You’ll also be able to filter devices by type.

We’ve also added pricing information to the Spotlight. The price of a device can vary a lot, depending on the options you select. We give you the price we paid for the device as we tested it. Because prices can vary over time, we also give the date when we bought the device.

We think these changes will make the Spotlight an even more powerful tool for helping with buying decisions.

We also created a special page, the Back to School Roundup. It provides side-by-side comparisons of 13 of the hottest devices to help families and students choose the right ones for their educational needs. We include a range of device types—all-in-ones, Chromebooks, convertibles, laptops, and tablets—and operating systems—Chrome OS, OS X, and Windows.

Going forward, we’ll be adding more advanced querying features. Of course, we’ll also add new devices every week.

Check it out! And, if you have any ideas about how to make the XPRT Weekly Tech Spotlight more useful, let us know!

Eric

Posted in BenchmarkXPRT, Chrome OS, Chromebooks, Cross-platform benchmarks, Education, OS X, Windows 10, XPRT Weekly Tech Spotlight |

Smarter shopping for Windows 10 devices

By Eric Hale

on August 4, 2016

Microsoft released the Windows 10 Anniversary Update for PCs and tablets on Tuesday. (The Anniversary Update for Windows 10 Mobile is rolling out on August 9.) Justin explained a couple of weeks ago how to run HDXPRT on current builds of Windows 10, and we have verified that those instructions work for the released version of the Anniversary Update.

We’ve also made sure that TouchXPRT runs with the Anniversary Update. When we uploaded the latest TouchXPRT version to the Microsoft Store, we checked the box to say it supports holographic devices. We expect it will work, but we don’t have a HoloLens available for testing. We would love to hear from anyone who has the developer version of the HoloLens about any issues and any tips for resolving them.

If you’re considering buying a Windows 10 tablet or phone, you should be using TouchXPRT to inform your decision. TouchXPRT 2016 is a Universal Windows application, compatible with Windows 10 and Windows 10 Mobile. Like all the XPRTs, it produces a simple “bigger is better” score. You can find TouchXPRT scores online and in the XPRT Weekly Tech Spotlight.

Of course, you can also download and run TouchXPRT yourself. It’s available in the Windows Store or from TouchXPRT.com. Knowing the TouchXPRT score for your device is a great way to set a baseline for your next purchase!

Eric

Posted in HDXPRT, Holograph, TouchXPRT, TouchXPRT 2016, Windows 10, Windows 10 Anniversary Update, XPRT Weekly Tech Spotlight |

Author Archives: Eric Hale

A Chrome-plated example

Apples to apples?

The things we do now

New device shopping tool helps consumers make smarter buying choices

Side by side

Smarter shopping for Windows 10 devices

Check out the other XPRTs: