Category: Google Chrome

Planning for the next CrXPRT

on December 5, 2019

We’re currently planning the next version of CrXPRT, our benchmark that evaluates the performance and battery life of Chromebooks. If you’re unfamiliar with CrXPRT, you can find out more about how it works both here in the blog and at CrXPRT.com. If you’ve used CrXPRT, we’d love to hear any suggestions you may have. What do you like or dislike about CrXPRT? What features do you hope to see in a new version?

When we begin work on a new version of any benchmark, one of our first steps is to determine whether the workloads will provide value during the years ahead. As technology and user behavior evolve, we update test content to be more relevant. One example is when we replace photos with ones that use more contemporary file resolutions and sizes.

Sometimes the changing tech landscape prompts us to remove entire workloads and add new ones. The Photo Collage workload in CrXPRT uses Portable Native Client (PNaCl) technology, for which the Chrome team will soon end support. CrXPRT 2015 has a workaround for this issue, but the best course of action for the next version of CrXPRT will be to remove this workload altogether.

The battery life test will also change. Earlier this year, we started to see unusual battery life estimates and high variance when running tests at CrXPRT’s default battery life test length of 3.5 hours, so we’ve been recommending that users perform full rundowns instead. In the next CrXPRT, the battery life test will require full rundowns.

We’ll also be revamping the CrXPRT UI to improve the look of the benchmark and make it easier to use, as we’ve done with the other recent XPRT releases.

We really do want to hear your ideas, and any feedback you send has a chance to shape the future of the benchmark. Let us know what you think!

Justin

Posted in Battery life, benchmark, Benchmarking, BenchmarkXPRT, Browser-based benchmarks, Chrome OS, Chromebooks, Collaborative benchmark development, CrXPRT, Google Chrome, Performance benchmarking | Tagged battery life, benchmark, Chrome, Chrome OS, Chromebooks, CrXPRT |

A CrXPRT fix for Chrome 76

By Justin Greene

on August 22, 2019

After Chrome OS version 76 moved from Chrome’s Beta channel to the Stable channel last week, we became aware of an issue that occurs when CrXPRT’s Photo Collage workload runs on a Chrome 76 system. We found that the Photo Collage workload produces an error message—“This plugin is not supported on this device”—and the test run does not complete.

The error occurs because the Photo Collage workload uses Portable Native Client (PNaCl), and starting with version 76, the Chrome team changed the way the OS handles PNaCl tasks. Technically, Chrome still supports PNaCl, but the OS now disables the capability by default. Chrome’s current plan is to end support for PNaCl by the end of this year, focusing related development efforts on WebAssembly instead.

We’ll investigate the best path forward during this transition, but for now, testers can use the following workaround that allows CrXPRT to complete successfully. Simply navigate to chrome://flags on the test system, and find the Native Client flag, which is set to “Disabled” by default. Click the toggle switch to “Enabled” to allow native client capabilities, restart the system, and kick off a CrXPRT test in the normal manner.

We’ll update the CrXPRT web page and test documentation to include information about the workaround. In the long term, we’re interested in any suggestions you have for CrXPRT—whether they’re related to PNaCl or not. Please let us know your thoughts!

Justin

Posted in Chrome OS, Chromebooks, CrXPRT, Google Chrome | Tagged Beta, Chrome, Chrome 76, Chrome OS, Chromebooks, Chromium, Native Client, photos, PNaCl |

A new playing field for WebXPRT

By Justin Greene

on April 18, 2019

WebXPRT is one of the go-to benchmarks for evaluating browser performance, so we’re always interested in browser development news. Recently, Microsoft created a development channel where anyone can download early versions of an all-new Microsoft Edge browser. Unlike previous versions of Edge, Microsoft constructed the new browser using the Chromium open-source project, the same foundation underlying the Google Chrome browser and Chrome OS.

One interesting aspect of the new Edge development strategy is the changes that Microsoft is making to more than 50 services that Chromium has included. If you use Chrome daily, you’ve likely become accustomed to certain built-in services such as ad block, spellcheck, translate, maps integration, and form fill, among many others. While each of these is useful, a large number of background services running simultaneously can slow browsing and sap battery life. In the new Edge, Microsoft is either reworking each service or removing it altogether, with the hope of winning users by providing a cleaner, faster, and more power-efficient experience. You can read more about Microsoft’s goals for the new project on the Microsoft Edge Insider site.

As we’ve discussed before, many factors contribute to the speed of a browsing experience and its WebXPRT score. It’s too early to know how the new Microsoft Edge will stack up against other browsers, but when the full version comes out of development, you can be sure that we’ll be publishing some comparison scores. I’ve installed the Dev Channel version of Edge on my personal machine and run WebXPRT 3. While I can’t publish the scores from this early version, I can tell you that the results were interesting. Have you run WebXPRT 3 on the new Microsoft Edge? How do you think it compares to competitors? We’d love to hear your thoughts.

Justin

Posted in Benchmarking, Browser-based benchmarks, Chrome OS, Cross-platform benchmarks, Google, Google Chrome, Microsoft Edge, Performance benchmarking, WebXPRT, WebXPRT 3 | Tagged browser, Chrome, Chrome OS, Chromium, Edge, Microsoft, Performance, WebXPRT, WebXPRT 3 |

CrXPRT is more valuable than ever

By Justin Greene

on March 28, 2019

Digital Trends recently published an article discussing various rumors about the future of the Google Pixelbook line. Pixelbooks were some of the first Chromebooks with high-end hardware specs, and they were priced accordingly. Whether or not the rumors discussed in the article turn out to be true, the author points out that the Pixelbook prompted several other vendors, such as HP and Lenovo, to take a chance on high-end Chromebooks. It seems like high-end Chromebooks are here to stay, but given the unique constraints of the Chrome OS environment, buyers are often unsure if it’s worth it to shell out the extra money for a premium model.

We developed CrXPRT to help buyers answer these questions. CrXPRT is a benchmark tool that measures the battery life of your Chromebook as well as how fast it handles everyday tasks like playing video games, watching movies, editing pictures, and doing homework. The performance test gives you individual workload scores and an overall score based on the speed of the device. The battery life test produces an estimated battery life time, a separate performance score, and a frames-per-second (FPS) rate for a built-in HTML5 gaming component.

You don’t have to be a tech journalist or even a techie to use CrXPRT. To learn more, check out the links below.

Testing the performance or battery life of your Chromebook

Simply download CrXPRT from the Chrome Web Store. Installation is quick and easy, and the CrXPRT 2015 user manual provides step-by-step instructions. A typical performance test takes about 15 minutes, and a battery life test will take 3.5 hours once the system is charged and configured for testing. If you’d like to see how your score compares to other Chromebooks, visit the CrXPRT results page.

Want to know more?

Read the Exploring CrXPRT 2015 white paper, where we discuss the concepts behind CrXPRT, its development process, and the app’s structure. We also describe the component tests and explain the statistical processes used to calculate expected battery life.

BenchmarkXPRT Development Community members also have access to the CrXPRT source code, so if you’re interested, join today! There’s no obligation and membership is free for members of any company or organization with an interest in benchmarks.

Give CrXPRT a try and let us know what you think!

Justin

Posted in Battery life, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Chrome OS, Chromebooks, Google, Google Chrome, What makes a good benchmark? | Tagged battery life, BenchmarkXPRT, Chrome OS, Chromebook, CrXPRT, laptops, Performance, Pixelbook |

CrXPRT in the Chrome Web Store

By Justin Greene

on September 13, 2018

Testers searching for CrXPRT in the Chrome Web Store may be puzzled to see “no results.” While CrXPRT no longer appears in Chrome Web Store searches, the app is still live in the store and there’s nothing wrong with it. CrXPRT is freely available and functioning normally, but you can access it only through a direct link. It’s an unusual situation that results from a strategic decision by the Chrome team, a decision affecting many apps in addition to CrXPRT.

For several years, Chrome supported two types of apps—hosted apps that wrap online websites, and packaged apps that have offline capability. CrXPRT is a packaged app built for Chrome OS. In December 2017, the Chrome team stopped providing search and browse functions for hosted and packaged apps in the Chrome Web Store. Instead, they’re encouraging folks to develop Progressive Web Apps. You can read about the reasoning behind this decision on the Chromium blog. Despite this shift in focus, the Chrome Web store will continue to support existing apps packaged for Chrome OS.

In short, the Chrome Web Store will support CrXPRT for the foreseeable future, and we’ll continue to issue any necessary bug fixes through the store. With the help of the community, we’ll reevaluate CrXPRT next year and decide the best way forward for the app. In the meantime, you can access CrXPRT from the direct link below and from CrXPRT.com. Note that the direct link may take you to a splash page if you use an unsupported platform.

CrXPRT 2015: https://chrome.google.com/webstore/detail/crxprt/hiajijaeaacmnpjpkcfnhohmaijanjgf

Please let us know if you have any questions.

Justin

Posted in Chrome OS, Chromebooks, Collaborative benchmark development, CrXPRT, Google Chrome, Performance benchmarking | Tagged Apps, Chrome, Chrome Web Store, Chromebooks, CrXPRT |

A note about a recent CrXPRT update

By Justin Greene

on June 21, 2018

A tester from Acer recently contacted us about an issue where CrXPRT was freezing indefinitely during the Photo Effects workload. We initially thought the problem was limited to a specific hardware platform or Chrome OS version, but soon discovered the issue was affecting all CrXPRT tests, regardless of the system.

After quite a bit of troubleshooting, we were able to find and fix what turned out to be simple bug. The problem started with a change we made to increase security and strengthen compliance with GDPR by moving all our web pages to HTTPS. Specifically, we added a redirect that forced principledtechnologies.com to www.principledtechnologies.com. Chrome apps have a manifest property that defines which websites can connect to the application. Because we hadn’t reconfigured the CrXPRT path permissions to account for the new redirect, the test failed. We made the necessary edits to the manifest, tested the fix, and uploaded the updated package (build number 1.0.2.1) to the Chrome Web Store.

If you’re still encountering this problem during testing, check to be sure the app has updated on your system. The changes we made do not affect performance, and all completed CrXPRT test scores from before and after the update are valid and comparable.

We’re grateful whenever community members report issues! If you ever have any problems, questions, or comments regarding any of the XPRTs, please feel free to contact us.

Justin

Posted in Benchmarking, BenchmarkXPRT, Chrome OS, Chromebooks, CrXPRT, Google Chrome, JavaScript, Performance benchmarking |

Category: Google Chrome

Planning for the next CrXPRT

A CrXPRT fix for Chrome 76

A new playing field for WebXPRT

CrXPRT is more valuable than ever

CrXPRT in the Chrome Web Store

A note about a recent CrXPRT update

Check out the other XPRTs: