BenchmarkXPRT Blog banner

Category: Chromebooks

The CrXPRT 2 Community Preview is almost here!

Justin Greene
By Justin Greene
on March 12, 2020

We appreciate everyone’s patience and feedback during the CrXPRT 2 development process. We’re excited to say that we’re now wrapping up some final details and expect to release the Community Preview (CP) within the next week.

Here is a summary of the key differences between CrXPRT 2015 and CrXPRT 2:

  • As we mentioned a few weeks ago, CrXPRT 2 has a completely new UI in line with the functional and aesthetic themes we used for the latest versions of WebXPRT, MobileXPRT, and HDXPRT, with a focus on intuitive navigation.
  • The CrXPRT 2 performance test includes six of the seven workloads in CrXPRT 2015. Newer versions of Chrome can’t run the Photo Collage workload without a workaround, so we removed it from CrXPRT 2.
  • We updated the images in the Photo Effects and Face Detection workloads to reflect more contemporary file resolutions and sizes.
  • The CrXPRT 2 battery life test requires a full rundown, so you’ll need charge your device to 100 percent before you can start the test, and the length of battery life tests will vary according to the battery life of the systems under test.
  • We no longer require testers to enter luminance and audio measurements in order to run a battery life test.
  • We added a second video playback segment to each battery life iteration.


We allow testers to publish CP test scores, but CrXPRT 2 overall performance test scores and battery life measurements are not comparable to CrXPRT 2015 scores.

Only BenchmarkXPRT Development Community members will be able to download the CrXPRT 2 CP. Because the Chrome team stopped providing search and browse functions for hosted and packaged Chrome apps in the Chrome Web Store, members will need a direct link to access the app. Once the app is available, we’ll post that link, along with the CrXPRT 2 CP user manual, on the CrXPRT tab in the XPRT Members’ Area (login required). We’ll also send a message to the community and post a notice here in the blog.

If you have any questions about CrXPRT 2 or joining the community, please let us know!

Justin

Posted in Battery life, benchmark, BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Chrome OS, Chromebooks, Community Preview, CrXPRT, Google Chrome, HDXPRT, MobileXPRT, Performance benchmarking, WebXPRT | Tagged , , , , , |

A preview of the new CrXPRT 2 UI

Justin Greene
By Justin Greene
on January 30, 2020

As we get closer to the CrXPRT 2 Community Preview (CP), we want to provide readers with a glimpse of the new CrXPRT 2 UI. In line with the functional and aesthetic themes we used for the latest versions of WebXPRT, MobileXPRT, and HDXPRT, we’re implementing a clean, bright look with a focus on intuitive navigation. The screenshots below show how we’ve used that approach to rework the home, battery life test, performance test, and battery life test results screens. (We’re still tweaking the UI, so the screens you see in the CP may differ slightly.)

On the home screen, we kept the performance test and battery life test buttons, but made it clearer that you can choose only one. We also added a link to the user manual to the bottom ribbon for quick access.

If you choose to run a battery life test and click Next, the screen below appears. The CrXPRT 2 battery life test requires a full rundown, so you’ll need charge your device to 100 percent before you can start the test. Once you’ve done that, enter a name for the test run, unplug the system, and click Start. (Note that you no longer need to enter values for screen brightness and audio levels.)

The CrXPRT 2 performance test includes updated versions of six of the seven workloads in CrXPRT 2015. (As we discussed in a previous blog post, newer versions of Chrome can’t run the Photo Collage workload without a workaround, so we removed it from CrXPRT 2.)  To run the performance test, enter a name for the test run, customize the workloads if you wish, and click Start.

For the results screens, we wanted to highlight the most important end-of-test information while still offering clear paths for options such as getting additional details on the test, submitting results, and running the test again. Below, we show the results screen from a battery life test. Note the “Main menu” link in the upper-left corner, which we added to all screens to give users a quick way to navigate back to the home screen.

CrXPRT 2 development and testing are still underway. We don’t yet have an exact release date for the CP, but once we do, we’ll announce it here in the blog.

What do you think about the new CrXPRT 2 UI? Let us know!

Justin

Posted in Battery life, benchmark, BenchmarkXPRT, BenchmarkXPRT development community, Chrome OS, Chromebooks, Community Preview, CrXPRT, Performance benchmarking | Tagged , , , , , , |

The XPRT activity we have planned for first half of 2020

Justin Greene
By Justin Greene
on January 23, 2020

Today, we want to let readers know what to expect from the XPRTs over the next several months. Timelines and details can always change, but we’re confident that community members will see CloudXPRT Community Preview (CP), updated AIXPRT, and CrXPRT 2 releases during the first half of 2020.

CloudXPRT

Last week, Bill shared some details about our new datacenter-oriented benchmark, CloudXPRT. If you missed that post, we encourage you to check it out and learn more about the need for a new kind of cloud benchmark, and our plans for the benchmark’s structure and metrics. We’re already testing preliminary builds, and aim to release a CloudXPRT CP in late March, followed by a version for general availability roughly two months later.

AIXPRT

About a month ago, we explained how the number of moving parts in AIXPRT will necessitate a different development approach than we’ve used for other XPRTs. AIXPRT will require more frequent updating than our other benchmarks, and we anticipate releasing the second version of AIXPRT by mid-year. We’re still finalizing the details, but it’s likely to include the latest versions of ResNet-50 and SSD-MobileNet, selected SDK updates, ease-of-use improvements for the harness, and improved installation scripts. We’ll share more detailed information about the release timeline here in the blog as soon as possible.

CrXPRT 2

As we mentioned in December, we’re working on CrXPRT 2, the next version of our benchmark that evaluates the performance and battery life of Chromebooks. You can find out more about how CrXPRT works both here in the blog and at CrXPRT.com.

We’re currently testing an alpha version of CrXPRT 2. Testing is going well, but we’re tweaking a few items and refining the new UI. We should start testing a CP candidate in the next few weeks, and will have firmer information for community members about a CP release date very soon.

We’re excited about these new developments and the prospect of extending the XPRTs into new areas. If you have any questions about CloudXPRT, AIXPRT, or CrXPRT 2, please feel free to ask!

Justin

Posted in AI, AIXPRT, benchmark, Benchmark metrics, BenchmarkXPRT, BenchmarkXPRT development community, Chromebooks, Cloud, CloudXPRT, Collaborative benchmark development, Community Preview, CrXPRT, Datacenter, ResNet-50, SSD-MobileNet v1 | Tagged , , , , , , , , |

Planning for the next CrXPRT

Justin Greene
By Justin Greene
on December 5, 2019

We’re currently planning the next version of CrXPRT, our benchmark that evaluates the performance and battery life of Chromebooks. If you’re unfamiliar with CrXPRT, you can find out more about how it works both here in the blog and at CrXPRT.com. If you’ve used CrXPRT, we’d love to hear any suggestions you may have. What do you like or dislike about CrXPRT? What features do you hope to see in a new version?

When we begin work on a new version of any benchmark, one of our first steps is to determine whether the workloads will provide value during the years ahead. As technology and user behavior evolve, we update test content to be more relevant. One example is when we replace photos with ones that use more contemporary file resolutions and sizes.

Sometimes the changing tech landscape prompts us to remove entire workloads and add new ones. The Photo Collage workload in CrXPRT uses Portable Native Client (PNaCl) technology, for which the Chrome team will soon end support. CrXPRT 2015 has a workaround for this issue, but the best course of action for the next version of CrXPRT will be to remove this workload altogether.

The battery life test will also change. Earlier this year, we started to see unusual battery life estimates and high variance when running tests at CrXPRT’s default battery life test length of 3.5 hours, so we’ve been recommending that users perform full rundowns instead. In the next CrXPRT, the battery life test will require full rundowns.

We’ll also be revamping the CrXPRT UI to improve the look of the benchmark and make it easier to use, as we’ve done with the other recent XPRT releases.

We really do want to hear your ideas, and any feedback you send has a chance to shape the future of the benchmark. Let us know what you think!

Justin

Posted in Battery life, benchmark, Benchmarking, BenchmarkXPRT, Browser-based benchmarks, Chrome OS, Chromebooks, Collaborative benchmark development, CrXPRT, Google Chrome, Performance benchmarking | Tagged , , , , , |

The XPRT Spotlight Black Friday Showcase helps you shop with confidence

Justin Greene
By Justin Greene
on November 14, 2019

Black Friday and Cyber Monday are almost here, and you may be feeling overwhelmed by the sea of tech gifts to choose from. The XPRTs are here to help. We’ve gathered the product specs and performance facts for some of the hottest tech devices in one convenient place—the XPRT Spotlight Black Friday Showcase. The Showcase is a free shopping tool that provides side-by-side comparisons of some of the season’s most popular smartphones, laptops, Chromebooks, tablets, and PCs. It helps you make informed buying decisions so you can shop with confidence this holiday season.

Want to know how the Google Pixel 4 stacks up against the Apple iPhone 11 or Samsung Galaxy Note10 in web browsing performance or screen size? Simply select any two devices in the Showcase and click Compare. You can also search by device type if you’re interested in a specific form factor such as consoles or tablets.

The Showcase doesn’t go away after Black Friday. We’ll rename it the XPRT Holiday Showcase and continue to add devices such as the Microsoft Surface Pro X throughout the shopping season. Be sure to check back in and see how your tech gifts measure up.

If this is the first you’ve heard about the XPRT Tech Spotlight, here’s a little background. Our hands-on testing process equips consumers with accurate information about how devices function in the real world. We test devices using our industry-standard BenchmarkXPRT tools: WebXPRT, MobileXPRT, TouchXPRT, CrXPRT, BatteryXPRT, and HDXPRT. In addition to benchmark results, we include photographs, specs, and prices for all products. New devices come online weekly, and you can browse the full list of almost 200 that we’ve featured to date on the Spotlight page.

If you represent a device vendor and want us to feature your product in the XPRT Tech Spotlight, please visit the website for more details.

Justin

Posted in BenchmarkXPRT, Chromebooks, Cross-platform benchmarks, HDXPRT, Laptops, Mobile devices, MobileXPRT, Performance benchmarking, Performance of computing devices, Performance testing on tablets, Tablets, TouchXPRT, WebXPRT, XPRT Tech Spotlight |

A CrXPRT fix for Chrome 76

Justin Greene
By Justin Greene
on August 22, 2019

After Chrome OS version 76 moved from Chrome’s Beta channel to the Stable channel last week, we became aware of an issue that occurs when CrXPRT’s Photo Collage workload runs on a Chrome 76 system. We found that the Photo Collage workload produces an error message—“This plugin is not supported on this device”—and the test run does not complete.

The error occurs because the Photo Collage workload uses Portable Native Client (PNaCl), and starting with version 76, the Chrome team changed the way the OS handles PNaCl tasks. Technically, Chrome still supports PNaCl, but the OS now disables the capability by default. Chrome’s current plan is to end support for PNaCl by the end of this year, focusing related development efforts on WebAssembly instead.

We’ll investigate the best path forward during this transition, but for now, testers can use the following workaround that allows CrXPRT to complete successfully. Simply navigate to chrome://flags on the test system, and find the Native Client flag, which is set to “Disabled” by default. Click the toggle switch to “Enabled” to allow native client capabilities, restart the system, and kick off a CrXPRT test in the normal manner.

We’ll update the CrXPRT web page and test documentation to include information about the workaround. In the long term, we’re interested in any suggestions you have for CrXPRT—whether they’re related to PNaCl or not. Please let us know your thoughts!

Justin

Posted in Chrome OS, Chromebooks, CrXPRT, Google Chrome | Tagged , , , , , , , , |

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x