Category: Performance benchmarking

CrXPRT in the Chrome Web Store

on September 13, 2018

Testers searching for CrXPRT in the Chrome Web Store may be puzzled to see “no results.” While CrXPRT no longer appears in Chrome Web Store searches, the app is still live in the store and there’s nothing wrong with it. CrXPRT is freely available and functioning normally, but you can access it only through a direct link. It’s an unusual situation that results from a strategic decision by the Chrome team, a decision affecting many apps in addition to CrXPRT.

For several years, Chrome supported two types of apps—hosted apps that wrap online websites, and packaged apps that have offline capability. CrXPRT is a packaged app built for Chrome OS. In December 2017, the Chrome team stopped providing search and browse functions for hosted and packaged apps in the Chrome Web Store. Instead, they’re encouraging folks to develop Progressive Web Apps. You can read about the reasoning behind this decision on the Chromium blog. Despite this shift in focus, the Chrome Web store will continue to support existing apps packaged for Chrome OS.

In short, the Chrome Web Store will support CrXPRT for the foreseeable future, and we’ll continue to issue any necessary bug fixes through the store. With the help of the community, we’ll reevaluate CrXPRT next year and decide the best way forward for the app. In the meantime, you can access CrXPRT from the direct link below and from CrXPRT.com. Note that the direct link may take you to a splash page if you use an unsupported platform.

CrXPRT 2015: https://chrome.google.com/webstore/detail/crxprt/hiajijaeaacmnpjpkcfnhohmaijanjgf

Please let us know if you have any questions.

Justin

Posted in Chrome OS, Chromebooks, Collaborative benchmark development, CrXPRT, Google Chrome, Performance benchmarking | Tagged Apps, Chrome, Chrome Web Store, Chromebooks, CrXPRT |

XPRTs in the datacenter

By Bill Catchings

on September 6, 2018

The XPRTs have been very successful on desktops, notebooks, tablets, and phones. People have run WebXPRT over 295,000 times. It and other benchmarks such as MobileXPRT, HDXPRT, and CrXPRT are important tools globally for evaluating device performance on various consumer and business client platforms.

We’ve begun branching out with tests for edge devices with AIXPRT, our new artificial intelligence benchmark. While typical consumers won’t be able to run AIXPRT on their devices initially, we feel that it is important for the XPRTs to play an active role in a critical emerging market. (We’ll have some updates on the AIXPRT front in the next few weeks.)

Recently, both community members and others have asked about the possibility of the XPRTs moving into the datacenter. Folks face challenges in evaluating the performance and suitability to task of such datacenter mainstays as servers, storage, networking infrastructure, clusters, and converged solutions. These challenges include the lack of easy-to-run benchmarks, the complexity and cost of the equipment (multi-tier servers, large amounts of storage, and fast networks) necessary to run tests, and confusion about best testing practices.

PT has a lot of expertise in measuring datacenter performance, as you can tell from the hundreds of datacenter-focused test reports on our website. We see great potential in our working with the BenchmarkXPRT Development Community to help in this area. It is very possible that, as with AIXPRT, our approach to datacenter benchmarks would differ from the approach we’ve taken with previous benchmarks. While we have ideas for useful benchmarks we might develop down the road, more immediate steps could be drafting white papers, developing testing guidelines, or working with vendors to set up a lab.

Right now, we’re trying to gauge the level of interest in having such tools and in helping us carry out these initiatives. What are the biggest challenges you face in datacenter-focused performance and suitability to task evaluations? Would you be willing to work with us in this area? We’d love to hear from you and will be reaching out to members of the community over the coming weeks.

As always, thanks for your help!

Bill

Posted in AI, AIXPRT, Benchmarking, Collaborative benchmark development, Datacenter, Future of performance evaluation, Networking, Performance benchmarking, Principled Technologies, Servers, Storage | Tagged benchmark, cluster, converged solutions, Data centers, networking, servers, storage |

HDXPRT 4: A little lighter and a lot faster

By Justin Greene

on August 30, 2018

This week, we’re sharing a little more about the upcoming HDXPRT 4 Community Preview. Just like previous versions of HDXPRT, HDXPRT 4 will use trial versions of commercial applications to complete workload tasks. We will include installers for some of those programs, such as Audacity and HandBrake, in the HDXPRT installation package. For other programs, such as Adobe Photoshop Elements 2018 and CyberLink Media Espresso 7.5, users will need to download the necessary installers prior to testing using links and instructions that we will provide. The HDXPRT 4 installation package is just over 4.7 GB, slightly smaller than previous versions.

I can also report that the new version requires fewer pre-test configuration steps and a full test run takes much less time than before. Some systems that took over an hour to complete an HDXPRT 2014 run are completing HDXPRT 4 runs in about 25 minutes.

We’ll continue to provide more information as we get closer to releasing the community preview. If you’re interested in testing with HDXPRT 4 before the general release but have not yet joined the community, we invite you to join now. If you have any questions or comments about HDXPRT or the community, please contact us.

Justin

Posted in Application-based benchmarks, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, HDXPRT, HDXPRT 4, HDXPRT development process, HDXPRT release cycle, Performance benchmarking | Tagged Adobe, Audacity, benchmark, BenchmarkXPRT, CyberLink, Handbrake, HDXPRT, HDXPRT 4 |

Sneak a peek at HDXPRT 4

By Justin Greene

on August 23, 2018

A few months ago, we shared some details about HDXPRT 4 development progress. Now that we’re closer to releasing a community preview build, we wanted to offer a sneak peek at the new benchmark. We may still tweak a few things during pre-release testing, but we’re close to the final look.

Below, you can see the benchmark’s new start page. After installation and completing a few brief pre-test configuration steps, running HDXPRT 4 is as easy as entering a test name and clicking the start button.

During the test, you’ll see HDXPRT’s real-world trial applications such as Adobe Photoshop Elements and CyberLink Media Espresso open and close during each iteration, though you won’t see workload graphics within the HDXPRT UI harness. When the test finishes, the results screen pops up. As you can see below, the results screen displays the overall and individual workload category scores in a straightforward and easy-to-understand manner. Below the workload scores, a button provides access to additional test and system information.

We’re not yet ready to share a date for the community preview, but we’ll provide more information in the coming weeks. As always, XPRT community previews are only available to BenchmarkXPRT Development Community members. If you’re interested in testing the HDXPRT 4 Community Preview, we invite you to join the community now. If you have any questions or comments about HDXPRT or the community, please contact us.

Justin

Posted in Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, HDXPRT, HDXPRT 4, HDXPRT development process, HDXPRT release cycle, PCs, Performance benchmarking, Windows 10 |

AI and the next MobileXPRT

By Justin Greene

on August 16, 2018

As we mentioned a few weeks ago, we’re in the early planning stages for the next version of MobileXPRT—MobileXPRT 3. We’re always looking for ways to make XPRT benchmark workloads more relevant to everyday users, and a new version of MobileXPRT provides a great opportunity to incorporate emerging tech such as AI into our apps. AI is everywhere and is beginning to play a huge role in our everyday lives through smarter-than-ever phones, virtual assistants, and smart homes. The challenge for us is to identify representative mobile AI workloads that have the necessary characteristics to work well in a benchmark setting. For MobileXPRT, we’re researching AI workloads that have the following characteristics:

They work offline, not in the cloud.
They don’t require additional training prior to use.
They support common use cases such as image processing, optical character recognition (OCR), etc.

We’re researching the possibility of using Google’s Mobile Vision library, but there may be other options or concerns that we’re not aware of. If you have tips for places we should look, or ideas for workloads or APIs we haven’t mentioned, please let us know. We’ll keep the community informed as we narrow down our options.

Justin

Posted in AI, Android, Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Future of performance evaluation, Google, Machine learning, Mobile devices, MobileXPRT, Performance benchmarking | Tagged AI, benchmark, Google, image processing, mobile, MobileXPRT, optical character recognition |

MWCS18 and AIXPRT: a new video

By Justin Greene

on July 12, 2018

A few weeks ago, Bill shared his first impressions from this year’s Mobile World Congress Shanghai (MWCS). “5G +” was the major theme, and there was a heavy emphasis on 5G + AI. This week, we published a video about Bill’s MWCS experience and the role that the XPRTs can play in evaluating emerging technologies such as 5G, AI, and VR. Check it out!

You can read more about AIXPRT development here. We’re still accepting responses to the AIXPRT Request for Comments, so if you would like to share your ideas on developing an AI/machine learning benchmark, please feel free to contact us.

Justin

Posted in 5G, AI, AIXPRT, AR, Augmented reality, Future of performance evaluation, Machine learning, Mobile devices, Mobile World Congress, Mobile World Congress, Performance benchmarking, Virtual reality | Tagged 5G, AI, AIXPRT, MWCS18, Shanghai, VR |

Category: Performance benchmarking

CrXPRT in the Chrome Web Store

XPRTs in the datacenter

HDXPRT 4: A little lighter and a lot faster

Sneak a peek at HDXPRT 4

AI and the next MobileXPRT

MWCS18 and AIXPRT: a new video

Check out the other XPRTs: