Category: Benchmark metrics

The XPRTs can help with your holiday shopping!

on November 3, 2022

The holiday shopping season is fast approaching, and choosing the right tech gift can often be a daunting task. If you’re considering phones, tablets, Chromebooks, or laptops as gifts, and are unsure where to get reliable device information, the XPRTs can help!

The XPRTs provide objective, reliable measures of a device’s performance that can help to cut through the marketing noise. For example, instead of guessing whether the performance of a new laptop lives up to its billing, you can use its WebXPRT performance score to see how it stacks up against the competition on everyday tasks.

A good place to start looking for device scores is our XPRT results browser, which lets you access our database of more than 3,200 test results from over 165 sources, including major tech review publications around the world, OEMs, and independent testers. You can find a wealth of current and historical performance data across all the XPRT benchmarks and hundreds of devices. Learn how to use the results browser here.

If you’re considering a popular device, chances are good that a recent tech review includes an XPRT score for it. You can find these reviews by going to your favorite tech review site and searching for “XPRT,” or entering the name of the device and the appropriate XPRT (e.g., “iPhone” and “WebXPRT”) in a search engine. Here are a few recent tech reviews that used the XPRTs to evaluate popular devices:

AnandTech used WebXPRT to evaluate the MSI Titan GT77 gaming laptop.
Notebookcheck used WebXPRT in reviews of the Apple iPhone 14 Plus, the ASUS ZenFone 9, Google Pixel 6a, and the Samsung Galaxy Z Fold4 5G.
Tom’s Guide used WebXPRT to test the Apple MacBook Pro (M1 2022).
PCMag used WebXPRT to evaluate the Apple MacBook Air (M2 20022) and Lenovo Legion 7 Gen 7 laptops.
PCWorld used WebXPRT in a review of the Microsoft Surface Pro 9.

The XPRTs can help consumers make better-informed and more confident tech purchases this holiday season, and we hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database please feel free to ask!

Justin

Posted in AnandTech, Apple, benchmark, Benchmark metrics, BenchmarkXPRT, Chromebooks, Google, Laptops, PCs, Performance benchmarking, Phones, results, Tablets, WebXPRT | Tagged AnandTech, Apple, ASUS, BenchmarkXPRT, BenchmarkXPRT Development Community, Google, Lenovo, Microsoft, Notebookcheck, PCMag, PCWorld, results, Samsung, Tom's Hardware, WebXPRT, WebXPRT 4 |

The CloudXPRT v1.2 update package is now available!

By Justin Greene

on October 27, 2022

We’re happy to announce that the CloudXPRT v1.2 update package is now available! The update prevents potential installation failures on Google Cloud Platform and Microsoft Azure, and ensures that the web microservices workload works on Ubuntu 22.04. The update uses updated software components such as Kubernetes v1.23.7, Kubespray v2.18.1, and Kubernetes Metrics Server v1, and incorporates some additional minor script changes.

The CloudXPRT v1.2 web microservices workload installation package is available at the CloudXPRT.com download page and the BenchmarkXPRT GitHub repository.

Before you get started with v1.2, please note the following updated system requirements:

Ubuntu 20.04.2 or 22.04 for on-premises testing
Ubuntu 18.04, 20.04.2, or 22.04 for CSP (AWS/Azure/GCP) testing

Because CloudXPRT is designed to run on high-end servers, physical nodes or VMs under test must meet the following minimum specifications:

16 logical or virtual CPUs
8 GB of RAM
10 GB of available disk space (50 GB for the data analytics workload)

The update package includes only the updated v1.2 test harness and the updated web microservices workload. It does not include the data analytics workload. As we stated in the blog, now that we’ve published the web microservices package, we will assess the level of interest users express about a possible refresh of the v1.1 data analytics workload. For now, the v1.1 data analytics workload will continue to be available via CloudXPRT.com for some time to serve as a reference resource for users who have worked with the package in the past.

Please let us know if you have any questions about the CloudXPRT v1.2 test package. Happy testing!

Justin

Posted in Amazon Web Services, benchmark, Benchmark metrics, Cloud, CloudXPRT, Cross-platform benchmarks, CSP, data analytics, Datacenter, Google Cloud Platform, Kubernetes, microservices, Microsoft Azure, On-premise, Performance benchmarking, Servers, Ubuntu, web microservices | Tagged Amazon Web Services, benchmark, cloud, cloud applications, CloudXPRT, data analytics, Google Cloud Platform, Kubernetes, Kubespray, Microsoft Azure, servers, Ubuntu, web microservices |

The versatility of XPRT benchmarks

By Justin Greene

on October 13, 2022

We’ve designed each of the XPRT benchmarks to assess the performance of specific types of devices in scenarios that mirror the ways consumers typically use those devices. While most XPRT benchmark users are interested in producing official overall scores, some members of the tech press have been using the XPRTs in unconventional, creative ways.

One example is the use of WebXPRT by Tweakers, a popular tech review site based in The Netherlands. (The site is in Dutch, so the Google Translate extension in Chrome was helpful for me.) As Tweakers uses WebXPRT to evaluate all kinds of consumer hardware, they also measure the sound output of each device. Tweakers then publishes the LAeq metric for each device, giving readers a sense of how loud a system may be, on average, while it performs common browser tasks.

If you’re interested in seeing Tweakers’ use of WebXPRT for sound output testing firsthand, check out their Apple MacBook Pro M2, HP Envy 34 All-in-One, and Samsung Galaxy Book 2 Pro reviews.

Other labs and tech publications have also used the XPRTs in unusual ways such as automating the benchmarks to run during screen burn-in tests or custom battery-life rundowns. If you’ve used any of the XPRT benchmarks in creative ways, please let us know! We are interested in learning more about your tests, and your experiences may provide helpful information that we can share with other XPRT users.

Justin

Posted in Apple, Benchmark metrics, Benchmarking, Browser-based benchmarks, Cross-platform benchmarks, Google Chrome, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 3, WebXPRT 4 | Tagged Apple, battery life, HP, Netherlands, Samsung, Sound, Tweakers, WebXPRT, WebXPRT 4, XPRTs |

CloudXPRT status and next steps

By Justin Greene

on August 11, 2022

We developed our first cloud benchmark, CloudXPRT, to measure the performance of cloud applications deployed on modern infrastructure as a service (IaaS) platforms. When we first released CloudXPRT in February of 2021, the benchmark included two test packages: a web microservices workload and a data analytics workload. Both supported on-premises and cloud service provider (CSP) testing with Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure.

CloudXPRT is our most complex benchmark, requiring sustained compatibility between many software components across multiple independent test environments. As vendors roll out updates for some components and stop supporting others, it’s inevitable that something will break. Since CloudXPRT’s launch, we’ve become aware of installation failures while attempting to set up CloudXPRT on Ubuntu virtual machines with GCP and Microsoft Azure. Additionally, while the web microservices workload continues to run in most instances with a few configuration tweaks and workarounds, the data analytics workload fails consistently due to compatibility issues with Minio, Prometheus, and Kafka within the Kubernetes environment.

In response, we’re working to fix problems with the web microservices workload and bring all necessary components up to date. We’re developing an updated test package that will work on Ubuntu 22.04, using Kubernetes v1.23.7 and Kubespray v2.18.1. We’re also updating Kubernetes Metrics Server from v1beta1 to v1, and will incorporate some minor script changes. Our goal is to ensure successful installation and testing with the on-premises and CSP platforms that we supported when we first launched CloudXPRT.

We are currently focusing on the web microservices workload for two reasons. First, more users have downloaded it than the data analytics workload. Second, we think we have a clear path to success. Our plan is to publish the updated web microservices test package, and see what feedback and interest we receive from users about a possible data analytics refresh. The existing data analytics workload will remain available via CloudXPRT.com for the time being to serve as a reference resource.

We apologize for the inconvenience that these issues have caused. We’ll provide more information about a release timeline and final test package details here in the blog as we get closer to publication. If you have any questions about the future of CloudXPRT, please feel free to contact us!

Justin

Posted in Amazon Web Services, benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, Cloud, CloudXPRT, Cross-platform benchmarks, CSP, data analytics, Datacenter, Google Cloud Platform, Hosted cloud, Hybrid cloud, IaaS, Kubernetes, microservices, Microsoft Azure, On-premise, Performance benchmarking, web microservices | Tagged Amazon Web Services, cloud, cloud applications, CloudXPRT, Google Cloud Platform, Kubernetes, Kubespray, Microsoft Azure, on-premises, public cloud, Ubuntu |

The XPRTs can help with your back-to-school shopping

By Justin Greene

on August 4, 2022

The new school year is upon us, and learners of all ages are looking for tech devices that have the capabilities they will need in the coming year. The tech marketplace can be confusing, and competing claims can be hard to navigate. The XPRTs are here to help! Whether you’re shopping for a new phone, tablet, Chromebook, laptop, or desktop, the XPRTs can provide reliable, industry-trusted performance scores that can cut through all the noise.

A good place to start looking for scores is the WebXPRT 4 results viewer. The viewer displays WebXPRT 4 scores from over 175 devices—including many hot new releases—and we’re adding new scores all the time. To learn more about the viewer’s capabilities and how you can use it to compare devices, check out this blog post.

Another resource we offer is the XPRT results browser. The browser is the most efficient way to access the XPRT results database, which currently holds more than 3,000 test results from over 120 sources, including major tech review publications around the world, OEMs, and independent testers. It offers a wealth of current and historical performance data across all of the XPRT benchmarks and hundreds of devices. You can read more about how to use the results browser here.

Also, if you’re considering a popular device, chances are good that a recent tech review includes an XPRT score for that device. Two quick ways to find these reviews: (1) go to your favorite tech review site and search for “XPRT” and (2) go to a search engine and enter the device name and XPRT name (e.g., “Apple MacBook Air” and “WebXPRT”). Here are a few recent tech reviews that use one of the XPRTs to evaluate a popular device:

Notebookcheck used WebXPRT in reviews of the Acer Swift X 16, Apple MacBook Air, ASUS ROG Flow X16, Lenovo V17G2, Nothing Phone (1); and a recent article titled, “The Best Smartphones.”
PCMag used WebXPRT 3 to compare the M1 Max and M1 Ultra versions of the Apple Mac Studio, and to review the Apple Macbook Air (2022, M2).
PCWorld used CrXPRT 2 in a feature called, “The best Chromebooks: Best overall, best battery life, and more.”
ZDNet used CrXPRT 2 in a review titled, “The 5 best Chromebooks for students: Top back-to-school picks.”

The XPRTs can help consumers make better-informed and more confident tech purchases. As this school year begins, we hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database please feel free to ask!

Justin

Posted in Apple, benchmark, Benchmark metrics, Browser-based benchmarks, Chromebooks, CrXPRT, Education, Laptops, Mobile devices, Performance benchmarking, Phones, results, Tablets, WebXPRT, WebXPRT 3, WebXPRT 4 | Tagged Acer, Apple, ASUS, Chromebooks, education, Lenovo, Notebookcheck, PCMag, PCWorld, results, school, WebXPRT, ZDNet |

WebXPRT passes the million-run milestone!

By Justin Greene

on July 21, 2022

We’re excited to see that users have successfully completed over 1,000,000 WebXPRT runs! If you’ve run WebXPRT in any of the 924 cities and 81 countries from which we’ve received complete test data—including newcomers Bahrain, Bangladesh, Mauritius, The Philippines, and South Korea —we’re grateful for your help. We could not have reached this milestone without you!

As the chart below illustrates, WebXPRT use has grown steadily since the debut of WebXPRT 2013. On average, we now record more WebXPRT runs in one month than we recorded in the entirety of our first year. With over 104,000 runs so far in 2022, that growth is continuing.

For us, this moment represents more than a numerical milestone. Developing and maintaining a benchmark is never easy, and a cross-platform benchmark that will run on a wide variety of devices poses an additional set of challenges. For such a benchmark to succeed, developers need not only technical competency, but the trust and support of the benchmarking community. WebXPRT is now in its ninth year, and its consistent year-over-year growth tells us that the benchmark continues to hold value for manufacturers, OEM labs, the tech press, and end users like you. We see it as a sign of trust that folks repeatedly return to the benchmark for reliable performance metrics. We’re grateful for that trust, and for everyone that’s contributed to the WebXPRT development process throughout the years.

We’ll have more to share related to this exciting milestone in the weeks to come, so stay tuned to the blog. If you have any questions or comments about WebXPRT, we’d love to hear from you!

Justin

Posted in benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, Browser-based benchmarks, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2013, WebXPRT 2015, WebXPRT 3, WebXPRT 4 | Tagged browser benchmark, browser performance, cross-platform, Performance, WebXPRT, WebXPRT 2013, WebXPRT 2015, WebXPRT 3, WebXPRT 4 |

Category: Benchmark metrics

The XPRTs can help with your holiday shopping!

The CloudXPRT v1.2 update package is now available!

The versatility of XPRT benchmarks

CloudXPRT status and next steps

The XPRTs can help with your back-to-school shopping

WebXPRT passes the million-run milestone!

Check out the other XPRTs: