BenchmarkXPRT Blog banner

Category: Cross-platform benchmarks

Archiving AIXPRT and CloudXPRT

Some of our readers have been following the XPRTs since the early days, and they may remember using legacy versions of benchmarks such as HDXPRT 2014 or WebXPRT 2013. For many years, whenever we released a new version of a benchmark, we would maintain a link to the previous version on the benchmark’s main page. However, as interest in the older versions understandably waned and we stopped formally supporting them, many of those legacy XPRTs stopped working on the latest versions of the operating systems or browsers that we designed them to test. While we wanted to continue to provide a way for users to access those legacy XPRTs, we also wanted to avoid potential confusion for new users who might see links to old versions on our site. We decided that the best solution was to archive older tests in a separate section of the site—the XPRT archive.

Recently, as we discussed XPRT plans for 2025, it became clear that we needed to add AIXPRT and CloudXPRT to the archive. Both benchmarks represent landmark efforts toward our ongoing goal of providing cutting-edge performance assessment tools, but even though a few tech press publications and OEM labs experimented with them, neither benchmark gained enough widespread adoption to justify their continued support. As a result, we decided to focus our resources elsewhere and halt development on both benchmarks. Since then, ongoing updates to their respective software components and target platforms have rendered them largely unusable. By archiving both benchmarks, we hope to avoid any future confusion for visitors who may otherwise try to use them.

Over the coming weeks, we’ll be moving the AIXPRT and CloudXPRT installation packages to the XPRT archive page. We’re grateful to everyone who has used AIXPRT and CloudXPRT in the past, and we apologize for any inconvenience this change may cause.

If you have any questions or concerns about access to either of these benchmarks—or about anything else related to the XPRTs, please let us know

Justin

February 2025 WebXPRT 4 browser performance comparisons

Once or twice per year, we refresh our ongoing series of WebXPRT comparison tests to see if software version updates have reordered the performance rankings of popular web browsers. We published our most recent comparison last June, when we used WebXPRT 4 to compare the performance of five browsers—Brave, Google Chrome, Microsoft Edge, Mozilla Firefox, and Opera—on a Lenovo ThinkPad T14s Gen 3. When assessing performance differences, it’s worth noting that all the browsers—except for Firefox—are built on a Chromium foundation. In the last round of tests, the scores were very tight, with a difference of only four percent between the last-place browser (Brave) and the winner (Chrome). Firefox’s score landed squarely in the middle of the pack.

Recently, we conducted a new set of tests to see how performance scores may have changed. To maintain continuity with our last comparison, we stuck with the same ThinkPad T14s as our reference system. That laptop is still in line with current mid-range laptops, so our comparison scores are likely to fall within the range of scores we would see from a typical user today. The ThinkPad is equipped with an Intel Core i7-1270P processor and 16 GB of RAM, and it’s running Windows 11 Pro, version 23H2 (22631.4890).

Before testing, we installed all current Windows updates, and we updated each of the browsers to the latest available stable version. After the update process was complete, we turned off updates to prevent any interference with test runs. We ran WebXPRT 4 five times on each of the five browsers. In Figure 1 below, each browser’s score is the median of the five test runs.

In this round of tests, the gap widened a bit between first and last place scores, with a difference of just over six percent between the lowest median score of 303 (Brave) and the highest median score of 322 (Firefox).

Figure 1: The median scores from running WebXPRT 4 five times with each browser on the Lenovo ThinkPad T14s Gen 3.

In this round of tests, the distribution of scores indicates that most users would not see a significant performance difference if they switched between the latest versions of these browsers. The one exception may be a change from the latest version of Brave to the latest version of Firefox. Even then, the quality of your browsing experience will often depend on other factors. The types of things you do on the web (e.g., gaming, media consumption, or multi-tab browsing), the type and number of extensions you’ve installed, and how frequently the browsers issue updates and integrate new technologies—among other things—can all affect browser performance over time. It’s important to keep such variables in mind when thinking about how browser performance comparison results may translate to your everyday web experience.

Have you tried using WebXPRT 4 in your own browser performance comparison? If so, we’d love to hear about it! Also, please let us know if there are other types of WebXPRT comparisons you’d like to see!

Justin

More than two million XPRT benchmark runs and downloads!

As we near the end of 2024, we’re excited to share that the XPRTs have passed another notable milestone—over 2,000,000 combined runs and downloads! The rate of growth in the total number of XPRT runs and downloads is exciting. It took about seven and a half years for the XPRTs to pass one million total runs and downloads—but it’s taken less than half that, three and a half years, to add another million. Figure 1 shows the climb to the two-million-run mark.

Figure 1: The cumulative number of total yearly XPRT runs and downloads over time.

As you would expect, most of the runs contributing to that total come from WebXPRT tests. If you’ve run WebXPRT in any of the 983 cities and 84 countries from which we’ve received completed test data—including newcomers El Salvador, Malaysia, Morocco, and Saudi Arabia—we’re grateful for your help in reaching this milestone! As Figure 2 illustrates, WebXPRT use has grown steadily since the debut of WebXPRT 2013. On average, we now record more than twice as many WebXPRT runs each month than we recorded in WebXPRT’s entire first year. With over 340,000 runs so far in 2024—an increase of more than 16 percent over last year’s total—that growth is showing no signs of slowing down.

Figure 2: The cumulative number of total yearly WebXPRT runs over time.

This milestone isn’t just about numbers. Establishing and maintaining a presence in the industry and experiencing year-over-year growth requires more than technical know-how and marketing efforts. It requires the ongoing trust and support of the benchmarking community—including OEM labs, the tech press, and independent computer enthusiasts—and those who simply want to know how good their devices are at web browsing.

Once again, we’re thankful for the support of everyone who’s used the XPRTs over the years, and we look forward to another million!

If you have any questions or comments about any of the XPRTs, we’d love to hear from you!

Justin

Using WebXPRT 4’s language options

Sohu, a major Chinese site, recently published a tech review discussing their first impressions from Intel Core Ultra 5 245K and Intel Core Ultra 9 285K white box testing. In the article, they included screenshots of the WebXPRT 4 test results they produced during their evaluation. The screenshots showed that the testers had enabled WebXPRT 4’s Simplified Chinese UI. They’re not the first to use this option, and it’s one we are glad worked for them.

Though WebXPRT’s language settings menu has proven to be a popular feature for many users around the world, some folks may not even know the option is there. In today’s blog, we’ll go over the basics of this simple but helpful testing option.

On WebXPRT’s Start screen, you can choose from three language options in the WebXPRT 4 UI: Simplified Chinese, German, and English. We included Simplified Chinese and German because of the large number of tests we see from China and Central Europe. We wanted to make testing a little easier for users who prefer those languages and we’re glad to see people using the options.

Changing languages in the WebXPRT UI is quick and easy. Locate the “Change Language?” prompt under the WebXPRT 4 logo at the top of the Start screen, and click or tap the arrow beside it. After the drop-down menu appears, select the language you want. The language of the start screen will then change to the language you selected, and the in-test workload headers and end-of-test results screen will also appear in the language you selected.

Figures 1–3 below my sig show the “Change Language?” drop-down menu and how the Start screen appears when you select Simplified Chinese or German. It’s important to note that if you have a translation extension installed in your browser, it may override the WebXPRT UI by reverting the language back to your browser’s default. You can avoid this conflict by temporarily disabling the browser’s translation extension for the duration of WebXPRT testing.

We hope WebXPRT 4’s language options will help facilitate the testing process for many users around the world. If you’re a frequent WebXPRT user and would like to see us add support for another language, please contact us. And, of course, if you have any questions about WebXPRT 4 testing, please let us know!

Justin

Figure 1: A screenshot of the WebXPRT 4 Start screen showing the language options drop-down menu.
Figure 2: A screenshot of the WebXPRT 4 Start screen with a Simplified Chinese UI.
Figure 3: A screenshot of the WebXPRT 4 Start screen with a German UI.

Speaking of potential future WebXPRT workloads

In recent blog posts, we’ve discussed several types of potential future WebXPRT workloads—from an auxiliary AI-focused workload to a WebXPRT battery life test—and many of the factors that we would need to consider when developing those workloads. In today’s post, we’re discussing other types of workloads that we may consider for future WebXPRT versions. We’re also inviting you to send us your WebXPRT workload ideas!

Currently, the most promising web technology for future WebXPRT workloads is WebAssembly (Wasm). Wasm is a binary instruction format that works across all modern browsers, provides a sandboxed environment that operates at native speeds, and takes advantage of common hardware specs across platforms. Wasm’s capabilities offer web developers significant flexibility in running complex client applications within the browser.

We first made use of Wasm in WebXPRT 4’s Organize Album and Encrypt Notes workloads, but Wasm has the potential to support many more types of test scenarios. Here are just a few of the use-case categories that Wasm supports:

  • Gaming
  • Image and video editing
  • Video augmentation
  • CAD applications
  • Interactive learning portals
  • Language translation

Those categories and the possibilities they open for additional workloads are exciting! When thinking through possible new workload scenarios, it’s important to remember that workload proposals need to fit within a set of basic guidelines that uphold WebXPRT’s strengths as a benchmark. You can read about those guidelines in more detail in this blog post, but in short, new workloads ideally should

  • be relevant to real-life scenarios
  • have cross-platform support
  • clearly differentiate in their performance between different types of devices
  • produce consistent and easily replicated results

After testing with WebXPRT or reviewing the list of use cases that Wasm supports, have you considered a new workload or test scenario that you would like to see? If so, please let us know! Your ideas could end up playing a role in shaping the next version of WebXPRT!

Justin

Thinking through a potential WebXPRT 4 battery life test

In recent blog posts, we’ve discussed some of the technical considerations we’re working through on our path toward a future AI-focused WebXPRT 4 auxiliary workload. While we’re especially excited about adding to WebXPRT 4’s AI performance evaluation capabilities, AI is not the only area of potential WebXPRT 4 expansion that we’ve thought about. We’re always open to hearing suggestions for ways we can improve WebXPRT 4, including any workload proposals you may have. Several users have asked about the possibility of a WebXPRT 4 battery life test, so today we’ll discuss what one might look like and some of the challenges we’d have to overcome to make it a reality.

Battery life tests fall into two primary categories: simple rundown tests and performance-weighted tests. Simple rundown tests measure battery life during extreme idle periods and loops of movie playbacks, etc., but do not reflect the wide-ranging mix of activities that characterize a typical day for most users. While they can be useful for performing very specific apples-to-apples comparisons, these tests don’t always give consumers an accurate estimate of the battery life they would experience in daily use.

In contrast, performance-weighted battery life tests, such as the one in CrXPRT 2, attempt to reflect real-world usage. The CrXPRT battery life test simulates common daily usage patterns for Chromebooks by including all the productivity workloads from the performance test, plus video playback, audio playback, and gaming scenarios. It also includes periods of wait/idle time. We believe this mixture of diverse activity and idle time better represents typical real-life behavior patterns. This makes the resulting estimated battery life much more helpful for consumers who are trying to match a device’s capabilities with their real-world needs.

From a technical standpoint, WebXPRT’s cross-platform nature presents us with several challenges that we did not face while developing the CrXPRT battery life test for ChromeOS. While the WebXPRT performance tests run in almost any browser, cross-browser differences and limitations in battery life reporting may restrict any future battery life test to a single browser or browser family. For instance, with the W3C Battery Status API, we can currently query battery status data from non-mobile Chromium-based browsers (e.g., Chrome, Edge, Opera, etc.), but not from Firefox or Safari. If a WebXPRT 4 battery life test supported only a single browser family, such as Chromium-based browsers, would you still be interested in using it? Please let us know.

A browser-based battery life workflow also presents other challenges that we do not face in native client applications, such as CrXPRT:

  • A browser-based battery life test may require the user to check the starting and ending battery capacities, with no way for the app to independently verify data accuracy.
  • The battery life test could require more babysitting in the event of network issues. We can catch network failures and try to handle them by reporting periods of network disconnection, but those interruptions could influence the battery life duration.
  • The factors above could make it difficult to achieve repeatability. One way to address that problem would be to run the test in a standardized lab environment with a steady internet connection, but a long list of standardized environmental requirements would make the battery life test less attractive and less accessible to many testers.

We’re not sharing these thoughts to make a WebXPRT 4 battery life test seem like an impossibility. Rather, we want to offer our perspective on what the test might look like and describe some of the challenges and considerations in play. If you have thoughts about battery life testing, or experience with battery life APIs in one or more of the major browsers, we’d love to hear from you!

Justin

Check out the other XPRTs:

Forgot your password?