BenchmarkXPRT Blog banner

Category: Benchmarking

How to automate WebXPRT 4 testing

Justin Greene
By Justin Greene
on November 23, 2022

As the number of WebXPRT runs continues to grow, we realize many new WebXPRT users may be unfamiliar with all the features and capabilities of the benchmark. To help inform users about features that might facilitate their testing, we’ve decided to highlight a few WebXPRT features here in the blog. A few weeks ago, we discussed the multiple language options available in the WebXPRT 4 UI. This week, we look at WebXPRT 4 test automation.

WebXPRT 4 allows users to run scripts in an automated fashion. You can control the execution of WebXPRT 4 by appending parameters and values to the WebXPRT URL. Three parameters are available: testtype, tests, and result. Below, you’ll find a description of those parameters and instructions for utilizing automation.

Test type

The WebXPRT automation framework accounts for two test types: (1) the six core workloads and (2) any experimental workloads we might add in future builds. There are currently no experimental tests in WebXPRT 4, so always set the test type variable to 1.

  • Core tests: 1

Test scenario

This parameter lets you specify which tests to run by using the following codes:

  • Photo enhancement: 1
  • Organize album using AI: 2
  • Stock option pricing: 4
  • Encrypt notes and OCR scan using WASM: 8
  • Sales graphs: 16
  • Online homework: 32

To run a single individual test, use its code. To run multiple tests, use the sum of their codes. For example, to run Stocks (4) and Notes (8), use the sum of 12. To run all core tests, use 63, the sum of all the individual test codes (1 + 2 + 4 + 8 + 16 + 32 = 63).

Results format

This parameter lets you select the format of the results:

  • Display the result as an HTML table: 1
  • Display the result as XML: 2
  • Display the result as CSV: 3
  • Download the result as CSV: 4

To use the automation feature, start with the URL http://www.principledtechnologies.com/benchmarkxprt/webxprt/2021/wx4_build_3_7_3, append a question mark (?), and add the parameters and values separated by ampersands (&). For example, to run all the core tests and download the results, you would use the following URL: http://principledtechnologies.com/benchmarkxprt/webxprt/2021/wx4_build_3_7_3/auto.php?testtype=1&tests=63&result=4

We hope the WebXPRT automation features will make testing easier for you. If you have any questions about WebXPRT or the automation process, please feel free to ask!

Justin

Posted in Automation, benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, Browser-based benchmarks, Cross-platform benchmarks, Performance benchmarking, results, Web-based testing, WebAssembly, WebXPRT, WebXPRT 4 | Tagged , , , , , , |

The XPRTs can help with your holiday shopping!

Justin Greene
By Justin Greene
on November 3, 2022

The holiday shopping season is fast approaching, and choosing the right tech gift can often be a daunting task. If you’re considering phones, tablets, Chromebooks, or laptops as gifts, and are unsure where to get reliable device information, the XPRTs can help!

The XPRTs provide objective, reliable measures of a device’s performance that can help to cut through the marketing noise. For example, instead of guessing whether the performance of a new laptop lives up to its billing, you can use its WebXPRT performance score to see how it stacks up against the competition on everyday tasks.

A good place to start looking for device scores is our XPRT results browser, which lets you access our database of more than 3,200 test results from over 165 sources, including major tech review publications around the world, OEMs, and independent testers. You can find a wealth of current and historical performance data across all the XPRT benchmarks and hundreds of devices. Learn how to use the results browser here.

If you’re considering a popular device, chances are good that a recent tech review includes an XPRT score for it. You can find these reviews by going to your favorite tech review site and searching for “XPRT,” or entering the name of the device and the appropriate XPRT (e.g., “iPhone” and “WebXPRT”) in a search engine. Here are a few recent tech reviews that used the XPRTs to evaluate popular devices:

The XPRTs can help consumers make better-informed and more confident tech purchases this holiday season, and we hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database please feel free to ask!

Justin

Posted in AnandTech, Apple, benchmark, Benchmark metrics, BenchmarkXPRT, Chromebooks, Google, Laptops, PCs, Performance benchmarking, Phones, results, Tablets, WebXPRT | Tagged , , , , , , , , , , , , , , , |

The CloudXPRT v1.2 update package is now available!

Justin Greene
By Justin Greene
on October 27, 2022

We’re happy to announce that the CloudXPRT v1.2 update package is now available! The update prevents potential installation failures on Google Cloud Platform and Microsoft Azure, and ensures that the web microservices workload works on Ubuntu 22.04. The update uses updated software components such as Kubernetes v1.23.7, Kubespray v2.18.1, and Kubernetes Metrics Server v1, and incorporates some additional minor script changes.

The CloudXPRT v1.2 web microservices workload installation package is available at the CloudXPRT.com download page and the BenchmarkXPRT GitHub repository.

Before you get started with v1.2, please note the following updated system requirements:

  • Ubuntu 20.04.2 or 22.04 for on-premises testing
  • Ubuntu 18.04, 20.04.2, or 22.04 for CSP (AWS/Azure/GCP) testing

Because CloudXPRT is designed to run on high-end servers, physical nodes or VMs under test must meet the following minimum specifications:

  • 16 logical or virtual CPUs
  • 8 GB of RAM
  • 10 GB of available disk space (50 GB for the data analytics workload)

The update package includes only the updated v1.2 test harness and the updated web microservices workload. It does not include the data analytics workload. As we stated in the blog, now that we’ve published the web microservices package, we will assess the level of interest users express about a possible refresh of the v1.1 data analytics workload. For now, the v1.1 data analytics workload will continue to be available via CloudXPRT.com for some time to serve as a reference resource for users who have worked with the package in the past.

Please let us know if you have any questions about the CloudXPRT v1.2 test package. Happy testing!

Justin

Posted in Amazon Web Services, benchmark, Benchmark metrics, Cloud, CloudXPRT, Cross-platform benchmarks, CSP, data analytics, Datacenter, Google Cloud Platform, Kubernetes, microservices, Microsoft Azure, On-premise, Performance benchmarking, Servers, Ubuntu, web microservices | Tagged , , , , , , , , , , , , |

We have a new XPRTs around the world infographic!

Justin Greene
By Justin Greene
on October 20, 2022

If you’ve followed the XPRT blog for a while, you know that we occasionally update the community on some of the reach metrics we track by publishing a new version of the “XPRTs around the world” infographic. The metrics we track include completed test runs, benchmark downloads, and mentions of the XPRTs in advertisements, articles, and tech reviews. Gathering this information gives us insight into how many people are using the XPRT tools, and updating the infographic helps readers and community members see the impact the XPRTs are having around the world.

This week, we published a new infographic, which includes the following highlights:

  • The XPRTs have been mentioned more than 19,500 times on over 4,000 unique sites.
  • Those mentions include more than 12,300 articles and reviews.
  • Users have tested gear with the XPRTs in over 924 cities located in 81 countries on six continents. New cities of note include Dhaka, Bangladesh; Zagreb, Croatia; Hamilton, New Zealand; and Medina, Saudi Arabia.

In addition to the reach metrics we mention above, the XPRTs have now delivered more than 1,330,000 real-world results! We’re grateful for everyone who’s helped us get this far. Your participation is vital to our achieving our goal: to provide benchmark tools that are reliable, relevant, and easy to use.

Justin

Posted in benchmark, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, results, tech press | Tagged , , , , , |

The versatility of XPRT benchmarks

Justin Greene
By Justin Greene
on October 13, 2022

We’ve designed each of the XPRT benchmarks to assess the performance of specific types of devices in scenarios that mirror the ways consumers typically use those devices. While most XPRT benchmark users are interested in producing official overall scores, some members of the tech press have been using the XPRTs in unconventional, creative ways.

One example is the use of WebXPRT by Tweakers, a popular tech review site based in The Netherlands. (The site is in Dutch, so the Google Translate extension in Chrome was helpful for me.) As Tweakers uses WebXPRT to evaluate all kinds of consumer hardware, they also measure the sound output of each device. Tweakers then publishes the LAeq metric for each device, giving readers a sense of how loud a system may be, on average, while it performs common browser tasks.

If you’re interested in seeing Tweakers’ use of WebXPRT for sound output testing firsthand, check out their Apple MacBook Pro M2, HP Envy 34 All-in-One, and Samsung Galaxy Book 2 Pro reviews.

Other labs and tech publications have also used the XPRTs in unusual ways such as automating the benchmarks to run during screen burn-in tests or custom battery-life rundowns. If you’ve used any of the XPRT benchmarks in creative ways, please let us know! We are interested in learning more about your tests, and your experiences may provide helpful information that we can share with other XPRT users.

Justin

Posted in Apple, Benchmark metrics, Benchmarking, Browser-based benchmarks, Cross-platform benchmarks, Google Chrome, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 3, WebXPRT 4 | Tagged , , , , , , , , , |

Let us know if you encounter this Adobe PSE 2020 issue with HDXPRT 4

Justin Greene
By Justin Greene
on October 6, 2022

Last week, a member of the tech press let us know that they encountered an error while preparing a system for HDXPRT 4 testing. Specifically, while attempting to install the trial version of Adobe Photoshop Elements (PSE) 2020, they encountered the following error:

Update Required

Your browser or operating system is no longer supported. You may need to install the latest updates to your operating system.

They were working with an MSI Sword 15 A12UE, which had all the latest Windows 11 and Microsoft Edge updates, and they were able to complete installation and testing on other Windows 11 systems in their lab. This eliminates compatibility between the Adobe PSE 2020 installer package and Windows 11 or Microsoft Edge as the issue.

We do not have the same MSI Sword system in our lab, but we tried to replicate the issue by performing the HDXPRT 4 installation and setup process on a Dell G7 15 laptop running on an up-to-date version of Windows 11 (22H2, 22621.521). We successfully installed Adobe PSE 2020 and completed several HDXPRT 4 iterations.

The error this user encountered could be specific to their system or situation. However, we would like to know if other HDXPRT 4 users have run into the same issue. If you’ve experienced this issue in your testing, please contact us. We may be able to identify and publish a solution. 

Justin

Posted in Adobe, Adobe Photoshop, benchmark, BenchmarkXPRT, HDXPRT, HDXPRT 4, Microsoft Edge, Performance benchmarking, Windows 11 | Tagged , , , , , , , |

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x