If you’ve
followed the XPRT blog for a while, you know that we occasionally update the
community on some of the reach metrics we track by publishing a new version of the
“XPRTs
around the world” infographic. The metrics we track include completed test
runs, benchmark downloads, and mentions of the XPRTs in advertisements, articles,
and tech reviews. Gathering this information gives us insight into how many people
are using the XPRT tools, and updating the infographic helps readers and
community members see the impact the XPRTs are having around the world.
This week, we
published a new infographic, which includes the following highlights:
The XPRTs have been mentioned more than 19,500 times on over 4,000 unique sites.
Those mentions include more than 12,300 articles and reviews.
Users have tested gear with the XPRTs in over 924 cities located in 81 countries on six continents. New cities of note include Dhaka, Bangladesh; Zagreb, Croatia; Hamilton, New Zealand; and Medina, Saudi Arabia.
In addition to the reach metrics we mention above, the XPRTs have now delivered more than 1,330,000 real-world results! We’re grateful for everyone who’s helped us get this far. Your participation is vital to our achieving our goal: to provide benchmark tools that are reliable, relevant, and easy to use.
Last week, we
published the Exploring WebXPRT 4 white paper.
The paper describes the design and structure of WebXPRT 4, including detailed
information about the benchmark’s harness, HTML5 and WebAssembly capability
checks, and the structure of the performance test workloads. This week, to
help WebXPRT 4 testers understand how the benchmark calculates results, we’ve published
the WebXPRT 4 results calculation and confidence interval white
paper.
The white
paper explains the WebXPRT 4 confidence interval and how it differs from typical
benchmark variability, and the formulas the benchmark uses to calculate the
individual workload scenario scores and overall score. The paper also provides
an overview of the statistical techniques WebXPRT uses to translate raw timings
into scores.
To supplement
the white paper’s discussion of the results calculation process, we’ve also
published a results calculation spreadsheet that shows the
raw data from a sample test run and reproduces the calculations WebXPRT uses to
produce workload scores and the overall score.
This week, we published the Exploring WebXPRT 4 white paper. It describes the design and structure of WebXPRT 4, including detailed information about the benchmark’s harness, HTML5 and WebAssembly (WASM) capability checks, and changes we’ve made to the structure of the performance test workloads. We explain the benchmark’s scoring methodology, how to automate tests, and how to submit results for publication. The white paper also includes information about the third-party functions and libraries that WebXPRT 4 uses during the HTML5 and WASM capability checks and performance workloads.
The Exploring WebXPRT 4 white paper promotes
the high level of transparency and disclosure that is a core value of the
BenchmarkXPRT Development Community. We’ve always believed that transparency
builds trust, and trust is essential for a healthy benchmarking community.
That’s why we involve community members in the benchmark development process
and disclose how we build our benchmarks and how they work.
Recently,
Tom’s Guide published an interesting article
about how they used ChromeOS Flex to turn a
ten-year-old Apple MacBook Pro into a functioning Chromebook by replacing the
laptop’s macOS operating system with ChromeOS. ChromeOS Flex is a free Google
tool that allows users to create a bootable USB drive that they can then use to
install ChromeOS on a wide variety
of hardware platforms that traditionally run other operating systems such as
macOS or Windows. Because ChromeOS is a cloud-first, relatively low-overhead
operating system, the ChromeOS Flex option could breathe new life into an old
laptop that you have lying around.
Never having encountered a MacBook Pro with ChromeOS, we were interested to learn about Tom’s experience running XPRT benchmarks in this new environment.WebXPRT 4, WebXPRT 3, and the CrXPRT 2 performance test apparently ran without any issues, but we have not yet seen a CrXPRT 2 battery life result from a ChromeOS Flex environment. We plan to experiment with this soon.
We
were happy to publish the results on our site, and will consider any ChromeOS
Flex results we receive for publication. If you submit results from ChromeOS
Flex testing, we ask that you use the “Additional information” field in the
results submission form to clarify that you ran the tests in a ChromeOS Flex
environment. This will prevent any possible confusion when we see a submission
that lists a traditional macOS or Windows hardware platform along with a
ChromeOS version number.
Do you have experience running CrXPRT or WebXPRT with ChromeOS Flex? We’d love to hear about it!
The new school year is
upon us, and learners of all ages are looking for tech devices that have the
capabilities they will need in the coming year. The tech marketplace can be
confusing, and competing claims can be hard to navigate. The XPRTs are here to
help! Whether you’re shopping for a new phone, tablet, Chromebook, laptop, or
desktop, the XPRTs can provide reliable, industry-trusted performance scores
that can cut through all the noise.
A good place to start looking
for scores is the WebXPRT 4 results viewer. The viewer displays WebXPRT 4 scores from
over 175 devices—including many hot new releases—and we’re adding new scores
all the time. To learn more about the viewer’s capabilities and how you can use
it to compare devices, check out this blog post.
Another resource we
offer is the XPRT results browser. The browser is the most efficient way to access the XPRT
results database, which currently holds more than 3,000 test results from over 120
sources, including major tech review publications around the world, OEMs, and
independent testers. It offers a wealth of current and historical performance
data across all of the XPRT benchmarks and hundreds of devices. You can read
more about how to use the results browser here.
Also, if you’re considering a popular device, chances are good that a recent tech review includes an XPRT score for that device. Two quick ways to find these reviews: (1) go to your favorite tech review site and search for “XPRT” and (2) go to a search engine and enter the device name and XPRT name (e.g., “Apple MacBook Air” and “WebXPRT”). Here are a few recent tech reviews that use one of the XPRTs to evaluate a popular device:
The XPRTs can help consumers make better-informed and more confident tech purchases. As this school year begins, we hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database please feel free to ask!
Testers
new to the XPRT benchmarks may not know about one of the free resources we
offer. The XPRT results database currently holds more than 3,000 test results
from over 120 sources, including major tech review publications around the
world, OEMs, and independent testers. It offers a wealth of current and
historical performance data across all the XPRT benchmarks and hundreds of
devices.
We update the results
database several times a week, adding selected results from our own internal
lab testing, reliable tech media sources, and end-of-test user submissions.
(After you run one of the XPRTs, you can choose to submit the results, but they
don’t automatically appear in the database.) Before adding a result, we
evaluate whether the score makes sense and is consistent with general
expectations, which we can do only when we have sufficient system information details.
For that reason, we ask testers to disclose as much hardware and software
information as possible when publishing or submitting a result.
We encourage visitors to our site to explore the XPRT results database. There are three primary ways to do so. The first is by visiting the main BenchmarkXPRT results browser, which displays results entries for all of the XPRT benchmarks in chronological order (see the screenshot below). You can narrow the results by selecting a benchmark from the drop-down menu and can type values, such as vendor or the name of a tech publication, into the free-form filter field. For results we’ve produced in our lab, clicking “PT” in the Source column takes you to a page with additional disclosure information for the test system. For sources outside our lab, clicking the source name takes you to the original article or review that contains the result.
The second way to access our published results is by visiting the results page for an individual XPRT benchmark. Go the page of the benchmark that interests you, and look for the blue View Results button. Clicking it takes you to a page that displays results for only that benchmark. You can use the free-form filter on the page to filter those results, and can use the Benchmarks drop-down menu to jump to the other individual XPRT results pages.
The third way to view
information in our results database is with the WebXPRT 4 results viewer.
The viewer provides an information-packed, interactive environment in which
users can explore data from the curated set of WebXPRT 4 results we’ve
published on our site. To learn more about the viewer’s capabilities and
features, check out this blog post
from March.
We hope you’ll take
some time to browse the information in our results database. We welcome your feedback
about what you’d like to see in the future and suggestions for improvement. Our
database contains the XPRT scores that we’ve gathered, but we publish them as a
resource for you. Let us know
what you think!
Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.
