Last week, we provided readers with an overview of what to expect in the WebXPRT 4 Preview, as well as an update on the Preview’s release schedule. Since then, we’ve been working on UI adjustments and bug fixes, additional technical tweaks, and follow-up testing. We’re very close, but won’t be able to meet our original goal of publishing the Preview today. We believe it will be ready for release early next week.
As a reminder, once we release the WebXPRT 4 Preview, testers will be able to publish scores from Preview build testing. We will limit any changes that we make between the Preview and the final release to the UI or to features we do not expect to affect test scores.
If you have any questions about WebXPRT 4 or the Preview build, please let us know!
A few months ago, we shared detailed information about the changes we expected
to make in WebXPRT 4. We are currently doing internal testing of the WebXPRT 4 Preview
build in preparation for releasing it to the public. We want to let our readers
know what to expect.
We’ve made some changes since our
last update and some of the details we present below could still change before
the preview release. However, we are much closer to the final product. Once we
release the WebXPRT 4 Preview, testers will be able to publish scores from Preview
build testing. We will limit any changes that we make between the Preview and
the final release to the UI or features that are not expected to affect test
scores.
General changes
Some of the non-workload changes we’ve
made in WebXPRT 4 relate to our typical benchmark update process.
We have updated the aesthetics of the WebXPRT UI to make WebXPRT 4 visually distinct from older versions. We did not significantly change the flow of the UI.
We have updated content in some of the workloads to reflect changes in everyday technology, such as upgrading most of the photos in the photo processing workloads to higher resolutions.
We have not yet added a looping function to the automation scripts, but are still considering it for the future.
We investigated the possibility of shortening the benchmark by reducing the default number of iterations from seven to five, but have decided to stick with seven iterations to ensure that score variability remains acceptable across all platforms.
Workload changes
Photo
Enhancement. We increased the efficiency of the
workload’s Canvas object creation function, and replaced the existing photos
with new, higher-resolution photos.
Organize Album Using AI. We replaced ConvNetJS with WebAssembly (WASM) based OpenCV.js for both the face detection and image classification tasks. We changed the images for the image classification tasks to images from the ImageNet dataset.
Stock Option Pricing. We updated the dygraph.js library.
Sales Graphs. We made no changes to this workload.
Encrypt Notes and OCR Scan. We replaced ASM.js with WASM for the Notes task and updated the WASM-based Tesseract version for the OCR task.
Online Homework. In addition to the existing scenario which uses four Web Workers, we have added a scenario with two Web Workers. The workload now covers a wider range of Web Worker performance, and we calculate the score by using the combined run time of both scenarios. We also updated the typo.js library.
Experimental workloads
As part of the WebXPRT 4 development
process, we researched the possibility of including two new workloads: a
natural language processing (NLP) workload, and an Angular-based message
scrolling workload. After much testing and discussion, we have decided to not
include these two workloads in WebXPRT 4. They will be good candidates for us
to add as experimental WebXPRT 4 workloads in 2022.
The release timeline
Our goal is to publish the WebXPRT 4
preview build by December 15th, which will allow testers to publish
scores in the weeks leading up to the Consumer Electronics Show in Las Vegas in
January 2022. We will provide more detailed information about the GA timeline
here in the blog as soon as possible.
If you have any questions about the details we’ve shared above, please feel free to ask!
As the WebXPRT 4 development process has progressed, we’ve started to discuss the possibility of offering experimental WebXPRT 4 workloads in 2022. These would be optional workloads that test cutting-edge browser technologies or new use cases. The individual scores for the experimental workloads would stand alone, and would not factor in the WebXPRT 4 overall score.
WebXPRT testers would be able to run the experimental workloads one of two ways: by manually selecting them on the benchmark’s home screen, or by adjusting a value in the WebXPRT 4 automation scripts.
Testers would benefit from experimental workloads by being able to compare how well certain browsers or systems handle new tasks (e.g., new web apps or AI capabilities). We would benefit from fielding workloads for large-scale testing and user feedback before we commit to including them as core WebXPRT workloads.
Do you have any general thoughts about experimental workloads for browser performance testing, or any specific workloads that you’d like us to consider? Please let us know.
The holiday season is
fast approaching, and with widespread product shortages and supply chain
interruptions in the tech industry, it’s wise to start your holiday shopping
now. If you’re considering phones, tablets, Chromebooks, or laptops as gifts,
but are unsure where to get reliable information about the devices, the XPRTs can
help!
One of the core
functions of the XPRTs is to cut through the marketing noise by providing
objective, reliable measures of a device’s performance. For example, instead of
trying to guess whether a new Chromebook is fast enough to handle the demands
of remote learning, you can use its CrXPRT and WebXPRT performance scores to see how it stacks up against the
competition on everyday tasks.
A good place to start looking
for device scores is our XPRT results browser, which lets you access our database of more than 2,800 test
results from over 110 sources, including major tech review publications around
the world, OEMs, and independent testers. You can find a wealth of current and
historical performance data across all the XPRT benchmarks and hundreds of
devices. Learn how to use the results browser here.
If you’re considering
a popular device, chances are good that a recent tech review includes an XPRT
score for that device. You can find these reviews by going to your favorite
tech review site and searching for “XPRT,” or entering the name of the device
and the appropriate XPRT (e.g., “Apple iPad” and “WebXPRT”) in a search engine.
Here are a few recent tech reviews that use one or more of the XPRTs to
evaluate popular devices:
The XPRTs can help
consumers make better-informed and more confident tech purchases this holiday
season, and we hope you’ll find the data you need on our site or in an
XPRT-related tech review. If you have any questions about the XPRTs, XPRT
scores, or the results database please feel free to ask!
People choose a default web browser based on several factors.
Speed is sometimes the deciding factor, but privacy settings, memory load,
ecosystem integration, and web app capabilities can also come into play.
Regardless of the motivations behind a person’s go-to browser choice, the
dominance of software-as-a-service (SaaS) computing means that new updates are
always right around the corner. In previous blog posts, we’ve talked about how browser speed can increase
or decrease significantly after an update, only to swing back in the other
direction shortly thereafter. OS-specific optimizations can also affect
performance, such as with Microsoft Edge on Windows and Google Chrome on Chrome
OS.
Windows 11 began rolling out earlier this month, and tech press outlets
such as AnandTech and PCWorld have used WebXPRT
3 to evaluate the impact of the new OS—or
specific settings in the OS—on browser performance. Our own in-house tests, which
we discuss below, show a negligible impact on browser performance when updating
our test system from Windows 10 to Windows 11. It’s important to note that depending
on a system’s hardware setup, the impact might be more significant in certain
scenarios. For more information about such scenarios, we encourage you to read the
PCWorld article discussing the impact of the Windows 11 default virtualization-based
security (VBS) settings on
browser performance in some instances.
In our comparison tests, we used a Dell
XPS 13 7930 with an Intel
Core i3-10110U processor and 4 GB of RAM. For the Windows 10 tests, we used a
clean Windows 10 Home image updated to version 20H2 (19042.1165). For the
Windows 11 tests, we updated the system to Windows 11 Home version 21H2 (22000.282).
On each OS version, we ran WebXPRT 3 three times on the latest versions of five
browsers: Brave, Google Chrome, Microsoft Edge, Mozilla Firefox, and Opera. For
each browser, the score we post below is the median of the three test runs.
In our last
round of tests on Windows 10, Firefox was the clear winner. Three of the
Chromium-based browsers (Chrome, Edge, and Opera) produced very close scores,
and the performance of Brave lagged by about 7 percent. In this round of
Windows 10 testing, performance on every browser improved slightly, with Google
Chrome taking a slight lead over Firefox.
In our Windows 11 testing, we were interested to find that without exception, browser scores were slightly lower than in Windows 10 testing. However, none of the decreases were statistically significant. Most users performing daily tasks are unlikely to notice that degree of difference.
Have you observed any significant differences in WebXPRT 3 scores
after upgrading to Windows 11? If so, let us know!
Last month, we discussed a potential fix for the error that was preventing CrXPRT 2 testers from successfully completing battery life tests on systems running Chrome v89.x and later. Since then, we’ve been testing an updated, unpublished version of the app package across several Chromebook models to ensure that the new build is stable and produces consistent results. We’re happy to report that our testing was successful, and we’ve published the new CrXPRT build (v1.2.0.0) in the Chrome Web Store and it is live as of 12:45 PM EDT today.
Note
that it might take some time for the update to appear on your Chromebook and,
once it does, you might have to manually approve the update notice.
Neither
the tests nor the method of calculating the overall score and battery-life
score in this new build have changed, so results are comparable with previous
CrXPRT 2 results.
We appreciate everyone’s patience while we found a solution to the error. If you have any questions or comments about the CrXPRT 2 battery life test, please feel free to contact us!
Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.
