Last month, we
reported the good news that our WebXPRT 4 tests successfully ran to completion on the beta releases of
iOS 17.2, iPadOS 17.2, and macOS Sonoma 14.2 with Safari 17.2. When we tested
with those beta builds, WebXPRT 4 did not encounter the issue of test runs getting stuck on iOS 17.1 while attempting to
complete the receipt scanning task in the Encrypt Notes and OCR Scan subtest.
Unfortunately, during the past several weeks, this fix was only available to
Apple users running beta software through the Apple Developer Program.
We’re happy to report
that Apple has now finalized and published the general releases of iOS 17.2,
iPadOS 17.2, and Safari 17.2. WebXPRT 4 tests running on those platforms should
now complete without any problems.
We do appreciate everyone’s patience as we worked to find a solution to this problem, and we look forward to seeing your WebXPRT 4 scores from all the latest Apple devices! If you have any questions or concerns about WebXPRT 4, or you encounter any additional issues when running the test on any platform, please let us know.
Each month, we send out
a BenchmarkXPRT Development Community newsletter that contains the latest updates
from the XPRT world and provides a summary of the previous month’s XPRT-related
activity, including mentions of the XPRTs in the tech press. More people read
the weekly XPRT blog than receive the monthly newsletter, so we realized that
some blog readers may be unaware of the wide variety of tech outlets that regularly
use or mention the XPRTs.
For today’s blog, we want to give readers a sampling
of the XPRT press mentions we see on a weekly basis. Recent mentions include:
Other outlets that have published articles, ads, or reviews mentioning the XPRTs in the last few months include: 3DNews.ru, AnandTech, Android Authority, BOTech News (Mexico), eTeknix, Enos Tech, Gadgets360, Hardware.info, ITC.ua (Ukraine), Mac Life, PCMag, QQ.com (China), Tech4Gamers, Tech Radar, Tom’s Hardware, and Tweakers.
If you don’t currently receive the monthly
BenchmarkXPRT newsletter, but would like to join the mailing list, please let us know! There is no cost to join, and we will not publish or sell any
of the contact information you provide. We will send only the monthly
newsletter and occasional benchmark-related announcements, such as patch
notifications or news of upcoming benchmark releases.
Over the past few months, we’ve been excited to see a substantial increase in the total number of completed WebXPRT runs. To put the increase in perspective, we had more total WebXPRT runs last month alone (40,453) than we had in the first two years WebXPRT was available (36,674)! This boost has helped us to reach two important milestones as we close in on the end of 2023.
The first milestone is that the number of WebXPRT 4 runs per month now exceeds the number of WebXPRT 3 runs per month. When we release a new version of an XPRT benchmark, it can take a while for users to transition from using the older version. For OEM labs and tech journalists, adding a new benchmark to their testing suite often involves a significant investment in back testing and gathering enough test data for meaningful comparisons. When the older version of the benchmark has been very successful, adoption of the new version can take longer. WebXPRT 3 has been remarkably popular around the world, so we’re excited to see WebXPRT 4 gain traction and take the lead even as the total number of WebXPRT runs increases each month. The chart below shows the number of WebXPRT runs per month for each version of WebXPRT over the past ten years. WebXPRT 4 usage first surpassed WebXPRT 3 in August of this year, and after looking at data for the last three months, we think its lead is here to stay.
The second important milestone is the cumulative number of WebXPRT runs, which recently passed 1.25 million, as the chart below shows. For us, this moment represents more than a numerical milestone. For a benchmark to succeed, developers need the trust and support of the benchmarking community. WebXPRT’s consistent year-over-year growth tells us that the benchmark continues to hold value for manufacturers, OEM labs, the tech press, and end users. We see it as a sign of trust that folks repeatedly return to the benchmark for reliable performance metrics. We’re grateful for that trust, and for everyone that has contributed to the WebXPRT development process over the years.
We look forward to seeing how far
WebXPRT’s reach can extend in 2024! If you have any
questions or comments about using WebXPRT, let us know!
The holiday shopping season
is right around the corner, and choosing the right tech gift can be a daunting
task. If you’re considering new phones, tablets, Chromebooks, laptops, or desktops
as gifts this year, and are unsure where to get reliable device information, the
XPRTs can help!
The XPRTs provide
objective, reliable measures of a device’s performance that can help cut
through competing marketing claims. For example, instead of guessing whether the
performance of a new phone justifies its price, you can use its WebXPRT performance score to see how it stacks up against both older
models and competitors while tackling everyday tasks.
A good place to start looking for device scores is our XPRT results browser, which lets you access our database of more than 3,500 test results from over 150 sources, including major tech review publications around the world, OEMs, and independent testers. You can find a wealth of current and historical performance data across all the XPRT benchmarks and hundreds of devices. Learn how to use the results browser here.
If you’re considering
a popular device, chances are good that a recent tech review includes an XPRT
score for it. Go to your favorite tech review site and search for “XPRT,” or enter
the name of the device and the appropriate XPRT (e.g., “Pixel” and “WebXPRT”)
in a search engine. Here are a few recent tech reviews that used the XPRTs to
evaluate popular devices:
The XPRTs can help
consumers make better-informed and more confident tech purchases this holiday
season, and we hope you’ll find the data you need on our site or in an
XPRT-related tech review. If you have any questions about the XPRTs, XPRT
scores, or the results database please feel free to ask!
Over the past several weeks, we’ve
been working to find a solution to a problem with WebXPRT 4
test failures on Apple devices running iOS 17/17.1, iPadOS 17/17.1, and macOS
Sonoma with Safari 17/17.1. While we put significant effort
into an updated WebXPRT version that would mitigate this issue, we are happy to
report that it now looks like we’ll be able to stick with the current version!
Last Thursday, Apple released the
iOS 17.2 beta for participants in the Apple Developer Program. When we tested
the current version of WebXPRT 4 on iOS 17.2, the tests completed without any
issues. We then successfully completed tests on iPadOS 17.2 and macOS Sonoma
14.2 with Safari 17.2. Now that we have good reasons to believe that the iOS
17.2 release will solve the problem, sticking with the current WebXPRT 4 build
will maximize continuity and minimize disruption for WebXPRT users.
Apple has not yet published a public
release date for iOS/iPad OS/Safari 17.2. Based on past development schedules,
it seems likely that they will release it between mid-November and early
December, but that’s simply our best guess. Until then, users who want to test
WebXPRT 4 on devices running iOS 17/17.1, iPadOS 17/17.1, or macOS Sonoma with
Safari 17/17.1 will need to update those devices to iOS/iPad OS/Safari 17.2 via
the Apple Developer Program.
To help Apple users better navigate
testing until the public 17.2 release, we’ve added a function to the current
WebXPRT 4 start page that will notify users if they need to update their
operating system to test.
We appreciate everyone’s patience as
we worked to find a solution to this problem! If you have any questions or
concerns about WebXPRT 4, please let us know.
In recent blog posts, we discussed
an issue that we encountered when attempting to run WebXPRT 4 on iOS 17 devices.
If you missed those posts, you can find more details about the nature of the
problem here. In
short, the issue is that the Encrypt Notes and OCR scan subtest in WebXPRT 4
gets stuck when the Tesseract.js Optical Character Recognition (OCR) engine
attempts to scan a shopping receipt. We’ve verified that the issue occurs on
devices running iOS 17, iPadOS 17, and macOS Sonoma with Safari 17.
After a good bit of troubleshooting and research to try and identify the cause of the problem, we decided to build an updated version of WebXPRT 4 that uses a newer version of Tesseract for the OCR task. Aside from updating Tesseract in the new build, we aimed to change as little as possible. To try and maximize continuity, we’re still using the original input image for the receipt scanning task, and we decided to stick with using the WASM library instead of a WASM-SIMD library. Aside from a new version of tesseract.js, WebXPRT 4 version number updates, and updated documentation where necessary, all other aspects of WebXPRT 4 will remain the same.
We’re currently
testing a candidate build of this new version on a wide array of devices. The
results so far seem promising, but we want to complete our due diligence and
make sure this is the best approach to solving the problem. We know that OEM
labs and tech reviewers put a lot of time and effort into compiling databases
of results, so we hope to provide a solution that minimizes results disruption
and inconvenience for WebXPRT 4 users. Ideally, folks would be able to
integrate scores from the new build without any questions or confusion about comparability.
We don’t yet have an exact release date for a new WebXPRT 4 build, but we can say that we’re shooting for the end of October. We appreciate everyone’s patience as we work towards the best possible solution. If you have any questions or concerns about an updated version of WebXPRT 4, please let us know.
Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.
