Category: WebXPRT 3

Passing two important WebXPRT milestones

on November 15, 2023

Over the past few months, we’ve been excited to see a substantial increase in the total number of completed WebXPRT runs. To put the increase in perspective, we had more total WebXPRT runs last month alone (40,453) than we had in the first two years WebXPRT was available (36,674)! This boost has helped us to reach two important milestones as we close in on the end of 2023.

The first milestone is that the number of WebXPRT 4 runs per month now exceeds the number of WebXPRT 3 runs per month. When we release a new version of an XPRT benchmark, it can take a while for users to transition from using the older version. For OEM labs and tech journalists, adding a new benchmark to their testing suite often involves a significant investment in back testing and gathering enough test data for meaningful comparisons. When the older version of the benchmark has been very successful, adoption of the new version can take longer. WebXPRT 3 has been remarkably popular around the world, so we’re excited to see WebXPRT 4 gain traction and take the lead even as the total number of WebXPRT runs increases each month. The chart below shows the number of WebXPRT runs per month for each version of WebXPRT over the past ten years. WebXPRT 4 usage first surpassed WebXPRT 3 in August of this year, and after looking at data for the last three months, we think its lead is here to stay.

The second important milestone is the cumulative number of WebXPRT runs, which recently passed 1.25 million, as the chart below shows. For us, this moment represents more than a numerical milestone. For a benchmark to succeed, developers need the trust and support of the benchmarking community. WebXPRT’s consistent year-over-year growth tells us that the benchmark continues to hold value for manufacturers, OEM labs, the tech press, and end users. We see it as a sign of trust that folks repeatedly return to the benchmark for reliable performance metrics. We’re grateful for that trust, and for everyone that has contributed to the WebXPRT development process over the years.

We look forward to seeing how far WebXPRT’s reach can extend in 2024! If you have any questions or comments about using WebXPRT, let us know!

Justin

Posted in benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, Browser-based benchmarks, Cross-platform benchmarks, History of benchmarking, Performance benchmarking, results, WebXPRT, WebXPRT 3, WebXPRT 4 | Tagged benchmark, BenchmarkXPRT, browser benchmark, cross-platform, results, WebXPRT, WebXPRT 3, WebXPRT 4 |

Investigating a possible issue with WebXPRT 4 in iOS 17

By Justin Greene

on September 14, 2023

Yesterday, Apple revealed the iPhone 15 and iPhone 15 Pro at its annual fall event, along with a new version of the iOS mobile operating system (iOS 17). The official iOS 17 launch will take place on September 18^th, but before then, users of newer iPhones can install the OS via the Apple Beta Software Program.

Today, a tech journalist informed us that during their testing of iPhone 15 and iPhone 15 Pro with iOS 17 Beta models, WebXPRT 4 has been freezing while running the Encrypt Notes and OCR Scan workload in the Safari 17 browser. Here in the lab, we were able to immediately replicate the issue on an iPhone 12 Pro with iOS 17 Beta model.

Our initial troubleshooting confirmed that WebXPRT 3 successfully runs to completion on iOS 17 Beta, so it appears that the problem is specific to WebXPRT 4. We also confirmed that WebXPRT 4 freezes at the same place when running in the Google Chrome browser on iOS 17 Beta, so we know that the problem does not occur only in Safari.

We’re currently investigating the issue, and will publish our findings here in the blog as soon as we feel confident that we’ve identified both the root cause and a workable solution, if a solution is necessary. One reason a solution would not be necessary is that the issue is a bug on the iOS 17 Beta side that Apple will resolve before the official launch.

We apologize for any inconvenience this issue might cause for tech reviewers and iPhone users, and we appreciate your patience while we figure out what’s going on. If you have any questions about WebXPRT 4, please don’t hesitate to ask!

Justin

Posted in Apple, benchmark, browser performance, Browser-based benchmarks, Cross-platform benchmarks, Google Chrome, iOS, iPhone, Performance benchmarking, Phones, Safari, WebXPRT, WebXPRT 3, WebXPRT 4 | Tagged Apple, browser benchmark, browsers, Chrome, iOS, iOS 17, iPhone, iPhone 15, iPhone 15 Pro, Safari, WebXPRT, WebXPRT 3, WebXPRT 4 |

WebXPRT’s mirror host site in Singapore

By Justin Greene

on May 25, 2023

If you’ve ever spent time exploring WebXPRT.com, you may have noticed a line that says, “If you are in East Asia, you can run WebXPRT from our Singapore host,” followed by a hyperlink with Simplified Chinese characters. We realize that some people may not know why we have a WebXPRT mirror host site in Singapore—or how to use it—so today’s post will cover the basics.

When we first released WebXPRT 2013, some users in mainland China reported slow download times when running the benchmark. These slowdowns affected initial page and workload content load times, but not workload execution, which happens locally. As a result, subtest and overall scores were still consistent with expectations for the devices under test, but it took longer than normal for test runs to complete. In response, we set up a mirror host site in Singapore to facilitate WebXPRT testing in China and other East Asian countries. We continued this practice with subsequent WebXPRT versions, and currently offer Singapore-based instances of WebXPRT 4, WebXPRT 3, and WebXPRT 2015.

The link to WebXPRT 4 Singapore on WebXPRT.com

The default UI language on the Singapore site is Simplified Chinese, but users can opt to change the language to English or German. Apart from a different default language, the WebXPRT mirror instances hosted in Singapore are identical to the instances on the main WebXPRT site. If you test a device on WebXPRT Singapore and WebXPRT.com, you should see similar performance scores from both sites.

The start page for WebXPRT 4 Singapore, with the default Simplified Chinese UI

We hope that the WebXPRT mirror host site in Singapore will make it easier for people in East Asia to use the benchmark. Do you find the site useful? If so, we’d love to hear from you! Also, if you encounter any unexpected issues or interruptions while testing, please let us know!

Justin

Posted in Automation, benchmark, Benchmarking, Browser-based benchmarks, Cross-platform benchmarks, Performance benchmarking, Simplified Chinese, WebXPRT, WebXPRT 2013, WebXPRT 2015, WebXPRT 3, WebXPRT 4 | Tagged browser benchmark, browser performance, Simplified Chinese, Singapore, WebXPRT, WebXPRT 2013, WebXPRT 2015, WebXPRT 3, WebXPRT 4 |

WebXPRT runs: A decade of growth

By Justin Greene

on April 13, 2023

In our last blog post, we celebrated the 10-year anniversary of the WebXPRT launch by looking back on the WebXPRT team’s accomplishments over the last decade. The incremental steps and milestone improvements we discussed all contributed to carving out a lasting place for WebXPRT in the benchmarking world and helped to grow its reputation for being a reliable, effective, and easy to use measurement tool.

WebXPRT’s growth is most evident when we look at the rising number of completed test runs over the last 10 years. Since the first WebXPRT launch in 2013, we’ve seen a steady increase in the number of tests people are running. To put the increase in perspective, we had more runs last month alone (17,300) than we recorded in the first 10 months that WebXPRT was available (11,984).

That growth has helped us to reach and surpass the million-run mark, but the most exciting aspect of seeing a consistent increase in WebXPRT testing is the knowledge that the benchmark is proving to be useful to more people in more places around the world. In our next blog post, we’ll discuss WebXPRT’s truly global reach and some of the surprising cities and countries where people have been using it to test their gear.

We’re grateful for all the testers that have helped WebXPRT grow during the last decade. If you have any questions or comments about using WebXPRT, let us know!

Justin

Posted in benchmark, Benchmarking, BenchmarkXPRT, Browser-based benchmarks, Cross-platform benchmarks, History of benchmarking, Performance benchmarking, WebXPRT, WebXPRT 2013, WebXPRT 2015, WebXPRT 3, WebXPRT 4 | Tagged benchmark, BenchmarkXPRT, WebXPRT, WebXPRT 2013, WebXPRT 2015, WebXPRT 3, WebXPRT 4 |

Celebrating 10 years of WebXPRT!

By Justin Greene

on March 30, 2023

We’re excited to announce that it’s been 10 years since the initial launch of WebXPRT! In early 2013, we introduced WebXPRT as a unique browser performance benchmark in a market space that was already crowded with a variety of specialized measurement tools. Our goal was to offer a benchmark that could compare the performance of almost any web-enabled device, using scenarios created to mirror real-world tasks. We wanted it to be a free, easily accessible, easy-to-run, useful, and appealing testing option for OEM labs, vendors, and the tech press.

When we look back on the last 10 years of WebXPRT, we can’t help but conclude that our efforts have been successful. Since those early days, the WebXPRT market presence has grown from humble beginnings into a worldwide industry standard. Hundreds of tech press publications have used WebXPRT in thousands of articles and reviews, and testers have now run the benchmark well over 1.1 million times.

Below, I’ve listed some of the WebXPRT team’s accomplishments over the last decade. If you’ve been following WebXPRT from the beginning, this may all be familiar, but if you’re new to the community, it may be interesting to see some of the steps that contributed to making WebXPRT what it is today.

We published four versions of the benchmark: WebXPRT 2013, WebXPRT 2015, WebXPRT 3, and WebXPRT 4.
We added subtests and updated existing workloads along the way to measure browser performance with newer HTML5, JavaScript, and WebAssembly technologies.
We added Simplified Chinese and German language options to the WebXPRT UI.
We added a WebXPRT host server in Singapore for testers in East Asia.
We published six WebXPRT white papers, along with three results calculation spreadsheets.
We published the WebXPRT: See how well your device handles common web tasks and What is WebXPRT and why should I care? videos.
We developed the WebXPRT 3 Processor Comparison Chart.
We developed the WebXPRT 4 results viewer tool.

In future blog posts, we’ll look at how the number of WebXPRT runs has grown over time, and how WebXPRT use has grown among OEMs, vendors, and the tech press worldwide. Do you have any thoughts that you’d like to share from your WebXPRT testing experience? If so, let us know!

Justin

Posted in benchmark, Benchmarking, BenchmarkXPRT, Browser-based benchmarks, Cross-platform benchmarks, German, History of benchmarking, JavaScript, Performance benchmarking, results, Simplified Chinese, tech press, WebAssembly, WebXPRT, WebXPRT 2013, WebXPRT 2015, WebXPRT 3, WebXPRT 4, WebXPRT Processor Comparison Chart, White papers | Tagged browser, browser benchmark, browser performance, German, HTML5, JavaScript, results, Simplified Chinese, WebAssembly, WebXPRT, WebXPRT 2015, WebXPRT 3, WebXPRT 4, white paper |

Looking forward to an important WebXPRT milestone

By Justin Greene

on December 1, 2022

February 28, 2013 was a momentous day for the BenchmarkXPRT Development Community. On that day, we published a press release announcing the official launch of the first version of the WebXPRT benchmark, WebXPRT 2013. As difficult as it is for us to believe, the 10-year anniversary of the initial WebXPRT launch is in just a few short months!

We introduced WebXPRT as a truly unique browser performance benchmark in a field that was already crowded with a variety of measurement tools. Since those early days, the WebXPRT market presence has grown from a small foothold into a worldwide industry standard. Over the years, hundreds of tech press publications have used WebXPRT in thousands of articles and reviews, and the WebXPRT completed-runs counter rolled over the 1,000,000-run mark.

New web technologies are continually changing the way we use the web, and browser-performance benchmarks should evaluate how well new devices handle the web of today, not the web of several years ago. While some organizations have stopped development for other browser performance benchmarks, we’ve had the opportunity to continue updating and refining WebXPRT. We can look back at each of the four major iterations of the benchmark—WebXPRT 2013, WebXPRT 2015, WebXPRT 3, and WebXPRT 4—and see a consistent philosophy and shared technical lineage contributing to a product that has steadily improved.

As we get closer to the 10-year anniversary of WebXPRT next year, we’ll be sharing more insights about its reach and impact on the industry, discussing possible future plans for the benchmark, and announcing some fun anniversary-related opportunities for WebXPRT users. We think 2023 will be the best year yet for WebXPRT!

Justin

Posted in benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Collaborative benchmark development, Cross-platform benchmarks, Future of performance evaluation, History of benchmarking, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2013, WebXPRT 2015, WebXPRT 3, WebXPRT 4 | Tagged benchmark, BenchmarkXPRT, BenchmarkXPRT Development Community, browser benchmark, browser performance, cross-platform, WebXPRT, WebXPRT 3, WebXPRT 4 |

Category: WebXPRT 3

Passing two important WebXPRT milestones

Investigating a possible issue with WebXPRT 4 in iOS 17

WebXPRT’s mirror host site in Singapore

WebXPRT runs: A decade of growth

Celebrating 10 years of WebXPRT!

Looking forward to an important WebXPRT milestone

Check out the other XPRTs: