BenchmarkXPRT Blog banner

Category: Web-based testing

WebXPRT: What would you like to see?

Justin Greene
By Justin Greene
on July 11, 2019

At over 412,000 runs and counting, WebXPRT is our most popular benchmark. From the first release in 2013, it’s been popular with device manufacturers, developers, tech journalists, and consumers because it’s easy to run, it runs on almost anything with a web browser, and it evaluates device performance using the types of web-based tasks that people are likely to encounter on a daily basis.

With each new version of WebXPRT, we analyze browser development trends to make sure the test’s underlying web technologies and workload scenarios adequately reflect the ways people are using their browsers to work and play. BenchmarkXPRT Development Community members can play an important part in that process by sending us feedback on existing tests and suggestions for new workloads to include.

For example, when we released WebXPRT 3, we updated the photo workloads with new images and a deep learning task used for image classification. We also added an optical character recognition task in the Encrypt Notes and OCR scan workload, and combined part of the DNA Sequence Analysis scenario with a writing sample/spell check scenario to simulate online homework in an all-new Online Homework workload.

Consider for a moment what an ideal future version of WebXPRT would look like for you. Are there new web technologies or workload scenarios that you would like to see? Would you be interested in an associated battery life test? Should we include experimental tests? We’re interested in what you have to say, so please feel free to contact us with your thoughts or questions.

If you’re just now learning about WebXPRT, we offer several resources to help you better understand the benchmark and its range of uses. For a general overview of why WebXPRT matters, watch our video titled What is WebXPRT and why should I care? To read more about the details of the benchmark’s development and structure, check out the Exploring WebXPRT 3 white paper. To see WebXPRT 2015 and WebXPRT 3 scores from a wide range of processors, visit the WebXPRT 3 Processor Comparison Chart.

We look forward to hearing from you!

Justin

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Collaborative benchmark development, Cross-platform benchmarks, Future of performance evaluation, JavaScript, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2013, WebXPRT 2015, WebXPRT 3, WebXPRT Processor Comparison Chart, What makes a good benchmark?, White papers | Tagged , , , , , |

WebXPRT in action

Justin Greene
By Justin Greene
on November 1, 2018

Just this past summer, WebXPRT passed the 250,000-run milestone, and since then, the run total has already passed 330,000. September was our biggest month ever, with over 28,000 WebXPRT runs! We sometimes like to show the community how far a reach the XPRTs have around the world by reporting the latest stats on the number of articles and reviews that mention the XPRTs, and the fact is that most of those mentions involve WebXPRT. Today, I thought it would be interesting to bring the numbers to life and provide a glimpse of how the tech press uses WebXPRT. Here’s a sample of WebXPRT in action during the past couple of weeks.



While WebXPRT continues to be a useful tool for tech enthusiasts around the world, you don’t have to be a tech expert to benefit from it. If you’d like to know more about WebXPRT, check out our recent video, What is WebXPRT and why should I care?

Justin

Posted in Browser-based benchmarks, Cross-platform benchmarks, Web-based testing, WebXPRT, WebXPRT 2015, WebXPRT 3 | Tagged , , , , , , , , |

Check out our new WebXPRT video!

Justin Greene
By Justin Greene
on September 20, 2018

At over 305,000 runs and counting, WebXPRT is our most popular benchmark app. Device manufacturers, tech journalists, and developers around the world use WebXPRT because test runs are quick and easy, it runs on almost anything with a web browser, and it provides reliable data about how well devices perform when completing real-world tasks.

WebXPRT is not just for “techies,” however. To help explain what WebXPRT does and why it matters to everyday consumers, we’ve published a new video, What is WebXPRT and why should I care? The video explains the concepts behind some of WebXPRT’s workloads and how even small delays in common online tasks can add up to big headaches and a significant amount of wasted time. We all want to avoid those problems, and WebXPRT can help anyone that wants to see how their device, or a new device they’re thinking about buying, stacks up against the alternatives. We encourage you to check out the video below, which you can also find on YouTube and WebXPRT.com. If you have any questions about WebXPRT, please let us know!

Justin

What is WebXPRT and why should I care?

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Cross-platform benchmarks, Mobile devices, Performance benchmarking, Video, Web-based testing, WebXPRT, WebXPRT 3 |

Which browser is the fastest? It’s complicated.

Justin Greene
By Justin Greene
on July 5, 2018

PCWorld recently published the results of a head-to-head browser performance comparison between Google Chrome, Microsoft Edge, Mozilla Firefox, and Opera. As we’ve noted about similar comparisons, no single browser was the fastest in every test. Browser speed sounds like a straightforward metric, but the reality is complex.

For the comparison, PCWorld used three JavaScript-centric test suites (JetStream, SunSpider, and Octane), one benchmark that simulates user actions (Speedometer), a few in-house tests of their own design, and one benchmark that simulates real-world web applications (WebXPRT). Edge came out on top in JetStream and SunSpider, Opera won in Octane and WebXPRT, and Chrome had the best results in Speedometer and PCWorld’s custom workloads.

The reason that the benchmarks rank the browsers so differently is that each one has a unique emphasis and tests a specific set of workloads and technologies. Some focus on very low-level JavaScript tasks, some test additional technologies such as HTML5, and some are designed to identify strengths or weakness by stressing devices in unusual ways. These approaches are all valid, and it’s important to understand exactly what a given score represents. Some scores reflect a very broad set of metrics, while others assess a very narrow set of tasks. Some scores help you to understand the performance you can expect from a device in your everyday life, and others measure performance in scenarios that you’re unlikely to encounter. For example, when Eric discussed a similar topic in the past, he said the tests in JetStream 1.1 provided information that “can be very useful for engineers and developers, but may not be as meaningful to the typical user.”

As we do with all the XPRTs, we designed WebXPRT to test how devices handle the types of real-world tasks consumers perform every day. While lab techs, manufacturers, and tech journalists can all glean detailed data from WebXPRT, the test’s real-world focus means that the overall score is relevant to the average consumer. Simply put, a device with a higher WebXPRT score is probably going to feel faster to you during daily use than one with a lower score. In today’s crowded tech marketplace, that piece of information provides a great deal of value to many people.

What are your thoughts on browser testing? We’d love to hear from you.

Justin

Posted in Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Collaborative benchmark development, Cross-platform benchmarks, HTML5, JavaScript, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2015, WebXPRT 3, What makes a good benchmark? |

The WebXPRT 3 results calculation white paper is now available

Justin Greene
By Justin Greene
on May 31, 2018

As we’ve discussed in prior blog posts, transparency is a core value of our open development community. A key part of being transparent is explaining how we design our benchmarks, why we make certain development decisions, and how the benchmarks actually work. This week, to help WebXPRT 3 testers understand how the benchmark calculates results, we published the WebXPRT 3 results calculation and confidence interval white paper.

The white paper explains what the WebXPRT 3 confidence interval is, how it differs from typical benchmark variability, and how the benchmark calculates the individual workload scenario and overall scores. The paper also provides an overview of the statistical techniques WebXPRT uses to translate raw times into scores.

To supplement the white paper’s overview of the results calculation process, we’ve also published a spreadsheet that shows the raw data from a sample test run and reproduces the calculations WebXPRT uses.

The paper and spreadsheet are both available on WebXPRT.com and on our XPRT white papers page. If you have any questions about the WebXPRT results calculation process, please let us know, and be sure to check out our other XPRT white papers.

Justin

Posted in Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 3, What makes a good benchmark? | Tagged , , , |

The WebXPRT 3 source code is now available

Justin Greene
By Justin Greene
on April 26, 2018

We’re excited to announce that the WebXPRT 3 source code is now available to BenchmarkXPRT Development Community members!

Download the WebXPRT 3 source here (login required).

We’ve also posted a download link on the WebXPRT tab in the Members’ Area. The source code package contains instructions for setting up and configuring a local instance of WebXPRT for those who wish to do so.

As part of our community model for software development, source code for each of the XPRTs is available to anyone who joins the community. If you’d like to review XPRT source code, but haven’t yet joined the community, we encourage you to join! Registration is quick and easy, and if you work for a company or organization with an interest in benchmarking, you can join the community for free. Simply fill out the form with your company e-mail address and select the option to be considered for a free membership. We’ll contact you to verify the address and then activate your membership.

If you have any other questions about community membership or XPRT source code, feel free to contact us. We look forward to hearing from you!

Justin

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Open development, Source code, Web-based testing, WebXPRT, WebXPRT 3 |

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x