Category: Cross-platform benchmarks

New features for the WebXPRT Processor Comparison Chart

on February 1, 2018

We’re excited that the WebXPRT 2015 Processor Comparison Chart is becoming a valuable resource for community members and visitors interested in comparing web-browsing performance scores. When we introduced the chart, we mentioned that the first iteration was only the beginning. We planned to add additional capabilities on a regular basis, and today we’re happy to announce some new features.

The chart now offers a drop-down menu for displaying more than 20 results per page. (You can choose 20, 50, 100, or all.) When you click on a system to view the details of its score, you will see that we added a column displaying browser information. If the “Browser” column entry shows as “Unknown,” this means we can’t find browser information for a specific test result.

We’re also working on adding a drop-down menu for filtering scores by device type (laptop, tablet, etc.).

We hope these features will prove to be helpful. We want the chart to continue to be a great asset for its users, so if you have ideas for features and types of data you’d like to see, let us know!

Justin

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Cross-platform benchmarks, Performance benchmarking |

The WebXPRT 3 Community Preview is here!

By Justin Greene

on December 14, 2017

Today we’re releasing the WebXPRT 3 Community Preview (CP). As we discussed in the blog last month, in the new version of WebXPRT, we updated the photo-related workloads with new images and a new deep learning task for the Organize Album workload. We also added an optical character recognition task to the Local Notes workload and combined a portion of the DNA Sequence Analysis scenario with a writing sample/spell check scenario to simulate an online homework hub in the new “Online Homework” workload.

Also, longtime WebXPRT users will immediately notice a completely new, but clean and straightforward, UI. We’re still tweaking aspects of the UI and implementing full functionality for certain features such as social media sharing and German language translation, but we don’t anticipate making any significant changes to the overall test or individual workloads before the general release.

As with all community previews, the WebXPRT 3 CP is available only to BenchmarkXPRT Development Community members, who can access the link from the WebXPRT tab in the Members’ Area.

After you try the WebXPRT 3 CP, please send us your comments. Thanks and happy testing!

Justin

Posted in BenchmarkXPRT development community, Browser-based benchmarks, Collaborative benchmark development, Community Preview, Cross-platform benchmarks, Future of performance evaluation, German, Performance benchmarking, WebXPRT, WebXPRT 3 |

News about WebXPRT and BatteryXPRT

By Justin Greene

on December 7, 2017

Last month, we gave readers a glimpse of the updates in store for the next WebXPRT, and now we have more news to report on that front.

The new version of WebXPRT will be called WebXPRT 3. WebXPRT 3 will retain the convenient features that made WebXPRT 2013 and WebXPRT 2015 our most popular tools, with more than 200,000 combined runs to date. We’ve added new elements, including AI, to a few of the workloads, but the test will still run in 15 minutes or less in most browsers and produce the same easy-to-understand results that help compare browsing performance across a wide variety of devices.

We’re also very close to publishing the WebXPRT 3 Community Preview. For those unfamiliar with our open development community model, BenchmarkXPRT Development Community members have the ability to preview and test new benchmark tools before we release them to the general public. Community previews are a great way for members to evaluate new XPRTs and send us feedback. If you’re interested in joining, you can register here.

In BatteryXPRT news, we recently started to see unusual battery life estimates and high variance when running battery life tests at the default length of 5.25 hours. We think this may be due to changes in how new OS versions are reporting battery life on certain devices, but we’re in the process of extensive testing to learn more. In the meantime, we recommend that BatteryXPRT users adjust the test run time to allow for a full rundown.

Do you have questions or comments about WebXPRT or BatteryXPRT? Let us know!

Justin

Posted in AI, Battery life, BatteryXPRT 2014 for Android, BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Community Preview, Cross-platform benchmarks, Future of performance evaluation, Performance benchmarking, WebXPRT, WebXPRT 3 |

Glimpses of the next WebXPRT

By Justin Greene

on November 2, 2017

Development work on the next version of WebXPRT is well underway, and we think it’s a good time to offer a glimpse of what’s to come.

We’ve updated the photo-related workloads with new images and are experimenting with adding a new task to the Organize Album workload. The task utilizes ConvNetJS, a JavaScript library designed for training neural networks within the browser itself, to assign classifications to a set of images. It’s an example of the type of integrated deep learning tasks that will be showing up in all sorts of devices in the years to come.

We’re also testing an additional task in the Local Notes workload using Tesseract.js, a popular OCR (optical character recognition) engine. Our scenario uses OCR technology to scan images of purchase receipts and gather relevant information.

We’re testing these new tasks now, and will include them only once we’re confident that they produce consistent and reliable results without extending the benchmark’s runtime unnecessarily. Consequently, the next WebXPRT might contain variations of these tasks, or other new technologies altogether. We mention them now to offer some insight into the types of workload enhancements that we’re considering.

We’ve been working hard on the new WebXPRT UI as well. The image below shows the new start page from an early development build. We’re still making adjustments, so the final product will probably differ, but you do get a sense of the new UI’s clean look.

As we’ve said before, we’re committed to making sure that WebXPRT runs in most browsers and produces results that are useful for comparing web browsing performance across a wide variety of devices. We appreciate the feedback we’ve gotten so far, and are happy to receive more. Do you have ideas for the next WebXPRT? Let us know!

Justin

Posted in Browser-based benchmarks, Cross-platform benchmarks, Future of performance evaluation, Machine learning, Web-based testing, WebXPRT, WebXPRT 2017 |

Introducing the WebXPRT 2015 Processor Comparison Chart

By Justin Greene

on October 5, 2017

Today, we’re excited to announce the WebXPRT 2015 Processor Comparison Chart, a new tool that makes it easier to access hundreds of PT-curated, real-world performance scores from a wide range of devices covering everything from TVs to phones to tablets to PCs.

The chart offers a quick way to browse and compare WebXPRT 2015 results grouped by processor. Unlike benchmark-score charts that may contain results from unknown sources, PT hand-selected each of the results from internal lab testing and reliable tech media sources. If we published multiple scores for an individual processor, the score presented in the chart will be an average of those scores. Users can hover over and click individual score bars for additional information about the test results and test sources for each processor.

We think the WebXPRT Processor Comparison Chart will be a valuable resource for folks interested in performance testing and product evaluation, but the current iteration is only the beginning. We plan to add additional capabilities on a regular basis, such a detailed filtering and enhanced viewing and navigational options. It’s also possible that we may integrate other XPRT benchmarks down the road.

Most importantly, we want the chart to be a great asset for its users, and that’s where you come in. We’d love to hear your feedback on the features and types of data you’d like to see. If you have suggestions, please let us know!

Justin

Posted in Benchmark metrics, Browser-based benchmarks, Cross-platform benchmarks, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2015 |

Taking a detour

By Eric Hale

on October 13, 2016

Back in April, Bill announced that we would be starting development of a cross-platform benchmark. This announcement generated a lot of interest and we got lots of good feedback and ideas.

We knew from the start that getting a cross-platform benchmark right would be hard. However, it proved to be even trickier than we thought. As I explained before, benchmarks not only have to run well, but the results must be fair to all platforms involved. Achieving both of these requirements has been a challenge.

At the same time we’ve been devoting a great deal of effort and resources to the cross-platform benchmark, some increasingly popular new use cases have been receiving less attention than they deserve. We’ve decided that the cross-platform benchmark is not the best use of the Community’s resources, and are going to put it on the shelf for a while. This will free up the resources to let us really dig into some newer technologies.

Thanks again to everyone who responded.

Eric

Posted in Benchmarking, BenchmarkXPRT development community, Collaborative benchmark development, Cross-platform benchmarks |

Category: Cross-platform benchmarks

New features for the WebXPRT Processor Comparison Chart

The WebXPRT 3 Community Preview is here!

News about WebXPRT and BatteryXPRT

Glimpses of the next WebXPRT

Introducing the WebXPRT 2015 Processor Comparison Chart

Taking a detour

Check out the other XPRTs: