BenchmarkXPRT Blog banner

Category: Google

Shop confidently this holiday season with the XPRTs!

Justin Greene
By Justin Greene
on November 14, 2024

The holiday shopping season is upon us, and trying to find the right tech gift for your friends or loved ones (or yourself!) can be a daunting task. If you’re considering new phones, tablets, Chromebooks, laptops, or desktops as gifts this year—and are unsure where to get reliable device information—the XPRTs can help!

The XPRTs provide industry-trusted and time-tested measures of a device’s performance that can help you cut through the fog of competing marketing claims. For example, instead of guessing whether the performance of a new gaming laptop justifies its price, you can use its WebXPRT performance score to see how it stacks up against both older models and competitors while tackling everyday tasks.

A great place to start looking for device scores is our XPRT results browser, which lets you access our database of more than 3,700 test results—across all the XPRT benchmarks and hundreds of devices—from over 155 sources, including major tech review publications around the world, OEMs, our own Principled Technologies (PT) testing, and independent submissions. For tips on how to use the XPRT results browser, check out this blog post.

Another way to view information in our results database is by using the WebXPRT 4 results viewer. The viewer provides an information-packed, interactive tool that we created to help people explore data from the set of almost 800 WebXPRT 4 results we’ve curated and published to date on our site. You’ll find detailed instructions in this blog post for how to use the WebXPRT 4 results viewer tool.

If you’re considering a popular device, it’s likely that a recent tech press review includes an XPRT score for it. To find those scores, go to your favorite tech review site and search for “XPRT,” or enter the name of the device and the appropriate XPRT (e.g., “iPhone” and “WebXPRT”) in a search engine. Here are a few recent tech reviews that used the XPRTs to evaluate popular devices:

In addition to XPRT-related resources in the tech press, here at PT we frequently publish reports that evaluate the performance of hot new consumer devices, and many of those reports include WebXPRT scores. For example, check out the results from our extensive testing of a Dell Latitude 7450 AI PC or our in-depth evaluation of three new Lenovo ThinkPad and ThinkBook laptops.

The XPRTs can help you make better-informed and more confident tech purchases this holiday season. We hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database, please feel free to ask!

Justin

Posted in Acer, Apple, ASUS, benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, browser performance, Chromebooks, CrXPRT, Dell, Google, holiday shopping, HP, iPhone, Lenovo, PCs, Phones, Samsung, WebXPRT, WebXPRT 4 | Tagged , , , , , , , , , , , , , , |

Web APIs: Possible paths for the AI-focused WebXPRT 4 auxiliary workload

Justin Greene
By Justin Greene
on September 19, 2024

In our last blog post, we discussed one of the major decision points we’re facing as we work on what we hope will be the first new AI-focused WebXPRT 4 auxiliary workload: choosing a Web AI framework. In today’s blog, we’re discussing another significant decision that we need to make for the future workload’s development path: choosing a web API.

Many of you are familiar with the concept of an application programming interface (API). Simply put, APIs implement sets of software rules, tools, and/or protocols that serve as intermediaries that make it possible for different computer programs or components to communicate with each other. APIs simplify many development tasks for programmers and provide standardized ways for applications to share data, functions, and system resources.

Web APIs fulfill the intermediary role of an API—through HTTP-based communication—for web servers (on the server side) or web browsers (on the client side). Client-side web APIs make it possible for browser-based applications to expand browser functionality. They execute the kinds of JavaScript, HTML5, and WebAssembly (Wasm) workloads—among other examples—that support the wide variety of browser extensions many of us use every day. WebXPRT uses those types of browser-based workloads to evaluate system performance. To lay a solid foundation for the first future browser-based AI workload, we need to choose a web API that will be compatible with WebXPRT and the Web AI framework and AI inference workload(s) we ultimately choose.

Currently, there are three main web API paths for running AI inference in a web browser: Web Neural Network (WebNN), Wasm, and WebGPU. These three web technologies are in various stages of development and standardization. Each has different levels of support within the major browsers. Here are basic overviews of each of the three options, as well as a few of our thoughts on the benefits and limitations that each may bring to the table for a future WebXPRT AI workload:

  • WebNN is a JavaScript API that enables developers to directly execute machine learning (ML) tasks on neural networks within web-based applications. WebNN makes it easier to integrate ML models into web apps, and it allows web apps to leverage the power of neural processing units (NPUs). WebNN has a lot going for it. It’s hardware-agnostic and works with various ML frameworks. It’s likely to be a major player in future browser-based inference applications. However, as a web standard, WebNN is still in the development stage and is only available in developer previews for Chromium-based browsers. Full default WebNN support could take a year or more.
  • Wasm is a binary instruction format that works across all modern browsers. Wasm provides a sandboxed environment that operates at near-native speeds and takes advantage of common hardware specs across platforms. Wasm’s capabilities offer web developers a great deal of flexibility for running complex client applications in the browser. Simply put, Wasm can help developers adapt their existing code for additional platforms and browser-based applications without requiring extensive code rewrites. Wasm’s flexibility and cross-platform compatibility is one of the reasons that we’ve already made use of Wasm in two existing WebXPRT 4 workloads that feature AI tasks: Organize Album using AI, and Encrypt Notes and OCR Scan. Wasm can also work together with other web APIs, such as WebGPU.
  • WebGPU enables web-based applications to directly access the graphics rendering and computational capabilities of a system’s GPU. The parallel computational abilities of GPUs make them especially well-suited to efficiently handle some of the demands of AI inference workloads, including image-based GenAI workloads or large language models. Google Chrome and Microsoft Edge currently support WebGPU, and it’s available in Safari through a tech preview.

Right now, we don’t think that WebNN will be fully out of the development phase in time to serve as our go-to web API for a new WebXPRT AI workload. Wasm and/or WebGPU appear to our best options for now. When WebNN is fully baked and available in mainstream browsers, it’s possible that we could port any existing Wasm- or WebGPU-based WebXPRT AI workloads to WebNN, which may open the possibility of cross-platform browser-based NPU performance comparisons.

All that said and as we mentioned in our previous post about Web AI frameworks, we have not made any final decisions about a web API or any aspect of the future workload. We’re still in the early stages of this project. We want your input.

If this discussion has sparked web AI ideas that you think would benefit the process, or if you have feedback you’d like to share, please feel free to contact us!

Justin

Posted in AI, benchmark, Benchmarking, BenchmarkXPRT, browser performance, Browser-based benchmarks, Chrome, Chromium, Collaborative benchmark development, Cross-platform benchmarks, Future of performance evaluation, GenAI, Google, Google Chrome, GPU, HTML5, inference, JavaScript, large language models, Machine learning, Microsoft, Microsoft Edge, NPU, Performance benchmarking, Wasm, Web AI, web API, WebAssembly, WebXPRT, WebXPRT 4 | Tagged , , , , , , , , , , , , , , , , , , |

Web AI frameworks: Possible paths for the AI-focused WebXPRT 4 auxiliary workload

Justin Greene
By Justin Greene
on September 5, 2024

A few months ago, we announced that we’re moving forward with the development of a new auxiliary WebXPRT 4 workload focused on local, browser-side AI technology. Local AI has many potential benefits, and it now seems safe to say that it will be a common fixture of everyday life for many people in the future. As the growth of browser-based inference technology picks up steam, our goal is to equip WebXPRT 4 users with the ability to quickly and reliably evaluate how well devices can handle substantial local inference tasks in the browser.

To reach our goal, we’ll need to make many well-researched and carefully considered decisions along the development path. Throughout the decision-making process, we’ll be balancing our commitment to core XPRT values, such as ease of use and widespread compatibility, with the practical realities of working with rapidly changing emergent technologies. In today’s blog, we’re discussing one of the first decision points that we face—choosing a Web AI framework.

AI frameworks are suites of tools and libraries that serve as building blocks for developers to create new AI-based models and apps or integrate existing AI functions in custom ways. AI frameworks can be commercial, such as OpenAI, or open source, such as Hugging Face, PyTorch, and TensorFlow. Because the XPRTs are available at no cost for users and we publish our source code, open-source frameworks are the right choice for WebXPRT.

Because the new workload will focus on locally powered, browser-based inference tasks, we also need to choose an AI framework that has browser integration capabilities and does not rely on server-side computing. These types of frameworks—called Web AI—use JavaScript (JS) APIs and other web technologies, such as WebAssembly and WebGPU, to run machine learning (ML) tasks on a device’s CPU, GPU, or NPU.

Several emerging Web AI frameworks may provide the compatibility and functionality we need for the future WebXPRT workload. Here are a few that we’re currently researching:

  • ONNX Runtime Web: Microsoft and other partners developed the Open Neural Network Exchange (ONNX) as an open standard for ML models. With available tools, users can convert models from several AI frameworks to ONNX, which can then be used by ONNX Runtime Web. ONNX Runtime Web allows developers to leverage the broad compatibility of ONNX-formatted ML models—including pre-trained vision, language, and GenAI models—in their web applications.
  • Transformers.js: Transformers.js, which uses ONNX Runtime Web, is a JS library that allows users to run AI models from the browser and offline. Transformers.js supports language, computer vision, and audio ML models, among others.
  • MediaPipe: Google developed MediaPipe as a way for developers to adapt TensorFlow-based models for use across many platforms in real-time on-device inference applications such as face detection and gesture recognition. MediaPipe is particularly useful for inference work in images, videos, and live streaming.
  • TensorFlow.js: TensorFlow has been around for a long time, and the TensorFlow ecosystem provides users with a broad variety of models and datasets. TensorFlow is an end-to-end ML solution—training to inference—but with available pre-trained models, developers can focus on inference. TensorFlow.js is an open-source JS library that helps developers integrate TensorFlow with web apps.

We have not made final decisions about a Web AI framework or any aspect of the future workload. We’re still in the research, discussion, and experimentation stages of development, but we want to be transparent with our readers about where we are in the process. In future blog posts, we’ll discuss some of the other major decision points in play.

Most of all, we invite you to join us in these discussions, make recommendations, and give us any other feedback or suggestions you may have, so please feel free to share your thoughts!

Justin

Posted in AI, benchmark, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, browser performance, Browser-based benchmarks, Collaborative benchmark development, computer vision, Cross-platform benchmarks, face detection, Future of performance evaluation, Google, GPU, image classification, inference, JavaScript, Machine learning, MediaPipe, Microsoft, NPU, on-device AI, ONNX Runtime Web, Open Source, OpenAI, Performance benchmarking, PyTorch, TensorFlow, Transformers.js, Video, Web AI, Web-based testing, WebAssembly, WebGPU, WebXPRT, WebXPRT 4 | Tagged , , , , , , , , , , , , , , , , , |

Let the XPRTs be your holiday shopping companion!

Justin Greene
By Justin Greene
on November 9, 2023

The holiday shopping season is right around the corner, and choosing the right tech gift can be a daunting task. If you’re considering new phones, tablets, Chromebooks, laptops, or desktops as gifts this year, and are unsure where to get reliable device information, the XPRTs can help!

The XPRTs provide objective, reliable measures of a device’s performance that can help cut through competing marketing claims. For example, instead of guessing whether the performance of a new phone justifies its price, you can use its WebXPRT performance score to see how it stacks up against both older models and competitors while tackling everyday tasks.

A good place to start looking for device scores is our XPRT results browser, which lets you access our database of more than 3,500 test results from over 150 sources, including major tech review publications around the world, OEMs, and independent testers. You can find a wealth of current and historical performance data across all the XPRT benchmarks and hundreds of devices. Learn how to use the results browser here.

If you’re considering a popular device, chances are good that a recent tech review includes an XPRT score for it. Go to your favorite tech review site and search for “XPRT,” or enter the name of the device and the appropriate XPRT (e.g., “Pixel” and “WebXPRT”) in a search engine. Here are a few recent tech reviews that used the XPRTs to evaluate popular devices:

The XPRTs can help consumers make better-informed and more confident tech purchases this holiday season, and we hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database please feel free to ask!

Justin

Posted in Acer, Apple, ASUS, benchmark, Chromebooks, CrXPRT, Google, HDXPRT, Laptops, Phones, results, Samsung, Tablets, WebXPRT, WebXPRT 4 | Tagged , , , , , , , , , , , , , , , , , |

Support for MobileXPRT 3 will likely end soon

Justin Greene
By Justin Greene
on October 20, 2023

In a past blog post, we discussed our plan to move several older versions of XPRT benchmarks to an XPRT archive page. Some of those legacy XPRTs still function correctly, and testers occasionally use them, but a few no longer work on the latest versions of the operating systems or browsers that we designed them to test. With the archive page, we can prevent potential confusion for new users who visit current XPRT pages, but still provide longtime users with continued access to old tests.

You can find more information about the XPRTs that we’ll be moving to the archive page here, but today, we want to let MobileXPRT users know that there’s a high likelihood that MobileXPRT 3 will be joining the list of archived XPRTs in the very near future. The Google Play Store has notified us that, due to evolving requirements for apps in newer versions of Android, we must update our MobileXPRT 3 app package to target an Android API level within one year of the latest Android release. If we don’t update the app to meet that requirement by November 1, users will no longer be able to access MobileXPRT 3 through the Google Play Store.

Though a small number of labs and reviewers still use MobileXPRT 3 to test phones and tablets around the world, we don’t feel current usage is high enough for us to justify committing resources to an update at this point. We had hoped that even if MobileXPRT 3 became inaccessible via the Google Play Store, it would still be possible to sideload the app for testing on newer Android devices. After experimenting with installation options in the lab, however, we think it’s likely that settings on devices running Android 11 and up will prevent both Google Play and sideload installations after November 1. The situation may change, but right now, we don’t expect any method to work after that date. If you try, you’ll likely encounter a message during the installation process that says, “This app was built for an older version of Android and may not work properly. Try checking for updates, or contact the developer.” If you attempt to continue the installation process after that message appears, the app will crash.

Both Android and Chrome developers know that the respective stores sometimes extend these types of deadlines. We hope that will be the case here, but we have no information that would lead us to anticipate an extension. If there is no extension, we will still make MobileXPRT 3 available for testing on older Android devices, but we will then have to move it to the XPRT archive page.

We’re grateful for everyone who has used MobileXPRT 3 in the past, and we apologize for any convenience this change may cause. If you have any questions or concerns about MobileXPRT 3 access, please let us know! 

Justin

Posted in Android, Android 11, Benchmarking, BenchmarkXPRT, Google, Google Chrome, Google Play Store, Mobile devices, MobileXPRT, MobileXPRT 3, Performance testing on tablets, Phones | Tagged , , , , , , , , |

The XPRTs can help with your holiday shopping!

Justin Greene
By Justin Greene
on November 3, 2022

The holiday shopping season is fast approaching, and choosing the right tech gift can often be a daunting task. If you’re considering phones, tablets, Chromebooks, or laptops as gifts, and are unsure where to get reliable device information, the XPRTs can help!

The XPRTs provide objective, reliable measures of a device’s performance that can help to cut through the marketing noise. For example, instead of guessing whether the performance of a new laptop lives up to its billing, you can use its WebXPRT performance score to see how it stacks up against the competition on everyday tasks.

A good place to start looking for device scores is our XPRT results browser, which lets you access our database of more than 3,200 test results from over 165 sources, including major tech review publications around the world, OEMs, and independent testers. You can find a wealth of current and historical performance data across all the XPRT benchmarks and hundreds of devices. Learn how to use the results browser here.

If you’re considering a popular device, chances are good that a recent tech review includes an XPRT score for it. You can find these reviews by going to your favorite tech review site and searching for “XPRT,” or entering the name of the device and the appropriate XPRT (e.g., “iPhone” and “WebXPRT”) in a search engine. Here are a few recent tech reviews that used the XPRTs to evaluate popular devices:

The XPRTs can help consumers make better-informed and more confident tech purchases this holiday season, and we hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database please feel free to ask!

Justin

Posted in AnandTech, Apple, benchmark, Benchmark metrics, BenchmarkXPRT, Chromebooks, Google, Laptops, PCs, Performance benchmarking, Phones, results, Tablets, WebXPRT | Tagged , , , , , , , , , , , , , , , |

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x