Tag Archives: Intel

Contribute to WebXPRT’s AI capabilities with your NPU-equipped gear

on July 11, 2024

A few weeks ago, we announced that we’re developing a new auxiliary WebXPRT 4 workload focused on local, browser-based AI technology. This is an exciting project for us, and as we work to determine the best approach from the perspective of frameworks, APIs, inference models, and test scenarios, we’re also thinking ahead to the testing process. To best understand how the new workload will impact system performance, we’re going to need to test it on hardware equipped with the latest generation of neural processing units (NPUs).

NPUs are not new, but the technology is advancing rapidly, and a growing number of PC and laptop manufacturers are releasing NPU-equipped systems. Several vendors have announced plans to release systems equipped with all-new NPUs in the latter half of this year. As is often the case with bleeding-edge technology, however, official release dates do not always coincide with widespread availability.

We want to evaluate new AI-focused WebXPRT workloads on the widest possible range of new systems, but getting a wide selection of gear equipped with the latest NPUs may take quite a while through normal channels. For that reason, we’ve decided to ask our readers for help to expedite the process.

If you’re an OEM or vendor representative with access to the latest generation of NPU-equipped gear and want to contribute to WebXPRT’s evolution, consider sending us any PCs, white boxes, laptops, 2-in-1s, or tablets (on loan) that would be suitable for NPU-focused testing. We have decades of experience serving as trusted testers of confidential and pre-release gear, so we’re well-acquainted with concerns about confidentiality that may come into play, and we won’t publish any information about the systems or related test results without your permission.

We will, though, be happy to share with you our test results on your systems, and we’d love to hear any guidance or other feedback from you on this new workload.

We’re open to any suitable gear, but we’re especially interested in AMD Ryzen AI, Apple M4, Intel Lunar Lake and Arrow Lake, and Qualcomm Snapdragon X Elite systems.

If you’re interested in sending us gear for WebXPRT development testing, please contact us. We’ll work out all the necessary details. Thanks in advance for your help!

Justin

Posted in AI, AMD, Apple, Arrow Lake, benchmark, Benchmarking, browser performance, Browser-based benchmarks, Collaborative benchmark development, Cross-platform benchmarks, Future of performance evaluation, Intel, Laptops, Lunar Lake, M4, Machine learning, Mobile devices, NPU, on-device AI, PCs, Performance benchmarking, Qualcomm, Ryzen AI, Snapdragon X Elite, Tablets, Web-based testing, WebXPRT, WebXPRT 4 | Also tagged 2-in-1s, AI, AMD, Apple, Arrow Lake, artificial intelligence, browser performance, desktops, laptops, Lunar Lake, M4, NPU, PCs, Qualcomm, Ryzen, Ryzen AI, Snapdragon, Snapdragon X Elite, tablets, WebXPRT, WebXPRT 4, white box

XPRT mentions in the tech press

By Justin Greene

on April 4, 2024

One of the ways we monitor the effectiveness of the XPRT family of benchmarks is to regularly track XPRT usage and reach in the global tech press. Many tech journalists invest a lot of time and effort into producing thorough device reviews, and relevant and reliable benchmarks such as the XPRTs often serve as indispensable parts of a reviewer’s toolkit. Trust is hard-earned and easily lost in the benchmarking community, so we’re happy when our benchmarks consistently achieve “go-to” status for a growing number of tech assessment professionals around the world.

Because some of our newer readers may be unaware of the wide variety of outlets that regularly use the XPRTs, we occasionally like to share an overview of recent XPRT-related tech press activity. For today’s blog, we want to give readers a sampling of the press mentions we’ve seen over the past few months.

Recent mentions include:

AnandTech used WebXPRT 4 to assess the performance of the ASRock Industrial 4X4 BOX-7840U and GEEKOM A5 mini-PCs.
Android Headlines used CrXPRT 2 to measure the performance of the ASUS Chromebook Plus CX34.
Mashable measured the performance and battery life of the HP Chromebook Plus x360 14c with CrXPRT 2.
Notebookcheck used WebXPRT 4 in dozens of device reviews, including evaluations of the Apple MacBook Air 15 (M3, 2024), the HP Omen 16 (2024), the HP Spectre x360 16, the Lenovo ThinkPad X1 Carbon G12, and the Valve Steam Deck.
PCMag used WebXPRT 4 in a review of the Apple MacBook Air 15 (M3, 2024).
TechPowerUp used WebXPRT 4 in a review of the Intel Core i9-14900KS processor.
Tom’s Guide used WebXPRT 4 to compare the performance of the Snapdragon and Exynos variants of the Samsung Galaxy S24.
Other outlets that have published articles, ads, or reviews mentioning the XPRTs in the last few months include: 3DNews (Russia), Android Authority, Benchlife.info, Delkom (Poland), DigitalWorld Italia, Digitec (Switzerland), Expert Reviews, Galaxus (Germany), Hardware.info, HIPC (Japan), ITC.ua (Ukraine), ITWorld (Korea), iXBT.com (Russia), PCMag, PC-Welt (Germany), QQ.com (China), SMZDM (China), and Tweakers.

Each month, we send out a BenchmarkXPRT Development Community newsletter that contains the latest updates from the XPRT world and provides a summary of the previous month’s XPRT-related activity, including new mentions of the XPRTs in the tech press. If you don’t currently receive the monthly BenchmarkXPRT newsletter but would like to join the mailing list, please let us know! There is no cost to join, and we will not publish or sell any of the contact information you provide. We will send only the monthly newsletter and occasional benchmark-related announcements, such as news about patches or new releases.

Justin

Posted in AnandTech, Apple, ASUS, battery life, benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Chromebooks, CrXPRT, HP, Lenovo, Performance benchmarking, Performance of computing devices, Samsung, tech press, What makes a good benchmark? | Also tagged AnandTech, Android, Apple, ASRock, ASUS, GEEKOM, HP, Lenovo, Mashable, Notebookcheck, PCMag, PCWorld, Samsung, TechPowerUp, Tom's Hardware, WebXPRT, WebXPRT 4

Local AI and new frontiers for performance evaluation

By Justin Greene

on December 19, 2023

Recently, we discussed some ways the PC market may evolve in 2024, and how new Windows on Arm PCs could present the XPRTs with many opportunities for benchmarking. In addition to a potential market shakeup from Arm-based PCs in the coming years, there’s a much broader emerging trend that could eventually revolutionize almost everything about the way we interact with our personal devices—the development of local, dedicated AI processing units for consumer-oriented tech.

AI already impacts daily life for many consumers through technologies such as such as predictive text, computer vision, adaptive workflow apps, voice recognition, smart assistants, and much more. Generative AI-based technologies are rapidly establishing a permanent, society-altering presence across a wide range of industries. Aside from some localized inference tasks that the CPU and/or GPU typically handle, the bulk of the heavy compute power that fuels those technologies has been in the cloud or in on-prem servers. Now, several major chipmakers are working to roll out their own versions of AI-optimized neural processing units (NPUs) that will enable local devices to take on a larger share of the AI load.

Examples of dedicated AI hardware in recently-released or upcoming consumer devices include Intel’s new Meteor Lake NPU, Apple’s Neural Engine for M-series SoCs, Qualcomm’s Hexagon NPU, and AMD’s XDNA 2 architecture. The potential benefits of localized, NPU-facilitated AI are straightforward. On-device AI could reduce power consumption and extend battery life by offloading those tasks from the CPUs. It could alleviate certain cloud-related privacy and security concerns. Without the delays inherent in cloud queries, localized AI could execute inference tasks that operate much closer to real time. NPU-powered devices could fine-tune applications around your habits and preferences, even while offline. You could pull and utilize relevant data from cloud-based datasets without pushing private data in return. Theoretically, your device could know a great deal about you and enhance many areas of your daily life without passing all that data to another party.

Will localized AI play out that way? Some tech companies envision a role for on-device AI that enhances the abilities of existing cloud-based subscription services without decoupling personal data. We’ll likely see a wide variety of capabilities and services on offer, with application-specific and SaaS-determined privacy options.

Regardless of the way on-device AI technology evolves in the coming years, it presents an exciting new frontier for benchmarking. All NPUs will not be created equal, and that’s something buyers will need to understand. Some vendors will optimize their hardware more for computer vision, or large language models, or AI-based graphics rendering, and so on. It won’t be enough for business and consumers to simply know that a new system has dedicated AI processing abilities. They’ll need to know if that system performs well while handling the types of AI-related tasks that they do every day.

Here at the XPRTs, we specialize in creating benchmarks that feature real-world scenarios that mirror the types of tasks that people do in their daily lives. That approach means that when people use XPRT scores to compare device performance, they’re using a metric that can help them make a buying decision that will benefit them every day. We look forward to exploring ways that we can bring XPRT benchmarking expertise to the world of on-device AI.

Do you have ideas for future localized AI workloads? Let us know!

Justin

Posted in AI, AMD, Apple, Arm, battery life, benchmark, Benchmark metrics, Benchmarking, Cloud, computer vision, Cross-platform benchmarks, Data privacy, Future of performance evaluation, graphics, Intel, large language models, Machine learning, Meteor Lake, NPU, on premises, on-device AI, PCs, Qualcomm, SaaS, WebXPRT, WebXPRT 4 | Also tagged AI, AMD, Apple, benchmark, BenchmarkXPRT, computer vision, large language models, local AI, machine learning, Meteor Lake, on-device AI, Qualcomm, rendering, SaaS, WebXPRT, WebXPRT 4

The evolving PC market brings new opportunities for WebXPRT

By Justin Greene

on December 7, 2023

Here at the XPRTs, we have to spend time examining what’s next in the tech industry, because the XPRTs have to keep up with the pace of innovation. In our recent discussions about 2024, a major recurring topic has been the potential impact of Qualcomm’s upcoming line of SOCs designed for Windows on Arm PCs.

Now, Windows on Arm PCs are certainly not new. Since Windows RT launched on the Arm-based Microsoft Surface RT in 2012, various Windows on Arm devices have come and gone, but none of them—except for some Microsoft SQ-based Surface devices—have made much of a name for themselves in the consumer market.

The reasons for these struggles are straightforward. While Arm-based PCs have the potential to offer consumers the benefits of excellent battery life and “always-on” mobile communications, the platform has historically lagged Intel- and AMD-based PCs in performance. Windows on Arm devices have also faced the challenge of a lack of large-scale buy-in from app developers. So, despite the past involvement of device makers like ASUS, HP, Lenovo, and Microsoft, the major theme of the Windows on Arm story has been one of very limited market acceptance.

Next year, though, the theme of that story may change. If it does, WebXPRT 4 is well-positioned to play an important part.

At the recent Qualcomm Technology Summit, the company unveiled the new 4nm Snapdragon X Elite SOC, which includes an all-new 12-core Oryon CPU, an integrated Adreno GPU, and an integrated Hexagon NPU (neural processing unit) designed for AI-powered applications. Company officials presented performance numbers that showed the X Elite surpassing the performance of late-gen AMD, Apple, and Intel competitor platforms, all while using less power.

Those are massive claims, and of course the proof will come—or not—only when systems are available for test. (In the past, companies have made similar claims about Windows on Arm advantages, only to see those claims evaporate by the time production devices show up on store shelves.)

Will Snapdragon X Elite systems demonstrate unprecedented performance and battery life when they hit the market? How will the performance of those devices stack up to Intel’s Meteor Lake systems and Apple’s M3 offerings? We don’t yet know how these new devices may shake up the PC market, but we do know that it looks like 2024 will present us with many golden opportunities for benchmarking. Amid all the marketing buzz, buyers everywhere will want to know about potential trade-offs between price, power, and battery life. Tech reviewers will want to dive into the details and provide useful data points, but many traditional PC benchmarks simply won’t work with Windows on ARM systems. As a go-to, cross-platform favorite of many OEMs—that runs on just about anything with a browser—WebXPRT 4 is in a perfect position to provide reviewers and consumers with relevant performance comparison data.

It’s quite possible that 2024 may be the biggest year for WebXPRT yet!

Justin

Posted in AI, AMD, Apple, Arm, ASUS, battery life, Benchmark metrics, Benchmarking, BenchmarkXPRT, browser performance, Browser-based benchmarks, Cross-platform benchmarks, Future of performance evaluation, Intel, Laptops, M3, Meteor Lake, PCs, Performance benchmarking, Qualcomm, WebXPRT, WebXPRT 4 | Also tagged Adreno, AMD, Apple, Arm, ASUS, benchmark, BenchmarkXPRT, Hexagon, HP, laptops, Lenovo, M3, Meteor Lake, Microsoft, Oryon, PCs, Qualcomm, Snapdragon, SOC, Surface

Using WebXPRT 3 to compare the performance of popular browsers

By Justin Greene

on February 6, 2020

Microsoft recently released a new Chromium-based version of the Edge browser, and several tech press outlets have released reviews and results from head-to-head browser performance comparison tests. Because WebXPRT is a go-to benchmark for evaluating browser performance, PCMag, PCWorld, and VentureBeat, among others, used WebXPRT 3 scores as part of the evaluation criteria for their reviews.

We thought we would try a quick experiment of our own, so we grabbed a recent laptop from our Spotlight testbed: a Dell XPS 13 7930 running Windows 10 Home 1909 (18363.628) with an Intel Core i3-10110U processor and 4 GB of RAM. We tested on a clean system image after installing all current Windows updates, and after the update process completed, we turned off updates to prevent them from interfering with test runs. We ran WebXPRT 3 three times on six browsers: a new browser called Brave, Google Chrome, the legacy version of Microsoft Edge, the new version of Microsoft Edge, Mozilla Firefox, and Opera. The posted score for each browser is the median of the three test runs.

As you can see in the chart below, five of the browsers (legacy Edge, Brave, Opera, Chrome, and new Edge) produced scores that were nearly identical. Mozilla Firefox was the only browser that produced a significantly different score. The parity among Brave, Chrome, Opera, and the new Edge is not that surprising, considering they are all Chromium-based browsers. The rank order and relative scaling of these results is similar to the results published by the tech outlets mentioned above.

Do these results mean that Mozilla Firefox will provide you with a speedier web experience? Generally, a device with a higher WebXPRT score is probably going to feel faster to you during daily use than one with a lower score. For comparisons on the same system, however, the answer depends in part on the types of things you do on the web, how the extensions you’ve installed affect performance, how frequently the browsers issue updates and incorporate new web technologies, and how accurately the browsers’ default installation settings reflect how you would set up the same browsers for your daily workflow.

In addition, browser speed can increase or decrease significantly after an update, only to swing back in the other direction shortly thereafter. OS-specific optimizations can also affect performance, such as with Edge on Windows 10 and Chrome on Chrome OS. All of these variables are important to keep in mind when considering how browser performance comparison results translate to your everyday experience. In such a competitive market, and with so many variables to consider, we’re happy that WebXPRT can help consumers by providing reliable, objective results.

What are your thoughts on today’s competitive browser market? We’d love to hear from you.

Justin

Posted in benchmark, Benchmarking, Browser-based benchmarks, Chrome OS, Cross-platform benchmarks, Google Chrome, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 3 | Also tagged Brave, Chrome, Dell, Firefox, Microsoft Edge, Opera, PCMag, PCWorld, VentureBeat, WebXPRT

How to use alternate configuration files with AIXPRT

By Justin Greene

on October 10, 2019

In last week’s AIXPRT Community Preview 3 announcement, we mentioned the new public GitHub repository that we’re using to publish AIXPRT-related information and resources. In addition to the installation readmes for each AIXPRT installation package, the repository contains a selection of alternative test config files that testers can use to quickly and easily change a test’s parameters.

As we discussed in previous blog entries about batch size, levels of precision, and number of concurrent instances, AIXPRT testers can adjust each of these key variables by editing the JSON file in the AIXPRT/Config directory. While the process is straightforward, editing each of the variables in a config file can take some time, and testers don’t always know the appropriate values for their system. To address both of these issues, we are offering a selection of alternative config files that testers can download and drop into the AIXPRT/Config directory.

In the GitHub repository, we’ve organized the available config files first by operating system (Linux_Ubuntu and Windows) and then by vendor (All, Intel, and NVIDIA). Within each section, testers will find preconfigured JSON files set up for several scenarios, such as running with multiple concurrent instances on a system’s CPU or GPU, running with FP32 precision instead of FP16, etc. The picture below shows the preconfigured files that are currently available for systems running Ubuntu on Intel hardware.

Because potential AIXPRT use cases cut across a wide range of hardware segments, including desktops, edge devices, and servers, not all AIXPRT workloads and configs will be applicable to each segment. As we move towards the AIXPRT GA, we’re working to find the best way to parse out these distinctions and communicate them to end users. In many cases, the ideal combination of test configuration variables remains an open question for ongoing research. However, we hope the alternative configuration files will help by giving testers a starting place.

If you experiment with an alternative test configuration file, please note that it should replace the existing default config file. If more than one config file is present, AIXPRT will run all the configurations and generate a separate result for each. More information about the config files and detailed instructions for how to handle the files are available in the EditConfig.md document in the public repository.

We’ll continue to keep everyone up to date with AIXPRT news here in the blog. If you have any questions or comments, please let us know.

Justin

Posted in AI, AIXPRT, benchmark, Benchmark metrics, Collaborative benchmark development, Community Preview, GitHub, Intel, Linux, Machine learning, NVIDIA, Performance benchmarking, Ubuntu | Also tagged AI, AIXPRT, batch size, concurrent instances, FP16, FP32, GitHub, NVIDIA, precision, Windows

Tag Archives: Intel

Contribute to WebXPRT’s AI capabilities with your NPU-equipped gear

XPRT mentions in the tech press

Local AI and new frontiers for performance evaluation

The evolving PC market brings new opportunities for WebXPRT

Using WebXPRT 3 to compare the performance of popular browsers

How to use alternate configuration files with AIXPRT

Check out the other XPRTs: