BenchmarkXPRT Blog banner

Tag Archives: AIXPRT

A note about AIXPRT

Justin Greene
By Justin Greene
on June 30, 2022

Recently, a member of the tech press asked us about the status of AIXPRT, our benchmark that measures machine learning inference performance. We want to share our answer here in the blog for the benefit of other readers. The writer said it seemed like we had not updated AIXPRT in a long time, and wondered whether we had any immediate plans to do so.

It’s true that we haven’t updated AIXPRT in quite some time. Unfortunately, while a few tech press publications and OEM labs began experimenting with AIXPRT testing, the benchmark never got the traction we hoped for, and we’ve decided to invest our resources elsewhere for the time being. The AIXPRT installation packages are still available for people to use or reference as they wish, but we have not updated the benchmark to work with the latest platform versions (OpenVINO, TensorFlow, etc.). It’s likely that several components in each package are out of date.

If you are interested in AIXPRT and would like us to bring it up to date, please let us know. We can’t promise that we’ll revive the benchmark, but your feedback could be a valuable contribution as we try to gauge the benchmarking community’s interest.

Justin

Posted in AI, AIXPRT, benchmark, Machine learning, OpenVINO, Performance benchmarking, TensorFlow | Also tagged , , , , ,

Following up on XPRT compatibility with Windows 11

Justin Greene
By Justin Greene
on September 16, 2021

Last week, we discussed the upcoming Windows 11 GA launch on October 5, and our hope is that the transition period from Windows 10 to Windows 11 will go smoothly for the three XPRTs that run on Windows 10, HDXPRT 4, TouchXPRT 2016, and AIXPRT. We’re happy to report that so far, we’ve been able to install HDXPRT 4 and TouchXPRT 2016 on the latest stable preview of Windows 11 without any problems. For TouchXPRT 2016, we successfully installed the benchmark using both available methods—directly from the Microsoft Store and through the manual sideload process—and ran it without issues.

We’re still testing Windows 11 compatibility with the AIXPRT OpenVINO, TensorFlow, and TensorRT test packages, and will share our findings here in the blog as soon as possible. Also, because Microsoft might still publish through the stable preview channel Windows 11 changes that interfere with the HDXPRT 4 or TouchXPRT 2016 installation or testing processes, we’ll continue to verify each benchmark’s Windows 11 compatibility up through and beyond launch day.

If you’re conducting your own HDXPRT 4, TouchXPRT 2016, or AIXPRT testing on the Windows 11 beta, you could encounter issues with newly published updates before we do due to the timing of our update cycles. You could also run into problems that are specific to your test gear. In either case, please don’t assume that we already know about the problem. Let us know!

Justin

Posted in Adobe, AIXPRT, Application-based benchmarks, benchmark, Benchmarking, HDXPRT, HDXPRT 4, Performance benchmarking, TouchXPRT, TouchXPRT 2016, Windows 11, Windows Update | Also tagged , , , , ,

Testing XPRT compatibility with Windows 11

Justin Greene
By Justin Greene
on September 9, 2021

Last week, Microsoft announced that the Windows 11 GA build will officially launch Tuesday October 5, earlier than the initial late 2021 estimate. The update will start rolling out with select new laptops and existing Windows 10 PCs that satisfy specific system requirements, and only some Windows 10 PCs will be eligible for the update right away. Through a phased Windows Update process, additional Windows 10 PCs will be able to access the update throughout the first half of 2022.

Between the phased Windows 11 rollout and the pledge Microsoft has made to continue Windows 10 support through October 2025, it will likely be a while before the majority of Windows users transition to the new version. We hope the transition period will go smoothly for the XPRTs. However, because we designed three of our benchmarks to run on Windows 10 (HDXPRT 4, TouchXPRT 2016, and AIXPRT), we might encounter compatibility issues with Windows 11.

Over the coming weeks, we’ll be testing HDXPRT 4, TouchXPRT 2016, and AIXPRT on beta versions of Windows 11, and we’ll test again after the GA launch. In addition to obvious compatibility issues and test failures, we’ll note any changes we need to make to our documentation to account for differences in the Windows 11 installation or test processes.

We hope that testers will be able to successfully use all three benchmarks on both OS versions throughout the transition process. If problems arise, we will keep our blog readers informed while exploring solutions. As always, we’re also open to feedback from the community, so if you are participating in the Windows Insider Program and have encountered Windows 11 beta compatibility issues with any of the Windows-focused XPRTs, please let us know!

Justin

Posted in AIXPRT, benchmark, Benchmarking, BenchmarkXPRT, HDXPRT 4, Performance benchmarking, TouchXPRT, Windows, Windows 10, Windows 11 | Also tagged , , , , , ,

The XPRTs in 2020: a year to remember

Justin Greene
By Justin Greene
on December 17, 2020

As 2020 comes to a close, we want to take this opportunity to review another productive year for the XPRTs. Readers of our newsletter are familiar with the stats and updates we include each month, but for our blog readers who don’t receive the newsletter, we’ve compiled some highlights below.

Benchmarks
In the past year, we released CrXPRT 2 and updated MobileXPRT 3 for testing on Android 11 phones. The biggest XPRT benchmark news was the release of CloudXPRT v1.0 and v1.01. CloudXPRT, our newest  benchmark, can accurately measure the performance of cloud applications deployed on modern infrastructure-as-a-service (IaaS) platforms, whether those platforms are paired with on-premises, private cloud, or public cloud deployments. 

XPRTs in the media
Journalists, advertisers, and analysts referenced the XPRTs thousands of times in 2020, and it’s always rewarding to know that the XPRTs have proven to be useful and reliable assessment tools for technology publications such as AnandTech, ArsTechnica, Computer Base, Gizmodo, HardwareZone, Laptop Mag, Legit Reviews, Notebookcheck, PCMag, PCWorld, Popular Science, TechPowerUp, Tom’s Hardware, VentureBeat, and ZDNet.

Downloads and confirmed runs
So far in 2020, we’ve had more than 24,200 benchmark downloads and 164,600 confirmed runs. Our most popular benchmark, WebXPRT, just passed 675,000 runs since its debut in 2013! WebXPRT continues to be a go-to, industry-standard performance benchmark for OEM labs, vendors, and leading tech press outlets around the globe.

Media, publications, and interactive tools
Part of our mission with the XPRTs is to produce materials that help testers better understand the ins and outs of benchmarking in general and the XPRTs in particular. To help achieve this goal, we’ve published the following in 2020:

We’re thankful for everyone who has used the XPRTs, joined the community, and sent questions and suggestions throughout 2020. This will be our last blog post of the year, but there’s much more to come in 2021. Stay tuned in early January for updates!

Justin

Posted in AIXPRT, AnandTech, Android 11, benchmark, Benchmarking, BenchmarkXPRT, Cloud, CloudXPRT, CrXPRT, IaaS, Interactive tools, MobileXPRT, MobileXPRT 3, Performance benchmarking, Phones, WebXPRT | Also tagged , , , , , , , , ,

The AIXPRT learning tool is now live (and a CloudXPRT version is on the way)!

Justin Greene
By Justin Greene
on December 10, 2020

We’re happy to announce that the AIXPRT learning tool is now live! We designed the tool to serve as an information hub for common AIXPRT topics and questions, and to help tech journalists, OEM lab engineers, and everyone who is interested in AIXPRT find the answers they need in as little time as possible.

The tool features four primary areas of content:

  • The Q&A section provides quick answers to the questions we receive most from testers and the tech press.
  • The AIXPRT: the basics section describes specific topics such as the benchmark’s toolkits, networks, workloads, and hardware and software requirements.
  • The testing and results section covers the testing process, metrics, and how to publish results.
  • The AI/ML primer provides brief, easy-to-understand definitions of key AI and ML terms and concepts for those who want to learn more about the subject.

The first screenshot below shows the home screen. To show how some of the popup information sections appear, the second screenshot shows the Inference tasks (workloads) entry in the AI/ML Primer section. 

We’re excited about the new AIXPRT learning tool, and we’re also happy to report that we’re working on a version of the tool for CloudXPRT. We hope to make the CloudXPRT tool available early next year, and we’ll post more information in the blog as we get closer to taking it live.

If you have any questions about the tool, please let us know!

Justin

Posted in AI, AIXPRT, benchmark, BenchmarkXPRT, Cloud, CloudXPRT, computer vision, Cross-platform benchmarks, data analytics, image classification, Machine learning, object detection, OpenVINO, Performance benchmarking, TensorFlow, TensorRT | Also tagged , , , , ,

Following up

Justin Greene
By Justin Greene
on October 1, 2020

This week, we’re sharing news on two topics that we’ve discussed here in the blog over the past several months: CloudXPRT v1.01 and a potential AIXPRT OpenVINO update.

CloudXPRT v1.01

Last week, we announced that we were very close to releasing an updated CloudXPRT build (v1.01) with two minor bug fixes, an improved post-test results processing script, and an adjustment to one of our test configuration recommendations. Our testing and prep is complete, and the new version is live in the CloudXPRT GitHub repository and on our site!

None of the v1.01 changes affect performance or test results, so scores from the new build are comparable to those from previous CloudXPRT builds. If you’d like to know more about the changes, take a look at last week’s blog post.

The AIXPRT OpenVINO update

In late July, we discussed our plans to update the AIXPRT OpenVINO packages with OpenVINO 2020.3 Long-Term Support (LTS). While there are no known problems with the existing AIXPRT OpenVINO package, the LTS version targets environments that benefit from maximum stability and don’t require a constant stream of new tools and feature changes, so we thought it would be well suited for a benchmark like AIXPRT.

We initially believed that the update process would be relatively simple, and we’d be able to release a new AIXPRT OpenVINO package in September. However, we’ve discovered that the process is involved enough to require substantial low-level recoding. At this time, it’s difficult to estimate when the updated build will be ready for release. For any testers looking forward to the update, we apologize for the delay.

If you have any questions or comments about these or any other XPRT-related topics, please let us know!

Justin

Posted in AI, AIXPRT, benchmark, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Cloud, CloudXPRT, Datacenter, GitHub, Machine learning, OpenVINO, Performance benchmarking | Also tagged , , , ,

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x