BenchmarkXPRT Blog banner

Author Archives: Justin Greene

Answering questions about the AIXPRT Community Preview

Over the last two weeks, we’ve received a few questions about the AIXPRT Community Preview. Specifically, community members have asked about the project’s focus, possible future steps, and the results table. We decided to answer each of these here in the blog, since others are likely to have the same questions. We encourage folks to submit any new questions they may have.

PT previously stated that AIXPRT would be focused on edge devices. The current published results are from desktops and laptops. Is the focus of AIXPRT changing?

In the past, we did say that the focus of AIXPRT would be edge inference devices. After much feedback, we’ve come to understand that focus is probably too restrictive. PCs and laptops are using inference machine learning, and a decent amount of inference is taking place on servers in the cloud until phones are capable enough to handle the workloads. We now see all of these devices as potential targets for AIXPRT.

How did you choose the current results in your database?

We ran the AIXPRT CP on some of the systems we used during development and testing. We will continue to publish additional results as we test available systems in our lab. We’d love to get results from the community that cover a wider base of devices.

Will you be publishing results from servers?

We welcome server results submissions from the community, and will review them for publication on our site.

Will AIXPRT ever be available for Windows systems?

This is a possibility we’re actively exploring, and we hope to be able to share more about it soon.

What’s the best way to navigate the results table?

AIXPRT can run three toolkits, utilize two networks, and target CPU or GPU hardware. Together, these configuration options produce a lot of data points. To make it easier to handle all these variables, we’re working to improve the navigation, sorting, and filtering capabilities of the results table. In the meantime, a few tips:

  • There are two tabs at the top of the table, one for the ResNet-50 network and one for the SSD-MobileNet network. You can click the tabs to move between results for these networks.
  • Clicking any of the column headers will sort the data in that column A-Z (with the first click) or Z-A (with a second click).
  • To see if an individual test targeted a system’s CPU or GPU, read the description in the Summary column, e.g. Intel Core i7-7600U GPU / OpenVINO.
  • Clicking the entry in the Source column will take you to a more detailed page listing additional test configuration and system hardware information.

 

We’ll continue to share more information about AIXPRT in the coming weeks. Do you have additional questions or comments about AIXPRT? Let us know.

Justin

Three years of the XPRT Weekly Tech Spotlight

February marked the three-year anniversary of the XPRT Weekly Tech Spotlight, and we now have over 150 devices in our Spotlight library! We started the Spotlight to provide consumers with objective information on device hardware and performance, and to provide vendors with a trusted third-party showcase for their gear. Each week, we measure and verify the Spotlight device’s specs ourselves, never relying on vendor-published data. We also test each device with every applicable XPRT benchmark, and publish the data that lets consumers know how a device measures up to its competitors.

Over the past three years, we’ve featured a wide array of devices:

  • 49 phones
  • 28 laptops
  • 26 tablets
  • 24 2-in-1 devices
  • 12 small-form-factor PCs
  • 7 desktops
  • 6 game consoles
  • 6 all-in-ones

 

In addition to a wide variety of device types, we try to include a wide range of vendors. We’ve featured devices from ACEPC, Acer, Alcatel, Alienware, Amazon, Apple, ASUS, Barnes and Noble, BlackBerry, BLU, CHUWI, Dell, Essential, Fujitsu, Fusion5, Google, Honor, HP, HTC, Huawei, Intel, LeEco, Lenovo, LG, Microsoft, MINIX, Motorola, Nokia, NVIDIA, OnePlus, Razer, Samsung, Sony, Syber, Xiaomi, and ZTE.

XPRT Spotlight is a great way for device vendors and manufacturers to share PT-verified specs and test results with buyers around the world. We test many of the devices that appear each year and will test—at no charge—any device a manufacturer or vendor sends us. If you’d like us to test your device, please contact us at XPRTSpotlight@PrincipledTechnologies.com.

There’s a lot more to come for the XPRT Spotlight, and we’re constantly working on new features and improvements for the page. Are there any specific devices or features that you would like to see in the Spotlight? Let us know.

Justin

Principled Technologies and the BenchmarkXPRT Development Community release a preview of AIXPRT, a tool designed to help testers evaluate machine learning performance

Durham, NC, March 4 — Principled Technologies and the BenchmarkXPRT Development Community release the AIXPRT Community Preview. AIXPRT is a free tool that makes it easier to evaluate a system’s machine learning inference performance by running several common image-classification workloads.

The AIXPRT Community Preview build includes support for the Intel© OpenVINO™, TensorFlow™, and TensorFlow with NVIDIA© TensorRT™ toolkits to run image-classification workloads with ResNet-50 and SSD-MobileNet v1 networks. The test reports FP32, FP16, and INT8 levels of precision.

“This AIXPRT preview build is the next step towards our goal of making it easier for folks to evaluate how well systems handle machine learning tasks,” said Bill Catchings, co-founder of Principled Technologies, which administers the BenchmarkXPRT Development Community. “We invite all industry experts and interested parties to try out the AIXPRT Community Preview and send us their feedback.”

The AIXPRT Community Preview is available to anyone with a GitHub© account who is interested in participating. To request access, please contact the BenchmarkXPRT Development Community by sending a message to BenchmarkXPRTsupport@PrincipledTechnologies.com.

AIXPRT is part of the BenchmarkXPRT suite of performance evaluation tools, which includes WebXPRT, MobileXPRT, TouchXPRT, CrXPRT, HDXPRT, and BatteryXPRT. The XPRTs help users get the facts before they buy, use, or evaluate tech products such as computers, tablets, and phones.

To learn more about the AIXPRT, go to www.AIXPRT.com. To learn more about the BenchmarkXPRT Development Community, go to www.BenchmarkXPRT.com.

About Principled Technologies, Inc.
Principled Technologies, Inc. is a leading provider of technology marketing, as well as learning and development services. It administers the BenchmarkXPRT Development Community.

Principled Technologies, Inc. is located in Durham, North Carolina, USA. For more information, please visit www.PrincipledTechnologies.com.

Company Contact
Justin Greene
BenchmarkXPRT Development Community
Principled Technologies, Inc.
1007 Slater Road, Ste. 300
Durham, NC 27704

BenchmarkXPRTsupport@PrincipledTechnologies.com

All about the AIXPRT Community Preview

Last week, Bill discussed our plans for the AIXPRT Community Preview (CP). I’m happy to report that, despite some last-minute tweaks and testing, we’re close to being on schedule. We expect to take the CP build live in the coming days, and will send a message to community members to let them know when the build is available in the AIXPRT GitHub repository.

As we mentioned last week, the AIXPRT CP build includes support for the Intel OpenVINO, TensorFlow (CPU and GPU), and TensorFlow with NVIDIA TensorRT toolkits to run image-classification workloads with ResNet-50 and SSD-MobileNet v1 networks. The test reports FP32, FP16, and INT8 levels of precision. Although the minimum CPU and GPU requirements vary by toolkit, the test systems must be running Ubuntu 16.04 LTS. You’ll be able to find more detail on those requirements in the installation instructions that we’ll post on AIXPRT.com.

We’re making the AIXPRT CP available to anyone interested in participating, but you must have a GitHub account. To gain access to the CP, please contact us and let us know your GitHub username. Once we receive it, we’ll send you an invitation to join the repository as a collaborator.

We’re allowing folks to quote test results during the CP period, and we’ll publish results from our lab and other members of the community at AIXPRT.com. Because this testing involves so many complex variables, we may contact testers if we see published results that seem to be significantly different than those from comparable systems. During the CP period, On the AIXPRT results page, we’ll provide detailed instructions on how to send in your results for publication on our site. For each set of results we receive , we’ll disclose all of the detailed test, software, and hardware information that the tester provides. In doing so, our goal is to make it possible for others to reproduce the test and confirm that they get similar numbers.

If you make changes to the code during testing, we ask that you email us and describe those changes. We’ll evaluate if those changes should become part of AIXPRT. We also require that users do not publish results from modified versions of the code during the CP period.

We expect the AIXPRT CP period to last about four to six weeks, placing the public release around the end of March or beginning of April. In the meantime, we welcome your thoughts and suggestions about all aspects of the benchmark.

Please let us know if you have any questions. Stay tuned to AIXPRT.com and the blog for more developments, and we look forward to seeing your results!

JNG

Principled Technologies and the BenchmarkXPRT Development Community release HDXPRT 4, a benchmark designed to show how well Windows devices handle real-world media tasks

Durham, NC, February 25 — Principled Technologies and the BenchmarkXPRT Development Community have released HDXPRT 4, a free benchmark that gives objective information about how well Windows 10 devices handle common media-creation tasks. HDXPRT 4 uses real commercial applications, like Photoshop and MediaEspresso, to perform tasks based on three everyday scenarios: photo editing, video conversion, and music editing. After the test is finished, the tool provides an overall measure by generating a single performance score. Anyone can go to HDXPRT.com to compare existing performance results on a variety of devices, or to download the app for themselves.

“When we started working on HDXPRT 4, we knew we wanted to create a benchmark that accurately reflects the kind of work average consumers do when creating content on their PCs,” said Bill Catchings, co-founder of Principled Technologies, which administers the BenchmarkXPRT Development Community. “HDXPRT delivers clear results that make sense to the wide audience of buyers shopping for new Windows systems.”

HDXPRT is part of the BenchmarkXPRT suite of performance evaluation tools, which includes WebXPRT, MobileXPRT, TouchXPRT, CrXPRT, and BatteryXPRT. The XPRTs help users get the facts before they buy, use, or evaluate tech products such as computers, tablets, and phones.

To learn more about the BenchmarkXPRT Development Community, go to www.BenchmarkXPRT.com.

About Principled Technologies, Inc.
Principled Technologies, Inc. is a leading provider of technology marketing, as well as learning and development services. It administers the BenchmarkXPRT Development Community.

Principled Technologies, Inc. is located in Durham, North Carolina, USA. For more information, please visit www.PrincipledTechnologies.com.

Company Contact
Justin Greene
BenchmarkXPRT Development Community
Principled Technologies, Inc.
1007 Slater Road, Ste. 300
Durham, NC 27704

BenchmarkXPRTsupport@PrincipledTechnologies.com

HDXPRT 4 is here!

We’re excited to announce that HDXPRT 4 is now available to the public! Just like previous versions of HDXPRT, HDXPRT 4 uses trial versions of commercial applications to complete real-world media tasks. The HDXPRT 4 installation package includes installers for some of those programs, such as Audacity and HandBrake. For other programs, such as Adobe Photoshop Elements and CyberLink Media Espresso, users will need to download the necessary installers prior to testing by using the links and instructions in the HDXPRT 4 User Manual.

In addition to the editing photos, editing music, and converting videos workloads from prior versions of the benchmark, HDXPRT 4 includes two new Photoshop Elements scenarios. The first utilizes an AI tool that corrects closed eyes in photos, and the second creates a single panoramic photo from seven separate photos.

HDXPRT 4 is compatible with systems running Windows 10, and is available for download at HDXPRT.com. The installation package is about 4.8 GB, so the download may take several minutes. The setup process takes about 30 minutes on most computers, and a standard test run takes approximately an hour.

After trying out HDXPRT 4, please submit your scores here and send any comments to BenchmarkXPRTsupport@principledtechnologies.com. To see test results from a variety of systems, go to HDXPRT.com and click View Results, where you’ll find scores from a variety of devices. We look forward to seeing your results!

Check out the other XPRTs:

Forgot your password?