Category: Linux

An update on AIXPRT development

on September 26, 2019

It’s been a while since we last discussed the AIXPRT Community Preview 3 (CP3) release schedule, so we want to let everyone know where things stand. Testing for CP3 has taken longer than we predicted, but we believe we’re nearly ready for the release.

Testers can expect three significant changes in AIXPRT CP3. First, we updated support for the Ubuntu test packages. During the initial development phase of AIXPRT, Ubuntu version 16.04 LTS (Long Term Support) was the most current LTS version, but version 18.04 is now available.

Second, we have added TensorRT test packages for Windows and Ubuntu. Previously, AIXPRT testers could test only the TensorFlow variant of TensorRT. Now, they can use TensorRT to test systems with NVIDIA GPUs.

Third, we have added the Wide and Deep recommender system workload with the MXNet toolkit. Recommender systems are AI-based information-filtering tools that learn from end user input and behavior patterns and try to present them with optimized outputs that suit their needs and preferences. If you’ve used Netflix, YouTube, or Amazon accounts, you’ve encountered recommender systems that learn from your behavior.

Currently, the recommender system workload in AIXPRT CP3 is available for Ubuntu testing, but not for Windows. Recommender system inference workloads typically run on datacenter hardware, which tends to be Linux based. If enough community members are interested in running the MXNet/Wide and Deep test package on Windows, we can investigate what that would entail. If you’d like to see that option, please let us know.

As always, if you have any questions about the AIXPRT development process, feel free to ask!

Justin

Posted in AI, AIXPRT, benchmark, Benchmarking, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, Linux, Machine learning, MXNet, Performance benchmarking, Ubuntu, What makes a good benchmark?, Wide and Deep, Windows |

Navigating the AIXPRT Community Preview download page just got easier

By Justin Greene

on July 25, 2019

AIXPRT Community Preview 2 (CP2) has been generating quite a bit of interest among the BenchmarkXPRT Development Community and members of the tech press. We’re excited that the tool has piqued curiosity and that folks are recognizing its value for technical analysis. When talking with folks about test setup and configuration, we keep hearing the same questions:

How do I find the exact toolkit or package that I need?
How do I find the instructions for a specific toolkit?
What test configuration variables are most important for producing consistent, relevant results?
How do I know which values to choose when configuring options such as iterations, concurrent instances, and batch size?

In the coming weeks, we’ll be working to provide detailed answers to questions about test configuration. In response to the confusion about finding specific packages and instructions, we’ve redesigned the CP2 download page to make it easier for you to find what you need. Below, we show a snapshot from the new CP2 download table. Instead of having to download the entire CP2 package that includes the OpenVINO, TensorFlow, and TensorRT in TensorFlow test packages, you can now download one package at a time. In the Documentation column, we’ve posted package-specific instructions, so you won’t have to wade through the entire installation guide to find the instructions you need.

We hope these changes make it easier for people to experiment with AIXPRT. As always, please feel free to contact us with any questions or comments you may have.

Justin

Posted in AI, AIXPRT, AMD, benchmark, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, Intel, Linux, NVIDIA, OpenVINO, TensorFlow, TensorRT, Windows 10 | Tagged AI, AIXPRT, Community Preview, CPU, GPU, Linux, machine learning, OpenVINO, TensorFlow, TensorRT |

Making AIXPRT easier to use

By Justin Greene

on June 20, 2019

We’re glad to see so much interest in the AIXPRT CP2 build. Over the past few days, we’ve received two questions about the setup process: 1) where to find instructions for setting up AIXPRT on Windows, and 2) whether we could make it easier to install Intel OpenVINO on test systems.

In response to the first question, testers can find the relevant instructions for each framework in the readme files included in the AIXPRT install package. Instructions for Windows installation are in section 3 of the OpenVINO and TensorFlow readmes. Please note that whether you’re running AIXPRT on Ubuntu or Windows, be sure to read the “Known Issues” section in the readme, as there may be issues relevant to your specific configuration.

The readme files for each respective framework in the CP2 package are located here:

AIXPRT_0.5_CP2\AIXPRT_OpenVINO_0.5_CP2.zip\AIXPRT\Modules\Deep-Learning
AIXPRT_0.5_CP2\AIXPRT_TensorFLow_0.5_CP2.zip\AIXPRT\Modules\Deep-Learning
AIXPRT_0.5_CP2\AIXPRT_TensorFlow_TensorRT_0.5_CP2.zip\AIXPRT\Modules\Deep-Learning

We’re also working on consolidating the instructions into a central document that will make it easier for everyone to find the instructions they need.

In response to the question about OpenVINO installation, we’re working on an AIXPRT CP2 package that includes a precompiled version of OpenVINO R5.0.1 for easy installation on Windows via a few quick commands, and a script that installs the necessary OpenVINO dependencies. We’re currently testing the build, and we’ll make it available to testers as soon as possible.

The tests themselves will not change, so the new build will not influence existing results from Ubuntu or Windows. We hope it will simply facilitate the setup and testing process for many users.

We appreciate each bit of feedback that we receive, so if you have any suggestions for AIXPRT, please let us know!

Justin

Posted in AI, AIXPRT, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, Linux, Machine learning, OpenVINO, TensorFlow, TensorRT, Ubuntu, What makes a good benchmark?, Windows 10 | Tagged AI, AIXPRT, machine learning, OpenVINO, TensorFlow, TensorRT, Ubuntu, Windows |

An update on AIXPRT development

By Bill Catchings

on May 2, 2019

It’s been almost two months since the AIXPRT Community Preview went live, and we want to provide folks with a quick update. Community Preview periods for the XPRTs generally last about a month. Because of the complexity of AIXPRT and some of the feedback we’ve received, we plan to release a second AIXPRT Community Preview (CP2) later this month.

One of the biggest additions in CP2 will be the ability to run AIXPRT on Windows. AIXPRT currently requires test systems to run Ubuntu 16.04 LTS. This is fine for testers accustomed to Linux environments, but presents obstacles for those who want to test in a traditional Windows environment. We will not be changing the tests themselves, so this update will not influence existing results from Ubuntu. We plan to make CP2 available for download from the BenchmarkXPRT website for people who don’t wish to deal with GitHub.

Also, after speaking with testers and learning more about the kinds of data points people are looking for in AIXPRT results, we’ve decided to make significant adjustments to the AIXPRT results viewer. To make it easier for visitors to find what they’re looking for, we’ll add filters for key categories such as batch size, toolkit, and latency percentile (e.g., 50th, 90th, 99th), among others. We’ll also allow users to set desired ranges for metrics such as throughput and latency.

Finally, we’re adding a demo mode that displays some images and other information on the screen while a test is running to give users a better idea what is happening. While we haven’t seen results change while running in demo mode, users should not publish demo results or use them for comparison.

We hope to release CP2 in the second half of May and a GA version in mid-June. However, this project has more uncertainties than we usually encounter with the XPRTs, so that timeline could easily change.

We’ll continue to keep everyone up to date with AIXPRT news here in the blog. As always, we appreciate your suggestions. If you have any questions or comments about AIXPRT, please let us know.

Bill

Posted in AI, AIXPRT, Benchmarking, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, Cross-platform benchmarks, Future of performance evaluation, Linux, Machine learning, OpenVINO, Performance benchmarking, TensorFlow, TensorRT, Ubuntu | Tagged AI, AIXPRT, Community Preview, Ubuntu, Windows |

CrXPRT helps to navigate the changing Chromebook market

By Justin Greene

on June 7, 2018

Some people envision Chromebooks as low-end, plastic-shelled laptops that large organizations buy in bulk because they’re inexpensive and easy to manage. While many sub-$200 Chromebooks are still available, the platform is no longer limited to budget chipsets and little memory. Consumers can now choose systems that feature up to 16 GB of RAM, 8th generation Intel Core CPUs, and Core i7 configurations for those willing to pay around $1,600. In addition, some Chromebooks can now run Android apps, Microsoft Office mobile apps, Linux apps, and even Windows apps. While Chromebooks still depend heavily on connectivity and cloud storage, an increasing number of Chrome apps let you perform substantial productivity tasks offline. The Chrome OS landscape has changed so much that for certain use cases, the practical hardware gap between Chromebooks and traditional laptops is narrowing.

More consumers might be interested in Chromebooks than was the case a few years ago, but how they make sense of all the devices on the market? CrXPRT can help by providing objective data on Chromebook performance and battery life. Steven J. Vaughan Nichols offered a great example of the value CrXPRT can provide in his recent ZDNet article on the new Core i7-based Google Pixelbook. The Pixelbook’s CrXPRT score of 226 showed that it performs everyday tasks faster than any of the Chromebooks in our results database. When trying to decide whether it’s worth spending a few hundred or even a thousand dollars more on a new Chromebook, having the right data in hand can transform guesses into well-informed decisions.

You don’t have to be a tech journalist or even a techie to use CrXPRT. If you’d like to learn more about CrXPRT, we encourage you to read the CrXPRT feature here in the blog or visit CrXPRT.com.

Justin

Posted in Chrome OS, Chromebooks, CrXPRT, Google, Google Chrome, Linux, Uncategorized |

Category: Linux

An update on AIXPRT development

Navigating the AIXPRT Community Preview download page just got easier

Making AIXPRT easier to use

An update on AIXPRT development

CrXPRT helps to navigate the changing Chromebook market

Check out the other XPRTs: