Category: AIXPRT

Make confident choices about your company’s future tech with the XPRTs

on April 23, 2020

Durham, NC, April 23, 2020 — Principled Technologies and the BenchmarkXPRT Development Community have released a video on the benefits of consulting the XPRTs before committing to new technology purchases.

AIXPRT, one of the battery of XPRT benchmark tools, runs image-classification and object-detection workloads to determine how well tech handles AI and machine learning.

CloudXPRT, another XPRT tool, accurately measures the end-to-end performance of modern, cloud-first applications deployed on infrastructure as a service (IaaS) platforms – allowing corporate decision-makers to select the best configuration for every objective.

All of the XPRTs give companies the real-world information necessary to determine which prospective future tech p – and which will disappoint

According to the video, “The XPRTs don’t just look at specs and features; they gauge a technology solution’s real-world performance and capabilities. So you know whether switching environments is worth the investment. How well solutions support machine learning and other AI capabilities. If next-gen releases beat their rivals or fall behind the curve.”

Watch the video at facts.pt/pyt88k5. To learn more about how AIXPRT, CloudXPRT, WebXPRT, MobileXPRT, TouchXPRT, CrXPRT, and HDXPRT can help IT decision-makers can make confident choices about future purchases, go to www.BenchmarkXPRT.com.

About Principled Technologies, Inc.
Principled Technologies, Inc. is the leading provider of technology marketing and learning & development services. It administers the BenchmarkXPRT Development Community.

Principled Technologies, Inc. is located in Durham, North Carolina, USA. For more information, please visit www.principledtechnologies.com.

Company Contact
Justin Greene
BenchmarkXPRT Development Community
Principled Technologies, Inc.
1007 Slater Road, Suite #300
Durham, NC 27703
BenchmarkXPRTsupport@PrincipledTechnologies.com

Posted in AI, AIXPRT, benchmark, Benchmarking, BenchmarkXPRT, Cloud, CloudXPRT, Cross-platform benchmarks, IaaS, image classification, object detection, Performance benchmarking, Press Releases | Tagged AI, AIXPRT, cloud, CloudXPRT, IaaS, image classification, object detection |

Adapting to a changing tech landscape

By Justin Greene

on April 23, 2020

The BenchmarkXPRT Development Community started almost 10 years ago with the development of the High Definition Experience & Performance Ratings Test, also known as HDXPRT. Back then, we distributed the benchmark to interested parties by mailing out physical DVDs. We’ve come a long way since then, as testers now freely and easily access six XPRT benchmarks from our site and major app stores.

Developers, hardware manufacturers, and tech journalists—the core group of XPRT testers—work within a constantly changing tech landscape. Because of our commitment to providing those testers with what they need, the XPRTs grew as we developed additional benchmarks to expand the reach of our tools from PCs to servers and all types of notebooks, Chromebooks, and mobile devices.

As today’s tech landscape continues to evolve at a rapid pace, our desire to play an active role in emerging markets continues to drive us to expand our testing capabilities into areas like machine learning (AIXPRT) and cloud-first applications (CloudXPRT). While these new technologies carry the potential to increase efficiency, improve quality, and boost the bottom line for companies around the world, it’s often difficult to decide where and how to invest in new hardware or services. The ever-present need for relevant and reliable data is the reason many organizations use the XPRTs to help make confident choices about their company’s future tech.

We just released a new video that helps to explain what the XPRTs provide and how they can play an important role in a company’s tech purchasing decisions. We hope you’ll check it out!

We’re excited about the continued growth of the XPRTs, and we’re eager to meet the challenges of adapting to the changing tech landscape. If you have any questions about the XPRTs or suggestions for future benchmarks, please let us know!

Justin

Posted in AI, AIXPRT, benchmark, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Chromebooks, Cloud, CloudXPRT, Collaborative benchmark development, Future of performance evaluation, HDXPRT, History of benchmarking, Performance benchmarking | Tagged AIXPRT, benchmark, BenchmarkXPRT, cloud applications, CloudXPRT, HDXPRT, machine learning |

News about the CloudXPRT source code

By Justin Greene

on April 9, 2020

For much of the BenchmarkXPRT Development Community’s history, we offered community members exclusive access to XPRT benchmark source code. Back in February, we started to experiment with a different approach when we made the AIXPRT source code publicly available on GitHub. By allowing anyone who is interested in AIXPRT to download and review the source code, we reinforced our commitment to making the XPRT development process as transparent as possible. We also want the XPRTs to continue to contribute to fair practices in the benchmarking world, and we believe that expanded access to the source code encourages constructive feedback to help in this goal.

The feedback we received after publishing the AIXPRT source code was very positive; thank you to all who reached out. Because of that feedback and our desire to increase openness, we’ve decided use standard open source licenses to make the CloudXPRT source code available to the public when we release of the first build, or shortly thereafter. As with AIXPRT, folks will be able to download the CloudXPRT source code and submit potential workloads for future consideration, but we reserve the right to control derivative works.

We’ll share more information about the first CloudXPRT release and its source code in the coming weeks. If you have any questions about XPRT source code, feel free to ask. We also welcome any thoughts about using this approach to release the source code of other XPRT benchmarks. As always, feel free to comment below or reach out by email.

Justin

Posted in AI, AIXPRT, benchmark, BenchmarkXPRT, BenchmarkXPRT development community, Cloud, CloudXPRT, Collaborative benchmark development, Open development, Open Source, Source code |

More details about CloudXPRT’s workloads

By Justin Greene

on April 2, 2020

About a month ago, we posted an update on the CloudXPRT development process. Today, we want to provide more details about the three workloads we plan to offer in the initial preview build:

In the web-tier microservices workload, a simulated user logs in to a web application that does three things: provides a selection of stock options, performs Monte-Carlo simulations with those stocks, and presents the user with options that may be of interest. The workload reports performance in transactions per second, which testers can use to directly compare IaaS stacks and to evaluate whether any given stack is capable of meeting service-level agreement (SLA) thresholds.
The machine learning (ML) training workload calculates XGBoost model training time. XGBoost is a gradient-boosting framework that data scientists often use for ML-based regression and classification problems. The purpose of the workload in the context of CloudXPRT is to evaluate how well an IaaS stack enables XGBoost to speed and optimize model training. The workload reports latency and throughput rates. As with the web-tier microservices workload, testers can use this workload’s metrics to compare IaaS stack performance and to evaluate whether any given stack is capable of meeting SLA thresholds.
The AI-themed container scaling workload starts up a container and uses a version of the AIXPRT harness to launch Wide and Deep recommender system inference tasks in the container. Each container represents a fixed amount of work, and as the number of Wide and Deep jobs increases, CloudXPRT launches more containers in parallel to handle the load. The workload reports both the startup time for the containers and the Wide and Deep throughput results. Testers can use this workload to compare container startup time between IaaS stacks; optimize the balance between resource allocation, capacity, and throughput on a given stack; and confirm whether a given stack is suitable for specific SLAs.

We’re continuing to move forward with CloudXPRT development and testing and hope to add more workloads in subsequent builds. Like most organizations, we’ve adjusted our work patterns to adapt to the COVID-19 situation. While this has slowed our progress a bit, we still hope to release the CloudXPRT preview build in April. If anything changes, we’ll let folks know as soon as possible here in the blog.

If you have any thoughts or comments about CloudXPRT workloads, please feel free to contact us.

Justin

Posted in AI, AIXPRT, benchmark, Benchmark metrics, BenchmarkXPRT, Cloud, CloudXPRT, container scaling, Datacenter, Hybrid cloud, Machine learning, microservices, On-premise, recommender system, Servers, Wide and Deep | Tagged AIXPRT, cloud, CloudXPRT, container scaling, IaaS, machine learning, SLA |

The Introduction to AIXPRT white paper is now available!

By Justin Greene

on March 26, 2020

Today, we published the Introduction to AIXPRT white paper. The paper serves as an overview of the benchmark and a consolidation of AIXPRT-related information that we’ve published in the XPRT blog over the past several months. For folks who are completely new to AIXPRT and veteran testers who need to brush up on pre-test configuration procedures, we hope this paper will be a quick, one-stop reference that helps reduce the learning curve.

The paper describes the AIXPRT toolkits and workloads, adjusting key test parameters (batch size, level of precision, number of concurrent instances, and default number of requests), using alternate test configuration files, understanding and submitting results, and accessing the source code.

We hope that Introduction to AIXPRT will prove to be a valuable resource. Moving forward, readers will be able to access the paper from the Helpful Info box on AIXPRT.com and the AIXPRT section of our XPRT white papers page. If you have any questions about AIXPRT, please let us know!

Justin

Posted in AI, AIXPRT, benchmark, Machine learning, MXNet, ResNet-50, SSD-MobileNet v1, White papers | Tagged AIXPRT, batch size, concurrent instances, OpenVINO, precision, ResNet-50, SSD-MobileNet v1, TensorFlow, TensorRT, white paper, Wide and Deep |

Odds and ends

By Justin Greene

on March 19, 2020

Today, we want to share quick updates on a few XPRT topics.

In case you missed yesterday’s announcement, the CrXPRT 2 Community Preview (CP) is now available. BenchmarkXPRT Development Community members can access the preview using a direct link we’ve posted on the CrXPRT tab in the XPRT Members’ Area (login required). This tab also provides a link to the CrXPRT 2 CP user manual. You can find a summary of what’s new with CrXPRT 2 in last week’s blog. During the preview period, we allow testers to publish CP test scores. Note that CrXPRT 2 overall performance test scores and battery life measurements are not comparable to those from CrXPRT 2015.

We’ll soon be publishing our first AIXPRT whitepaper, Introduction to AIXPRT. It will summarize the AIXPRT toolkits and workloads; how to adjust test parameters such as batch size, levels of precision, and concurrent instances; how to use alternate test configuration files; and how to understand test results. When the paper is available, we’ll post it on the XPRT white papers page and make an announcement here in the blog.

Finally, in response to decreased downloads and usage of BatteryXPRT, we have ended support for the benchmark. We’re always monitoring usage of the XPRTs so that we can better direct our resources to the current needs of users. We’ve removed BatteryXPRT from the Google Play Store, but it is still available for download on BatteryXPRT.com.

If you have any questions about CrXPRT 2, AIXPRT, or BatteryXPRT, please let us know!

Justin

Posted in AI, AIXPRT, Battery life, BenchmarkXPRT, BenchmarkXPRT development community, Chrome OS, Chromebooks, CrXPRT, White papers | Tagged AI, AIXPRT, Android, BatteryXPRT, Chrome, CrXPRT, white paper |

Category: AIXPRT

Make confident choices about your company’s future tech with the XPRTs

Adapting to a changing tech landscape

News about the CloudXPRT source code

More details about CloudXPRT’s workloads

The Introduction to AIXPRT white paper is now available!

Odds and ends

Check out the other XPRTs: