Tag Archives: AI

An update on the AIXPRT Request for Comments preview

on January 24, 2019

As we approach the end of the original feedback window for the AIXPRT Request for Comments preview build, we want to update folks on the status of the project and what to expect in the coming weeks.

First, thanks to those who’ve downloaded the AIXPRT OpenVINO package and sent in their questions and comments. We value your feedback, and it’s instrumental in making AIXPRT a better tool. We’re currently working through some issues with the TensorFlow and TensorRT packages, and hope to add support for those to the RFC preview build repository very soon.

We’re also hoping to have a full-fledged community preview (CP) ready in mid to late February. Like our other community previews, the AIXPRT CP would be solid enough to allow folks to start quoting numbers. We typically make our benchmarks available to the general public four to six weeks after the community preview period begins, so if that schedule holds, it would place the public AIXPRT release around the end of March.

In light of the schedule described above, you still have time to gain access to the AIXPRT RFC preview build and give your feedback, so let us know if you’d like to check it out. The installation and testing process can take less than an hour, but getting everything properly set up can take a few tries. We are hard at work trying to make that process more straightforward. We welcome your input on all aspects of the benchmark, including workloads, ease of use, metrics, scores, and reporting.

Thanks for your help!

Justin

Posted in AI, AIXPRT, Benchmark metrics, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, Cross-platform benchmarks, Future of performance evaluation, OpenVINO, TensorFlow, TensorRT | Also tagged AIXPRT, BenchmarkXPRT, OpenVINO, TensorFlow, TensorRT

AI is the heartbeat of CES 2019

By Justin Greene

on January 10, 2019

This year’s CES features a familiar cast of characters: gigantic, super-thin 8K screens; plenty of signage promising the arrival of 5G; robots of all shapes, sizes, and levels of competency; and acres of personal grooming products that you can pair with your phone. In all seriousness, however, one main question keeps coming to mind as I walk the floor: Are we approaching the tipping point where AI truly starts to affect most people in meaningful ways on a daily basis? I think we’re still a couple of years away from ubiquitous AI, but it’s the heartbeat of this year’s show, and it’s going play a part in almost everything we do in the very near future. AI applications at this year’s show include manufacturing, transportation, energy, medicine, education, photography, communications, farming, grocery shopping, fitness, sports, defense, and entertainment, just to name a few. The AI revolution is just starting, but once it gets going, AI will continually reshape society for decades to come. This year’s show reinforces our decision to explore the roles that the XPRTs, beginning with AIXPRT, can play in the AI revolution.

Now for the fun stuff. Here’s a peek at a couple of my favorite displays so far. As is often the case, the most awe-inducing displays at CES are those that overwhelm attendees with light and sound. LG’s enormous curved OLED wall, dubbed the Massive Curve of Nature, was truly something to behold.

Another big draw has been Bell’s Nexus prototype, a hybrid-electric VTOL (vertical takeoff and landing) air taxi. Some journalists can’t resist calling it a flying car, but I refuse to do so, because it has nothing in common with cars apart from the fact that people sit in it and use it to travel from place to place. As Elon Musk once said of an earlier, but similar, concept, “it’s just a helicopter in helicopter’s clothing.” Semantics aside, it’s intriguing to imagine urban environments full of nimble aircraft that are quieter, easier to fly, and more energy efficient than traditional helicopters, especially if they’re paired with autonomous driving technologies.

Finally, quite a few companies are displaying props that put some of the “reality” back into “virtual reality.” Driving and flight simulators with full range of motion that are small enough to fit in someone’s basement or game room, full-body VR suits that control your temperature and deliver electrical stimulus based on game play (yikes!), and portable roller-coaster-like VR rides were just a few of the attractions.

It’s been a fascinating show so far!

Justin

Posted in AI, Augmented reality, Automation, CES, Drones, Education, IoT, Machine learning, Trade Shows, Virtual reality, VR Demo | Also tagged 5G, Bell, CES, Consumer Electronics Show, Las Vegas, LG, OLED, VR

XPRT collaborations: North Carolina State University

By Justin Greene

on December 6, 2018

For those of us who work on the BenchmarkXPRT tools, a core goal is involving new contributors and interested parties in the benchmark development process. Adding voices to the discussion fosters the collaboration and innovation that lead to powerful benchmark tools with lasting relevance.

One vehicle for outreach that we especially enjoy is sponsoring a student project through North Carolina State University. Each semester, the Senior Design Center in the university’s Department of Computer Science partners with external companies and organizations to provide student teams with an opportunity to work on real-world programming projects. If you’ve followed the XPRTs for a while, you may remember previous student projects such as Nebula Wolf, a mini-game that shows how well different devices handle games, and VR Demo, a virtual reality prototype workload based on a room escape scenario.

This fall, a team of NC State students is developing a software console for automating machine learning tests. Ideally, the tool will let future testers specify custom workload combinations, compute a performance metric, and upload results to our database. The project will also assess the impact of the framework on performance scores. In fact, the console will perform many of the same functions we plan to implement with AIXPRT.

The students have worked very hard on the project, and have learned quite a bit about benchmarking practices and several new software tools. The project will wrap up in the next couple of weeks, and we’ll share additional details as soon as possible. Early next year, we’ll publish a video about the experience.

If you’d like to join the NC State students and hundreds of other XPRT community members in the future of benchmark development, please let us know!

Justin

Posted in AI, AIXPRT, Benchmarking, BenchmarkXPRT development community, Collaborative benchmark development, Education, Future of performance evaluation, Nebula Wolf, North Carolina State University, Virtual reality, VR Demo | Also tagged benchmarks, computer science, gaming, machine learning, NC State, NCSU, North Carolina, North Carolina State University, university, virtual reality

AI and the next MobileXPRT

By Justin Greene

on August 16, 2018

As we mentioned a few weeks ago, we’re in the early planning stages for the next version of MobileXPRT—MobileXPRT 3. We’re always looking for ways to make XPRT benchmark workloads more relevant to everyday users, and a new version of MobileXPRT provides a great opportunity to incorporate emerging tech such as AI into our apps. AI is everywhere and is beginning to play a huge role in our everyday lives through smarter-than-ever phones, virtual assistants, and smart homes. The challenge for us is to identify representative mobile AI workloads that have the necessary characteristics to work well in a benchmark setting. For MobileXPRT, we’re researching AI workloads that have the following characteristics:

They work offline, not in the cloud.
They don’t require additional training prior to use.
They support common use cases such as image processing, optical character recognition (OCR), etc.

We’re researching the possibility of using Google’s Mobile Vision library, but there may be other options or concerns that we’re not aware of. If you have tips for places we should look, or ideas for workloads or APIs we haven’t mentioned, please let us know. We’ll keep the community informed as we narrow down our options.

Justin

Posted in AI, Android, Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Future of performance evaluation, Google, Machine learning, Mobile devices, MobileXPRT, Performance benchmarking | Also tagged benchmark, Google, image processing, mobile, MobileXPRT, optical character recognition

MWCS18 and AIXPRT: a new video

By Justin Greene

on July 12, 2018

A few weeks ago, Bill shared his first impressions from this year’s Mobile World Congress Shanghai (MWCS). “5G +” was the major theme, and there was a heavy emphasis on 5G + AI. This week, we published a video about Bill’s MWCS experience and the role that the XPRTs can play in evaluating emerging technologies such as 5G, AI, and VR. Check it out!

You can read more about AIXPRT development here. We’re still accepting responses to the AIXPRT Request for Comments, so if you would like to share your ideas on developing an AI/machine learning benchmark, please feel free to contact us.

Justin

Posted in 5G, AI, AIXPRT, AR, Augmented reality, Future of performance evaluation, Machine learning, Mobile devices, Mobile World Congress, Mobile World Congress, Performance benchmarking, Virtual reality | Also tagged 5G, AIXPRT, MWCS18, Shanghai, VR

Thoughts from MWC Shanghai 2018

By Bill Catchings

on June 28, 2018

Ni hao from Shanghai! It is amazing the change that happens in a year. This year’s MWC Shanghai, like last year’s, took up about half of the Shanghai New International Expo Centre (SNIEC). “5G +” is the major theme and, unlike last year, 5G is not something in the distant future. It is now assumed to be in progress.

The biggest of the pluses was AI, with a number of booths explicitly sporting 5G + AI signage. There were also 5G plus robots, cars, and cloud services. Many of those are really about AI as well. The show makes it feel like 5G is everywhere and will make everything better (or at least a lot faster). And Asia is leading the way.

[caption id="attachment_3447" align="alignleft" width="640"]

5G + robotics at MWCS 18.[/caption]

Most of the booths touted their 5G support as they did last year, but rather than talking about the future, they tried to say that their 5G was now. They claimed their products were in real-world tests with anticipated deployment schedules. One of the keynote speakers talked about 1.2 billion 5G connections by 2025, with more than half of those in Asia. The purported scale and speed of the transition to 5G is staggering.

[caption id="attachment_3449" align="alignleft" width="640"]

The keynote stage, displaying some big numbers.[/caption]

The last two halls I visited showed that world is not all 5G and AI. These halls looked at current fun applications of mobile technologies and companies developing technologies in the near future. MWC allowed children into one of the halls, where they (and we adults) could fly drones and experience VR technology. I watched in some amusement as people crashed drones, rode bikes with VR gear to simulate horses, were 3D scanned, and generally tried out new tech that didn’t always work.

The second hall included small booths from new companies working on future technologies that might be ready “4 years from now” (4YFN). These companies did not have much to show yet, but each booth displayed the company name and a short phrase summing up their future tech. That led to “Deepscent Labs is a smart scent data company,” ChineSpain is a “Marketplace of experiences for Chinese tourists in Spain,” and “Juice is a tech-based music contents startup that creates an ecosystem of music.” The mind boggles!

The XPRTs’ foray into AI with AIXPRT seems well timed based on this show. Other areas from this show that may be worth considering for the XPRTs are 5G and the cloud. We would love to hear your thoughts on those areas. We know they are important, but do you need the XPRTs and their emphasis on real-world benchmarks and workloads in those areas? Drop us a line and let us know!

Bill

Posted in 5G, AI, AIXPRT, AR, Augmented reality, Automated cars, Future of performance evaluation, IoT, Mobile data, Mobile devices, Mobile World Congress, Virtual reality | Also tagged 5G, China, mobile, Mobile World Shanghai, MWC, Shanghai, VR

Tag Archives: AI

An update on the AIXPRT Request for Comments preview

AI is the heartbeat of CES 2019

XPRT collaborations: North Carolina State University

AI and the next MobileXPRT

MWCS18 and AIXPRT: a new video

Thoughts from MWC Shanghai 2018

Check out the other XPRTs: