Category: AI

Thoughts from MWC Shanghai 2018

on June 28, 2018

Ni hao from Shanghai! It is amazing the change that happens in a year. This year’s MWC Shanghai, like last year’s, took up about half of the Shanghai New International Expo Centre (SNIEC). “5G +” is the major theme and, unlike last year, 5G is not something in the distant future. It is now assumed to be in progress.

The biggest of the pluses was AI, with a number of booths explicitly sporting 5G + AI signage. There were also 5G plus robots, cars, and cloud services. Many of those are really about AI as well. The show makes it feel like 5G is everywhere and will make everything better (or at least a lot faster). And Asia is leading the way.

[caption id="attachment_3447" align="alignleft" width="640"]

5G + robotics at MWCS 18.[/caption]

Most of the booths touted their 5G support as they did last year, but rather than talking about the future, they tried to say that their 5G was now. They claimed their products were in real-world tests with anticipated deployment schedules. One of the keynote speakers talked about 1.2 billion 5G connections by 2025, with more than half of those in Asia. The purported scale and speed of the transition to 5G is staggering.

[caption id="attachment_3449" align="alignleft" width="640"]

The keynote stage, displaying some big numbers.[/caption]

The last two halls I visited showed that world is not all 5G and AI. These halls looked at current fun applications of mobile technologies and companies developing technologies in the near future. MWC allowed children into one of the halls, where they (and we adults) could fly drones and experience VR technology. I watched in some amusement as people crashed drones, rode bikes with VR gear to simulate horses, were 3D scanned, and generally tried out new tech that didn’t always work.

The second hall included small booths from new companies working on future technologies that might be ready “4 years from now” (4YFN). These companies did not have much to show yet, but each booth displayed the company name and a short phrase summing up their future tech. That led to “Deepscent Labs is a smart scent data company,” ChineSpain is a “Marketplace of experiences for Chinese tourists in Spain,” and “Juice is a tech-based music contents startup that creates an ecosystem of music.” The mind boggles!

The XPRTs’ foray into AI with AIXPRT seems well timed based on this show. Other areas from this show that may be worth considering for the XPRTs are 5G and the cloud. We would love to hear your thoughts on those areas. We know they are important, but do you need the XPRTs and their emphasis on real-world benchmarks and workloads in those areas? Drop us a line and let us know!

Bill

Posted in 5G, AI, AIXPRT, AR, Augmented reality, Automated cars, Future of performance evaluation, IoT, Mobile data, Mobile devices, Mobile World Congress, Virtual reality | Tagged 5G, AI, China, mobile, Mobile World Shanghai, MWC, Shanghai, VR |

AIXPRT: We want your feedback!

By Justin Greene

on April 12, 2018

Today, we’re publishing the AIXPRT Request for Comments (RFC) document. The RFC explains the need for a new artificial intelligence (AI)/machine learning benchmark, shows how the BenchmarkXPRT Development Community plans to address that need, and provides preliminary design specifications for the benchmark.

We’re seeking feedback and suggestions from anyone interested in shaping the future of machine learning benchmarking, including those not currently part of the Development Community. Usually, only members of the BenchmarkXPRT Development Community have access to our RFCs and the opportunity to provide feedback. However, because we’re seeking input from non-members who have expertise in this field, we will be posting this RFC in the New events & happenings section of the main BenchmarkXPRT.com page and making it available at AIXPRT.com.

We welcome input on all aspects of the benchmark, including scope, workloads, metrics and scores, UI design, and reporting requirements. We will accept feedback through May 13, 2018, after which BenchmarkXPRT Development Community administrators will collect and evaluate the feedback and publish the final design specification.

Please share the RFC with anyone interested in machine learning benchmarking and please send us your feedback before May 13.

Justin

Posted in AI, Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Future of performance evaluation, Machine learning, Performance benchmarking, What makes a good benchmark? |

MWC18 and technology on the brink

By Mark Van Name

on March 8, 2018

This year’s Mobile World Congress in Barcelona bristled with technologies on the brink of superstardom. The long-awaited 5G high-speed mobile standard again dominated the conversations, and is one year closer to creating a world of high-speed connections that will make possible mobile usages we’ve only begun to discover. Intelligent, connected cars promise a self-driving and highly interconnected automotive experience that should ultimately make driving better for all of us. Artificial intelligence, already a star, showed glimmers of its vast and still barely tapped potential. In keeping with the show’s name, mobile devices of all sorts proved that phones and tablets and laptops are nowhere near done, with new models and capabilities available all over the many halls that comprised the MWC campus.

Each of those technologies will continue to evolve rapidly over the coming years, and each will create new opportunities for us all to benefit. Those opportunities will appear both in ways we understand now—faster connections and quicker devices, for example—and in fashions we don’t yet understand. The new benefits will lead to new usage models, change the ways we interact with the world, and create whole new markets. (When the first smartphones appeared, they changed photography forever, but that wasn’t their primary goal.) These new technologies will help us in ways we can now only glimpse.

These changes and new capabilities will breed both competition and, inevitably, confusion. How are we to know which of the new products deliver the best implementations of these technologies heading toward stardom, and how are we to know when to upgrade to new generations of these products?

Answering those questions, and clarifying some of the confusing aspects of the always shifting tech market, are the reasons the XPRT community and tools exist. New tech creates new usage models that require new tools to assess–XPRT tools.

If there’s one last lesson I learned from MWC18, it’s that our work is only just beginning. The new technologies that are on the brink today will become superstars soon, and we’ll be there with the tools you need to assess and compare them.

Mark

Posted in 5G, AI, Automated cars, BenchmarkXPRT development community, Future of performance evaluation, Mobile World Congress, Mobile World Congress, Trade Shows |

Thinking ahead at CES 2018

By Justin Greene

on January 11, 2018

It may sound trite to say that a show like CES is all about the future, but this year’s show is prompting me to think about how our lives will evolve in the coming years. Some technological breakthroughs change the way we do everyday things like play music or hail a cab—while some transform the way we do everything. For technological innovation to truly shift society on a wide scale, it has to coincide with markets of scale in a way that makes life-changing tech accessible to almost everyone (think: smartphones in 2005 versus smartphones in 2018).

These technical and economic forces are coinciding once again in the areas of AI, automation, the Internet of Things (IoT), and consumer robotics. While many of our daily activities will stay the same, the ways we organize and engage with those activities are changing dramatically.

I’ll leave you with a few general observations from the show:

Huawei has a huge presence here. A new tagline for them that I haven’t seen before is a play on their name, “Wow Way.” I suspect we may have a Mate 10 Pro XPRT Spotlight entry in the near future.
The Kino-mo Hypervsn 3D Holograph display blew me away. People were crowding in to see it and couldn’t stop staring. It’s straight out of sci-fi, and its appearance is similar to Princess Leia’s hologram message in Star Wars. (By the way, it looks way better in real life than in the video.)
Sony is making a big push into smart homes by building systems that work cross-platform with a range of smart speakers and assistants. Between their smart home push and some of the cool home theater tech they had on display, I can see them gaining some brand power.
To me, the most exciting concepts at the show involved smart infrastructure, which promises enormous potential to boost the efficient distribution of water, energy, and transportation resources.
Surprisingly, I saw automation, smart city, and smart infrastructure displays from companies that I don’t always associate with IoT or AI, like Bosch and Panasonic. Panasonic was marketing an array of semi-autonomous vehicle cockpit prototypes, and had a section highlighting their partnership with Tesla.
By far, the strangest thing I’ve seen at CES has been the Psychasec booth, staffed by eerie attendants in pure white outfits who talked confidently about “downloading your cortical stack into customized bodies made from organic materials.” The Psychasec staff absolutely refused to break character, which made the whole scene even stranger. Check out the link above for the story behind Psychasec.

And, while I’m probably not supposed to admit this, my favorite part of the show so far has been the line of expensive massage chair vendors doling out free sessions…

More to follow soon,

Justin

Posted in AI, Automated cars, Automation, CES, Holograph, Internet of things, IoT, Trade Shows |

News about WebXPRT and BatteryXPRT

By Justin Greene

on December 7, 2017

Last month, we gave readers a glimpse of the updates in store for the next WebXPRT, and now we have more news to report on that front.

The new version of WebXPRT will be called WebXPRT 3. WebXPRT 3 will retain the convenient features that made WebXPRT 2013 and WebXPRT 2015 our most popular tools, with more than 200,000 combined runs to date. We’ve added new elements, including AI, to a few of the workloads, but the test will still run in 15 minutes or less in most browsers and produce the same easy-to-understand results that help compare browsing performance across a wide variety of devices.

We’re also very close to publishing the WebXPRT 3 Community Preview. For those unfamiliar with our open development community model, BenchmarkXPRT Development Community members have the ability to preview and test new benchmark tools before we release them to the general public. Community previews are a great way for members to evaluate new XPRTs and send us feedback. If you’re interested in joining, you can register here.

In BatteryXPRT news, we recently started to see unusual battery life estimates and high variance when running battery life tests at the default length of 5.25 hours. We think this may be due to changes in how new OS versions are reporting battery life on certain devices, but we’re in the process of extensive testing to learn more. In the meantime, we recommend that BatteryXPRT users adjust the test run time to allow for a full rundown.

Do you have questions or comments about WebXPRT or BatteryXPRT? Let us know!

Justin

Posted in AI, Battery life, BatteryXPRT 2014 for Android, BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Community Preview, Cross-platform benchmarks, Future of performance evaluation, Performance benchmarking, WebXPRT, WebXPRT 3 |

Machine learning performance tool update

By Bill Catchings

on October 19, 2017

Earlier this year we started talking about our efforts to develop a tool to help in evaluating machine learning performance. We’ve given some updates since then, but we’ve also gotten some questions, so I thought I’d do my best to summarize our answers for everyone.

Some have asked what kinds of algorithms we’ve been looking into. As we said in an earlier blog, we’re looking at algorithms involved in computer vision, natural language processing, and data analytics, particularly different aspects of computer vision.

One seemingly trivial question we’ve received regards the proposed name, MLXPRT. We have been thinking of this tool as evaluating machine learning performance, but folks have raised a valid concern that it may well be broader than that. Does machine learning include deep learning? What about other artificial intelligence approaches? I’ve certainly seen other approaches lumped into machine learning, probably because machine learning is the hot topic of the moment. It feels like everything is boasting, “Now with machine learning!”

While there is some value in being part of such a hot movement, we’ve begun to wonder if a more inclusive name, such as AIXPRT, would be better. We’d love to hear your thoughts on that.

We’ve also had questions about the kind of devices the tool will run on. The short answer is that we’re concentrating on edge devices. While there is a need for server AI/ML tools, we’ve been focusing on the evaluating the devices close to the end users. As a result, we’re looking at the inference aspect of machine learning rather than the training aspect.

Probably the most frequent thing we’ve been asked about is the timetable. While we’d hoped to have something available this year, we were overly optimistic. We’re currently working on a more detailed proposal of what the tool will be, and we aim to make that available by the end of this year. If we achieve that goal, our next one will be to have a preliminary version of the tool itself ready in the first half of 2018.

As always, we seek input from folks, like yourself, who are working in these areas. What would you most like to see in an AI/machine learning performance tool? Do you have any questions?

Bill

Posted in AI, Benchmark metrics, Collaborative benchmark development, computer vision, Future of performance evaluation, Machine learning, What makes a good benchmark? |

Category: AI

Thoughts from MWC Shanghai 2018

AIXPRT: We want your feedback!

MWC18 and technology on the brink

Thinking ahead at CES 2018

News about WebXPRT and BatteryXPRT

Machine learning performance tool update

Check out the other XPRTs: