Category: Benchmarking

BatteryXPRT 2014 gets an update

on January 26, 2017

After Android 7 (Nougat) was released on select devices this past fall, we discovered an issue with BatteryXPRT on devices running Android 7 and above. The battery life tests were completing accurately and reliably, but the test was not producing a performance score.

The problem was a result of significant changes in the Android development environment. Android 7 restricted the flags used for different target architectures when linking native code components, and that caused issues while executing part of the Create Slideshow workload. We resolved the issue by changing the linked flags. Also, we migrated the BatteryXPRT code from the Eclipse and Android SDK development environments to the up-to-date Android Studio environment. This allowed us to rebuild the app in a way that maintains compatibility with the most recent versions of Android.

Today, we’re releasing a new build of BatteryXPRT 2014 (v104) at BatteryXPRT.com and the Google Play store. Scores from this build are comparable with previous BatteryXPRT scores, and if you’re testing with a version of BatteryXPRT that you downloaded from the Google Play store, you should receive the new build via an app update.

Click here to download the new BatteryXPRT installer (330 MB) directly from our site.

For users who have limited bandwidth or trouble accessing the Google Play store, downloading the APK files (26.7 MB total) may make installation easier.

Download the updated BatteryXPRT APK (2.8 MB) directly from our site.

Download the updated BatteryXPRT Tests APK (23.9 MB) directly from our site.

If you have any questions about the update or any other XPRT-related topic, feel free to contact us at BenchmarkXPRTSupport@principledtechnologies.com.

Justin

Posted in Android, Android N, Battery life, BatteryXPRT 2014 for Android, Performance benchmarking |

CrXPRT’s future

By Justin Greene

on January 19, 2017

This week, we’re continuing our review of the XPRT portfolio by discussing the future of CrXPRT. CrXPRT, designed for use with Chrome OS, is a tool for evaluating the performance and battery life of Chromebooks as they handle everyday tasks. The app’s performance test, which measures Chromebook speed, produces an overall score and individual scores for each workload. The battery life test produces an estimated battery life and a separate performance score. CrXPRT is easy to install and use, and like BatteryXPRT, it evaluates battery life in half a workday.

We developed CrXPRT in response to the growing popularity of Chromebooks, especially in the education sector. The number of OEMs manufacturing Chromebooks has grown dramatically, along with the range of Chromebook price points and form factors. That growth shows no signs of slowing down, so CrXPRT is more relevant than ever as a tool for helping consumers make informed buying decisions.

As Chromebook market share continues to grow, however, it’s clear that significant changes to the Chrome OS environment are on the way. One big change is Google’s decision to bring Android apps, and the Google Play store itself, to Chrome OS. Another change is the plan to “begin the evolution away” from the Chrome apps platform and phase out Chrome app support on other platforms within the next two years.

There are also reports of a hybrid Android-Chrome OS operating system. Codenamed “Andromeda,” it would unite the Android and Chrome OS environments in a manner similar to the way Microsoft Continuum allows Windows 10 to run on a wide variety of device types. Details on Andromeda are few and far between, but it would obviously be a game changer.

The Google Play store rollout to select Chromebooks is already well underway. As for the other changes, it remains to be seen exactly when and how they will be implemented. The Chromium team did state that all types of Chrome apps will remain supported and maintained on Chrome OS for the foreseeable future.

For us, it’s important to maintain the ability to measure both performance and battery life on Chromebooks. The current version of CrXPRT does the job well, so we don’t see a need for a new version until the situation becomes more clear. In the meantime, we’ll be keeping an eye on Chrome-related news.

As always, we’re interested in your feedback. If you have any thoughts on CrXPRT 2015 or the future of Chromebook evaluation, let us know!

Justin

Posted in Android, BatteryXPRT 2014 for Android, Chrome OS, Chromebooks, CrXPRT, Performance benchmarking, Windows 10 |

CES 2017

By Bill Catchings

on January 6, 2017

I’ve attended many tech shows over the years, but this year’s CES has more energy than any I’ve attended in a long time. Part of the energy is the breadth of products. There are amazingly slim TVs that make my TV at home, which I thought was slim, look fat. And, there are beautiful 8K TVs that make my new 4K one feel old.

I’m seeing all manner of smartphones. I’m seeing mobile remote presence devices such as the one from Beam, and after seeing the latest Fenix 5 from Garmin, I think I’ve found my next smartwatch.

There are many differing devices and approaches to VR and AR. There are drones everywhere. And, lots of massage chairs.

There are also plenty of products, let’s call them longshots, that contribute to the Wild West feel of the show. Maybe you’d like the Hydreon FakeTV, a small device to make it seem like a TV is on in your house to keep away burglars? Or Dr. Fuji’s Body Shaper, a vibrating platform to “accelerate your workout”?

Another part of the dynamic feeling is the breadth of vendors. Almost all the big tech vendors are here, except for Apple, of course. The excitement for me, however, is the small vendors displaying things that may well never see the light of day, but give glimpses of the future. For example, while I doubt most of the drone vendors at CES will be around in a few years, I think the trend toward small, inexpensive selfie drones will be.

The main reason for the energy at this year’s CES could be the convergence of multiple big industries. The most obvious example of this phenomenon is the large auto presence. Cars have been at CES before—the first time I drove my current car, a BMW i3, was at CES 2014—but this time around they seem to really want to make a statement. Faraday Future is making a splash by trying to be the next Tesla with its FF91.

Multiple vendors, including VW, Nissan, Toyota, and Mercedes, have concept cars on display—most of them are electric and all of them are heavy on technology. The biggest tech they’re touting is autonomous driving. The auto companies are showing their products while companies like NVIDIA, Intel, and Magic Eye are displaying the tech they have as well.

Regardless of the source of the energy at this CES, I see many opportunities for the existing XPRTs to continue to be important resources. I also see how important emerging technologies like machine learning and VR/AR are going to be and how the XPRTs can be of help there as well.

Exciting times!

Bill

Posted in Augmented reality, CES, Drones, Future of performance evaluation, Trade Shows, Uncategorized, Virtual reality |

Creating a machine-learning benchmark

By Bill Catchings

on December 22, 2016

Recently, we wrote about one of the most exciting emerging technology areas, machine learning, and the question of what role the XPRTs could play in the field.

Experts expect machine learning to be the analytics backbone of the IoT data explosion. It is a disruptive technology with potential to influence a broad range of industries. Consumer and industrial applications that take advantage of machine-learning advancements in computer vision, natural language processing, and data analytics are already available and many more are on the way.

Currently, there is no comprehensive machine-learning or deep-learning benchmark that includes home, automotive, industrial, and retail use cases. The challenge with developing a benchmark for machine learning is that these are still the early days of the technology. A fragmented software and hardware landscape and lack of standardized implementations makes benchmarking machine learning complex and challenging.

Based on the conversations we’ve had over the last few weeks, we’ve decided to take on that challenge. With the community’s help, of course!

As we outlined in a blog entry last month, we will work with interested folks in the community, key vendors, and academia to pull together what we are internally calling MLXPRT.

While the result may differ substantially from the existing XPRTs, we think the need for something is great. Whether that will turn out to be a packaged tool or just sample code and workloads remains to be seen.

What we need most your help. We need both general input about what you would like to see as well as any expertise you may have. Let us know any questions you may have or ways you can help.

On a related note, I’ll be at CES 2017 in Las Vegas during the first week of January. I’d love to meet and talk more about machine learning, benchmarking, or the XPRTs. If you’re planning to be there and would like to connect, let us know.

We will not have a blog entry next week over the holidays, so we wish all of you a wonderful time with your families and a great start to the new year.

Bill

Posted in BenchmarkXPRT, BenchmarkXPRT development community, CES, Collaborative benchmark development, Machine learning, What makes a good benchmark? |

Exploring virtual reality

By Eric Hale

on December 15, 2016

We’ve talked a lot in recent weeks about new technologies we are evaluating for the XPRTs. You may remember that back in June, we also wrote about sponsoring a second senior project with North Carolina State University. Last week, the project ended with the traditional Posters and Pies event. The team gave a very well thought‑out presentation.

As you can tell from the photo below, the team designed and implemented a nifty virtual reality app. It’s a room escape puzzle, and it looks great!

The app is a playable game with the ability to record the gameplay for doing repeatable tests. It also includes a recording that allows you to test a device without playing the game. Finally, the app lets you launch directly into the prerecorded game without using a viewer, which will be handy for testing multiple devices.

The team built the app using the Google Cardboard API and the Unity game engine, which allowed them to create Android and iOS versions. We’re looking forward to seeing what that may tell us!

After Posters and Pies, the team came to PT to present their work and answer questions. We were all very impressed with their knowledge and with how well thought out the application was.

Many thanks to team members Christian McCurdy, Gregory Manning, Grayson Jones, and Shon Ferguson (not shown).

Thanks also to Dr. Lina Battestilli, the team’s technical advisor, and Margaret Heil, Director of the Senior Design Center.

We are currently evaluating the app, and expect to make it available to the community in early 2017!

Eric

Posted in Android, Benchmarking, Collaborative benchmark development, iOS, North Carolina State University, Virtual reality |

WebXPRT in 2017

By Bill Catchings

on December 8, 2016

Over the last few weeks, we’ve discussed the future of HDXPRT and BatteryXPRT. This week, we’re discussing what’s in store for WebXPRT in 2017.

WebXPRT is our most popular tool. Manufacturers, developers, consumers, and media outlets in more than 350 cities and 57 countries have run WebXPRT over 113,000 times to date. The benchmark runs quickly and simply in most browsers and produces easy-to-understand results that are useful for comparing web browsing performance across a wide variety of devices and browsers. People love the fact that WebXPRT runs on almost any platform that has a web browser, from PCs to phones to game consoles.

More people are using WebXPRT in more places and in more ways than ever before. It’s an unquestioned success, but we think this is a good time to make it even better by beginning work on WebXPRT 2017. Any time change comes to a popular product, there’s a risk that faithful fans will lose the features and functionality they’ve grown to love. Relevant workloads, ease of use, and extensive compatibility have always been the core components of WebXPRT’s success, so we want to reassure users that we’re committed to maintaining all of those in future versions.

Some steps in the WebXPRT 2017 process are straightforward, such as the need to reassess the existing workload lineup and update content to reflect advances in commonly used technologies. Other steps, such as introducing new workloads to test emerging browser technologies, may be tricky to implement, but could offer tremendous value in the months and years ahead.

Are there test scenarios or browser technologies you would like to see in WebXPRT 2017, or tests you think we should get rid of? Please let us know. We want to hear from you and make sure that we’re crafting a performance tool that continues to meet your needs.

Bill

Posted in BatteryXPRT 2014 for Android, Future of performance evaluation, HDXPRT, HTML5, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2017 |

Category: Benchmarking

BatteryXPRT 2014 gets an update

CrXPRT’s future

CES 2017

Creating a machine-learning benchmark

Exploring virtual reality

WebXPRT in 2017

Check out the other XPRTs: