Category: MobileXPRT 2015

Notes from the lab

on June 15, 2017

This week’s XPRT Weekly Tech Spotlight featured the Alcatel A30 Android phone. We chose the A30, an Amazon exclusive, because it’s a budget phone running Android 7.0 (Nougat) right out of the box. That may be an appealing combination for consumers, but running a newer OS on inexpensive hardware such as what’s found in the A30 can cause issues for app developers, and the XPRTs are no exception.

Spotlight fans may have noticed that we didn’t post a MobileXPRT 2015 or BatteryXPRT 2014 score for the A30. In both cases, the benchmark did not produce an overall score because of a problem that occurs during the Create Slideshow workload. The issue deals with text relocation and significant changes in the Android development environment.

As of Android 5.0, on 64-bit devices, the OS doesn’t allow native code executables to perform text relocation. Instead, it is necessary to compile the executables using position-independent code (PIC) flags. This is how we compiled the current version of MobileXPRT, and it’s why we updated BatteryXPRT earlier this year to maintain compatibility with the most recent versions of Android.

However, the same approach doesn’t work for SoCs built with older 32-bit ARMv7-A architectures, such as the A30’s Qualcomm Snapdragon 210, so testers may encounter this issue on other devices with low-end hardware.

Testers who run into this problem can still use MobileXPRT 2015 to generate individual workload scores for the Apply Photo Effects, Create Photo Collages, Encrypt Personal Content, and Detect Faces workloads. Also, BatteryXPRT will produce an estimated battery life for the device, but since it won’t produce a performance score, we ask that testers use those numbers for informational purposes and not publication.

If you have any questions or have encountered additional issues, please let us know!

Justin

Posted in Android, Android N, Battery life, BatteryXPRT 2014 for Android, Benchmarking computing devices, MobileXPRT, MobileXPRT 2015 |

Running Android-oriented XPRTs on Chrome OS

By Justin Greene

on March 30, 2017

Since last summer, we’ve been following Google’s progress in bringing Android apps and the Google Play store to Chromebooks, along with their plan to gradually phase out support for Chrome apps over the next few years. Because we currently offer apps that assess battery life and performance for Android devices (BatteryXPRT and MobileXPRT) and Chromebooks (CrXPRT), the way this situation unfolds could affect the makeup of the XPRT portfolio in the years to come.

For now, we’re experimenting to see how well the Android app/Chrome OS merger is working with the devices in our lab. One test case is the Samsung Chromebook Plus, which we featured in the XPRT Weekly Tech Spotlight a few weeks ago. Normally, we would publish only CrXPRT and WebXPRT results for a Chromebook, but installing and running MobileXPRT 2015 from the Google Play store was such a smooth and error-free process that we decided to publish the first MobileXPRT score for a device running Chrome OS.

We also tried running BatteryXPRT on the Chromebook Plus, but even though the installation was quick and easy and the test kicked off without a hitch, we could not generate a valid result. Typically, the test would complete several iterations successfully, but terminate before producing a result. We’re investigating the problem, and will keep the community up to date on what we find.

In the meantime, we continue to recommend that Chromebook testers use CrXPRT for performance and battery life assessment. While we haven’t encountered any issues running MobileXPRT 2015 on Chromebooks, CrXPRT has a proven track record.

If you have any questions about running Android-oriented XPRTs on Chrome OS, or insights that you’d like to share, please let us know.

Justin

Posted in Android, BatteryXPRT 2014 for Android, Chrome OS, Chromebooks, CrXPRT, Google, Google Chrome, MobileXPRT, MobileXPRT 2015, WebXPRT |

BatteryXPRT’s future

By Justin Greene

on December 1, 2016

A few weeks ago, we discussed the future of HDXPRT. This week, we’re focusing on the current state of BatteryXPRT 2014 for Android, and how the benchmark may evolve in 2017.

BatteryXPRT continues to provide users with reliable evaluations of their Android device’s performance and battery life under real-world conditions. Originally designed to be compatible with Android 4.2 (Jelly Bean) and above, the benchmark continues to work well on subsequent versions of Android, up to and including Android 6.0 (Marshmallow).

Since Android 7 (Nougat) began to roll out on select devices in the past few months, our internal testing has shown that we’ll need to adjust the BatteryXPRT source code to maintain compatibility with devices running Android 7 and above. We developed the existing source when Eclipse was the officially supported SDK environment, and now we need to bring the code in line with the current Android Studio SDK.

Practically speaking, BatteryXPRT does run on Nougat, and to the best of our knowledge, battery life results are still accurate and reliable. However, the test will not produce a performance score. As more Nougat devices are released in the coming months, it’s possible that other aspects of the benchmark may encounter issues. If this happens during your testing, we encourage you to let us know.

Because MobileXPRT 2015 and BatteryXPRT 2014 performance workloads are so closely related, the next obvious question is whether MobileXPRT 2015 runs on Nougat devices. As of now, MobileXPRT 2015 does run successfully and reliably on Android 7, and this is because the most recent build of MobileXPRT 2015 was compiled using a newer SDK.

We think the best course of action for MobileXPRT 2015 and BatteryXPRT will be to eventually combine them into a single, easy-to-use Android benchmark for performance and battery life. We’ll talk more about that plan in the coming months, and we look forward to hearing your input. Until that transition is successful, we’ll continue to support both BatteryXPRT and MobileXPRT 2015.

As always, we welcome your feedback on these developments, as well as any ideas you may have for future XPRTs.

Justin

Posted in Android, Android L, Android M, Android N, Battery life, BatteryXPRT 2014 for Android, Collaborative benchmark development, MobileXPRT 2015 |

The big weekend is almost here!

By Eric Hale

on March 10, 2016

A few weeks ago, I talked about the XPRT Women Code-a-Thon. Well, all the work that we and our friends at ChickTech Seattle have done is about to pay off! On Saturday, March 12, dozens of women will go to the Sole Repair Shop in Seattle for two days of coding, good food, and networking opportunities.

The goal of each team at the code-a-thon is to create a new workload that might be included in a future version of WebXPRT or MobileXPRT. Judges will award prizes to the top three workloads: $2,500 for first place, $1,500 for second place, and $1,000 for third place. I can’t wait to see the winning workloads!

We’re very fortunate to have Kristin Toth Smith as the keynote speaker. She is an avid supporter of women in tech, current COO of Dolly, and former CEO of Code Fellows.

It should be a great time for all. If you or anyone you know can get to Seattle this weekend, registration is still open.

Eric

Last week in the XPRTs
We published the XPRT Weekly Tech Spotlight on the ASUS ZenFone 2.
We added one new BatteryXPRT ’14 result.
We added five new WebXPRT ’15 results.

Posted in ChickTech Seattle, Collaborative benchmark development, MobileXPRT 2015, WebXPRT 2015, XPRT Women Code-a-Thon |

An update on TouchXPRT 2016

By Eric Hale

on October 8, 2015

We’ll be releasing the MobileXPRT 2015 white paper tomorrow. It contains lots of information about MobileXPRT 2015 that you won’t find anywhere else. We hope you’ll find it very informative.

A couple of weeks ago, we released the design document for TouchXPRT 2016 (login required). This week, we put the first build of TouchXPRT 2016 into testing. It’s a Universal Windows app that runs on Windows 10 tablets, PCs, and phones. This means that TouchXPRT can now run on a wider variety of devices. However, it also means that TouchXPRT 2016 will not be backward compatible with Windows 8 and 8.1.

Given the current state of the SDKs, installing the test builds on phones is more complicated than we would like. We’re looking into ways to simplify the install before releasing the community preview. Testing on phones is particularly important because we made many of the UI changes to enable TouchXPRT to work acceptably on a small display.

We’ll keep you informed as testing proceeds. We’re hoping to release the community preview in the next couple of weeks.

Eric