BenchmarkXPRT Blog banner

Category: MobileXPRT 3

The XPRTs in 2020: a year to remember

Justin Greene
By Justin Greene
on December 17, 2020

As 2020 comes to a close, we want to take this opportunity to review another productive year for the XPRTs. Readers of our newsletter are familiar with the stats and updates we include each month, but for our blog readers who don’t receive the newsletter, we’ve compiled some highlights below.

Benchmarks
In the past year, we released CrXPRT 2 and updated MobileXPRT 3 for testing on Android 11 phones. The biggest XPRT benchmark news was the release of CloudXPRT v1.0 and v1.01. CloudXPRT, our newest  benchmark, can accurately measure the performance of cloud applications deployed on modern infrastructure-as-a-service (IaaS) platforms, whether those platforms are paired with on-premises, private cloud, or public cloud deployments. 

XPRTs in the media
Journalists, advertisers, and analysts referenced the XPRTs thousands of times in 2020, and it’s always rewarding to know that the XPRTs have proven to be useful and reliable assessment tools for technology publications such as AnandTech, ArsTechnica, Computer Base, Gizmodo, HardwareZone, Laptop Mag, Legit Reviews, Notebookcheck, PCMag, PCWorld, Popular Science, TechPowerUp, Tom’s Hardware, VentureBeat, and ZDNet.

Downloads and confirmed runs
So far in 2020, we’ve had more than 24,200 benchmark downloads and 164,600 confirmed runs. Our most popular benchmark, WebXPRT, just passed 675,000 runs since its debut in 2013! WebXPRT continues to be a go-to, industry-standard performance benchmark for OEM labs, vendors, and leading tech press outlets around the globe.

Media, publications, and interactive tools
Part of our mission with the XPRTs is to produce materials that help testers better understand the ins and outs of benchmarking in general and the XPRTs in particular. To help achieve this goal, we’ve published the following in 2020:

We’re thankful for everyone who has used the XPRTs, joined the community, and sent questions and suggestions throughout 2020. This will be our last blog post of the year, but there’s much more to come in 2021. Stay tuned in early January for updates!

Justin

Posted in AIXPRT, AnandTech, Android 11, benchmark, Benchmarking, BenchmarkXPRT, Cloud, CloudXPRT, CrXPRT, IaaS, Interactive tools, MobileXPRT, MobileXPRT 3, Performance benchmarking, Phones, WebXPRT | Tagged , , , , , , , , , , |

New MobileXPRT 3 installations may crash on Android 11

Justin Greene
By Justin Greene
on October 15, 2020

We recently received a tech support inquiry about problems with new MobileXPRT 3 installations on some Android 11 phones. The tester installed MobileXPRT 3 on a selection of phones running Android 11, and the app crashed immediately upon opening. We were able to reproduce the issue on multiple phones in our lab, and currently know that the issue may happen on the Google Pixel 3, Google Pixel 4a 5G, Google Pixel 4XL, Google Pixel 5, and the OnePlus 8T (running Android 11 with an Oxygen OS skin).

MobileXPRT 3 continues to run without issues on Android 9 and 10 phones. When we updated an Android 10 phone with an existing MobileXPRT 3 installation to Android 11, we found that the benchmark ran successfully. This suggests a lack of fundamental incompatibilities between MobileXPRT 3 and current versions of Android 11. Because some of our lab techs experienced crashes immediately after the app asked for permissions, we think it’s possible that new permissions-setting requirements in Android 11 are causing the problem.

We’re currently working to isolate the problem and identify a course of action. We’ll share more information here in the blog as soon as possible. If you’ve encountered this problem in your testing, we apologize for the inconvenience, and we’re thankful for your patience as we work towards a solution.

If you have any information you’d like to share about running MobileXPRT 3 on Android 11, please let us know!

Justin

Posted in Android, Android 10, Android 11, Android 9, benchmark, Benchmarking, Google, Mobile devices, MobileXPRT, MobileXPRT 3, Performance benchmarking, Phones, Tablets | Tagged , , , , , , , , , , |

The XPRTs in 2019: Looking back on an exciting and productive year

Justin Greene
By Justin Greene
on December 19, 2019

2019 is winding down, and we want to take this opportunity to review another exciting and productive year for the BenchmarkXPRT Development Community. Readers of our newsletter are familiar with the stats and updates we post in each month’s mailing, but we know that not all our blog readers receive the newsletter, so we’ve compiled the highlights below.

Trade shows
Earlier this year, Justin attended CES in Las Vegas and Mark travelled to MWC Barcelona. These shows help us keep up with the latest industry trends and gather insights that help to lay the groundwork for XPRT development in the years ahead.

Benchmarks
In the past year, we released MobileXPRT 3, HDXPRT 4, and AIXPRT, our new AI benchmark tool that helps you evaluate a system’s machine learning inference performance. There’s much more to come in 2020 with AIXPRT and several other projects, so expect more news about benchmark development early in the year.

Web mentions
In 2019 so far, journalists, advertisers, and analysts have referenced the XPRTs over 5,000 times, including mentions in more than 190 articles and 1,350 device reviews. This represents a more than 50% increase over 2018.

Downloads and confirmed runs
To date, we’ve had more than 24,800 benchmark downloads and 153,000 confirmed runs in 2019, increases of more than 8% and 10%, respectively, over 2018. Within the last month, our most popular benchmark, WebXPRT, passed the 500,000-run milestone! WebXPRT continues to be an industry-standard performance benchmark upon which OEM labs, vendors, and leading tech press outlets rely.

XPRT Tech Spotlight
We put 47 new devices in the XPRT Tech Spotlight throughout the year and published updated back-to-school, Black Friday, and holiday showcases to help buyers compare devices.

Media and interactive tools
We published a new XPRTs around the world infographic and an interactive AIXPRT installation package selector tool. We’ve received a lot of positive feedback about the tool. We encourage you to give it a try if you’re curious about AIXPRT but aren’t sure how to get started.

We’re thankful for everyone who used the XPRTs, joined the community, and sent questions and suggestions throughout 2019. This will be our last blog post for 2019, but there’s much more to come in 2020, including some exciting new developments. Stay tuned in early January for updates!

Justin

Posted in AIXPRT, benchmark, BenchmarkXPRT, BenchmarkXPRT development community, CES, HDXPRT 4, Mobile World Congress, MobileXPRT 3, Trade Shows, WebXPRT, XPRT Tech Spotlight | Tagged , , , , , , , |

Planning for the next TouchXPRT

Justin Greene
By Justin Greene
on August 29, 2019

We’re in the very early planning stages for the next version of TouchXPRT, and we’d love to hear any suggestions you may have. What do you like or dislike about TouchXPRT? What features do you hope to see in a new version?

For those who are unfamiliar with TouchXPRT, it’s a benchmark for evaluating the performance of Windows 10 devices. TouchXPRT 2016, the most recent version, runs tests based on five everyday scenarios (Beautify Photos, Blend Photos, Convert Videos for Sharing, Create Music Podcast, and Create Slideshow from Photos) and produces results for each of the five scenarios plus an overall score. The benchmark is available two ways: as a Universal Windows App in the Microsoft Store and as a sideload installer package on TouchXPRT.com.

When we begin work on a new version of any benchmark, one of the first steps we take is to assess its workloads to determine whether they will provide value during the years ahead. This step involves evaluating whether to update test content such as photos and videos to more contemporary file resolutions and sizes, and can also involve removing workloads or adding completely new ones. Should we keep the TouchXPRT workloads listed above or investigate other use cases? Should we research potential AI-related workloads? What do you think?

As we did with MobileXPRT 3 and HDXPRT 4 earlier this year, we’re also planning to update the TouchXPRT UI to improve the look of the benchmark and make it easier to use. We’re just at the beginning of this process, so any feedback you send has a chance to really shape the future of the benchmark.

On a related note, TouchXPRT 2016 testers who use the installer package available on TouchXPRT.com may have noticed that the package has a new file name (TX2016.6.52.0_8.19.19.zip). Microsoft requires developers to assign a security certificate to all sideload apps, and the new TouchXPRT file contains a refreshed certificate. We did not change the benchmark in any other way, so scores from this package are comparable to previous TouchXPRT 2016 scores.

Justin

Posted in Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Future of performance evaluation, HDXPRT 4, MobileXPRT 3, Performance benchmarking, TouchXPRT, TouchXPRT 2016, What makes a good benchmark?, Windows, Windows 10 | Tagged , , , , |

A new MobileXPRT 3 build is available

Justin Greene
By Justin Greene
on April 25, 2019

Today, we published an updated MobileXPRT 3 build, version 3.114.2.1, on MobileXPRT.com and in the Google Play Store. The new build fixes an issue that was causing crashes on Xiaomi phones. Xiaomi holds significant market share in China, so we wanted to address the issue as soon as possible.

Xiaomi phones use a proprietary Android-based firmware called MIUI, which requires apps to communicate with the system in a specific way. When we originally built MobileXPRT 3, Android allowed an app’s code to send implicit messages calling certain classes of actions. In MIUI, the code must broadcast explicit messages that call the exact action necessary (e.g., waking from sleep). The requirement can improve security by allowing more granular levels of user control, and save power by restricting the number of unseen tasks that apps can run in the background without a user’s knowledge. The new MobileXPRT 3 build code complies with MIUI’s requirements. Other Android-based platforms will likely require explicit messages in the near future, so we’re hoping the new build will be relatively future proof.

We also fixed a few small UI bugs and improved the accuracy of the system hardware information that the app reports when a user submits a set of results. None of these changes affect performance, so scores from prior MobileXPRT 3 builds are comparable to those from the new build. If you have any questions or comments about MobileXPRT 3, please let us know.

Justin

Posted in Android, Android O, Benchmarking, MobileXPRT, MobileXPRT 3, MobileXPRT 3, Performance benchmarking, Phones, Tablets | Tagged , , , , , , |

MobileXPRT 3 is here!

Justin Greene
By Justin Greene
on January 31, 2019

We’re excited to announce that MobileXPRT 3 is now available to the public! MobileXPRT 3 is the latest version of our popular tool for evaluating the performance of Android devices. The BenchmarkXPRT Development Community has been using a community preview for several weeks, but now anyone can run the tool and publish their results.

Compatible with systems running Android 5.0 and above, MobileXPRT 3 includes the same performance workloads as MobileXPRT 2015 (Apply Photo Effects, Create Photo Collages, Create Slideshow, Encrypt Personal Content, and Detect Faces to Organize Photos), plus a new optical character recognition-based workload called Scan Receipts for Spreadsheet.

MobileXPRT 3 is available at MobileXPRT.com and on the Google Play Store. Alternatively, you can download the app using either of the links below:



After trying out MobileXPRT 3, please submit your scores here and send any comments to BenchmarkXPRTsupport@principledtechnologies.com. To see test results from a variety of systems, go to MobileXPRT.com and click View Results, where you’ll find scores from a variety of Android devices. We look forward to seeing your results!

Justin

Posted in Android, Android L, Android M, Android N, Android O, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, Future of performance evaluation, Google, Mobile devices, MobileXPRT, MobileXPRT 3, MobileXPRT 3, Performance benchmarking, Performance testing on tablets, Phones |

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x