Category: Google Chrome

Running Android-oriented XPRTs on Chrome OS

on March 30, 2017

Since last summer, we’ve been following Google’s progress in bringing Android apps and the Google Play store to Chromebooks, along with their plan to gradually phase out support for Chrome apps over the next few years. Because we currently offer apps that assess battery life and performance for Android devices (BatteryXPRT and MobileXPRT) and Chromebooks (CrXPRT), the way this situation unfolds could affect the makeup of the XPRT portfolio in the years to come.

For now, we’re experimenting to see how well the Android app/Chrome OS merger is working with the devices in our lab. One test case is the Samsung Chromebook Plus, which we featured in the XPRT Weekly Tech Spotlight a few weeks ago. Normally, we would publish only CrXPRT and WebXPRT results for a Chromebook, but installing and running MobileXPRT 2015 from the Google Play store was such a smooth and error-free process that we decided to publish the first MobileXPRT score for a device running Chrome OS.

We also tried running BatteryXPRT on the Chromebook Plus, but even though the installation was quick and easy and the test kicked off without a hitch, we could not generate a valid result. Typically, the test would complete several iterations successfully, but terminate before producing a result. We’re investigating the problem, and will keep the community up to date on what we find.

In the meantime, we continue to recommend that Chromebook testers use CrXPRT for performance and battery life assessment. While we haven’t encountered any issues running MobileXPRT 2015 on Chromebooks, CrXPRT has a proven track record.

If you have any questions about running Android-oriented XPRTs on Chrome OS, or insights that you’d like to share, please let us know.

Justin

Posted in Android, BatteryXPRT 2014 for Android, Chrome OS, Chromebooks, CrXPRT, Google, Google Chrome, MobileXPRT, MobileXPRT 2015, WebXPRT |

They’re coming!

By Eric Hale

on June 16, 2016

We’ve been hearing for a while about Google’s plan to bring Android apps to Chrome. They recently published a video on the Google Developers channel that gives us some idea of what running Android apps on a Chromebook would look like.

Because I’m very interested in performance, the claim “Android apps can run full speed, with no overhead and no performance penalties” got my attention. You can bet we’ll be using the XPRTs to check that out! We’re using a Google developer tool called ARC Welder to do some experiments. However, it’s not fair or valid to print performance results based on a developer tool, so we’ll have to wait until the official release to see what the performance is really like.

Obviously, the use cases for Chrome will be changing. The demos in the video are for workloads we associate with PCs. MobileXPRT-type workloads might be more appropriate, or, assuming the scripting tools are available, perhaps HDXPRT-type workloads. We’ll be watching these developments closely to see how they will affect our future cross-platform benchmark.

Eric

Posted in Android, Chrome OS, Chromebooks, Google Chrome, HDXPRT, MobileXPRT, PCs |

Focusing the spotlight

By Justin Greene

on March 24, 2016

As you may have heard, the Samsung Galaxy S7 is the XPRT Weekly Tech Spotlight this week. As we were testing it, we noticed that our WebXPRT scores were about 8 percent lower than those reported by AnandTech.

The folks at AnandTech do a good job on their reviews, so we wanted to understand the discrepancy in scores. The S7 comes in a couple of models, so we started by verifying that our model was the same as theirs. It was.

The next step was to check their configuration against ours, and this is where we found the difference. Both phones were running the same version of Android, but the S7 AnandTech tested used Chrome 48 while the S7 we tested came preloaded with Chrome 49. In our testing, we’ve noticed that upgrading from Chrome 48 to Chrome 49 has a noticeable performance impact on certain devices. On the Samsung Galaxy S6, the scores went down about 10 percent. In all cases we’ve seen, the decrease is driven largely by the Stock Option Pricing workload.

This isn’t the first time we’ve written about browser versions affecting results. WebXPRT is a browsing benchmark, and the browser has a legitimate impact on performance. When you’re comparing results, it’s always important to look at all the factors involved.

Justin

Last week in the XPRTs

We published the XPRT Weekly Tech Spotlight on the Samsung Galaxy S7.
We added two new BatteryXPRT ’14 results.
We added one new MobileXPRT ’15 result.
We added four new WebXPRT ’15 results.

Posted in AnandTech, Android, Android M, Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Google Chrome, WebXPRT, WebXPRT 2015 |

A couple of things

By Eric Hale

on April 2, 2015

We’ve heard about a couple of issues this week. The HDXPRT 2014 Convert Videos test uses the MediaEspresso application and requires hardware acceleration be configured. On some systems, this setting is not configuring automatically, which may result in lower scores. We’re working on a solution, but in the meantime there’s an easy workaround.

We’ve also found that some Chromebooks report extremely low battery use, as low as 0%, for the first couple iterations of the CrXPRT battery life test. This can cause CrXPRT to report results with a very wide confidence interval, with an interval greater than 15%. We’re looking at ways to detect and compensate for this. However, if you see results with a very wide confidence interval, we recommend that you use the rundown test.

In other news, we’ve been talking about Intel Developer Forum 2015 – Shenzhen for weeks, and the time is finally here! Bill and Justin get on the plane tomorrow. If you’d like to talk about the XPRTs, or any of PT’s other offerings, Bill and the team would be happy to meet with you!

Eric

Posted in Collaborative benchmark development, CrXPRT, Google Chrome, HDXPRT, IDF |

A guest blog from Wilson Cheng at Acer Inc.

By Wilson Cheng

on February 12, 2015

Two years ago, I got a job to evaluate cross-platform benchmarks for comparing Windows and Android.

Usually, Web-based benchmarks focus on only Java-script and HTML5 functionality. I found WebXPRT is easy to use and includes the tasks that you do every day. It reliably tests the user experience and gives a simple score for comparison. I suggested to our manager that we use it to evaluate our platform and he agreed.

Recently, Chromebooks have become very popular. Testing the battery life of Chromebooks has been difficult because you have needed to use the power_LoadTest image from the Chromium project or the test image from the Chrome OS Partners Web site to do power load testing. Without reimaging the system, you couldn’t test battery life. Because CrXPRT is designed to test battery life as well as performance on ChromeOS, this extra step is no longer needed. We can run CrXPRT without updating the test image, which is very useful in our battery life testing.

The advantage of XPRT benchmarks is that they are easy to use and give real-world results. The disadvantage is that they are not yet popular on review sites.

Posted in Android, BenchmarkXPRT, Chrome OS, Chromebooks, CrXPRT, Google Chrome, WebXPRT, What makes a good benchmark? |

An updated CrXPRT 2015 build is available

By Eric Hale

on February 10, 2015

Today we’re releasing a new build of CrXPRT 2015 (v1.0.2) in the Chrome Web Store. The updates included in this build correct the handling of outlier scores for individual workloads, check the duration of each battery-life test iteration, and introduce a check to ensure that the system is not plugged in during battery-life testing. The tests have not changed, and the method of calculating the overall score and battery-life score is the same, so the main scores are comparable with previous CrXPRT scores.

We published the update today in the Chrome Web Store as of 3:00 PM EST, but it may take some time for the update to appear on your system. You may have to manually approve the update notice when it appears on your system.

If you have any questions about the update or any other XPRT-related topic, feel free to contact us at BenchmarkXPRTsupport@principledtechnologies.com.

Posted in Battery life, Chrome OS, Chromebooks, CrXPRT, Google Chrome |

Category: Google Chrome

Running Android-oriented XPRTs on Chrome OS

They’re coming!

Focusing the spotlight

A couple of things

A guest blog from Wilson Cheng at Acer Inc.

An updated CrXPRT 2015 build is available

Check out the other XPRTs: