Category: CrXPRT

A very good question

on December 4, 2014

A couple of weeks ago, we released the second community preview of CrXPRT (CP2). One of the changes was to make CrXPRT handle its test results more like WebXPRT. It now automatically uploads some test data to the PT servers, so that we can use it to improve future versions of the benchmark. Like WebXPRT, no personally identifying information is collected, and the results are never made public without your permission.

This week we got a question about the new results collection in CP2. The member wanted to know if the uploading of the results affected the battery life score at all. It does not. The results collection happens after the test. If you’re doing a battery rundown test, the results aren’t uploaded until the device boots the next time. I’m sorry we didn’t make that clearer in the documentation.

When we release a community preview, we commit that the results from that preview will be comparable to any subsequent previews and to the general release. In the case of CP2, we compared the results from CP2 to those from CP1 to make sure that the results stayed comparable. We take this commitment very seriously. When you test with a community preview, you can be sure that the results will still be good when the general release comes out.

In other news, we released a patch for BatteryXPRT yesterday. As we explained in the last blog post, this new version of BatteryXPRT is much more stable on devices running Android 5.

Have any other good questions? Send them our way.

Eric

Comment on this post in the forums

Posted in Android, BatteryXPRT 2014 for Android, Chrome OS, Collaborative benchmark development, Community Preview, CrXPRT, WebXPRT |

A new model

By Eric Hale

on November 20, 2014

Today we released a new community preview for CrXPRT (CP2). It contains several enhancements and UI changes. It also fixes the problem we discussed in last week’s blog, so CP2 will now correctly execute a rundown test on a system using Chinese language settings.

The most exciting enhancement is that CP2 will let you test battery life without having to put the device into developer mode. The new Battery Status API, which became available with Chrome 38, makes this possible.

Another enhancement is that results submission is now integrated into the benchmark UI. CrXPRT follows the WebXPRT model, with you having opt-in control over whether the results are published on the PT Web site. However, CrXPRT goes further by allowing you the option to upload more extensive disclosure information, and the option to download results as a text file.

These and other enhancements are covered in more detail in the CrXPRT forum and the user manual. Because this is a community preview, you have to be a community member to download it. However, joining is very easy.

We hope you like the new features in CrXPRT. Let us know what you think!

Eric

Discuss CrXPRT CP2 in the forums

Posted in Chrome OS, Collaborative benchmark development, Community Preview, CrXPRT |

The value of diversity

By Eric Hale

on November 13, 2014

It’s great to hear from community members who are using the XPRTs. Having a pool of users with diverse viewpoints is critical for successful software development. While we try very hard to find problems, there will always be cases we miss.

For example, we talked recently about the challenges of working in a multilingual world. Just this week, Acer reported an interesting bug in the CrXPRT community preview. If your language is set to Chinese, the battery test will estimate the battery life as expected. However, a rundown test will return the message “Cannot calculate total run time.” This does not happen when the language is set to English.

We have verified this problem. In the next week or so, we’ll be releasing a second community preview containing a fix for this and a few other issues.

We’ve talked before about the difficulties of maintaining software in a fast moving world. Late last week, we received a new Nexus 9. It encountered a problem during the MobileXPRT tests, so we fixed the problem and will be releasing a patched build tomorrow.

Don’t forget, as we said last week, we’ll be releasing the WebXPRT 2015 design overview tomorrow. We are looking forward to the feedback from all our community members.

Eric

Comment on this post in the forums

Posted in Android, Chrome OS, CrXPRT, MobileXPRT, WebXPRT, WebXPRT 2015 |

CrXPRT is here!

By Eric Hale

on October 15, 2014

Today we are releasing the CrXPRT 2014 Community Preview (CP1). As mentioned in a previous post, CrXPRT contains performance and battery life tests. The performance suite includes five scenarios utilizing Web browsing and JavaScript workloads, plus Portable Native Client (PNaCl) and WebGL-based scenarios. The battery life test incorporates all of the performance workloads and adds video playback, audio playback, and HTML5 gaming scenarios.

The battery life test in CP1 builds on the lessons we learned from developing BatteryXPRT 2014 for Android. In fact, we’ve been able to improve on the testing time. BatteryXPRT 2014 requires 5.5 hours to estimate battery life; CP1 can estimate battery life in only 3.5 hours. The battery test in CP1 still requires the device be put in developer mode, so we’re investigating the new Chrome OS battery status APIs. We hope these will make it possible to remove this restriction in a future release.

The estimates for battery life are generally pretty accurate. However, we have seen runs where the battery life results were much higher than expected. We are continuing to investigate this. If you see an anomalous result, please let us know. It is worth noting that the performance scores have been very consistent.

Because this is a community preview, you have to be a community member to download it. However, joining is very easy.

Check out the new CrXPRT, and let us know what you think!

Eric

Comment on this post in the forums

Posted in Battery life, BenchmarkXPRT development community, Chrome OS, Community Preview, CrXPRT, Performance benchmarking |

News from the factory floor

By Eric Hale

on October 9, 2014

As we mentioned last week, we have BatteryXPRT and MobileXPRT news:

Today, we’re releasing a new build of MobileXPRT 2013 at MobileXPRT.com and the Google Play store. This build addresses issues we saw when testing MobileXPRT on the beta build of Android L and the experimental ART runtime. The tests have not changed, and the scores are comparable with previous MobileXPRT 2013 scores.

Also, the BatteryXPRT 2014 for Android APKs are now available at BatteryXPRT.com. Up to now, only the full installer, including the content for the tests, was available on our Web site. The APKs are much smaller and allow you to download the test content during installation. For users who have trouble accessing the Google Play store, these APKs may make it easier to download the benchmark.

If you have any questions or concerns, please don’t hesitate to contact us at BenchmarkXPRTsupport@principledtechnologies.com.

Don’t forget: we’re releasing the community preview of CrXPRT next week!

Eric

Comment on this post in the forums

Posted in Android, BatteryXPRT 2014 for Android, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Chrome OS, Community Preview, CrXPRT, MobileXPRT |

Activity all around

By Justin Greene

on October 2, 2014

It’s a busy time in XPRT land, and we wanted to share some news about upcoming events.

The first bit is an update about the CrXPRT 2014 Community Preview. We’ve been busy refining the benchmark, and feel like the current candidate build under test is likely to ship as our Community Preview. We’ll publish more details about the app and its workloads soon, but don’t be surprised to see a Community Preview announcement in the near future.

In Patching and future proofing, we discussed two potential problems while running MobileXPRT 2013 on some versions of Android 4.4 and the developer’s preview of Android L. The first problem involves the benchmark not showing scores when running on a Nexus 5 with the ART runtime enabled. The second problem was a failure of the Create Slideshow workload on Android L. In order to address the underlying cause of both problems, we’ll soon be issuing a patched MobileXPRT 2013 build. We’re still wrapping up our testing, but expect to release the patch at some point next week.

We’ll also be posting BatteryXPRT 2014 for Android APKs directly on the BatteryXPRT.com site. For users who have low bandwidth or trouble accessing the Google Play store, these APKs will make it easier to download the benchmark.

If you have any questions or concerns, please don’t hesitate to contact us at BenchmarkXPRTsupport@principledtechnologies.com.

Justin

Comment on this post in the forums

Posted in Android, BatteryXPRT 2014 for Android, Community Preview, CrXPRT, MobileXPRT |

Category: CrXPRT

A very good question

A new model

The value of diversity

CrXPRT is here!

News from the factory floor

Activity all around

Check out the other XPRTs: