BenchmarkXPRT Blog banner

Search Results for: webxprt

The busiest year ever!

Eric Hale
By Eric Hale
on December 25, 2014

As 2014 winds down, it’s a good time to look back at the year. And what a year it has been! This is the year that the XPRTs really went global. The benchmarks are in wider use than ever. The community continues to grow.

Here are a few of the big things that happened this year:

 

It’s going to be hard to top this year, but we are certainly going to try! There’s another video coming soon. We’re already working on WebXPRT 2015. (If you haven’t commented on the WebXPRT 2015 design document yet, it’s not too late.) We’re making plans for MWC 2015. And there’s a lot more in the works!

Whatever your traditions are, we hope you have a wonderful holiday season. See you in 2015.

Eric

Posted in Android, BatteryXPRT 2014 for Android, BenchmarkXPRT development community, CES, CrXPRT, HDXPRT, Mobile World Congress, TouchXPRT 2014, WebXPRT 2015 |

A very good question

Eric Hale
By Eric Hale
on December 4, 2014

A couple of weeks ago, we released the second community preview of CrXPRT (CP2). One of the changes was to make CrXPRT handle its test results more like WebXPRT. It now automatically uploads some test data to the PT servers, so that we can use it to improve future versions of the benchmark. Like WebXPRT, no personally identifying information is collected, and the results are never made public without your permission.

This week we got a question about the new results collection in CP2. The member wanted to know if the uploading of the results affected the battery life score at all. It does not. The results collection happens after the test. If you’re doing a battery rundown test, the results aren’t uploaded until the device boots the next time. I’m sorry we didn’t make that clearer in the documentation.

When we release a community preview, we commit that the results from that preview will be comparable to any subsequent previews and to the general release. In the case of CP2, we compared the results from CP2 to those from CP1 to make sure that the results stayed comparable. We take this commitment very seriously. When you test with a community preview, you can be sure that the results will still be good when the general release comes out.

In other news, we released a patch for BatteryXPRT yesterday. As we explained in the last blog post, this new version of BatteryXPRT is much more stable on devices running Android 5.

Have any other good questions? Send them our way.

Eric

Comment on this post in the forums

Posted in Android, BatteryXPRT 2014 for Android, Chrome OS, Collaborative benchmark development, Community Preview, CrXPRT, WebXPRT |

An update for BatteryXPRT 2014 for Android

Eric Hale
By Eric Hale
on December 1, 2014

For those of you in the US, I hope you had a happy Thanksgiving.

A couple of weeks ago, we updated MobileXPRT so that it would run correctly on Android 5 (Android L) devices. Later this week, we will be updating BatteryXPRT as well. The current BatteryXPRT build doesn’t work with Android 5 due to failures of the Create Slideshow workload. This is the same workload failure we fixed with an updated MobileXPRT build.

The new build greatly improves things. We successfully tested the new build on an array of Android 5 and 4.4 devices with no failures. However, we have seen the new build of BatteryXPRT return intermittent errors during the Create Photo Collage and Apply Photo Effects workloads when testing on the Nexus 9.

Because the error on the Nexus 9 is intermittent, it’s possible to get results from some runs. These results are still valid.

We haven’t seen this failure on any device other than the Nexus 9, but cannot guarantee that there aren’t others. If you see any other failures, or have any questions, please contact us.

We’re currently working on a solution for this problem. We will, as always, keep you informed.

Eric

Comment on this post in the forums

Posted in Android, Battery life, BatteryXPRT 2014 for Android, Collaborative benchmark development, MobileXPRT |

A new model

Eric Hale
By Eric Hale
on November 20, 2014

Today we released a new community preview for CrXPRT (CP2). It contains several enhancements and UI changes. It also fixes the problem we discussed in last week’s blog, so CP2 will now correctly execute a rundown test on a system using Chinese language settings.

The most exciting enhancement is that CP2 will let you test battery life without having to put the device into developer mode.  The new Battery Status API, which became available with Chrome 38, makes this possible.

Another enhancement is that results submission is now integrated into the benchmark UI. CrXPRT follows the WebXPRT model, with you having opt-in control over whether the results are published on the PT Web site. However, CrXPRT goes further by allowing you the option to upload more extensive disclosure information, and the option to download results as a text file.

These and other enhancements are covered in more detail in the CrXPRT forum and the user manual. Because this is a community preview, you have to be a community member to download it. However, joining is very easy.

We hope you like the new features in CrXPRT. Let us know what you think!

Eric

Discuss CrXPRT CP2 in the forums

Posted in Chrome OS, Collaborative benchmark development, Community Preview, CrXPRT |

The value of diversity

Eric Hale
By Eric Hale
on November 13, 2014

It’s great to hear from community members who are using the XPRTs. Having a pool of users with diverse viewpoints is critical for successful software development. While we try very hard to find problems, there will always be cases we miss.

For example, we talked recently about the challenges of working in a multilingual world. Just this week, Acer reported an interesting bug in the CrXPRT community preview. If your language is set to Chinese, the battery test will estimate the battery life as expected. However, a rundown test will return the message “Cannot calculate total run time.” This does not happen when the language is set to English.

We have verified this problem. In the next week or so, we’ll be releasing a second community preview containing a fix for this and a few other issues.

We’ve talked before about the difficulties of maintaining software in a fast moving world. Late last week, we received a new Nexus 9. It encountered a problem during the MobileXPRT tests, so we fixed the problem and will be releasing a patched build tomorrow.

Don’t forget, as we said last week, we’ll be releasing the WebXPRT 2015 design overview tomorrow. We are looking forward to the feedback from all our community members.

Eric

Comment on this post in the forums

Posted in Android, Chrome OS, CrXPRT, MobileXPRT, WebXPRT, WebXPRT 2015 |

Upcoming experiments

Eric Hale
By Eric Hale
on November 6, 2014

Next week, we’ll be releasing the design overview for WebXPRT 2015. WebXPRT 2013 has been an enormous success, having been run tens of thousands of times.

One of the big improvements we are considering for WebXPRT 2015 is adding experimental tests. A big reason for WebXPRT’s success is that it runs on almost every Web-enabled device. We consider it essential to preserve this broad compatibility. However, there are interesting Web technologies that simply are not available on all devices.

Our proposal is to add experimental tests to WebXPRT. These tests would be optional and would not be included in the Overall score, so WebXPRT would still be able to compare the performance of widely different devices. We are looking at technologies such as Web Workers, WebGL, and pre-compiled JavaScript (asm.js).

In addition to adding experimental tests, we are looking at ways to improve the UI, add automation, add new tests, update old tests, and more!

If you are a community member, you’ll get a notice when the overview is available. We will definitely want to know what you think! If you are not a member, it’s a great time to join.

If you have any thoughts on these ideas, or have ideas of your own, please let us know!

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT development community, Collaborative benchmark development, Performance benchmarking, WebXPRT, WebXPRT 2015, What makes a good benchmark? |

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x