BenchmarkXPRT Blog banner

Category: WebXPRT

More power, more control

Eric Hale
By Eric Hale
on March 19, 2015

As I said last week, the community preview for WebXPRT 2015 is coming up soon. One of the changes that will be exciting to anyone who does a lot of testing is that we made it simpler to automate WebXPRT tests.

WebXPRT 2015 will let you automatically select any set of tests you want to run. However, as always, you must run the entire suite of tests to get an overall score. Although the community preview will not include any experimental tests, the automation includes control for those future tests as well.

You may choose from several output formats: HTML table, XML, and CSV, or you can download the results as a text file.

Using the automation is simple: you just append the desired test parameters to the end of the URL. The format allows you to mix and match a lot of options, while still being very concise. The details will be in the release notes.

As people who test a lot of devices, we are very excited about this new capability.

Eric

Posted in Automation, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, Future of performance evaluation, Performance benchmarking, WebXPRT, What makes a good benchmark? |

Getting closer

Eric Hale
By Eric Hale
on March 12, 2015

Mobile World Congress ended last week. If you’ve been following us on Twitter, you know that Bill had some really good meetings while he was there. That’s not to say that Bill didn’t have some fun too!

Bill MWC

Next, we go to Intel Developer Forum 2015 – Shenzhen!

While Bill has been out of the country, we’ve been continuing to work on WebXPRT 2015. We’ve talked before about some of the improvements the new WebXPRT: new and more-demanding tests, on-screen descriptions of the tests, a progress indicator, and more. We’ve also given the UI a sleeker, more modern look. Also, WebXPRT 2015 will be the second XPRT to include a Chinese UI.

WebXPRT '15 screenshot

Testing has been going well. We have a few final tweaks to make but are expecting to release the community preview for WebXPRT 2015 in the next week or two. We’re really excited! I can’t wait to hear what you think of it.

Eric

Posted in IDF, Mobile World Congress, Simplified Chinese, Trade Shows, WebXPRT |

A guest blog from Wilson Cheng at Acer Inc.

Wilson Cheng
By Wilson Cheng
on February 12, 2015

Two years ago, I got a job to evaluate cross-platform benchmarks for comparing Windows and Android.

Usually, Web-based benchmarks focus on only Java-script and HTML5 functionality. I found WebXPRT is easy to use and includes the tasks that you do every day. It reliably tests the user experience and gives a simple score for comparison. I suggested to our manager that we use it to evaluate our platform and he agreed.

Recently, Chromebooks have become very popular. Testing the battery life of Chromebooks has been difficult because you have needed to use the power_LoadTest image from the Chromium project or the test image from the Chrome OS Partners Web site to do power load testing. Without reimaging the system, you couldn’t test battery life. Because CrXPRT is designed to test battery life as well as performance on ChromeOS, this extra step is no longer needed. We can run CrXPRT without updating the test image, which is very useful in our battery life testing.

The advantage of XPRT benchmarks is that they are easy to use and give real-world results. The disadvantage is that they are not yet popular on review sites.

Posted in Android, BenchmarkXPRT, Chrome OS, Chromebooks, CrXPRT, Google Chrome, WebXPRT, What makes a good benchmark? |

A first look

Eric Hale
By Eric Hale
on January 29, 2015

The beta build of WebXPRT 2015 is complete. Although the benchmark is still under development, it looks great!

As we described in the design document, it includes a number of improvements over WebXPRT 2013. There are more, and more demanding, tests. The UI has a cleaner, more modern look, and includes a couple of improvements people have been asking for. There is now descriptive text so that you know what’s happening during the test. Also, a progress indicator in the upper right corner shows you how close the test is to completion. Although the screens aren’t final, here’s a preview of one of the new DNA Sequence Analysis test screens:

WebXPRT DNA

One of the great virtues of WebXPRT is that it runs on almost anything. However, that level of compatibility has limited the range of devices WebXPRT can test. WebXPRT 2015 introduces a set of experimental tests. Running these tests is optional and their results are not included in the Overall Score. We hope that folks will use them to evaluate both new technologies and platform-specific technologies. The first Community Preview will include a WebGL-based 3D experimental test. We expect to add new experimental tests over time and we’re hoping community members will contribute tests or ideas for them.

I’m very excited to see the new WebXPRT. We’re testing now, and I hope to be able to let you know when the Community Preview will be available soon.

Eric

Posted in BenchmarkXPRT development community, Community Preview, WebGL, WebXPRT, WebXPRT 2015 |

The busiest year ever!

Eric Hale
By Eric Hale
on December 25, 2014

As 2014 winds down, it’s a good time to look back at the year. And what a year it has been! This is the year that the XPRTs really went global. The benchmarks are in wider use than ever. The community continues to grow.

Here are a few of the big things that happened this year:

 

It’s going to be hard to top this year, but we are certainly going to try! There’s another video coming soon. We’re already working on WebXPRT 2015. (If you haven’t commented on the WebXPRT 2015 design document yet, it’s not too late.) We’re making plans for MWC 2015. And there’s a lot more in the works!

Whatever your traditions are, we hope you have a wonderful holiday season. See you in 2015.

Eric

Posted in Android, BatteryXPRT 2014 for Android, BenchmarkXPRT development community, CES, CrXPRT, HDXPRT, Mobile World Congress, TouchXPRT 2014, WebXPRT 2015 |

A very good question

Eric Hale
By Eric Hale
on December 4, 2014

A couple of weeks ago, we released the second community preview of CrXPRT (CP2). One of the changes was to make CrXPRT handle its test results more like WebXPRT. It now automatically uploads some test data to the PT servers, so that we can use it to improve future versions of the benchmark. Like WebXPRT, no personally identifying information is collected, and the results are never made public without your permission.

This week we got a question about the new results collection in CP2. The member wanted to know if the uploading of the results affected the battery life score at all. It does not. The results collection happens after the test. If you’re doing a battery rundown test, the results aren’t uploaded until the device boots the next time. I’m sorry we didn’t make that clearer in the documentation.

When we release a community preview, we commit that the results from that preview will be comparable to any subsequent previews and to the general release. In the case of CP2, we compared the results from CP2 to those from CP1 to make sure that the results stayed comparable. We take this commitment very seriously. When you test with a community preview, you can be sure that the results will still be good when the general release comes out.

In other news, we released a patch for BatteryXPRT yesterday. As we explained in the last blog post, this new version of BatteryXPRT is much more stable on devices running Android 5.

Have any other good questions? Send them our way.

Eric

Comment on this post in the forums

Posted in Android, BatteryXPRT 2014 for Android, Chrome OS, Collaborative benchmark development, Community Preview, CrXPRT, WebXPRT |

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x