Category: Community Preview

WebXPRT 2015 Community Preview is available!

on March 27, 2015

As we said yesterday, we’re releasing the community preview of a WebXPRT 2015 Community Preview. Members of the BenchmarkXPRT community can now run the community preview and publish results.

Run the community preview here.

After trying out the community preview, please send in your comments. Either post them to the forum or mail them to benchmarkxprtsupport@principledtechnologies.com. If you mail information that’s of interest to the entire community, we may post an anonymous version of the comments to the forum.

Thanks for being part of the community,

Eric

Posted in Automation, BenchmarkXPRT development community, Chinese, Community Preview, WebXPRT, WebXPRT 2015 |

Finally!

By Eric Hale

on March 26, 2015

We’re releasing the community preview of WebXPRT 2015 tomorrow. We’re very excited that it’s finally here. In the past few weeks, we’ve discussed some of the new features in WebXPRT 2015, such as test automation, its new and improved tests, and its Chinese UI. We think you’re really going to enjoy the new WebXPRT.

The design document (login required) specifies that WebXPRT will contain an experimental workload. That workload is not in the community preview, although we plan for it to be in the general release. However, because any experimental workloads are not included in the overall score, this will not affect any results you generate.

We’re also investigating the use of the JavaScript navigator object to improve system disclosure, but we are still determining if we can get reliable enough information to display. So this information is not displayed in the community preview.

As with all the BenchmarkXPRT community previews, we’re not putting any publication restrictions on this preview release. Test at will, and publish your findings. We guarantee the results for the community preview will be comparable to results from the general release.

If you’re not a community member, join us and check out the new WebXPRT.

Eric

Posted in Automation, BenchmarkXPRT development community, Chinese, Collaborative benchmark development, Community Preview, WebXPRT, WebXPRT 2015 |

More power, more control

By Eric Hale

on March 19, 2015

As I said last week, the community preview for WebXPRT 2015 is coming up soon. One of the changes that will be exciting to anyone who does a lot of testing is that we made it simpler to automate WebXPRT tests.

WebXPRT 2015 will let you automatically select any set of tests you want to run. However, as always, you must run the entire suite of tests to get an overall score. Although the community preview will not include any experimental tests, the automation includes control for those future tests as well.

You may choose from several output formats: HTML table, XML, and CSV, or you can download the results as a text file.

Using the automation is simple: you just append the desired test parameters to the end of the URL. The format allows you to mix and match a lot of options, while still being very concise. The details will be in the release notes.

As people who test a lot of devices, we are very excited about this new capability.

Eric

Posted in Automation, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, Future of performance evaluation, Performance benchmarking, WebXPRT, What makes a good benchmark? |

A first look

By Eric Hale

on January 29, 2015

The beta build of WebXPRT 2015 is complete. Although the benchmark is still under development, it looks great!

As we described in the design document, it includes a number of improvements over WebXPRT 2013. There are more, and more demanding, tests. The UI has a cleaner, more modern look, and includes a couple of improvements people have been asking for. There is now descriptive text so that you know what’s happening during the test. Also, a progress indicator in the upper right corner shows you how close the test is to completion. Although the screens aren’t final, here’s a preview of one of the new DNA Sequence Analysis test screens:

One of the great virtues of WebXPRT is that it runs on almost anything. However, that level of compatibility has limited the range of devices WebXPRT can test. WebXPRT 2015 introduces a set of experimental tests. Running these tests is optional and their results are not included in the Overall Score. We hope that folks will use them to evaluate both new technologies and platform-specific technologies. The first Community Preview will include a WebGL-based 3D experimental test. We expect to add new experimental tests over time and we’re hoping community members will contribute tests or ideas for them.

I’m very excited to see the new WebXPRT. We’re testing now, and I hope to be able to let you know when the Community Preview will be available soon.

Eric

Posted in BenchmarkXPRT development community, Community Preview, WebGL, WebXPRT, WebXPRT 2015 |

Timing is everything

By Eric Hale

on December 11, 2014

We have a couple instances of interesting timing this week.

A few weeks ago, we released the second community preview for CrXPRT (CP2). It’s been doing very well and we’re planning to release CrXPRT to the public next week. Many thanks to all the community members who have helped make this possible.

Because the release is so close to the end of the year, the general release will be CrXPRT 2015. However, the community previews were dated 2014, which is potentially confusing. We want to reassure you that the results will still be comparable. As we discussed last week, for any of the XPRT benchmarks, the results from the community previews are always comparable to those from the final release.

In other news, we fixed the intermittent problem BatteryXPRT was having on the Nexus 9. The tests are looking good. In a bit of really good timing, this week Google started rolling out Android 5.01. We will be going back and retesting the devices with the newer version of the OS. We don’t expect any new problems, but it’s very nice that we got to check it out before we released the patch.

Eric

Comment on this post in the forums

Posted in Android, BatteryXPRT 2014 for Android, Chrome OS, Community Preview, CrXPRT |

A very good question

By Eric Hale

on December 4, 2014

A couple of weeks ago, we released the second community preview of CrXPRT (CP2). One of the changes was to make CrXPRT handle its test results more like WebXPRT. It now automatically uploads some test data to the PT servers, so that we can use it to improve future versions of the benchmark. Like WebXPRT, no personally identifying information is collected, and the results are never made public without your permission.

This week we got a question about the new results collection in CP2. The member wanted to know if the uploading of the results affected the battery life score at all. It does not. The results collection happens after the test. If you’re doing a battery rundown test, the results aren’t uploaded until the device boots the next time. I’m sorry we didn’t make that clearer in the documentation.

When we release a community preview, we commit that the results from that preview will be comparable to any subsequent previews and to the general release. In the case of CP2, we compared the results from CP2 to those from CP1 to make sure that the results stayed comparable. We take this commitment very seriously. When you test with a community preview, you can be sure that the results will still be good when the general release comes out.

In other news, we released a patch for BatteryXPRT yesterday. As we explained in the last blog post, this new version of BatteryXPRT is much more stable on devices running Android 5.

Have any other good questions? Send them our way.

Eric

Comment on this post in the forums

Posted in Android, BatteryXPRT 2014 for Android, Chrome OS, Collaborative benchmark development, Community Preview, CrXPRT, WebXPRT |

Category: Community Preview

WebXPRT 2015 Community Preview is available!

Finally!

More power, more control

A first look

Timing is everything

A very good question

Check out the other XPRTs: