Category: Community Preview

Things are looking good!

on October 29, 2015

It’s been a couple of weeks since we talked about TouchXPRT 2016. I’m happy to say that we have made great progress since then.

The UI is adapting to the different form factors very well. We’ve tested the resizing on phones and on tablets with the screen snapped to half and quarter screen. While a couple of UI elements still need work, we’ve had no problems running the tests. Here’s how TouchXPRT looks on a phone:

We have also greatly simplified installing the application on phones, so we’re no longer concerned about that.

All in all, things are looking good for releasing the community preview soon. However, we’re going to wait until we can test on the Windows 10 Fall Update (Threshold Build 2). That’s supposed to be released on November 2. We’re not expecting any problems with the Fall Update, but it’s always to wise to check.

November can’t come quickly enough!

Eric

Posted in Benchmarking, BenchmarkXPRT, Collaborative benchmark development, Community Preview, TouchXPRT, TouchXPRT 2016, TouchXPRT development process, TouchXPRT release cycle |

Upping our game

By Eric Hale

on August 27, 2015

As we wrote last week, we’re releasing MobileXPRT 2015 to the public tomorrow. Thanks to everyone who helped make the community preview a success!

We’re working on the TouchXPRT 2016 design document and will make it available for the community to review soon.

As you know, we’re always investigating initiatives that could improve our game. We’re continuing to investigate creating experimental tests for future XPRTs. Experimental tests will allow us to maintain broad compatibility for each XPRT tool while providing testers with an opportunity to evaluate cutting-edge technologies.

Another initiative involves looking for new partnerships with people who are not yet part of the community, but could add valuable input to the development process. It’s too soon to say much more about this, but we’re having fruitful conversations and hoping that these partnerships will grow the community even more!

If you have ideas about experimental tests, improving the XPRTs, or expanding the community, please let us know.

Eric

Posted in Android, Collaborative benchmark development, Community Preview, Future of performance evaluation, MobileXPRT, MobileXPRT 2015, TouchXPRT, TouchXPRT 2016, What makes a good benchmark? |

MobileXPRT 2015 Community Preview is here!

By Eric Hale

on August 14, 2015

Today we are releasing the MobileXPRT 2015 Community Preview (CP1). MobileXPRT 2015 is compatible with systems running Android 4.4 and above. It is a 64-bit application, but will work on both 32-bit and 64-bit hardware.

MobileXPRT 2015 includes the same performance workloads as MobileXPRT 2013, but not the UX Tests. Should you need the UX tests, MobileXPRT 2013 will continue to be available.

Because this is a community preview, you have to be a community member to download it. Members may download the preview from the MobileXPRT tab in the Members’ Area. There are two options for downloading:

MobileXPRT 2015 CP1 application and test data, 249 MB
MobileXPRT 2015 CP1 application only, 18 MB (downloads test data the first time it’s run)

After trying out CP1, please send in your comments. Either post them to the forum or mail them to BenchmarkXPRTsupport@principledtechnologies.com. If you send us information that’s relevant to the entire community, we may post an anonymous version of your comments to the forum.

Thanks for your participation in the community!

Posted in Android, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, MobileXPRT, MobileXPRT 2015 |

It’s almost here!

By Eric Hale

on August 13, 2015

This Friday, we will be releasing the community preview of MobileXPRT 2015. BenchmarkXPRT Development Community members will be able to download the installation APK from the MobileXPRT tab in the Members’ Area. We are happy to say that, although we thought that MobileXPRT 2015 would only run on Android 5.0 and above, we have been able to extend its compatibility back to Android 4.4 (KitKat).

As we’ve said, the new MobileXPRT is a 64-bit application. Although it will work on both 32-bit and 64-bit hardware, the code will automatically select 64-bit when available. If you want to compare 32-bit performance to 64-bit on a single device, MobileXPRT 2013 will still be available. It’s not a problem to have both versions of MobileXPRT installed at the same time, and results from the two versions of the benchmark are comparable. On the same note, remember that the UX tests are no longer available in MobileXPRT 2015, but they are available as part of MobileXPRT 2013 if you need them.

As always, the results from the community preview will be comparable to the general release.

We hope you enjoy the new MobileXPRT, and we look forward to seeing your results!

Eric

Posted in BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, MobileXPRT, MobileXPRT 2015 |

An update on MobileXPRT 2015

By Eric Hale

on July 23, 2015

As I mentioned last week, we’ve been testing MobileXPRT 2015.

We’ve rebuilt the current MobileXPRT as a 64-bit application. This means that MobileXPRT 2015 will only run on Android 5.0 and above. For this reason, we’ll make MobileXPRT 2013 available for testing older versions of the operating system. Because the workloads haven’t changed, you’ll be able to compare results from MobileXPRT 2013 to those from MobileXPRT 2015.

Another change to MobileXPRT 2015 will be the removal of the UX tests. While these tests were useful on lower-end devices when MobileXPRT 2013 came out, they don’t distinguish modern devices. However, should you need them, these tests will continue to be available as part of MobileXPRT 2013.

There are a few other features that we’ll be talking about over the next few weeks. We’ll be releasing the MobileXPRT 2015 community preview before Android M is released. We’re looking forward to see how Android M performs!

Eric

Posted in Android, Android L, Android M, Collaborative benchmark development, Community Preview, MobileXPRT 2015 |

Looking at the data

By Eric Hale

on April 30, 2015

We’re planning the general release of WebXPRT 2015 for late next week. The testing is looking good and the response has been positive.

We’ve been looking at the hundreds of runs in the database for the community preview. As we’ve said before, we’ve been looking at the information from the JavaScript navigator object in the hope that we could improve the disclosure information from WebXPRT 2015. However, the information is not reliable enough for us to use it at this time. Hopefully, that will improve in the near future.

For now, we will continue to use the information from the user agent string. We’ve discussed the user agent string before. It does give us some information about the device, although not as much as we are able to gather in some of the XPRTs.

Looking at the data, the most common OS has been Windows. This may be in part because you needed to be logged in to run the CP. However, Android devices represented over a third of the runs, and Chrome OS represented about 25 percent of the runs. While we had healthy numbers of iOS devices, there were only a handful of Mac OSX runs. Chrome was the most common browser. Other browsers identified themselves as Safari, Firefox, Opera, and MS IE.

As you can see, the new WebXPRT continues the tradition that WebXPRT 2013 started of running everywhere. We can’t wait to make it available to general public!

Eric

Posted in BenchmarkXPRT development community, Community Preview, WebXPRT, WebXPRT 2013 release, WebXPRT 2015 |

Category: Community Preview

Things are looking good!

Upping our game

MobileXPRT 2015 Community Preview is here!

It’s almost here!

An update on MobileXPRT 2015

Looking at the data

Check out the other XPRTs: