Category: Android

Upping our game

on August 27, 2015

As we wrote last week, we’re releasing MobileXPRT 2015 to the public tomorrow. Thanks to everyone who helped make the community preview a success!

We’re working on the TouchXPRT 2016 design document and will make it available for the community to review soon.

As you know, we’re always investigating initiatives that could improve our game. We’re continuing to investigate creating experimental tests for future XPRTs. Experimental tests will allow us to maintain broad compatibility for each XPRT tool while providing testers with an opportunity to evaluate cutting-edge technologies.

Another initiative involves looking for new partnerships with people who are not yet part of the community, but could add valuable input to the development process. It’s too soon to say much more about this, but we’re having fruitful conversations and hoping that these partnerships will grow the community even more!

If you have ideas about experimental tests, improving the XPRTs, or expanding the community, please let us know.

Eric

Posted in Android, Collaborative benchmark development, Community Preview, Future of performance evaluation, MobileXPRT, MobileXPRT 2015, TouchXPRT, TouchXPRT 2016, What makes a good benchmark? |

We’ve been thinking

By Eric Hale

on August 20, 2015

Last week, we released the community preview for MobileXPRT 2015, and things are looking great. We anticipate a general release next week, probably on the 28th. Many thanks to the community members who are making this preview a success.

Of course, MobileXPRT isn’t the only thing we’ve been working on. Development of TouchXPRT 2016 is in full swing. As we noted in an earlier blog, Microsoft did not release the Windows 10 SDK until the end of July, but this delay has been a bit of a blessing in disguise. The extra time has allowed us to consider additional ways to improve TouchXPRT. As always, we want to hear the community’s input, so please send any comments or suggestions our way. We’ll add them to our ideas and publish a design document in the next couple of weeks.

Like the XPRT previews, the design document is only available to members of the BenchmarkXPRT Development Community. If you’re wondering what membership entails, this video will show you everything you want to know.

Eric

Posted in Android, BenchmarkXPRT, BenchmarkXPRT development community, MobileXPRT, MobileXPRT 2015, TouchXPRT, TouchXPRT 2016, Windows 10 |

MobileXPRT 2015 Community Preview is here!

By Eric Hale

on August 14, 2015

Today we are releasing the MobileXPRT 2015 Community Preview (CP1). MobileXPRT 2015 is compatible with systems running Android 4.4 and above. It is a 64-bit application, but will work on both 32-bit and 64-bit hardware.

MobileXPRT 2015 includes the same performance workloads as MobileXPRT 2013, but not the UX Tests. Should you need the UX tests, MobileXPRT 2013 will continue to be available.

Because this is a community preview, you have to be a community member to download it. Members may download the preview from the MobileXPRT tab in the Members’ Area. There are two options for downloading:

MobileXPRT 2015 CP1 application and test data, 249 MB
MobileXPRT 2015 CP1 application only, 18 MB (downloads test data the first time it’s run)

After trying out CP1, please send in your comments. Either post them to the forum or mail them to BenchmarkXPRTsupport@principledtechnologies.com. If you send us information that’s relevant to the entire community, we may post an anonymous version of your comments to the forum.

Thanks for your participation in the community!

Posted in Android, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, MobileXPRT, MobileXPRT 2015 |

An update on MobileXPRT 2015

By Eric Hale

on July 23, 2015

As I mentioned last week, we’ve been testing MobileXPRT 2015.

We’ve rebuilt the current MobileXPRT as a 64-bit application. This means that MobileXPRT 2015 will only run on Android 5.0 and above. For this reason, we’ll make MobileXPRT 2013 available for testing older versions of the operating system. Because the workloads haven’t changed, you’ll be able to compare results from MobileXPRT 2013 to those from MobileXPRT 2015.

Another change to MobileXPRT 2015 will be the removal of the UX tests. While these tests were useful on lower-end devices when MobileXPRT 2013 came out, they don’t distinguish modern devices. However, should you need them, these tests will continue to be available as part of MobileXPRT 2013.

There are a few other features that we’ll be talking about over the next few weeks. We’ll be releasing the MobileXPRT 2015 community preview before Android M is released. We’re looking forward to see how Android M performs!

Eric

Posted in Android, Android L, Android M, Collaborative benchmark development, Community Preview, MobileXPRT 2015 |

Get the 411 on WebXPRT 2015

By Eric Hale

on July 16, 2015

We’re publishing the WebXPRT 2015 white paper tomorrow. It contains lots of information about WebXPRT 2015 that you won’t find anywhere else. In addition to explaining what the tests do, the test descriptions detail the particular aspects of Web browsing performance that the test stresses.

The white paper also contains a list of the third-party libraries that WebXPRT uses, what those libraries do, and where you can find them. It also lists the HTML 5 capabilities that WebXPRT 2015 requires. And, as in past XPRT white papers, we include a spreadsheet showing how the benchmark calculates its results. There’s a lot of information, so check it out!

Meanwhile, we’re hard at work on MobileXPRT 2015. The Android M Developer Preview 2 hasn’t presented any new problems, so things are looking good. I expect to have more details about the upcoming MobileXPRT 2015 community preview soon!

Eric

Posted in Android, Android M, MobileXPRT, MobileXPRT 2015, WebXPRT 2015, White papers |

Rolling with the changes

By Eric Hale

on June 4, 2015

While WebXPRT 2015 has been running fine on earlier beta versions of Windows 10, we have found a problem on some recent versions. Starting with build 10122, the Local Notes test hangs when using the Microsoft Edge browser. (Note: This browser still identifies itself as Spartan in the builds we have seen.) Chrome and Firefox on Windows 10 have successfully run WebXPRT 2015, so the problem appears to be restricted to Edge/Spartan.

Because WebXPRT ran successfully on earlier builds of Windows 10, we are hoping that upcoming builds will resolve the problem. However, we have been investigating the issue in case there is something we can address. The problem is that the encrypted strings that the test is trying to write to LocalStorage are not being written correctly. Non-encrypted strings are being written correctly.

As soon as the problem gets resolved, we’ll let you know.

In other news, we’ve been looking at Android M. There are a lot of interesting changes coming, such as the difference in the way that Android M manages app permissions. We’ve decided to delay releasing the design document for the next version of MobileXPRT so that we can make sure that the design deals with these changes appropriately.

Eric

Posted in Android, Benchmarking, Chrome OS, Collaborative benchmark development, MobileXPRT, WebXPRT, WebXPRT 2015, Windows 10 |

Category: Android

Upping our game

We’ve been thinking

MobileXPRT 2015 Community Preview is here!

An update on MobileXPRT 2015

Get the 411 on WebXPRT 2015

Rolling with the changes

Check out the other XPRTs: