BenchmarkXPRT Blog banner

Category: Collaborative benchmark development

Looking ahead

It’s only been a couple of weeks since we announced a cross-platform XPRT. It’s still early days, but we’ve already started getting ideas from vendors and media—from both people within the community and those who have not yet joined. We’re incorporating these ideas into our investigations, and plan to be sending a design document for the community to critique in a few weeks.

However, we are always looking ahead and Bill’s trip to IDF16 got us thinking about future benchmarks. Virtual reality is obviously going to be big. Bill said that he thinks he saw more things using the Oculus Rift than there are Oculus Rifts in the world! The Internet of Things has been ramping up for a while now, and shows no sign of slowing down. Computer vision is another emerging area, one with many possible applications. There are a lot of exciting possibilities!

As always, we want to know what you think. What upcoming technologies are you excited about? What would like to see in these benchmarks? Please let us know!

Eric

One benchmark to test them all

It’s no secret that the XPRTs are a great way to get device results you can count on. Tens of thousands of people over six continents have used the XPRTs to help them make smart buying choices, and over a thousand media outlets have quoted XPRT results when reporting on the hottest tech. WebXPRT has always been the “go to” XPRT, because you can use it to test the widest range of devices. WebXPRT runs in the browser, however, so browser performance influences the results.

For a long time, our members and others have asked for a tool that would let you compare application performance on any type of device. People want a cross-platform XPRT that runs on devices the same way apps do.

We’re excited to announce that we’re going to create just that tool! Specifically, we’re going to create a version of MobileXPRT that runs on Android, iOS, and Windows.

This will not be easy. At one point in my career, I was in charge of a group that ported applications between platforms, and I learned from hands-on experience that doing that job well is very difficult. It’s not enough to simply make the application run; it also has to run efficiently on each type of system. MobileXPRT works at the application level, so we’ll have to deal with the many differences in the operating system architectures and APIs. We’ll have to make sure the code runs well on all three target OSes.

We’re willing to do all this work because the need for such a tool has never been greater. More and more devices hit the market all the time, and choosing the ones you want is tougher than ever. iPhone or Android phone? Windows tablet, Android tablet, or iPad?

The coming MobileXPRT will let buyers around the world answer those questions.

We’re not going to do this work in isolation. We will reach out to the OS vendors, because we want their input, comments, and help. We’ll make the source available to them, and we welcome their critiques and guidance in creating the best possible version for each OS.

Of course, we very much want your input, too. Do you have any thoughts about what you’d like to see in a cross-platform XPRT? If so, let us know!

Bill

Last week in the XPRTs
We published the XPRT Weekly Tech Spotlight on the Apple iPhone SE.
We added one new MobileXPRT ’15 result.
We added seven new WebXPRT ’15 results.

Quarterly review

It’s been one of our busiest quarters ever! Here’s a quick review of what’s been happening:

The XPRTs were on the road a lot!

 
While I was at CES, I was lucky enough to be able to sit down and talk on-the-record with a couple of community members:

 
Many thanks to them for being so generous with their time and their insights.

We also gave folks a lot to look at:

 
That is a great start to the year, but we’re going to top it – Next week, we’ll kick off Q2 with one of our biggest announcements ever!

Eric

Women develop new perspectives for the XPRTs

Last weekend, we had the great privilege of co-hosting the first XPRT Women Code-a-Thon with the Seattle chapter of ChickTech. We couldn’t be happier with the results!

Our goal was to bring together a group of women and invite them to develop ideas for new device workloads—workloads that we might include in future versions of MobileXPRT and WebXPRT. The 20 participants—some working individually, and others working as teams—not only met that goal, they did a great deal more.

On the coding front, the participants achieved an impressive amount of work in a very short time. Though we awarded only three prizes, everyone generated interesting and useful ideas. Our prizes went to the following people:

1st place: Viveret Steele, for a 3D-modeling workload

2nd place: Annmarie Aidoo, for a geolocation workload

3rd place: Molly Fallen and Alex Trimble, for an audio-enhancement workload

These four people went home with checks, but winning wasn’t what motivated anyone to participate. Everyone was excited about developing software and working with others. The social side of the event proved to be as meaningful as the technical. People talked, formed friendships and mentoring relationships, and discussed seeking other events like this one. Two people said the event changed their lives.

In the weeks ahead, we’ll be sharing some more information about the event. In the meantime, we’re proud to have been part of it.

Jennie Faries

Last week in the XPRTs

We published the XPRT Weekly Tech Spotlight on the Microsoft Surface 3.
We added two new CrXPRT ’15 results.
We added two new MobileXPRT ’13 results.
We added six new WebXPRT ’15 results.

The big weekend is almost here!

A few weeks ago, I talked about the XPRT Women Code-a-Thon. Well, all the work that we and our friends at ChickTech Seattle have done is about to pay off! On Saturday, March 12, dozens of women will go to the Sole Repair Shop in Seattle for two days of coding, good food, and networking opportunities.

The goal of each team at the code-a-thon is to create a new workload that might be included in a future version of WebXPRT or MobileXPRT. Judges will award prizes to the top three workloads: $2,500 for first place, $1,500 for second place, and $1,000 for third place. I can’t wait to see the winning workloads!

We’re very fortunate to have Kristin Toth Smith as the keynote speaker. She is an avid supporter of women in tech, current COO of Dolly, and former CEO of Code Fellows.

It should be a great time for all. If you or anyone you know can get to Seattle this weekend, registration is still open.

Eric

Last week in the XPRTs
We published the XPRT Weekly Tech Spotlight on the ASUS ZenFone 2.
We added one new BatteryXPRT ’14 result.
We added five new WebXPRT ’15 results.

Is it hot in here?

One of the great meetings I had at CES was with another community member, Patrick Chang, Senior System Engineer in the Tablet Performance group at Dell.  I was glad to hear that, when he tests his devices, Patrick makes frequent use of TouchXPRT.

While TouchXPRT stresses the system appropriately, Patrick’s job requires him to understand not only how well the device performs, but why it performs that way. He was wondering what we could do to help him correlate the temperature, power consumption, and performance of a device.

That information is not typically available to software and apps like the XPRTs. However, it may be possible to add some hooks that would let the XPRTs coordinate with third-party utilities and hardware that do.

As always, the input from the community guides the design of the XPRTs. So, we’d love to know how much interest the community has in having this type of information. If you have thoughts about this, or other kinds of information you’d like the XPRTs to gather, please let us know!

Eric

Last week in the XPRTs
We published the XPRT Weekly Tech Spotlight on the Apple iPad Pro.
We added one new BatteryXPRT ’14 result.
We added one new CrXPRT ’15 result.
We added one new MobileXPRT ’13 result.
We added four new WebXPRT ’15 results.

Check out the other XPRTs:

Forgot your password?