Category: Android

Has it only been 2 weeks?

on January 16, 2014

2014 has barely started, but things are already popping!

The new battery test has a name: BatteryXPRT for Android. We are busy at work on the latest member of the BenchmarkXPRT family. The comment period for BatteryXPRT for Android ended last Friday. We greatly appreciate the comments we received on the RFC. We used them to shape the RFC into a design document, which is available on the member forum now.

In other news, we’re testing what we hope will be the TouchXPRT 2014 community preview (CP) now. Testing is going well and we will be releasing the TouchXPRT 2014 CP to the community soon. As I said in TouchXPRT 2014, we’ve revamped the tests. They are more visual and more demanding than the old tests, and they introduce new types of work. The tests look great!

Speaking of looking great, we announced a while back that we were doing a UI redesign across the XPRT benchmarks. Unfortunately, that effort was not quite in sync with the release of the TouchXPRT 2014 CP, so the CP will use the older style UI. We will update the UI for the final version, but doing so will not affect the results.

Remember that you have to belong to the community to get the community preview. If you’re still not a member, come join us.

A new member of the family and a new version of an old friend. 2014 is off to a great start.

Eric

Comment on this post in the forums

Posted in Android, Battery life, Benchmarking, BenchmarkXPRT development community, Collaborative benchmark development, TouchXPRT, TouchXPRT release cycle |

Thankful for consumers

By Bill Catchings

on November 27, 2013

This week in the US occur a pair of days that push in opposite directions—Thanksgiving and Black Friday. The former implores us to be thankful and the latter to consume. It seems like Black Friday is surpassing Thanksgiving in terms of importance, or at least media coverage.

I don’t particularly want to fight the Black Friday crowds, but I love reading all the ads. In them, I see all manner of consumer devices from smartphones to 4K TVs to computers to tablets. I see lots of cool devices I might want, but little help in choosing between them.

Which of the many Android smartphones will provide me with a better experience? Are any of the TVs really capable of browsing the Web or is that just a checkbox item? Is this notebook computer the best choice for creating the complex content my Web site needs? Which of these Windows 8.1 tablets will best let me manipulate my photos and videos?

I see consumers in need of tools like the XPRTs to help select products. Their ability to help uncover and understand the facts that differentiate products can be an important part of the buying process. For that, I’m thankful! My hope is that over the coming months, the next version of each of the XPRTs will make great strides in helping buyers make more informed decisions when choosing between products.

I’m also thankful to the community that makes the XPRTs possible. And, to the folks who work hard making the devices we all want to buy on Black Friday.

Finally, thanks to all of you who read this blog. Have a great Thanksgiving!

Bill

Comment on this post in the forums

Posted in Android, BenchmarkXPRT |

Staying out in the open

By Eric Hale

on October 3, 2013

Back in July, Anandtech publicized some research about possible benchmark optimizations in the Galaxy S4. Yesterday, Anandtech published a much more comprehensive article, “The State of Cheating in Android Benchmarks.” It’s well worth the read.

Anandtech doesn’t accuse any of the benchmarks of being biased—it’s the OEMS who are supposedly doing the optimizations. I will note that none of the XPRT benchmarks are among the whitelisted CPU tests. That being said, I imagine that everyone in the benchmark game is concerned about any implication that their benchmark could be biased.

When I was a kid, my parents taught me that it’s a lot harder to cheat in the open. This is one of the reasons we believe so strongly in the community model for software development. The source code is available to anyone who joins the community. It’s impossible to hide any biases. At the same time, it allows us to control derivative works. That’s necessary to avoid biased versions of the benchmarks being published. We think the community model strikes the right balance.

However, any time there is a system, someone will try to game it. We’ll always be on the lookout for optimizations that happen outside the benchmarks.

Eric

Comment on this post in the forums

Posted in Android, Benchmark metrics, Benchmarks in general, BenchmarkXPRT development community, Collaborative benchmark development, MobileXPRT |

Time to Play

By Eric Hale

on August 15, 2013

As Bill said in MobileXPRT 2013 lives!, we released MobileXPRT 2013 to the public on Monday. Since that time, we have published MobileXPRT as a free app in the Google Play Store. You can find it by searching for “mobilexprt” or you can get it here. Downloading the app from the store simplifies the installation, but in all other regards, it’s the same application that you can download from the MobileXPRT site.

We will make the source for the release available to the community in a few days.

We have some MobileXPRT results on the Web site. Some of these results are from our own testing and some are from published articles like this one from Tom’s Hardware. We will be adding more results from devices we test and looking for more articles as time goes on.

We’d love to add your results as well. You can find out how to submit your results here. If you have a blog or publish your own reviews, send us the link. If we include your results, we will link back to your site.

Now on to the next great thing!

Eric

Comment on this post in the forums

Posted in Android, MobileXPRT |

A new version and a new video

By Eric Hale

on June 27, 2013

As we said in It’s finally here!, the Android ecosystem is very diverse and we knew that MobileXPRT might have problems on some devices. So far, we have encountered a couple of issues:

The Slideshow test failed during post validation check on the Onda tablet.
The Zoom-n-pinch test was unstable, failing part of the time, on the Zopo phone

On Friday, we will be releasing MobileXPRT 2013 community preview 1.1 (CP1.1), which fixes these problems. The results from version CP1.1 are comparable to the current community preview. If you have any problems, or questions, please e-mail us at benchmarkxprtsupport@principledtechnologies.com

We will release the source for CP1.1 next week.

In other news, we released a new video this week, BenchmarkXPRT: It’s not a benchmark. In this video, Bill answers some common questions about the BenchmarkXPRT Development Community. If you’ve ever wondered what exactly BenchmarkXPRT is, or why the world needs more benchmarks, this is the video for you.

Eric

Comment on this post in the forums

Posted in Android, BenchmarkXPRT, BenchmarkXPRT development community, MobileXPRT, MobileXPRT 2013 source, Video |

Chaos and opportunity

By Eric Hale

on June 13, 2013

With both E3 and Apple’s WWDC happening this week, there’s been a lot of news. There’s also been a lot of hyperbolic commentary. I am not about to get into the arguments about the PS4 vs. the Xbox One or iOS 7 vs. Android.

It was Tim Cook’s presentation at WWDC that really got my attention. It’s unusual in an executive presentation to focus so much attention on a particular competitor, but Android was clearly on his mind. At one point, he focused harsh attention on fragmentation in the Android market, calling it “terrible” for developers. You can see the video here, at about 74 minutes.

As we saw in the 90s, chaos can breed innovation. At that time, the paradigm was that Macs always worked, but if you wanted the most advanced hardware, you should get a PC. I remember the editors at MacWorld, who deeply, truly loved the Mac, lusting over the (by the standards of the time) small, light, cheap notebooks PC users could get.

That being said, we understand the challenges of developing in the Android market. As I said in It’s finally here!, the Android ecosystem is sufficiently diverse that we know the benchmark will encounter configurations we’ve not seen before. If you have any problems with the MobileXPRT CP, please let us know at benchmarkxprtsupport@principledtechnologies.com. We want the benchmark to be the best it can be.

Eric

Comment on this post in the forums

Posted in Android, BenchmarkXPRT development community, History of benchmarking, History of computers, MobileXPRT |

Category: Android

Has it only been 2 weeks?

Thankful for consumers

Staying out in the open

Time to Play

A new version and a new video

Chaos and opportunity

Check out the other XPRTs: