Search Results for: webxprt

Three names, two hosts

on April 18, 2013

As Bill mentioned a couple of weeks ago in The Name Game, we’ve been considering changing the name of PhoneXPRT. The rationale for this is that the tests in PhoneXPRT are useful for a range of devices, from phones to tablets. We asked for your opinions about the name. After getting your input and talking amongst ourselves, we are considering three possibilities:

Leaving the name unchanged.
Changing the name to MobileXPRT. While this would convey the scope of the benchmark, some people thought the name might be too general.
Changing the name to TouchXPRT for Android. While there is some similarity between the tests in TouchXPRT and the ones we are developing for this benchmark, the two benchmarks would not initially be comparable.

Let us know what you think. We hope to settle the name question soon.

As we mentioned in Loose Ends, some users in mainland China are reporting extremely slow download times when running WebXPRT. We have set up a trial host for WebXPRT in Singapore to see if this improves the situation. Preliminary, US-based tests have seen no significant difference in scores when running from the Singapore host.

If you are in China and want to try running WebXPRT from the new host, you will find it at http://54.251.252.204/webxprt/. Please let us know your experience.

Eric

Comment on this post in the forums

Posted in Android, MobileXPRT, PhoneXPRT, WebXPRT |

Lies, damned lies, and statistics

By Eric Hale

on April 11, 2013

No one knows who first said “lies, damned lies, and statistics,” but it’s easy to understand why they said it. It’s no surprise that the bestselling statistics book in history is titled How to Lie with Statistics. While the title is facetious, it is certainly true that statistics can be confusing—consider the word “average,” which can refer to the mean, median, or mode. “Mean average,” in turn, can refer to the arithmetic mean, the geometric mean, or the harmonic mean. It’s enough to make a non-statistician’s head spin.

In fact, a number of people have been confused by the confidence interval WebXPRT reports. We believe that the best way to stand behind your results is to be completely open about how you crunch the numbers. To this end, we released the white paper WebXPRT 2013 results calculation and confidence interval this past Monday.

This white paper, which does not require a background in mathematics, explains what the WebXPRT confidence interval is and how it differs from the benchmark variability we sometimes talk about. The paper also gives an overview of the statistical and mathematical techniques WebXPRT uses to translate the raw timing numbers into results.

Because sometimes the devil is in the details, we wanted to augment our overview by showing exactly how WebXPRT calculates results. The white paper is accompanied by a spreadsheet that reproduces the calculations WebXPRT uses. If you are mathematically inclined and would like to suggest improvements to the process, by all means let us know!

Eric

Comment on this post in the forums

Posted in Benchmark metrics, WebXPRT, WebXPRT 2013 results, White papers |

The name game

By Bill Catchings

on March 29, 2013

There’s a lot going on in the world of the XPRTs. We’re working on HDXPRT 2013, writing white papers, building up our results database, and thinking ahead to the next versions of TouchXPRT and WebXPRT. At the same time, we are giving quite a bit of attention to PhoneXPRT.

As we said in the PhoneXPRT press release last month, PhoneXPRT will use the same kinds of realistic scenarios the other XPRT benchmarks do. These scenarios include tasks that people perform on phones as well as other mobile devices.

We’ve gone back and forth on the seemingly simple question of how to define what a phone is. At one point, defining a phone as a mobile device that you put to your ear seemed to suffice. As phones grow larger and tablets smaller, that doesn’t really hold up.

We also have a decent bit of interest in using the benchmark on Android-based tablets as well as phones. That seems like a good idea to us, as we are all for getting the most out of any benchmark.

One issue, however, is that PhoneXPRT is not a great name for a benchmark that may be commonly used on devices other than phones. So we’ve started thinking about what else we could call it.

As always, we look to the community. Do you think calling the benchmark PhoneXPRT would limit its usefulness for benchmarking tablets? Do you have any ideas for more inclusive names? We really need your feedback here and look forward to getting it. Please send suggestions to benchmarkxprtsupport@principledtechnologies.com or post your comments in the forum. Thanks!

Bill

Comment on this post in the forums

Posted in Android, BenchmarkXPRT, HDXPRT 2013, PhoneXPRT |

Loose ends

By Eric Hale

on March 21, 2013

As we mentioned last week, the Samsung debuted the Galaxy S4 this past week. It seems to have hit all the expectations – including eye-scrolling. Looking at the reviews, it’s somewhere between the greatest phone ever made and something not quite as good as the iPhone. I’m looking forward to seeing one for myself and hoping someone submits a WebXPRT score for it soon.

We’ll be releasing the HDXPRT 2013 design document tomorrow. As we’ve said, the number one comment has been that HDXPRT 2012 is too big and takes too long to run. We have put a lot of thought into how to trim HDXPRT 2013 and still keep the essential value of the benchmark. We’ve also received some other good feedback that we’re incorporating, such as making installing and running HDXPRT scriptable.

We’ve been doing some investigation during the RFC period, and we’ve encountered problems scripting some of the applications, notably iTunes and PowerDirector. We’re working to overcome these problems but if they prove to be insurmountable, we might have to change the list of applications.

Again, thanks to everyone who commented on the HDXPRT 2013 RFC.

We’ve learned that some WebXPRT users in mainland China are having problems with very slow downloads. The good news is that results from the runs are valid. However, we understand that this is frustrating and are investigating solutions. If you are in China and have experienced slow downloads, please send an e-mail to benchmarkxprtsupport@principledtechnologies.com. We would like your help in evaluating any solutions we come up with.

Speaking of WebXPRT, over the next few weeks we’ll be releasing several white papers that look at WebXPRT results in more depth. The first paper will explain the WebXPRT confidence interval and how it relates to run to run variability, which has been confusing to a number of people. The second paper will look at the effect on the browser on WebXPRT scores. The third paper will look at the influence of the operating system.

Eric

Comment on this post in the forums

Posted in HDXPRT 2012 RFC, HDXPRT 2013, HDXPRT 2013 design document, WebXPRT, WebXPRT 2013 results |

Soon, we’ll know the truth

By Eric Hale

on March 14, 2013

I’m writing this on the morning of one of the most anticipated events in tech, Samsung’s rollout of the new Galaxy S4 phone. There have been many leaks, and it will be interesting to see how many are true. One feature generating buzz would let you scroll just by moving your eyes. Samsung filed for the patent, but there are conflicting reports about whether eye scrolling made it into the Galaxy S4.

The leaked reports say that the U.S. version of the Galaxy S4 will use Qualcomm Inc.’s quad-core chip, while versions for other markets will use Samsung’s Exynos 5 Octa-core chip. As we continue development of PhoneXPRT, I will be very interested to see how the different processors stack up performance wise.

For browser-level performance, WebXPRT 2013 is available today. If you get a new Galaxy S4, try out WebXPRT 2013 on it. As it says on the WebXPRT.com page, your run is confidential. However, anyone who shares a Galaxy S4 score in the next week gets a free t-shirt (until they are gone). You can tag BenchmarkXPRT on a Facebook post, use the hashtag #WebXPRT on Twitter, or email us at BenchmarkXPRTsupport@principledtechnologies.com.

In other news, the HDXPRT 2013 comment period has ended and we’re working on the design document. We’ll be sharing that with the community next week. If you have comments you haven’t sent in, please do so, and we’ll try to address them in the design document.

Eric

Comment on this post in the forums

Posted in HDXPRT 2012 RFC, HDXPRT 2013, PhoneXPRT, WebXPRT |

No time to rest

By Eric Hale

on March 7, 2013

Last week was possibly our biggest week ever. We announced PhoneXPRT, a new benchmark for evaluating the performance of smartphones, and released TouchXPRT 2013 and WebXPRT 2013 to the general public.

Since then, there’s been a lot of interest. The numbers keep going up! We’re not just talking about page views – people are downloading TouchXPRT and lots of people are running WebXPRT.

People have also been downloading the TouchXPRT source, which is very exciting. We strongly encourage community members to look at how the benchmark is put together. If you have programming skills and want to submit code, get in touch with us at benchmarkxprtsupport@principledtechologies.com. We’d love to hear from you!

Oh, and there’s a new video! This one introduces WebXPRT to the public. It gives a good idea of the range of devices WebXPRT will run on.

That’s a lot for one week. We’re not resting on our laurels, though. Obviously, we’re working on PhoneXPRT. However, let’s not forget about HDXPRT. The comment period for HDXPRT 2013 officially closed on March 6 and we are starting to work on the HDXPRT 2013 design document. If you have any feedback you haven’t sent, please do send it on. We’ll do our best to incorporate it into the design document.

Eric

Comment on this post in the forums

Posted in HDXPRT 2013, HDXPRT 2013 RFC, PhoneXPRT, TouchXPRT 2013 release, TouchXPRT 2013 source, Video, WebXPRT, WebXPRT 2013 release |

Search Results for: webxprt

Three names, two hosts

Lies, damned lies, and statistics

The name game

Loose ends

Soon, we’ll know the truth

No time to rest

Check out the other XPRTs: