Category: WebXPRT

Fragments

on August 1, 2013

The folks over at OpenSignal have created some pretty impressive visualizations of how fragmented the Android marketplace is. Pictures like that are a little daunting as we head to the public release of MobileXPRT 2013. However, the benchmark is looking pretty stable. Since we released MobileXPRT CP1.1 back in June, we have not encountered any problem configurations.

That includes our first encounter with the recently released Android 4.3. We tested MobileXPRT 2013 CP 1.1 on a Nexus 7 running Android 4.3. The benchmark ran with no problem. Here are the scores, along with scores from the same device running Android 4.2.2.

	Android 4.2.2	Android 4.3
MobileXPRT performance	116	120
MobileXPRT user experience	97	98

Of course, WebXPRT 2013 ran on Android 4.3 as well. If you’re curious, here are the scores.

	Android 4.2.2	Android 4.3
WebXPRT	183	185

While the upgrade did not have a big effect on the score in the case of the Nexus 7, we will need to test on more devices before we can make a definitive statement about the effect of Android 4.3.3 on performance.

Eric

Comment on this post in the forums

Posted in MobileXPRT, WebXPRT |

There’s always something new to measure

By Eric Hale

on July 25, 2013

Scientific Reports published an article this month showing that electrically charged insects, such as honey bees, can cause some spider webs to deform. The effect, as pictures in the report show, is that the webs can literally reach out a short distance to snag the passing insect. It’s a cool optimization for the web, and one no one had measured before.

Like the scientists in the report, we are always looking for interesting things to measure. Even as we head toward the public release of MobileXPRT, we’re already thinking ahead to the next versions of all the XPRT benchmarks. This week, I’d like to share a few of the things we have been thinking about, based on conversations with folks (both within and outside of the Development Community) for WebXPRT.

Enhancing the benchmark UI to provide better feedback about the test progress
Extending coverage to other Web technologies such as Web Workers and CSS 3
Extending the workload categories to include other areas, such as productivity
Enabling WebXPRT to be used as a platform-independent battery life test

There are just some of the ideas we’ve been talking about. Which of these excite you? Even better, do you have any cool ideas of your own? Either way, post to the forums or e-mail BenchmarkXPRT@principledtechnologies.com

Eric

Comment on this post in the forums

Posted in Benchmarking, BenchmarkXPRT, Collaborative benchmark development, WebXPRT |

The show is in previews

By Eric Hale

on July 11, 2013

I love theater, both as an actor and as an audience member. Seeing a show in preview means you have the chance to see what might be the next big thing before the critics do. It also means you may be watching all the things they need to fix before opening night.

Microsoft released the Windows 8.1 preview at the end of June. Of course we had to give it a look. After installing it, we ran the compatible XPRT benchmarks: HDXPRT, TouchXPRT, and WebXPRT. (Because MobileXPRT is Android based, it does not run on Windows.)

WebXPRT ran without any problems. To date, WebXPRT has run on everything we’ve thrown at it.

We took a Windows 8 system with TouchXPRT already installed and upgraded it to Windows 8.1. The previously installed version of TouchXPRT had some problems. However, when we did a clean install of Windows 8.1 and then installed TouchXPRT, it ran just fine.

HDXPRT failed while configuring the applications. We are looking into this and hope to have a solution soon.

I’m not going to talk about the results, because this is a preview of Windows. By the time of the official release, the results from the benchmarks certainly may change.

The Windows 8.1 RTM is currently set for August. I’m looking forward to seeing how it performs!

Eric

Comment on this post in the forums

Posted in HDXPRT 2012, TouchXPRT, WebXPRT, Windows 8.1 |

Our baby has a new name!

By Eric Hale

on May 16, 2013

The upcoming Android benchmark will be called MobileXPRT. Thanks to everyone who sent in suggestions. We are testing development builds now and look forward to having a community preview available in the next few weeks.

In other news, the developer license used to build TouchXPRT expired this week. We have created a new version to fix this problem. If you are a TouchXPRT user, you’ll need to download the new version for any future testing. You can find the details here.

As we mentioned in the post Three names, two hosts, we set up a second WebXPRT hosting site to see if that would improve the slow downloads reported in China. To help us better understand the situation, we are going to start logging IP addresses for the host at http://54.251.252.204/webxprt/. We are doing this to see if there are regional differences in the download time. This change does not affect users of WebXPRT at http://www.principledtechnologies.com/benchmarkxprt/webxprt/. Neither host will gather personally identifying information.

Eric

Comment on this post in the forums

Posted in MobileXPRT, PhoneXPRT, TouchXPRT, WebXPRT |

Extreme makeover

By Eric Hale

on April 25, 2013

Last week, we unveiled redesigned Web pages for BenchmarkXPRT. We’ve been working on this redesign for a while. We think you’ll find the pages to be a lot sleeker and more attractive. The HDXPRT page, for example, is far less cluttered and easier to navigate. There’s a new white papers page. The members’ area has a new tabbed design that will let you access the member resources for any benchmark form a single page.

We will be redesigning the blog and forums over the next few weeks. Log into the forum or send an e-mail to benchmarkxprtsupport@principledtechnologies.com and tell us what you think about the new design!

As we mentioned in the blog post What a week!, WebXPRT does not collect any personally identifying information. (The WebXPRT data collection page details all the information the benchmark collects.) The benchmark does not attempt to verify that the user agent string is correct under the assumption that the user or browser had some reason for setting it the way it is.

This has caused some people to be confused when, for example, the results for a phone running the stock Android browser say that the phone used Safari. Most modern browsers have the ability to change the user agent string and misidentify themselves, as that version of the Android browser did by default. In fact, you can usually override the browser’s default, should you want to. For example, Google Chrome version 26.0, the version I’m using right now, allows you to choose from multiple versions of IE, Firefox, Chrome, iPhone, iPad, Android, and others. You can even type in a custom string.

So, if you think WebXPRT misidentified your browser, it’s worth checking the user agent string. The instructions for doing this vary by browser, but are usually pretty straightforward. If you’re curious about why browsers offer this feature, you can search for “user agent spoofing” to find explanations of the pros and cons.

Eric

Comment on this post in the forums

Posted in BenchmarkXPRT, HDXPRT, Principled Technologies, WebXPRT, WebXPRT 2013 results |

Three names, two hosts

By Eric Hale

on April 18, 2013

As Bill mentioned a couple of weeks ago in The Name Game, we’ve been considering changing the name of PhoneXPRT. The rationale for this is that the tests in PhoneXPRT are useful for a range of devices, from phones to tablets. We asked for your opinions about the name. After getting your input and talking amongst ourselves, we are considering three possibilities:

Leaving the name unchanged.
Changing the name to MobileXPRT. While this would convey the scope of the benchmark, some people thought the name might be too general.
Changing the name to TouchXPRT for Android. While there is some similarity between the tests in TouchXPRT and the ones we are developing for this benchmark, the two benchmarks would not initially be comparable.

Let us know what you think. We hope to settle the name question soon.

As we mentioned in Loose Ends, some users in mainland China are reporting extremely slow download times when running WebXPRT. We have set up a trial host for WebXPRT in Singapore to see if this improves the situation. Preliminary, US-based tests have seen no significant difference in scores when running from the Singapore host.

If you are in China and want to try running WebXPRT from the new host, you will find it at http://54.251.252.204/webxprt/. Please let us know your experience.

Eric

Comment on this post in the forums

Posted in Android, MobileXPRT, PhoneXPRT, WebXPRT |

Category: WebXPRT

Fragments

There’s always something new to measure

The show is in previews

Our baby has a new name!

Extreme makeover

Three names, two hosts

Check out the other XPRTs: