Search Results for: webxprt

Has it only been a month?

on October 23, 2014

Operating systems will continue to evolve. Whether you consider that a promise or a threat, it’s a fact. Those who write software know the day will come when it’s running in an environment that did not exist when you wrote it. Sometimes you get lucky. WebXPRT, for example, has sailed through the release of new versions of Windows, Android, and iOS with no problems.

At other times, you have to take action. Last month, we alerted you to an issue MobileXPRT had with the pre-release version of Android L. We’ve released an update to MobileXPRT that resolves the issue with Android L.

The technical preview for Windows 10 became available to members of Microsoft’s Windows Insider Program on the first of October, and we’ve had a report that TouchXPRT does not run reliably on it. We are currently investigating this and will let you know the details as soon as we have them. We are checking HDXPRT on Windows 10 as well.

Of course, this is what previews are for. By addressing these issues now, the XPRTs will be ready to support these operating systems when they’re released to the public.

If you are running a preview version of Windows 10 and see a problem with any of our benchmarks, please let us know.

Operating systems continue to evolve. There’s already a new build of the Windows 10 preview. More are on the way. As they come, we will be there testing the XPRTs on them.

Eric

Comment on this post in the forums

Posted in Android, BenchmarkXPRT development community, HDXPRT, HDXPRT 2014, MobileXPRT, Windows 10 |

Time to get creative

By Justin Greene

on September 4, 2014

The CrXPRT Community Preview is right around the corner, and there’s no sign of things slowing down. We’re exploring new opportunities on a number of fronts, and we’d love to hear what you think! We’re considering possible changes to WebXPRT and MobileXPRT, and since the mobile device market is changing all the time, we’re looking for the next great benchmark opportunity. In both cases, the development community is a rich source of ideas, so we’d like to tap into it one more time.

A while back, we added a new Web form in the members’ area for submitting benchmark ideas. Some of the ideas we have so far include:

A benchmark to evaluate camera features and photo quality on phones and tablets
A benchmark for measuring the performance of cloud services
A benchmark for measuring the performance and battery life of iOS-based devices

So, what would you like to see? Any of these, or do you have ideas we haven’t mentioned? Also, we’d love to hear your feedback on ways we can improve, both with the XPRTs themselves and with community life. Either way, send a message to BenchmarkXPRTSupport@principledtechnologies.com and let us know what you think!

Justin

Comment on this post in the forums

Posted in BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, CrXPRT, MobileXPRT, WebXPRT, What makes a good benchmark? |

More details to come

By Eric Hale

on July 10, 2014

As we’ve been saying the past couple of months, we’re working on a benchmark for Chrome OS. The experimentation phase is winding down, and we are starting to shape the code into a useable benchmark. The design plan will leverage existing WebXPRT tests, of course. However, we’ve gone far beyond that. The benchmark will include video playback, 3D modeling via WebGL, and even an HTML5 game. The test also uses Chrome OS’ native execution capability. The benchmark will actually use the Portable Native Client (PNaCl), as PNaCl is the recommended tool chain for native client. It also gives the benchmark the ability to run on more platforms.

As we mentioned before, we’re including a battery test as part of the new benchmark. So far, we haven’t found a way to remove the requirement to put the device in developer mode for the battery test.

Next week, we’ll publish a design document for the community to review. As always, the design document is based on the comments and suggestions we received combined with our own research and experimentation.

Eric

Comment on this post in the forums

Posted in Chrome OS, Future of performance evaluation, HTML5, Performance benchmarking, WebXPRT |

It makes a difference

By Eric Hale

on July 3, 2014

Ars Technica reported this week that they tested the developer preview of Android L and saw a whopping 36 percent improvement in battery life! Google made improving battery life a priority, and it sounds like they are succeeding. I can’t wait to test Android L with BatteryXPRT.

This is a spectacular example of how a change in software can change benchmark results, but it’s hardly unique. I’ve written before about how background activity on a phone depressed my friend’s WebXPRT scores. AnandTech used both IE 11 and Chrome 30 to test the Surface Pro 2 with a variety of benchmarks, including WebXPRT, SunSpider, Octane, Browsermark, and others. Browser choice had a noticeable impact on results – about a 40 percent difference for WebXPRT and a 76 percent difference for SunSpider!

People are generally pretty aware that changing the hardware changes performance. However, sometimes they lose track of software differences. When you compare scores, it’s not always possible to keep all the variables the same, but it’s crucial to know what the differences are.

In other BenchmarkXPRT news, we’re making some final adjustments to HDXPRT 2014, and the general release is just around the corner.

Eric

Comment on this post in the forums

Posted in Android, Battery life, HDXPRT 2014, Performance benchmarking, Performance testing on tablets, WebXPRT |

Testing the waters

By Eric Hale

on June 19, 2014

A couple of weeks ago, we talked about some of our ideas for a new XPRT designed for Google’s Chrome OS. We’ve been working with some of these ideas and, while we’re still in the experimental stage, things look promising so far.

As we mentioned in the earlier blog, we’re trying WebXPRT as a base for the performance part of the test. So far, the performance component is working well. In addition to modified WebXPRT tests, we’re also trying some things that are not part of the WebXPRT 2013 workload.

We’ve been able to get battery life, but it’s been challenging and we haven’t found a way to avoid using Chrome’s Developer Mode. Accessing Developer Mode in Chrome can be tricky and requires different steps for each hardware manufacturer. We’re hoping to find ways to make battery life testing easier.

I’ve been vague about the tests because they’re likely to change over the next few weeks. We’re experimenting with both browser-based and Native Client-based performance tasks. As they firm up, I’ll be able to share more information.

Challenges aside, we’re excited about this new benchmark, and committed to making it as effective as possible. We’d still love feedback on a name, so feel free to contact us at BenchmarkXPRTsupport@principledtechnologies.com with your ideas.

Eric

Comment on this post in the forums

Posted in Battery life, Chrome OS, Collaborative benchmark development, Performance benchmarking, What makes a good benchmark? |

The name game

By Eric Hale

on June 5, 2014

In Something shiny, we discussed the leading contender in our search for new benchmark ideas, a benchmark tailored especially for the Chrome OS, and we’ve been looking at what workloads would make sense.

As we said, the ability to measure battery life would be useful. That’s not easy in the Chrome environment. We think we may be able to do it, but the Chromebook may have to be in developer mode. Even so, we can leverage what we’ve learned from BatteryXPRT to get a reliable estimate of battery life in less than a working day.

Measuring performance, however, is a must. We’ve been looking at the existing WebXPRT workloads as well as other applications, such as education apps, online games, HD video playback, music playback, and more. We’re also looking for areas where using native client execution makes sense, such as higher-resolution photo editing.

In addition, we’re thinking about what we might call this benchmark. ChromeXPRT would be obvious, but probably wouldn’t pass Google’s naming restrictions.

Do you have ideas for the benchmark’s name? Are there Chrome-based benchmark workloads you’d love to see? Let us know at BenchmarkXPRTsupport@principledtechnologies.com!

Eric

Comment on this post in the forums

Posted in Chrome OS, Collaborative benchmark development, Future of performance evaluation, WebXPRT |

Search Results for: webxprt

Has it only been a month?

Time to get creative

More details to come

It makes a difference

Testing the waters

The name game

Check out the other XPRTs: