Category: HDXPRT 2012 RFC

Loose ends

on March 21, 2013

As we mentioned last week, the Samsung debuted the Galaxy S4 this past week. It seems to have hit all the expectations – including eye-scrolling. Looking at the reviews, it’s somewhere between the greatest phone ever made and something not quite as good as the iPhone. I’m looking forward to seeing one for myself and hoping someone submits a WebXPRT score for it soon.

We’ll be releasing the HDXPRT 2013 design document tomorrow. As we’ve said, the number one comment has been that HDXPRT 2012 is too big and takes too long to run. We have put a lot of thought into how to trim HDXPRT 2013 and still keep the essential value of the benchmark. We’ve also received some other good feedback that we’re incorporating, such as making installing and running HDXPRT scriptable.

We’ve been doing some investigation during the RFC period, and we’ve encountered problems scripting some of the applications, notably iTunes and PowerDirector. We’re working to overcome these problems but if they prove to be insurmountable, we might have to change the list of applications.

Again, thanks to everyone who commented on the HDXPRT 2013 RFC.

We’ve learned that some WebXPRT users in mainland China are having problems with very slow downloads. The good news is that results from the runs are valid. However, we understand that this is frustrating and are investigating solutions. If you are in China and have experienced slow downloads, please send an e-mail to benchmarkxprtsupport@principledtechnologies.com. We would like your help in evaluating any solutions we come up with.

Speaking of WebXPRT, over the next few weeks we’ll be releasing several white papers that look at WebXPRT results in more depth. The first paper will explain the WebXPRT confidence interval and how it relates to run to run variability, which has been confusing to a number of people. The second paper will look at the effect on the browser on WebXPRT scores. The third paper will look at the influence of the operating system.

Eric

Comment on this post in the forums

Posted in HDXPRT 2012 RFC, HDXPRT 2013, HDXPRT 2013 design document, WebXPRT, WebXPRT 2013 results |

Soon, we’ll know the truth

By Eric Hale

on March 14, 2013

I’m writing this on the morning of one of the most anticipated events in tech, Samsung’s rollout of the new Galaxy S4 phone. There have been many leaks, and it will be interesting to see how many are true. One feature generating buzz would let you scroll just by moving your eyes. Samsung filed for the patent, but there are conflicting reports about whether eye scrolling made it into the Galaxy S4.

The leaked reports say that the U.S. version of the Galaxy S4 will use Qualcomm Inc.’s quad-core chip, while versions for other markets will use Samsung’s Exynos 5 Octa-core chip. As we continue development of PhoneXPRT, I will be very interested to see how the different processors stack up performance wise.

For browser-level performance, WebXPRT 2013 is available today. If you get a new Galaxy S4, try out WebXPRT 2013 on it. As it says on the WebXPRT.com page, your run is confidential. However, anyone who shares a Galaxy S4 score in the next week gets a free t-shirt (until they are gone). You can tag BenchmarkXPRT on a Facebook post, use the hashtag #WebXPRT on Twitter, or email us at BenchmarkXPRTsupport@principledtechnologies.com.

In other news, the HDXPRT 2013 comment period has ended and we’re working on the design document. We’ll be sharing that with the community next week. If you have comments you haven’t sent in, please do so, and we’ll try to address them in the design document.

Eric

Comment on this post in the forums

Posted in HDXPRT 2012 RFC, HDXPRT 2013, PhoneXPRT, WebXPRT |

RFC responses

By Bill Catchings

on December 9, 2011

The official RFC response period ended last week on December 2. We’ve been working our way through the responses and have published all of the comments and our responses here. Please do read through all of them. I wanted to touch on a couple of them here.

First, we heard differing opinions on what the minimum system requirements should be. This is a tough one. On the one hand, we want to keep the benchmark current with the latest versions of applications. Those applications typically increase their minimum system requirements over time. Those system requirements in turn are what we use as the basis for HDXPRT’s requirements. On the other hand, we really want HDXPRT 2012 to be useful on the new class of tablet, netbook, and nettop systems coming out in the next year. Our best idea is to go with the same method for determining the requirements, but to test on those slower systems and try to make sure that they are able to run.

Although we only addressed one in the responses, we heard from multiple people about making HDXPRT 2012 run on versions of Windows other than English. As we are trying to have HDXPRT used around the world, we’d like to make this happen. We face two challenges, however—first, getting systems with OEM images running non-English versions of Windows and second, the effort required for testing and debugging on those systems. You can help with the first by sending us systems. Please contact me if you may be able to help. We will test on any systems we receive and will make it a priority to work out any problems we encounter, but we can’t guarantee that all systems will work.

We are off working on the final design specifications based on the RFC and the responses. While we cannot include your comments in the design specs, we do still want to hear from you. If you have a good idea, we will try to see what we can do to accommodate it in our development cycle. So, please do let us know any comments you may have about either the RFC or the responses to it. Thanks!

Bill

Comment on this post in the forums

Posted in HDXPRT 2012, HDXPRT 2012 RFC, Let us know your thoughts |

Responding to the RFC

By Bill Catchings

on November 30, 2011

We are reaching the end of the RFC period. Your feedback plays a crucial role in defining what HDXPRT 2012 will be. Now is the time to let us know what you agree and disagree with in the RFC. We want to hear from as many of you as possible by the end of this week (December 2nd). We will still accept responses after that date, but they will be more difficult to include in the design specifications, which we will create next week based on the RFC and your feedback.

In case you have not yet had a chance to look at the RFC, it is available for Development Community members at www.hdxprt.com/forum/hdxprt2012RFC.php. You can give us your feedback either in the forums to help stimulate discussion or in email by sending them to HDXPRTsupport@hdxprt.com. However you choose to respond, we appreciate you taking the effort to do so. Thanks!

Bill

Comment on this post in the forums

Posted in HDXPRT 2012, HDXPRT 2012 RFC, Let us know your thoughts |

Thankful for change

By Bill Catchings

on November 23, 2011

With the American holiday of Thanksgiving almost here, I have been thinking of the many things I’m thankful for. I realized how thankful I am for the constant state of change in the computer industry. Part of that change is because of Moore’s Law and part of it is due to general innovation of how to take advantage of additional processing power. The consequences are that there are always new (and usually better) products out there. I love the opportunity both to play with the latest and greatest products (like my new Kindle Fire) and to use them to be more productive and do things I was not able to do previously. I genuinely look forward to when it is time to upgrade to a new computer.

A consequence of constant innovation is that it’s often important to understand how much faster new products are than their predecessors. I certainly don’t feel like going to the hassle of moving to a new computer for 10 percent more performance. But, when it is twice as fast, I have trouble resisting. And, it is important to be able to decide which of two products offers either the most performance or the best performance for the dollar. That, of course, is where benchmarks like HDXPRT come in.

Someday, the constraints of physics may put an end to Moore’s Law. When that happens, computers will be more like cars. Instead of the new model being twice or ten times as fast as your old one, it will be just a little faster, but have a really cool paint job and boss fins on it. That is not a day I look forward to! Not only will the computer industry be much less interesting, but there won’t be much need for benchmarks. I am thankful for the constant change in computers!

And, I’m thankful for all of you in the Development Community for your help in defining, developing, testing, and promoting HDXPRT. Now, before the tryptophan kicks in, please send us your responses to the RFC!

Bill

Comment on this post in the forums

Posted in HDXPRT 2012 RFC, Let us know your thoughts |

Turning a role into a scenario

By Bill Catchings

on November 9, 2011

To give you an idea about our process for proposing the scenarios in the RFC, I want to walk you through one of them. (Download the RFC at http://www.hdxprt.com/forum/hdxprt2012RFC.php if you haven’t already.) The role I’m going to discuss is that of a photo blogger. By photo blogger, I am thinking of someone who is a photo enthusiast, not someone who snaps pictures with his iPhone and puts them on Facebook. By my thinking, this type of photo blogger is someone who enjoys photography, has substantial knowledge of digital tools, and is interested in processing and posting photos on enthusiast Web sites. That person would commonly use raw rather than compressed photos from their camera and convert them to a standard format as the last step. The person would certainly edit the photos using different filters and effects. That person might also do some more esoteric things like stitch together multiple photos to create a panorama or work with high dynamic range (HDR) images. The latter is growing in popularity due to its ability both to better represent the color palette found in nature and to go farther and create amazing surreal images. Some newer cameras (like my Sony NEX-5) will produce the necessary multiple exposures to make HDR fairly easy to do.

What software would such a person use to do those things? Because HDXPRT is a consumer-oriented benchmark, we want to find affordable (or free) applications that a typical consumer might use. For photo manipulation, I think Adobe Photoshop Elements is the leading consumer application. The software lets users create, edit, organize, and share images. For HDR images, HDRsoft Photomatix is a good tool for high-quality two-stage HDR processing. It provides capabilities for both image overlaying and adjustable tone mapping. Another useful tool is GIMP, which stands for GNU Image Manipulation Program. It’s a free graphics editor that allows users to edit and retouch images. I think including some free tools like this mimics what many consumers with limited budgets are using.

Having seen the basic process we used in the RFC, what do you think? Does that process make sense to you? In this particular one, do you agree with the basic activities? The applications? Thanks, I look forward to hearing from you!

Bill

Comment on this post in the forums

Posted in HDXPRT 2012 RFC, Let us know your thoughts |

Category: HDXPRT 2012 RFC

Loose ends

Soon, we’ll know the truth

RFC responses

Responding to the RFC

Thankful for change

Turning a role into a scenario

Check out the other XPRTs: