Category: HDXPRT 2013

Unchecked ambition

on January 17, 2013

With 10 applications and five scenarios, HDXPRT 2012 is the most ambitious version of HDXPRT ever. However, as we said in the blog post of the same name, There is such a thing as too much. We heard that message from you as well: HDXPRT 2012 is too big. At 11 GB, we can’t make it available for download, and it won’t fit on a single DVD.

It also takes longer to use than many of you would like. The installation takes a couple of hours, and each iteration takes 2 to 3 hours. Because a valid run of HDXPRT 2012 comprises three iterations, getting a single result takes all day.

As we work to reduce the size of HDXPRT 2013, we’re being careful. While we need to shrink the benchmark and its running time, we want to be sure that we don’t compromise its essential value. As part of this process, we’re looking at both the selection of applications and the ways we can reengineer the scenarios.

If you have ideas about HDXPRT 2012 that you haven’t sent yet, now is a good time to do that. We’ll be sending out the HDXPRT 2013 RFC in the next couple of weeks. Once it’s out, we will look forward to hearing your comments.

I also wanted to mention that as we had to cancel CES at the last minute, we’re planning to do a Webinar next week (Tuesday, January 22) to cover the material we planned to present there. We’ll send out an email later this week with more details.

Bill

Comment on this post in the forums

Posted in BenchmarkXPRT, CES, HDXPRT, HDXPRT 2013, HDXPRT Development Community benefits, Let us know your thoughts |

There is such a thing as too much

By Eric Hale

on December 6, 2012

There’s been a lot of excitement about TouchXPRT recently. However, we haven’t been ignoring HDXPRT. On November 9, we released a patch that lets HDXPRT support Windows 8. We’ve now integrated the patch into HDXPRT2012, so all copies of HDXPRT 2012 going forward will install on Windows 8 without the need for a separate step.

As promised, we will be releasing the source code for HDXPRT 2012. We anticipate having it available for community members by December 14.

During the comment period for HDXPRT, this message came through loud and clear: HDXPRT 2012 is too big and takes too long to run. So we are working hard to find the best way to reduce the number of applications and scenarios. While we want to make the benchmark smaller and faster, we want to make sure that HDXPRT 2013 is comprehensive enough to provide useful performance metrics for the greatest number of people.

We’re working toward having an RFC in late January that will define a leaner, meaner HDXPRT 2013, and will reflect the other comments we have as received as well. If you have thoughts about which applications and scenarios are most important to you, please let us know.

In other news, CES is coming in January, and Principled Technologies will be there! Once again, Bill is hoping to meet with as many of you in the Development Community as possible. We’ll have a suite at the Hilton and would love for you to come, kick back, and talk about HDXPRT, TouchXPRT, the future of benchmarks, or about the cool things you’ve seen at the show. (Bill loves talking about gadgets. Last year, he went into gadget overload!)

If you plan to be at CES, but are stuck working a booth or suite, let us know and Bill will try to stop by and say hi. Drop us an email at hdxrpt_CES@principledtechnologies.com and we will set up an appointment.

Finally, we’re really excited about the big changes at the Principled Technologies Web site. The new Web site gives us a lot of opportunities. Over the next few weeks, we’ll be looking at ways the Development Community can take advantage of them.

Eric

on this post in the forums

Posted in Application-based benchmarks, Benchmarking, CES, Collaborative benchmark development, HDXPRT 2013, HDXPRT Development Community benefits, Principled Technologies, What makes a good benchmark?, Windows 8 | Tagged CES |

Make sure your voice is heard

By Eric Hale

on October 4, 2012

One thing about the community model we use for developing HDXPRT is that is depends on the community. Your input is essential to making the benchmark the best it can be. As the community grows, we’re learning more about your priorities.

During the development of HDXPRT 2012, we made the decision to remove the playback tests from the benchmark. While the design document called for the playback test to include 4K H.264, Windows Media Player does not play that format by default. Because less demanding codecs were not differentiating systems, and because the stars used to report the results confused some people, it seemed like a reasonable decision. Bill announced the decision in a blog post, More HDXPRT 2012 changes.

Fast forward to September 18, when Bill hosted the HDXPRT 2012 Webinar. During the Q&A session, a new member of the community said that the playback tests from HDXPRT 2011 were what got him interested in the benchmark. For now, he has to use HDXPRT 2011 for those tests, although, as per Bill’s original blog post, we may release a more demanding playback test as a standalone inspection test later this year.

The suggestion period for HDXPRT 2013 started on October 1. Now is the time to let us know what tests are the most useful to you. If there are tests you’d like us to add, tests you’d like us to change, applications you’d like us to consider, we need to know that too. You can post your suggestions to the forum in the HDXPRT 2013 Suggestions section or mail them to hdxprtsupport@hdxprt.com.

In November, we’ll develop an RFC for HDXPRT 2013 and send it to the community for review.

While the suggestions we receive early have the best chance of being implemented, comments we receive after the formal suggestion period still get our attention. We’re always listening. Contact us anytime and make sure that HDXPRT 2013 includes the things that are important to you.

Eric

Comment on this post in the forums

Posted in Collaborative benchmark development, HDXPRT 2012, HDXPRT 2013, HDXPRT Webinar, Let us know your thoughts |

HDXPRT 2012 has left the building! Now what?

By Bill Catchings

on July 27, 2012

HDXPRT 2012 is finished! We are shipping out the HDXPRT 2012 DVDs to the Development Community membership. If you don’t get yours next week, please let us know. If you would like a copy of HDXPRT 2012, please join the community and we will gladly send you one.

We are also putting out a press release, preparing white papers, and doing the myriad things involved in shipping a product. The benchmark itself, however, is done.

Now that we have had a couple minutes to celebrate that moment, it is time to ask, “What next?” The first and most obvious thing will be to get the RTM of Windows 8 and see what we need to fix to make HDXPRT 2012 work with it. We hope we won’t need to change much–maybe we’ll get lucky and everything will work fine. Regardless, we’ll let you know what is necessary to make HDXPRT 2012 work on Windows 8 or will create an updated version that works on Windows 8.

We will also begin the next cycle for HDXPRT 2013. Now, however, is the best time for you to let us know what you would like to see. Are there particular tests you would like to see? We can add inspection tests to HDXPRT even before the next full version. By an inspection test, I mean a test that may not be relevant for all users or environments and is not part of the overall score. The test might also be an experimental one that we want to try out for possible inclusion in the overall score in the future.

But, for the moment, we plan to enjoy our weekend. We hope you have a great weekend as well!

Bill

Comment on this post in the forums

Posted in HDXPRT 2012, HDXPRT 2013, Let us know your thoughts, Windows 8 |

Category: HDXPRT 2013

Unchecked ambition

There is such a thing as too much

Make sure your voice is heard

HDXPRT 2012 has left the building! Now what?

Check out the other XPRTs: