Category: HDXPRT development process

Back in September

on November 8, 2012

Back in September, we made a prerelease version of a patch to let HDXPRT run on Windows 8. (See Working towards Windows 8.) We’ve been testing the final version of the patch and hope to release it in the next few days.

We wanted to give you a heads up that there are a few differences between the versions of Windows that may affect your HDXPRT testing. For example, we have found that some that older systems that meet the minimum specs for HDXPRT 2012 and Windows 8 may not have Windows 8 drivers available. The lack of these drivers can cause the benchmark to fail.

Also, while it was possible for a lab like ours to test without activation of Windows 7 during the 30-day grace period, activation is different under Windows 8. To reliably run HDXPRT on Windows 8, activation is required.

We are updating the HDXPRT documentation based on our testing. Generally, though, testing on Windows 8 is very similar to testing on Windows 7.

We are releasing the patch on Friday, November 9 . Please try it out and let us know what you think!

Eric

Comment on this post in the forums

Posted in HDXPRT 2012, HDXPRT development process, Let us know your thoughts, Windows 8 |

What you’re going to need

By Eric Hale

on September 27, 2012

We’re planning to release the source code for HDXPRT 2012 soon after we release the update for Windows 8, probably in mid-November. The source code will be freely available to the members of the community. In preparation for that, here’s some information about what you’ll need to build the benchmark.

As we’ve discussed before, HDXPRT is a complicated entity, with a test harness, multiple workloads, and an installer. Consequently, you’ll need several tools to edit and build
HDXPRT 2012:

• Microsoft Visual Studio 2005
• Microsoft SDK, (version 6.0.6000.0)
• Microsoft .Net Framework 2.0 SDK x64
• AutoIT v3
• InstallShield 2012 Standalone version. Note: The InstallShield Professional Version is required to edit the install script.
• Visual Build Pro 7 Note: It is possible to edit and build HDXPRT without Visual Build Pro. However, Visual Build Pro enables an automated build process.

If you install these products per their instructions, your system will be ready to build HDXPRT 2012 when the source code becomes available.

Remember, the HDXPRT 2013 suggestion period starts Monday. We’ll open a section of the forum for suggestions then. However, if you want to get an early start, feel free to go ahead and send suggestions to hdxprtsupport@hdxprt.com.

Eric

Comment on this post in the forums

Posted in HDXPRT 2012, HDXPRT development process, HDXPRT source code, Windows 8 |

Counting down

By Eric Hale

on July 6, 2012

We’ve been hard at work since the end of the beta period, driving toward the release of HDXPRT 2012. Things are looking good. The RTM is coming soon, so we thought we’d share the next few milestones with you.

RTM candidate: 7/13/12. At this point, we stop development. There are no feature changes after this point.
HDXPRT 2012 launch: 7/27/12. Having tested the RTM and manufactured the DVDs, we mail the benchmark to the community members. This is when the press release goes out. It’s also when we publish the HDXPRT 2012 white paper. Unlike the design document, this paper will explain HDXPRT 2012 to the general public.
Webinar: 8/3/28 – We talk about HDXPRT 2012 and take your questions.
First scaling paper: 8/31/12. As with HDXPRT 2011, we will be publishing a set of scaling studies. The first will test several modern desktop processors, looking at factors such as varying the amount of RAM, comparing HDDs to SSDs, and comparing internal graphics to a couple of popular graphics cards.
Second scaling paper: 9/28/12. In this paper, we expand on the testing we did in the first scaling paper.

We will release an update of HDXPRT 2012 that supports Windows 8 and includes bug fixes within a month of Windows 8 launch.

We’re very excited about the release of HDXPRT 2012, and look forward to seeing what you do with it!

Eric

Comment on this post in the forums

Posted in Application-based benchmarks, Collaborative benchmark development, HDXPRT, HDXPRT 2012, HDXPRT 2012 beta, HDXPRT development process, HDXPRT release cycle |

Software updating when everything uses software

By Bill Catchings

on June 8, 2012

Software updating when everything uses software

The latest beta of Windows 8 Release Preview is out and we are scrambling to see what works with it and what doesn’t. After many years in the software industry, I’ve come to expect that. Now, however, pretty much everything goes through many cycles of software updates. The results can be annoying. Or worse.

Two weeks ago, I took my Nissan LEAF in for its 6-month check-up. While there, the dealer did a software update. I’ve had no real problems with it and, in general, I love the car . However, after the update, the LCD display on my console is different. It now displays the time in the upper right. Unfortunately, that time is different from the one that displays beneath the windshield. I guess I need to read some manuals. Annoying.

This week, the Garmin Edge 500 bicycle computer I use had an available update and I updated it. Everything seemed to go fine on my 36-mile bike ride a couple days later. When I went to upload the ride to the Garmin Connect Web site, it told me I need to update my Edge 500 (to the same version) and claimed there were no new rides on the device. After some frustration, I managed to get it all working without losing the data from my ride. Very annoying.

It seems like I’m constantly updating my iPhone, my Xbox 360, my TV, my iPad, Windows on my PC, Mac OS on my Mac, and, of course, applications everywhere. Usually, the updates work fine. But, when the glitches happen, I am tempted to never update anything again. Very annoying, indeed.

In the meantime, it’s back to working on updating HDXPRT 2012!

Bill

Comment on this post in the forums

Posted in HDXPRT 2012, HDXPRT development process |

HDXPRT 2012 and Windows 8

By Bill Catchings

on May 25, 2012

One of our community members asked whether HDXPRT 2012 would support Windows 8. It certainly is one of our goals. The current beta does not work with Windows 8, however. Problems with a couple of the applications themselves prevent the current version from installing on Windows 8.

We will continue to test beta versions of HDXPRT 2012 with beta versions of Windows 8 over the coming weeks. There is rumored to be a newer beta of Windows 8 out next week and we are hoping to be able to resolve the issues in a later beta or at least by the shipping version.

Regardless, there will be multiple versions of Windows 8 before it releases later this year. We plan to release HDXPRT 2012 this summer. We assume there will be some glitches. So, we expect that we will have to release an update to HDXPRT 2012 once Windows 8 is closer to its final version.

Also, once there is a version of Windows 8 that allows publishing performance results, we plan to look at what performance differences there are between Windows 7 and Windows 8.

A quick update on the beta—we are in the process of making the DVDs and will send them out next week. We sent emails to all the community members to confirm their correct shipping addresses. If you received that email, please respond. If you haven’t received that email, please let us know. We want to make sure the DVDs actually get to you!

Bill

Comment on this post in the forums

Posted in HDXPRT 2012 beta, HDXPRT development process |

Today is a milestone

By Bill Catchings

on May 18, 2012

Today is a milestone. We are freezing the beta for HDXPRT 2012. It’s taken a lot of work–developing something with this many moving parts is a bit like herding cockroaches. However, the workloads are set and the benchmark seems stable.

So, what happens next?

First, the beta goes to a team who will spend a few days intensively testing it. The test team is isolated from the development team because, let’s face it, developers can’t test their own code. (I should know. I’ve developed a lot of products in my time.)

We’ll run it on a variety of systems, checking it for stability and repeatability. We will also look at the results and make sure that they are sensible. That is, they reflect that capabilities of the systems we are running the benchmark on.

The beta is approximately 10 GB. It includes installers for nine applications and a number of large content files. Because of its size, the beta will only be available on DVD. Once the tests look good, we’ll start reproducing and verifying the DVDs. Once the DVDs are verified, we’ll mail them out. The whole process should take about a week, so community members should have their beta DVDs within a couple of weeks.

While the testing is happening, we’ll be contacting each community member to confirm that we have their correct physical mail address.

Once you have the DVDs, please run the benchmark and send feedback. We need your help to make this beta into a truly great release candidate. We will work hard to address any feedback we get. If you find a problem, let us know. If you have a complaint, let us know. It you have a compliment, by all means, let us know!

Next week, Bill will be back from his travels.

Eric

Comment on this post in the forums

Posted in HDXPRT 2012 beta, HDXPRT development process, Let us know your thoughts |

Category: HDXPRT development process

Back in September

What you’re going to need

Counting down

Software updating when everything uses software

HDXPRT 2012 and Windows 8

Today is a milestone

Check out the other XPRTs: