We sometimes get questions about comparing results from older versions of benchmarks to the current version. Unfortunately, it’s never safe to compare the results from different versions of benchmarks. This principle has been around much longer than the XPRTs. A major update will use different workloads and test data, and will probably be built with updated or different tools.
To avoid confusion, we rescale the results every time we release a new version of an existing benchmark. By making the results significantly different, we hope to reduce the likelihood that results from two different versions will get mixed together.
As an example, we scaled the results from WebXPRT 2015 to be significantly lower than those from WebXPRT 2013. Here are some scores from the published results for WebXPRT 2013 and WebXPRT 2015.
Please note that the results above are not necessarily from the same device configurations, and are meant only to illustrate the difference in results between the two versions of WebXPRT.
If you have any questions, please let us know.
Eric
