Category: HDXPRT

Looking under the hood

on April 6, 2017

In the next couple of weeks, we’ll publish the source code and build instructions for the latest HDXPRT 2014 and BatteryXPRT 2014 builds. Access to XPRT source code is one of the benefits of BenchmarkXPRT Development Community membership. For readers who may not know, this a good time to revisit the reasons we make the source code available.

The primary reason is transparency; we want the XPRTs to be as open as possible. As part of our community model for software development, the source code is available to anyone who joins the community. Closed-source benchmark development can lead some people to infer that a benchmark is biased in some way. Our approach makes it impossible to hide any biases.

Another reason we publish source code is to encourage collaborative development and innovation. Community members are involved in XPRT development from the beginning, helping to identify emerging technologies in need of reliable benchmarking tools, suggesting potential workloads and improvements, reviewing design documents, and offering all sorts of general feedback.

Simply put, if you’re interested in benchmarking and the BenchmarkXPRT Development Community, then we’re interested in what you have to say! Community input helps us at every step of the process, and ultimately helps us to create benchmarking tools that are as reliable and relevant as possible.

If you’d like to review XPRT source code, but haven’t yet joined the community, we encourage you to go ahead and join! It’s easy, and if you work for a company or organization with an interest in benchmarking, you can join the community for free. Simply fill out the form with your company e-mail address and click the option to be considered for a free membership. We’ll contact you to verify the address is real and then activate your membership.

If you have any other questions about community membership or XPRT source code, feel free to contact us. We look forward to hearing from you!

Justin

Posted in BatteryXPRT 2014 for Android, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, HDXPRT, HDXPRT 2014, Open Source, Source code |

A new HDXPRT 2014 build is available

By Justin Greene

on February 9, 2017

Last fall, we identified a way to run HDXPRT 2014, originally developed for Windows 8, on Windows 10. The method involved overwriting the HDXPRT CPU-Z files with newer versions and performing a few additional pre-test configuration steps. You can read more details about those steps here.

Today, we’re releasing a new build of HDXPRT 2014 (v1.2) that eliminates the need to overwrite the CPU-Z files. The new build is available for download at HDXPRT.com. Please note that the app package is 5.08 GB, so allow time and space for the download process.

We also updated the HDXPRT 2014 User Manual to reflect changes in pre-test system configuration and to include the settings we recommend for newer builds of Windows 10.

The changes in the new build do not affect results, so v1.2 scores are comparable to v1.1 scores on the same system.

The new build ran well during testing in our labs, but issues could emerge as Microsoft releases new Windows updates. If you have any questions about HDXPRT or encounter any issues during testing, we encourage you to let us know.

We look forward to seeing your test results!

Justin

Posted in Application-based benchmarks, Collaborative benchmark development, HDXPRT, HDXPRT 2014, Performance benchmarking, Windows 10 |

TouchXPRT’s future

By Bill Catchings

on February 2, 2017

If you’ve been following the blog, you know that we’ve been reviewing each part of the XPRT portfolio. If you missed our discussions of HDXPRT, BatteryXPRT, WebXPRT, and CrXPRT, we encourage you to check them out and send us any thoughts you may have. This week, we continue that series by discussing the state of TouchXPRT and what we see down the road for it in 2017.

We released TouchXPRT 2016, an app for evaluating the performance of Windows 10 and Windows 10 Mobile devices, last February. We built the app by porting TouchXPRT 2014 performance workloads to the new Universal Windows App format, which allows a single app package to run on PCs, phones, tablets, and even consoles.

TouchXPRT 2016 installation is quick and easy, and the test completes in under 15 minutes on most devices. The app runs tests based on five everyday tasks (Beautify Photos, Blend Photos, Convert Videos for Sharing, Create Music Podcast, and Create Slideshow from Photos). It measures how long your device takes to complete each task, produces results for each scenario, and gives you an overall score.

As we think about the path forward for TouchXPRT, we’re aware that many expect 2017 to be a year of significant change in the Windows world, with two updates scheduled for release. Microsoft is slated to release the Windows 10 Creators Update (Build 1704) in April, and a subsequent version of Windows codenamed Redstone 3 may arrive this fall. Many tech observers believe that the Creators Update will introduce new creativity and gaming features, along with a UI upgrade named Project NEON. Major foundational shifts in the OS’s structure are more likely to appear with Redstone 3. At this point, quite a lot is still up in the air, but we’ll be following developments closely.

As we learn more about upcoming changes, we’ll have the opportunity to reevaluate TouchXPRT workloads and determine the best way to incorporate new technologies. Virtual reality, 3D, and 4K are especially exciting, but it’s too soon to know how we might incorporate them in a future version of TouchXPRT.

Because TouchXPRT 2016 continues to run well on a wide range of Windows 10 devices, we think it’s best to keep supporting the current version until we get a better idea of what’s in store for Windows.

If you have any thoughts on the future of Windows performance testing, please let us know!

Bill

Posted in BatteryXPRT 2014 for Android, BenchmarkXPRT development community, CrXPRT, Future of performance evaluation, HDXPRT, MobileXPRT, PCs, TouchXPRT, TouchXPRT 2016, TouchXPRT development process, TouchXPRT release cycle, WebXPRT 2017, Windows 10 |

WebXPRT in 2017

By Bill Catchings

on December 8, 2016

Over the last few weeks, we’ve discussed the future of HDXPRT and BatteryXPRT. This week, we’re discussing what’s in store for WebXPRT in 2017.

WebXPRT is our most popular tool. Manufacturers, developers, consumers, and media outlets in more than 350 cities and 57 countries have run WebXPRT over 113,000 times to date. The benchmark runs quickly and simply in most browsers and produces easy-to-understand results that are useful for comparing web browsing performance across a wide variety of devices and browsers. People love the fact that WebXPRT runs on almost any platform that has a web browser, from PCs to phones to game consoles.

More people are using WebXPRT in more places and in more ways than ever before. It’s an unquestioned success, but we think this is a good time to make it even better by beginning work on WebXPRT 2017. Any time change comes to a popular product, there’s a risk that faithful fans will lose the features and functionality they’ve grown to love. Relevant workloads, ease of use, and extensive compatibility have always been the core components of WebXPRT’s success, so we want to reassure users that we’re committed to maintaining all of those in future versions.

Some steps in the WebXPRT 2017 process are straightforward, such as the need to reassess the existing workload lineup and update content to reflect advances in commonly used technologies. Other steps, such as introducing new workloads to test emerging browser technologies, may be tricky to implement, but could offer tremendous value in the months and years ahead.

Are there test scenarios or browser technologies you would like to see in WebXPRT 2017, or tests you think we should get rid of? Please let us know. We want to hear from you and make sure that we’re crafting a performance tool that continues to meet your needs.

Bill

Posted in BatteryXPRT 2014 for Android, Future of performance evaluation, HDXPRT, HTML5, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2017 |

Machine learning

By Bill Catchings

on November 23, 2016

A couple months ago I wrote about doing an inventory of our XPRT tools. Part of that is taking a close look at the six existing XPRTs. The first result of that effort was what I recently wrote about HDXPRT. We’re also looking at emerging technology areas where the BenchmarkXPRT Community has expertise that can guide us.

One of the most exciting of these areas is machine learning. It has rapidly gone from interesting theoretical research (they called them “neural nets” back when I was getting my computer science degree) to something we all use whether we realize it or not. Machine learning (or deep learning) is in everything from intelligent home assistants to autonomous automobiles to industrial device monitoring to personalized shopping in retail environments.

The challenge with developing a benchmark for machine learning is that these are still the early days of the technology. In the past, XPRTs have targeted technologies later in the product cycle. We’re wondering how the XPRT model and the members of its community can play a role here.

One possible use of a machine-learning XPRT is with drones, a market that includes many vendors. Consumers, hobbyists, builders, and the companies creating off-the-shelf models could all benefit from tools and techniques that fairly compare drone performance.

The best approach we’ve come up with to define a machine-learning XPRT starts with identifying common areas such as computer vision, natural language processing, and data analytics, and then, within each of those areas, identifying common algorithms such as AlexNet, GoogLeNet, and VGG. We would also look at the commonly used frameworks such as Caffe, Theano, TensorFlow, and CNTK.

The result might differ from an existing XPRT where you simply run a tool and get a result. Instead, it might take the form of sample code and workloads. Or, maybe even one or two executables that could be used in the most common environments.

At this point, our biggest question is, What do you think? Is this an area you’re interested in? If so, what would you like to see a machine-learning XPRT do?

We’re actively engaging with people in these emerging markets to gauge their interest as well. Regardless of the feedback, we’re excited about the possibilities!

Bill

Posted in Collaborative benchmark development, Drones, Future of performance evaluation, HDXPRT, Machine learning |

How do you say that?

By Eric Hale

on November 10, 2016

I recently saw this video, and heard something that I had never imagined: “Next we tested with what I assume is pronounced web-export.” I’ve had people ask if it was an acronym, but I ‘ve never heard it pronounced “export.”

How do we pronounce XPRT? The same way we pronounce “expert.” So, it’s “Benchmark expert,” “Web expert,” “Touch expert,” and so on. CrXPRT is pronounced “C‑R expert” and HDXPRT is pronounced “H‑D‑expert.”

When I was working in Australia, I got teased about my accent quite a bit, and my case-hardened American R was a particular target. So, when I say the letters out loud, is comes out something like “eks‑pee‑arrr‑tee,” (arrr like a pirate would say it) and “expert” is the closest match. This is true for most Americans. However, in many other accents, it’s more like “eks‑pee‑ah‑tee,” and “ex-paht” is much closer to “export.”

Yes, I think way too much about this stuff.

Eric

Posted in BenchmarkXPRT, BenchmarkXPRT development community, CrXPRT, HDXPRT, MobileXPRT, TouchXPRT, WebXPRT |

Category: HDXPRT

Looking under the hood

A new HDXPRT 2014 build is available

TouchXPRT’s future

WebXPRT in 2017

Machine learning

How do you say that?

Check out the other XPRTs: