Category: Future of performance evaluation

IDF16 Shenzhen

on April 14, 2016

I just spent the last couple of days at IDF16 Shenzhen. It was a great opportunity to talk to folks about the XPRTs, see some future technology demos, and think about the future of the XPRTs.

The technology and product demos included lots of interesting technology. I saw everything from the latest computers to games to VR to body monitoring.

Of particular interest to me were the future-looking technologies beyond the usual array of notebooks, tablets, and servers. I was able to see drones that could video a person by following them, while avoiding obstacles such as trees. I saw a number of demos using the Oculus Rift. I got to see some robot demos that were impressive in their use of the fairly off-the-shelf technology driving them. I would have had myself scanned and then had a small 3D model of myself printed, but I was pressed for time and the line was too long.

I was particularly interested in a mirror that could scan a person and tell things about their health. I also found somewhat amusing a technology demo that was able to “beautify” a person in real time for use with teleconferencing such as Skype. While I might quibble about the definition of beautify, the idea of real-time video enhancement is intriguing. (Given the raw material I gave it to work with, it was no easy task to accomplish!) Maybe I won’t need to shave before my next WebEx meeting…

All of these technologies give some hints as to areas the XPRTs may go in the future. While I don’t think we are quite ready for BeautificationXPRT, there may well be some workloads we should consider such as path finding, real-time video enhancement, health monitoring, virtual reality, and gaming. Please let us know your thoughts about what near-term technologies we should be considering in future workloads.

We definitely have exciting times still ahead of us in technology and the XPRTs!

Bill

Posted in BenchmarkXPRT development community, Future of performance evaluation, IDF, Internet of things, IoT, Mobile devices, Performance of computing devices, Virtual reality |

One benchmark to test them all

By Bill Catchings

on April 7, 2016

It’s no secret that the XPRTs are a great way to get device results you can count on. Tens of thousands of people over six continents have used the XPRTs to help them make smart buying choices, and over a thousand media outlets have quoted XPRT results when reporting on the hottest tech. WebXPRT has always been the “go to” XPRT, because you can use it to test the widest range of devices. WebXPRT runs in the browser, however, so browser performance influences the results.

For a long time, our members and others have asked for a tool that would let you compare application performance on any type of device. People want a cross-platform XPRT that runs on devices the same way apps do.

We’re excited to announce that we’re going to create just that tool! Specifically, we’re going to create a version of MobileXPRT that runs on Android, iOS, and Windows.

This will not be easy. At one point in my career, I was in charge of a group that ported applications between platforms, and I learned from hands-on experience that doing that job well is very difficult. It’s not enough to simply make the application run; it also has to run efficiently on each type of system. MobileXPRT works at the application level, so we’ll have to deal with the many differences in the operating system architectures and APIs. We’ll have to make sure the code runs well on all three target OSes.

We’re willing to do all this work because the need for such a tool has never been greater. More and more devices hit the market all the time, and choosing the ones you want is tougher than ever. iPhone or Android phone? Windows tablet, Android tablet, or iPad?

The coming MobileXPRT will let buyers around the world answer those questions.

We’re not going to do this work in isolation. We will reach out to the OS vendors, because we want their input, comments, and help. We’ll make the source available to them, and we welcome their critiques and guidance in creating the best possible version for each OS.

Of course, we very much want your input, too. Do you have any thoughts about what you’d like to see in a cross-platform XPRT? If so, let us know!

Bill

Last week in the XPRTs
We published the XPRT Weekly Tech Spotlight on the Apple iPhone SE.
We added one new MobileXPRT ’15 result.
We added seven new WebXPRT ’15 results.

Posted in Android, Application-based benchmarks, Benchmark metrics, Benchmarking, BenchmarkXPRT development community, Collaborative benchmark development, Future of performance evaluation, iOS, MobileXPRT, Performance benchmarking, WebXPRT, Windows 10 |

Women develop new perspectives for the XPRTs

By Jennie Faries

on March 18, 2016

Last weekend, we had the great privilege of co-hosting the first XPRT Women Code-a-Thon with the Seattle chapter of ChickTech. We couldn’t be happier with the results!

Our goal was to bring together a group of women and invite them to develop ideas for new device workloads—workloads that we might include in future versions of MobileXPRT and WebXPRT. The 20 participants—some working individually, and others working as teams—not only met that goal, they did a great deal more.

On the coding front, the participants achieved an impressive amount of work in a very short time. Though we awarded only three prizes, everyone generated interesting and useful ideas. Our prizes went to the following people:

1st place: Viveret Steele, for a 3D-modeling workload

2nd place: Annmarie Aidoo, for a geolocation workload

3rd place: Molly Fallen and Alex Trimble, for an audio-enhancement workload

These four people went home with checks, but winning wasn’t what motivated anyone to participate. Everyone was excited about developing software and working with others. The social side of the event proved to be as meaningful as the technical. People talked, formed friendships and mentoring relationships, and discussed seeking other events like this one. Two people said the event changed their lives.

In the weeks ahead, we’ll be sharing some more information about the event. In the meantime, we’re proud to have been part of it.

Jennie Faries

Last week in the XPRTs

We published the XPRT Weekly Tech Spotlight on the Microsoft Surface 3.
We added two new CrXPRT ’15 results.
We added two new MobileXPRT ’13 results.
We added six new WebXPRT ’15 results.

Posted in BenchmarkXPRT, BenchmarkXPRT development community, ChickTech Seattle, Collaborative benchmark development, Future of performance evaluation, MobileXPRT, WebXPRT, What makes a good benchmark?, XPRT Women Code-a-Thon |

Is it hot in here?

By Eric Hale

on March 3, 2016

One of the great meetings I had at CES was with another community member, Patrick Chang, Senior System Engineer in the Tablet Performance group at Dell. I was glad to hear that, when he tests his devices, Patrick makes frequent use of TouchXPRT.

While TouchXPRT stresses the system appropriately, Patrick’s job requires him to understand not only how well the device performs, but why it performs that way. He was wondering what we could do to help him correlate the temperature, power consumption, and performance of a device.

That information is not typically available to software and apps like the XPRTs. However, it may be possible to add some hooks that would let the XPRTs coordinate with third-party utilities and hardware that do.

As always, the input from the community guides the design of the XPRTs. So, we’d love to know how much interest the community has in having this type of information. If you have thoughts about this, or other kinds of information you’d like the XPRTs to gather, please let us know!

Eric

Last week in the XPRTs
We published the XPRT Weekly Tech Spotlight on the Apple iPad Pro.
We added one new BatteryXPRT ’14 result.
We added one new CrXPRT ’15 result.
We added one new MobileXPRT ’13 result.
We added four new WebXPRT ’15 results.

Posted in Benchmark metrics, Benchmarking, BenchmarkXPRT, CES, Collaborative benchmark development, Dell, Future of performance evaluation, Patrick Chang, Performance benchmarking, Power consumption, Temperature, TouchXPRT, Trade Shows, What makes a good benchmark? |

XPRT Women Code-a-Thon: Make your voice heard and win a cash prize

By Eric Hale

on March 1, 2016

DURHAM, NC –(Marketwired – March 01, 2016) – The BenchmarkXPRT Development Community and ChickTech are co-hosting the XPRT Women Code-a-Thon on March 12-13 in Seattle. The code-a-thon encourages Seattle software programmers to create small apps, or “workloads,” that mimic actions they take on their devices every day.

The top three participants or teams will receive cash prizes of up to $2,500, and all participants’ workloads will be considered for inclusion in future versions of the BenchmarkXPRT tools, or XPRTs. Any programmer familiar with Web development or Android development is encouraged to participate.

The XPRTs are apps that empower people all over the world to test how well devices handle everyday activities. They do this by running workloads that simulate common tasks – just like the workloads code-a-thon participants will be building.

“We want the XPRTs to reflect how people actually use their technology every day,” said Jennie Faries. Faries is one of the code-a-thon’s judges and a developer at Principled Technologies, which administers the BenchmarkXPRT Development Community. “By gaining the perspectives of this group of women, we’re making the tools stronger and more realistic. And when the tools we use to measure technology get better, the technology itself gets better too.”

All participants will receive a t-shirt and locally sourced breakfast and lunch on both days of the code-a-thon. The event will include time for networking and conclude with a talk from a special keynote speaker.

Add your voice to the tools that measure today’s hottest tech. Register today at facts.pt/XPRTcodeathon2016_registration, learn more at facts.pt/XPRT codeathon2016, and get all the details at facts.pt/XPRTcodeathon2016_FAQ.

About ChickTech

ChickTech envisions a safe, inclusive, and innovative technology future that includes equal pay, participation, and treatment of women. It is dedicated to retaining women in the technology workforce and increasing the number of women and girls pursuing technology-based careers. For more information, please visit http://chicktech.org

About the BenchmarkXPRT Development Community

The BenchmarkXPRT Development Community is a forum where registered members can contribute to the process of creating and improving the XPRTs. For more information, please visit http://www.principledtechnologies.com/benchmarkxprt

About Principled Technologies, Inc.

Principled Technologies, Inc. is a leading provider of technology marketing and learning & development services. It administers the BenchmarkXPRT Development Community.

Principled Technologies, Inc. is located in Durham, North Carolina, in NC’s Research Triangle Park region. For more information, please visit www.PrincipledTechnologies.com.

Company Contact
Jennie Faries
Principled Technologies, Inc.
1007 Slater Road, Suite #300
Durham, NC 27703

Posted in Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, BenchmarkXPRT Press Releases, Collaborative benchmark development, Future of performance evaluation, MobileXPRT, Performance benchmarking, Press Releases, Principled Technologies, WebXPRT, XPRT Women Code-a-Thon |

Mobile World Congress 2016 and the need for more

By Mark Van Name

on February 25, 2016

Nothing shows you how much more bandwidth we need than a techie trade show like Mobile World Congress 2016. No matter how much the show’s organizers made available, the attendees swamped it with data from their many devices—phones, tablets, and PCs.

This show also demonstrated that we’re going to need a lot more of something else: device performance.

Some people like to say that our current devices are fast enough, but those people either weren’t at MWC or weren’t paying attention. New, demanding workloads were on display everywhere. High-end graphics. Support for an ever-growing range of wearables. Virtual reality. Augmented reality. The ability to act as a hub for all sorts of home automation devices. These and other new capabilities place ever-increasing demands on devices—and they’re all just getting started.

As I walked all of the MWC buildings—and I did at least walk by every single exhibit—I was struck again and again by how many cool technologies are on the cusp of being ready for prime time. They’ll bring nifty features to our everyday lives, and they’ll place heavy demands on our devices to support them and enable them to run well.

Some devices will handle these demands better than others, but we won’t be able to tell the winners from the losers just by looking at them. We’ll need reliable, relevant, real-world benchmarks to sort the winners from the posers—and that means we’ll need the XPRTs. We’ll need the XPRTs we have today, and we’ll need new XPRTs and/or new XPRT workloads for the future. We’ll need help from everyone—members of the BenchmarkXPRT Development Community and vendors yet to join it—to create these new tools, so that buyers everywhere can make smart purchase decisions.

It’s an exciting time. The future for tech, for devices, and for the XPRTs is bright. Let’s get busy creating it.

Posted in BenchmarkXPRT, BenchmarkXPRT development community, Future of performance evaluation, Internet of things, IoT, Mobile devices, Mobile World Congress, PCs, Performance benchmarking, Trade Shows |