Category: IDF

Reflecting on 2016

on January 12, 2017

The beginning of a new year is a good time to look back on the previous 12 months and take stock of everything that happened. Here’s a quick recap of a very busy year:

In 2016, the XPRTs travelled quite a bit. Eric went to CES in Las Vegas, Mark attended MWC in Barcelona, and Bill flew out to IDF16 in Shenzhen.

We also sent a team to Seattle for the first XPRT Women Code-A-Thon, an event we’re very proud to have sponsored and co-hosted along with ChickTech, a nonprofit organization dedicated to increasing the number of women in tech-related fields. The Code-a-thon also served as inspiration for an eight-part video series entitled Women Coding for Change. The series explains the motivation behind the Code-a-thon and profiles several of the participants. If you haven’t watched the videos, check them out. They’re well worth the time.

Speaking of videos, we also published one about Nebula Wolf, the mini-game workload produced through our first collaboration with the North Carolina State Senior Design Center. That experience was promising enough for us to partner with another student team this past fall, which resulted in a virtual reality app that we hope to share with the community in the near future.

Of course, we also continued work on our suite of benchmark tools and related resources. We released TouchXPRT 2016 to the public, published the Exploring TouchXPRT 2016 white paper, and released the TouchXPRT 2016 source code to community members.

In 2016, we unveiled the XPRT Weekly Tech Spotlight, a new way for device vendors and manufacturers to share verified test results with buyers around the world. We put 46 devices in the spotlight throughout the year and published Back-to-School, Black Friday, and Holiday device showcases.

In the last quarter of 2016, we celebrated our most widely-used benchmark, WebXPRT, passing the 100,000-run milestone. WebXPRT is still going strong and is as useful and relevant as ever!

Finally, we ended the year with the exciting news that we’re moving forward with efforts to develop a machine-learning performance evaluation tool. We look forward to engaging with the community in the coming year as we tackle this challenge!

As always, we’re grateful for everyone who’s helped to make the BenchmarkXPRT Development Community a strong, vibrant, and relevant resource for people all around the world. Here’s to a great 2017!

Justin

Posted in CES, ChickTech Seattle, IDF, Machine learning, Mobile World Congress, Nebula Wolf, North Carolina State University, Source code, TouchXPRT 2016, Trade Shows, Video, Virtual reality, White papers, Women Coding for Change, Women in STEM, XPRT Weekly Tech Spotlight, XPRT Women Code-a-Thon |

Seeing the future

By Eric Hale

on July 7, 2016

Back in April we wrote about how Bill’s trip to IDF16 in Shenzhen got us thinking about future benchmarks. Technologies like virtual reality, the Internet of things, and computer vision are going to open up lots of new applications.

Yesterday I saw an amazing article that talked about an automatic computer vision system that is able to detect early-stage esophageal cancer from endoscopy images. These lesions can be difficult for physicians to detect, and the system did very well when compared to four experts who participated in the test. The article contains a link to the original study, for those of you who want more detail.

To me, this is the stuff of science fiction. It’s a very impressive accomplishment. Clearly, new technologies are going to lead to many new and exciting applications.

While this type of application is more specialized than the typical XPRT, things like this get us really excited about the possibilities for the future. Have you seen an application that impressed you recently? Let us know!

Eric

Posted in Benchmarking, BenchmarkXPRT development community, Collaborative benchmark development, computer vision, Future of performance evaluation, IDF, Internet of things, IoT, Virtual reality, What makes a good benchmark? |

Looking ahead

By Eric Hale

on April 21, 2016

It’s only been a couple of weeks since we announced a cross-platform XPRT. It’s still early days, but we’ve already started getting ideas from vendors and media—from both people within the community and those who have not yet joined. We’re incorporating these ideas into our investigations, and plan to be sending a design document for the community to critique in a few weeks.

However, we are always looking ahead and Bill’s trip to IDF16 got us thinking about future benchmarks. Virtual reality is obviously going to be big. Bill said that he thinks he saw more things using the Oculus Rift than there are Oculus Rifts in the world! The Internet of Things has been ramping up for a while now, and shows no sign of slowing down. Computer vision is another emerging area, one with many possible applications. There are a lot of exciting possibilities!

As always, we want to know what you think. What upcoming technologies are you excited about? What would like to see in these benchmarks? Please let us know!

Eric

Posted in Android, Benchmark metrics, BenchmarkXPRT development community, Collaborative benchmark development, Cross-platform benchmarks, Future of performance evaluation, IDF, iOS, MobileXPRT, Virtual reality, Windows 10 |

IDF16 Shenzhen

By Bill Catchings

on April 14, 2016

I just spent the last couple of days at IDF16 Shenzhen. It was a great opportunity to talk to folks about the XPRTs, see some future technology demos, and think about the future of the XPRTs.

The technology and product demos included lots of interesting technology. I saw everything from the latest computers to games to VR to body monitoring.

Of particular interest to me were the future-looking technologies beyond the usual array of notebooks, tablets, and servers. I was able to see drones that could video a person by following them, while avoiding obstacles such as trees. I saw a number of demos using the Oculus Rift. I got to see some robot demos that were impressive in their use of the fairly off-the-shelf technology driving them. I would have had myself scanned and then had a small 3D model of myself printed, but I was pressed for time and the line was too long.

I was particularly interested in a mirror that could scan a person and tell things about their health. I also found somewhat amusing a technology demo that was able to “beautify” a person in real time for use with teleconferencing such as Skype. While I might quibble about the definition of beautify, the idea of real-time video enhancement is intriguing. (Given the raw material I gave it to work with, it was no easy task to accomplish!) Maybe I won’t need to shave before my next WebEx meeting…

All of these technologies give some hints as to areas the XPRTs may go in the future. While I don’t think we are quite ready for BeautificationXPRT, there may well be some workloads we should consider such as path finding, real-time video enhancement, health monitoring, virtual reality, and gaming. Please let us know your thoughts about what near-term technologies we should be considering in future workloads.

We definitely have exciting times still ahead of us in technology and the XPRTs!

Bill

Posted in BenchmarkXPRT development community, Future of performance evaluation, IDF, Internet of things, IoT, Mobile devices, Performance of computing devices, Virtual reality |

Explaining the BenchmarkXPRT Development Community

By Bill Catchings

on August 6, 2015

Over the last year, I’ve spoken about the XPRT benchmarks with people across America, in China (at IDF Shenzhen), and in Europe (at Mobile World Congress). I regularly found myself having to explain how the BenchmarkXPRT Development Community works. While I was glad to do so, I found myself wishing that the many people I wasn’t able to talk with could also learn about how the community works.

To help make that happen, we’ve developed a simple and engaging video. My (admittedly prejudiced) opinion is that it does a great job of explaining how the community works in less than two minutes. That’s a lot faster than I was able to explain it to folks!

We hope you enjoy the video. And we hope you’ll pass it along to other folks who aren’t already part of the community so they can learn how it works and hopefully be persuaded to join us. Thanks!

Bill

Posted in Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, IDF, What makes a good benchmark? |

The XPRTs at IDF15 Shenzhen

By Justin Greene

on April 23, 2015

We recently traveled to Shenzhen, China to anchor Principled Technologies’ booth in the IDF15 Shenzhen technology showcase. Over 60 companies set up displays to interact with approximately 2,000 attendees, and it was a great opportunity for us to connect with innovators in one of the fastest-growing technology hubs in the world.

We spent most of our time talking with people about the benefits of the BenchmarkXPRT Development Community, demonstrating the XPRTs on our display systems, and describing the scope and abilities of each XPRT. Many of the people we talked with showed great interest in the XPRTs’ commitment to developing easy-to-use benchmarks that measure performance and battery life while doing everyday tasks.

Of course, we also spent a good deal of free time exploring Shenzhen and nearby Hong Kong. If you ever get the opportunity to visit either of these dynamic cities, you won’t be disappointed!

The IDF15 Shenzhen technology showcase floor. Our booth is just to the right of the gray-and-white balloon. The booming Shenzhen skyline, including the soon-to-be-second-tallest building in the world.

If we missed you at IDF15, or you have questions or comments about the Benchmark XPRT Development Community, feel free to contact us.