BenchmarkXPRT Development Community Blog

There’s a lot going on!

on February 11, 2016

We recently released TouchXPRT 2016 CP3 to the community. Testing has been going well, so we are releasing TouchXPRT 2016 to the general public on Monday. Thanks to everyone who tried out the previews!

Back in December, we told you about Nebula Wolf, a game-based workload developed by students at North Carolina State University. Now we have a new video that talks about the project and why efforts such as this one matter for the XPRTs. It’s a gorgeous video. I think it’s one of our best!

We’ve also created a page that talks about the Nebula Wolf project. And, of course, the page has links for the community members to see the game in action and check out the source code. If you’re not a member, it’s easy to join!

Our next community outreach effort will be the XPRT Women Code-a-Thon March 12-13 in Seattle. It’s a great chance to contribute to the XPRTs, hang with other coders, and maybe even win some prize money!

Later this month, Mark will be at Mobile World Congress in Barcelona. If you’d like to chat with him, just let us know.

Finally, the XPRT Weekly Tech Spotlight shines on the Google Pixel C this week.

That’s a lot, and there’s more to come!

Eric

Posted in Mobile World Congress, Nebula Wolf, North Carolina State University, TouchXPRT, TouchXPRT 2016, TouchXPRT development process, TouchXPRT release cycle, XPRT Weekly Tech Spotlight, XPRT Women Code-a-Thon |

North Carolina State University Senior Design Center students help the XPRTs

By Eric Hale

on February 9, 2016

DURHAM, NC –(Marketwired – February 09, 2016) – Principled Technologies (PT) today announced that it sponsored a senior project at the NC State University Computer Science Department Senior Design Center. PT asked a team of three seniors involved in this project to build a workload that reflected their interests. PT and the BenchmarkXPRT Development Community will evaluate adding this workload to existing BenchmarkXPRT tests in the future.

The BenchmarkXPRT tools, or XPRTs, are apps that empower people all over the world to measure how well their devices handle everyday activities. The XPRTs do this by running workloads that simulate common tasks — like editing photos and writing notes. To help make sure these workloads represent a wide range of viewpoints and ways of using devices, PT reached out to young adults to see what mattered to them.

Senior Design Center team members Rachel Ashburn, Brien Croft, and AJ Smith chose to create a mini-game that shows how well different devices handle games. Input like this potential workload helps better reflect how people really use their devices.

Learn more at: facts.pt/NebulaWolfTale

See why this matters to you at: facts.pt/NebulaWolf

About Principled Technologies, Inc.

Principled Technologies, Inc. is the leading provider of technology marketing and learning & development services. It administers the BenchmarkXPRT Development Community.

Principled Technologies, Inc. is located in Durham, North Carolina, in NC’s Research Triangle Park region. For more information, please visit www.PrincipledTechnologies.com.

About the BenchmarkXPRT Development Community

The BenchmarkXPRT Development Community is a forum where registered members can contribute to the process of creating and improving the XPRTs. For more information, please visit: www.BenchmarkXPRT.com

About the NC State University Computer Science Department Senior Design Center

The Senior Design Center brings together Computer Science seniors and sponsor companies to work on a specific project for a semester. This collaboration provides valuable hands-on experience for the students and important project results for the companies sponsoring them. For more information, please visit: https://sdc.csc.ncsu.edu

Company Contact
Eric Hale
Principled Technologies, Inc.
1007 Slater Road, Suite #300
Durham, NC 27703

Posted in Press Releases |

Comparing apples and oranges?

By Eric Hale

on February 4, 2016

My first day at CES, I had breakfast with Brett Howse from AnandTech. It was a great opportunity to get the perspective of a savvy tech journalist and frequent user of the XPRTs.

During our conversation, Brett raised concerns about comparing mobile devices to PCs. As mobile devices get more powerful, the performance and capability gaps between them and PCs are narrowing. That makes it more common to compare upper-end mobile devices to PCs.

People have long used different versions of benchmarks when comparing these two classes of devices. For example, the images for benchmarking a phone might be smaller than those for benchmarking a PC. Also, because of processor differences, the benchmarks might be built differently, say a 16- or 32-bit executable for a mobile device, and a 64-bit version for a PC. That was fine when no one was comparing the devices directly, but can be a problem now.

This issue is more complicated than it sounds. For those cases where a benchmark uses a dumbed-down version of the workload for mobile devices, comparing the results is clearly not valid. However, let’s assume that the workload stays the same, and that you run a 32-bit benchmark on a tablet, and a 64-bit version on a PC. Is the comparison valid? It may be, if you are talking about the day-to-day performance a user is likely to encounter. However, it may not be valid if you are making statement about the potential performance of the device itself.

Brett would like the benchmarking community to take charge of this issue and provide guidance about how to compare mobile devices and PCs. What are your thoughts?

Eric

Posted in AnandTech, BenchmarkXPRT development community, Brett Howse, CES, Collaborative benchmark development, Future of performance evaluation, Mobile devices, PCs, Performance of computing devices, Performance testing on tablets, What makes a good benchmark? |

The XPRT Women Code-a-Thon

By Eric Hale

on January 28, 2016

As Justin explained last week, we’ve resolved the issue we found with the TouchXPRT CP. I’m happy to say that the testing went well and that we released CP3 this week.

It’s been only three weeks since we announced the XPRT Weekly Tech Spotlight, and we already have another big announcement! Principled Technologies has joined with ChickTech Seattle to host the first ever XPRT Women Code-a-Thon! In this two-day event, participants will compete to create the best new candidate workload for WebXPRT or MobileXPRT. The workloads can’t duplicate existing workloads, so we are looking forward to seeing the new ideas.

Judges will study all the workloads and award prizes to the top three: $2,500 for first place, $1,500 for second place, and $1,000 for third place. Anyone interested can register here.

PT and the BenchmarkXPRT Development Community are committed to promoting the advancement of women in STEM, but we also win by doing good. As with the NCSU senior project, the BenchmarkXPRT Development Community will get some fresh perspectives and some new experimental test tools. Everyone wins!

So much has happened in 2016 and January isn’t even over yet. The year is off to a great start!

Eric

Posted in Benchmark metrics, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development, Future of performance evaluation, MobileXPRT, Performance benchmarking, TouchXPRT, TouchXPRT 2016, WebXPRT, What makes a good benchmark?, XPRT Weekly Tech Spotlight, XPRT Women Code-a-Thon |

TouchXPRT 2016 Community Preview 3 is available

By Eric Hale

on January 28, 2016

Today we are releasing TouchXPRT 2016 Community Preview 3 (CP 3). As we discussed in the blog, CP 3 resolves an issue reported on some systems, where the Create Slideshow test would take longer than expected to complete due to the output video not rendering correctly.

As with all community previews, TouchXPRT 2016 CP 3 is available only to community members. Members may download the preview from the TouchXPRT tab in the Members’ Area.

For more details about TouchXPRT 2016, please consult the release notes for further details (login required).

After you try out CP 3, please send us your comments. Either post them to the forum or mail them to BenchmarkXPRTsupport@principledtechnologies.com. If you send us information that’s relevant to the entire community, we may post an anonymous version of your comments to the forum.

Thanks for your participation!

Posted in BenchmarkXPRT development community, Collaborative benchmark development, Community Preview, Performance benchmarking, TouchXPRT, TouchXPRT 2016, TouchXPRT development process, TouchXPRT release cycle, Windows 10 |

Detective work

By Justin Greene

on January 21, 2016

A few weeks ago, we described an issue with TouchXPRT’s Create Slideshow test. On some systems, this test would take longer than expected to complete due to the output video not rendering correctly. Since then, we released TouchXPRT 2016 CP2, which includes a check that verifies the output and reports an error if there’s a problem.

It took a lot of detective work, but I’m happy to say that we’ve been able to resolve this issue. We had to make a couple of changes. First, we changed the pixel format in which the photo content is read from GUID_WICPixelFormat32bppPBGRA to GUID_WICPixelFormat32bppBGR. Second, we changed the media format of the input into the video converter from MFVideoFormat_ARGB32 to MFVideoFormat_RGB32.

After making these changes, the Create Slideshow test ran in the expected amount of time, and rendered the output video correctly.

In our testing so far, results have not changed noticeably. However, we’re continuing to test. We want to be sure that the updated build is stable and that results are comparable to earlier TouchXPRT CPs. We’ll then release it to the community.

Here’s looking forward to the release of TouchXPRT 2016 to the world at large!

Posted in BenchmarkXPRT development community, TouchXPRT, TouchXPRT 2016, TouchXPRT development process, TouchXPRT release cycle, TouchXPRT results, Windows 10 |

There’s a lot going on!

North Carolina State University Senior Design Center students help the XPRTs

Comparing apples and oranges?

The XPRT Women Code-a-Thon

TouchXPRT 2016 Community Preview 3 is available

Detective work

Check out the other XPRTs: