Category: Benchmarking

Watching students become masters

on June 23, 2016

As you know, last year, PT sponsored a senior project at the Senior Design Center of North Carolina State University (NCSU). The students created Nebula Wolf, a mini game that might evolve into a future benchmark test. It was a valuable collaboration for us and a very educational experience for the students involved.

I’ve talked before about the emerging technologies we’re considering for new benchmarks. Today, I met with the folks at the NCSU Senior Design Center to discuss a possible future project. We’re hoping to harness the immense energy of these students by having them explore one of these new technologies, and then build on what they discover. Nothing is set yet, but we will, as always, keep you informed as things develop.

We’ll be sharing some exciting news about the XPRT Women Code-a-Thon tomorrow. Check back to find out more! Meanwhile, we hope you enjoy as much as we did the University of Washington Tacoma article on student Viveret, the first place winner of the XPRT Women Code-a-Thon.

Eric

Posted in Benchmarking, BenchmarkXPRT development community, Collaborative benchmark development, Future of performance evaluation, Nebula Wolf, North Carolina State University, What makes a good benchmark?, XPRT Women Code-a-Thon |

Getting it right

By Eric Hale

on June 2, 2016

Back in April Bill announced that we are working on a cross-platform benchmark. We asked for your thoughts and comments, and you’ve been great! We really appreciate all the great ideas.

We’ve been using code from MobileXPRT and TouchXPRT as the basis for some experiments. In his post, Bill talked about the difficulty of porting applications. However, even though we have expertise in porting applications, it’s proving more difficult than we originally thought. Benchmarks are held to a higher standard than most applications. It’s not enough for the code to run reliably and efficiently, it must compare the different platforms fairly.

One thing we know for sure: getting it right is going to take a while. However, we owe it to you to make sure that the benchmark is reliable and fair on all platforms it supports. We will, of course, keep you informed as things progress.

In the meantime, keep sending your ideas!
Eric

Posted in Android, Application-based benchmarks, Benchmark metrics, Benchmarking, BenchmarkXPRT development community, Collaborative benchmark development, Future of performance evaluation, iOS, MobileXPRT, Performance benchmarking, TouchXPRT, WebXPRT, Windows 10 |

Windows 10 upgrade?

By Eric Hale

on May 26, 2016

We’ve gotten reports that HDXPRT 2014 no longer works on newer versions of Windows 10. We ran tests in our labs and found that to be true.

At least one user has reported that the problem may be the version of CPU-Z the benchmark uses.

We’re working to track down the problem and hope to provide a workaround in the near future and a more definitive fix (if necessary) later.

Please let us know if you’ve encountered this issue and if you’ve found any ways to work around it.

Eric

Posted in BenchmarkXPRT development community, Collaborative benchmark development, HDXPRT, HDXPRT 2014, HDXPRT capabilities, HDXPRT development process, Performance benchmarking, Windows 10 |

Personal preference

By Eric Hale

on May 19, 2016

I saw an interesting article recently, Here’s why I gave up my beloved Galaxy S7 for a boring old iPhone. It’s only been a few weeks since we featured the Samsung S7 in the XPRT Weekly Tech Spotlight, so of course I had to read it. The interesting thing is this guy really loved his Samsung S7, and even declared it “the best smartphone I’ve ever used.” He loved its VR capabilities, camera, and its look. He even prefers Android as an operating system.

So why would he give it up for an iPhone 6s Plus? Simply put, battery life. As a self-described heavy user, he found his Samsung S7 dying before 5 PM every day. The iPhone 6s Plus lasted much longer.

This is a good reminder that people have different priorities. Your priority could be having the fastest phone, the longest battery life, the best screen, or the broadest compatibility. This is why there is no such this as “the best device.”

This is why we are always asking for your input. Knowing your priorities helps the community build better tests!

Eric

Posted in Android, Battery life, BenchmarkXPRT development community, Collaborative benchmark development, Phones, What makes a good benchmark?, XPRT Weekly Tech Spotlight |

Sleep studies

By Justin Greene

on May 12, 2016

Last week, we discussed the increasing complexity of power options in Android 6.0. Features such as Doze and App Standby have changed the way that the operating system manages app activity, and the wide array of UI skins used by many vendors ensures that the steps needed for pre-test configuration differ considerably from device to device.

Managing Android’s Doze feature is critical to getting a good BatteryXPRT score. To show how involved this process can be, we thought it might be helpful to present the steps for one device. Below my sig are the configuration steps we used for the Huawei Mate 8, which we recently featured in the XPRT Weekly Tech Spotlight. For other phones we’ve tested, the steps have been quite different. We’re working on distilling our experience for our tips and tricks document, and the updated version of the document will be available soon. If you have any useful tips, please let us know.

Justin

Whitelist BatteryXPRT (there are two ways to do this)
1) Access Battery manager from Settings/Advanced settings or from the Phone Manager app on the home screen.
2) Select Protected apps.
3) Use the toggles beside BatteryXPRT and BatteryXPRT Tests to allow them to keep running after the screen turns off.

Configure sleep settings
1) Open Settings from the home screen.
2) Select Display.
3) Select Sleep.
4) Select Never. This may reset to a default setting on its own. In our case, it reset to 10 minutes.

Configure screen lock settings
1) Open Settings from the home screen.
2) Select Advanced settings.
3) Select Security.
4) Scroll to the bottom of the list and use the toggle to turn off Screen lock. This keeps the device screen from locking after standby periods during the test.

Posted in Android M, Android N, Battery life, BatteryXPRT 2014 for Android, Benchmarking, Mobile devices, Power consumption |

Staying awake

By Eric Hale

on May 5, 2016

Android developers continue to respond to user demands for longer battery life. Android 6 brought a number of enhancements to improve battery life, and Android N promises even more! Device makers often add device-specific options in their customized UIs, which means the options can differ greatly from device to device. A power-conscious user has a lot more control, but with that control comes complexity.

Not that long ago, preparing a device to run BatteryXPRT was simple: turn off automatic updates, kill unnecessary backgrounds apps, and set the screen to stay awake for at least 30 minutes. That’s still the case for many devices. However, it’s often much more complex. You may have to whitelist BatteryXPRT to keep Android from killing it when the device is in standby. Always-on features may interfere with standby. Security lock timers can override screen-timeout settings for the display – and the list goes on.

We’ve been testing a lot of devices and taking notes. In the next couple of weeks, we’ll be updating the BatteryXPRT Tips and Tricks document to reflect what we’ve learned. If you’ve encountered any of these issues, we’d love to add your tips to the list, so please send them our way!

Eric

Posted in Android, Android M, Android N, Battery life, BatteryXPRT 2014 for Android, Benchmarking, Mobile devices, Power consumption |

Category: Benchmarking

Watching students become masters

Getting it right

Windows 10 upgrade?

Personal preference

Sleep studies

Staying awake

Check out the other XPRTs: