A few weeks ago, we discussed the beginnings of a
WebXPRT 4 development plan, and asked for reader feedback about potential
workload changes. So far, the two most common feedback topics have been the
possible addition of a WebAssembly workload, and the feasibility
of including a browser-based battery life test. Today, we discuss what a
WebXPRT 4 battery life test would look like, and some of the challenges we’d
have to overcome to make it a reality.
Battery life tests
fall into two primary categories: simple rundown tests and performance-weighted tests. Simple rundown tests measure battery life during
extreme idle periods and loops of movie playbacks, etc., but do not reflect the
wide-ranging mix of activities that characterize a typical day for most users.
While they can be useful for performing very specific apple-to-apples
comparisons, these tests have limited value when it comes to giving consumers a
realistic estimation of the battery life they would experience during everyday
use.
In contrast,
performance-weighted battery life tests, such as the one in CrXPRT 2, attempt to reflect
real-world usage. The
CrXPRT battery life test simulates common daily usage patterns for Chromebooks
by including all the productivity workloads from the performance test, plus
video playback, audio playback, and gaming scenarios. It also includes periods
of wait/idle time. We believe this mixture of diverse activity and idle time better
represents typical real-life behavior patterns. This makes the resulting
estimated battery life much more helpful for consumers who are trying to match
a device’s capabilities with their real-world needs.
From a technical standpoint, WebXPRT’s cross-platform nature presents us with several challenges that we did not face while developing the CrXPRT battery life test for Chrome OS. While the WebXPRT performance tests run in almost any browser, cross-browser differences in battery life reporting may restrict the battery life test to a single browser. For instance, Mozilla has deprecated the battery status API for Firefox, and we’re not yet sure if there’s another approach that would work. If a WebXPRT 4 battery life test supported only a single browser, such as Chrome or Safari, would you still be interested in using it? Please let us know.
A browser-based battery life
workflow also presents other challenges that we do not face in native client
applications such as CrXPRT:
- A
browser-based battery life test would require the user to check the starting and
ending battery capacities, with no way for the app to independently verify data
accuracy.
- The
battery life test could require more babysitting in the event of network
issues. We can catch network failures and try to handle them by reporting
periods of network disconnection, but those interruptions could influence the
battery life duration.
- The
factors above could make it difficult to achieve repeatability. One way to address
that problem would be to run the test in a standardized lab environment lab
with a steady internet connection, but a long list of standardized
environmental requirements would make the battery life test less attractive and
less accessible to many testers.
Our intention with
today’s blog is not to make a WebXPRT 4 battery life test seem like an
impossibility. Rather, we want to share our perspective on what the test might
look like, and describe some of the challenges and considerations in play. If you
have thoughts about battery life testing, or experience with battery life APIs
in one or more of the major browsers, we’d love to hear from you!
Justin
