We
have a very talented studio team here at Principled Technologies, and this
week, the team worked with the XPRTs to put together a CES 2023 recap video. In
it, I discuss why we traveled to CES, my overall impressions from the show, and
how the ideas and technologies presented at the show may influence the development
of future XPRT benchmarks. If you missed last week’s post
about my initial thoughts on the advances in AR and VR technology at the show,
or you didn’t get a chance to see some of our photos from the show on social media, this week’s video
provides a good summary of our CES-related activity.
To view the video, you
can follow this link or click the
screenshot below. If you followed CES this year and have any thoughts about how
the XPRTs can help to evaluate emerging technologies, we’d love to hear
from you!
The
last time the XPRTs attended the Consumer Electronics Show
in Las Vegas was in January 2020,
shortly before shutdowns due to the global pandemic began. More than 171,000
people attended that year’s show, the 2021 show was totally virtual, and CES
shortened the 2022 show after many exhibitors and media pulled out during the
Omicron surge. While some aspects of the event are returning to normal this
year, about one-third of the typically jam-packed Las Vegas Convention Center
space is empty, and only about 100,000 people are likely to attend. Nevertheless,
the show is still enormous and full of fascinating new technology.
Just one day into the show, I’ve already noticed some interesting changes in the virtual reality (VR) and augmented reality (AR) areas since I last attended in 2020. One change is a significant expansion in the sensory capabilities of VR equipment. For a long time, VR technologies have focused almost solely on visual and audio input technology and the graphics-rendering capabilities necessary for lag-free, immersive experiences. In 2020, I saw companies working on various types of haptic feedback gear, including full-body suits, that pushed the boundaries of VR beyond sight and sound. Now, several companies are demonstrating significant progress in “real-feel touch” technologies for VR. One such company is HaptX, which is developing a set of gloves (see the picture below) that pump air through “microfluidic actuators” so that users can feel the size and shape of virtual objects they interact with in a VR environment. While we often think of VR being used for gaming and entertainment, advances in realistic, multi-sensory capabilities can lead to VR becoming a valuable tool for all kinds of industrial and professional training applications.
A show attendee tries out HaptX gear.
Another change I’ve noticed is how AR seems poised to move from demos to everyday life by means of integration with all types of smartphone apps. I enjoyed speaking with a representative from a Korean AR company called Arbeon. Arbeon is developing an app that will allow users to point their phone’s camera at an object (a wine bottle in the picture below), and see an array of customizable, interactive AR animations surrounding the object. You’ll be able to find product info, see and leave feedback similar to “likes” and reviews, attach emojis, tag friends, and even purchase the product, all from your phone’s AR-enhanced camera and screen. It’s an interesting concept with limitless applications. While VR is here to stay and getting better all the time, I personally think that AR will become much more integrated into everyday life in the coming years. I also think AR apps for phones will allow the technology to take off more quickly in the near term than clunkier options like AR eyeglasses.
The large screen displays how Arbeon’s AR phone app interacts with objects like a wine bottle.
Of course, thinking about AR has led me to wonder if we’ll be able to incorporate AR-related workloads into future XPRTs. As new technologies place new and unprecedented levels of processing demand on our computing hardware, the need for objective performance evaluation will continue. Providing reliable, objective performance data is why the XPRTs exist, and planning for the future of the XPRTs is why we’re at CES 2023. If you have any thoughts about how the XPRTs can help to evaluate new technologies, we’d love to hear from you!
Two weeks ago, we discussed how users can automate WebXPRT 4 testing by appending several parameters and values to the benchmark’s URL. One of these lets you enable any available experimental workloads during the test run. While we don’t currently offer any experimental workloads for WebXPRT 4, we are seeking suggestions for possible future workload scenarios, or specific web technologies that you’d like to be able to test with an experimental workload.
The main purpose of optional, experimental workloads would be to test cutting-edge browser technologies or new use cases, even if the experimental workload doesn’t work on all browsers or devices. The individual scores for the experimental workloads would stand alone, and would not factor in the WebXPRT 4 overall score. WebXPRT 4 testers would be able to run the experimental workloads one of two ways: by adjusting a value in the WebXPRT 4 automation scripts, as mentioned above, or by manually selecting them on the benchmark’s home screen.
Testers would benefit
from experimental workloads by learning how well certain browsers or systems
handle new tasks (e.g., new web apps or AI capabilities). We would benefit from
fielding workloads for large-scale testing and user feedback before we commit
to including them as core WebXPRT workloads.
Do you have any general thoughts about experimental workloads for browser performance testing, or any specific workloads that you’d like us to consider? Please let us know.
February 28, 2013 was
a momentous day for the BenchmarkXPRT Development Community. On that day, we
published a press release announcing the official launch of the first version
of the WebXPRT benchmark, WebXPRT 2013. As difficult as it is for us to
believe, the 10-year anniversary of the initial WebXPRT launch is in just a few
short months!
We introduced WebXPRT
as a truly unique browser performance benchmark in a field that was already
crowded with a variety of measurement tools. Since those early days, the WebXPRT
market presence has grown from a small foothold into a worldwide industry
standard. Over the years, hundreds of tech press publications have used WebXPRT
in thousands of articles and reviews, and the WebXPRT completed-runs counter
rolled over the 1,000,000-run mark.
New web technologies
are continually changing the way we use the web, and browser-performance
benchmarks should evaluate how well new devices handle the web of today, not
the web of several years ago. While some organizations have stopped development
for other browser performance benchmarks, we’ve had the opportunity to continue
updating and refining WebXPRT. We can look back at each of the four major
iterations of the benchmark—WebXPRT 2013, WebXPRT 2015, WebXPRT 3, and WebXPRT 4—and
see a consistent philosophy and shared technical lineage contributing to a product
that has steadily improved.
As we get closer to the 10-year anniversary of WebXPRT next year, we’ll be sharing more insights about its reach and impact on the industry, discussing possible future plans for the benchmark, and announcing some fun anniversary-related opportunities for WebXPRT users. We think 2023 will be the best year yet for WebXPRT!
As the number
of WebXPRT runs continues to grow, we realize many new WebXPRT users may be unfamiliar with all the
features and capabilities of the benchmark. To help inform users about features
that might facilitate their testing, we’ve decided to highlight a few WebXPRT
features here in the blog. A few weeks ago, we discussed the multiple language options available in the WebXPRT 4 UI.
This week, we look at WebXPRT 4 test automation.
WebXPRT 4 allows
users to run scripts in an automated fashion. You can control the execution of
WebXPRT 4 by appending parameters and values to the WebXPRT URL. Three
parameters are available: testtype, tests, and result. Below, you’ll find a
description of those parameters and instructions for utilizing automation.
Test type
The WebXPRT
automation framework accounts for two test types: (1) the six core workloads
and (2) any experimental workloads we might add in future builds. There are
currently no experimental tests in WebXPRT 4, so always set the test type
variable to 1.
Core tests: 1
Test scenario
This
parameter lets you specify which tests to run by using the following codes:
Photo enhancement: 1
Organize album using AI: 2
Stock option pricing: 4
Encrypt notes and OCR scan using WASM: 8
Sales graphs: 16
Online homework: 32
To run a single individual test, use its code. To run multiple
tests, use the sum of their codes. For example, to run Stocks (4) and Notes
(8), use the sum of 12. To run all core tests, use 63, the sum of all the
individual test codes (1 + 2 + 4 + 8 + 16 + 32 = 63).
Results format
This parameter
lets you select the format of the results:
We hope the WebXPRT automation features will make testing easier
for you. If you have any questions about WebXPRT or the automation process,
please feel free to ask!
The holiday shopping season
is fast approaching, and choosing the right tech gift can often be a daunting
task. If you’re considering phones, tablets, Chromebooks, or laptops as gifts, and
are unsure where to get reliable device information, the XPRTs can help!
The XPRTs provide
objective, reliable measures of a device’s performance that can help to cut
through the marketing noise. For example, instead of guessing whether the
performance of a new laptop lives up to its billing, you can use its WebXPRT performance score to see how it stacks up against the
competition on everyday tasks.
A good place to start looking
for device scores is our XPRT results browser, which lets you access our database of more than 3,200 test
results from over 165 sources, including major tech review publications around
the world, OEMs, and independent testers. You can find a wealth of current and
historical performance data across all the XPRT benchmarks and hundreds of
devices. Learn how to use the results browser here.
If you’re considering
a popular device, chances are good that a recent tech review includes an XPRT
score for it. You can find these reviews by going to your favorite tech review
site and searching for “XPRT,” or entering the name of the device and the
appropriate XPRT (e.g., “iPhone” and “WebXPRT”) in a search engine. Here are a
few recent tech reviews that used the XPRTs to evaluate popular devices:
AnandTech
used WebXPRT to evaluate the MSI Titan GT77 gaming laptop.
The XPRTs can help consumers make better-informed and more confident tech purchases this holiday season, and we hope you’ll find the data you need on our site or in an XPRT-related tech review. If you have any questions about the XPRTs, XPRT scores, or the results database please feel free to ask!
Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.
