Category: Benchmarking

The 2018 XPRT Spotlight Back-to-School Roundup

on August 9, 2018

With the new school year approaching, we’re pleased to announce that we’ve just published our third annual XPRT Spotlight Back-to-School Roundup! The Roundup allows shoppers to view side-by-side comparisons of XPRT test scores and hardware specs from some of this school year’s most popular Chromebooks, laptops, tablets, and convertibles. We tested the devices in our lab using the XPRT benchmarks and provide not only performance scores, but also photo galleries, PT-verified device specs, and prices. Parents, teachers, students, and administrators who are considering purchases for use in their education environments have many options, and the Roundup can help make their decisions easier by gathering product and performance facts in one convenient place. We’ll continue to add devices to the Roundup, including the new Microsoft Surface Go, as they arrive over the next few weeks.

The Back-to-School Roundup is just one of the features we offer through the XPRT Weekly Tech Spotlight. Every week, the Spotlight highlights a new device, making it easier for consumers to select a new laptop, phone, tablet, or PC. Recent devices in the Spotlight include the Acer Chromebook Tab 10, the 2018 Samsung Chromebook Plus, the Intel Hades Canyon NUC mini PC, and the OnePlus 6 phone. The Spotlight device comparison page lets you view side-by-side comparisons of all of the devices we’ve tested.

If you’re interested in having your devices featured in the XPRT Weekly Tech Spotlight or in this year’s Black Friday and Holiday Showcases, which we publish in late November, visit the website for more details.

If you have any ideas for the Spotlight page or suggestions for devices you’d like to see, let us know!

Justin

Posted in Battery life, BatteryXPRT 2014 for Android, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, CrXPRT, Education, Mobile data, TouchXPRT, WebXPRT, XPRT Weekly Tech Spotlight | Tagged convertible, education, laptop, PC, school, Spotlight, tech, XPRT Spotlight |

Worldwide influence

By Justin Greene

on August 2, 2018

On a weekly basis, we track several XPRT-related metrics such as completed test runs and downloads, content views, and social media engagement. We also search for instances where one or more of the XPRTs has been mentioned in an advertisement, article, or review. Compiling all this information gives us some insight into how, where, and how much people are using the XPRT tools.

From time to time, we share some of that data with readers and community members so they can see the impact that the XPRTs are having around the world. One way we do this is through our “XPRTs around the world” infographic, which provides a great picture of the XPRT’s reach.

Here are some key numbers from the latest update:

The XPRTs have been mentioned more than 11,400 times on over 3,600 unique sites.
Those mentions include more than 9,200 articles and reviews.
Those mentions originated in over 590 cities located in 65 countries on six continents. New cities of note include Phnom Penh, Cambodia; Osijek, Croatia; Kano, Nigeria; and Lahore, Pakistan.
The BenchmarkXPRT Development Community now includes 210 members from 74 companies and organizations around the world.

In addition to the growth in web mentions and community members, the XPRTs have now delivered more than 380,000 real-world results!

We’re grateful for everyone who’s helped us get this far! Every contribution to the community, no matter how small, contributes to our goal of providing reliable, relevant, and easy-to-use benchmark tools.

Justin

Posted in Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Collaborative benchmark development |

Planning the next version of MobileXPRT

By Justin Greene

on July 26, 2018

We’re in the early planning stages for the next version of MobileXPRT, and invite you to send us any suggestions you may have. What do you like or not like about MobileXPRT? What features would you like to see in a new version?

When we begin work on a new version of any XPRT, one of the first steps we take is to assess the benchmark’s workloads to determine whether they will provide value during the years ahead. This step almost always involves updating test content such as photos and videos to more contemporary file resolutions and sizes, and it can also involve removing workloads or adding completely new scenarios. MobileXPRT currently includes five performance scenarios (Apply Photo Effects, Create Photo Collages, Create Slideshow, Encrypt Personal Content, and Detect Faces to Organize Photos). Should we stick with these five or investigate other use cases? What do you think?

As we did with WebXPRT 3 and the upcoming HDXPRT 4, we’re also planning to update the MobileXPRT UI to improve the look of the benchmark and make it easier to use.

Crucially, we’ll also build the app using the most current Android Studio SDK. Android development has changed significantly since we released MobileXPRT 2015 and apps must now conform to stricter standards that require explicit user permission for many tasks. Navigating these changes shouldn’t be too difficult, but it’s always possible that we’ll encounter unforeseen challenges at some point during the process.

Do you have suggestions for test scenarios that we should consider for MobileXPRT? Are there existing features we should remove? Are there elements of the UI that you find especially useful or have ideas for improving? Please let us know. We want to hear from you and make sure that MobileXPRT continues to meet your needs.

Justin

Posted in Android, Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Mobile devices, MobileXPRT, MobileXPRT 2015, MobileXPRT 3, Phones, Tablets | Tagged Android, benchmark, benchmarks, mobile, MobileXPRT, MobileXPRT 2015, MobileXPRT 3, phones, photos, tablets |

MWCS18 and AIXPRT: a new video

By Justin Greene

on July 12, 2018

A few weeks ago, Bill shared his first impressions from this year’s Mobile World Congress Shanghai (MWCS). “5G +” was the major theme, and there was a heavy emphasis on 5G + AI. This week, we published a video about Bill’s MWCS experience and the role that the XPRTs can play in evaluating emerging technologies such as 5G, AI, and VR. Check it out!

You can read more about AIXPRT development here. We’re still accepting responses to the AIXPRT Request for Comments, so if you would like to share your ideas on developing an AI/machine learning benchmark, please feel free to contact us.

Justin

Posted in 5G, AI, AIXPRT, AR, Augmented reality, Future of performance evaluation, Machine learning, Mobile devices, Mobile World Congress, Mobile World Congress, Performance benchmarking, Virtual reality | Tagged 5G, AI, AIXPRT, MWCS18, Shanghai, VR |

Which browser is the fastest? It’s complicated.

By Justin Greene

on July 5, 2018

PCWorld recently published the results of a head-to-head browser performance comparison between Google Chrome, Microsoft Edge, Mozilla Firefox, and Opera. As we’ve noted about similar comparisons, no single browser was the fastest in every test. Browser speed sounds like a straightforward metric, but the reality is complex.

For the comparison, PCWorld used three JavaScript-centric test suites (JetStream, SunSpider, and Octane), one benchmark that simulates user actions (Speedometer), a few in-house tests of their own design, and one benchmark that simulates real-world web applications (WebXPRT). Edge came out on top in JetStream and SunSpider, Opera won in Octane and WebXPRT, and Chrome had the best results in Speedometer and PCWorld’s custom workloads.

The reason that the benchmarks rank the browsers so differently is that each one has a unique emphasis and tests a specific set of workloads and technologies. Some focus on very low-level JavaScript tasks, some test additional technologies such as HTML5, and some are designed to identify strengths or weakness by stressing devices in unusual ways. These approaches are all valid, and it’s important to understand exactly what a given score represents. Some scores reflect a very broad set of metrics, while others assess a very narrow set of tasks. Some scores help you to understand the performance you can expect from a device in your everyday life, and others measure performance in scenarios that you’re unlikely to encounter. For example, when Eric discussed a similar topic in the past, he said the tests in JetStream 1.1 provided information that “can be very useful for engineers and developers, but may not be as meaningful to the typical user.”

As we do with all the XPRTs, we designed WebXPRT to test how devices handle the types of real-world tasks consumers perform every day. While lab techs, manufacturers, and tech journalists can all glean detailed data from WebXPRT, the test’s real-world focus means that the overall score is relevant to the average consumer. Simply put, a device with a higher WebXPRT score is probably going to feel faster to you during daily use than one with a lower score. In today’s crowded tech marketplace, that piece of information provides a great deal of value to many people.

What are your thoughts on browser testing? We’d love to hear from you.

Justin

Posted in Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Collaborative benchmark development, Cross-platform benchmarks, HTML5, JavaScript, Performance benchmarking, Web-based testing, WebXPRT, WebXPRT 2015, WebXPRT 3, What makes a good benchmark? |

Thoughts from MWC Shanghai 2018

By Bill Catchings

on June 28, 2018

Ni hao from Shanghai! It is amazing the change that happens in a year. This year’s MWC Shanghai, like last year’s, took up about half of the Shanghai New International Expo Centre (SNIEC). “5G +” is the major theme and, unlike last year, 5G is not something in the distant future. It is now assumed to be in progress.

The biggest of the pluses was AI, with a number of booths explicitly sporting 5G + AI signage. There were also 5G plus robots, cars, and cloud services. Many of those are really about AI as well. The show makes it feel like 5G is everywhere and will make everything better (or at least a lot faster). And Asia is leading the way.

[caption id="attachment_3447" align="alignleft" width="640"]

5G + robotics at MWCS 18.[/caption]

Most of the booths touted their 5G support as they did last year, but rather than talking about the future, they tried to say that their 5G was now. They claimed their products were in real-world tests with anticipated deployment schedules. One of the keynote speakers talked about 1.2 billion 5G connections by 2025, with more than half of those in Asia. The purported scale and speed of the transition to 5G is staggering.

[caption id="attachment_3449" align="alignleft" width="640"]

The keynote stage, displaying some big numbers.[/caption]

The last two halls I visited showed that world is not all 5G and AI. These halls looked at current fun applications of mobile technologies and companies developing technologies in the near future. MWC allowed children into one of the halls, where they (and we adults) could fly drones and experience VR technology. I watched in some amusement as people crashed drones, rode bikes with VR gear to simulate horses, were 3D scanned, and generally tried out new tech that didn’t always work.

The second hall included small booths from new companies working on future technologies that might be ready “4 years from now” (4YFN). These companies did not have much to show yet, but each booth displayed the company name and a short phrase summing up their future tech. That led to “Deepscent Labs is a smart scent data company,” ChineSpain is a “Marketplace of experiences for Chinese tourists in Spain,” and “Juice is a tech-based music contents startup that creates an ecosystem of music.” The mind boggles!

The XPRTs’ foray into AI with AIXPRT seems well timed based on this show. Other areas from this show that may be worth considering for the XPRTs are 5G and the cloud. We would love to hear your thoughts on those areas. We know they are important, but do you need the XPRTs and their emphasis on real-world benchmarks and workloads in those areas? Drop us a line and let us know!

Bill

Posted in 5G, AI, AIXPRT, AR, Augmented reality, Automated cars, Future of performance evaluation, IoT, Mobile data, Mobile devices, Mobile World Congress, Virtual reality | Tagged 5G, AI, China, mobile, Mobile World Shanghai, MWC, Shanghai, VR |

Category: Benchmarking

The 2018 XPRT Spotlight Back-to-School Roundup

Worldwide influence

Planning the next version of MobileXPRT

MWCS18 and AIXPRT: a new video

Which browser is the fastest? It’s complicated.

Thoughts from MWC Shanghai 2018

Check out the other XPRTs: