BenchmarkXPRT Blog banner

Category: WebGL

Considering WebAssembly for WebXPRT 4

Justin Greene
By Justin Greene
on March 25, 2021

Earlier this month, we discussed a few of our ideas for possible changes in WebXPRT 4, including new web technologies that may work well in a browser benchmark. Today, we’re going to focus on one of those technologies, WebAssembly, in more detail.

WebAssembly (WASM) is a binary instruction format that works across all modern browsers. WASM provides a sandboxed environment that operates at native speeds and takes advantage of common hardware specs across platforms. WASM’s capabilities offer web developers a great deal of flexibility for running complex client applications in the browser. That level of flexibility may enable workload scenario options for WebXPRT 4 such as gaming, video editing, VR, virtual machines, and image recognition. We’re excited about those possibilities, but it remains to be seen which WASM use cases will meet the criteria we look for when considering new WebXPRT workloads, such as relevancy to real life, consistency and replicability, and the broadest-possible level of cross-browser support.

One WASM workload that we’re investigating is a web-based machine learning workload with TensorFlow for JavaScript (TensorFlow.js). TensorFlow.js offers pre-trained models for a wide variety of tasks, including image classification, object detection, sentence encoding, and natural language processing. TensorFlow.js originally used WebGL technology on the back end, but now it’s possible to run the workload using WASM. We could also use this technology to enhance one of WebXPRT’s existing AI-themed workloads, such as Organize Album using AI or Encrypt Notes and OCR Scan.

We’re can’t yet say that a WASM workload will definitely appear in WebXPRT 4, but the technology is promising. Do you have any experience with WASM, or ideas for WASM workloads? There’s still time for you to help shape the future of WebXPRT 4, so let us know what you think!

Justin

Posted in benchmark, BenchmarkXPRT, Browser-based benchmarks, Collaborative benchmark development, Cross-platform benchmarks, image processing, Machine learning, Performance benchmarking, TensorFlow, Virtual reality, Web-based testing, WebAssembly, WebGL, WebXPRT, WebXPRT 3 | Tagged , , , , , , , , , |

Moving forward with WebXPRT 4

Justin Greene
By Justin Greene
on March 4, 2021

In the coming months, we’ll be moving forward with the first stages of the WebXPRT 4 development process. It’s been a while since we last asked readers to send their thoughts about web technologies and workloads that may be a good fit for WebXPRT 4, but we’re still very much open to ideas. If you missed our previous posts about possible changes for WebXPRT 4, we recap the most prominent ideas below. We also request specific feedback regarding a potential battery life component.

  • Community members have asked about a WebXPRT 4 battery life test. Any such test would likely be very similar to the performance-weighted battery life test in CrXPRT 2 (as opposed to a simple rundown test). While WebXPRT runs in almost any browser, cross-browser compatibility issues could cause a WebXPRT battery life test to run in only one browser. If this turned out to be the case, would you still be interested in using the battery life test? Please let us know.
  • One of the most promising ideas is the potential addition of one or more WebAssembly (WASM) workloads. WASM is a low-level, binary instruction format that works across all modern browsers. It offers web developers a great deal of flexibility and provides the speed and efficiency necessary for running complex client applications in the browser. WASM enables a variety of workload scenario options, including gaming, video editing, VR, virtual machines, image recognition, and interactive educational content.
  • We are also considering adding a web-based machine learning workload with TensorFlow for JavaScript (TensorFlow.js). TensorFlow.js offers pre-trained models for a wide range of tasks including image classification, object detection, sentence encoding, and natural language processing. We could also use this technology to enhance one of WebXPRT’s existing AI-themed workloads, such as Organize Album using AI or Encrypt Notes and OCR Scan.
  • Other ideas include using a WebGL-based workload to target GPUs, and simulating common web applications.

We’ll start work on WebXPRT 4 soon, but there’s still time to send your comments and ideas, so please do so as quickly as possible!

Justin

Posted in battery life, benchmark, BenchmarkXPRT, Browser-based benchmarks, Collaborative benchmark development, Future of performance evaluation, Performance benchmarking, TensorFlow, WebAssembly, WebGL, WebXPRT, WebXPRT 3 | Tagged , , , , , , , , , |

Potential web technology additions for WebXPRT 4

Justin Greene
By Justin Greene
on August 20, 2020

A few months ago, we invited readers to send in their thoughts and ideas about web technologies and workload scenarios that may be a good fit for the next WebXPRT. We’d like to share a few of those ideas today, and we invite you to continue to send your feedback. We’re approaching the time when we need to begin firming up plans for a WebXPRT 4 development cycle in 2021, but there’s still plenty of time for you to help shape the future of the benchmark.

One of the most promising ideas for WebXPRT 4 is the potential addition of one or more WebAssembly (WASM) workloads. WASM is a low-level, binary instruction format that works across all modern browsers. It offers web developers a great deal of flexibility and provides the speed and efficiency necessary for running complex client applications in the browser. WASM enables a variety of workload scenario options, including gaming, video editing, VR, virtual machines, image recognition, and interactive educational content.

In addition, the Chrome team is dropping Portable Native Client (PNaCL) support in favor of WASM, which is why we had to remove a PNaCL workload when updating CrXPRT 2015 to CrXPRT 2. We generally model CrXPRT workloads on existing WebXPRT workloads, so familiarizing ourselves with WASM could ultimately benefit more than one XPRT benchmark.

We are also considering adding a web-based machine learning workload with TensorFlow for JavaScript (TensorFlow.js). TensorFlow.js offers pre-trained models for a wide variety of tasks including image classification, object detection, sentence encoding, natural language processing, and more. We could also use this technology to enhance one of WebXPRT’s existing AI-themed workloads, such as Organize Album using AI or Encrypt Notes and OCR Scan.

Other ideas include using a WebGL-based workload to target GPUs and investigating ways to incorporate a battery life test. What do you think? Let us know!

Justin

Posted in AI, battery life, benchmark, BenchmarkXPRT, BenchmarkXPRT development community, Browser-based benchmarks, Collaborative benchmark development, Cross-platform benchmarks, image classification, Machine learning, object detection, Performance benchmarking, Virtual reality, Web-based testing, WebAssembly, WebGL, WebXPRT, WebXPRT 3, What makes a good benchmark? | Tagged , , , , , , , , , , , , |

Apples to apples?

Eric Hale
By Eric Hale
on August 25, 2016

PCMag published a great review of the Opera browser this week. In addition to looking at the many features Opera offers, the review included performance data from multiple benchmarks, which look at areas such as hardware graphics acceleration, WebGL performance, memory consumption, and battery life.

Three of the benchmarks have a significant, though not exclusive, focus on JavaScript performance: Google Octane 2.0, JetStream 1.1, and WebXPRT 2015. The three benchmarks did not rank the browsers the same way, and in the past, we‘ve discussed some of the reasons why this happens. In addition to the difference in tests, there are also sometimes differences in approaches that are worth considering.

For example, consider the test descriptions for JetStream 1.1. You’ll immediately notice that the tests are much lower-level tests than the ones in WebXPRT. However, consider these phrases from a few of the test descriptions:

  • code-first-load “…This test attempts to defeat the browser’s caching capabilities…”
  • splay-latency “Tests the worst-case performance…”
  • zlib “…modified to restrict code caching opportunities…”

 

While the XPRTs test typical performance for higher level applications, the tests in JetStream are tweaked to stress devices in very specific ways, some of which are not typical. The information these tests provide can be very useful for engineers and developers, but may not be as meaningful to the typical user.

I have to stress that both approaches are valid, but they are doing somewhat different things. There’s a cliché about comparing apples to apples, but not all apples are the same. If you’re making a pie, a Granny Smith would be a good choice, but for snacking, you might be better off with a Red Delicious. Knowing a benchmark’s purpose will help you find the results that are most meaningful to you.

Eric

Posted in Benchmark metrics, Browser-based benchmarks, JavaScript, Performance benchmarking, WebGL, WebXPRT, What makes a good benchmark? |

A first look

Eric Hale
By Eric Hale
on January 29, 2015

The beta build of WebXPRT 2015 is complete. Although the benchmark is still under development, it looks great!

As we described in the design document, it includes a number of improvements over WebXPRT 2013. There are more, and more demanding, tests. The UI has a cleaner, more modern look, and includes a couple of improvements people have been asking for. There is now descriptive text so that you know what’s happening during the test. Also, a progress indicator in the upper right corner shows you how close the test is to completion. Although the screens aren’t final, here’s a preview of one of the new DNA Sequence Analysis test screens:

WebXPRT DNA

One of the great virtues of WebXPRT is that it runs on almost anything. However, that level of compatibility has limited the range of devices WebXPRT can test. WebXPRT 2015 introduces a set of experimental tests. Running these tests is optional and their results are not included in the Overall Score. We hope that folks will use them to evaluate both new technologies and platform-specific technologies. The first Community Preview will include a WebGL-based 3D experimental test. We expect to add new experimental tests over time and we’re hoping community members will contribute tests or ideas for them.

I’m very excited to see the new WebXPRT. We’re testing now, and I hope to be able to let you know when the Community Preview will be available soon.

Eric

Posted in BenchmarkXPRT development community, Community Preview, WebGL, WebXPRT, WebXPRT 2015 |

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x