BenchmarkXPRT Blog banner

Category: web microservices

The CloudXPRT v1.1 beta is on the way

Justin Greene
By Justin Greene
on February 11, 2021

As we’ve been working on improvements and updates for CloudXPRT, we’ve been using feedback from community members to determine which changes will help testers most in the short term. To make some of those changes available to the community as soon as possible, we plan to release a beta version of CloudXPRT v1.1 in the coming weeks.

During the v1.1 beta period, the CloudXPRT v1.01 installation packages on CloudXPRT.com and our GitHub repository will continue to include the officially supported version of CloudXPRT. However, interested testers can experiment with the v1.1 beta version in new environments while we finalize the build for official release. 

The CloudXPRT v1.1 beta includes the following primary changes:

  • We’re adding support for Ubuntu 20.04.2 or later, the number one request we’ve received.
  • We’re consolidating and standardizing the installation packages for both workloads. Instead of one package for the data analytics workload and four separate packages for the web microservices workload, each workload will have two installation packages: one for all on-premises testing and one for testing with all three supported CSPs.
  • We’re incorporating Terraform to help create and configure VMs, which will help to prevent situations when testers do not allocate enough storage per VM prior to testing.
  • We use Kubespray to manage Kubernetes clusters, and Kubespray uses Calico as the default network plug in. Calico has not always worked well for CloudXPRT in the CSP environment, so we’re replacing Calico with Weave.


At the start of the beta period, we will share a link to the v1.1 beta download page here in the blog. You’ll be free to share this link. To avoid confusion, we will not add the beta download to the v1.01 downloads available on CloudXPRT.com.

As the beta release date approaches, we’ll share more details about timelines, access, and any additional changes to the benchmark. If you have any questions about the upcoming CloudXPRT v1.1 beta, please let us know!

Justin

Posted in benchmark, Benchmark metrics, BenchmarkXPRT, BenchmarkXPRT development community, Cloud, CloudXPRT, Clusters, Collaborative benchmark development, Cross-platform benchmarks, data analytics, Datacenter, GitHub, Hosted cloud, IaaS, Kubernetes, Linux, on premises, Performance benchmarking, web microservices | Tagged , , , , , , , , , |

The CloudXPRT learning tool is now live!

Justin Greene
By Justin Greene
on January 7, 2021

We’re happy to announce that the CloudXPRT learning tool is now live! We designed the tool to serve as an information hub for common CloudXPRT topics and questions, and to help tech journalists, OEM lab engineers, and everyone who is interested in CloudXPRT find the answers they need as quickly as possible.

The tool features four primary areas of content:

  • The Q&A section provides quick answers to the questions we receive most from testers and the tech press.
  • The CloudXPRT: the basics section describes specific topics such as the benchmark’s target platforms, workloads, companion cloud software, and hardware and software requirements.
  • The Testing and results section covers the testing process, metrics, and how to publish results.
  • The cloud primer provides brief, easy-to-understand definitions of key cloud computing terms and concepts.

The first screenshot below shows the home screen. To illustrate how some of the pop-up information sections appear, the second screenshot shows part of the Key terms and concepts module in the Cloud primer section. 

We’re excited about the new CloudXPRT learning tool! If you have any questions about the tool, or suggestions for additional content to include in it, please let us know!

Justin

Posted in benchmark, BenchmarkXPRT, Cloud, CloudXPRT, data analytics, Hosted cloud, Hybrid cloud, IaaS, Kubernetes, microservices, Performance benchmarking, Servers, web microservices | Tagged , , , , , , , , |

Next up: a white paper about the CloudXPRT data analytics workload

Justin Greene
By Justin Greene
on October 8, 2020

Soon, we’ll be publishing a CloudXPRT white paper that focuses on the benchmark’s data analytics workload. We summarized the workload in the Introduction to CloudXPRT white paper, but in the same way that the Overview of the CloudXPRT Web Microservices Workload paper did, the new paper will discuss the workload in much greater detail.

In addition to providing practical information about the installation package and minimum system requirements for the data analytics workload, the paper will describe test configuration variables, structural components, task workflows, and test metrics. It will also include guidance on interpreting test results and submitting them for publication.

As we’ve noted, CloudXPRT is one of the more complex tools in the XPRT family, with no shortage of topics to explore. Possible future topics include the impact of adjusting specific test configuration options, recommendations for results reporting, and methods for results analysis. If there are specific topics that you’d like us to address in future white papers, please feel free to send us your ideas!

We hope that the upcoming Overview of the CloudXPRT Data Analytics Workload paper will serve as a go-to resource for CloudXPRT testers, and will answer any questions you have about the workload. Once it goes live, we’ll provide links in the Helpful Info box on CloudXPRT.com and the CloudXPRT section of our XPRT white papers page.

If you have any questions, please let us know!

Justin

Posted in benchmark, Benchmark metrics, BenchmarkXPRT, Cloud, CloudXPRT, Clusters, data analytics, Datacenter, Hosted cloud, Hybrid cloud, IaaS, microservices, Performance benchmarking, web microservices |

Fixes for minor CloudXPRT bugs are on the way

Justin Greene
By Justin Greene
on September 24, 2020

We want to let CloudXPRT testers know that we’re close to releasing an updated version (build 1.01) with two minor bug fixes, an improved post-test results processing script, and an adjustment to one of our test configuration recommendations. None of these changes will affect performance or test results, so scores from previous CloudXPRT builds will be comparable to those from the new build.

The most significant changes in CloudXPRT build 1.01 are as follows:

  • In previous builds, some testers encountered warnings during setup to update the version of Kubernetes Operations (kops) when testing on public-cloud platforms (the CloudXPRT 1.00 recommendation is kops version 1.16.0). We are adjusing the kops installation instructions in the setup instructions for the web microservices and data analytics workloads to prevent these warnings.
  • In previous builds, post-test cleanup instructions for public-cloud testing environments do not always delete all of the resources that CloudXPRT creates during setup. We are updating instructions to ensure a more thorough cleanup process. This change applies to test instructions for the web microservices and data analytics workloads.
  • We are reformatting the optional results graphs the web microservices postprocess program creates to make them easier to interpret.
  • In previous builds, the recommended time interval for the web-microservices workload is 120 seconds if the hpamode option is enabled and 60 seconds if it is disabled. Because we’ve found that the 60-second difference has no significant impact on test results, we are changing the recommendation to 60 seconds for both hpamode settings.


We hope these changes will improve the CloudXPRT setup and testing experience. We haven’t set the release date for the updated build yet, but when we do, we’ll announce it here in the blog. If you have any questions about CloudXPRT, or would like to report bugs or other issues, please feel free to contact us!

Justin

Posted in benchmark, Benchmarking, BenchmarkXPRT, CloudXPRT, Clusters, data analytics, Hosted cloud, Performance benchmarking, web microservices | Tagged , , , , , , |

Coming soon: a white paper about the CloudXPRT web microservices workload

Justin Greene
By Justin Greene
on September 3, 2020

Soon, we’ll be expanding our portfolio of CloudXPRT resources with a white paper that focuses on the benchmark’s web microservices workload. While we summarized the workload in the Introduction to CloudXPRT white paper, the new paper will discuss the workload in much greater detail.

In addition to providing practical information about the web microservices installation packages and minimum system requirements, the paper describes the workload’s test configuration variables, structural components, task workflows, and test metrics. It also discusses interpreting test results and the process for submitting results for publication.

As we’ve noted, CloudXPRT is one of the more complex tools in the XPRT family, with no shortage of topics to explore further. We plan to publish a companion overview for the data analytics workload, and possible future topics include the impact of adjusting specific test configuration options, recommendations for results reporting, and methods for analysis.

We hope that the upcoming Overview of the CloudXPRT Web Microservices Workload paper will serve as a go-to resource for CloudXPRT testers, and will answer any questions you have about the workload. Once it goes live, we’ll provide links in the Helpful Info box on CloudXPRT.com and the CloudXPRT section of our XPRT white papers page.

If you have any questions, please let us know!

Justin

Posted in benchmark, Benchmark metrics, Benchmarking, BenchmarkXPRT, BenchmarkXPRT development community, Cloud, CloudXPRT, Clusters, Cross-platform benchmarks, data analytics, Datacenter, IaaS, microservices, Performance benchmarking, Server, web microservices, White papers | Tagged , , , , , , , |

CloudXPRT version 1.0 is here!

Justin Greene
By Justin Greene
on August 27, 2020

The CloudXPRT Preview period has ended, and CloudXPRT version 1.0 installation packages are now available on CloudXPRT.com and the BenchmarkXPRT GitHub repository! Like the Preview build, CloudXPRT version 1.0 includes two workloads: web microservices and data analytics (you can find more details about the workloads here). Testers can use metrics from the workloads to compare IaaS stack (both hardware and software) performance and to evaluate whether any given stack is capable of meeting SLA thresholds. You can configure CloudXPRT to run on local datacenter, Amazon Web Services, Google Cloud Platform, or Microsoft Azure deployments.

Several different test packages are available for download from the CloudXPRT download page. For detailed installation instructions and hardware and software requirements for each, click the package’s readme link. On CloudXPRT.com, the Helpful Info box contains resources such as links to the Introduction to CloudXPRT white paper, the CloudXPRT master readme, and the CloudXPRT GitHub repository.

The GitHub repository also contains the CloudXPRT source code. The source code is freely available for testers to download and review.

Performance results from this release are comparable to performance results from the CloudXPRT Preview build. Testers who wish to publish results on CloudXPRT.com can find more information about the results submission and review process in the blog. We post the monthly results cycle schedule on the results submission page.

We’re thankful for all the input we received during the CloudXPRT development process and Preview period. If you have any questions about CloudXPRT, please let us know.

Justin

Posted in benchmark, Benchmark metrics, BenchmarkXPRT, BenchmarkXPRT development community, Cloud, CloudXPRT, data analytics, Datacenter, GitHub, Google, Hosted cloud, Hybrid cloud, IaaS, microservices, Performance benchmarking, Server, web microservices | Tagged , , , , , , , , , , |

Check out the other XPRTs:

Forgot your password?
© Principled Technologies, Inc. 2024   |   Privacy policy

Cookie Notice: Our website uses cookies to deliver a smooth experience by storing logins and saving user information. By continuing to use our site, you agree with our usage of cookies as our privacy policy outlines.

x