A Principled Technologies report: Hands-on testing. Real-world results.
Microsoft Azure Database for MySQL delivered better performance and lower price per performance than competing Amazon Web Services (AWS) and Google Cloud solutions
Cloud-based MySQL database services provide database administrators and developers with many helpful advantages, such as scalability, automatic firmware updates, and more. Many cloud service providers (CSPs) offer these advantages through their MySQL database services or instances. When it comes to performance, however, our testing showed that Microsoft Azure stands out from the competition.
We assessed the performance of four MySQL services from three CSPs: Microsoft Azure Database for MySQL – Flexible Server, Amazon Aurora™ MySQL, Amazon Relational Database Service (RDS) for MySQL, and Google Cloud™ SQL for MySQL. We saw better performance from Azure Database for MySQL, which handled more database transactions per second (TPS) with lower latency than the other three MySQL services and cost less for the same performance. Better MySQL performance could mean more ecommerce sales, support for more gaming users, increased transactions, or better database performance wherever MySQL serves as the back end.
About Azure Database for MySQL – Flexible Server
According to Microsoft, Azure Database for MySQL – Flexible Server is a “fully managed production-ready database service designed for more granular control and flexibility over database management functions and configuration settings.”1 With the service, users can also enable high availability within either a single availability zone or across multiple availability zones.
The service also helps users optimize costs with on-demand automatic scaling provisioning of additional storage and input/output operations per second (IOPS) as needed, which could be ideal for workloads that do not need continuous provisioned storage and I/O capacities. This automatic storage and I/O provisioning is available for each compute tier: Burstable, General Purpose, and Business Critical. The service runs on Linux and supports the open-source community versions of MySQL 8.0 and 5.7.
We compared the performance of the following cloud-based MySQL services, testing one instance each:
Standard_Eds32_v5 Business Critical Azure Database for MySQL – Flexible Server
Memory-optimized db.r6i.8xlarge Amazon Aurora MySQL on AWS™
Memory-optimized db.r6i.8xlarge Amazon RDS for MySQL, also on AWS
db-perf-optimized-N-32 Google Cloud SQL for MySQL
Each instance had 32-core vCPUs and 256 GB of memory. We maximized disk performance, in IOPS, for each service. Azure automatically adjusted performance via the Auto-IOPS feature, and AWS used a similar function, called IO-Optimized, to maximize performance for the Aurora service. For the AWS RDS and Google Cloud services, we used larger-capacity data volumes to increase the provisioned IOPS to the instance maximums. The nominal maximum IOPS for each of the backing instances were as follows:
Azure Standard_E32ds_v5 - 38,000 IOPS
AWS db.r6i.8xlarge - 40,000 IOPS
Google Cloud db-perf-optimized-N-32 - 60,000 IOPS
Using the benchmark tool sysbench, we ran an online transaction processing (OLTP) MySQL workload of 70 percent reads and 30 percent writes on each CSP service at 128 and 256 threads. We ran both workloads with 5- and 10-minute runtimes, generating four sets of results for each service.
About Amazon Aurora MySQL
Amazon states that Aurora MySQL is “a fully managed, MySQL-compatible, relational database engine that combines the speed and reliability of high-end commercial databases with the simplicity and cost-effectiveness of open-source databases.”2 Part of Amazon RDS, Aurora is also compatible with PostgreSQL and offers “a high-performance storage subsystem” with underlying storage that “grows automatically as needed.”3 To learn more about Amazon Aurora MySQL, visit https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/CHAP_AuroraOverview.html.
Unlike the other offerings we tested, Aurora uses its own proprietary version of MySQL, which differs slightly from the open-source version generally available to the public. Certain features of MySQL 8.0 are not available in Aurora MySQL ver3.4 This could create challenges for organizations choosing MySQL—for example, a company trying to port an application from MySQL 8.0 to Aurora may see issues with their code.
About Amazon RDS for MySQL
According to Amazon, Amazon RDS “makes it easier to set up, operate, and scale MySQL deployments in the cloud” and allows users to “deploy scalable MySQL servers in minutes with cost-efficient and resizable hardware capacity.”5 It offers automated backups, Amazon CloudWatch metrics, and two storage options: General Purpose, for small and medium workloads, and Provisioned IOPS, for OLTP applications that require high performance. Companies such as Intuit Mint and Airbnb utilize Amazon RDS for MySQL.6 To explore more about Amazon RDS for MySQL, visit https://aws.amazon.com/rds/mysql/.
About Google Cloud SQL for MySQL
Google Cloud SQL for MySQL, which Google abbreviates as Cloud SQL, “uses built-in software optimizations to reduce transaction commit latency and improve write throughput” and supports “all major versions of MySQL” (8.0, 5.7, and 5.6), per Google.7 It offers integrations with Google Compute Engine, Google Kubernetes Engine, Datastream, Looker, and BigQuery as well as observability through Cloud SQL Insights and up to a 99.99% availability SLA.8 Google offers more information about Cloud SQL at https://cloud.google.com/sql/mysql.
Comparing features across services
Azure Database for MySQL – Flexible Server supports a variety of features and services, and we compared this support across all four services. Details of our comparison appear in Table 1.
Table 1: A comparison of features and services supported by each of the cloud-based MySQL services we examined, based on publicly available sources.
Service
Azure Database for MySQL
Amazon Aurora MySQL
Amazon RDS for MySQL
Google Cloud SQL for MySQL
MySQL 5.7 support
Yes
Yes
Yes
Yes
MySQL 8.0 support
Yes
Yes
Yes
Yes
Free tier offering
Yes
No
Yes
Yes
Autoscale IOPS
Yes
Yes
No
No
Storage autogrowth
Yes
Yes
Yes
Yes
Local caching SSDs
Yes
Yes
Yes
Yes
Automated patching
Yes
Yes
Yes
Yes
Automatic backups
Yes
Yes
Yes
Yes
Point-in-time-recovery (PITR)
Yes
Yes
Yes
Yes
Enterprise grade security
Yes
Yes
Yes
Yes
Private networking
Yes
Yes
Yes
Yes
Public networking
Yes
Yes
Yes
Yes
Geo-redundant* storage for backup
Yes
Yes*
Yes*
Yes
Zone Redundant High Availability
Yes
Yes
Yes
Yes
Same Zone High Availability
Yes
No
No
No
Dynamic scalability
Yes
Yes
Yes
Yes
Built-in performance monitoring
Yes
Yes
Yes
Yes
Read replicas
Yes
Yes
Yes
Yes
Data-in replication
Yes
Yes
Yes
Yes
Application migration (community MySQL)
Yes
No
Yes
Yes
*Note: While it is technically possible to create geo-redundant backups with AWS using AWS Backup,9 Amazon does not advertise it as a feature of Aurora or RDS, and the feature requires users to perform additional setup using an additional service.
About sysbench
Sysbench is an open-source, scriptable, multi-threaded benchmark tool. According to the tool’s GitHub repository, sysbench has “low overhead even with thousands of concurrent threads.”10 The page also states that the tool can generate and track hundreds of millions of events per second.11Users can “evaluate any MySQL database with sysbench to determine the associated reads, writes, queries/second (QPS), transactions/second (TPS), and latency.”12
Results of our testing
Table 2 shows the TPS and latency outputs from our testing. Following this table, we break down the comparisons and results and discuss how the performance advantages of Azure Database for MySQL could benefit your organization.
Table 2: Transactions per second (TPS) and average latency, in milliseconds, results from our testing. For TPS, higher is better. For average latency, lower is better. Source: Principled Technologies.
Azure Database for MySQL
Amazon Aurora MySQL
Amazon RDS for MySQL
Google Cloud SQL for MySQL
5-min runtime
128 threads
TPS
9,969.88
8,703.15
6,607.71
4,404.33
Avg. latency (ms)
12.84
14.71
19.37
29.06
256 threads
TPS
10,251.60
9,140.24
8,370.71
5,712.49
Avg. latency (ms)
24.97
28.00
30.58
44.80
10-min runtime
128 threads
TPS
9,996.74
8,565.33
7,101.09
4,714.79
Avg. latency (ms)
12.80
14.94
18.02
27.15
256 threads
TPS
10,324.25
9,101.28
8,532.13
5,955.9
Avg. latency (ms)
24.79
28.12
30.00
42.98
Process more MySQL transactions
When we ran the MySQL workload from sysbench on the instances, Azure Database for MySQL handled up to:
16 percent more TPS than the Amazon Aurora MySQL service*
50 percent more TPS than the Amazon RDS for MySQL service
2.26 times the TPS of the Google Cloud SQL for MySQL service
This performance advantage from Azure Database for MySQL could enable your organization to provide a better experience in many use cases where MySQL serves as the back end for an application. For example, the service could enable a better experience so that customer-facing ecommerce sites could process more sales, financial institutions could support more transactions, and gaming applications could handle more users.
As just one visual example of the performance differences between the four MySQL services, Figure 1 shows the TPS each processed during the 5-minute test using 128 threads.
Get lower latency
Azure Database for MySQL service also had lower average latency while running the MySQL workload, with up to:
14 percent less than the Amazon Aurora MySQL service*
33 percent less than the Amazon RDS for MySQL service
55 percent less than the Google Cloud SQL for MySQL service
These results indicate that using Azure Database for MySQL to support MySQL workloads could translate to faster load and response times. In addition to these user-focused benefits, cloud solutions with lower latency could potentially deliver better value by consuming fewer resources to process the same workload.
As an example of the performance differences between the four services, Figure 2 shows the average latency each service delivered during the 5-minute test using 128 threads.
*Based on results from the 10-minute tests, which we did not visualize.
Achieve better performance for your money
For organizations that rely on the cloud for development or production database workloads, cloud costs may comprise a substantial line item in the budget. Keeping those costs in check must be a focus for IT.
To understand how cost might intersect with performance for the four instances we tested, we looked at a price per performance scenario. In this scenario, we examine the monthly cost to run each instance at peak performance for one hour per business day and at 25 percent of peak performance for seven hours per business day.
For the Azure instance, IOPS usage affects price, so we estimated cost with that in mind. To approximate IOPS for cost estimates, we took the average 180 million requests per hour we observed in our testing and divided by 3,600 seconds/hour to convert this into IOPS. During our tests, our Azure instance averaged approximately 50,000 IOPS. (Note some Azure pricing pages refer to both pricing per million “requests” and “IOPS” interchangeably. We confirmed with Microsoft that these terms were equivalent for pricing estimates.)
Figure 3 shows the normalized price per performance that each instance achieved at different thread counts and run times in this scenario, with lower costs being better. With Azure Database for MySQL, organizations could pay up to 54 percent less for the same performance, helping them accomplish more work while remaining in budget.
We took advantage of an accelerated logging feature for the Azure service, which is now available at no additional cost. (During our testing, the feature was available only in preview.) That feature can help boost performance and is included with Azure Database for MySQL.13
Note that expenses vary depending on the plan an organization chooses, which could include using the cloud for short-term performance bursts or needing longer-term options (e.g., one year, three years).
Conclusion
Leveraging a cloud service for MySQL could enhance overall operational efficiency while allowing for seamless database scaling as needed. However, when it comes to MySQL performance, Azure Database for MySQL delivered advantages over Amazon Aurora MySQL, Amazon RDS for MySQL, and Google Cloud SQL for MySQL. In our tests, Azure Database for MySQL handled more TPS with lower latency than the three competitors—up to 2.26 times the TPS and up to 55 percent lower latency—than the three competitors. The Azure service also cost up to 54 percent less for the same performance in our scenario. If the advantages of running MySQL in the cloud appeal to you, consider leveraging Azure Database for MySQL for the performance boost it could provide to your essential workloads.
Thota, Shireesh, “Microsoft Azure innovation powers leading price-performance for MySQL database in the cloud,” accessed November 15, 2023, https://aka.ms/AzureMySQLBenchmarkBlog.
This project was commissioned by Microsoft.
November 2023
Principled Technologies is a registered trademark of Principled Technologies, Inc.
All other product names are the trademarks of their respective owners.
Principled Technologies disclaimer
Principled Technologies is a registered trademark of Principled Technologies, Inc. All other product names are the trademarks of their respective owners.
DISCLAIMER OF WARRANTIES; LIMITATION OF LIABILITY: Principled Technologies, Inc. has made reasonable efforts to ensure the accuracy and validity of its testing, however, Principled Technologies, Inc. specifically disclaims any warranty, expressed or implied, relating to the test results and analysis, their accuracy, completeness or quality, including any implied warranty of fitness for any particular purpose. All persons or entities relying on the results of any testing do so at their own risk, and agree that Principled Technologies, Inc., its employees and its subcontractors shall have no liability whatsoever from any claim of loss or damage on account of any alleged error or defect in any testing procedure or result.
In no event shall Principled Technologies, Inc. be liable for indirect, special, incidental, or consequential damages in connection with its testing, even if advised of the possibility of such damages. In no event shall Principled Technologies, Inc.’s liability, including for direct damages, exceed the amounts paid in connection with Principled Technologies, Inc.’s testing. Customer’s sole and exclusive remedies are as set forth herein.