AWS EC2: T3 vs. M5 Instances (with Benchmark)

Our link checker heavily relies on AWS and EC2 instances in particular. One of the more difficult decisions when dealing with EC2 is choosing the right instance type. Will burstable and (potentially) cheap T3 instances do the job, or should you pay more for general purpose M5 instances? In this blog post, I will try to shed some light on this and provide answers to the following questions:

  • How fast is a T3 instance compared to an equivalently sized M5 instance?
  • How does the T3 CPU credit model work, exactly?
  • What is T3 Unlimited and how reliable is it?

CPU Performance

Without further ado, let me dive directly into some actual testing. I create a t3.large instance (without Unlimited mode) in the us-east-2 region and select Ubuntu 18.04 as the operating system. The first thing I check is the processor the machine is running on:

$ cat /proc/cpuinfo | grep -E 'processor|model name|cpu MHz'

processor       : 0
model name      : Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
cpu MHz         : 2500.000
processor       : 1
model name      : Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
cpu MHz         : 2500.000

As you can see from the output, the T3 instance has access to two vCPUs (= hyperthreads) of an Intel Xeon Platinum 8175M processor.

In order to test the performance of the processor, I install sysbench …

$ sudo apt-get install sysbench

… and start the benchmark (which stresses the CPU for ten seconds by calculating prime numbers):

$ sysbench --threads=2 cpu run

sysbench 1.0.11 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 2
Initializing random number generator from current time

Prime numbers limit: 10000

Initializing worker threads...

Threads started!

CPU speed:
    events per second:  1623.24

General statistics:
    total time:                          10.0012s
    total number of events:              16237

Latency (ms):
         min:                                  1.16
         avg:                                  1.23
         max:                                  9.89
         95th percentile:                      1.27
         sum:                              19993.46

Threads fairness:
    events (avg/stddev):           8118.5000/0.50
    execution time (avg/stddev):   9.9967/0.00

The events per second (1623 in this case) is the number you should care about. The higher this number, the higher the performance of the CPU. As points of reference, here are the results from the same test on several other cloud machines as well as some desktop and mobile processors:

Instance Type Processor Threads Burst Baseline
t3.micro Intel Xeon Platinum 8175M 2 1654 175
t3.medium Intel Xeon Platinum 8175M 2 1653 343
t3.large Intel Xeon Platinum 8175M 2 1623 510
m5.large Intel Xeon Platinum 8175M 2 1647 1647
t3a.nano AMD EPYC 7571 2 1510 75
t2.large Intel Xeon E5-2686 v4 2 1830 538
a1.medium AWS Graviton 1 2281 2281
t4g.micro AWS Graviton2 2 5922 592
m4.large Intel Xeon E5-2686 v4 2 1393 1393
c4.large Intel Xeon E5-2666 v3 2 1583 1583
GCP f1-micro Intel Xeon (Skylake) 1 956 190
GCP e2-medium Intel Xeon 2 1434 717
GCP n1-standard-2 Intel Xeon (Skylake) 2 1435 1435
Linode 2GB AMD EPYC 7501 1 1236 1170
Hetzner CX11 Intel Xeon (Skylake) 1 970 860
Dedicated Server Intel Xeon E3-1271 v3 8 7530 7530
Desktop Intel Core i9-9900K 16 19274 19274
Laptop Intel Core i7-8565U 8 8372 8372

While bursting, T3 instances provide the same CPU performance as equally sized M5 instances, which answers my first question from above.

Investigating the T3 CPU Credit Model

For the next test, I let the t3.large instance sit unused for 24 hours until it has accrued the maximum number of CPU credits.

t3.large CPU Credit Balance after 24 hours

T3 instances always start with zero credits and earn credits at a rate determined by the size of the instance. A t3.large instance earns 36 CPU credits per hour with a maximum of 864 credits. According to the AWS documentation, one CPU credit is equal to one vCPU running at 100% for one minute. So how long should my t3.large instance be able to burst to 100% CPU utilization?

If the instance has 864 CPU credits, uses 2 credits per minute, and refills its credits at a rate of 0.6 (= 36/60) per minute, it should have the capacity to burst for 864 / (2 - 0.6) = 617 minutes = 10.3 hours. Let me put that to a test.

sysbench --time=0 --threads=2 --report-interval=60 cpu run

Just as expected, the performance drops to about 30% (the baseline performance for t3.large instances) after about ten and a half hours.

t3.large CPU Utilization

T3 Unlimited Mode

T3 instances with activated Unlimited Mode are allowed to burst even if no CPU credits are available. This comes at a price: a T3 instance that continuously bursts at 100% CPU costs approximately 1.5 times the price of an equally sized M5 instance. However, how reliable is Unlimited Mode? My worry is that AWS puts too many instances on a single physical machine, so not enough spare burst capacity is available. To answer this question, I launch a t3.nano instance with Unlimited Mode and let it run at full steam for about four days.

t3.nano CPU Utilization (Unlimited Mode)

As promised, there is no drop in CPU performance. The t3.nano instance delivers the full capacity of 2 vCPUs (almost) all the time. Quite impressive!

Network Performance

Instead of running my own network performance tests, I rely on the results that Andreas Wittig published on the cloudonaut blog. He used iperf3 to determine the baseline and burst network throughput for different EC2 instance types. Here are the values for different T3 instances and an m5.large instance:

Instance Type Burst (Gbit/s) Baseline (Gbit/s)
t3.nano 5.06 0.03
t3.micro 5.09 0.06
t3.small 5.11 0.13
t3.medium 4.98 0.25
t3.large 5.11 0.51
m5.large 10.04 0.74

Although an m5.large instance costs only about 15% more than a t3.large instance, it provides a 50% higher baseline throughput and more than double the burst capacity.

When determining how much network throughput you need, consider that EBS volumes are network-attached.

Conclusion and Recommendation

T3 instances are great! Even a t3.nano instance at a monthly on-demand price of less than $4 gives you access to the full power of two hyperthreads on an Intel Xeon processor and at burst runs as fast as a $70 m5.large instance. By activating Unlimited Mode, you can easily insure yourself against running out of CPU credits and being throttled.

If you don’t need the 8 GiB of memory (RAM) that an m5.large instance provides and can live with the lower network throughput, one of the smaller T3 instances with activated Unlimited Mode might be the much more cost-effective choice. In the end, it depends on how high the average CPU usage of your instance is. The table below lists the CPU utilizations up to which bursting T3 instances remain cheaper than an m5.large instance. Please note that the calculations are based on the on-demand prices and might be different when using reserved instances.

Instance Type Memory Cost-effective if average CPU usage is less than
t3.nano 0.5 GiB 95.8%
t3.micro 1 GiB 95.6%
t3.small 2 GiB 95.2%
t3.medium 4 GiB 74.4%
t3.large 8 GiB 42.5%

Last but not least, let me state that you shouldn’t rely solely on the benchmarks and comparisons in this post. You are welcome to use my finding as a first guide when choosing the right instance type, but don’t forget to run your own tests and make your own calculations.

Older Post Newer Post