Load Balancing & Auto Scaling

1. Scalability and High Availability

Scalability means that an application or system can handle greater loads by adapting. There are two kinds:

• Vertical Scalability
• Horizontal Scalability (= elasticity)

Scalability is linked but different to High Availability. The deck teaches the distinction with a call center.

Vertical Scalability

• Increasing the size of the instance. For example, your application runs on a t2.micro; scaling it vertically means running it on a t2.large.
• Very common for non-distributed systems, such as a database.
• There is usually a limit to how much you can vertically scale (a hardware limit).

In the call-center analogy: replacing a junior operator with a senior operator.

Horizontal Scalability

• Increasing the number of instances / systems for your application.
• Horizontal scaling implies distributed systems.
• Very common for web applications / modern applications.
• It’s easy to scale horizontally thanks to cloud offerings such as Amazon EC2.

In the analogy: hiring more operators.

High Availability

• Usually goes hand in hand with horizontal scaling.
• Means running your application / system in at least 2 Availability Zones.
• The goal is to survive a data center loss (a disaster).

In the analogy: a first building in New York and a second building in San Francisco.

Exam tip

Scaling up / down = vertical (bigger instance). Scaling out / in = horizontal (more instances). High availability = at least two AZs. A question that says “survive the loss of a data center” is asking for multi-AZ, not for a bigger instance.

2. High Availability and Scalability for EC2

• Vertical Scaling: increase instance size (= scale up / down). The deck’s range runs from t2.nano — 0.5 GB of RAM, 1 vCPU — all the way to u-12tb1.metal — 12.3 TB of RAM, 448 vCPUs.
• Horizontal Scaling: increase the number of instances (= scale out / in), using an Auto Scaling Group and a Load Balancer.
• High Availability: run instances for the same application across multiple AZs, with a multi-AZ Auto Scaling Group and a multi-AZ Load Balancer.

3. Scalability vs Elasticity vs Agility

• Scalability — the ability to accommodate a larger load by making the hardware stronger (scale up) or by adding nodes (scale out).
• Elasticity — once a system is scalable, elasticity means there is some “auto-scaling” so the system scales based on the load. This is “cloud-friendly”: pay-per-use, match demand, optimize costs.
• Agility — not related to scalability; it’s a distractor. New IT resources are only a click away, so you reduce the time to make resources available to your developers from weeks to minutes.

Exam tip

The deck flags agility explicitly as a distractor in scaling questions. If the scenario is about capacity following demand, the word you want is elasticity; agility is about how fast you can get resources at all.

4. What is load balancing?

Load balancers are servers that forward internet traffic to multiple servers (EC2 instances) downstream. Several users hit one load balancer, and it spreads them over the fleet behind it.

Why use a load balancer?

• Spread load across multiple downstream instances.
• Expose a single point of access (DNS) to your application.
• Seamlessly handle failures of downstream instances.
• Do regular health checks on your instances.
• Provide SSL termination (HTTPS) for your websites.
• High availability across zones.

Why use an Elastic Load Balancer?

An ELB (Elastic Load Balancer) is a managed load balancer:

• AWS guarantees that it will be working.
• AWS takes care of upgrades, maintenance, high availability.
• AWS provides only a few configuration knobs.
• It costs less to set up your own load balancer, but it is a lot more effort on your end (maintenance, integrations).

Real-world example

The single-DNS point is what makes the rest of this chapter work: your users only ever know the load balancer’s address, so instances behind it can be added, replaced or killed by an Auto Scaling Group without anything changing for them.

5. The four kinds of load balancer

Load balancer	Layer	Protocols	Notable features
Application Load Balancer (ALB)	Layer 7	HTTP / HTTPS / gRPC	HTTP routing features, static DNS (URL)
Network Load Balancer (NLB)	Layer 4	TCP / UDP	High performance — millions of requests per second, static IP through Elastic IP
Gateway Load Balancer (GWLB)	Layer 3	GENEVE protocol on IP packets	Routes traffic to firewalls you manage on EC2 instances, intrusion detection
Classic Load Balancer (CLB)	Layer 4 & 7		Retired in 2023

The Gateway Load Balancer sits between users (the source) and the application (the destination), sending traffic through third-party security virtual appliances before it reaches the application.

Exam tip

Learn the layer and the giveaway of each: ALB = Layer 7 = HTTP/HTTPS only, static DNS. NLB = Layer 4 = TCP/UDP, static IP, millions of requests/second. GWLB = Layer 3 = GENEVE, third-party firewalls. CLB is retired — if it appears as an option in a “which should you use today” question, it is wrong.

6. Auto Scaling Groups

In real life the load on your websites and applications changes. In the cloud, you can create and get rid of servers very quickly. The goal of an Auto Scaling Group (ASG) is to:

• Scale out (add EC2 instances) to match an increased load.
• Scale in (remove EC2 instances) to match a decreased load.
• Ensure you have a minimum and a maximum number of machines running.
• Automatically register new instances to a load balancer.
• Replace unhealthy instances.
• Cost savings — only run at an optimal capacity, which is the principle of the cloud.

An ASG is defined by three numbers: a minimum size, an actual size / desired capacity, and a maximum size. It scales out as needed between the minimum and the maximum, and when paired with a load balancer the web traffic reaching the load balancer is distributed over whatever instances the ASG currently holds.

Real-world example

Put the two together and you get the standard AWS web tier: a load balancer taking traffic at a single DNS name, an Auto Scaling Group spanning multiple AZs behind it, and EC2 instances that are created, registered, health-checked and replaced without anyone touching them.

7. Scaling strategies

• Manual Scaling — update the size of an ASG manually.
• Dynamic Scaling — respond to changing demand:
  • Simple / Step Scaling — when a CloudWatch alarm is triggered (for example CPU > 70%), add 2 units; when another alarm is triggered (CPU < 30%), remove 1.
  • Target Tracking Scaling — for example: I want the average ASG CPU to stay at around 40%.
• Scheduled Scaling — anticipate scaling based on known usage patterns; for example, increase the minimum capacity to 10 at 5 pm on Fridays.
• Predictive Scaling — uses Machine Learning to predict future traffic ahead of time, automatically provisioning the right number of EC2 instances in advance. Useful when your load has predictable time-based patterns.

Exam tip

Match the strategy to the wording: a CloudWatch alarm threshold → Simple / Step. A target value to hold (“keep CPU around 40%”) → Target Tracking. A known clock time (“every Friday at 5 pm”) → Scheduled. Machine Learning / predicting traffic in advance → Predictive.

Quick recap

Concept	What to remember for the exam
Vertical scalability	Bigger instance (scale up / down); common for non-distributed systems like databases; limited by hardware
Horizontal scalability	More instances (scale out / in); implies distributed systems; the normal pattern for web applications
High availability	Run in at least 2 Availability Zones to survive a data center loss
Elasticity	Auto-scaling based on load — pay-per-use, match demand, optimize costs
Agility	New resources a click away, weeks down to minutes — a distractor in scaling questions
Why a load balancer	Spread load, single DNS entry point, handle instance failures, health checks, SSL termination, HA across zones
ELB	A managed load balancer — AWS guarantees it works and handles upgrades, maintenance and HA
ALB	Layer 7, HTTP / HTTPS / gRPC, HTTP routing, static DNS
NLB	Layer 4, TCP / UDP, millions of requests per second, static IP via Elastic IP
GWLB	Layer 3, GENEVE protocol, routes traffic to firewalls / intrusion detection on EC2
CLB	Layer 4 & 7, retired in 2023
Auto Scaling Group	Minimum / desired / maximum; scales out and in, registers instances with the ELB, replaces unhealthy instances, multi-AZ
Scaling strategies	Manual · Dynamic (Simple/Step, Target Tracking) · Scheduled · Predictive (Machine Learning)