High Availability & Scalability – ELB and Auto Scaling Groups

1. Scalability and High Availability

Scalability means an application or system can cope with a greater load by adapting to it. The deck insists on splitting that single word into two very different mechanisms, because the exam tests the distinction constantly:

Vertical scalability — make the instance bigger.
Horizontal scalability (also called elasticity) — run more instances.

Scalability is related to High Availability but is not the same thing. The deck uses a call center to make the difference concrete: you can replace a junior operator with a senior operator who handles calls faster (vertical), you can hire a room full of operators (horizontal), or you can open a second call center in another city so that losing the first one does not take you offline (high availability).

Vertical scalability

Vertical scaling means increasing the size of the instance. If your application runs on a t2.micro, scaling it vertically means moving it to a t2.large.

It is the normal answer for non-distributed systems, a database being the classic example.
RDS and ElastiCache are the two AWS services the deck names as scaling vertically.
There is always a ceiling: at some point you run into the hardware limit of the largest instance available, and you cannot go further.

Horizontal scalability

Horizontal scaling means increasing the number of instances or systems serving the application. That implies a distributed system — several machines doing the same job behind something that spreads work between them.

It is the normal shape of web applications and other modern applications.
Cloud offerings such as Amazon EC2 make it easy, because adding a machine is an API call rather than a purchase order.

High availability

High availability usually travels together with horizontal scaling. It means running the application in at least two data centers, which on AWS means at least two Availability Zones. The goal is narrow and specific: survive the loss of a data center.

HA can be passive — RDS Multi-AZ is the deck’s example, where a standby sits idle and takes over only on failure.
HA can be active — the horizontal-scaling case, where every instance is serving traffic all the time.

The deck draws this as one building in New York and a second building in San Francisco: lose one, the service continues from the other.

Applied to EC2

Goal	How you do it with EC2
Vertical scaling (scale up / down)	Increase the instance size — from `t2.nano` (0.5 GB of RAM, 1 vCPU) up to `u-12tb1.metal` (12.3 TB of RAM, 448 vCPUs)
Horizontal scaling (scale out / in)	Increase the number of instances — Auto Scaling Group plus a Load Balancer
High availability	Run instances for the same application across multiple AZs — the Auto Scaling Group multi-AZ, the Load Balancer multi-AZ

2. What load balancing is

A load balancer is simply a server that sits in front of your fleet and forwards incoming traffic to several servers downstream, typically EC2 instances. Clients only ever talk to the load balancer; which backend actually answers is the load balancer’s decision.

The deck gives eight reasons to put one in front of your fleet. It spreads the load over several downstream instances while offering the application a single point of access — one DNS name. It swallows failures of those downstream instances without the client noticing, which it can only do because it runs regular health checks against them. It can terminate SSL so the public side of the site is served over HTTPS. It can enforce stickiness with cookies when a client needs to keep landing on the same backend. It provides high availability across zones. And it keeps public traffic separated from private traffic.

Why the Elastic Load Balancer specifically

You could run your own load balancer software on EC2. It would be cheaper in raw instance cost and far more work in every other way. An Elastic Load Balancer is the managed alternative:

AWS guarantees it will be working, and takes care of upgrades, maintenance and the load balancer’s own high availability.
In exchange you get only a handful of configuration knobs — less flexibility than a self-managed proxy.
It is integrated with the rest of AWS: EC2, EC2 Auto Scaling Groups, Amazon ECS, AWS Certificate Manager (ACM), CloudWatch, Route 53, AWS WAF and AWS Global Accelerator.

Health checks

Health checks are what make the rest of the mechanism work: without them the load balancer has no way of knowing whether a target can still answer. A health check is configured with a protocol, a port and a route — /health is the conventional path. If the target does not answer 200 (OK), it is considered unhealthy and stops receiving traffic.

Load balancer security groups

The two-security-group pattern is the one to memorize. The load balancer’s security group allows HTTP/HTTPS from anywhere, because it is the public entry point. The application’s security group allows HTTP only from the load balancer’s security group — not from a CIDR range, but by referencing the other security group. The instances then have no way of being reached directly from the internet.

3. The four load balancer types on AWS

AWS offers four managed load balancers, and they are distinguished mainly by the network layer they operate at and the protocols they carry.

Load balancer	Year	Layer / protocols
Classic Load Balancer (CLB) — v1, old generation	2009	HTTP, HTTPS, TCP, SSL (secure TCP)
Application Load Balancer (ALB) — v2	2016	HTTP, HTTPS, WebSocket
Network Load Balancer (NLB) — v2	2017	TCP, TLS (secure TCP), UDP
Gateway Load Balancer (GWLB)	2020	Layer 3 (network layer) — IP protocol

The deck’s general advice is to prefer the newer-generation load balancers, which have more features. Any of them can be set up as internal (private) or external (public).

Classic Load Balancer (v1)

The CLB supports TCP at layer 4 and HTTP/HTTPS at layer 7, its health checks are TCP- or HTTP-based, and it exposes a fixed hostname of the form XXX.region.elb.amazonaws.com. It is the old generation.

Application Load Balancer (v2)

The ALB is a layer 7 (HTTP) load balancer. It can balance across multiple HTTP applications on different machines through target groups, and across multiple applications on the same machine — which is exactly the container case. It supports HTTP/2 and WebSocket, and it can issue redirects, for example from HTTP to HTTPS.

Its distinguishing feature is rule-based routing to different target groups:

Routing on the path in the URL (example.com/users versus example.com/posts).
Routing on the hostname in the URL (one.example.com versus other.example.com).
Routing on query string or headers (example.com/users?id=123&order=false).

That makes the ALB a natural fit for microservices and container-based applications such as Docker and Amazon ECS, helped by a port mapping feature that lets it reach a dynamic port chosen by ECS. With Classic Load Balancers you would need one CLB per application instead.

ALB target groups can be:

EC2 instances (which can be managed by an Auto Scaling Group) — over HTTP.
ECS tasks (managed by ECS itself) — over HTTP.
Lambda functions — the HTTP request is translated into a JSON event.
IP addresses — which must be private IPs.

An ALB can route to multiple target groups, and health checks are configured at the target group level, not on the load balancer as a whole.

Two behaviors are worth memorizing. The ALB has a fixed hostname (XXX.region.elb.amazonaws.com). And because the ALB terminates the connection, your application servers do not see the client’s IP directly — they see the load balancer’s private IP. The real client IP is passed in the X-Forwarded-For header, with the port in X-Forwarded-Port and the protocol in X-Forwarded-Proto.

Network Load Balancer (v2)

The NLB works at layer 4. It forwards TCP and UDP traffic, handles millions of requests per second, and offers ultra-low latency. Its other signature property: an NLB has one static IP per Availability Zone, and supports assigning an Elastic IP — which is what you want when a client or partner needs to whitelist a fixed address. Reach for it when the requirement is extreme performance, or plain TCP/UDP traffic.

NLB target groups can be EC2 instances, IP addresses (private IPs only), or an Application Load Balancer. Its health checks support the TCP, HTTP and HTTPS protocols.

Gateway Load Balancer

The GWLB exists for a very specific job: deploying, scaling and managing a fleet of third-party network virtual appliances — firewalls, intrusion detection and prevention systems, deep packet inspection systems, payload manipulation and similar. It operates at layer 3, on IP packets.

It combines two functions in one device:

A transparent network gateway — a single entry and exit point for all traffic.
A load balancer — distributing that traffic across your virtual appliances.

Traffic is steered to it through the route table, sent to the appliances for inspection, and then continues to its real destination. It uses the GENEVE protocol on port 6081. Its target groups are EC2 instances or IP addresses (private IPs only).

4. Features shared across ELBs

Sticky sessions (session affinity)

Stickiness makes the load balancer send the same client to the same instance every time. It works for the Classic, Application and Network load balancers. For CLB and ALB the cookie used for stickiness has an expiration date that you control. The use case is keeping a user from losing session data held in that instance’s memory. The cost is balance: pinning clients to instances can leave the load unevenly spread across the backend fleet.

Cookie names are a detail the exam likes:

Application-based cookies
- Custom cookie — the target produces this one, so it can carry whatever attributes the application needs. Its name is set per target group, and three names are off limits because the ELB reserves them: AWSALB, AWSALBAPP and AWSALBTG.
- Application cookie — the load balancer produces this one, under the name AWSALBAPP.
Duration-based cookies — also produced by the load balancer, under the name AWSALB on an ALB and AWSELB on a CLB.

Cross-zone load balancing

With cross-zone load balancing, every load balancer node distributes requests evenly across all registered instances in all AZs. Without it, each node only spreads requests across the instances in its own AZ — so if AZ 1 holds two instances and AZ 2 holds eight, each AZ still receives half the traffic and the two instances in AZ 1 each get far more work than the eight in AZ 2.

Load balancer	Default	Inter-AZ data charges
Application Load Balancer	Enabled by default (can be disabled at the target group level)	No charge
Network Load Balancer and Gateway Load Balancer	Disabled by default	You pay for inter-AZ data if enabled
Classic Load Balancer	Disabled by default	No charge if enabled

SSL/TLS on the load balancer

An SSL certificate lets traffic between clients and the load balancer be encrypted in transit (in-flight encryption). SSL stands for Secure Sockets Layer and TLS for Transport Layer Security, the newer version; in practice TLS certificates are what everyone uses, while everyone keeps saying “SSL”. Public certificates are issued by Certificate Authorities such as Comodo, Symantec, GoDaddy, GlobalSign, Digicert or Let’s Encrypt, they have an expiration date you set, and they must be renewed.

On the load balancer side, the certificate is an X.509 server certificate. You can manage certificates with ACM (AWS Certificate Manager) or upload your own. On an HTTPS listener you must specify a default certificate, and you may add an optional list of further certificates to serve multiple domains. Clients then use SNI to say which hostname they want. You can also set a security policy to keep supporting older SSL/TLS versions for legacy clients. The usual shape is HTTPS from the users over the internet, plain HTTP from the load balancer to the instances over the private VPC.

SNI (Server Name Indication)

SNI solves the problem of loading multiple SSL certificates onto one server in order to serve multiple websites. It is a newer protocol that requires the client to indicate the hostname it wants during the initial SSL handshake; the server then picks the matching certificate, or returns the default one.

Its support matrix is the exam point: SNI works for ALB, NLB and CloudFront, and does not work for the CLB.

Load balancer	Certificates
Classic Load Balancer (v1)	Only one SSL certificate — you need multiple CLBs to serve multiple hostnames with multiple certificates
Application Load Balancer (v2)	Multiple listeners with multiple certificates, using SNI
Network Load Balancer (v2)	Multiple listeners with multiple certificates, using SNI

Connection draining

The same feature has two names depending on the load balancer: it is called Connection Draining on the CLB and Deregistration Delay on the ALB and NLB. It is the grace period given to in-flight requests while an instance is deregistering or has been marked unhealthy: the load balancer stops sending it new requests but lets the existing ones finish.

The value ranges from 1 to 3600 seconds, with a default of 300 seconds, and can be disabled by setting it to 0. If your requests are short, set it low so instances leave the fleet quickly.

5. Auto Scaling Groups (ASG)

Real-world load changes through the day, and in the cloud you can create and destroy servers in minutes. An Auto Scaling Group automates that. Its jobs are to:

Scale out (add EC2 instances) when load increases.
Scale in (remove EC2 instances) when load decreases.
Keep the fleet between a minimum and a maximum number of instances.
Register new instances with a load balancer automatically.
Re-create an instance when a previous one is terminated, for example because it became unhealthy.

ASGs themselves are free — you only pay for the underlying EC2 instances.

An ASG is defined by three capacity numbers: minimum capacity, desired capacity and maximum capacity. It scales out from desired towards maximum as needed and never goes below minimum. Paired with an Elastic Load Balancer, the ELB’s health checks also feed the ASG’s view of which instances are alive, so an instance failing its health check gets replaced.

ASG attributes

An ASG is built on a Launch Template — the older Launch Configurations are deprecated. The template carries:

The AMI and the instance type.
EC2 User Data.
EBS volumes.
Security groups.
The SSH key pair.
IAM roles for the instances.

Alongside the template, the ASG itself holds the network and subnet information, the load balancer information, the min size / max size / initial capacity, and the scaling policies.

CloudWatch alarms and scaling

An ASG can scale on CloudWatch alarms. An alarm watches a metric — average CPU, or a custom metric — and metrics such as average CPU are computed across all the instances in the group, not per instance. From an alarm you create scale-out policies that add instances and scale-in policies that remove them.

Scaling policies

Dynamic scaling
- Target tracking scaling — the simplest to set up. You state the target and AWS keeps you near it: “I want the average ASG CPU to stay around 40%.”
- Simple / step scaling — tied directly to CloudWatch alarms. When CPU goes above 70%, add 2 units; when it drops below 30%, remove 1.
Scheduled scaling — anticipate a known pattern rather than react to it. For example, raise the minimum capacity to 10 at 5 pm on Fridays.
Predictive scaling — AWS keeps projecting what the load will be and puts the scaling in place before it arrives.

Good metrics to scale on

CPUUtilization — average CPU utilization across your instances.
RequestCountPerTarget — keeps the number of requests handled per EC2 instance stable; the deck illustrates it with an ALB and a target value of 3.
Average Network In / Out — the right choice if the application is network bound.
Any custom metric you push to CloudWatch.

Scaling cooldowns

After a scaling activity, the ASG enters a cooldown period, 300 seconds by default. During the cooldown the group will not launch or terminate further instances, which gives the metrics time to stabilise and stops the group from over-reacting to its own changes.

The deck’s practical advice: use a ready-to-use AMI so new instances need little configuration time, start serving requests faster, and let you shorten the cooldown.

Quick recap

Topic	What to remember for the exam
Vertical scaling	Increase instance size; used for non-distributed systems such as RDS and ElastiCache; bounded by a hardware limit
Horizontal scaling	Increase the number of instances; the same thing as elasticity; requires a distributed system
High availability	Run in at least 2 AZs; passive (RDS Multi-AZ) or active (horizontal scaling)
CLB (2009)	HTTP, HTTPS, TCP, SSL; only one SSL certificate; no SNI
ALB (2016)	Layer 7; routing by path / hostname / query string / header; targets are EC2, ECS tasks, Lambda, private IPs; client IP arrives in `X-Forwarded-For`
NLB (2017)	Layer 4, TCP/UDP; millions of requests per second, ultra-low latency; one static IP per AZ plus Elastic IP; can target an ALB
GWLB (2020)	Layer 3, IP packets; fronts third-party virtual appliances (firewall, IDS/IPS); GENEVE on port 6081
Sticky sessions	Same client to the same instance; cookies `AWSALB` (ALB), `AWSELB` (CLB), `AWSALBAPP` (application cookie); never name a custom cookie `AWSALB`, `AWSALBAPP` or `AWSALBTG`
Cross-zone LB	ALB enabled by default, free · NLB/GWLB disabled by default, charged · CLB disabled by default, free
SNI	Several certificates on one listener; ALB, NLB and CloudFront only — not the CLB
Connection draining	Called Connection Draining on the CLB, Deregistration Delay on ALB/NLB; 1–3600 seconds, default 300, 0 disables it
ASG	Min / desired / max; Launch Template (Launch Configurations deprecated); free, you pay only for EC2; replaces unhealthy instances
Scaling policies	Target tracking · simple/step · scheduled · predictive
Metrics to scale on	`CPUUtilization`, `RequestCountPerTarget`, average network in/out, custom metrics
Cooldown	300 seconds by default; a ready-to-use AMI lets you shorten it