RDS, Aurora & ElastiCache

1. What Amazon RDS is

RDS stands for Relational Database Service. It is a managed service for databases that use SQL as their query language, letting you create databases in the cloud that AWS operates for you.

The engines it supports are:

• PostgreSQL
• MySQL
• MariaDB
• Oracle
• Microsoft SQL Server
• IBM DB2
• Aurora — the AWS proprietary database

Why RDS rather than a database on EC2

You could install PostgreSQL on an EC2 instance yourself. What you give up by doing so is everything RDS provides as a managed service:

• Automated provisioning and OS patching.
• Continuous backups and restore to a specific timestamp — Point in Time Restore.
• Monitoring dashboards.
• Read replicas for better read performance.
• Multi-AZ setup for disaster recovery.
• Maintenance windows for upgrades.
• Scaling capability, both vertical and horizontal.
• Storage backed by EBS.

The one thing you lose: you cannot SSH into an RDS instance.

Exam tip

“We need shell access to the database host / we need to install a native OS package” rules out plain RDS. The only exception in this chapter is RDS Custom, covered in section 4.

RDS Storage Auto Scaling

Storage Auto Scaling grows the storage of an RDS DB instance dynamically, so you do not have to notice the disk filling up and resize it by hand. When RDS detects the instance is running out of free database storage, it scales the volume automatically.

You set a Maximum Storage Threshold — the upper limit past which it will not grow. RDS then modifies storage automatically when:

• Free storage is less than 10% of allocated storage.
• The low-storage condition has lasted at least 5 minutes.
• 6 hours have passed since the last storage modification.

It is useful for applications with unpredictable workloads and is supported by all RDS database engines.

2. RDS read replicas — scaling reads

• You can have up to 15 read replicas.
• They can be within the same AZ, cross-AZ or cross-Region.
• Replication is asynchronous, so reads from a replica are eventually consistent.
• A replica can be promoted to become its own standalone database.
• Applications must update their connection string to actually use the replicas — nothing routes reads for you.

Use case

The canonical scenario: you have a production database handling its normal load, and someone wants to run a reporting or analytics application against it. Instead of letting that workload compete with production, you create a read replica and point the reporting application at it. Production is unaffected.

Read replicas are for SELECT statements only — no INSERT, UPDATE or DELETE against a replica.

Network cost

In AWS, data crossing from one AZ to another normally costs money. RDS read replicas within the same Region are exempt: replication traffic between us-east-1a and us-east-1b is free. Cross-Region replication, for example us-east-1 to eu-west-1, is billed.

Exam tip

Two traps in one section. First, read replicas are for scaling reads, not for disaster recovery — that is Multi-AZ. Second, same-Region replication traffic is free while cross-Region is not, which is often the deciding factor in a cost question.

3. RDS Multi-AZ — for disaster recovery

Multi-AZ keeps a standby instance in another Availability Zone, kept up to date with synchronous replication.

• There is one DNS name, and failover to the standby is automatic — the application does not change anything.
• It increases availability, and covers the loss of an AZ, loss of the network, or an instance or storage failure.
• No manual intervention is needed in the applications.
• It is not used for scaling: the standby does not serve reads.
• Read replicas can themselves be set up as Multi-AZ for disaster recovery.

Going from Single-AZ to Multi-AZ

This is a zero-downtime operation — you do not stop the database. You click “modify” on the database, and internally AWS:

1. Takes a snapshot.
2. Restores a new DB from that snapshot in another AZ.
3. Establishes synchronisation between the two databases.

Exam tip

Remember the pairing: read replicas = asynchronous, scale reads; Multi-AZ = synchronous, disaster recovery, standby serves nothing.

4. RDS Custom

RDS Custom is a managed Oracle and Microsoft SQL Server database that additionally gives you OS and database customization.

The difference from plain RDS is access. RDS automates the setup, operation and scaling of the database and keeps the whole stack under AWS management. RDS Custom hands you the underlying database and OS so you can:

• Configure settings.
• Install patches.
• Enable native features.
• Access the underlying EC2 instance with SSH or SSM Session Manager.

To perform your customizations you deactivate Automation Mode first, and it is wise to take a DB snapshot before you do.

In one line: RDS means AWS manages the entire database and the OS; RDS Custom gives you full admin access to both.

5. Amazon Aurora

Aurora is proprietary AWS technology — not open source. It speaks the PostgreSQL and MySQL protocols, meaning your existing drivers work as if they were talking to a Postgres or MySQL database.

• Aurora is “AWS cloud optimized” and claims 5x the performance of MySQL on RDS and over 3x the performance of Postgres on RDS.
• Storage grows automatically in 10 GB increments, up to 128 TB.
• It supports up to 15 replicas, and replication is faster than MySQL’s — sub-10 ms replica lag.
• Failover is instantaneous; high availability is native to the design.
• It costs about 20% more than RDS, but is more efficient.

High availability and read scaling

Aurora’s design separates compute from a shared, distributed storage layer:

• 6 copies of your data across 3 AZs.
• 4 copies out of 6 are needed for writes.
• 3 copies out of 6 are needed for reads.
• Storage is self-healing with peer-to-peer replication, and is striped across hundreds of volumes.
• One Aurora instance takes the writes (the master), with automated failover in less than 30 seconds.
• The master plus up to 15 Aurora read replicas serve reads.
• Cross-Region replication is supported.

Endpoints

Clients do not track which instance is the master. Aurora exposes:

• A Writer Endpoint, always pointing at the current master — so a failover is transparent.
• A Reader Endpoint, which performs connection load balancing across the read replicas.

Replica Auto Scaling adds read replicas when the existing ones come under load (for example rising CPU from many requests), and the reader endpoint is extended to cover the new ones automatically.

Custom endpoints

You can define a subset of Aurora instances as a Custom Endpoint — for example, sending analytical queries only to two large db.r5.2xlarge replicas while ordinary traffic goes to the smaller ones. Once you define custom endpoints, the reader endpoint is generally not used any more.

Aurora features

Beyond the storage design, the deck credits Aurora with failover that happens on its own, backup and recovery, isolation and security, compliance with industry standards, scaling at the push of a button, patching that is automated and costs no downtime, advanced monitoring and routine maintenance. The entry worth singling out is Backtrack, which rewinds the data to an arbitrary point in time without going through a backup.

Aurora Serverless

Aurora Serverless does automated database instantiation and auto-scaling based on actual usage, fronted by a proxy fleet that Aurora manages. It suits infrequent, intermittent or unpredictable workloads, requires no capacity planning, and is billed per second, which can make it more cost-effective than a permanently provisioned cluster.

Global Aurora

Two options exist for spanning Regions:

• Aurora Cross-Region Read Replicas — useful for disaster recovery and simple to put in place.
• Aurora Global Database — the recommended option. 1 primary Region (read/write) and up to 10 secondary read-only Regions with a replication lag under 1 second, and up to 16 read replicas per secondary Region. It reduces latency for distant users, and promoting another Region for disaster recovery has an RTO of under 1 minute.

Aurora Machine Learning

Aurora ML lets you add ML-based predictions to your applications through SQL, with an optimized and secure integration between Aurora and AWS ML services. The supported services are Amazon SageMaker (usable with any ML model) and Amazon Comprehend (for sentiment analysis). You do not need ML experience. Use cases: fraud detection, ads targeting, sentiment analysis and product recommendations — the application issues a SQL query, Aurora sends the data to the ML service and returns the predictions inside the query results.

Babelfish for Aurora PostgreSQL

Babelfish lets Aurora PostgreSQL understand commands aimed at Microsoft SQL Server, such as T-SQL. SQL Server based applications can therefore run against Aurora PostgreSQL with no or very little code change, still using the same SQL Server client driver. The same applications keep working after you migrate the database with AWS SCT and DMS.

Exam tip

Keep the Aurora numbers straight: 6 copies across 3 AZs, 4/6 for writes, 3/6 for reads, 10 GB increments to 128 TB, 15 read replicas, failover under 30 seconds, Global Database: 10 secondary Regions, sub-second lag, RTO under 1 minute.

6. Backups, restore and cloning

RDS backups

• Automated backups: a daily full backup during the backup window, plus transaction logs backed up every 5 minutes. Together these give you the ability to restore to any point in time, from the oldest backup up to 5 minutes ago. Retention is 1 to 35 days; set it to 0 to disable automated backups.
• Manual DB snapshots: triggered by you, retained for as long as you want.

One cost trick worth remembering: a stopped RDS database still bills you for storage. If you plan to stop it for a long time, snapshot it and restore later instead.

Aurora backups

• Automated backups: 1 to 35 days, and cannot be disabled. Point-in-time recovery within that window.
• Manual DB snapshots: triggered by you, retained for as long as you want.

Restore options

Restoring an RDS or Aurora backup or snapshot always creates a new database. Two migration-shaped restores are called out:

• Restoring a MySQL RDS database from S3 — back up your on-premises database, store the backup in Amazon S3, and restore the file onto a new RDS instance running MySQL.
• Restoring a MySQL Aurora cluster from S3 — back up the on-premises database using Percona XtraBackup, store the file in S3, and restore it onto a new Aurora cluster running MySQL.

Aurora Database Cloning

Cloning creates a new Aurora DB cluster from an existing one, and it is faster than snapshot and restore. It uses a copy-on-write protocol: initially the new cluster points at the same data volume as the original, so nothing is copied; only when the new cluster’s data is updated is additional storage allocated and the changed data separated out.

It is very fast and cost-effective, and the obvious use is creating a staging database from a production database without impacting production.

Real-world example

Before a risky schema migration, clone the production Aurora cluster into a staging cluster. The clone is ready in moments because no data is copied, the migration runs against real-sized data, and production never feels it.

7. RDS and Aurora security

• At-rest encryption: AWS KMS encrypts the master and its replicas, but the decision has to be made when the database is launched. A read replica cannot be encrypted if its master is not, and turning encryption on for a database that was created without it means taking a snapshot and restoring it as encrypted.
• In-flight encryption: every database is TLS-ready out of the box; on the client side you use the AWS TLS root certificates.
• IAM authentication: IAM roles can stand in for the username-and-password login.
• Security groups: these are what gate network access to an RDS or Aurora database.
• SSH is not available anywhere except RDS Custom.
• Audit logs: switch them on and ship them to CloudWatch Logs when you need to keep them longer.

8. Amazon RDS Proxy

RDS Proxy is a fully managed database proxy for RDS. It lets applications pool and share the database connections they establish, which improves database efficiency by reducing pressure on database resources such as CPU and RAM and by minimizing the number of open connections and the timeouts they cause.

• It is serverless, autoscaling and highly available (multi-AZ).
• It reduces RDS and Aurora failover time by up to 66%.
• It supports RDS (MySQL, PostgreSQL, MariaDB, MS SQL Server) and Aurora (MySQL, PostgreSQL).
• No code changes are required for most applications.
• It can enforce IAM authentication for the database and store credentials securely in AWS Secrets Manager.
• RDS Proxy is never publicly accessible — it must be reached from within a VPC.

Exam tip

The deck draws Lambda functions sitting in a private subnet in front of RDS Proxy — the setting where pooling matters most, because the point of the proxy is to keep the number of open connections (and the timeouts that follow from them) down.

9. Amazon ElastiCache

What RDS is for managed relational databases, ElastiCache is for managed Redis or Memcached.

• Caches are in-memory databases with very high performance and low latency.
• They reduce the load on databases for read-intensive workloads.
• They help make your application stateless, by moving session data out of the web servers.
• AWS handles OS maintenance and patching, optimizations, setup, configuration, monitoring, failure recovery and backups.
• The catch: using ElastiCache involves heavy application code changes — the application has to know about the cache.

Architecture 1 — database cache

The application queries ElastiCache first. On a cache hit it gets the answer immediately. On a cache miss it reads from Amazon RDS and writes the result into the cache. This relieves load on RDS. The cache must have an invalidation strategy so that only current data is served.

Architecture 2 — user session store

The user logs into one instance of the application, which writes the session data into ElastiCache. When the user’s next request lands on a different instance, that instance retrieves the session from ElastiCache and the user is still logged in. This is what makes the web tier stateless.

Redis versus Memcached

Redis	Memcached
Multi-AZ with auto-failover	Multi-node partitioning of data (sharding)
Read replicas to scale reads and provide high availability	No high availability (no replication)
Data durability using AOF persistence	Non-persistent
Backup and restore features	Backup and restore (Serverless)
Supports Sets and Sorted Sets	Multi-threaded architecture

Cache security

• ElastiCache supports IAM authentication for Redis. IAM policies on ElastiCache are otherwise only used for AWS API-level security.
• Redis AUTH — you can set a password or token when creating a Redis cluster, an extra layer of security on top of security groups. Redis also supports SSL in-flight encryption.
• Memcached supports SASL-based authentication (advanced).

Caching patterns

• Lazy Loading — everything that gets read ends up in the cache, at the price of entries that may no longer match the database: cached data can go stale.
• Write Through — a write to the database also adds or updates the value in the cache, which is why this pattern leaves no stale data.
• Session Store — temporary session data lives in the cache and is expired using its TTL features.

The deck closes the topic with the old joke that there are only two hard things in computer science: cache invalidation and naming things.

Real-world example

Gaming leaderboards are computationally expensive to recompute. Redis Sorted Sets guarantee both uniqueness and element ordering, so each new score is ranked in real time and inserted in the right position — a real-time leaderboard served straight from ElastiCache for Redis.

Quick recap

Topic	What to remember for the exam
RDS engines	PostgreSQL, MySQL, MariaDB, Oracle, MS SQL Server, IBM DB2, Aurora
RDS managed features	Provisioning, OS patching, continuous backup with Point in Time Restore, monitoring, read replicas, Multi-AZ, maintenance windows, EBS storage — but no SSH
Storage Auto Scaling	Needs a Maximum Storage Threshold; triggers at free < 10%, lasting 5 minutes, 6 hours since the last change
Read replicas	Up to 15, async so eventually consistent, same AZ / cross-AZ / cross-Region, can be promoted; same-Region replication traffic is free
Multi-AZ	Synchronous, one DNS name, automatic failover, disaster recovery not scaling; Single-AZ to Multi-AZ is zero downtime
RDS Custom	Oracle and MS SQL Server only; SSH / SSM access to the underlying EC2; deactivate Automation Mode first
Aurora core numbers	6 copies over 3 AZs, 4/6 writes, 3/6 reads; 10 GB increments to 128 TB; 15 replicas, sub-10 ms lag; failover < 30 s; costs ~20% more
Aurora endpoints	Writer endpoint to the master, Reader endpoint load-balances replicas, Custom endpoints for a subset of instances
Aurora Serverless	Auto instantiation and scaling, pay per second, for intermittent or unpredictable workloads
Aurora Global Database	1 primary Region, up to 10 secondary read-only Regions, lag < 1 s, 16 replicas per secondary, promotion RTO < 1 minute
Babelfish / Aurora ML	Babelfish lets Aurora PostgreSQL accept T-SQL; Aurora ML calls SageMaker and Comprehend from SQL
Backups	RDS automated: daily full plus transaction logs every 5 min, retention 1–35 days, 0 disables · Aurora automated: 1–35 days, cannot be disabled
Aurora cloning	Copy-on-write, faster than snapshot and restore, ideal for staging from production
Encryption	KMS at launch time; an unencrypted master means unencrypted replicas; encrypt later via snapshot and restore as encrypted; TLS in flight by default
RDS Proxy	Connection pooling, serverless and multi-AZ, failover time down by up to 66%, enforces IAM auth, never publicly accessible
ElastiCache engines	Redis: Multi-AZ auto-failover, read replicas, AOF persistence, backup/restore, Sorted Sets · Memcached: sharding, no HA, non-persistent, multi-threaded
Cache security	Redis AUTH password/token plus SSL; Memcached SASL; IAM auth for Redis
Caching patterns	Lazy Loading (can go stale) · Write Through (no stale data) · Session Store (TTL)