Engineering Challenge

Build an API endpoint that returns a list of articles. Implement Redis caching. If the user creates a new article, automatically invalidate (delete) the cached list so the next request fetches fresh data from the database. Measure the response time difference between a cache miss and a cache hit using console.time().

Revision Sheet

• Redis: In-memory key-value store. Blazing fast.
• TTL: Time-To-Live. How long data stays in cache before auto-deleting.
• Eviction: What Redis does when RAM is full (e.g., deletes oldest keys).
• Pub/Sub: Fire-and-forget real-time messaging between services.
• Rate Limiting: Easily implemented with Redis INCR and EXPIRE.
• NoSQL vs SQL: NoSQL scales horizontally easier, handles unstructured data well, but sacrifices ACID guarantees and strict relational integrity.

Connections

This cache layer directly connects to our previous chapter on Database Scaling. By introducing Redis, we delay the need to implement complex database sharding or read replicas. In the next chapter, we will look at Message Brokers (Kafka/RabbitMQ) and see why they are preferred over Redis Pub/Sub for guaranteed message delivery.

Learning Objectives

• Understand the architectural differences between relational (PostgreSQL) and NoSQL databases.
• Grasp the mechanics of Key-Value stores and Document databases.
• Master Redis for caching, session management, and rate limiting.
• Implement Pub/Sub and message queues using Redis.
• Understand distributed locks and TTL (Time-To-Live).

Prerequisites

• Basic understanding of databases and SQL.
• Node.js and TypeScript fundamentals.
• Familiarity with HTTP and REST APIs.

Why Does This Exist?

Relational databases like PostgreSQL are excellent for structured data with complex relationships, enforcing ACID properties. However, as applications scale to millions of users, we encounter scenarios where we need ultra-fast read/write operations (microsecond latency), flexible schemas (unstructured data), or transient data storage (sessions, caches). NoSQL and Redis exist to solve these specific scaling and latency problems by relaxing certain constraints (like strict consistency or relationships) in favor of performance and flexibility.

The Problem Before the Solution

Imagine you have a popular e-commerce site built on PostgreSQL. Every time a user loads the homepage, your application queries the database for the top 10 best-selling products. During a flash sale, 100,000 users hit the homepage simultaneously. That's 100,000 complex SQL queries executing against your database in seconds.

Why the Old Approach Breaks

PostgreSQL reads from disk (mostly, although it has some memory caching) and computes the query execution plan each time. The massive spike in CPU and I/O operations will overload the database, causing connections to drop, queries to time out, and eventually bringing down the entire application. Scaling PostgreSQL vertically (buying a bigger server) is expensive and has a hard limit.

History

In the late 2000s, web giants like Google, Amazon, and Facebook realized traditional RDBMS couldn't handle their web-scale traffic. Amazon published the Dynamo paper (leading to key-value stores), and Google published Bigtable. "NoSQL" (Not Only SQL) became a movement. In 2009, Salvatore Sanfilippo created Redis (Remote Dictionary Server) to improve the performance of a real-time web analytics product, building an in-memory key-value store that operates blisteringly fast.

Mental Model (Analogy -> Reality)

Analogy: Think of a relational database (PostgreSQL) as a massive, highly organized filing cabinet in a library. Every document must follow a strict template (schema) and be placed in a specific folder (table). It's great for audits and complex reports.

Think of a document database (MongoDB) as a set of flexible folders where you can drop in any JSON-like document without pre-defining the fields.

Think of Redis as the librarian's notepad on their desk (RAM). It holds small pieces of frequently accessed information (like "Where is the bathroom?") so they don't have to walk to the filing cabinet every time someone asks. It's incredibly fast but temporary (if the library loses power, the notepad might get thrown away).

Reality: PostgreSQL writes to persistent disk. Redis writes to RAM (memory), which is orders of magnitude faster but volatile by default.

Internal Working

When you query Redis, it looks up the key in a hash table stored directly in RAM. Since there is no disk seek time, the operation typically completes in under a millisecond. Redis is single-threaded, meaning it executes one command at a time. This avoids complex locking mechanisms and context switching overhead, making it incredibly fast for simple atomic operations.

For TTL (Time-To-Live), Redis passively expires keys (when you try to access an expired key, it deletes it) and actively expires them (sampling keys periodically and deleting expired ones in the background).

Visual Explanation

[User Request] --> [Node.js Server]
                          |
                          v
                    Is Data in Redis?
                     /            \
                 [YES]            [NO]
               (Cache Hit)      (Cache Miss)
                   |                |
             Return Data       Query PostgreSQL
                                    |
                               Save to Redis (with TTL)
                                    |
                               Return Data
    

Syntax

// Redis commands via redis-cli
SET mykey "Hello"
GET mykey
SETEX session:123 3600 "user_data"  // Set with TTL (3600s)
INCR page_views
    

Tiny Example

import { createClient } from 'redis';

async function main() {
  const client = createClient();
  await client.connect();

  await client.set('greeting', 'Hello World!');
  const value = await client.get('greeting');
  console.log(value); // Hello World!
  
  await client.disconnect();
}
    

Walkthrough

1. We import createClient from the redis package.
2. We create a client and connect to the local Redis instance (defaults to localhost:6379).
3. We use set to write a key-value pair to memory.
4. We use get to retrieve it instantly.
5. We gracefully disconnect to close the socket.

Break It

What happens if we try to store a massive JSON object directly in Redis without stringifying it, or try to use a relational query?

await client.set('user:1', { name: "John", age: 30 }); // Error!
    

Redis only understands strings, hashes, lists, sets, etc., natively. You must JSON.stringify() objects before storing them as simple strings.

Debug It

If Redis says "OOM command not allowed", it means Out Of Memory. Redis lives in RAM. If you don't set TTLs (Time-To-Live) on your cached items, your server's RAM will fill up entirely, causing Redis to crash or reject new writes.

Mini Project (20-30 min)

Rate Limiter: Build a simple middleware that limits users to 10 requests per minute using Redis.

async function rateLimiter(req, res, next) {
  const ip = req.ip;
  const key = `rate_limit:${ip}`;
  
  const requests = await redis.incr(key);
  
  if (requests === 1) {
    await redis.expire(key, 60); // Set TTL to 60 seconds on first request
  }
  
  if (requests > 10) {
    return res.status(429).send("Too Many Requests");
  }
  
  next();
}
    

Bigger Project (1-2 hours)

Implement a Redis caching layer for an API endpoint, and build a simple rate-limiter using Redis SETEX and INCR.

// Implementation for NoSQL & Redis
console.log("Bigger project solution");

Interview Questions