Engineering Challenge

Design a rate limiter from scratch using Redis. Ensure it supports sliding window algorithms to handle bursts accurately without punishing consistent, valid traffic. Write unit tests mocking Redis.

Revision Sheet

• Layers: Router -> Controller -> Service -> Repository.
• Queues: Offload slow tasks (Emails, Webhooks, Image Resizing).
• Middleware: Cross-cutting concerns (Auth, Logs, Errors).
• Pagination: Prefer cursor over offset for large tables.

Connections

Understanding these backend principles leads directly into Distributed Systems and Cloud Architecture, where these same concepts scale across multiple servers and containers.

Learning Objectives

By the end of this chapter, you will be able to engineer a robust, scalable, and production-ready backend system. You will master the separation of concerns (Controllers, Services, Repositories), handle cross-cutting concerns (Middleware, Error Handling, Logging, Validation), secure your application (Authentication, Authorization, Rate Limiting), and scale operations (Background Jobs, Queues, Caching, File Uploads, Webhooks, Scheduled Jobs).

Prerequisites

Solid understanding of HTTP, basic Node.js and TypeScript, introductory relational database knowledge, and comfort with asynchronous programming.

Why Does This Exist?

A backend is not just an API that reads from a database. It is the engine of a product. It exists to securely process business logic, persist data reliably, handle immense traffic seamlessly, and communicate with external systems. Without a structured backend architecture, codebases devolve into unmaintainable "spaghetti" where business rules, database queries, and HTTP parsing are jumbled together.

The Problem Before the Solution

In a naive application, a developer creates a single massive function for a route (e.g., POST /users). This function parses the request, validates input, hashes the password, writes to the database, sends a welcome email, and returns a response. It handles everything.

Why the Old Approach Breaks

As the application grows, testing becomes impossible without mocking everything. If you want to create a user from an internal admin script instead of an HTTP request, you can't reuse the logic. When errors occur, it's hard to trace where it failed. Scaling specific tasks (like sending emails) blocks the main HTTP thread, leading to timeout errors and a degraded user experience.

History

Backend architectures evolved from monolithic CGI scripts to Model-View-Controller (MVC) paradigms popularized by frameworks like Ruby on Rails and Django. As systems grew, service-oriented architectures and microservices emerged. The industry standardized on patterns like Domain-Driven Design (DDD) and layered architectures (Controllers -> Services -> Repositories) to maintain order in massive codebases.

Mental Model (Analogy -> Reality)

Think of your backend as a high-end restaurant.

• Router/Middleware: The Maître D' checks reservations (Authentication), ensures you are dressed appropriately (Validation/Rate Limiting), and seats you (Routing).
• Controller: The Waiter takes your order (HTTP Request), translates it into a standard ticket, hands it to the kitchen, and delivers the food back (HTTP Response).
• Service: The Head Chef executes the recipe (Business Logic). They don't care how the order arrived; they just cook.
• Repository: The Sous Chef fetches raw ingredients from the pantry (Database Queries).
• Background Jobs/Queues: The dishwashers and prep cooks working asynchronously so the Head Chef is never blocked.

In reality, this translates to distinct classes or modules where each layer only communicates with the one directly below it.

Internal Working (Memory, stack, process, network, etc.)

When an HTTP request arrives, Node's event loop offloads network I/O to the OS. The request passes through middleware functions that operate on the request object. Controllers parse data and push it down the call stack to Services. For Database access (Repository) or API calls, asynchronous operations are placed in the libuv thread pool. Background jobs are pushed to external systems (like Redis queues) to free the V8 engine to handle the next request immediately.

Visual Explanation (ASCII diagrams)

[Client] 
   | (HTTP POST /users)
   v
[Router] --> [Middleware (Auth, Rate Limit, Logging)]
   |
   v
[Controller] (Parses HTTP, Validates DTO)
   |
   v
[Service] (Business Logic: Check if user exists, hash password)
   | \
   |  \----> [Queue (Redis)] -> [Background Worker] (Sends Email)
   v
[Repository] (SQL Queries)
   |
   v
[Database] (PostgreSQL)

Syntax

We use TypeScript classes and Dependency Injection (DI) to construct these layers securely.

Tiny Example

// Controller
class UserController {
  constructor(private userService: UserService) {}
  
  async create(req: Request, res: Response) {
    const user = await this.userService.createUser(req.body);
    res.status(201).json(user);
  }
}

Walkthrough

In this architecture:

1. Routing: Directs URL paths to specific Controller methods.
2. Validation/Serialization: Middleware validates payload against schemas (e.g., Zod) before it reaches the Controller.
3. Controllers: Extract data and call the Service.
4. Services: Orchestrate business rules and call Repositories.
5. Repositories: Isolate ORM/SQL logic.
6. Error Handling: A centralized catch-all middleware formats exceptions into standard JSON responses.

Break It

If you inject the Express req object directly into your Service layer, you have broken the separation of concerns. The Service is now tightly coupled to HTTP. If you try to run the Service via a cron job, it fails because there is no HTTP request object.

Debug It

To fix coupling, extract the required fields from req.body in the Controller and pass a simple TypeScript object (DTO) to the Service.

Mini Project (20-30 min)

Build a User Onboarding API. Implement a Router, a rate-limiter middleware, Zod validation, a Controller, a Service that hashes passwords, and a Repository that saves to Postgres. Enqueue a welcome email using BullMQ.

Bigger Project (1-2 hours)

Build a RESTful API for a Todo app with Zod validation, request logging, and a global error handler.

// Implementation for Backend Engineering
console.log("Bigger project solution");

Interview Questions