Learning Objectives

• Understand the Node.js runtime, V8 engine, and libuv.
• Master the Event Loop, asynchronous I/O, and non-blocking execution.
• Differentiate between CommonJS and ES Modules.
• Work with the file system, streams, and buffers efficiently.
• Manage processes, environment variables, child processes, worker threads, and clustering.
• Understand package management via npm, package.json, lockfiles, semantic versioning, and dependency management.

Prerequisites

• Solid understanding of JavaScript (ES6+).
• Familiarity with asynchronous JavaScript (Promises, async/await).
• Basic terminal and command-line usage.

Why Does This Exist?

JavaScript was originally confined to the browser, manipulating the DOM and handling user events. Node.js was created to take JavaScript out of the browser and let it run on the server, interacting with the operating system, file system, and network, making full-stack JavaScript possible.

The Problem Before the Solution

Before Node.js, most web servers (like Apache with PHP, Ruby on Rails, Java Tomcat) handled concurrent connections by spawning a new OS thread for each request. This is the Thread-per-Request model.

Why the Old Approach Breaks

Threads are expensive. They consume memory (often a few megabytes each) and require the OS to perform expensive context switches. If you have 10,000 concurrent requests waiting for a database to respond (the C10k problem), you need 10,000 threads. Most of these threads are just sitting there blocked, waiting for network I/O, wasting system resources.

History

In 2009, Ryan Dahl created Node.js. He combined Google's fast V8 JavaScript engine (from Chrome) with a C library called libuv for asynchronous I/O. His realization: JavaScript was single-threaded and heavily relied on callbacks, making it the perfect language for an event-driven, non-blocking I/O model.

Mental Model (Analogy -> Reality)

Analogy: A restaurant kitchen.

Old model (Thread-per-Request): Each customer gets their own dedicated waiter. The waiter takes the order, walks to the kitchen, and stands there waiting for the food to cook before bringing it back. 100 customers = 100 waiters.

Node.js model (Event Loop): One incredibly fast waiter (the main thread). The waiter takes an order, passes it to the kitchen (libuv / OS), and immediately goes to the next customer to take their order. When the kitchen finishes cooking, they ring a bell (callback). The waiter then delivers the food. 100 customers = 1 waiter + an asynchronous kitchen.

Reality: Node runs your JavaScript on a single thread. When you perform an I/O operation (like reading a file or querying a DB), Node offloads it to the OS or a thread pool via libuv. The main thread continues executing. When the I/O is done, a callback/promise is placed in a queue, and the Event Loop eventually picks it up to resume execution.

Internal Working (Memory, stack, process, network, etc.)

Node.js is not just JavaScript. It is a C++ wrapper around:

• V8: Compiles and executes JavaScript synchronously. Handles memory heap and call stack.
• libuv: The C library that provides the Event Loop, thread pool (for operations the OS can't do asynchronously, like some file I/O or DNS lookups), and async network I/O.
• Bindings/APIs: Bridges the C++ world to JavaScript.

When V8 encounters an async I/O function (e.g., fs.readFile), it hands the work to Node's C++ bindings, which pass it to libuv, and V8 pops the function off the call stack.

Visual Explanation (ASCII diagrams)

[ JS Call Stack (V8) ] ---> executes synchronous code
         |
    (async call)
         v
[ Node APIs (C++) ] ---> delegates work
         |
         v
[ libuv / OS kernel / Thread Pool ] ---> does the heavy lifting
         |
    (completion)
         v
[ Task Queues ] (Microtask Queue, Macrotask Queue)
         |
         v
[ Event Loop ] ---> pulls callbacks back to JS Call Stack
    

Syntax

CommonJS vs ESM:

CommonJS (CJS) uses require() and module.exports. It is synchronous and dynamic.

ES Modules (ESM) use import and export. It is asynchronous and statically analyzable. Add "type": "module" in package.json to use it.

Tiny Example

import fs from 'node:fs/promises';

async function run() {
  console.log('1. Start reading');
  const data = await fs.readFile('package.json', 'utf8');
  console.log('2. File read complete');
}

run();
console.log('3. Script continues immediately');
    

Walkthrough

In the tiny example, Node prints "1. Start reading". The fs.readFile call is handed off to libuv. V8 suspends the run function context. It then moves on to print "3. Script continues immediately". Once the file system finishes reading, the callback microtask is queued, the Event Loop picks it up, and V8 resumes run to print "2. File read complete".

Break It

Let's block the Event Loop.

setInterval(() => console.log('I should run every second!'), 1000);

// CPU-intensive blocking task
let sum = 0;
for (let i = 0; i < 10_000_000_000; i++) {
  sum += i;
}
console.log('Done calculating:', sum);
    

The setInterval will NOT fire until the billion-loop loop finishes. The single thread is occupied doing math; it cannot check the task queue.

Debug It

To fix CPU-blocking code in Node, we use Worker Threads.

import { Worker, isMainThread, parentPort } from 'node:worker_threads';

if (isMainThread) {
  const worker = new Worker(new URL(import.meta.url));
  worker.on('message', msg => console.log('From worker:', msg));
  setInterval(() => console.log('Event loop is free!'), 1000);
} else {
  let sum = 0;
  for (let i = 0; i < 10_000_000_000; i++) sum += i;
  parentPort.postMessage(`Done: ${sum}`);
}
    

Mini Project

A fast log processor using Streams and Buffers. Reading a 5GB file with fs.readFile crashes Node because it tries to load 5GB into RAM (V8 heap limit). Instead, we process data in chunks.

import { createReadStream, createWriteStream } from 'node:fs';
import { pipeline } from 'node:stream/promises';
import { Transform } from 'node:stream';

const upperCaseTransform = new Transform({
  transform(chunk, encoding, callback) {
    // chunk is a Buffer
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

async function processLogs() {
  await pipeline(
    createReadStream('input.log'),
    upperCaseTransform,
    createWriteStream('output.log')
  );
  console.log('Processing complete, with low memory usage!');
}
processLogs();
    

Real Application Feature

Utilizing Environment Variables (process.env) and Child Processes to trigger shell scripts in a CI pipeline runner.

import { spawn } from 'node:child_process';

const script = spawn('bash', ['deploy.sh'], {
  env: { ...process.env, DEPLOY_ENV: 'production' }
});

script.stdout.on('data', (data) => console.log(`stdout: ${data}`));
script.stderr.on('data', (data) => console.error(`stderr: ${data}`));
script.on('close', (code) => console.log(`Process exited with code ${code}`));
    

Production Implementation

Clustering for maximizing CPU core usage. Node is single-threaded. If your server has 8 cores, running one Node process wastes 7 cores. We use the cluster module (or process managers like PM2) to fork multiple instances listening on the same port.

import cluster from 'node:cluster';
import http from 'node:http';
import os from 'node:os';

const numCPUs = os.cpus().length;

if (cluster.isPrimary) {
  console.log(`Primary ${process.pid} is running`);
  for (let i = 0; i < numCPUs; i++) cluster.fork();
  cluster.on('exit', (worker) => cluster.fork()); // self-healing
} else {
  http.createServer((req, res) => {
    res.writeHead(200);
    res.end(`Hello from worker ${process.pid}\n`);
  }).listen(8000);
}
    

Production Usage

In production, you manage dependencies and versions strictly. Node relies on npm (Node Package Manager). package.json declares dependencies. Semantic Versioning (SemVer) rules apply:

• "express": "^4.17.1" - Updates minor and patch versions, but keeps major at 4 (non-breaking).
• "express": "~4.17.1" - Updates only patch versions (bug fixes).
• "express": "4.17.1" - Exact version.

A lockfile (package-lock.json) is vital. It freezes the exact dependency tree (including dependencies of dependencies) so every developer and the CI server installs the exact same bytes, preventing "it works on my machine" bugs.

Performance

Node.js shines at heavily concurrent I/O (handling thousands of simultaneous database reads, API calls, chat connections). It struggles with CPU-bound tasks (image processing, heavy crypto, large array sorting) on the main thread. Always offload CPU work to Worker Threads, external microservices (Go/Rust), or use streams to minimize memory footprint.

Best Practices

• Never block the Event Loop.
• Use streams for handling large payloads.
• Use environment variables for configuration.
• Always commit your lockfile.
• Graceful shutdown: Handle SIGTERM and SIGINT to close database connections cleanly before exiting.

Interview Questions

Engineering Challenge

Write a script using the fs and stream modules that reads a 2GB CSV file, filters out rows where column 3 is "false", and writes it to a new file without exceeding 100MB of RAM.

Revision Sheet

• Node.js = V8 (JS) + libuv (I/O) + C++ bindings.
• Single-threaded, non-blocking, event-driven.
• package.json holds metadata; package-lock.json guarantees deterministic installs.
• Use Worker Threads for CPU heavy lifting; Cluster for scaling across cores.
• Always process big files using Streams.

Connections

Now that we understand the runtime that powers backend JavaScript, we can confidently build robust APIs on top of it. Next, we will abstract the raw http module and look at Express and Fastify to build scalable RESTful architectures.