🌙
☀️ Dark
PART 36

Production Security

Threat modeling, secure CI/CD, audit logging.

Advanced 45 min read

Part 36: Production Security

Learning Objectives

You will learn to think like a defender. You will understand threat modeling, attack surface reduction, secrets management, dependency security, and how to build security into every layer of a production system rather than bolting it on at the end.

Why Does This Exist?

Security breaches are not rare events. They are a certainty for systems that treat security as an afterthought. The average cost of a data breach is $4.5M. More importantly, breaches destroy user trust in ways that are hard to recover from. Production security is not a checklist — it is a mindset applied at every engineering decision.

Threat Modeling

Before writing a security control, understand what you are protecting against.

STRIDE threat model:
┌──────────────────┬───────────────────────────────────────────┐
│ Spoofing         │ Pretending to be someone else             │
│ Tampering        │ Modifying data in transit or at rest      │
│ Repudiation      │ Denying an action occurred                │
│ Info Disclosure  │ Exposing sensitive information            │
│ Denial of Service│ Making the service unavailable            │
│ Elevation of Priv│ Gaining unauthorized permissions          │
└──────────────────┴───────────────────────────────────────────┘

Practical threat modeling for a new feature:
  1. What data does this feature handle?
  2. Who are the legitimate users?
  3. Who is the adversary? (external attacker, malicious insider, competitor)
  4. What is the worst thing an adversary could do?
  5. What controls prevent or detect it?
  6. What is the recovery path if controls fail?

Least Privilege

Every component should have only the permissions it needs to function.

Database users:
  -- API server: read/write on specific tables
  CREATE USER api_service WITH PASSWORD 'strong-password';
  GRANT SELECT, INSERT, UPDATE ON users, orders TO api_service;
  -- NOT: GRANT ALL ON ALL TABLES TO api_service;

IAM roles (AWS/GCP):
  Lambda function reading S3?
    → Grant: s3:GetObject on the specific bucket only
    → NOT: s3:* on all buckets, NOT: AdministratorAccess

Environment configuration:
  Development: can only connect to dev DB
  Staging:     can only connect to staging DB
  Production:  separate credentials, MFA required for human access

Secrets Management

NEVER:
  ✗ Hardcode secrets in source code
  ✗ Commit secrets to git (even in private repos)
  ✗ Store secrets in environment variable files committed to git
  ✗ Pass secrets via command-line arguments (visible in ps aux)
  ✗ Log secrets (even accidentally)

DO:
  ✓ Use a secrets manager (AWS Secrets Manager, HashiCorp Vault, Doppler)
  ✓ Rotate credentials regularly and automatically
  ✓ Use short-lived credentials where possible (AWS IAM roles, OIDC tokens)
  ✓ Audit access to secrets (who accessed what, when)
  ✓ Use .env files locally, add to .gitignore immediately
  ✓ Scan git history for accidentally committed secrets (trufflehog, git-secrets)

// Load secrets from environment at runtime
const config = {
  dbUrl: process.env.DATABASE_URL,      // From Secrets Manager
  stripeKey: process.env.STRIPE_SECRET, // From Secrets Manager
  // Fail fast if missing
  jwtSecret: (() => {
    const s = process.env.JWT_SECRET;
    if (!s) throw new Error('JWT_SECRET is required');
    return s;
  })()
};

Dependency Security

Your application is as secure as its weakest dependency.

npm audit:
  Run on every CI pipeline build.
  npm audit --audit-level=high
  Fail the build on high/critical vulnerabilities.

Dependency evaluation (before adding any package):
  ✓ When was the last release? (stale packages are risky)
  ✓ How many open GitHub issues? (indicators of maintenance)
  ✓ Is the maintainer trustworthy? (check their GitHub history)
  ✓ How many weekly downloads? (widely used = more audited)
  ✓ Does this package actually need this permission/access?
  ✓ Can I implement this myself in 50 lines?

Supply chain attacks:
  In 2022, malicious code was added to popular npm packages.
  Use exact versions in package.json (not ^1.2.3, use 1.2.3).
  Use package-lock.json. Review lockfile changes in PRs.
  Consider using npm ci in CI (respects lockfile exactly).

Secure CI/CD

CI/CD pipelines are a high-value attack target.
Compromising CI = arbitrary code execution in your prod environment.

Principles:
  ✓ Minimal permissions for CI service account
  ✓ Secrets injected at runtime, never stored in repo
  ✓ Pin action versions by SHA in GitHub Actions, not tags
  ✓ Separate deploy credentials per environment
  ✓ Require human approval for production deploys
  ✓ Audit trail of every deployment (who, what, when)

// GitHub Actions: pin by SHA, not tag
uses: actions/checkout@v4           # BAD: tag can be changed
uses: actions/checkout@11bd71901bbe96b187efa665cf23001fc11e5b9d  # GOOD: immutable

// Never do this
- name: Deploy
  env:
    AWS_SECRET_KEY: ${{ secrets.AWS_SECRET_KEY }}
  run: echo $AWS_SECRET_KEY  # Logs the secret!

Container Security

Docker security fundamentals:

1. Never run containers as root
   # Dockerfile
   RUN addgroup -g 1001 appgroup && adduser -u 1001 -G appgroup appuser
   USER appuser   ← runs as non-root

2. Use minimal base images
   FROM node:20-alpine  ← smaller attack surface than node:20
   FROM distroless/nodejs20-debian11  ← no shell at all

3. Scan images for vulnerabilities
   docker scout cves myapp:latest
   trivy image myapp:latest

4. Read-only filesystem
   docker run --read-only myapp  ← prevents runtime modifications

5. Drop capabilities
   docker run --cap-drop=ALL --cap-add=NET_BIND_SERVICE myapp

Audit Logging

For compliance, security, and incident investigation, maintain an immutable audit trail of all sensitive actions.

What to audit log:
  ✓ Authentication events (login, logout, failed attempts)
  ✓ Authorization failures (access denied)
  ✓ Data access (who read what sensitive record)
  ✓ Data modification (who changed what, what was the old value)
  ✓ Admin actions (user created, role changed, config modified)
  ✓ API key usage (which key, from which IP)

// Audit log structure
interface AuditEvent {
  timestamp: string;     // ISO 8601
  actor: {
    userId: string;      // who did it
    ip: string;          // from where
    userAgent: string;
  };
  action: string;        // 'user.password_changed'
  resource: {
    type: string;        // 'user'
    id: string;          // '123'
  };
  result: 'success' | 'failure';
  metadata: Record<string, unknown>;
}

// Audit logs must be:
// 1. Append-only (no modification or deletion)
// 2. Stored separately from application logs
// 3. Retained per compliance requirements (often 7 years)

Mini Project (20-30 min)

Implement a secure secret fetching utility that sanitizes inputs and uses zero-trust principles.

▶ View Solution
typescript
async function getSecret(key: string, requesterContext: any) {
  if (!requesterContext.isAuthenticated || !requesterContext.hasMfa) {
    throw new Error('Access Denied: Zero Trust policy requires MFA');
  }
  if (!/^[a-zA-Z0-9_]+$/.test(key)) {
    throw new Error('Invalid secret key format');
  }
  // Simulate secure vault fetch
  return process.env[key] || null;
}

Bigger Project (1-2 hours)

Design a secure, Zero Trust authentication gateway. Implement mutual TLS (mTLS) between two internal services, integrate a secrets manager (like HashiCorp Vault), and simulate a supply chain attack mitigation strategy.

\n
▶ View Solution
typescript
// Example JWT validation
const token = req.headers.authorization.split(" ")[1];
jwt.verify(token, process.env.JWT_SECRET, { algorithms: ["HS256"] }, (err, decoded) => {
  if (err) return res.status(401).send("Unauthorized");
  req.user = decoded;
  next();
});

Interview Questions

\n
Core: What is Cross-Site Request Forgery (CSRF) and how is it prevented?

CSRF is an attack where an authenticated user is tricked into executing unwanted actions on a web application where they are currently authenticated. It is typically prevented by using anti-CSRF tokens (a unique, secret, and unpredictable value) that are validated on the server for state-changing requests, or by using SameSite cookie attributes.

Hard: How do you defend against Server-Side Request Forgery (SSRF) in a cloud environment?

SSRF allows an attacker to make requests from the server to internal resources. To defend against it, implement strict allow-lists for outbound URLs, validate and sanitize all user input, and block access to sensitive internal IPs like the AWS Instance Metadata Service (IMDSv2 mitigates this by requiring a session token) or internal subnets.

Senior: What is a supply chain attack in the context of NPM packages, and how do you mitigate it at scale?

A supply chain attack occurs when a malicious actor compromises an upstream dependency (e.g., typosquatting or hijacking a package maintainer's account). Mitigation involves pinning exact dependency versions, using lockfiles, employing vulnerability scanners (like Snyk or Dependabot), requiring 2FA for publishing, and mirroring dependencies in a private registry to audit code before it enters the build pipeline.

Revision Sheet

  • Threat model first: STRIDE. Who is the adversary? What do they want?
  • Least privilege: Every service, user, and role gets minimum needed permissions.
  • Secrets: Secrets manager, never in code/git, rotate regularly.
  • Dependencies: npm audit in CI, pin versions, evaluate before adding.
  • Containers: Non-root user, minimal image, scan for CVEs.
  • Audit logs: Append-only, separate storage, long retention.
🏠 Curriculum NextVolume 2