PM2, Graceful Shutdown and Clustering

Node.js runs in a single thread by default. On a multi-core server, this means one CPU does all the work while the others idle. PM2 is the standard production process manager for Node.js: it runs your app as a cluster (one process per CPU core), restarts it automatically on crash, manages logs, and provides zero-downtime deployments. Graceful shutdown ensures that when your process needs to stop (deployment, crash recovery), it first finishes processing in-flight requests before terminating. Together, these three concerns — clustering, graceful shutdown, and process management — determine whether your application handles production load reliably.

Analogy🏏Cricket

Think of it like cricket: A single-wicket match has one batsman facing an entire bowling attack — inefficient and exhausting. A full eleven-player squad distributes the batting workload across multiple batsmen in a rotating order. PM2's cluster mode is exactly this: instead of one Node.js process handling all requests, PM2 starts one process per CPU core and a load balancer distributes incoming requests across them. Graceful shutdown is like a batsman completing their final over before retiring hurt — they don't walk off mid-delivery; they finish the current ball, then hand over to the next player. Zero-downtime deployment is the super-substitute rule: a fresh player can enter the field without stopping the match.

Installing and Configuring PM2

PM2 is a global npm package. You define your application configuration in an ecosystem.config.js file which PM2 reads at startup. The key settings are instances (number of processes), exec_mode ('cluster' for multi-core), and autorestart (restart on crash).

bash

npm install -g pm2

javascript

// ecosystem.config.js
module.exports = {
  apps: [{
    name:         'cricket-api',
    script:       './server.js',
    instances:    'max',        // one per CPU core
    exec_mode:    'cluster',    // share the port across instances
    watch:        false,        // never watch in production
    max_memory_restart: '500M', // restart if process exceeds 500 MB
    env: {
      NODE_ENV: 'development',
      PORT:     3000
    },
    env_production: {
      NODE_ENV:  'production',
      PORT:      3000,
      LOG_LEVEL: 'info'
    },
    error_file:    './logs/pm2-error.log',
    out_file:      './logs/pm2-out.log',
    merge_logs:    true,
    log_date_format: 'YYYY-MM-DD HH:mm:ss Z'
  }]
};

bash

# Start in production mode
pm2 start ecosystem.config.js --env production

# List running processes
pm2 list

# Zero-downtime reload (completes in-flight requests before restarting each worker)
pm2 reload cricket-api

# View logs
pm2 logs cricket-api

# Save process list to survive server reboots
pm2 save
pm2 startup    # generates systemd/upstart startup script

Analogy🏏Cricket

Think of it like cricket: pm2 start is the team bus arriving at the ground — all eleven players (processes) start walking toward the field. pm2 reload is the team management protocol for injury replacements: the substitute is fully ready before the injured player walks off, so there is no gap in fielding coverage. pm2 stop is the formal end-of-match handshake: every player completes their current task and walks off in an orderly fashion.

Graceful Shutdown

Graceful shutdown means that when a SIGTERM signal is received (sent by PM2 on reload, Kubernetes on pod termination, or Docker on container stop), your app stops accepting new connections, waits for in-flight requests to complete, closes database connections and message queue connections, and then exits cleanly. Without graceful shutdown, in-flight requests receive a connection reset error, database connections are abruptly closed (causing timeouts for the next connection attempt), and queued messages may be lost.

Analogy🏏Cricket

🏏 Think of it like cricket: graceful shutdown is the umpire calling a proper close of play rather than someone yanking the stumps mid-delivery. When a SIGTERM arrives — from PM2 on reload, Kubernetes terminating a pod, or Docker stopping a container — your app stops accepting new connections the way the gates shut to new spectators, but lets the batsmen already at the crease finish the over: it waits for in-flight requests to complete. Only then does it close database and message-queue connections and exit cleanly, like the groundstaff covering the pitch and locking up once the last ball is genuinely bowled. Skip this and it's like pulling the stumps while a ball is in the air — in-flight requests get a connection reset, the equivalent of a delivery abandoned mid-flight, and database connections are left dangling. The payoff: draining in-flight work before exiting means reloads and shutdowns finish every request already in play, so no user ever gets a jarring connection-reset error.

javascript

// server.js — graceful shutdown implementation
const app      = require('./src/app');
const mongoose = require('mongoose');
const logger   = require('./src/utils/logger');

const PORT = process.env.PORT || 3000;

const server = app.listen(PORT, () => {
  logger.info('Server started', { port: PORT, pid: process.pid });
});

// Handle graceful shutdown
async function shutdown(signal) {
  logger.info('Shutdown signal received', { signal });

  // 1. Stop accepting new connections
  server.close(async () => {
    logger.info('HTTP server closed — no new connections accepted');

    try {
      // 2. Close database connections
      await mongoose.disconnect();
      logger.info('Database disconnected');

      // 3. Close queue workers (if any)
      // await emailWorker.close();

      logger.info('Graceful shutdown complete');
      process.exit(0);
    } catch (err) {
      logger.error('Error during shutdown', { error: err.message });
      process.exit(1);
    }
  });

  // Safety net: force exit after 30 seconds if graceful shutdown hangs
  setTimeout(() => {
    logger.error('Forced shutdown after timeout');
    process.exit(1);
  }, 30000).unref();  // .unref() prevents this timeout from keeping the process alive
}

process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGINT',  () => shutdown('SIGINT'));   // Ctrl+C in development

// Handle uncaught exceptions — log and exit
process.on('uncaughtException', (err) => {
  logger.error('Uncaught exception', { error: err.message, stack: err.stack });
  process.exit(1);
});

process.on('unhandledRejection', (reason) => {
  logger.error('Unhandled rejection', { reason: String(reason) });
  process.exit(1);
});