100% Free Forever
AI-Powered Learning
Industry Expert Content
Certificates & Badges
Learn At Your Own Pace
Linux & Shell Scripting
55 minbeginner

Scripting Practice — Backup Automation Script

What You'll Build

In this exercise you will build `cricket_backup.sh` — a production-grade backup automation script for CricketPulse that backs up the application data, database dump, and configuration files to a structured archive directory. The script implements all the scripting fundamentals from Module 5: structured functions, argument parsing, input validation, error handling with traps, retry logic, and comprehensive logging with structured output.

The backup script handles three backup types: a full backup (all components), a data-only backup (application data without database), and a config backup (configuration files only). Each type is implemented as a function, the main function orchestrates them based on the requested type, and the cleanup trap ensures partial backups are removed if any step fails. This mirrors the structure of real backup automation used in production environments.

Backup automation is a safety-critical script — a backup that appears to succeed but produces a corrupt or incomplete archive provides false confidence and fails exactly when it is needed most. This exercise pays particular attention to verification: each backup component is verified after creation, and the final script exits with a non-zero code if any verification fails, ensuring a CI/CD pipeline can detect and alert on backup failures.

Analogy🏏Cricket
🏏 Think of it like cricket: This script is like the pre-match ground inspection conducted by the match referee, pitch curator, and captains before a Test match begins. Before Rohit Sharma and the opposition captain walk out for the toss, the curator has measured pitch moisture, taken grass length readings, and documented the surface condition — creating a baseline against which any afternoon deterioration can be measured.Just as this structured inspection prevents surprises and creates a documented record, the inventory script creates a documented baseline for a server against which future anomalies can be compared. Just as a ground inspection without a checklist might miss a drainage issue that affects the afternoon session, a server assessment without a structured script might miss a nearly-full disk that causes a midnight deployment failure.The insight is that the value of a structured inspection is not just the current findings but the reproducible method — the same script run tomorrow highlights exactly what changed.

Prerequisites

  • Completion of Lessons 17–19: Bash scripting fundamentals (set -euo pipefail, structure), variables/conditionals/loops (arrays, retry patterns), and functions/error handling (traps, return codes).
  • A Linux system with `tar`, `gzip`, `pg_dump` (or mock substitute), and `sha256sum` installed — all standard on Ubuntu 20.04+.
  • The CricketPulse directory structure from previous exercises, or the setup commands in this lesson.
  • Understanding of the `trap` and `cleanup` pattern from Lesson 19 — the backup script relies on this for partial backup cleanup.
  • Understanding of the structured `log()` function pattern — the backup script uses this for all output.

Setup & Project Structure

The setup creates the CricketPulse directory structure with realistic data to back up: application data files, a simulated database dump, and configuration files. It also creates the backup destination directory with appropriate permissions — the backup directory must be writable by the script's user but should not be world-readable if it will contain database dumps with credentials.

Analogy🏏Cricket
🏏 Think of it like cricket: building `cricket_inventory.sh` incrementally across four steps is like a batter building an innings session by session rather than swinging for six off the first ball. Just as you only need a bat and a net to start practising — no fancy equipment — this exercise needs only a text editor and a shell, nothing installed. Just as a batter grooves one shot at a time, the cover drive before the pull, each step adds exactly one report section and introduces only the commands needed to gather that piece of information. Just as marking your guard and taking centre sets a repeatable stance, creating a dedicated exercise directory and making the file executable gives you a clean, runnable base to return to. And just as a solid start makes the big total possible, getting the script structure right first means the later sections drop in without collapse. The payoff: incremental practice builds a working system-inventory script the same disciplined way a batter builds a match-winning score.

The script will be built incrementally — the function stubs are created first, then each function is implemented and tested in isolation before being integrated into the main function. This incremental approach mirrors how production scripts are developed and reviewed: each component is validated independently before composition, preventing the scenario where a complex integrated script fails in a way that obscures which individual component is broken.

bash
#!/bin/bash
# setup_backup_target.sh — creates CricketPulse data to back up
set -euo pipefail

mkdir -p /opt/cricketpulse/{data,assets,cache}
mkdir -p /etc/cricketpulse
mkdir -p /var/backup/cricketpulse

# Application data
echo 'match_id,team1,team2,score' > /opt/cricketpulse/data/matches.csv
echo '1001,India,Australia,287/5' >> /opt/cricketpulse/data/matches.csv
echo 'player_id,name,runs' > /opt/cricketpulse/data/players.csv
dd if=/dev/urandom bs=1k count=50 2>/dev/null | base64 > /opt/cricketpulse/assets/banner.b64

# Configuration
cat > /etc/cricketpulse/config.yaml << 'EOF'
app:
  name: cricketpulse
  version: v2.1.0
database:
  host: localhost
  port: 5432
EOF
chmod 640 /etc/cricketpulse/config.yaml

# Mock pg_dump (since we may not have PostgreSQL)
cat > /usr/local/bin/mock_pg_dump << 'EOF'
#!/bin/bash
echo '-- PostgreSQL database dump'
echo 'CREATE TABLE matches (id INT, team1 VARCHAR(50));'
echo "INSERT INTO matches VALUES (1001, 'India');";
EOF
chmod +x /usr/local/bin/mock_pg_dump

chown -R root:root /var/backup/cricketpulse
chmod 750 /var/backup/cricketpulse
echo 'Setup complete'

Step 1 — Foundation: Script Structure and Logging

Step 1 establishes the script's structure: the safety header, constants, exit codes, the logging functions, and the cleanup trap. At this stage, no backup logic exists — the goal is to verify that the framework is correct before adding domain logic on top. Running the partial script should produce a usage message and exit cleanly.

Analogy🏏Cricket
🏏 Think of it like cricket: building the safety header, constants, exit codes, logging functions, and cleanup trap before any backup logic is like a batter grooving stance, guard, and footwork in the nets before ever trying to score — get the frame right and the runs follow safely. Just as a coach checks the basics hold up before adding aggressive shots, running the partial script should just print a usage message and exit cleanly, proving the framework is sound with no domain logic yet. Just as a fielder rehearses backing-up before the game so it is automatic under pressure, the cleanup trap is wired in first so it fires however the run ends. And just as clear signals between fielders prevent confusion, the logging functions give every later step a consistent voice. The payoff: verifying the skeleton is correct first means the backup logic drops onto a stable base instead of a shaky stance.

The logging function uses structured output with timestamp, level, and source location — the same format that log aggregators can parse automatically. The cleanup trap captures the exit code before any other command can overwrite it and uses that code to both clean up partial backups and set the final exit code for the caller. Getting this infrastructure right before adding domain logic prevents debugging confusion where error handling bugs obscure domain logic bugs.

bash
#!/bin/bash
# cricket_backup.sh — Step 1: Structure and logging
set -euo pipefail

# --- Constants ---
readonly BACKUP_DIR='/var/backup/cricketpulse'
readonly APP_DIR='/opt/cricketpulse'
readonly CONFIG_DIR='/etc/cricketpulse'
readonly DB_NAME='cricketpulse'
readonly RETENTION_DAYS=30

# --- Exit codes ---
readonly E_OK=0
readonly E_ARGS=2
readonly E_IO=3
readonly E_VERIFY=4

# --- Logging ---
SCRIPT='cricket_backup.sh'
log()   { echo "[INFO]  $(date -u +%Y-%m-%dT%H:%M:%SZ) [$SCRIPT] $*"; }
warn()  { echo "[WARN]  $(date -u +%Y-%m-%dT%H:%M:%SZ) [$SCRIPT] $*" >&2; }
error() { echo "[ERROR] $(date -u +%Y-%m-%dT%H:%M:%SZ) [$SCRIPT] $*" >&2; }
die()   { error "$*"; exit 1; }

# --- Cleanup ---
BACKUP_FILE=''
cleanup() {
    local ec=$?     # Capture before any command overwrites it
    if [[ "$ec" -ne 0 && -n "$BACKUP_FILE" && -f "$BACKUP_FILE" ]]; then
        warn 'Removing partial backup due to failure'
        rm -f "$BACKUP_FILE" "${BACKUP_FILE}.sha256" 2>/dev/null || true
    fi
    exit "$ec"
}
trap cleanup EXIT

# --- Usage ---
usage() {
    echo "Usage: $0 <type> [--dry-run]"
    echo "  type: full | data | config"
    exit $E_ARGS
}

[[ $# -ge 1 ]] || usage
BACKUP_TYPE=${1:-''}
DRY_RUN=${2:-''}

[[ $BACKUP_TYPE =~ ^(full|data|config)$ ]] || { error "Invalid type: $BACKUP_TYPE"; usage; }

log "Starting $BACKUP_TYPE backup (dry-run: ${DRY_RUN:-no})"

Step 2 — Core Logic: Backup Functions

Step 2 implements the three backup component functions: `backup_database`, `backup_appdata`, and `backup_config`. Each function creates its component, verifies the archive is not empty and has the expected structure, and logs the result. Each returns a named exit code so the caller can distinguish between an I/O failure and a verification failure and respond appropriately.

The verification step uses `tar -tzf archive.tar.gz | head -5` to list the archive contents without extracting — this confirms the archive is a valid gzip-compressed tar file with actual content, not just a zero-byte file or a corrupted header. A backup that passes this check is recoverable; one that fails is useless. Every production backup system implements some form of this smoke-test verification.

Analogy🏏Cricket
🏏 Think of it like cricket: The backup verification is like the ICC's match scorecard validation process. After the scorers submit the official scorecard, the match referee does not simply file it — they verify that the total wickets match the declared/all-out state, that the runs add up correctly, and that all player entries are complete. A scorecard that fails validation is returned for correction before filing.Just as a filed-but-invalid scorecard creates historical record problems, a backup that reports success but contains a corrupt archive creates disaster recovery problems. Just as the match referee verifies the scorecard's internal consistency before filing it — not trusting that 'the scorer submitted it so it must be correct' — the backup script verifies the archive's readability before reporting success.The insight is that verification is what transforms 'a backup ran' into 'a backup that can be restored' — the former provides false confidence, the latter provides real security.
bash
# Step 2: Backup component functions

backup_database() {
    local dest_file=${1:?'dest_file required'}
    log 'Backing up database...'

    # Mock pg_dump for exercise; use pg_dump in production
    /usr/local/bin/mock_pg_dump --no-password "$DB_NAME" 2>/dev/null \
        | gzip > "$dest_file" || return $E_IO

    # Verify archive is valid
    gzip -t "$dest_file" 2>/dev/null || { error 'Database backup corrupted'; return $E_VERIFY; }
    local size; size=$(stat -c '%s' "$dest_file")
    [[ "$size" -gt 0 ]] || { error 'Database backup is empty'; return $E_VERIFY; }
    log "Database backup OK: $(numfmt --to=iec $size)"
}

backup_appdata() {
    local dest_file=${1:?'dest_file required'}
    log 'Backing up application data...'

    [[ -d "$APP_DIR/data" ]] || { warn 'No app data directory'; return 0; }
    tar czf "$dest_file" -C "$APP_DIR" data assets 2>/dev/null || return $E_IO

    # Verify archive structure
    tar -tzf "$dest_file" | head -3 | grep -q 'data/' || \
        { error 'App backup missing data/ directory'; return $E_VERIFY; }
    log "App data backup OK: $(stat -c '%s' "$dest_file" | numfmt --to=iec)"
}

backup_config() {
    local dest_file=${1:?'dest_file required'}
    log 'Backing up configuration...'

    [[ -d "$CONFIG_DIR" ]] || { warn 'No config directory'; return 0; }
    tar czf "$dest_file" -C / \
        "${CONFIG_DIR#/}" 2>/dev/null || return $E_IO

    tar -tzf "$dest_file" | grep -q 'cricketpulse' || \
        { error 'Config backup missing cricketpulse entries'; return $E_VERIFY; }
    log "Config backup OK: $(stat -c '%s' "$dest_file" | numfmt --to=iec)"
}

Step 3 — Integration: Main Function and Checksums

Step 3 implements the main function that orchestrates the backup components, generates a SHA-256 checksum file for the final archive, implements retention cleanup for old backups, and logs a summary. The checksum enables future integrity verification — before restoring a backup, `sha256sum --check backup.tar.gz.sha256` confirms the archive has not been corrupted or tampered with since creation.

The retention cleanup uses `find` with `-mtime +RETENTION_DAYS -delete` to remove old backups automatically. Before the deletion, it lists what would be deleted (logging the files by name) so the log provides a clear record of which old backups were removed. This log is valuable during audits that require proof of backup retention policy enforcement.

Analogy🏏Cricket
🏏 Think of it like cricket: The checksum file is like the authentication seal on an ICC official document. When a historical scorecard is retrieved from the ICC archives, the authentication seal confirms it has not been altered since filing — the document is the same one the referee approved and filed on match day. Without the seal, a retrieved document cannot be distinguished from a forgery.Just as `sha256sum --check backup.sha256` confirms the archive has not been modified since creation — providing the same assurance as the ICC's authentication seal — a backup without a checksum cannot be verified as unmodified. Just as the retention cleanup removing old scorecards is like the ICC's document management policy that keeps only the last five years of detailed records and summarises older ones, the backup retention policy removes archives older than the configured threshold.The insight is that checksums transform storage into trusted storage — the difference between having a backup and having a backup you can rely on.
bash
# Step 3: Main function

rotate_old_backups() {
    local count; count=$(find "$BACKUP_DIR" -name '*.tar.gz' -mtime +"$RETENTION_DAYS" | wc -l)
    if [[ "$count" -gt 0 ]]; then
        log "Removing $count backup(s) older than ${RETENTION_DAYS} days"
        find "$BACKUP_DIR" -name '*.tar.gz' -mtime +"$RETENTION_DAYS" -print -delete
        find "$BACKUP_DIR" -name '*.sha256' -mtime +"$RETENTION_DAYS" -delete
    fi
}

main() {
    local type=${1:?}
    local dry_run=${2:-''}
    local timestamp; timestamp=$(date +%Y%m%d_%H%M%S)
    local staging; staging=$(mktemp -d)
    BACKUP_FILE="${BACKUP_DIR}/cricketpulse_${type}_${timestamp}.tar.gz"

    [[ -d "$BACKUP_DIR" ]] || die "Backup directory $BACKUP_DIR does not exist"

    [[ "$dry_run" == '--dry-run' ]] && { log "Dry run: would create $BACKUP_FILE"; return 0; }

    # Run requested backup type
    case "$type" in
        full)   backup_database "${staging}/db.sql.gz"
                backup_appdata  "${staging}/app.tar.gz"
                backup_config   "${staging}/config.tar.gz" ;;
        data)   backup_appdata  "${staging}/app.tar.gz" ;;
        config) backup_config   "${staging}/config.tar.gz" ;;
    esac

    # Bundle all components into final archive
    tar czf "$BACKUP_FILE" -C "$staging" . || die 'Failed to create final archive'
    rm -rf "$staging"

    # Generate checksum
    sha256sum "$BACKUP_FILE" > "${BACKUP_FILE}.sha256"
    log "Checksum: $(cat ${BACKUP_FILE}.sha256 | awk '{print $1}')"

    # Retention
    rotate_old_backups

    local size; size=$(stat -c '%s' "$BACKUP_FILE")
    log "Backup complete: $BACKUP_FILE ($(numfmt --to=iec $size))"
}

main "$BACKUP_TYPE" "$DRY_RUN"

Step 4 — Testing & Verification

Run all three backup types and verify each produces a valid, verifiable archive. Test the dry-run mode to confirm it exits with code 0 and produces no files. Simulate a backup failure by temporarily making the app data directory unreadable, confirm the script exits with a non-zero code, and verify the cleanup trap removed the partial backup file. Finally, restore a file from the full backup to confirm the archive is actually restorable.

Analogy🏏Cricket
🏏 Think of it like cricket: running the full script and cross-checking its numbers against `nproc`, `free -h`, and `df -h` is like a scorer confirming the big-screen total against the umpire's notebook and the third umpire's feed before it goes in the record. Just as a total that only appears on the scoreboard is untrusted until verified against an independent source, the report's CPU, memory, and disk figures are only trustworthy once they match the raw commands. Just as the score must show correctly to both the crowd in the ground and in the official book, the output should appear simultaneously in the terminal and in the timestamped file. Just as a good scorer confirms every figure comes from the right source rather than a stale note, you confirm the script reads from the correct system sources. The payoff: cross-checking against independent commands proves the inventory report is accurate, not just plausible-looking.

The restoration test is the most important verification step — it confirms the backup is not just a valid archive but one that actually contains the expected content in a restorable format. Run `tar -tzf /var/backup/cricketpulse/cricketpulse_full_*.tar.gz` to list contents, then `tar -xzf archive.tar.gz -C /tmp/restore_test` to extract and verify the extracted files match the originals with `diff`.

bash
# Step 4: Full test cycle

echo '=== TEST 1: Full backup ==='
bash cricket_backup.sh full
LATEST=$(ls -t /var/backup/cricketpulse/cricketpulse_full_*.tar.gz 2>/dev/null | head -1)
echo "Latest backup: $LATEST"
sha256sum --check "${LATEST}.sha256" && echo '[PASS] Checksum verified'

echo '=== TEST 2: Dry run ==='
bash cricket_backup.sh config --dry-run
echo "Exit code: $?"   # Should be 0
ls /var/backup/cricketpulse/   # Should show no new files

echo '=== TEST 3: Failure cleanup ==='
chmod 000 /opt/cricketpulse/data   # Make unreadable
bash cricket_backup.sh data && echo 'FAIL: should have failed' || {
    echo "Exit code $? — expected non-zero"
    ls /var/backup/cricketpulse/*.tar.gz 2>/dev/null && echo 'FAIL: partial backup remains' \
        || echo '[PASS] Cleanup trap removed partial backup'
}
chmod 755 /opt/cricketpulse/data   # Restore permissions

echo '=== TEST 4: Restore verification ==='
RESTORE_DIR=$(mktemp -d)
tar -xzf "$LATEST" -C "$RESTORE_DIR"
ls -la "$RESTORE_DIR"
# Verify original data is intact in the restore
diff /opt/cricketpulse/data/matches.csv \
    <(tar -xzf "${RESTORE_DIR}/app.tar.gz" --to-stdout data/matches.csv 2>/dev/null) && \
    echo '[PASS] Restore verified: matches.csv matches original' || echo '[FAIL] Restore mismatch'
rm -rf "$RESTORE_DIR"

Warning: Never test backup cleanup by running the full backup against production data and then verifying the cleanup deleted files — a bug in the cleanup logic might delete the backup files you just created instead of the old ones. Always test cleanup separately with a controlled set of old test files: create mock files with `touch -d '35 days ago' /var/backup/cricketpulse/test_old_backup.tar.gz`, run the cleanup logic in isolation, and verify the correct files were removed before integrating it into the full backup script.

Extension Challenge: Add three enhancements to `cricket_backup.sh`. First, add remote backup support using `rsync` — after the local backup completes, sync the backup directory to a remote server with `rsync -az --delete /var/backup/cricketpulse/ backup-server:/backups/cricketpulse/`. Second, add a restore function that takes a backup file path and a target directory, verifies the checksum before extracting, and reports which files were restored. Third, add a backup inventory report: a `list` command that shows all available backups with their type, timestamp, size, and checksum status — using `sha256sum --check` to verify each archive's integrity as part of the inventory.

  • Capture `$?` as the first line of the cleanup trap — every subsequent command overwrites `$?`, so `local ec=$?` at the very start of the cleanup function is required to preserve the original exit code.
  • Verify backup archives immediately after creation — `gzip -t archive.gz` checks gzip integrity and `tar -tzf archive.tar.gz | head` confirms valid tar structure before reporting success.
  • Generate and store `sha256sum` checksums alongside every backup archive — checksums enable future integrity verification before restoration and detect silent corruption during storage.
  • Remove partial backups in the cleanup trap when the exit code is non-zero — a partial backup is worse than no backup because it occupies disk space and may be mistaken for a complete backup.
  • Test the restore path, not just the backup path — a backup that cannot be restored is useless, and the only way to confirm restorability is to actually extract and verify the archive contents.
  • Use `find -mtime +N -print -delete` for retention cleanup — the `-print` before `-delete` creates a log entry for each removed file, providing a retention audit trail.
Lesson 20 of 40
0% complete