Backup Automation: Scripts, Compression, and Integrity

Previously in this course, we explored shell scripting logic and shell functions to handle complex tasks, and learned how to automate recurring tasks with cron. In this lesson, we tie these concepts together to build a professional-grade backup pipeline for our web server.

The Philosophy of Robust Backups

A backup that hasn't been verified is just a collection of files that might be corrupted. To ensure our server data is safe, we need three distinct stages:

1. Packaging: Creating a compressed snapshot of the data.
2. Scheduling: Using cron to ensure backups happen without manual intervention.
3. Verification: Confirming the backup is complete and readable.

Building the Backup Script

We will create a script that compresses our project’s web root directory and moves it to a dedicated backup folder. We'll leverage the tar command for archiving and gzip for compression, as covered in our guide on file archiving.

Create a file named backup.sh in your project directory:

Bash
#!/bin/bash

# Configuration
SOURCE_DIR="/var/www/my-web-app"
BACKUP_DIR="/var/backups/web-app"
TIMESTAMP=$(date +"%Y-%m-%d_%H-%M-%S")
BACKUP_FILE="$BACKUP_DIR/backup_$TIMESTAMP style="color:#569CD6">.tar.gz"

# Ensure backup directory exists
mkdir -p "$BACKUP_DIR"

# Perform backup and log output
echo "Starting backup: $TIMESTAMP"
tar -czf "$BACKUP_FILE" -C "$SOURCE_DIR" . 2> /tmp/backup_error.log

# Verify integrity
if [ $? -eq 0 ]; then
    echo "Backup successful: $BACKUP_FILE"
    # Verify the archive content
    tar -tf "$BACKUP_FILE" > /dev/null
    if [ $? -eq 0 ]; then
        echo "Integrity check passed."
    else
        echo "Integrity check failed!" | mail -s "Backup Alert" admin
    fi
else
    echo "Backup failed. See /tmp/backup_error.log"
fi

Automating with Cron

Once your script is tested, you need to schedule it. Use crontab -e to add a job that runs at 3:00 AM daily. Remember, you might need sudo privileges to back up system directories:

Bash
0 3 * * * /usr/local/bin/backup.sh

Common Pitfalls in Backup Automation

• Assuming Success: Never rely on a backup without a status check. Always check the exit code ($?) of your tar command.
• Disk Exhaustion: If you run daily backups without cleaning up, your server will eventually run out of space. Add a line to your script to remove files older than 30 days: find "$BACKUP_DIR" -type f -mtime +30 -delete.
• Relative Path Nightmares: Always use absolute paths in cron jobs and scripts, as the environment cron runs in is often minimal and doesn't know your current working directory.
• Permission Errors: If the script runs as a user who doesn't have read access to the web files, the backup will be empty. Ensure your backup user has appropriate permissions or run the script via sudo.

Hands-on Exercise

1. Refactor: Modify the provided script to include the "cleanup" command mentioned above (deleting files older than 30 days).
2. Test: Manually run the script twice to verify that the mkdir -p command handles existing directories gracefully.
3. Audit: Check the exit status of your manual run using echo $? immediately after the script finishes.

FAQ

Q: Should I store backups on the same drive? A: Never. If that drive fails, you lose both your live data and your backups. Always use a remote location or a separate physical disk.

Q: How do I know if the backup is actually valid? A: tar -tf lists the contents of an archive. If it can list the files without error, the archive structure is likely intact. For mission-critical data, consider using checksums (like sha256sum) to verify the file hasn't changed.

Q: Why use tar instead of just copying files? A: tar preserves file permissions and metadata (ownership, timestamps), which is vital for restoring a web server to a working state.

Recap

We have moved from manual file management to automated, verified protection. By combining tar for compression, if logic for error handling, and cron for scheduling, you've implemented a fundamental piece of DevOps infrastructure.

Up next: Learn to troubleshoot and monitor your system's health by monitoring system logs.