Unfinished Bitness

Until you’ve experienced data loss first hand, or have seen someone go through the process of loss and recovery, you probably don’t pay much attention to backups.  I’ve been backing up my data regularly since the mid 1990’s.  A few years ago, a backup saved my bacon, and taught me a valuable lesson.  Here I detail the importance of backups, and tell you how to make them easy.

You may be thinking, so what, it won’t matter if I lose some documents.  Sure, an annoyance, but not severe.  But what about your family photos, your financial records, amongst others?  Some are not easily reproduced, others simply can’t be.

Causes

There are a bunch of other reasons you want a backup for.

Device Failure

The hard drives in our computers are mostly silent and go largely un-noticed.  The question is not “if” they will fail, it is “when” they will fail.  As with any device, the hard drive in your computer has a service lifetime.  Most desktop and laptop hard drives spin at between 5400 rpm and 7200 rpm. There are a myriad of things that can go wrong and cause a failure, including rated life span.

Theft / Lost

It happens.  Your desktop computer could be stolen from your home, or you might lose your laptop.  So you say you have them backed up to an external USB drive.  OK, but what about the following?

Flood / Fire / Natural Disaster

Yes, these things can ruin all of your equipment in a hurry.

Aliens

It might happen.  You never know.

Backup Planning

The first thing you want to do is make a backup plan.  You need to identify what needs backed up, their location, and how it will be done.

Here are some items you might want to consider.  You will need to have the directory location on your hard drive for each one:

• Pictures
• Home Movies
• Checkbook data files
• Tax software data files (returns)
• Legal documents in PDF form
• Awards / Certificates in PDF form
• Source code (if you’re a programmer)

Now you must decide how the backups will be done.  There are a plethora of options available.

External Hard Drive

Backing up to external hard drive is great first step.  It’s easily accessible if you need to restore.  This protects you from primary hard drive failure, but only doing this leaves you susceptible to failure of the backup device, and to all the other causes outlined above.

You should have at least two (2) external hard drives.  One should be stored AWAY from the computer being backup up, in the same geo location.  The other should be stored offsite, which could be a relatives house or safe deposit box.  The idea is that you swap them out after each backup so the most current is local, and the previous one is offsite. If you keep it at a persons house, you will want to make sure it is encrypted.  Encrypting it isn’t a bad idea anyway, just in case it is stolen.

This method will yield 2 to 3 copies of your data, depending on if you rotate a 2nd device offsite.

Network Attached Storage (NAS)

This is like an external hard drive backup, but you don’t have to plug it in each time because it is connected to, and accessible on your network.  This removes some physical hassle out of doing the backup.  However, a NAS device isn’t very suitable to offsite swapping.  It is also still susceptible to the disaster scenarios.

The beauty of a NAS is how they can be configured.  When implementing a NAS solution, you should look for one that employs at least two (2) drives.  When configuring the system, make sure the drives are mirrored (RAID 1).  This effectively cuts the storage capacity in half, but if one of the drives fails, the data is still intact on the other one.  So if you get a NAS with two 4TB drives, you won’t get 8TB of storage, you’ll get 4TB.  With two mirrored drives, you will have 3 copies of your data after a backup is completed.

Another feature some NAS devices offer is the ability to upload data to an offsite Internet Storage provider (see Internet Storage below).  This eliminates the physical loss scenarios, and gives you one more copy of your data.

Internet Service

You could use an internet service for backup, which works by using software installed to your computer to upload your data to the providers data center (aka: in the cloud).  These often run completely automated, removing all barriers to performing the backup.

Consider the sensitivity of the documents you choose to backup.  You may want to encrypt them locally and only have the encrypted versions backed up.  Unless of course you completely trust the company providing the service to a) keep your data out of employees hands; b) employ adequate account security to prevent outside access.

Some providers examples are (in no particular order): Carbonite, Crashplan, Mozy, and Amazon Glacier.  It’s noteworthy to mention that I tried Amazon Glacier and was not pleased with how the data files are stored.  They are plenty secure, but the container names are encoded in such a way that you can’t tell which file is which without using a client that can read the index.

Internet Storage

Similar to the Internet Services above, but these are just storage.  You will need to perform the backups by copying files on a schedule you set.  Some of them can do some degree of synchronization for you, helping to reduce the burden of rememberance.  These typically have smaller storage pools available than the Internet Services, unless you pay extra.

Some examples are (in no particular order): DropBox, Box, Google Drive, OneDrive, etc.

Make It Happen

You’ve identified the what and how.  Now what?  Implement it!

How I Did It

Let’s look at how I implemented a backup solution.  This may help you decide how you want to.

First I divided my primary documents folder into two sections:

• Local Only: Documents placed here are only backed up locally.  These include correspondence, instructions downloaded, owners manuals, etc.  Basically stuff that is not critical if it is lost, or can be easily replaced.  These files are still susceptible to the physical loss scenarios.
• Offsite: Documents placed here are backed up locally, AND to an offsite location.  These include checkbook data, tax returns, bank statements, certifications, legal documents, etc.  These files are no longer susceptible to ANY of the loss scenarios.

Next I created a backup script.  This script creates a tar file of the Offsite documents, and a separate tar file for my pictures.  It then creates a zip file of each tar file.  This zip is done with a long encryption key that only I know.  You’ll see why the encryption is done shortly.  Then the script removes the tar files, and copies the zip files to my NAS drive in a Backup folder.  Once copied, the zip files are removed as well.  An example of the backup script is here:

#!/bin/sh

iALL=0
iDocs=0
iPics=0
iDebug=0
cUser=`whoami`
cPass=""

# -------------------------------------------------------------------
# Function......: DoSyntax
# Description...: Displays Syntax
# -------------------------------------------------------------------
function DoSyntax {
    echo "Syntax: backup.sh [-a] [-d] [-p][-x]"
    echo "        -a = Backup all"
    echo "        -d = Backup Documents"
    echo "        -p = Backup Pictures"
    echo "        -x = Debug mode (perform no actions)"
    exit 1
}

# -------------------- M A I N   R O U T I N E --------------------

# Get parameters
while getopts ":adpx" opt; do
    case $opt in
        a) iALL=1 ;;
        d) iDocs=1 ;;
        p) iPics=1 ;;
        x) iDebug=1 ;;
        \?) DoSyntax ;;
    esac
done

if [[ ${iDebug} = 1 ]]; then
    echo "*** DEBUG MODE ***"
    echo "--- Actions will NOT occur! ---"
    echo " "
fi

# -------------------- GET PASSWORD FOR MOUNTS --------------------
read -s -p "Enter mount password for user ${cUser}:" cPass
echo " "
if [[ ${iDebug} = 1 ]]; then
    echo "Password: ${cPass}"
fi

# -------------------- BACKUP KEY DATA TO NAS FOR OFFSITE PUSH --------------------
# ----- Documents

if [[ ${iALL} = 1 ]] || [[ ${iDocs} = 1 ]]; then

    # -------------------- PREPARE DOCUMENTS --------------------
    # Tar, Zip Encrypt, Remove unencrypted tar
    if [[ ${iALL} = 1 ]] || [[ ${iDocs} = 1 ]]; then
        echo " "
        echo "Preparing documents for backup..."
        if [[ ${iDebug} = 0 ]]; then
            echo "     - tarring files into one..."
            tar -cvf /Users/username/0ffsite/MyDocuments.tar /Users/username/Documents/*
            echo "     - zipping file with encryption..."
            zip -v -P "super-secret-password" /Users/username/0ffsite/MyDocuments.zip /Users/username/0ffsite/MyDocuments.tar
            echo "     - removing unencrypted tar file..."
            rm -f /Users/username/0ffsite/MyDocuments.tar
        fi
    fi


    # -------------------- COPY DATA TO NAS --------------------

    # Mount the backup drive
    if [[ ${iDebug} = 0 ]]; then
        mkdir /Volumes/Data
        mount_afp -i afp://username:${cPass}@nas.home/Data /Volumes/Data
    fi

    # ---------- COPY DOCUMENTS ----------
    if [[ ${iALL} = 1 ]] || [[ ${iDocs} = 1 ]]; then
        echo " "
        echo "Copying critical (0ffsite) documents to NAS backup..."
        if [[ ${iDebug} = 0 ]]; then
            echo "     - copying zip file to NAS..."
            cp /Users/username/0ffsite/MyDocuments.zip /Volumes/Data/Backup
        fi
    fi

    # Unmount the backup drive
    if [[ ${iDebug} = 0 ]]; then
        umount /Volumes/Data
    fi
fi


# -------------------- BACKUP PICTURE DATA TO NAS FOR OFFSITE PUSH --------------------

if [[ ${iALL} = 1 ]] || [[ ${iPics} = 1 ]]; then

    # -------------------- PREPARE DOCUMENTS --------------------
    # Tar, Zip Encrypt, Remove unencrypted tar
    if [[ ${iALL} = 1 ]] || [[ ${iPics} = 1 ]]; then
        echo " "
        echo "Preparing pictures for backup..."
        if [[ ${iDebug} = 0 ]]; then
            echo "     - tarring files into one..."
            tar -cvf /Users/username/0ffsite/MyPictures.tar /Users/username/Pictures/*
            echo "     - zipping file with encryption..."
            zip -v -P "super-secret-password" /Users/username/0ffsite/MyPictures.zip /Users/username/0ffsite/MyPictures.tar
            echo "     - removing unencrypted tar file..."
            rm -f /Users/username/0ffsite/MyPictures.tar
        fi
    fi


    # -------------------- COPY DATA TO NAS --------------------

    # Mount the backup drive
    if [[ ${iDebug} = 0 ]]; then
        mkdir /Volumes/Data
        mount_afp -i afp://username:${cPass}@nas.home/Data /Volumes/Data
    fi

    # ---------- COPY DOCUMENTS ----------
    if [[ ${iALL} = 1 ]] || [[ ${iPics} = 1 ]]; then
        echo " "
        echo "Copying pictures to NAS backup..."
        if [[ ${iDebug} = 0 ]]; then
            echo "     - copying zip file to NAS..."
            cp /Users/username/0ffsite/MyPictures.zip /Volumes/Data/Backup
        fi
    fi

    # Unmount the backup drive
    if [[ ${iDebug} = 0 ]]; then
        umount /Volumes/Data
    fi
fi

exit 0

Note that I omitted music from my plan.  Music is too much data, and it’s easily recovered from the CD’s and/or re-downloaded from the sales provider.  Yes, re-ripping 600 CD’s would take some time, but it can be done.  You could back music up, just note that the internet location will need to be sized accordingly.

Home video’s were also omitted from my plan.  I haven’t determined the best offsite solution for these, primarily due to size of the data.  In all likelihood it will be a USB drive stored offsite.

Taking Action

Now, I employ two backup types described in the Backup Planning section above.

First, once a month I perform a USB drive backup.  This drive is stored in a safe, in part of the house away from the computer.  This provides a second copy of ALL my data.

Second, twice per month, I run my backup script for the critical data.  This feeds the encrypted data onto the NAS drive.  My NAS drive has 2 hard drives in a RAID 1 (mirrored) configuration.  This effectively gives me a third and fourth copy of my critical data.

At this point the data is still susceptible to the physical loss scenarios.  To solve that problem, at least for my critical data, I used a feature on my NAS drive to push the data to Dropbox.  I encrypted the data because it is going offsite.  Yes, Dropbox keeps your data encrypted, but their servers are outside of my control.  So if someone gets to the files either internally at Dropbox or externally by hacking, the files are still useless to them.  Now, should I lose access to all physical copies, once the computer is replaced, I can still pull the data down from Dropbox.  Granted the non critical data is lost, but life goes on.

My next post will detail how I setup the NAS Dropbox feed.

(Edit): The follow up post is Automated Offsite Backup.