rdiff-backup provides a simple easy means for providing remote backups. This is intended to be a simple, easy to follow set of instructions for implementing off site backups. There are some great advantage to using rdiff-backup for this, here are some of my favorite:
Files are stored on the remote site as normal files, not in some proprietary format making looking for something easy using standard *nix tools such as find, grep and locate.
A full history of previous revisions is kept so you can restore a copy of that file you accidently changed from several days ago, before the change was made.
Rsync's algorithms are used so only the changes from the previous day are transfered, not a full image.
Previous revisions of files are stored as differences from current version saving space where backups are stored.
Can use ssh for transferring files making the transmissions secure from packet sniffing.
Can use RSA or DSA public key cryptography for authenticating to the remote host so automated backups are still secure without a clear text password in a config file.
If you have any corrections or suggestions for this document, either technical or regarding readability, please do drop me a line at firstname.lastname@example.org so your suggestions can help other people.
› Conventions Used
In this document program output will look like this with any responses entered will be shown in bold. For hostnames source will be used for the server being backed up and destination for the host storing the backups.
This document assumes you already have rdiff-backup installed, if you need help with this, feel free to email me at the address above with any questions you may have. If there is enough demand, I will add instructions for getting it installed on the most popular distributions.
› Setting up SSH
This step gets ssh setup for secure unattended, password-less use. Public and private RSA keys will be generated. The private key is stored on the machine you are backing up too while the public key gets sent to your server you are backing up. These keys are used to authenticate a connection without passwords in a secure manner. Then we setup the remote end to authenticate against this key and allow only rdiff-backup to run.
Use the ssh-keygen program to generate a password-less RSA key pair by executing the following command. In this example /root is root's home directory. Adjust accordingly for your system.
destination# ssh-keygen -t rsa Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): /root/.ssh/id_rsa_backup Enter passphrase (empty for no passphrase): <ENTER> Enter same passphrase again: <ENTER> Your identification has been saved in /root/.ssh/id_rsa_backup. Your public key has been saved in /root/.ssh/id_rsa_backup.pub. The key fingerprint is: aa:63:1a:f8:0f:44:78:e9:d2:e1:25:40:51:2f:59:29 root@destination
Set this new private key as readable only by root for security. SSH will complain if it doesn't have secure file permissions.
destination# chmod go-r /root/.ssh/id_rsa_backup
Now we must transfer the public key to the source system.
destination# scp /root/.ssh/id_rsa_backup.pub source:/root/.ssh/id_rsa_backup.pub root@source's password: <enter root password> id_rsa_backup.pub 100% 224 0.0KB/s 00:00
Now setup the entry on the source system, on the source system open id_rsa_backup.pub in your favorite text editor. This should be one really long line and you want to add the following to the beginning of it. Note that this line has been significantly shortened here.
command="rdiff-backup --server" ssh-rsa AAAAB3Nz[....]iNM= root@destination
Add this new line to authorized_keys2. If there is no /root/.ssh/authorized_keys2 already this execute the following command:
source# mv /root/.ssh/id_rsa_backup.pub /root/.ssh/authorized_keys2
If this file already exists, use these commands instead to add your new line to it:
source# cat /root/.ssh/id_rsa_backup.pub >> /root/.ssh/authorized_keys2 source# rm /root/.ssh/id_rsa_backup.pub
Finally, set the permissions on this file to only readable by root:
source# chmod go-r /root/.ssh/authorized_keys2
Now we need to tell ssh to use our new private key and a few other settings, this will lessen the CPU load on both ends. Open in your favorite editor, creating if it doesn't already exist, the file /root/.ssh/config on the destination system. You want to add the following section to this file.
host source-backup hostname source identityfile /root/.ssh/id_rsa_backup compression yes cipher blowfish protocol 2
This causes ssh to use these settings when you ask it to connect to source-backup. The hostname specifies the real host to connect to. identityfile specifies the name of the private key to use when connecting.
Now you get to test out our settings for ssh.
destination# rdiff-backup --test-server source-backup::/ignored Testing server started by: ssh -C source-backup rdiff-backup --server Server OK
If your output doesn't match hopefully you will receive an error that make sense. The most important thing here is that ssh shouldn't ask you for your password. If it doesn't work out, check you filenames and permissions. If your still at a loss, feel free to send some output my way and we'll see if we can get it knocked out.
› Setting Up rdiff-backup
In this section we will setup a location for storing the backups, create an rdiff-backup config file, and set things up for automatic backup via a system cronjob.
You need to decide on a location to store the backups on destination. I store my backups in /backups/<hostname>. Once you decide where you want to put your backups. Create the path. For example, to create the location I use you would type:
destination# mkdir -p /backups/source
We need to specify the files to include and exclude when backing up with rdiff-backup. This can be done on the command line when it is run, but for an automated system it is much easier to create a file with this information. I use /etc/backup-source.conf but you may use something else if you prefer. It is important to note that this file is located on destination, not on source as you might expect.
- /dev - /proc - /tmp - /var/tmp - /usr/portage - /home/michael/dl
In this file a - at the start of a line excludes paths matching the contents of the line. To include a path, you just place the path on the line with no preceding symbols.
My example is for a Gentoo system, though the only Gentoo specific item is /usr/portage. /home/michael/dl is the location I download, extract and compile software and is easily replaceable, hence it's exclusion. The remainder of the entries you will most likely want to keep. The way we are setting stuff up, the root dir will be included, causing everything to be backed up except for the items listed in this file.
Finally we setup rdiff-backup to run daily from cron. You will need to locate the directory holding scripts for system cronjobs that run daily. In most cases this is /etc/cron.daily. In this directory you need to create a new shell script that looks like this:
#!/bin/sh export HOME=/root rdiff-backup --print-statistics \ --include-globbing-filelist /etc/backup-source.conf \ source-backup::/ /backup/source
You may choose what ever name you desire for this script. I use rdiff-backup.sh. This script will run rdiff-backup using the config file you created earlier. The --print-statistics line is optional and produces output that looks like this:
--------------[ Session statistics ]-------------- StartTime 1066622417.00 (Mon Oct 20 00:00:17 2003) EndTime 1066623159.32 (Mon Oct 20 00:12:39 2003) ElapsedTime 742.32 (12 minutes 22.32 seconds) SourceFiles 45652 SourceFileSize 429389316 (409 MB) MirrorFiles 44428 MirrorFileSize 414270694 (395 MB) NewFiles 1226 NewFileSize 15059120 (14.4 MB) DeletedFiles 2 DeletedFileSize 3501 (3.42 KB) ChangedFiles 119 ChangedSourceSize 1755983 (1.67 MB) ChangedMirrorSize 1692980 (1.61 MB) IncrementFiles 1349 IncrementFileSize 17377 (115 KB) TotalDestinationSizeChange 15235999 (14.5 MB) --------------------------------------------------
Lastly you may want to run it for the first time right now. The first run will be the longest as it will need to transfer everything that needs to be backed up.
A great way to do this and test out your script at the same time is by running it:
You should be now on your way to a happy backed up system. If you see any errors in this document, or think things could be clarified some where, please let me know!