This is a read-only archive. Find the latest Linux articles, documentation, and answers at the new Linux.com!

Linux.com

Feature

Making secure remote backups with Rsync

By Brice Burgess on November 04, 2004 (8:00:00 AM)

Share    Print    Comments   

Backups are more important than ever these days, as our digital information collections expand. Many Linux users know rsync as a file transfer utility, but rsync can also be an efficient tool for automating remote backups of your Linux, Windows, and even Mac OS X systems.

In an earlier article, I explained how to use rsync to make local backups of a Linux system. Remote backups, where you store your backed up data on a separate machine, further promote data safety by separating information both physically and geographically.

First steps

To perform secure remote backups, you must have rsync and SSH installed on both your local and your target remote machine. Rsync can use SSH as a secure transport agent.

Make sure rsync is installed by opening a terminal session and typing rsync --version on each machine. You should see a message like rsync version 2.X.X protocol version X. If you receive "command not found" or a similar message, you'll need to download and install rsync. Use your GNU/Linux distribution's package management system to do this, or download and install the source from the rsync Web site. If you're running Microsoft Windows I recommend installing cwRsync. Mac OS X comes with rsync, but if you want to try a different version, check out RsyncX.

SSH is likely to already be installed on your Mac OS X and GNU/Linux systems, while the Windows port of rsync, cwRsync, includes the key SSH programs. I'm going to assume you're running Linux or OS X on the remote machine where the backup is to be stored. Make sure your remote machine has Secure Shell Daemon (sshd) running and that the users of both machines have proper permissions to execute a backup.

To ensure that sshd is running on a remote machine, enter a terminal session and type ssh [user]@[remote.machines.address]. If all is well, you should be asked for the user's password and allowed to log in and check permissions. If the remote machine is the destination where the rsync backup will be stored, you'll want read and write permission to the destination directory.

Once you know all the necessary programs and permissions are in place, choose a directory with a few small files to use as a test backup. On the remote machine, create a destination directory to hold your backups:

rsync -avz -e ssh /some/small/directory/ remote_user@remotehost.com:/backup/destination/directory/

The trailing slash in the source directory causes rsync to copy only the contents of the source directory. Omitting the trailing slash causes rsync to copy both the directory name and its contents to the destination.

After rsync completes, you'll be left with a copy of the source files on the remote computer. Congratulations on your first remote backup! Now let's automate the process.

Automatic backups

The first step in automating remote backups is to remove any required user intervention -- namely requests for SSH passwords. To allow your systems to make an SSH connection without asking for a password, you must generate passphraseless keys. On the local machine, drop into the terminal and enter:

$ ssh-keygen -t dsa -b 2048 -f ~/rsync-key
Generating public/private dsa key pair.
Enter passphrase (empty for no passphrase): [press enter here]
Enter same passphrase again: [press enter here]
Your identification has been saved in /home/user/rsync-key.
Your public key has been saved in /home/user/rsync-key.pub.
The key fingerprint is:
8c:57:af:68:cd:b2:7c:aa:6d:d6:ee:0a:5a:a4:29:03 user@localhost

Now copy the public key to the remote machine using Secure Copy:

scp ~/rsync-key.pub user@remotehost:~

Finally, put the public key into the authorized_keys file on the remote host. SSH into the remote machine using ssh user@remotehost.com and execute:

mkdir ~/.ssh
chmod 700 ~/.ssh
mv ~/rsync-key.pub ~/.ssh/
cd ~/.ssh/
touch authorized_keys
chmod 600 authorized_keys
cat rsync-key.pub >> authorized_keys

You should now be able to SSH into the remote machine without being asked for a password. Give it a try by closing your previous SSH session and creating another one by typing ssh -i ~/rsync-key user@remotehost.

These entries with no passwords can originate from any host and execute anything. You can add additional security by limiting what the SSH connection can do via the authorized_keys file. I don't recommend employing any additional security until after your first backup in order to limit the troubleshooting process, but once you've completed that successfully, you can employ additional security by using SSH to connect to the remote machine and editing your ~/.ssh/authorized_keys file. It should look similar to:

ssh-dss AAAAB3NzaC1yc2EAAAABIwAAAIEAyNChQxw/+Da....= user@remotehost.com

To limit where connections are coming from, prefix the key with from="ip.address". To limit what command is executed, prefix the key with command="/path/to/validating/script". As an example, your secured authorized_keys file might look like:

from="192.168.0.1", command="/home/user/validate-rsync.sh" ssh-dss AAAAB3NzaC1yc2EAAAABIwAAAIEAyNChQxw/+Da....= user@remotehost.com

Finally, put something like the following in your validate-rsync.sh file:

#!/bin/sh
case "$SSH_ORIGINAL_COMMAND" in
*\&*)
echo "Rejected"
;;
*\;*)
echo "Rejected"
;;
rsync\ --server*)
$SSH_ORIGINAL_COMMAND
;;
*)
echo "Rejected"
;;
esac

Make it executable by typing: chmod +x ~/validate-rsync.sh. This will check to see if the ssh session is being used to execute an rsync backup. If it is being used for anything else, the session will be rejected and closed.

Rsync should now complete without prompting for a password if we modify the test backup we used earlier, telling it to use keys. Try it out by typing:

rsync -avz -e "ssh -i ~/rsync-key" /some/small/directory/ remote_user@remotehost.com:/backup/destination/directory/

If you're having problems, please ensure you have proper permissions to read from the source (/some/small/directory) and to write to the target (remotehost:/backup/destination). Also make sure passphraseless ssh sessions can be established between the two hosts, ~/rsync-key exists on the machine to be backed up, and that rsync is intalled on both machines.

Next: Making a real backup
 

Share    Print    Comments   

Comments

on Making secure remote backups with Rsync

Note: Comments are owned by the poster. We are not responsible for their content.

validate-rsync.sh needs password?

Posted by: Anonymous Coward on November 19, 2004 06:43 PM
Your instructions work very well. Thank you for that. However, when I include command="../validate-rsync.sh" in authorized_keys I am prompted for a password.

#

Re:validate-rsync.sh needs password?

Posted by: Anonymous Coward on February 21, 2005 08:17 PM
You can solve this problem by removing the blank space between from and command. Only the comma is allowed as separator.

Another thing I ran into was that the rotational backup is not working with the validate-rsync.sh. Unfortunately I have no experience in shell scripts, so I would be glad if someone could fix this.

#

Re:validate-rsync.sh needs password?

Posted by: Administrator on December 01, 2004 11:50 AM
For the security purpose can't we have all the source file and module restrictions in the rsyncd.conf file on the server side. This will make sure that no files are written or pulled out by the wrong person. Correct me if i am wrong.

#

great article, very useful script, keep it up!

Posted by: Anonymous Coward on January 17, 2006 02:02 AM
subject says it all<nobr> <wbr></nobr>;).

#

indentity file issues

Posted by: Anonymous Coward on September 10, 2006 08:17 AM
After following the directions I'm able to issue this command and reach my remote server w/out a password.
<tt>ssh -i ~/.ssh/rsync-key user@remotehost.com</tt>
but I get an error if I try:
<tt>rsync -avz -e "ssh -i ~/.ssh/rsync-key"<nobr> <wbr></nobr>/local/dir/ user@remotehost.com:/remote/dir/</tt>
The error is:
<tt>Warning: Identity file ~/.ssh/rsync-key not accessible: No such file or directory.</tt>
Why would the first work but not the second?

#

Re:indentity file issues

Posted by: Anonymous Coward on November 01, 2006 03:49 AM
I too am having this issue. did you ever find your answer? Anyone else able to help?

It works fine with the ssh connection command but fails in the rsync connection attempt.

#

Re:indentity file issues

Posted by: Anonymous Coward on December 10, 2006 08:28 AM
The reason is the ~ in the string. On the position it want be expanded by the shell. Replace it with $HOME or direct with you home dir name.

#

Re:indentity file issues

Posted by: Anonymous Coward on November 15, 2006 01:42 PM
Hi
my solution was
this<nobr> <wbr></nobr>:rsync -avz -e "ssh -i<nobr> <wbr></nobr>/root/rsync-key"<nobr> <wbr></nobr>/local/dir/ user@remotehost.com:/remote/dir/

if it's not working try
rsync -avz -e "ssh -v -i<nobr> <wbr></nobr>/root/rsync-key"<nobr> <wbr></nobr>/local/dir/ user@remotehost.com:/remote/dir/

I hop I help you

#

had to modify your script to get it working

Posted by: Anonymous Coward on October 02, 2006 12:26 PM
I had an error with your multi_rbackup.sh script and had to modify it slightly.

The fix was simply to add a trailing slash to this line:

ssh $RUSER@$RMACHINE "cp -al $NEWEST_BACKUP. $RTARGET/$BACKUP_DATE/"

Without the trailing slash, this command would create a time-stamped backup directory based on $NEWEST_BACKUP _inside_ the new $BACKUP_DATE directory.

Otherthan that this script works perfectly.

Thanks for sharing!

#

Do not recommend passphrase-less SSH keys, please!

Posted by: Administrator on November 10, 2004 11:09 AM
A fine article, but telling people to use no passphrase on their SSH keys is unwise at best. It certainly does not make a "secure" backup. A good passphrase on a key (SSH, GPG, whatnot) makes all the difference.


Using tools like <A HREF="http://www.gentoo.org/proj/en/keychain/index.xml" title="gentoo.org">keychain</a gentoo.org> or some other manipulation of ssh-agent on the backup repository and pulling the data negates the temptation to use such an insecure method.

#

ravindra mudumby--Rsync

Posted by: Administrator on August 17, 2005 11:23 AM
Want to automate your backups? Want to do it securely? Want it to be efficient and fast? Then Rsync backups over SSH using public/private keys are for you.

The following script will setup a nightly rsync backup of a directory you specify on your host to another Unix server (whatever you use as remote_host). The remote host must be running SSHd.

remote_host=ahostname.com
directory_to_backup=/a/directory/to/backup

{
cd<nobr> <wbr></nobr>/root
# create a private key with no pass phrase.
# You want to make sure no one ever gets access to this file
# (else they'll be able to log into your remote host)
ssh-keygen -f backup.private.key -t rsa -C "backupkey" -N ""
# pop the public key into a variable
public_key=`cat backup.private.key.pub`
# add a backup user on the remote machine, install the public key
ssh $remote_host "adduser backupuser;mkdir ~backupuser/backups; mkdir ~backupuser/.ssh;echo \"$public_key\" > ~backupuser/.ssh/authorized_keys"
# back on the localhost setup a job that will do the backup

echo "
#!/bin/bash
# invoked by<nobr> <wbr></nobr>/etc/cron.daily/rsyncbackupcronjob.sh
ssh-add<nobr> <wbr></nobr>/root/backup.private.key

# copy a directory to the backup server
rsync --delete --compress --archive --rsh=ssh $directory_to_backup backupuser@$remote_host:backups/
" ><nobr> <wbr></nobr>/root/rsyncbackup.sh
chmod +x<nobr> <wbr></nobr>/root/rsyncbackup.sh

# and then you need a job to lauch the previous one. This job will execute nightly via cron
echo '#!/bin/bash' ><nobr> <wbr></nobr>/etc/cron.daily/rsyncbackupcronjob.sh
echo "ssh-agent<nobr> <wbr></nobr>/root/rsyncbackup.sh" >><nobr> <wbr></nobr>/etc/cron.daily/rsyncbackupcronjob.sh
chmod +x<nobr> <wbr></nobr>/etc/cron.daily/rsyncbackupcronjob.sh
}

Edit<nobr> <wbr></nobr>/root/rsyncbackup.sh if you want to change the files/directories you need backed up

#

rsync.net

Posted by: Administrator on June 08, 2006 05:11 AM
The above configuration (including the use of ssh keys to automate) is exactly what I use to backup my important personal data.

The difference is that the server I back them up to does not belong to me - it is an rsync.net server, and the data gets replicated to both California and Colorado.

I highly recommend this service. Read their corporate philosophy and privacy/warrant policy if you like pleasant surprises...

#

Some Sample Scripts

Posted by: Administrator on June 08, 2006 07:49 AM

Based in part on the above, we've put together a series of how-tos, including sample scripts, for using rsync on Linux, Windows & Mac OS X. Might be of some help to folks.

<a href="http://www.exavault.com/rsync_setup_unix_linux_bsd.shtml" title="exavault.com">rsync on Linux/Unix/BSD</a exavault.com>

<a href="http://www.exavault.com/rsync_setup_mac_osx.shtml" title="exavault.com">rsync on Mac OS X</a exavault.com>

<a href="http://www.exavault.com/rsync_setup_windows.shtml" title="exavault.com">rsync on Windows (All Versions)</a exavault.com>

These are for our online backup service, <a href="http://www.exavault.com/" title="exavault.com">ExaVault</a exavault.com>, but if you want to use them for your own purposes you are welcome to. You just need to change the name of the server (username.exavault.com) to your own server, and you need to add your keys to your authorized_keys file manually, instead of using our 'addkeys'.

#

Making secure remote backups with Rsync

Posted by: Anonymous [ip: 124.43.251.154] on August 13, 2007 06:33 AM
This is cool! I've got it using without the additional security part. I found other articles related to this but this is the only article has discribed about the environmet variable passing to start SSH -e "ssh -i ~/rsync-key". Thanks :-)
W.M.T. Manjula

#

Making secure remote backups with Rsync

Posted by: Anonymous [ip: 217.128.184.142] on February 26, 2008 11:12 AM
Very good paper! Thanks a lot. Work very well

#

This story has been archived. Comments can no longer be posted.



 
Tableless layout Validate XHTML 1.0 Strict Validate CSS Powered by Xaraya