rsync

rsync

Overview

The rsync utility is used to transfer files from one computer to another across a network. Some of the more salient features of rsync include the ability to:

Transfer only files that are different.
Transfer only the differences in those files.
Compress data as it transfers it.
Preserve file modes, owners, and groups.
Copy devices and links.
Use ssh for network transport.
Do everything it does without needing to be setuid.
Accept or reject files according to parameter files.
Pipeline transfers to avoid network lockstep delays.

At setgetweb.com, files on the main server are copied to a backup server every night at 4AM, using a script called rsync.sh, invoked from the backup server, which contains:
rsync -avuzr -e ssh map@192.168.0.190:/usr/local/apache/htdocs/tech/* /var/www/tech
rsync -avuzr -e ssh map@192.168.0.190:/usr/local/apache/htdocs/wendy/* /var/www/wendy
rsync -avuzr -e ssh map@192.168.0.190:/usr/local/apache/cgi-bin/* /var/www/cgi-bin
rsync -avuzr -e ssh map@192.168.0.190:/usr/local/bin/* /usr/local/bin

To avoid having to put in a password, authentication with keys was set up between the local server and the remove server.

Other Examples

To retrieve a set of files from a remote computer to a local computer, one could run something like:

rsync -ravz -e ssh server:/path/directory/*files* /localpath

To send a set of files from a local computer to a remote computer, one could run something like:
rsync -ravz -e ssh /localpath/*files* server:/path/directory

Backups and Mirrors

One use for rsync is to back up directories whose master copy is always on one machine. In that case, you can set up a cron entry to periodically use rsync to synchronize e master copy with another copy on some other machine. The cron entry can be on either the machine with the master copy or the machine with the backup copy, since rsync can synchronize in either direction. Such backup transfers will be more efficient than the old traditional UNIX method of using dump, although the rsync backup method does require keeping a complete copy on the backup machine. Disks are cheap these days, so that's often not a big problem.
Similarly, FTP or HTTP servers that serve as mirrors of other servers can be synchronized via rsync called from cron.

Where to Get It

The standard FTP repository for rsync is: ftp://samba.anu.edu.au/pub/rsync/.
Mirrors are currently available at:

ftp://sunsite.auc.dk/pub/unix/rsync
ftp://ftp.sunet.se/pub/unix/admin/rsync
ftp://ftp.fu-berlin.de/pub/unix/network/rsync/

There is a web page, including an interface to a bug tracking system: <URL:http://samba.anu.edu.au/cgi-bin/rsync>.
You can also send bug reports by electronic mail to rsync-bugs@samba.anu.edu.au.

Options

Option Description

-v --verbose Increase verbosity

-q --quiet Decrease verbosity

-c --checksum Always checksum

-a --archive Archive mode

-r --recursive Recurse into directories

-R --relative Use relative path names

-b --backup Make backups (default ~ suffix)

--suffix=suffix Override backup suffix

-u --update Update only (don't overwrite newer files)

-l --links Preserve soft links

-L --copy-links Treat soft links like regular files

--copy-unsafe-links Copy links outside the source tree

--safe-links Ignore links outside the destination tree

-H --hard-links Preserve hard links

-p --perms Preserve permissions

-o --owner Preserve owner (root only)

-g --group Preserve group

-D --devices Preserve devices (root only)

-t --times Preserve times

-S --sparse Handle sparse files efficiently

-n --dry-run Show what would have been transferred

-W --whole-file Copy whole files. No incremental checks

-x --one-file-system Don't cross filesystem boundaries

-B --block-size=size Checksum blocking size (default 700)

-e --rsh=command Specify rsh replacement

--rsync-path=path Specify path to rsync on the remote machine

-C --cvs-exclude Auto ignore files in the same way CVS does

--delete Delete files that don't exist on the sending side

--delete-excluded Also delete excluded files on the receiving side

--partial Keep partially transferred files

--force Force deletion of directories even if not empty

--numeric-ids Don't map uid/gid values by user/group name

--timeout=time Set IO timeout in seconds

-I --ignore-times Don't exclude files that match length and time

--size-only Only use file size when determining if a file should be transferred

-T --temp-dir=directory Create temporary files in directory directory

--compare-dest=directory Also compare destination files relative to directory

-z --compress Compress file data

--exclude=pattern Exclude files matching pattern

--exclude-from=file Exclude patterns listed in file

--include=pattern Don't exclude files matching pattern

--include-from=file Don't exclude patterns listed in file

--version Print version number

--daemon Run as a rsync daemon

--config=file Specify alternate rsyncd.conf file

--port=port Specify alternate rsyncd port number

--stats Give some file transfer stats

--progress Show progress during transfer

--log-format=format Log file transfers using specified format

--password-file=file Get password from file

-h --help Show this help screen

Setup

Rsync normally uses rsh or ssh for communication. It does not need to be setuid and requires no special privileges for installation. You must, however, have a working rsh or ssh system. Using ssh is recommended for its security features.
Alternatively, rsync can run in `daemon' mode, listening on a socket. This is generally used for public file distribution, although authentication and access control are available.
To install rsync, first run the "configure" script. This will create a Makefile and config.h appropriate for your system. Then type "make".
Note that on some systems you will have to force configure not to use gcc because gcc may not support some features (such as 64 bit file offsets) that your system may support. Set the environment variable CC to the name of your native compiler before running configure in this case.
Once built put a copy of rsync in your search path on the local and remote systems (or use "make install").

rsync servers

rsync can also talk to "rsync servers" which can provide anonymous or authenticated rsync. See the rsyncd.conf(5) man page for details on how to setup a rsync server. See the rsync(1) man page for info on how to connect to a rsync server.

Option	Description
-v --verbose	Increase verbosity
-q --quiet	Decrease verbosity
-c --checksum	Always checksum
-a --archive	Archive mode
-r --recursive	Recurse into directories
-R --relative	Use relative path names
-b --backup	Make backups (default ~ suffix)
--suffix=suffix	Override backup suffix
-u --update	Update only (don't overwrite newer files)
-l --links	Preserve soft links
-L --copy-links	Treat soft links like regular files
--copy-unsafe-links	Copy links outside the source tree
--safe-links	Ignore links outside the destination tree
-H --hard-links	Preserve hard links
-p --perms	Preserve permissions
-o --owner	Preserve owner (root only)
-g --group	Preserve group
-D --devices	Preserve devices (root only)
-t --times	Preserve times
-S --sparse	Handle sparse files efficiently
-n --dry-run	Show what would have been transferred
-W --whole-file	Copy whole files. No incremental checks
-x --one-file-system	Don't cross filesystem boundaries
-B --block-size=size	Checksum blocking size (default 700)
-e --rsh=command	Specify rsh replacement
--rsync-path=path	Specify path to rsync on the remote machine
-C --cvs-exclude	Auto ignore files in the same way CVS does
--delete	Delete files that don't exist on the sending side
--delete-excluded	Also delete excluded files on the receiving side
--partial	Keep partially transferred files
--force	Force deletion of directories even if not empty
--numeric-ids	Don't map uid/gid values by user/group name
--timeout=time	Set IO timeout in seconds
-I --ignore-times	Don't exclude files that match length and time
--size-only	Only use file size when determining if a file should be transferred
-T --temp-dir=directory	Create temporary files in directory directory
--compare-dest=directory	Also compare destination files relative to directory
-z --compress	Compress file data
--exclude=pattern	Exclude files matching pattern
--exclude-from=file	Exclude patterns listed in file
--include=pattern	Don't exclude files matching pattern
--include-from=file	Don't exclude patterns listed in file
--version	Print version number
--daemon	Run as a rsync daemon
--config=file	Specify alternate rsyncd.conf file
--port=port	Specify alternate rsyncd port number
--stats	Give some file transfer stats
--progress	Show progress during transfer
--log-format=format	Log file transfers using specified format
--password-file=file	Get password from file
-h --help	Show this help screen