Linux/rsync

rsync
rsync

Description
"rsync is a software application for Unix systems which synchronizes files and directories from one location to another while minimizing data transfer using delta encoding when appropriate. An important feature of rsync not found in most similar programs/protocols is that the mirroring takes place with only one transmission in each direction. rsync can copy or display directory contents and copy files, optionally using compression and recursion.

In daemon mode, rsync listens to the default TCP port of 873, serving files in the native rsync protocol or via a remote shell such as RSH or SSH. In the latter case, the rsync client executable must be installed on both the local and the remote host.

Released under the GNU General Public License, rsync is free software." (rsync - Wikipedia)

Usage
rsync (Use the --dry-run option for testing) rsync -P rsync://rsync.server.com/path/to/file file	Only get diffs. Do multiple times for troublesome downloads rsync --bwlimit=1000 fromfile tofile	Locally copy with rate limit. It's like nice for I/O rsync -az -e ssh --delete ~/public_html/ remote.com:'~/public_html'	Mirror web site (using compression and encryption) rsync -auz -e ssh remote:/dir/. && rsync -auz -e ssh. remote:/dir/	Synchronize current directory with remote one

SSH
Standard format: rsync -avz -e ssh remoteuser@remotehost:/remote/dir /this/dir/

With ssh keys: rsync -avz -e "ssh -i /home/thisuser/cron/thishost-rsync-key" remoteuser@remotehost:/remote/dir /this/dir/

In reverse: rsync -avz -e "ssh -p 9999" linux/ trogdor@oeey.com:/www/linux-tested-stage/

References:
 * Using Rsync and SSH

Compile
The --no-motd option is included with the latest source.

Source: http://samba.anu.edu.au/rsync/download.html

Compile: cd ~/src wget http://samba.anu.edu.au/ftp/rsync/rsync-3.0.7.tar.gz tar -zvxf rsync-3.0.7.tar.gz cd rsync-3.0.7 ./configure --prefix=/opt/rsync make sudo make install
 * 1) git clone git://git.samba.org/rsync.git

Copy root
rsync -axv --progress / /newpu/ken/kesx/
 * 1) -x says not to cross fs boundaries

Rsync Hardlink Copy Backup
SOURCE=[SOURCE_PATH] MAX=10 rm -rf $MAX for i in $( seq $MAX -1 2 ) ; do mv $(( i - 1 )) $i done cp -al 'current' 1 # this is the key here rsync -avrt --delete $SOURCE 'current'

Readme
Full System Backup with rsync - ArchWiki - https://wiki.archlinux.org/index.php/Full_System_Backup_with_rsync
 * This rsync script allows creating a full backup copy across filesystems. It is setup so that the copy includes intact booting capabilities, optionally excluding selected files.
 * The approach has benefits over omitting system files by just copying personal data; if the system becomes corrupted in the main partition, overcoming the problem means booting into the backup as opposed to identifying and reinstalling affected programs.

How to Backup Linux? 15 rsync Command Examples - http://www.thegeekstuff.com/2010/09/rsync-command-examples/

Exclude
--exclude 'dir1'

rsync -avz --exclude 'dir1' source/ destination/ rsync -avz --exclude 'dir1' --exclude 'dir2' source/ destination/

References:
 * 6 rsync Examples to Exclude Multiple Files and Directories using exclude-from - http://www.thegeekstuff.com/2011/01/rsync-exclude-files-and-folders/

including only certain extensions
What you need is indeed a combination of --include and --exclude arguments. For example the following will replicate only mp3 files in a directory named source, or any of its subdirectories, to a directory named target:

rsync -a --include '*/' --include '*.mp3' --exclude '*' source/ target/

This will (1) include any directory or subdirectory, whatever its contents, (2) include all files with an mp3 extension, and (3) exclude all other files. Any number of other file types can be included by adding their one include pattern for each before the --exclude, e.g. --include '*.bmp'.

rsync server
Runs on TCP port 873 by default.

/etc/rsyncd.conf: motd file = /etc/rsyncd.motd log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock


 * 1) Default port: 873
 * 2) port = 873

[files] path = /home/public_rsync comment = Rsync Files Server read only = yes list = yes timeout = 300

Run server: rsync --daemon

List rsync shares: rsync rsync://localhost

Sync: rsync -avrt rsync://localhost/files.

Larger Sync Example: rsync --verbose --progress --stats --compress --rsh=/usr/local/bin/ssh \ --recursive --times --perms --links --delete \ --exclude "*bak" --exclude "*~" \ /www/* webserver:simple_path_name

xinetd
/etc/xinetd.d/rsync: service rsync {       #disable = yes disable = no       flags           = IPv6 socket_type    = stream wait           = no        user            = root server         = /usr/bin/rsync server_args    = --daemon log_on_failure += USERID }
 * 1) default: off
 * 2) description: The rsync server is a good addition to an ftp server, as it \
 * 3)       allows crc checksumming etc.

service xinetd restart

yum install rsync xinetd

References:
 * Everything Linux - A Tutorial on Using Rsync - http://everythinglinux.org/rsync/
 * Set up an Rsync Daemon on your Linux server | Atlantic.Net Community - https://www.atlantic.net/community/howto/setup-rsync-daemon/