rdup(8) - Linux man page

Name

Synopsis

Description

The FILELIST is a internal list rdup writes to, to keep track of which files are in a backup. If you don't want this (i.e. make a full backup), use /dev/null here. This file is needed to calculate which files are to removed from a backup.

The DIRS/FILES can be specified multiple times. These are the directories and files you want to backup.

If the -N timestamp option is not given, all paths founds are printed. Only when a -N timestamp file is given, times can be compared and an incremental output can be generated.

rdup prints a filelist to standard output. Subsequent programs in a pipe line can be used to actually implement to backup scheme. If FILELIST is empty or non existant all files in DIR are dumped. This is the same as a null dump. After a run a new FILELIST is written. No warning is given when FILELIST is an existing file, it just gets overwritten by rdup. New runs will print out only those files that have actually changed or are removed since the last run, thereby making incremental backups possible.

Files are checked for changes by comparing the m_time (modification time), if this time is NEWER than the m_time of FILELIST the pathname is printed to standard output. When files are removed they are also printed to standard output, but they are prefixed with a '-'. See FORMAT below. The default format rdup uses is: "%p%T %b %m %u %g %l %s %n\n"

Note, that rdup also supports hashing of files, this makes it possible to check the local hash with the hash of the backed up file.

All errors are written to standard error. If the directory or file does not exist, they are skipped and a warning is emitted.

The general idea is to be very unix like and create a bunch of simple programs which each do a their specific thing very well. With rdup and a small shell script (50 lines) one can implement encrypted and compressed backups.

Backup Policy

Options

-F format

Specify a printf-style format to use. See FORMAT below.

-N timestamp

use the m_time of file timestamp as the timestamp to decide what to include in the incremental backup list. If timestamp does not exist of full dump is performed. rdup will create/touch timestamp after it has printed the file list. This means if something goes wrong, you still have the original timestamp.

-E file

The file named 'file' contains a list of extended POSIX regular expressions, one per line, that rdup will use to exclude names. A '#' at the start of the line can be used to signal a comment. Empty lines are discarded. The -0 option also affects the format of this file.

If a directory is excluded, rdup won't descend in that directory, so all files in that directory are also excluded.

The directories leading up to the directory to be backed up cannot be excluded. If you use a command line like:

rdup /dev/null /home/miekg/bin

The directories '/home', '/home/miekg', '/home/miekg/bin' are always printed.

If you want to exclude the file '/home/miekg/blaat' you need to add the following regular expression: '/home/miekg/blaat'.

If you want to exclude all .mozilla/cache directories of all users you can use '/home/.*/.mozilla/cache/.*'. This doesn't exclude the directory itself.

Note that rdup does not print directories with a trailing slash.

-V

Print rdup's version.

-h

Give an overview of the options.

-n

Don't honor .nobackup files. Normally if such files are found the directory is not printed to standard output, now it is.

-a

Read the uid and gid information from the file's extended user attributes: 'r_uid' and 'r_gid'. This option only works if rdup was built with xattr support. Currently this is supported and tested on Linux and Solaris 10.

This option is usefull when restoring a remote backup. Normally these backups are transferred by a non-root user. This user will not be able to correctly set all uid/gid information. As a workaround rdup's utilities will set the extended attributes for the file to store the original uid/gid. With this option rdup will re-insert these original uid/gid in its output.

-c

Print the files' contents to standard output. This sets the FORMAT string to: "%p%T %b %u %g %l %s\n%n%C";

Note this creates a race condition: if a file changes in size after rdup has crawled it and before it has been sent to the remote server, the backup will fail. rdup will check for file size changes, but this doesn't remove the race condition.

This option is used when streaming your backup to a remote machine. Note that the remote machine must have rdup installed.

-r

Only print removed files; entries that start with a '-'. This option unsets -m.

-m

Only print modified/new files; entries that start with a '+'. This option unsets -r.

-l

Local backups, do not check for file size changes.

-v

Be more verbose. When used once, processed .nobackup files will be printed to standard error. When used twice each path will also be printed to standard error. This is usefull in case of a remote backup (-c) where the normal output is not seen.

-s size

Only output files smaller then size bytes. This can be used to limit the amount of data to be transferred when doing a remote backup. This option only applies to files and symbolic links, not to directories.

-0

Delimit filelist with NULL's instead of a newline. Use '\0' in the format string to change rdup's output.

-x

Stay on the local filesystem.

Backups

rm -f timestamp && rdup -N timestamp LIST DIR

All full-dump filelist is printed to standard output. And with:

rdup -N timestamp LIST DIR

An incremental dump filelist is printed. The file timestamp is used to save the exact time of rdup's run. The file LIST is used to calculate the correct incremental dump list, this is needed for files that are removed, or have a different type.

Format

The following escape sequences are understood by rdup:

'p': '+' if file is new/modified, '-' if removed
'b': permission bits from lstat(2)
'm;: the file mode bits, st_mode from lstat(2)
'u': uid
'g': gid
'l': path name length
's': file size (zero if directory)
'n': path
't': time of modification (seconds from epoch)
'H': the SHA1 hash of the file, all "0" for directories and
symlinks
'T': file type: -, l or d (normail file, symlink or directory)
'C': the content of the file/link (none for directories)

To delimit the outout of rdup with NULLs you can use '\0' in the format string.

Filelist

MODE PATH_SIZE PATH

Where MODE is the st_mode from stat(2), PATH_SIZE is the length of PATH. Where PATH is the path of the file. A typical example is:

33216 8 bin/rdup

Output Format

+|-TYPE BITS UID GID PATH_SIZE FILE_SIZE PATH

Where:

o +|-

plus or minus, indicating whether PATH should added or removed.

the type of the see %T in FORMAT.

o BITS

the permission of the file, this is a subset of the st_mode from lstat(2).

o UID

the numerical user id of PATH. Note that if the first character of the line is '-' (i.e. remove) the UID will be zero.

o GID

the numerical group id of PATH. Note that if the first character of the line is '-' (i.e. remove) the GID will be zero.

o PATH_SIZE

the size of PATH. Normally this is not needed, but when PATH contains spaces this can come in handy.

o FILE_SIZE

the size of file pointed to by PATH. Note that if the first character of the line is '-' (.ie. remove) the SIZE will be zero. For directories this size will always be zero.

o PATH

the pathname

A typical example look like this: +- 755 1000 1000 8 11288 bin/rdup

This example tells us, that it should be backed up, it has a user id and group id of 1000, the length of the path is 8 bytes, the size of the file it 11288 and it has "bin/rdup" as a path.

Directories are always printed, as the modification time of directories is not compared.

OUTPUT FORMAT WITH -c

+|-TYPE BITS UID GID PATH_SIZE FILE_SIZE\n
PATH FILE_CONTENTS

This makes it possible possible for a remote shell script to receive the actual file and make a backup.

All field are identical as described in OUTPUT FORMAT, but there is one extra field and the following cavaets.

o FILE_CONTENTS

The entire file is concatenated to standard output.

Cavaets

o directories:

the FILE_SIZE is zero and no content is printed. Thus:

+d 755 1000 1000 11 0\n
/home/miekg

o regular files:

the FILE_SIZE is used as-is and the entire file contents is printed to standard output. Thus:

+- 644 1000 1000 32 6\n
/home/miekg/svn/rdup/trunk/aaa/ahello

o symbolic links:

the FILE_SIZE is set to the size of the target name and instead of printing the file contents the target name is printed. Thus:

+l 777 1000 1000 34 1\n
/home/miekg/svn/rdup/trunk/aaa/defb

Utilities

The more important scripts are:

rdup-mirror

the main backup script that implements a mirroring backup. It also used for remote backups (with the -c switch).

rdup-snap

Create a (hard link) snapshot. Hard links saves space so you can keep a very long record of previous backups. This is wrapper around 'rdup-snap' and 'rdup'. Also supports the -c switch.

rdup-gzip

acts as a compression filter.

rdup-crypt

acts as a encryption filter.

rdup-dump

a wrapper script around 'rdup-mirror' and 'rdup' that creates a backup.

rdup-snapshot

a wrapper script around 'rdup-snap' and 'rdup' that creates a hardlinked backup.

rdup-restore

a wrapper scripts around 'rdup', 'rdup-mirror' to ease restoring.

Examples

Backups

full dump of /home

full gzipped dump of /home

-

c -b /vol/backup

full remote dump of /home

miekg@example.com rdup-mirror -c -b /vol/backup

The -C compresses the stream.

full encrypted remote dump of /home

full encrypted, gzipped remote dump of /home

rdup -c /tmp/HOMELIST ~ | rdup-crypt KEYFILE | rdup-gzip | ssh miekg@example.com rdup-mirror -c -b /vol/backup

Restores

restore of /home to /tmp/restore

-

r -b /tmp/restore

Usefull External Tools

o cstream

a tool the implement bandwidth shaping (for remote backups).

o dirsplit

split a directory into DVD sized volumes for easy burning.

Exit Codes

Files

Author

Reporting Bugs

Bugs

Hard links are handled as normal files.

See Also

Copyright

Licensed under the GPL. See the file LICENSE in the source distribution of rdup.

Referenced By