Added manpages

I've moved documentation out of README.md into the man/*.txt files to
convert to manpages.

7 files changed, 323 insertions, 88 deletions

diff --git a/Makefile b/Makefile
new file mode 100644
index 0000000..6a0cb38
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,17 @@
+
+all: docs
+
+man/%.man: ./man/%.txt
+	[ -d man ] || mkdir man
+	txt2man -s 1 -t $(basename $(notdir $<)) $< | sed -e 's/\\\\fB/\\fB/' > $@
+
+docs: man/byteback-prune.man man/byteback-restore.man man/byteback-backup.man
+
+# To be written
+# man/byteback-snapshot.man man/byteback-setup-client.man man/byteback-setup-client-receive.man man/byteback-receive.man
+
+clean:
+	$(RM)  man/*.man
+
+.PHONY: clean docs all
+
diff --git a/README.md b/README.md
index 0ca540e..fb10d43 100644
--- a/README.md
+++ b/README.md
@@ -68,65 +68,13 @@ client to start and watch the backup.
 
 Configuring byteback-backup
 ---------------------------
-You can now set "byteback-backup"  on a daily cron job to start backing up the
-server on a regular basis.
 
-Without any further options this will copy every file from the root downwards.
-
-It currently excludes /tmp, /var/tmp, /var/cache/apt/archives, /swap.file and
-/var/backups/localhost which (on Bytemark systems) do not need to be part of
-any backup.  To specify which locations are excluded, add them to
-/etc/byteback/excludes, one per line.  The filesystems /dev, /proc, /run and
-/sys are always excluded.
-
-It is possible to configure a full rsync filter by creating the file
-/etc/byteback/rsync_filter, which is parsed to rsync via the --filter flag.
-Note that excludes on the command line take precedence, unless the filter
-starts with an exclamation mark, which resets everything.  If you do this,
-you'll need to specify /proc, /sys, etc manually.  See the rsync manpage for
-more information about filters.
-
-It is also possible to add extra rsync flags to the configuration.  The
-following flags are always set:
-
-  --archive --numeric-ids --delete-during --inplace --relative --timeout 43200
-
-If you wish to add to that (e.g. --xattrs --acl --hard-links) then simply add
-them to /etc/byteback/rsync_flags.  These extra flags get appended after the
-default flags above, so they take precedence.
-
-When the backup has completed successfully, the server will take a snapshot
-so that the client can't alter the backups, and then "prune" the backup 
-snapshots to ensure that the next backup is likely to run OK.
-
-If the backup is interrupted or dies unexpected, running "byteback backup" 
-will cause the backup to be resumed, with rsync saving the work of re-copying
-any files that hadn't changed.  By default this will happen automatically up to 
-5 times, with a 10 minute pause in between each attempt.
+This is now documented in the manapge for byteback-backup(1).
 
 Viewing and restoring backups
 -----------------------------
 
-Backups can be viewed on the server filesystem, although the permissions will
-be wrong.  The rsync "fake-super" flag is used to store the permissions in a
-user attribute list.  To view this list on the server, run 
-
-  getfattr -d  $filename
-
-This command is part of the "attr" package in Debian.
-
-To restore a file you may use the `byteback-restore` command, which allows you to find all files matching a pattern via a command-like:
-
-    byteback-restore --file /etc/passwd
-
-To actually restore the file, to the current directory, run:
-
-    byteback-restore --file /etc/passwd --revision=current
-
-Recursive restorations are supported too:
-
-    byteback-restore --file /srv --revision=current
-
+This is now documented in the manapge for byteback-restore(1).
 
 The trust model
 ---------------
@@ -151,43 +99,15 @@ over a long enough period of time:
 
 Pruning behaviour
 -----------------
-Unless you are backing up a very small amount of data, backups will always 
-need pruning, i.e. old backups must be deleted to make way for newer ones.
-
-There is a program on the server called byteback-prune which deals with this
-operation.  It deletes old backups until a certain amount of free space is
-achieved, which is currently fixed at 10% free.  This can be changed by setting
-the --maxpercent option.
-
-It can choose which backups to delete by one of two methods:
 
-1) the 'age' method simply deletes the oldest backup;
-
-2) the 'importance' method tries to retain a more spread-out backup pattern
-by "scoring" each backup according to how close it is to a set of "target 
-times".  These are:
-
- * midday today;
- * midday on the previous 6 days;
- * midday on the previous 4 Sundays;
- * midday on every 4th Sunday before that.
-
-So when you ask the pruner to run, the backup closest to the present time will
-be the last one to be deleted.  The backup closes to "1 day ago" will be the
-second-last, and so on.  We score every backup in this way until we end up with
-a "least important" snapshot to delete.
-
-The upshot of the second strategy should be that we retain closely-spaced
-daily backups, but as they get too numerous, we make sure that we are reluctant
-to delete our very oldest.
-
-The "4th Sunday" is calculated as every 4th Sunday in the Unix epoch, i.e.
-every fourth Sunday since Sunday 4th Jan 1970. 
+This is now documented in byteback-prune(1).
 
 Acknowledgements
 ----------------
-For maximum portability, I've included two libraries.  Thanks very much to
+For maximum portability, I've included three libraries.  Thanks very much to
 their authors:
 
-sys-filesystem by Daniel J. Berger: https://github.com/djberg96/sys-filesystem
-trollop by William Morgan: https://github.com/wjessop/trollop
+* sys-filesystem by Daniel J. Berger: https://github.com/djberg96/sys-filesystem
+* trollop by William Morgan: https://github.com/wjessop/trollop
+* ffi-xattr by Jari Bakken: https://github.com/jarib/ffi-xattr
+
diff --git a/debian/manpages b/debian/manpages
new file mode 100644
index 0000000..5054144
--- /dev/null
+++ b/debian/manpages
@@ -0,0 +1 @@
+man/*.man
diff --git a/debian/rules b/debian/rules
index 0587334..b361ae1 100755
--- a/debian/rules
+++ b/debian/rules
@@ -3,3 +3,10 @@
 
 %:
 	dh $@ 
+
+override_dh_auto_build-indep:
+	$(MAKE) docs
+
+override_dh_auto_clean:
+	$(MAKE) clean
+
diff --git a/man/byteback-backup.txt b/man/byteback-backup.txt
new file mode 100644
index 0000000..15c448f
--- /dev/null
+++ b/man/byteback-backup.txt
@@ -0,0 +1,69 @@
+NAME
+
+  byteback-backup - Back up this system to a byteback-enabled server
+
+OPTIONS
+ 
+   --destination, -d <s>    Backup destination (i.e. user@host:/path).
+   --source, -s <s+>        Source paths (default: /)
+   --exclude, -x <s+>       Paths to exclude
+   --retry-number, -r <i>:  Number of retries on error (default: 3)
+   --io-timeout, -i <i>:    Number of seconds to allow I/O timeout for (default: 10800)
+   --retry-delay, -e <i>:   Number of seconds between retries after an error (default: 300)
+   --ssh-key, -k <s>        SSH key filename (default: /etc/byteback/key)
+   --verbose, -v            Show debugging messages
+   --help, -h               Show this message
+
+Additional excludes can be specified using /etc/byteback/rsync_filter, which is
+an rsync filter file.  See the rsync man page for information on how this
+works.
+
+DESCRIPTION
+
+You can set "byteback-backup"  on a daily cron job to start backing up
+the server on a regular basis.
+
+Without any further options this will copy every file from the root
+downwards.
+
+It currently excludes /tmp, /var/tmp, /var/cache/apt/archives,
+/swap.file and /var/backups/localhost which (on Bytemark systems) do
+not need to be part of any backup.  To specify which locations are
+excluded, add them to /etc/byteback/excludes, one per line.  The
+filesystems /dev, /proc, /run and /sys are always excluded.
+
+It is possible to configure a full rsync filter by creating the file
+/etc/byteback/rsync_filter, which is parsed to rsync via the --filter
+flag.  Note that excludes on the command line take precedence, unless
+the filter starts with an exclamation mark, which resets everything.
+If you do this, you'll need to specify /proc, /sys, etc manually.  See
+the rsync manpage for more information about filters.
+
+It is also possible to add extra rsync flags to the configuration.
+The following flags are always set:
+
+  --archive --numeric-ids --delete-excluded --delete-during --inplace --relative 
+
+If you wish to add to that (e.g. --xattrs --acl --hard-links) then
+simply add them to /etc/byteback/rsync_flags.  These extra flags get
+appended after the default flags above, so they take precedence.
+
+When the backup has completed successfully, the server will take a
+snapshot so that the client can't alter the backups, and then "prune"
+the backup snapshots to ensure that the next backup is likely to run
+OK.
+
+If the backup is interrupted or dies unexpected, running
+"byteback-backup" will cause the backup to be resumed, with rsync
+saving the work of re-copying any files that hadn't changed.  By
+default this will happen automatically up to 5 times, with a 10 minute
+pause in between each attempt.
+
+SEE ALSO
+
+  byteback-restore(1), byteback-prune(1)
+
+AUTHOR
+
+  Patrick J Cherry <patrick@bytemark.co.uk>
+
diff --git a/man/byteback-prune.txt b/man/byteback-prune.txt
new file mode 100644
index 0000000..f8074e2
--- /dev/null
+++ b/man/byteback-prune.txt
@@ -0,0 +1,47 @@
+NAME
+
+  byteback-prune - Prune old byteback backup directories to ensure there's enough space
+
+OPTIONS
+
+  --minpercent, -m <i>   Start prune when disk has less than this %age free (default: 5)
+  --maxpercent, -a <i>   Stop prune when disk has more than this %age free (default: 10)
+  --list, -l             List backups in pruning order, no other action
+  --prune, -p            Prune the next backup if necessary
+  --prune-force, -r      Prune the next backup regardless
+  --order, -o <s>        Order backups by 'age' or 'importance' (default: importance)
+  --verbose, -v          Show debugging messages
+   --help, -h            Show this message
+
+DESCRIPTION
+
+Unless you are backing up a very small amount of data, backups will
+always need pruning, i.e. old backups must be deleted to make way for
+newer ones.
+
+This program deals with this operation by deleting old backups until a
+certain amount of free space is achieved, which is currently fixed at
+10% free.  This can be changed by setting the --maxpercent option.
+
+ * midday today;
+ * midday on the previous 6 days;
+ * midday on the previous 4 Sundays;
+ * midday on every fourth Sunday before that, where the "fourth
+   Sunday" is calculated as every fourth Sunday in the Unix epoch,
+   i.e.  every fourth Sunday since Sunday 4th Jan 1970. 
+
+So when you ask the pruner to run, the backup closest to the present
+time will be the last one to be deleted.  The backup closes to "1 day
+ago" will be the second-last, and so on.  We score every backup in
+this way until we end up with a "least important" snapshot to delete.
+
+The pruner will not delete backups younger than 7 days.
+
+SEE ALSO
+
+  byteback-backup(1), byteback-restore(1), byteback-receive(1)
+
+AUTHOR
+
+  Patrick J Cherry <patrick@bytemark.co.uk>
+
diff --git a/man/byteback-restore.txt b/man/byteback-restore.txt
new file mode 100644
index 0000000..d4ff01e
--- /dev/null
+++ b/man/byteback-restore.txt
@@ -0,0 +1,174 @@
+NAME
+
+byteback-restore - Restore a file to this system from a byteback-enabled server
+
+SYNOPSIS
+
+byteback-restore [ --restore | --list ] [ --snapshot s ]
+     [ --destination d ]  [ --io-timeout i ] [ --ssh-key k ]
+     [ --list-all ] [ --help ] [ --verbose ] pattern pattern..
+
+OPTIONS
+
+  --list, -l           List or find files
+  --restore, -r        Restore files
+  --snapshot, -s  s    The snaphot to use. (Default: *)
+  --destination, -d s  Backup destination (i.e. user@host:/path). The default is read from /etc/bytebackde
+  --io-timeout, -i i   Number of seconds to allow I/O timeout for (default: 300)
+  --ssh-key, -k s      SSH key filename (default: /etc/byteback/key)
+  --all, -a            List/restore all versrions of each file
+  --verbose, -v        Show debugging messages.
+  --help, -h           Show a brief usage message.
+
+DESCRIPTION
+
+Firstly the pattern argument. This is expanded relative to the current directory,
+if it is not an absolute path, before being globbed. It is best to quote the
+pattern to avoid premature shell expansion. Multiple patterns can be supplied.
+
+To list files that could be restored, just give a pattern. The --list flag can
+be used for clairty, but it is entirely optional.
+
+To restore files, use the --restore flag. Restores are always done into the
+current working directory. See EXAMPLES below.
+
+GLOBBING
+
+The following character are "globbed":
+
+ - "*" matches any filename.
+ - "**" matches directories recursively.
+ - "?" matches any single character.
+ - "[set]" matches any set of characters, working in the same way as regular expressions.
+ - "{a,b}" matches either literal "a" or literal "b"
+ - "\\" escapes the next metacharacter.
+
+Note that to match "dotfiles", the pattern "{*,.*}" should be used.
+
+The full glob syntax is documented in the Ruby documentation for the Dir#glob
+method.
+
+EXAMPLES
+
+To list the latest copies of the files available, just call the command with
+no flags.. The --list flag is optional.
+
+  # byteback-restore '/home/patch/.bash*'
+  snapshot modestring size  uid  gid  mtime                     path
+  ----------------------------------------------------------------------------------------
+  current  -rw------- 47931 1000 1000 2015-11-29 09:26:10 +0000 /home/patch/.bash_history
+  current  -rw-r--r-- 220   1000 1000 2014-03-10 10:13:33 +0000 /home/patch/.bash_logout
+  current  -rw-r--r-- 3638  1000 1000 2015-07-30 16:33:30 +0100 /home/patch/.bashrc
+
+To list all different available versions of a file, use --all.
+
+  # byteback-restore --all '/home/patch/.bashrc'
+  snapshot              modestring size uid  gid  mtime                     path
+  ----------------------------------------------------------------------------------------------
+  2015-07-17T15:11+0100 -rw-r--r-- 3637 1000 1000 2014-03-10 10:13:33 +0000 /home/patch/.bashrc
+  current               -rw-r--r-- 3638 1000 1000 2015-07-30 16:33:30 +0100 /home/patch/.bashrc
+
+To list every single copy of a file, use --verbose. Notice that a number of
+these entries match on size and mtime.
+
+  # byteback-restore --verbose '/home/patch/.bashrc'
+  ssh -o BatchMode=yes -o ConnectionAttempts=5 -o ConnectTimeout=30 -o ServerAliveInterval=60 -o TCPKeepAlive=yes -x -a -i /etc/byteback/key -l byteback backups.default.cherry.uk0.bigv.io byteback-receive --snapshot * --list --verbose /home/patch/.bashrc
+  snapshot              modestring size uid  gid  mtime                     path
+  ----------------------------------------------------------------------------------------------
+  2015-04-30T11:44+0100 -rw-r--r-- 3637 1000 1000 2014-03-10 10:13:33 +0000 /home/patch/.bashrc
+  2015-05-05T15:05+0100 -rw-r--r-- 3637 1000 1000 2014-03-10 10:13:33 +0000 /home/patch/.bashrc
+  2015-07-17T15:11+0100 -rw-r--r-- 3637 1000 1000 2014-03-10 10:13:33 +0000 /home/patch/.bashrc
+  2015-11-25T11:14+0000 -rw-r--r-- 3638 1000 1000 2015-07-30 16:33:30 +0100 /home/patch/.bashrc
+  2015-11-26T13:38+0000 -rw-r--r-- 3638 1000 1000 2015-07-30 16:33:30 +0100 /home/patch/.bashrc
+  2015-11-27T10:48+0000 -rw-r--r-- 3638 1000 1000 2015-07-30 16:33:30 +0100 /home/patch/.bashrc
+  2015-11-30T15:37+0000 -rw-r--r-- 3638 1000 1000 2015-07-30 16:33:30 +0100 /home/patch/.bashrc
+  current               -rw-r--r-- 3638 1000 1000 2015-07-30 16:33:30 +0100 /home/patch/.bashrc
+
+Restoring is always to the current directory, and into directories named after the snapshots.
+This next command restores the last copies of the files shown in the first
+listing above into the "current" directory, as they were all from the "current"
+snapshot.
+
+  # byteback-restore --restore '/home/patch/.bash*'
+  Restoring:
+  snapshot modestring size  uid  gid  mtime                     path
+  ----------------------------------------------------------------------------------------
+  current  -rw------- 47931 1000 1000 2015-11-29 09:26:10 +0000 /home/patch/.bash_history
+  current  -rw-r--r-- 220   1000 1000 2014-03-10 10:13:33 +0000 /home/patch/.bash_logout
+  current  -rw-r--r-- 3638  1000 1000 2015-07-30 16:33:30 +0100 /home/patch/.bashrc
+
+Using find(1), we can see the files restored. Note that the permissions on the
+parent directory have been set correctly.
+
+  # find current/ -ls
+  422049    4 drwxr-xr-x   3 root     root         4096 Nov 16 16:04 current/
+  422050    4 drwxr-xr-x   3 root     root         4096 Mar 12  2014 current/home
+  422885    4 drwxr-xr-x   2 patch    patch        4096 Nov 29 09:26 current/home/patch
+  422886    4 -rw-r--r--   1 patch    patch         220 Mar 10  2014 current/home/patch/.bash_logout
+  422887    4 -rw-r--r--   1 patch    patch        3638 Jul 30 16:33 current/home/patch/.bashrc
+
+To restore the files from a specific snapshot, use the --snapshot flag.
+
+  # byteback-restore --restore --snapshot '2015-07-17T15:11+0100' '/home/patch/.bash*'
+  Restoring:
+  snapshot              modestring size  uid  gid  mtime                     path
+  -----------------------------------------------------------------------------------------------------
+  2015-07-17T15:11+0100 -rw------- 38504 1000 1000 2015-07-17 13:42:17 +0100 /home/patch/.bash_history
+  2015-07-17T15:11+0100 -rw-r--r-- 220   1000 1000 2014-03-10 10:13:33 +0000 /home/patch/.bash_logout
+  2015-07-17T15:11+0100 -rw-r--r-- 3637  1000 1000 2014-03-10 10:13:33 +0000 /home/patch/.bashrc
+
+These have been restored into the 2015-07-17T15:11+0100 directory.
+
+  # find 2015-07-17T15:11+0100 -type f -ls
+  422911    4 -rw-r--r--   1 patch    patch         220 Mar 10  2014 2015-07-17T15:11+0100/home/patch/.bash_logout
+  422913   40 -rw-------   1 patch    patch       38504 Jul 17 13:42 2015-07-17T15:11+0100/home/patch/.bash_history
+  422912    4 -rw-r--r--   1 patch    patch        3637 Mar 10  2014 2015-07-17T15:11+0100/home/patch/.bashrc
+
+Finally, to restore all available copies of the files, use the --all flag.
+This will restore all copies shown when --list is used with the --all flag.
+
+  # byteback-restore --restore --all '/home/patch/.bashrc'
+  Restoring:
+  snapshot              modestring size  uid  gid  mtime                     path
+  -----------------------------------------------------------------------------------------------------
+  2015-07-17T15:11+0100 -rw-r--r-- 3637  1000 1000 2014-03-10 10:13:33 +0000 /home/patch/.bashrc
+  current               -rw-r--r-- 3638  1000 1000 2015-07-30 16:33:30 +0100 /home/patch/.bashrc
+
+We now have two versions of the same file, in different restore directories.
+
+  # find -type f -ls
+  946020    4 -rw-r--r--   1 patch    patch        3637 Mar 10  2014 ./2015-07-17T15:11+0100/home/patch/.bashrc
+  422893    4 -rw-r--r--   1 patch    patch        3638 Jul 30 16:33 ./current/home/patch/.bashrc
+
+CAVEATS
+
+If a pattern given matches a directory, then the whole directory will be
+restored recursively, not just the directory itself. This can be shown by
+adding the --verbose flag.
+
+  # byteback-restore --verbose --restore '/home/patch'
+  rsync --archive --numeric-ids --inplace --relative --compress --timeout 300 --rsh ssh -o BatchMode=yes -x -a -i /etc/byteback/key -l byteback --verbose --rsync-path byteback-restore --fake-super --snapshot * byteback@backups.default.cherry.uk0.bigv.io:/home/patch .
+  Restoring:
+  snapshot modestring size uid  gid  mtime                     path
+  -------------------------------------------------------------------------
+  current  drwxr-xr-x 5152 1000 1000 2015-11-29 09:26:22 +0000 /home/patch
+  rsync --fake-super --server --sender -vlogDtprRze.iLsfx --timeout=300 --numeric-ids --inplace --exclude=. . ./current/home/patch
+  receiving incremental file list
+  current/home/patch/
+  current/home/patch/.mrconfig
+  current/home/patch/.pam_environment
+  current/home/patch/.plan
+  current/home/patch/.profile
+  current/home/patch/.psql_history
+  etc...
+
+
+SEE ALSO
+
+byteback-receive(1), byteback-prune(1), http://ruby-doc.org/core/Dir.html#method-c-glob
+
+AUTHOR
+
+Patrick J Cherry <patrick@bytemark.co.uk>
+
+