Merge remote-tracking branch 'upstream/master' into ssh-key-passphrase

13 files changed, 336 insertions, 313 deletions

diff --git a/docs/Configuration.md b/docs/Configuration.md
index e2db05e..59e3f6d 100644
--- a/docs/Configuration.md
+++ b/docs/Configuration.md
@@ -1,10 +1,12 @@
-## Configuration
-### Debugging
+# Configuration
+
+## Debugging
+
 In case a model plugin doesn't work correctly (ios, procurve, etc.), you can enable live debugging of SSH/Telnet sessions. Just add a `debug` option containing the value true to the `input` section. The log files will be created depending on the parent directory of the logfile option.
 
 The following example will log an active ssh/telnet session `/home/oxidized/.config/oxidized/log/<IP-Adress>-<PROTOCOL>`. The file will be truncated on each consecutive ssh/telnet session, so you need to put a `tailf` or `tail -f` on that file!
 
-```
+```yaml
 log: /home/oxidized/.config/oxidized/log
 
 ...
@@ -16,20 +18,20 @@ input:
     secure: false
 ```
 
-### Privileged mode
+## Privileged mode
 
 To start privileged mode before pulling the configuration, Oxidized needs to send the enable command. You can globally enable this, by adding the following snippet to the global section of the configuration file.
 
-```
+```yaml
 vars:
    enable: S3cre7
 ```
 
-### Removing secrets
+## Removing secrets
 
 To strip out secrets from configurations before storing them, Oxidized needs the the remove_secrets flag. You can globally enable this by adding the following snippet to the global sections of the configuration file.
 
-```
+```yaml
 vars:
   remove_secret: true
 ```
@@ -38,32 +40,33 @@ Device models can contain substitution filters to remove potentially sensitive d
 
 As a partial example from ios.rb:
 
-```
+```yaml
   cmd :secret do |cfg|
     cfg.gsub! /^(snmp-server community).*/, '\\1 <configuration removed>'
     (...)
     cfg
   end
 ```
+
 The above strips out snmp community strings from your saved configs.
 
 **NOTE:** Removing secrets reduces the usefulness as a full configuration backup, but it may make sharing configs easier.
 
-### Disabling SSH exec channels
+## Disabling SSH exec channels
 
 Oxidized uses exec channels to make information extraction simpler, but there are some situations where this doesn't work well, e.g. configuring devices.  This feature can be turned off by setting the `ssh_no_exec`
 variable.
 
-```
+```yaml
 vars:
   ssh_no_exec: true
 ```
 
-### SSH Proxy Command
+## SSH Proxy Command
 
 Oxidized can `ssh` through a proxy as well. To do so we just need to set `ssh_proxy` variable.
 
-```
+```yaml
 ...
 map:
   name: 0
@@ -74,20 +77,21 @@ vars_map:
 ...
 ```
 
-### FTP Passive Mode
+## FTP Passive Mode
 
 Oxidized uses ftp passive mode by default. Some devices require passive mode to be disabled. To do so, we can set `input.ftp.passive` to false
-```
+
+```yaml
 input:
   ftp:
     passive: false
 ```
 
-### Advanced Configuration
+## Advanced Configuration
 
 Below is an advanced example configuration. You will be able to (optionally) override options per device. The router.db format used is `hostname:model:username:password:enable_password`. Hostname and model will be the only required options, all others override the global configuration sections.
 
-```
+```yaml
 ---
 username: oxidized
 password: S3cr3tx
@@ -130,14 +134,13 @@ source:
 model_map:
   cisco: ios
   juniper: junos
-
 ```
 
-### Advanced Group Configuration
+## Advanced Group Configuration
 
 For group specific credentials
 
-```
+```yaml
 groups:
   mikrotik:
     username: admin
@@ -146,16 +149,19 @@ groups:
     username: ubnt
     password: ubnt
 ```
+
 and add group mapping
-```
+
+```yaml
 map:
   model: 0
   name: 1
   group: 2
 ```
+
 For model specific credentials
 
-```
+```yaml
 models:
   junos:
     username: admin
@@ -170,26 +176,26 @@ models:
     password: password
 ```
 
-### RESTful API and Web Interface
+## RESTful API and Web Interface
 
 The RESTful API and Web Interface is enabled by configuring the `rest:` parameter in the config file.  This parameter can optionally contain a relative URI.
 
-```
+```yaml
 # Listen on http://127.0.0.1:8888/
 rest: 127.0.0.1:8888
 ```
 
-```
+```yaml
 # Listen on http://10.0.0.1:8000/oxidized/
 rest: 10.0.0.1:8000/oxidized
 ```
 
-### Triggered backups
+## Triggered backups
 
 A node can be moved to head-of-queue via the REST API `GET/POST /node/next/[NODE]`.
 
 In the default configuration this node will be processed when the next job worker becomes available, it could take some time if existing backups are in progress. To execute moved jobs immediately a new job can be added:
 
-```
+```yaml
 next_adds_job: true
 ```
diff --git a/docs/Hooks.md b/docs/Hooks.md
index 080d301..1d263fb 100644
--- a/docs/Hooks.md
+++ b/docs/Hooks.md
@@ -1,25 +1,30 @@
 # Hooks
+
 You can define arbitrary number of hooks that subscribe different events. The hook system is modular and different kind of hook types can be enabled.
 
 ## Configuration
+
 Following configuration keys need to be defined for all hooks:
 
-  * `events`: which events to subscribe. Needs to be an array. See below for the list of available events.
-  * `type`: what hook class to use. See below for the list of available hook types.
+* `events`: which events to subscribe. Needs to be an array. See below for the list of available events.
+* `type`: what hook class to use. See below for the list of available hook types.
 
 ### Events
-  * `node_success`: triggered when configuration is succesfully pulled from a node and right before storing the configuration.
-  * `node_fail`: triggered after `retries` amount of failed node pulls.
-  * `post_store`: triggered after node configuration is stored (this is executed only when the configuration has changed).
-  * `nodes_done`: triggered after finished fetching all nodes.
+
+* `node_success`: triggered when configuration is succesfully pulled from a node and right before storing the configuration.
+* `node_fail`: triggered after `retries` amount of failed node pulls.
+* `post_store`: triggered after node configuration is stored (this is executed only when the configuration has changed).
+* `nodes_done`: triggered after finished fetching all nodes.
 
 ## Hook type: exec
+
 The `exec` hook type allows users to run an arbitrary shell command or a binary when triggered.
 
 The command is executed on a separate child process either in synchronous or asynchronous fashion. Non-zero exit values cause errors to be logged. STDOUT and STDERR are currently not collected.
 
 Command is executed with the following environment:
-```
+
+```text
 OX_EVENT
 OX_NODE_NAME
 OX_NODE_IP
@@ -35,13 +40,13 @@ OX_REPO_NAME
 
 Exec hook recognizes following configuration keys:
 
-  * `timeout`: hard timeout for the command execution. SIGTERM will be sent to the child process after the timeout has elapsed. Default: 60
-  * `async`: influences whether main thread will wait for the command execution. Set this true for long running commands so node pull is not blocked. Default: false
-  * `cmd`: command to run.
+* `timeout`: hard timeout for the command execution. SIGTERM will be sent to the child process after the timeout has elapsed. Default: 60
+* `async`: influences whether main thread will wait for the command execution. Set this true for long running commands so node pull is not blocked. Default: false
+* `cmd`: command to run.
 
+## exec hook configuration example
 
-## Hook configuration example
-```
+```yaml
 hooks:
   name_for_example_hook1:
     type: exec
@@ -55,23 +60,23 @@ hooks:
     timeout: 120
 ```
 
-### githubrepo
+### Hook type: githubrepo
 
 This hook configures the repository `remote` and _push_ the code when the specified event is triggerd. If the `username` and `password` are not provided, the `Rugged::Credentials::SshKeyFromAgent` will be used.
 
 `githubrepo` hook recognizes following configuration keys:
 
-  * `remote_repo`: the remote repository to be pushed to.
-  * `username`: username for repository auth.
-  * `password`: password for repository auth.
-  * `publickey`: publickey for repository auth.
-  * `privatekey`: privatekey for repository auth.
+* `remote_repo`: the remote repository to be pushed to.
+* `username`: username for repository auth.
+* `password`: password for repository auth.
+* `publickey`: publickey for repository auth.
+* `privatekey`: privatekey for repository auth.
 
 It is also possible to set the environment variable `OXIDIZED_SSH_PASSPHRASE` to a passphrase if your keypair requires it.
 
 When using groups repositories, each group must have its own `remote` in the `remote_repo` config.
 
-``` yaml
+```yaml
 hooks:
   push_to_remote:
     remote_repo:
@@ -80,10 +85,9 @@ hooks:
       firewalls: git@git.intranet:oxidized/firewalls.git
 ```
 
+## githubrepo hook configuration example
 
-## Hook configuration example
-
-``` yaml
+```yaml
 hooks:
   push_to_remote:
     type: githubrepo
@@ -99,14 +103,14 @@ The `awssns` hook publishes messages to AWS SNS topics. This allows you to notif
 
 Fields sent in the message:
 
-  * `event`: Event type (e.g. `node_success`)
-  * `group`: Group name
-  * `model`: Model name (e.g. `eos`)
-  * `node`: Device hostname
+* `event`: Event type (e.g. `node_success`)
+* `group`: Group name
+* `model`: Model name (e.g. `eos`)
+* `node`: Device hostname
 
-Configuration example:
+## awssns hook configuration example
 
-``` yaml
+```yaml
 hooks:
   hook_script:
     type: awssns
@@ -117,8 +121,8 @@ hooks:
 
 AWS SNS hook requires the following configuration keys:
 
-  * `region`: AWS Region name
-  * `topic_arn`: ASN Topic reference
+* `region`: AWS Region name
+* `topic_arn`: ASN Topic reference
 
 Your AWS credentials should be stored in `~/.aws/credentials`.
 
@@ -128,13 +132,13 @@ The `slackdiff` hook posts colorized config diffs to a [Slack](http://www.slack.
 
 You will need to manually install the `slack-api` gem on your system:
 
-```
+```shell
 gem install slack-api
 ```
 
-Configuration example:
+## slackdiff hook configuration example
 
-``` yaml
+```yaml
 hooks:
   slack:
     type: slackdiff
@@ -145,7 +149,7 @@ hooks:
 
 Optionally you can disable snippets and post a formatted message, for instance linking to a commit in a git repo. Named parameters `%{node}`, `%{group}`, `%{model}` and `%{commitref}` are available.
 
-``` yaml
+```yaml
 hooks:
   slack:
     type: slackdiff
@@ -164,13 +168,13 @@ The `xmppdiff` hook posts config diffs to a [XMPP](https://en.wikipedia.org/wiki
 
 You will need to manually install the `xmpp4r` gem on your system:
 
-```
+```shell
 gem install xmpp4r
 ```
 
-Configuration example:
+## xmppdiff hook configuration example
 
-``` yaml
+```yaml
 hooks:
   xmpp:
     type: xmppdiff
diff --git a/docs/Model-Notes/AireOS.md b/docs/Model-Notes/AireOS.md
index 0da9b57..9962f31 100644
--- a/docs/Model-Notes/AireOS.md
+++ b/docs/Model-Notes/AireOS.md
@@ -1,12 +1,12 @@
-Cisco WLC Configuration 
-========================
+Cisco WLC Configuration
+=======================
 
 Create a user with read-write privilege :
 
-```
+```text
 mgmtuser add oxidized **** read-write
 ```
 
-Oxidized needs read-write privilege in order to execute 'config paging disable'. 
+Oxidized needs read-write privilege in order to execute 'config paging disable'.
 
 Back to [Model-Notes](README.md)
diff --git a/docs/Model-Notes/ArbOS.md b/docs/Model-Notes/ArbOS.md
index 0f5b628..f68467f 100644
--- a/docs/Model-Notes/ArbOS.md
+++ b/docs/Model-Notes/ArbOS.md
@@ -3,7 +3,7 @@ Arbor Networks ArbOS notes
 
 If you are running ArbOS version 7 or lower then you may need to update the model to remove `exec true`:
 
-```
+```ruby
   cfg :ssh do
     pre_logout 'exit'
   end
diff --git a/docs/Model-Notes/Comware.md b/docs/Model-Notes/Comware.md
index 31eb002..0da019a 100644
--- a/docs/Model-Notes/Comware.md
+++ b/docs/Model-Notes/Comware.md
@@ -1,10 +1,12 @@
 Comware Configuration
 =====================
 
-If you find 3Com comware devices aren't being backed up this may be due to prompt detection not matching 
-because a previous login message is disabled after the first prompt. You can disable this on the devices 
-themselves by running this command:
+If you find 3Com comware devices aren't being backed up this may be due to prompt detection not matching because a previous login message is disabled after the first prompt.
 
-`info-center source default channel 1 log state off debug state off`
+You can disable this on the devices themselves by running this command:
+
+```text
+info-center source default channel 1 log state off debug state off
+```
 
 [Reference](https://github.com/ytti/oxidized/issues/1171)
diff --git a/docs/Model-Notes/JunOS.md b/docs/Model-Notes/JunOS.md
index ed8dbca..8093df0 100644
--- a/docs/Model-Notes/JunOS.md
+++ b/docs/Model-Notes/JunOS.md
@@ -1,9 +1,9 @@
 JunOS Configuration
-========================
+===================
 
 Create login class cfg-view
 
-```
+```text
 set system login class cfg-view permissions view-configuration
 set system login class cfg-view allow-commands "(show)|(set cli screen-length)|(set cli screen-width)"
 set system login class cfg-view deny-commands "(clear)|(file)|(file show)|(help)|(load)|(monitor)|(op)|(request)|(save)|(set)|(start)|(test)"
@@ -12,7 +12,7 @@ set system login class cfg-view deny-configuration all
 
 Create a user with cfg-view class
 
-```
+```text
 set system login user oxidized class cfg-view
 set system login user oxidized authentication plain-text-password "verysecret"
 ```
@@ -25,14 +25,10 @@ The commands Oxidized executes are:
 4. show version
 5. show chassis hardware
 6. show system license
-7. show system license keys
-ex22|ex33|ex4|ex8|qfx only
-8. show virtual-chassis
-MX960 only
+7. show system license keys (ex22|ex33|ex4|ex8|qfx only)
+8. show virtual-chassis (MX960 only)
 9. show chassis fabric reachability
 
-
 Oxidized can now retrieve your configuration!
 
-
 Back to [Model-Notes](README.md)
diff --git a/docs/Model-Notes/README.md b/docs/Model-Notes/README.md
index 9d7db32..8dcabd3 100644
--- a/docs/Model-Notes/README.md
+++ b/docs/Model-Notes/README.md
@@ -1,21 +1,17 @@
-
-
 Model Notes
-========================
-
+===========
 
 This directory contains implemention notes and caveats to assist you in your oxidized deployment.
 
 Use the table below for more information on the Vendor/Model caveats.
 
-
 Vendor          | Model           |Updated
 ----------------|-----------------|----------------
 3COM|[Comware](Comware.md)|15 Feb 2018
+AireOS|[AireOS](AireOS.md)|29 Nov 2017
 Arbor Networks|[ArbOS](ArbOS.md)|27 Feb 2018
 Huawei|[VRP](VRP-Huawei.md)|17 Nov 2017
 Juniper|[MX/QFX/EX/SRX/J Series](JunOS.md)|18 Jan 2018
 Xyzel|[XGS4600 Series](XGS4600-Zyxel.md)|23 Jan 2018
 
-
 If you discover additional caveats or problems please make sure to consult the [GitHub issues for oxidized](https://github.com/ytti/oxidized/issues) known issues.
diff --git a/docs/Model-Notes/VRP-Huawei.md b/docs/Model-Notes/VRP-Huawei.md
index d03c752..e68a959 100644
--- a/docs/Model-Notes/VRP-Huawei.md
+++ b/docs/Model-Notes/VRP-Huawei.md
@@ -3,7 +3,7 @@ Huawei VRP Configuration
 
 Create a user with no privileges
 
-```
+```text
     <HUAWEI> system-view
     [~HUAWEI] aaa
     [~HUAWEI-aaa] local-user oxidized password irreversible-cipher verysecret
@@ -21,7 +21,7 @@ The commands Oxidized executes are:
 
 Command 2 and 3 can be executed without issues, but 1 and 4 are only available for higher level users. Instead of making Oxidized a read/write user on your device, lower the priviledge-level for commands 1 and 4:
 
-```
+```text
     <HUAWEI> system-view
     [~HUAWEI] command-privilege level 1 view global display current-configuration all
     [*HUAWEI] command-privilege level 1 view shell screen-length
@@ -30,5 +30,4 @@ Command 2 and 3 can be executed without issues, but 1 and 4 are only available f
 
 Oxidized can now retrieve your configuration!
 
-
 Back to [Model-Notes](README.md)
diff --git a/docs/Model-Notes/XGS4600-Zyxel.md b/docs/Model-Notes/XGS4600-Zyxel.md
index 0ff2e7d..17cb2b5 100644
--- a/docs/Model-Notes/XGS4600-Zyxel.md
+++ b/docs/Model-Notes/XGS4600-Zyxel.md
@@ -1,18 +1,20 @@
 ZynOS Configuration
-========================
+===================
 
 ## FTP
+
 FTP access is only possible as admin, other users can login but cannot pull the files.
 For the XGS4600 series the config file is _config_ and not _config-0_
 
-The following line in _oxidized/lib/oxidized/model/zynos.rb_ with need changing
-```
-  cmd 'config-0'
+The following line in _oxidized/lib/oxidized/model/zynos.rb_ will need changing
 
+```text
+  cmd 'config-0'
 ```
 
 The inclusion of an extra ftp option is also require. Within _input_ add the following
-```
+
+```yaml
 input:
   ftp:
     passive: false
@@ -20,5 +22,4 @@ input:
 
 Oxidized can now retrieve your configuration!
 
-
 Back to [Model-Notes](README.md)
diff --git a/docs/Outputs.md b/docs/Outputs.md
index e3bd42d..92c672c 100644
--- a/docs/Outputs.md
+++ b/docs/Outputs.md
@@ -1,23 +1,22 @@
+# Outputs
 
-## Output
-
-### Output: File
+## Output: File
 
 Parent directory needs to be created manually, one file per device, with most recent running config.
 
-```
+```yaml
 output:
   file:
     directory: /var/lib/oxidized/configs
 ```
 
-### Output: Git
+## Output: Git
 
 This uses the rugged/libgit2 interface. So you should remember that normal Git hooks will not be executed.
 
 For a single repositories for all devices:
 
-``` yaml
+```yaml
 output:
   default: git
   git:
@@ -28,7 +27,7 @@ output:
 
 And for groups repositories:
 
-``` yaml
+```yaml
 output:
   default: git
   git:
@@ -40,14 +39,14 @@ output:
 Oxidized will create a repository for each group in the same directory as the `default.git`. For
 example:
 
-``` csv
+```csv
 host1:ios:first
 host2:nxos:second
 ```
 
 This will generate the following repositories:
 
-``` bash
+```bash
 $ ls /var/lib/oxidized/git-repos
 
 default.git first.git second.git
@@ -55,7 +54,7 @@ default.git first.git second.git
 
 If you would like to use groups and a single repository, you can force this with the `single_repo` config.
 
-``` yaml
+```yaml
 output:
   default: git
   git:
@@ -64,15 +63,14 @@ output:
 
 ```
 
-### Output: Git-Crypt
+## Output: Git-Crypt
 
 This uses the gem git and system git-crypt interfaces. Have a look at [GIT-Crypt](https://www.agwa.name/projects/git-crypt/) documentation to know how to install it.
 Additionally to user and email informations, you have to provide the users ID that can be a key ID, a full fingerprint, an email address, or anything else that uniquely identifies a public key to GPG (see "HOW TO SPECIFY A USER ID" in the gpg man page).
 
-
 For a single repositories for all devices:
 
-``` yaml
+```yaml
 output:
   default: gitcrypt
   gitcrypt:
@@ -86,7 +84,7 @@ output:
 
 And for groups repositories:
 
-``` yaml
+```yaml
 output:
   default: gitcrypt
   gitcrypt:
@@ -101,14 +99,14 @@ output:
 Oxidized will create a repository for each group in the same directory as the `default`. For
 example:
 
-``` csv
+```csv
 host1:ios:first
 host2:nxos:second
 ```
 
 This will generate the following repositories:
 
-``` bash
+```bash
 $ ls /var/lib/oxidized/git-repos
 
 default.git first.git second.git
@@ -116,7 +114,7 @@ default.git first.git second.git
 
 If you would like to use groups and a single repository, you can force this with the `single_repo` config.
 
-``` yaml
+```yaml
 output:
   default: gitcrypt
   gitcrypt:
@@ -130,11 +128,11 @@ output:
 
 Please note that user list is only updated once at creation.
 
-### Output: Http
+## Output: Http
 
 POST a config to the specified URL
 
-```
+```yaml
 output:
   default: http
   http:
@@ -143,13 +141,13 @@ output:
     url: "http://192.168.162.50:8080/db/coll"
 ```
 
-### Output types
+## Output types
 
 If you prefer to have different outputs in different files and/or directories, you can easily do this by modifying the corresponding model. To change the behaviour for IOS, you would edit `lib/oxidized/model/ios.rb` (run `gem contents oxidized` to find out the full file path).
 
 For example, let's say you want to split out `show version` and `show inventory` into separate files in a directory called `nodiff` which your tools will not send automated diffstats for. You can apply a patch along the lines of
 
-```
+```text
 -  cmd 'show version' do |cfg|
 -    comment cfg.lines.first
 +  cmd 'show version' do |state|
@@ -183,7 +181,7 @@ For example, let's say you want to split out `show version` and `show inventory`
 
 which will result in the following layout
 
-```
+```text
 diff/$FQDN--show_running_config
 nodiff/$FQDN--show_version
 nodiff/$FQDN--show_inventory
diff --git a/docs/Ruby-API.md b/docs/Ruby-API.md
index 2dbc10e..8621870 100644
--- a/docs/Ruby-API.md
+++ b/docs/Ruby-API.md
@@ -3,33 +3,39 @@
 The following objects exist in Oxidized.
 
 ## Input
- * gets config from nodes
- * must implement 'connect', 'get', 'cmd'
- * 'ssh', 'telnet, ftp, and tftp' implemented
+
+* gets config from nodes
+* must implement 'connect', 'get', 'cmd'
+* 'ssh', 'telnet, ftp, and tftp' implemented
 
 ## Output
- * stores config
- * must implement 'store' (may implement 'fetch')
- * 'git' and 'file' (store as flat ascii) implemented
+
+* stores config
+* must implement 'store' (may implement 'fetch')
+* 'git' and 'file' (store as flat ascii) implemented
 
 ## Source
- * gets list of nodes to poll
- * must implement 'load'
- * source can have 'name', 'model', 'group', 'username', 'password', 'input', 'output', 'prompt'
-   * name - name of the devices
-   * model - model to use ios/junos/xyz, model is loaded dynamically when needed (Also default in config file)
-   * input - method to acquire config, loaded dynamically as needed (Also default in config file)
-   * output - method to store config, loaded dynamically as needed (Also default in config file)
-   * prompt - prompt used for node (Also default in config file, can be specified in model too)
- * 'sql', 'csv' and 'http' (supports any format with single entry per line, like router.db)
+
+* gets list of nodes to poll
+* must implement 'load'
+* source can have 'name', 'model', 'group', 'username', 'password', 'input', 'output', 'prompt'
+  * name - name of the devices
+  * model - model to use ios/junos/xyz, model is loaded dynamically when needed (Also default in config file)
+  * input - method to acquire config, loaded dynamically as needed (Also default in config file)
+  * output - method to store config, loaded dynamically as needed (Also default in config file)
+  * prompt - prompt used for node (Also default in config file, can be specified in model too)
+* 'sql', 'csv' and 'http' (supports any format with single entry per line, like router.db)
 
 ## Model
+
 ### At the top level
+
 A model may use several methods at the top level in the class. `cfg` is
 executed in input/output/source context. `cmd` is executed within an instance
 of the model.
 
 #### `cfg`
+
 `cfg` may be called with a list of methods (`:ssh`, `:telnet`) and a block with
 zero parameters.  Calling `cfg` registers the given access methods and calling
 it at least once is required for a model to work.
@@ -38,6 +44,7 @@ The block may contain commands to change some behaviour for the given methods
 (e.g. calling `post_login` to disable the pager).
 
 #### `cmd`
+
 Is used to specify commands that should be executed on a model in order to
 gather its configuration. It can be called with:
 
@@ -69,18 +76,21 @@ Execution order is `:all`, `:secret`, and lastly the command specific block, if
 given.
 
 #### `comment`
+
 Called with a single string containing the string to prepend for comments in
 emitted configuration for this model.
 
 If not specified the default of `'# '` will be used (note the trailing space).
 
 #### `prompt`
+
 Is called with a regular expression that is used to detect when command output
 ends after a command has been executed.
 
 If not specified, a default of `/^([\w.@-]+[#>]\s?)$/` is used.
 
 #### `expect`
+
 Called with a regular expression and a block. The block takes two parameters:
 the regular expression, and the data containing the match.
 
@@ -90,26 +100,32 @@ The passed data is replaced by the return value of the block.
 it's further processed.
 
 ### At the second level
+
 The following methods are available:
 
 #### `comment`
+
 Used inside `cmd` invocations. Comments out every line in the passed string and
 returns the result.
 
 #### `password`
+
 Used inside `cfg` invocations to specify the regular expression used to detect
 the password prompt. If not specified, the default of `/^Password/` is used.
 
 #### `post_login`
+
 Used inside `cfg` invocations to specify commands to run once Oxidized has
 logged in to the switch. Takes one argument that is either a block (taking zero
 parameters) or a string containing a command to execute.
 
 #### `pre_logout`
+
 Used to specify commands to run before Oxidized closes the connection to the
 switch. Takes one argument that is either a block (taking zero parameters) or a
 string containing a command to execute.
 
 #### `send`
+
 Usually used inside `expect` or blocks passed to `post_login`/`pre_logout`.
 Takes a single parameter: a string to be sent to the switch.
diff --git a/docs/Sources.md b/docs/Sources.md
index 7029b72..5599409 100644
--- a/docs/Sources.md
+++ b/docs/Sources.md
@@ -1,10 +1,10 @@
-## Source
+# Sources
 
-### Source: CSV
+## Source: CSV
 
 One line per device, colon seperated. If `ip` isn't present, a DNS lookup will be done against `name`.  For large installations, setting `ip` will dramatically reduce startup time.
 
-```
+```yaml
 source:
   default: csv
   csv:
@@ -22,7 +22,7 @@ source:
 
 Example csv `/var/lib/oxidized/router.db`:
 
-```
+```text
 rtr01.local:192.168.1.1:ios:oxidized:5uP3R53cR3T:T0p53cR3t
 ```
 
@@ -41,25 +41,31 @@ source:
       model: 1
 ```
 
-> Please note, if you are running GPG v2 then you will be prompted for your gpg password on start up, if you use GPG >= 2.1 then you can add the following config to stop that behaviour:
+Please note, if you are running GPG v2 then you will be prompted for your gpg password on start up, if you use GPG >= 2.1 then you can add the following config to stop that behaviour:
 
-> Within `~/.gnupg/gpg-agent.conf`
+Within `~/.gnupg/gpg-agent.conf`
+
+```text
+allow-loopback-pinentry
+```
 
-> `allow-loopback-pinentry`
+and within: `~/.gnupg/gpg.conf`
 
-> and within: `~/.gnupg/gpg.conf`
+```text
+pinentry-mode loopback
+```
 
-> `pinentry-mode loopback`
+## Source: SQL
 
-### Source: SQL
  Oxidized uses the `sequel` ruby gem. You can use a variety of databases that aren't explicitly listed. For more information visit https://github.com/jeremyevans/sequel Make sure you have the correct adapter!
-### Source: MYSQL
+
+## Source: MYSQL
 
 `sudo apt-get install libmysqlclient-dev`
 
 The values correspond to your fields in the DB such that ip, model, etc are field names in the DB
 
-```
+```yaml
 source:
   default: sql
   sql:
@@ -77,11 +83,11 @@ source:
       enable: enable
 ```
 
-### Source: SQLite
+## Source: SQLite
 
 One row per device, filtered by hostname.
 
-```
+```yaml
 source:
   default: sql
   sql:
@@ -97,12 +103,11 @@ source:
       enable: enable
 ```
 
-### Custom SQL Query Support
+## Custom SQL Query Support
 
 You may also implement a custom SQL query to retreive the nodelist using  SQL syntax with the `query:` configuration parameter under the `sql:` stanza.
 
-
-#### Custom SQL Query Examples
+### Custom SQL Query Examples
 
 You may have a table named `nodes` which contains a boolean to indicate if the nodes should be enabled (fetched via oxidized). This can be used in the custom SQL query to avoid fetching from known impacted nodes.
 
@@ -119,21 +124,20 @@ In this example we limit the nodes to two "POPs" of `mypop1` and `mypop2`. We al
 ```sql
 query: "SELECT * FROM nodes WHERE pop IN ('mypop1','mypop2') AND enabled = True"
 ```
+
 The order of the nodes returned will influence the order that nodes are fetched by oxidized. You can use standard SQL `ORDER BY` clauses to influence the node order.
 
 You should always test your SQL query before using it in the oxidized configuration as there is no syntax or error checking performed before sending it to the database engine.
 
 Consult your database documentation for more information on query language and table optimization.
 
-
-
-### Source: HTTP
+## Source: HTTP
 
 One object per device.
 
 HTTP Supports basic auth, configure the user and pass you want to use under the http: section.
 
-```
+```yaml
 source:
   default: http
   http:
@@ -155,7 +159,7 @@ source:
 
 You can also pass `secure: false` if you want to disable ssl certificate verification:
 
-```
+```yaml
 source:
   default: http
   http:
diff --git a/docs/Supported-OS-Types.md b/docs/Supported-OS-Types.md
index d6d718f..a0ca66b 100644
--- a/docs/Supported-OS-Types.md
+++ b/docs/Supported-OS-Types.md
@@ -1,159 +1,160 @@
 # Supported OS types
- * Vendor
-   * OS model
- * A10 Networks
-   * [ACOS](/lib/oxidized/model/acos.rb)
- * Accedian Performance Elements (NIDs)
-   * [AEN](/lib/oxidized/model/aen.rb)
- * Alcatel-Lucent
-   * [AOS](/lib/oxidized/model/aos.rb)
-   * [AOS7](/lib/oxidized/model/aos7.rb)
-   * [ISAM](/lib/oxidized/model/isam.rb)
-   * [SR OS (Formerly TiMOS)](/lib/oxidized/model/timos.rb)
-   * Wireless
- * Allied Telesis
-   * [Alliedware Plus](/lib/oxidized/model/awplus.rb)
- * Alvarion
-   * [BreezeACCESS](/lib/oxidized/model/alvarion.rb)
- * APC
-   * [AOS](/lib/oxidized/model/apc_aos.rb)
- * Arbor Networks
-   * [ArbOS](/lib/oxidized/model/arbos.rb)
- * Arista
-   * [EOS](/lib/oxidized/model/eos.rb)
- * Arris
-   * [C4CMTS](/lib/oxidized/model/c4cmts.rb)
- * Aruba
-   * [AOSW](/lib/oxidized/model/aosw.rb)
- * AudioCodes
-   * [AudioCodes](/lib/oxdized/model/audiocodes.rb)
- * Avaya
-   * [VOSS (VSP Operating System Software)](/lib/oxidized/model/voss.rb)
-   * [BOSS (Baystack Operating System Software)](/lib/oxidized/model/boss.rb)
- * Brocade
-   * [FabricOS](lib/oxidized/model/fabricos.rb)
-   * [Ironware](lib/oxidized/model/ironware.rb)
-   * [NOS (Network Operating System)](lib/oxidized/model/nos.rb)
-   * [Vyatta](lib/oxidized/model/vyatta.rb)
-   * [6910](lib/oxidized/model/br6910.rb)
-   * [SLX-OS](lib/oxidized/model/slxos.rb)
- * Casa
-   * [Casa](/lib/oxidized/model/casa.rb)
- * Check Point
-   * [GaiaOS](/lib/oxidized/model/gaiaos.rb)
- * Ciena
-   * [SAOS](/lib/oxidized/model/saos.rb)
- * Cisco
-   * [ACSW](/lib/oxidized/model/acsw.rb)
-   * [AireOS](/lib/oxidized/model/aireos.rb)
-   * [ASA](/lib/oxidized/model/asa.rb)
-   * [AsyncOS](/lib/oxidized/model/asyncos.rb)
-   * [CatOS](/lib/oxidized/model/catos.rb)
-   * [IOS](/lib/oxidized/model/ios.rb)
-   * [IOSXR](/lib/oxidized/model/iosxr.rb)
-   * [NGA](/lib/oxidized/model/cisconga.rb)
-   * [NXOS](/lib/oxidized/model/nxos.rb)
-   * [SMA](/lib/oxidized/model/ciscosma.rb)
-   * [SMB (Nikola series)](/lib/oxidized/model/ciscosmb.rb)
-   * [UCS](/lib/oxidized/model/ucs.rb)
- * Citrix
-   * [NetScaler (Virtual Applicance)](/lib/oxidized/model/netscaler.rb)
- * Coriant (former Tellabs)
-   * [TMOS (8800)](/lib/oxidized/model/corianttmos.rb)
-   * [8600](/lib/oxidized/model/coriant8600.rb)
-   * [Groove](/lib/oxidized/model/coriantgroove.rb)
- * Cumulus
-   * [Linux](/lib/oxidized/model/cumulus.rb)
- * DataCom
-   * [DmSwitch 3000](/lib/oxidized/model/datacom.rb)
- * DCN
-   * [DCN](/lib/oxidized/model/ios.rb) - Map this to ios.
- * DELL
-   * [PowerConnect](/lib/oxidized/model/powerconnect.rb)
-   * [AOSW](/lib/oxidized/model/aosw.rb)
- * D-Link
-   * [D-Link](/lib/oxidized/model/dlink.rb)
- * Ericsson/Redback
-   * [IPOS (former SEOS)](/lib/oxidized/model/ipos.rb)
- * Extreme Networks
-   * [Enterasys](/lib/oxidized/model/enterasys.rb)
-   * [WM](/lib/oxidized/model/mtrlrfs.rb)
-   * [XOS](/lib/oxidized/model/xos.rb)
- * F5
-   * [TMOS](/lib/oxidized/model/tmos.rb)
- * Fiberstore
-   * [S3800](/lib/oxidized/model/gcombnps.rb)
- * Force10
-   * [DNOS](/lib/oxidized/model/dnos.rb)
-   * [FTOS](/lib/oxidized/model/ftos.rb)
- * FortiGate
-   * [FortiOS](/lib/oxidized/model/fortios.rb)
- * Fujitsu
-   * [PRIMERGY Blade switch 1/10Gbe](/lib/oxidized/model/fujitsupy.rb)
- * GCOM Technologies
-   * [Broadband Network Platform Software](/lib/oxidized/model/gcombnps.rb)
- * Hatteras
-   * [Hatteras](/lib/oxidized/model/hatteras.rb)
- * Hirschmann
-   * [HiOS](/lib/oxidized/model/hirschmann.rb)
- * HP
-   * [Comware (HP A-series, H3C, 3Com)](/lib/oxidized/model/comware.rb)
-   * [Procurve](/lib/oxidized/model/procurve.rb)
-   * [BladeSystem (Onboard Administrator)](/lib/oxidized/model/hpebladesystem.rb)
-   * [MSA](/lib/oxidized/model/hpemsa.rb)
- * Huawei
-   * [VRP](/lib/oxidized/model/vrp.rb)
- * Juniper
-   * [JunOS](/lib/oxidized/model/junos.rb)
-   * [ScreenOS (Netscreen)](/lib/oxidized/model/screenos.rb)
- * Mellanox
-   * [MLNX-OS](/lib/oxidized/model/mlnxos.rb)
-   * [Voltaire](/lib/oxidized/model/voltaire.rb)
- * Mikrotik
-   * [RouterOS](/lib/oxidized/model/routeros.rb)
- * Motorola
-   * [RFS](/lib/oxidized/model/mtrlrfs.rb)
- * MRV
-   * [MasterOS](/lib/oxidized/model/masteros.rb)
-   * [FiberDriver](/lib/oxidized/model/fiberdriver.rb)
- * Netgear
-   * [Netgear](/lib/oxidized/model/netgear.rb)
- * Netonix
-   * [WISP Switch (As Netonix)](/lib/oxidized/model/netonix.rb)
- * Nokia (formerly TiMetra, Alcatel, Alcatel-Lucent)
-   * [SR OS (TiMOS)](/lib/oxidized/model/timos.rb)
- * OneAccess
-   * [OneOS](/lib/oxidized/model/oneos.rb)
- * Opengear
-   * [Opengear](/lib/oxidized/model/opengear.rb)
- * [OPNsense](/lib/oxidized/model/opnsense.rb)
- * Palo Alto
-   * [PANOS](/lib/oxidized/model/panos.rb)
- * [PLANET SG/SGS Switches](/lib/oxidized/model/planet.rb)
- * [pfSense](/lib/oxidized/model/pfsense.rb)
- * Radware
-   * [AlteonOS](/lib/oxidized/model/alteonos.rb)
- * Quanta
-   * [Quanta / VxWorks 6.6 (1.1.0.8)](/lib/oxidized/model/quantaos.rb)
- * Siklu
-   * [EtherHaul](/lib/oxidized/model/siklu.rb)
- * Supermicro
-   * [Supermicro](/lib/oxidized/model/supermicro.rb)
- * Symantec
-   * [Blue Coat ProxySG / Security Gateway OS (SGOS)](/lib/oxidized/model/sgos.rb)
- * Trango Systems
-   * [Trango](/lib/oxidized/model/trango.rb)
- * TPLink
-   * [TPLink](/lib/oxidized/model/tplink.rb)
- * Ubiquiti
-   * [AirOS](/lib/oxidized/model/airos.rb)
-   * [Edgeos](/lib/oxidized/model/edgeos.rb)
-   * [EdgeSwitch](/lib/oxidized/model/edgeswitch.rb)
- * Watchguard
-   * [Fireware OS](/lib/oxidized/model/firewareos.rb)
- * Westell
-   * [Westell 8178G, Westell 8266G](/lib/oxidized/model/weos.rb)
- * Zhone
-   * [Zhone (OLT and MX)](/lib/oxidized/model/zhoneolt.rb)
- * Zyxel
-   * [ZyNOS](/lib/oxidized/model/zynos.rb)
+
+* Vendor
+  * OS model
+* A10 Networks
+  * [ACOS](/lib/oxidized/model/acos.rb)
+* Accedian Performance Elements (NIDs)
+  * [AEN](/lib/oxidized/model/aen.rb)
+* Alcatel-Lucent
+  * [AOS](/lib/oxidized/model/aos.rb)
+  * [AOS7](/lib/oxidized/model/aos7.rb)
+  * [ISAM](/lib/oxidized/model/isam.rb)
+  * [SR OS (Formerly TiMOS)](/lib/oxidized/model/timos.rb)
+  * Wireless
+* Allied Telesis
+  * [Alliedware Plus](/lib/oxidized/model/awplus.rb)
+* Alvarion
+  * [BreezeACCESS](/lib/oxidized/model/alvarion.rb)
+* APC
+  * [AOS](/lib/oxidized/model/apc_aos.rb)
+* Arbor Networks
+  * [ArbOS](/lib/oxidized/model/arbos.rb)
+* Arista
+  * [EOS](/lib/oxidized/model/eos.rb)
+* Arris
+  * [C4CMTS](/lib/oxidized/model/c4cmts.rb)
+* Aruba
+  * [AOSW](/lib/oxidized/model/aosw.rb)
+* AudioCodes
+  * [AudioCodes](/lib/oxdized/model/audiocodes.rb)
+* Avaya
+  * [VOSS (VSP Operating System Software)](/lib/oxidized/model/voss.rb)
+  * [BOSS (Baystack Operating System Software)](/lib/oxidized/model/boss.rb)
+* Brocade
+  * [FabricOS](lib/oxidized/model/fabricos.rb)
+  * [Ironware](lib/oxidized/model/ironware.rb)
+  * [NOS (Network Operating System)](lib/oxidized/model/nos.rb)
+  * [Vyatta](lib/oxidized/model/vyatta.rb)
+  * [6910](lib/oxidized/model/br6910.rb)
+  * [SLX-OS](lib/oxidized/model/slxos.rb)
+* Casa
+  * [Casa](/lib/oxidized/model/casa.rb)
+* Check Point
+  * [GaiaOS](/lib/oxidized/model/gaiaos.rb)
+* Ciena
+  * [SAOS](/lib/oxidized/model/saos.rb)
+* Cisco
+  * [ACSW](/lib/oxidized/model/acsw.rb)
+  * [AireOS](/lib/oxidized/model/aireos.rb)
+  * [ASA](/lib/oxidized/model/asa.rb)
+  * [AsyncOS](/lib/oxidized/model/asyncos.rb)
+  * [CatOS](/lib/oxidized/model/catos.rb)
+  * [IOS](/lib/oxidized/model/ios.rb)
+  * [IOSXR](/lib/oxidized/model/iosxr.rb)
+  * [NGA](/lib/oxidized/model/cisconga.rb)
+  * [NXOS](/lib/oxidized/model/nxos.rb)
+  * [SMA](/lib/oxidized/model/ciscosma.rb)
+  * [SMB (Nikola series)](/lib/oxidized/model/ciscosmb.rb)
+  * [UCS](/lib/oxidized/model/ucs.rb)
+* Citrix
+  * [NetScaler (Virtual Applicance)](/lib/oxidized/model/netscaler.rb)
+* Coriant (former Tellabs)
+  * [TMOS (8800)](/lib/oxidized/model/corianttmos.rb)
+  * [8600](/lib/oxidized/model/coriant8600.rb)
+  * [Groove](/lib/oxidized/model/coriantgroove.rb)
+* Cumulus
+  * [Linux](/lib/oxidized/model/cumulus.rb)
+* DataCom
+  * [DmSwitch 3000](/lib/oxidized/model/datacom.rb)
+* DCN
+  * [DCN](/lib/oxidized/model/ios.rb) - Map this to ios.
+* DELL
+  * [PowerConnect](/lib/oxidized/model/powerconnect.rb)
+  * [AOSW](/lib/oxidized/model/aosw.rb)
+* D-Link
+  * [D-Link](/lib/oxidized/model/dlink.rb)
+* Ericsson/Redback
+  * [IPOS (former SEOS)](/lib/oxidized/model/ipos.rb)
+* Extreme Networks
+  * [Enterasys](/lib/oxidized/model/enterasys.rb)
+  * [WM](/lib/oxidized/model/mtrlrfs.rb)
+  * [XOS](/lib/oxidized/model/xos.rb)
+* F5
+  * [TMOS](/lib/oxidized/model/tmos.rb)
+* Fiberstore
+  * [S3800](/lib/oxidized/model/gcombnps.rb)
+* Force10
+  * [DNOS](/lib/oxidized/model/dnos.rb)
+  * [FTOS](/lib/oxidized/model/ftos.rb)
+* FortiGate
+  * [FortiOS](/lib/oxidized/model/fortios.rb)
+* Fujitsu
+  * [PRIMERGY Blade switch 1/10Gbe](/lib/oxidized/model/fujitsupy.rb)
+* GCOM Technologies
+  * [Broadband Network Platform Software](/lib/oxidized/model/gcombnps.rb)
+* Hatteras
+  * [Hatteras](/lib/oxidized/model/hatteras.rb)
+* Hirschmann
+  * [HiOS](/lib/oxidized/model/hirschmann.rb)
+* HP
+  * [Comware (HP A-series, H3C, 3Com)](/lib/oxidized/model/comware.rb)
+  * [Procurve](/lib/oxidized/model/procurve.rb)
+  * [BladeSystem (Onboard Administrator)](/lib/oxidized/model/hpebladesystem.rb)
+  * [MSA](/lib/oxidized/model/hpemsa.rb)
+* Huawei
+  * [VRP](/lib/oxidized/model/vrp.rb)
+* Juniper
+  * [JunOS](/lib/oxidized/model/junos.rb)
+  * [ScreenOS (Netscreen)](/lib/oxidized/model/screenos.rb)
+* Mellanox
+  * [MLNX-OS](/lib/oxidized/model/mlnxos.rb)
+  * [Voltaire](/lib/oxidized/model/voltaire.rb)
+* Mikrotik
+  * [RouterOS](/lib/oxidized/model/routeros.rb)
+* Motorola
+  * [RFS](/lib/oxidized/model/mtrlrfs.rb)
+* MRV
+  * [MasterOS](/lib/oxidized/model/masteros.rb)
+  * [FiberDriver](/lib/oxidized/model/fiberdriver.rb)
+* Netgear
+  * [Netgear](/lib/oxidized/model/netgear.rb)
+* Netonix
+  * [WISP Switch (As Netonix)](/lib/oxidized/model/netonix.rb)
+* Nokia (formerly TiMetra, Alcatel, Alcatel-Lucent)
+  * [SR OS (TiMOS)](/lib/oxidized/model/timos.rb)
+* OneAccess
+  * [OneOS](/lib/oxidized/model/oneos.rb)
+* Opengear
+  * [Opengear](/lib/oxidized/model/opengear.rb)
+* [OPNsense](/lib/oxidized/model/opnsense.rb)
+* Palo Alto
+  * [PANOS](/lib/oxidized/model/panos.rb)
+* [PLANET SG/SGS Switches](/lib/oxidized/model/planet.rb)
+* [pfSense](/lib/oxidized/model/pfsense.rb)
+* Radware
+  * [AlteonOS](/lib/oxidized/model/alteonos.rb)
+* Quanta
+  * [Quanta / VxWorks 6.6 (1.1.0.8)](/lib/oxidized/model/quantaos.rb)
+* Siklu
+  * [EtherHaul](/lib/oxidized/model/siklu.rb)
+* Supermicro
+  * [Supermicro](/lib/oxidized/model/supermicro.rb)
+* Symantec
+  * [Blue Coat ProxySG / Security Gateway OS (SGOS)](/lib/oxidized/model/sgos.rb)
+* Trango Systems
+  * [Trango](/lib/oxidized/model/trango.rb)
+* TPLink
+  * [TPLink](/lib/oxidized/model/tplink.rb)
+* Ubiquiti
+  * [AirOS](/lib/oxidized/model/airos.rb)
+  * [Edgeos](/lib/oxidized/model/edgeos.rb)
+  * [EdgeSwitch](/lib/oxidized/model/edgeswitch.rb)
+* Watchguard
+  * [Fireware OS](/lib/oxidized/model/firewareos.rb)
+* Westell
+  * [Westell 8178G, Westell 8266G](/lib/oxidized/model/weos.rb)
+* Zhone
+  * [Zhone (OLT and MX)](/lib/oxidized/model/zhoneolt.rb)
+* Zyxel
+  * [ZyNOS](/lib/oxidized/model/zynos.rb)