diff options
Diffstat (limited to 'docs')
-rw-r--r-- | docs/Configuration.md | 216 | ||||
-rw-r--r-- | docs/Creating-Models.md | 78 | ||||
-rw-r--r-- | docs/Hooks.md | 247 | ||||
-rw-r--r-- | docs/Model-Notes/AireOS.md | 12 | ||||
-rw-r--r-- | docs/Model-Notes/ArbOS.md | 12 | ||||
-rw-r--r-- | docs/Model-Notes/Comware.md | 14 | ||||
-rw-r--r-- | docs/Model-Notes/EOS.md | 9 | ||||
-rw-r--r-- | docs/Model-Notes/JunOS.md | 34 | ||||
-rw-r--r-- | docs/Model-Notes/Netgear.md | 68 | ||||
-rw-r--r-- | docs/Model-Notes/README.md | 19 | ||||
-rw-r--r-- | docs/Model-Notes/VRP-Huawei.md | 35 | ||||
-rw-r--r-- | docs/Model-Notes/XGS4600-Zyxel.md | 39 | ||||
-rw-r--r-- | docs/Outputs.md | 189 | ||||
-rw-r--r-- | docs/Ruby-API.md | 135 | ||||
-rw-r--r-- | docs/Sources.md | 172 | ||||
-rw-r--r-- | docs/Supported-OS-Types.md | 166 |
16 files changed, 1445 insertions, 0 deletions
diff --git a/docs/Configuration.md b/docs/Configuration.md new file mode 100644 index 0000000..1d16c91 --- /dev/null +++ b/docs/Configuration.md @@ -0,0 +1,216 @@ +# Configuration + +## Debugging + +In case a model plugin doesn't work correctly (ios, procurve, etc.), you can enable live debugging of SSH/Telnet sessions. Just add a `debug` option containing the value true to the `input` section. The log files will be created depending on the parent directory of the logfile option. + +The following example will log an active ssh/telnet session `/home/oxidized/.config/oxidized/log/<IP-Address>-<PROTOCOL>`. The file will be truncated on each consecutive ssh/telnet session, so you need to put a `tailf` or `tail -f` on that file! + +```yaml +log: /home/oxidized/.config/oxidized/log + +... + +input: + default: ssh, telnet + debug: true + ssh: + secure: false +``` + +## Privileged mode + +To start privileged mode before pulling the configuration, Oxidized needs to send the enable command. You can globally enable this, by adding the following snippet to the global section of the configuration file. + +```yaml +vars: + enable: S3cre7 +``` + +## Removing secrets + +To strip out secrets from configurations before storing them, Oxidized needs the `remove_secret` flag. You can globally enable this by adding the following snippet to the global section of the configuration file. + +```yaml +vars: + remove_secret: true +``` + +Device models that contain substitution filters to remove sensitive data will now be run on any fetched configuration. + +As a partial example from ios.rb: + +```ruby + cmd :secret do |cfg| + cfg.gsub! /^(snmp-server community).*/, '\\1 <configuration removed>' + (...) + cfg + end +``` + +The above strips out snmp community strings from your saved configs. + +**NOTE:** Removing secrets reduces the usefulness as a full configuration backup, but it may make sharing configs easier. + +## Disabling SSH exec channels + +Oxidized uses exec channels to make information extraction simpler, but there are some situations where this doesn't work well, e.g. configuring devices. This feature can be turned off by setting the `ssh_no_exec` +variable. + +```yaml +vars: + ssh_no_exec: true +``` + +## SSH Auth Methods + +By default, Oxidized registers the following auth methods: `none`, `publickey` and `password`. However you can configure this globally, by groups, models or nodes. + +``` +vars: + auth_methods: none, publickey, password, keyboard-interactive +``` + +## SSH Proxy Command + +Oxidized can `ssh` through a proxy as well. To do so we just need to set `ssh_proxy` variable with the proxy host information. + +This can be provided on a per-node basis by mapping the proper fields from your source. + +An example for a `csv` input source that maps the 4th field as the `ssh_proxy` value. + +```yaml +... +map: + name: 0 + model: 1 +vars_map: + enable: 2 + ssh_proxy: 3 +... +``` + +## FTP Passive Mode + +Oxidized uses ftp passive mode by default. Some devices require passive mode to be disabled. To do so, we can set `input.ftp.passive` to false - this will make use of FTP active mode. + +```yaml +input: + ftp: + passive: false +``` + +## Advanced Configuration + +Below is an advanced example configuration. You will be able to (optionally) override options per device. The router.db format used is `hostname:model:username:password:enable_password`. Hostname and model will be the only required options, all others override the global configuration sections. + +```yaml +--- +username: oxidized +password: S3cr3tx +model: junos +interval: 3600 #interval in seconds +log: ~/.config/oxidized/log +debug: false +threads: 30 +timeout: 20 +retries: 3 +prompt: !ruby/regexp /^([\w.@-]+[#>]\s?)$/ +vars: + enable: S3cr3tx +groups: {} +rest: 127.0.0.1:8888 +pid: ~/.config/oxidized/oxidized.pid +input: + default: ssh, telnet + debug: false + ssh: + secure: false +output: + default: git + git: + user: Oxidized + email: oxidized@example.com + repo: "~/.config/oxidized/oxidized.git" +source: + default: csv + csv: + file: ~/.config/oxidized/router.db + delimiter: !ruby/regexp /:/ + map: + name: 0 + model: 1 + username: 2 + password: 3 + vars_map: + enable: 4 +model_map: + cisco: ios + juniper: junos +``` + +## Advanced Group Configuration + +For group specific credentials + +```yaml +groups: + mikrotik: + username: admin + password: blank + ubiquiti: + username: ubnt + password: ubnt +``` + +and add group mapping + +```yaml +map: + model: 0 + name: 1 + group: 2 +``` + +For model specific credentials + +```yaml +models: + junos: + username: admin + password: password + ironware: + username: admin + password: password + vars: + enable: enablepassword + apc_aos: + username: apc + password: password +``` + +## RESTful API and Web Interface + +The RESTful API and Web Interface is enabled by configuring the `rest:` parameter in the config file. This parameter can optionally contain a relative URI. + +```yaml +# Listen on http://127.0.0.1:8888/ +rest: 127.0.0.1:8888 +``` + +```yaml +# Listen on http://10.0.0.1:8000/oxidized/ +rest: 10.0.0.1:8000/oxidized +``` + +## Triggered backups + +A node can be moved to head-of-queue via the REST API `GET/POST /node/next/[NODE]`. This can be useful to immediately schedule a fetch of the configuration after some other event such as a syslog message indicating a configuration update on the device. + +In the default configuration this node will be processed when the next job worker becomes available, it could take some time if existing backups are in progress. To execute moved jobs immediately a new job can be added automatically: + +```yaml +next_adds_job: true +``` + +This will allow for a more timely fetch of the device configuration. diff --git a/docs/Creating-Models.md b/docs/Creating-Models.md new file mode 100644 index 0000000..3cb87c3 --- /dev/null +++ b/docs/Creating-Models.md @@ -0,0 +1,78 @@ +# Creating and Extending Models + +Oxidized supports a growing list of [operating system types](Supported-OS-Types.md). Out of the box, most model implementations collect configuration data. Some implementations also include a conservative set of additional commands that collect basic device information (device make and model, software version, licensing information, ...) which are appended to the configuration as comments. + +A user may wish to extend an existing model to collect the output of additional commands. Oxidized offers smart loading of models in order to facilitate this with ease, without the need to introduce changes to the upstream source code. + +This methodology allows local site changes to be preserved during Oxidized version updates / gem updates. + +## Extending an existing model with a new command + +The example below can be used to extend the `JunOS` model to collect the output of `show interfaces diagnostics optics` and append the output to the configuration file as a comment. This command retrieves DOM information on pluggable optics present in a `JunOS`-powered chassis. + +Create the file `~/.config/oxidized/model/junos.rb` with the following contents: + +```ruby +require 'oxidized/model/junos.rb' + + +class JunOS + + + cmd 'show interfaces diagnostics optics' do |cfg| + comment cfg + end + + +end +``` + +Due to smart loading, the user-supplied `~/.config/oxidized/model/junos.rb` file will take precedence over the model with the same name included in the Oxidized distribution. + +The code then uses `require` to initially load the Oxidized-supplied model, and extends the class defined therein with an additional command. + +Intuitively, it is also possible to: + +* Completely re-define an existing model by creating a file in `~/.config/oxidized/model/` with the same name as an existing model, but not `require`-ing the upstream model file. +* Create a named variation of an existing model, by creating a file with a new name (such as `~/.config/oxidized/model/junos-extra.rb`), Then `require` the original model and extend its base class as in the above example. The named variation can then be specified as an OS type for specific devices that can benefit from the extra functionality. This allows for preservation of the base functionality for the default model types. +* Create a completely new model, with a new name, for a new operating system type. +* Testing/validation of an updated model from the [Oxidized GitHub repo models](https://github.com/ytti/oxidized/tree/master/lib/oxidized/model) by placing an updated model in the proper location without disrupting the gem-supplied model files. + +## Advanced features + +The loosely-coupled architecture of Oxidized allows for easy extensibility in more advanced use cases as well. + +The example below extends the functionality of the `JunOS` model further to collect `display set` formatted configuration from the device, and utilizes the multi-output functionality of the `git` output to place the returned configuration in a separate file within a git repository. + +It is possible to configure the `git` output to create new subdirectories under an existing repository instead of creating new repositories for each new defined output type (the default) by including the following configuration in the `~/.config/oxidized/config` file: + +```yaml +output: + git: + type_as_directory: true +``` + +Then, `~/.config/oxidized/model/junos.rb` is adapted as following: + +```ruby +require 'oxidized/model/junos.rb' + + +class JunOS + + + cmd 'show interface diagnostic optics' do |cfg| + comment cfg + end + + cmd 'show configuration | display set' do |cfg| + cfg.type = "junos-set" + cfg.name = "set" + cfg + end + + +end +``` + +The output of the `show configuration | display set` command is marked with a new arbitrary alternative output type, `junos-set`. The `git` output will use the output type to create a new subdirectory by the same name. In this subdirectory, the `git` output will create files with the name `<devicename>--set` that will contain the output of this command for each device. diff --git a/docs/Hooks.md b/docs/Hooks.md new file mode 100644 index 0000000..09b6117 --- /dev/null +++ b/docs/Hooks.md @@ -0,0 +1,247 @@ +# Hooks + +You can define an arbitrary number of hooks that subscribe to different events. The hook system is modular and different kind of hook types can be enabled. + +## Configuration + +Following configuration keys need to be defined for all hooks: + +* `events`: which events to subscribe. Needs to be an array. See below for the list of available events. +* `type`: what hook class to use. See below for the list of available hook types. + +## Events + +* `node_success`: triggered when configuration is successfully pulled from a node and right before storing the configuration. +* `node_fail`: triggered after `retries` amount of failed node pulls. +* `post_store`: triggered after node configuration is stored (this is executed only when the configuration has changed). +* `nodes_done`: triggered after finished fetching all nodes. + +## Hook type: exec + +The `exec` hook type allows users to run an arbitrary shell command or a binary when triggered. + +The command is executed on a separate child process either in synchronous or asynchronous fashion. Non-zero exit values cause errors to be logged. STDOUT and STDERR are currently not collected. + +Command is executed with the following environment: + +```text +OX_EVENT +OX_NODE_NAME +OX_NODE_IP +OX_NODE_FROM +OX_NODE_MSG +OX_NODE_GROUP +OX_NODE_MODEL +OX_JOB_STATUS +OX_JOB_TIME +OX_REPO_COMMITREF +OX_REPO_NAME +``` + +Exec hook recognizes the following configuration keys: + +* `timeout`: hard timeout (in seconds) for the command execution. SIGTERM will be sent to the child process after the timeout has elapsed. Default: `60` +* `async`: Execute the command in an asynchronous fashion. The main thread by default will wait for the hook command execution to complete. Set this to `true` for long running commands so node configuration pulls are not blocked. Default: `false` +* `cmd`: command to run. + +### exec hook configuration example + +```yaml +hooks: + name_for_example_hook1: + type: exec + events: [node_success] + cmd: 'echo "Node success $OX_NODE_NAME" >> /tmp/ox_node_success.log' + name_for_example_hook2: + type: exec + events: [post_store, node_fail] + cmd: 'echo "Doing long running stuff for $OX_NODE_NAME" >> /tmp/ox_node_stuff.log; sleep 60' + async: true + timeout: 120 +``` + +## Hook type: githubrepo + +The `githubrepo` hook executes a `git push` to a configured `remote_repo` when the specified event is triggered. + +Several authentication methods are supported: + +* Provide a `password` for username + password authentication +* Provide both a `publickey` and a `privatekey` for ssh key-based authentication +* Don't provide any credentials for ssh-agent authentication + +The username will be set to the relevant part of the `remote_repo` URI, with a fallback to `git`. It is also possible to provide one by setting the `username` configuration key. + +For ssh key-based authentication, it is possible to set the environment variable `OXIDIZED_SSH_PASSPHRASE` to a passphrase if the private key requires it. + +`githubrepo` hook recognizes the following configuration keys: + +* `remote_repo`: the remote repository to be pushed to. +* `username`: username for repository auth. +* `password`: password for repository auth. +* `publickey`: public key file path for repository auth. +* `privatekey`: private key file path for repository auth. + +When using groups, each group must have a unique entry in the `remote_repo` config. + +```yaml +hooks: + push_to_remote: + remote_repo: + routers: git@git.intranet:oxidized/routers.git + switches: git@git.intranet:oxidized/switches.git + firewalls: git@git.intranet:oxidized/firewalls.git +``` + +### githubrepo hook configuration example + +Authenticate with a username and a password without groups in use: + +```yaml +hooks: + push_to_remote: + type: githubrepo + events: [post_store] + remote_repo: git@git.intranet:oxidized/test.git + username: user + password: pass +``` + +Authenticate with the username `git` and an ssh key: + +```yaml +hooks: + push_to_remote: + type: githubrepo + events: [post_store] + remote_repo: git@git.intranet:oxidized/test.git + publickey: /root/.ssh/id_rsa.pub + privatekey: /root/.ssh/id_rsa +``` + +## Hook type: awssns + +The `awssns` hook publishes messages to AWS SNS topics. This allows you to notify other systems of device configuration changes, for example a config orchestration pipeline. Multiple services can subscribe to the same AWS topic. + +Fields sent in the message: + +* `event`: Event type (e.g. `node_success`) +* `group`: Group name +* `model`: Model name (e.g. `eos`) +* `node`: Device hostname + + +The AWS SNS hook requires the following configuration keys: + +* `region`: AWS Region name +* `topic_arn`: ASN Topic reference +### awssns hook configuration example + +```yaml +hooks: + hook_script: + type: awssns + events: [node_fail,node_success,post_store] + region: us-east-1 + topic_arn: arn:aws:sns:us-east-1:1234567:oxidized-test-backup_events +``` + +Your AWS credentials should be stored in `~/.aws/credentials`. + +## Hook type: slackdiff + +The `slackdiff` hook posts colorized config diffs to a [Slack](http://www.slack.com) channel of your choice. It only triggers for `post_store` events. + +You will need to manually install the `slack-api` gem on your system: + +```shell +gem install slack-api +``` + +### slackdiff hook configuration example + +```yaml +hooks: + slack: + type: slackdiff + events: [post_store] + token: SLACK_BOT_TOKEN + channel: "#network-changes" +``` + +Optionally you can disable snippets and post a formatted message, for instance linking to a commit in a git repo. Named parameters `%{node}`, `%{group}`, `%{model}` and `%{commitref}` are available. + +```yaml +hooks: + slack: + type: slackdiff + events: [post_store] + token: SLACK_BOT_TOKEN + channel: "#network-changes" + diff: false + message: "%{node} %{group} %{model} updated https://git.intranet/network-changes/commit/%{commitref}" +``` + +Note the channel name must be in quotes. + +## Hook type: ciscosparkdiff + +The `ciscosparkdiff` hook posts config diffs to a [Cisco Spark](https://www.ciscospark.com/) space of your choice. It only triggers for `post_store` events. + +You will need to manually install the `cisco_spark` gem on your system (see [cisco_spark-ruby](https://github.com/NGMarmaduke/cisco_spark-ruby)) and generate either a [Bot or OAUTH access key](https://developer.ciscospark.com/apps.html), and retrieve the [Spark Space ID](https://developer.ciscospark.com/endpoint-rooms-get.html) + +```shell +gem install cisco_spark +``` + +### ciscosparkdiff hook configuration example + +```yaml +hooks: + ciscospark: + type: ciscosparkdiff + events: [post_store] + accesskey: SPARK_BOT_API_OR_OAUTH_KEY + space: SPARK_SPACE_ID + diff: true +``` + +Optionally you can disable snippets and post a formatted message, for instance linking to a commit in a git repo. Named parameters `%{node}`, `%{group}`, `%{model}` and `%{commitref}` are available. + +```yaml +hooks: + ciscospark: + type: ciscosparkdiff + events: [post_store] + accesskey: SPARK_BOT_API_OR_OAUTH_KEY + space: SPARK_SPACE_ID + diff: false + message: "%{node} %{group} %{model} updated https://git.intranet/network-changes/commit/%{commitref}" +``` + +Note the space and access tokens must be in quotes. + +## Hook type: xmppdiff + +The `xmppdiff` hook posts config diffs to a [XMPP](https://en.wikipedia.org/wiki/XMPP) chatroom of your choice. It only triggers for `post_store` events. + +You will need to manually install the `xmpp4r` gem on your system: + +```shell +gem install xmpp4r +``` + +### xmppdiff hook configuration example + +```yaml +hooks: + xmpp: + type: xmppdiff + events: [post_store] + jid: "user@server.tld/resource" + password: "password" + channel: "room@server.tld" + nick: "nickname" +``` + +Note the channel name must be in quotes. diff --git a/docs/Model-Notes/AireOS.md b/docs/Model-Notes/AireOS.md new file mode 100644 index 0000000..5674ae2 --- /dev/null +++ b/docs/Model-Notes/AireOS.md @@ -0,0 +1,12 @@ +Cisco WLC Configuration +======================= + +Create a user with read-write privilege: + +```text +mgmtuser add oxidized **** read-write +``` + +Oxidized needs read-write privilege in order to execute 'config paging disable'. + +Back to [Model-Notes](README.md) diff --git a/docs/Model-Notes/ArbOS.md b/docs/Model-Notes/ArbOS.md new file mode 100644 index 0000000..ebac997 --- /dev/null +++ b/docs/Model-Notes/ArbOS.md @@ -0,0 +1,12 @@ +Arbor Networks ArbOS notes +========================== + +If you are running ArbOS version 7 or lower then you may need to update the model to remove `exec true`: + +```ruby + cfg :ssh do + pre_logout 'exit' + end +``` + +Back to [Model-Notes](README.md) diff --git a/docs/Model-Notes/Comware.md b/docs/Model-Notes/Comware.md new file mode 100644 index 0000000..048f312 --- /dev/null +++ b/docs/Model-Notes/Comware.md @@ -0,0 +1,14 @@ +Comware Configuration +===================== + +If you find 3Com Comware devices aren't being backed up this may be due to prompt detection not matching because a previous login message is disabled after the first prompt. + +You can disable this on the devices themselves by running this command: + +```text +info-center source default channel 1 log state off debug state off +``` + +[Reference](https://github.com/ytti/oxidized/issues/1171) + +Back to [Model-Notes](README.md) diff --git a/docs/Model-Notes/EOS.md b/docs/Model-Notes/EOS.md new file mode 100644 index 0000000..66287e3 --- /dev/null +++ b/docs/Model-Notes/EOS.md @@ -0,0 +1,9 @@ +Arista EOS Configuration +======================== + +By default EOS requires keyboard-interactive to be added to your Oxidized config. You can avoid having to do this by configuring the following on the EOS device: + +``` +management ssh + authentication mode password +``` diff --git a/docs/Model-Notes/JunOS.md b/docs/Model-Notes/JunOS.md new file mode 100644 index 0000000..8093df0 --- /dev/null +++ b/docs/Model-Notes/JunOS.md @@ -0,0 +1,34 @@ +JunOS Configuration +=================== + +Create login class cfg-view + +```text +set system login class cfg-view permissions view-configuration +set system login class cfg-view allow-commands "(show)|(set cli screen-length)|(set cli screen-width)" +set system login class cfg-view deny-commands "(clear)|(file)|(file show)|(help)|(load)|(monitor)|(op)|(request)|(save)|(set)|(start)|(test)" +set system login class cfg-view deny-configuration all +``` + +Create a user with cfg-view class + +```text +set system login user oxidized class cfg-view +set system login user oxidized authentication plain-text-password "verysecret" +``` + +The commands Oxidized executes are: + +1. set cli screen-length 0 +2. set cli screen-width 0 +3. show configuration +4. show version +5. show chassis hardware +6. show system license +7. show system license keys (ex22|ex33|ex4|ex8|qfx only) +8. show virtual-chassis (MX960 only) +9. show chassis fabric reachability + +Oxidized can now retrieve your configuration! + +Back to [Model-Notes](README.md) diff --git a/docs/Model-Notes/Netgear.md b/docs/Model-Notes/Netgear.md new file mode 100644 index 0000000..d82bdcc --- /dev/null +++ b/docs/Model-Notes/Netgear.md @@ -0,0 +1,68 @@ +Netgear Configuration +===================== + +There are several models available with CLI management via telnet (port 60000), but they all behave like one of the following: +- older models: +``` +Connected to 192.168.3.201. + +(GS748Tv4) +Applying Interface configuration, please wait ...admin +Password:******** +(GS748Tv4) >enable +Password: + +(GS748Tv4) #terminal length 0 + +(GS748Tv4) #show running-config +``` + +- newer models: +``` +Connected to 172.0.3.203. + +User:admin +Password:******** +(GS724Tv4) >enable + +(GS724Tv4) #terminal length 0 + +(GS724Tv4) #show running-config +``` + +The main differences are: +- the prompt for username is different (looks quite strange for older models) +- enable password + - the older model prompts for enable password and it expects empty string + - the newer model does not prompt for enable password at all + +Configuration for older/newer models: make sure you have defined variable 'enable': +- `'true'` for newer models +- `''` empty string: for older models + +One possible configuration: +- oxidized config +```yaml +source: + default: csv + csv: + file: "/home/oxidized/.config/oxidized/router.db" + delimiter: !ruby/regexp /:/ + map: + name: 0 + model: 1 + username: 2 + password: 3 + vars_map: + enable: 4 + telnet_port: 5 +``` +- router.db +``` +switchOldFW:netgear:admin:adminpw::60000 +switchNewFW:netgear:admin:adminpw:true:60000 +``` + +[Reference](https://github.com/ytti/oxidized/pull/1268) + +Back to [Model-Notes](README.md) diff --git a/docs/Model-Notes/README.md b/docs/Model-Notes/README.md new file mode 100644 index 0000000..993eb77 --- /dev/null +++ b/docs/Model-Notes/README.md @@ -0,0 +1,19 @@ +Model Notes +=========== + +This directory contains implementation notes and caveats to assist you in your oxidized deployment. + +Use the table below for more information on the Vendor/Model caveats. + +Vendor | Model |Updated +----------------|-----------------|---------------- +3COM|[Comware](Comware.md)|15 Feb 2018 +AireOS|[AireOS](AireOS.md)|29 Nov 2017 +Arbor Networks|[ArbOS](ArbOS.md)|27 Feb 2018 +Arista|[EOS](EOS.md)|05 Feb 2018 +Huawei|[VRP](VRP-Huawei.md)|17 Nov 2017 +Juniper|[MX/QFX/EX/SRX/J Series](JunOS.md)|18 Jan 2018 +Netgear|[Netgear](Netgear.md)|11 Apr 2018 +Zyxel|[XGS4600 Series](XGS4600-Zyxel.md)|1 Feb 2018 + +If you discover additional caveats or problems please make sure to consult the [GitHub issues for oxidized](https://github.com/ytti/oxidized/issues) known issues. diff --git a/docs/Model-Notes/VRP-Huawei.md b/docs/Model-Notes/VRP-Huawei.md new file mode 100644 index 0000000..ff5426e --- /dev/null +++ b/docs/Model-Notes/VRP-Huawei.md @@ -0,0 +1,35 @@ +Huawei VRP Configuration +======================== + +Create a user with no privileges + +```text + <HUAWEI> system-view + [~HUAWEI] aaa + [~HUAWEI-aaa] local-user oxidized password irreversible-cipher verysecret + [*HUAWEI-aaa] local-user oxidized level 1 + [*HUAWEI-aaa] local-user oxidized service-type terminal ssh + [*HUAWEI-aaa] commit +``` + +The commands Oxidized executes are: + +1. screen-length 0 temporary +2. display version +3. display device +4. display current-configuration all + +Command 2 and 3 can be executed without issues, but 1 and 4 are only available for higher level users. Instead of making Oxidized a read/write user on your device, lower the privilege-level for commands 1 and 4: + +```text + <HUAWEI> system-view + [~HUAWEI] command-privilege level 1 view global display current-configuration all + [*HUAWEI] command-privilege level 1 view shell screen-length + [*HUAWEI] commit +``` + +Oxidized can now retrieve your configuration! + +Caveat: Some versions of VRP default to appending a timestamp prior to the output of each `display` command, which will lead to superfluous updates. The configuration statement `timestamp disable` can be used to disable this functionality. (Issue #1218) + +Back to [Model-Notes](README.md) diff --git a/docs/Model-Notes/XGS4600-Zyxel.md b/docs/Model-Notes/XGS4600-Zyxel.md new file mode 100644 index 0000000..8b58ed8 --- /dev/null +++ b/docs/Model-Notes/XGS4600-Zyxel.md @@ -0,0 +1,39 @@ +ZynOS Configuration +=================== + +## FTP + +FTP access is only possible as admin, other users can login but cannot pull the files. +For the XGS4600 series the config file is _config_ and not _config-0_ + +The following line in _oxidized/lib/oxidized/model/zynos.rb_ will need changing + +```text + cmd 'config-0' +``` + +The inclusion of an extra ftp option is also require. Within _input_ add the following + +```yaml +input: + ftp: + passive: false +``` + + +## SSH/TelNet + +Below is the table from the XGS4600 CLI Reference Guide (Version 3.79~4.50 Edition 1, 07/2017) +Take this table with a pinch of salt, level 3 will not allow _show running-config_! + +Privilege Level | Types of commands at this privilege level +----------------|------------------------------------------- +0|Display basic system information. +3|Display configuration or status. +13|Configure features except for login accounts, SNMP user accounts, the authentication method sequence and authorization settings, multiple logins, administrator and enable passwords, and configuration information display. +14|Configure login accounts, SNMP user accounts, the authentication method sequence and authorization settings, multiple logins, and administrator and enable passwords, and display configuration information. + + +Oxidized can now retrieve your configuration! + +Back to [Model-Notes](README.md) diff --git a/docs/Outputs.md b/docs/Outputs.md new file mode 100644 index 0000000..fab3bf8 --- /dev/null +++ b/docs/Outputs.md @@ -0,0 +1,189 @@ +# Outputs + +## Output: File + +Parent directory needs to be created manually, one file per device, with most recent running config. + +```yaml +output: + file: + directory: /var/lib/oxidized/configs +``` + +## Output: Git + +This uses the rugged/libgit2 interface. So you should remember that normal Git hooks will not be executed. + +For a single repository containing all devices: + +```yaml +output: + default: git + git: + user: Oxidized + email: o@example.com + repo: "/var/lib/oxidized/devices.git" +``` + +And for group-based repositories: + +```yaml +output: + default: git + git: + user: Oxidized + email: o@example.com + repo: "/var/lib/oxidized/git-repos/default.git" +``` + +Oxidized will create a repository for each group in the same directory as the `default.git`. For +example: + +```csv +host1:ios:first +host2:nxos:second +``` + +This will generate the following repositories: + +```bash +$ ls /var/lib/oxidized/git-repos + +default.git first.git second.git +``` + +If you would like to use groups and a single repository, you can force this with the `single_repo` config. + +```yaml +output: + default: git + git: + single_repo: true + repo: "/var/lib/oxidized/devices.git" + +``` + +## Output: Git-Crypt + +This uses the gem git and system git-crypt interfaces. Have a look at [GIT-Crypt](https://www.agwa.name/projects/git-crypt/) documentation to know how to install it. +Additionally to user and email informations, you have to provide the users ID that can be a key ID, a full fingerprint, an email address, or anything else that uniquely identifies a public key to GPG (see "HOW TO SPECIFY A USER ID" in the gpg man page). + +For a single repository containing all devices: + +```yaml +output: + default: gitcrypt + gitcrypt: + user: Oxidized + email: o@example.com + repo: "/var/lib/oxidized/devices" + users: + - "0x0123456789ABCDEF" + - "<user@example.com>" +``` + +And for group-based repositories: + +```yaml +output: + default: gitcrypt + gitcrypt: + user: Oxidized + email: o@example.com + repo: "/var/lib/oxidized/git-repos/default" + users: + - "0xABCDEF0123456789" + - "0x0123456789ABCDEF" +``` + +Oxidized will create a repository for each group in the same directory as the `default`. For +example: + +```csv +host1:ios:first +host2:nxos:second +``` + +This will generate the following repositories: + +```bash +$ ls /var/lib/oxidized/git-repos + +default.git first.git second.git +``` + +If you would like to use groups and a single repository, you can force this with the `single_repo` config. + +```yaml +output: + default: gitcrypt + gitcrypt: + single_repo: true + repo: "/var/lib/oxidized/devices" + users: + - "0xABCDEF0123456789" + - "0x0123456789ABCDEF" + +``` + +Please note that user list is only updated once at creation. + +## Output: Http + +The HTTP output will POST a config to the specified HTTP URL. Basic username/password authentication is supported. + +Example HTTP output configuration: +```yaml +output: + default: http + http: + user: admin + password: changeit + url: "http://192.168.162.50:8080/db/coll" +``` + +## Output types + +If you prefer to have different outputs in different files and/or directories, you can easily do this by modifying the corresponding model. To change the behaviour for IOS, you would edit `lib/oxidized/model/ios.rb` (run `gem contents oxidized` to find out the full file path). + +For example, let's say you want to split out `show version` and `show inventory` into separate files in a directory called `nodiff` which your tools will not send automated diffstats for. You can apply a patch along the lines of + +```text +- cmd 'show version' do |cfg| +- comment cfg.lines.first ++ cmd 'show version' do |state| ++ state.type = 'nodiff' ++ state + +- cmd 'show inventory' do |cfg| +- comment cfg ++ cmd 'show inventory' do |state| ++ state.type = 'nodiff' ++ state ++ end + +- cmd 'show running-config' do |cfg| +- cfg = cfg.each_line.to_a[3..-1].join +- cfg.gsub! /^Current configuration : [^\n]*\n/, '' +- cfg.sub! /^(ntp clock-period).*/, '! \1' +- cfg.gsub! /^\ tunnel\ mpls\ traffic-eng\ bandwidth[^\n]*\n*( ++ cmd 'show running-config' do |state| ++ state = state.each_line.to_a[3..-1].join ++ state.gsub! /^Current configuration : [^\n]*\n/, '' ++ state.sub! /^(ntp clock-period).*/, '! \1' ++ state.gsub! /^\ tunnel\ mpls\ traffic-eng\ bandwidth[^\n]*\n*( + (?:\ [^\n]*\n*)* + tunnel\ mpls\ traffic-eng\ auto-bw)/mx, '\1' +- cfg ++ state = Oxidized::String.new state ++ state.type = 'nodiff' ++ state +``` + +which will result in the following layout + +```text +diff/$FQDN--show_running_config +nodiff/$FQDN--show_version +nodiff/$FQDN--show_inventory +``` diff --git a/docs/Ruby-API.md b/docs/Ruby-API.md new file mode 100644 index 0000000..a9963b3 --- /dev/null +++ b/docs/Ruby-API.md @@ -0,0 +1,135 @@ +# Ruby API + +The following objects exist in Oxidized. + +## Input + +* gets config from nodes +* must implement 'connect', 'get', 'cmd' +* 'ssh', 'telnet', 'ftp', and 'tftp' implemented + +## Output + +* stores config +* must implement 'store' (may implement 'fetch') +* 'git' and 'file' (store as flat ascii) implemented + +## Source + +* gets list of nodes to poll +* must implement 'load' +* source can have 'name', 'model', 'group', 'username', 'password', 'input', 'output', 'prompt' for each device. + * `name` - name of the device + * `model` - model to use ('ios', 'junos', etc).The model is loaded dynamically by the first node of that model type. (Also default in config file) + * `input` - method to acquire config, loaded dynamically as needed (Also default in config file) + * `output` - method to store config, loaded dynamically as needed (Also default in config file) + * `prompt` - prompt used for node (Also default in config file, can be specified in model too) +* 'sql', 'csv' and 'http' (supports any format with single entry per line, like router.db) + +## Model + +### At the top level + +A model may use several methods at the top level in the class. `cfg` is +executed in input/output/source context. `cmd` is executed within an instance +of the model. + +#### `cfg` + +`cfg` may be called with a list of methods (`:ssh`, `:telnet`) and a block with +zero parameters. Calling `cfg` registers the given access methods and calling +it at least once is required for a model to work. + +The block may contain commands to change some behaviour for the given methods +(e.g. calling `post_login` to disable the pager). + +#### `cmd` + +Is used to specify commands that should be executed on a model in order to +gather its configuration. It can be called with: + +* Just a string +* A string and a block +* `:all` and a block +* `:secret` and a block + +The block takes a single parameter `cfg` containing the output of the command +being processed. + +Calling `cmd` with just a string will emit the output of the command given in +that string as configuration. + +Calling `cmd` with a string and a block will pass the output of the given +command to the block, then emit its return value (that must be a string) as +configuration. + +Calling `cmd` with `:all` and a block will pass all command output through this +block before emitting it. This is useful if some cleanup is required of the +output of all commands. + +Calling `cmd` with `:secret` and a block will pass all configuration to the +given block before emitting it to hide secrets if secret hiding is enabled. The +block should replace any secrets with `'<hidden>'` and return the resulting +string. + +Execution order is `:all`, `:secret`, and lastly the command specific block, if +given. + +#### `comment` + +Called with a single string containing the string to prepend for comments in +emitted configuration for this model. + +If not specified the default of `'# '` will be used (note the trailing space). + +#### `prompt` + +Is called with a regular expression that is used to detect when command output +ends after a command has been executed. + +If not specified, a default of `/^([\w.@-]+[#>]\s?)$/` is used. + +#### `expect` + +Called with a regular expression and a block. The block takes two parameters: +the regular expression, and the data containing the match. + +The passed data is replaced by the return value of the block. + +`expect` can be used to, for example, strip escape sequences from output before +it's further processed. + +### At the second level + +The following methods are available: + +#### `comment` + +Used inside `cmd` invocations. Comments out every line in the passed string and +returns the result. + +#### `password` + +Used inside `cfg` invocations to specify the regular expression used to detect +the password prompt. If not specified, the default of `/^Password/` is used. + +#### `post_login` + +Used inside `cfg` invocations to specify commands to run once Oxidized has +logged in to the device. Takes one argument that is either a block (taking zero +parameters) or a string containing a command to execute. + +This allows `post_login` to be used for any model-specific items prior to running the regular commands. This could include disabling the output pager or timestamp outputs that would cause constant differences. + +#### `pre_logout` + +Used to specify commands to run before Oxidized closes the connection to the +device. Takes one argument that is either a block (taking zero parameters) or a +string containing a command to execute. + +This allows `pre_logout` to be used to 'undo' any changes that may have been needed via `post_login` (restore pager output, etc.) + +#### `send` + +Usually used inside `expect` or blocks passed to `post_login`/`pre_logout`. +Takes a single parameter: a string to be sent to the device. diff --git a/docs/Sources.md b/docs/Sources.md new file mode 100644 index 0000000..eb3cd0c --- /dev/null +++ b/docs/Sources.md @@ -0,0 +1,172 @@ +# Sources + +## Source: CSV + +One line per device, colon separated. If `ip` isn't present, a DNS lookup will be done against `name`. For large installations, setting `ip` will dramatically reduce startup time. + +```yaml +source: + default: csv + csv: + file: /var/lib/oxidized/router.db + delimiter: !ruby/regexp /:/ + map: + name: 0 + ip: 1 + model: 2 + username: 3 + password: 4 + vars_map: + enable: 5 +``` + +Example csv `/var/lib/oxidized/router.db`: + +```text +rtr01.local:192.168.1.1:ios:oxidized:5uP3R53cR3T:T0p53cR3t +``` + +If you would like to use a GPG encrypted file as the source then you can use the following example: + +```yaml +source: + default: csv + csv: + file: ~/.config/oxidized/router.db + delimiter: !ruby/regexp /:/ + gpg: true + gpg_password: 'password' + map: + name: 0 + model: 1 +``` + +Please note, if you are running GPG v2 then you will be prompted for your gpg password on start up, if you use GPG >= 2.1 then you can add the following config to stop that behaviour: + +Within `~/.gnupg/gpg-agent.conf` + +```text +allow-loopback-pinentry +``` + +and within: `~/.gnupg/gpg.conf` + +```text +pinentry-mode loopback +``` + +## Source: SQL + + Oxidized uses the `sequel` ruby gem. You can use a variety of databases that aren't explicitly listed. For more information visit https://github.com/jeremyevans/sequel Make sure you have the correct adapter! + +**NOTE** - Many database engines have reserved keywords that may conflict with Oxidized configuration field names (such as 'name', 'group', etc). Pay attention to any names that are used and observed proper quoting methods to avoid errors or unpredictable results. + + +## Source: MYSQL + +`sudo apt-get install libmysqlclient-dev` + +The values correspond to your fields in the DB such that ip, model, etc are field names in the DB + +```yaml +source: + default: sql + sql: + adapter: mysql2 + database: oxidized + table: nodes + user: root + password: rootpass + map: + name: ip + model: model + username: username + password: password + vars_map: + enable: enable +``` + +## Source: SQLite + +One row per device, filtered by hostname. + +```yaml +source: + default: sql + sql: + adapter: sqlite + database: "/var/lib/oxidized/nodes.db" + table: nodes + map: + name: fqdn + model: model + username: username + password: password + vars_map: + enable: enable +``` + +## Custom SQL Query Support + +You may also implement a custom SQL query to retrieve the nodelist using SQL syntax with the `query:` configuration parameter under the `sql:` stanza. + +### Custom SQL Query Examples + +You may have a table named `nodes` which contains a boolean to indicate if the nodes should be enabled (fetched via oxidized). This can be used in the custom SQL query to avoid fetching from known impacted nodes. + +In your configuration, you would add the `query:` parameter and specify the SQL query. Make sure to put this within the `sql:` configuration section. + +```sql +query: "SELECT * FROM nodes WHERE enabled = True" +``` + +Since this is an SQL query, you can also provide a more advanced query to assist in more complicated oxidized deployments. The exact deployment is up to you on how you design your database and oxidized fetchers. + +In this example we limit the nodes to two "POPs" of `mypop1` and `mypop2`. We also require the nodes to have the `enabled` boolean set to `True`. + +```sql +query: "SELECT * FROM nodes WHERE pop IN ('mypop1','mypop2') AND enabled = True" +``` + +The order of the nodes returned will influence the order that nodes are fetched by oxidized. You can use standard SQL `ORDER BY` clauses to influence the node order. + +You should always test your SQL query before using it in the oxidized configuration as there is no syntax or error checking performed before sending it to the database engine. + +Consult your database documentation for more information on query language and table optimization. + +## Source: HTTP + +One object per device. + +HTTP Supports basic auth, configure the user and pass you want to use under the http: section. + +```yaml +source: + default: http + http: + url: https://url/api + scheme: https + delimiter: !ruby/regexp /:/ + user: username + pass: password + map: + name: hostname + model: os + username: username + password: password + vars_map: + enable: enable + headers: + X-Auth-Token: 'somerandomstring' +``` + +You can also pass `secure: false` if you want to disable ssl certificate verification: + +```yaml +source: + default: http + http: + url: https://url/api + scheme: https + secure: false +``` diff --git a/docs/Supported-OS-Types.md b/docs/Supported-OS-Types.md new file mode 100644 index 0000000..8839cc3 --- /dev/null +++ b/docs/Supported-OS-Types.md @@ -0,0 +1,166 @@ +# Supported OS types + +* Vendor + * OS model +* A10 Networks + * [ACOS](/lib/oxidized/model/acos.rb) +* Accedian Performance Elements (NIDs) + * [AEN](/lib/oxidized/model/aen.rb) +* Alcatel-Lucent + * [AOS](/lib/oxidized/model/aos.rb) + * [AOS7](/lib/oxidized/model/aos7.rb) + * [ISAM](/lib/oxidized/model/isam.rb) + * [SR OS (Formerly TiMOS)](/lib/oxidized/model/sros.rb) + * Wireless +* Allied Telesis + * [Alliedware Plus](/lib/oxidized/model/awplus.rb) +* Alvarion + * [BreezeACCESS](/lib/oxidized/model/alvarion.rb) +* APC + * [AOS](/lib/oxidized/model/apc_aos.rb) +* Arbor Networks + * [ArbOS](/lib/oxidized/model/arbos.rb) +* Arista + * [EOS](/lib/oxidized/model/eos.rb) +* Arris + * [C4CMTS](/lib/oxidized/model/c4cmts.rb) +* Aruba + * [AOSW](/lib/oxidized/model/aosw.rb) +* AudioCodes + * [AudioCodes](/lib/oxdized/model/audiocodes.rb) +* Avaya + * [VOSS (VSP Operating System Software)](/lib/oxidized/model/voss.rb) + * [BOSS (Baystack Operating System Software)](/lib/oxidized/model/boss.rb) +* Brocade + * [FabricOS](/lib/oxidized/model/fabricos.rb) + * [Ironware](/lib/oxidized/model/ironware.rb) + * [NOS (Network Operating System)](/lib/oxidized/model/nos.rb) + * [Vyatta](/lib/oxidized/model/vyatta.rb) + * [6910](/lib/oxidized/model/br6910.rb) + * [SLX-OS](/lib/oxidized/model/slxos.rb) +* Casa + * [Casa](/lib/oxidized/model/casa.rb) +* Check Point + * [GaiaOS](/lib/oxidized/model/gaiaos.rb) +* Ciena + * [SAOS](/lib/oxidized/model/saos.rb) +* Cisco + * [ACSW](/lib/oxidized/model/acsw.rb) + * [AireOS](/lib/oxidized/model/aireos.rb) + * [ASA](/lib/oxidized/model/asa.rb) + * [AsyncOS](/lib/oxidized/model/asyncos.rb) + * [CatOS](/lib/oxidized/model/catos.rb) + * [IOS](/lib/oxidized/model/ios.rb) + * [IOSXR](/lib/oxidized/model/iosxr.rb) + * [NGA](/lib/oxidized/model/cisconga.rb) + * [NXOS](/lib/oxidized/model/nxos.rb) + * [SMA](/lib/oxidized/model/ciscosma.rb) + * [SMB (Nikola series)](/lib/oxidized/model/ciscosmb.rb) + * [UCS](/lib/oxidized/model/ucs.rb) +* Citrix + * [NetScaler (Virtual Applicance)](/lib/oxidized/model/netscaler.rb) +* Coriant (former Tellabs) + * [TMOS (8800)](/lib/oxidized/model/corianttmos.rb) + * [8600](/lib/oxidized/model/coriant8600.rb) + * [Groove](/lib/oxidized/model/coriantgroove.rb) +* Cumulus + * [Linux](/lib/oxidized/model/cumulus.rb) +* DataCom + * [DmSwitch 3000](/lib/oxidized/model/datacom.rb) +* DCN + * [DCN](/lib/oxidized/model/ios.rb) - Map this to ios. +* DELL + * [PowerConnect](/lib/oxidized/model/powerconnect.rb) + * [AOSW](/lib/oxidized/model/aosw.rb) +* D-Link + * [D-Link](/lib/oxidized/model/dlink.rb) +* Ericsson/Redback + * [IPOS (former SEOS)](/lib/oxidized/model/ipos.rb) +* Extreme Networks + * [Enterasys](/lib/oxidized/model/enterasys.rb) + * [WM](/lib/oxidized/model/mtrlrfs.rb) + * [XOS](/lib/oxidized/model/xos.rb) +* F5 + * [TMOS](/lib/oxidized/model/tmos.rb) +* Fiberstore + * [S3800](/lib/oxidized/model/gcombnps.rb) +* Force10 + * [DNOS](/lib/oxidized/model/dnos.rb) + * [FTOS](/lib/oxidized/model/ftos.rb) +* FortiGate + * [FortiOS](/lib/oxidized/model/fortios.rb) +* Fujitsu + * [PRIMERGY Blade switch 1/10Gbe](/lib/oxidized/model/fujitsupy.rb) +* GCOM Technologies + * [Broadband Network Platform Software](/lib/oxidized/model/gcombnps.rb) +* Hatteras + * [Hatteras](/lib/oxidized/model/hatteras.rb) +* Hirschmann + * [HiOS](/lib/oxidized/model/hirschmann.rb) +* HP + * [Comware (HP A-series, H3C, 3Com)](/lib/oxidized/model/comware.rb) + * [Procurve](/lib/oxidized/model/procurve.rb) + * [BladeSystem (Onboard Administrator)](/lib/oxidized/model/hpebladesystem.rb) + * [MSA](/lib/oxidized/model/hpemsa.rb) +* Huawei + * [VRP](/lib/oxidized/model/vrp.rb) +* Juniper + * [JunOS](/lib/oxidized/model/junos.rb) + * [ScreenOS (Netscreen)](/lib/oxidized/model/screenos.rb) +* Mellanox + * [MLNX-OS](/lib/oxidized/model/mlnxos.rb) + * [Voltaire](/lib/oxidized/model/voltaire.rb) +* Mikrotik + * [RouterOS](/lib/oxidized/model/routeros.rb) +* Motorola + * [RFS](/lib/oxidized/model/mtrlrfs.rb) +* MRV + * [MasterOS](/lib/oxidized/model/masteros.rb) + * [FiberDriver](/lib/oxidized/model/fiberdriver.rb) +* Netgear + * [Netgear](/lib/oxidized/model/netgear.rb) +* Netonix + * [WISP Switch (As Netonix)](/lib/oxidized/model/netonix.rb) +* Nokia (formerly TiMetra, Alcatel, Alcatel-Lucent) + * [SR OS (TiMOS)](/lib/oxidized/model/sros.rb) +* OneAccess + * [OneOS](/lib/oxidized/model/oneos.rb) +* Opengear + * [Opengear](/lib/oxidized/model/opengear.rb) +* [OPNsense](/lib/oxidized/model/opnsense.rb) +* Palo Alto + * [PANOS](/lib/oxidized/model/panos.rb) +* [PLANET SG/SGS Switches](/lib/oxidized/model/planet.rb) +* [pfSense](/lib/oxidized/model/pfsense.rb) +* Radware + * [AlteonOS](/lib/oxidized/model/alteonos.rb) +* Quanta + * [Quanta / VxWorks 6.6 (1.1.0.8)](/lib/oxidized/model/quantaos.rb) +* Siklu + * [EtherHaul](/lib/oxidized/model/siklu.rb) +* SNR + * [SNR-S300G, S2xxx, S3xxx, S4xxx](/lib/oxidized/model/dcnos.rb) +* Supermicro + * [SSE-G2252, G2252P](/lib/oxidized/model/edgecos.rb) + * [SSE-G48-TG4, G24-TG4](/lib/oxidized/model/aricentiss.rb) + * [SSE-X24S, X24SR, X3348S, X3348SR, X3348T, X3348TR](/lib/oxidized/model/aricentiss.rb) + * [SBM-GEM-X2C, GEM-X2C+, GEM-X3S+, XEM-X10SM](/lib/oxidized/model/aricentiss.rb) +* Symantec + * [Blue Coat ProxySG / Security Gateway OS (SGOS)](/lib/oxidized/model/sgos.rb) +* Trango Systems + * [Trango](/lib/oxidized/model/trango.rb) +* TPLink + * [TPLink](/lib/oxidized/model/tplink.rb) +* Ubiquiti + * [AirOS](/lib/oxidized/model/airos.rb) + * [Edgeos](/lib/oxidized/model/edgeos.rb) + * [EdgeSwitch](/lib/oxidized/model/edgeswitch.rb) +* Watchguard + * [Fireware OS](/lib/oxidized/model/firewareos.rb) +* Westell + * [Westell 8178G, Westell 8266G](/lib/oxidized/model/weos.rb) +* Zhone + * [Zhone (OLT and MX)](/lib/oxidized/model/zhoneolt.rb) +* Zyxel + * [ZyNOS](/lib/oxidized/model/zynos.rb) + * [NDMS](/lib/oxidized/model/ndms.rb) |