diff options
29 files changed, 810 insertions, 30 deletions
diff --git a/CHANGELOG.md b/CHANGELOG.md index 09ae2da..34b192c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,20 @@ +# 0.20.0 +- FEATURE: gpg support for CSV source (@elmobp) +- FEATURE: slackdiff (@natm) +- FEATURE: gitcrypt output model (@clement-parisot) +- FEATURE: model specific credentials (@davromaniak) +- FEATURE: hierarchical json in http source model +- FEATURE: next-adds-job config toggle (to add new job when ever /next is called) +- FEATURE: netgear model (@aschaber1) +- FEATURE: zhone model (@rfdrake) +- FEATURE: tplink model (@mediumo) +- FEATURE: oneos model (@crami) +- FEATURE: cisco NGA model (@udhos) +- FEATURE: voltaire model (@clement-parisot) +- FEATURE: siklu model (@bdg-robert) +- FEATURE: voss model (@ospfbgp) +- BUGFIX: ios, cumulus, ironware, nxos, fiberdiver, aosw, fortios, comware, procurve, opengear, timos, routeros, junos, asa, aireos, mlnxos, pfsense, saos, powerconnect, firewareos, quantaos + # 0.19.0 - FEATURE: allow setting ssh_keys (not relying on openssh config) (@denvera) - FEATURE: fujitsupy model (@stokbaek) @@ -27,6 +27,7 @@ RUN apt-get -y autoremove ADD extra/oxidized.runit /etc/service/oxidized/run ADD extra/auto-reload-config.runit /etc/service/auto-reload-config/run +ADD extra/update-ca-certificates.runit /etc/service/update-ca-certificates/run VOLUME ["/root/.config/oxidized"] EXPOSE 8888/tcp diff --git a/Gemfile.lock b/Gemfile.lock index ab52715..2c887c1 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -33,6 +33,7 @@ PLATFORMS DEPENDENCIES bundler (~> 1.10) + git (~> 1) minitest (~> 5.8) mocha (~> 1.1) oxidized! @@ -19,6 +19,7 @@ Oxidized is a network device configuration backup tool. It's a RANCID replacemen 2. [Installation](#installation) * [Debian](#debian) * [CentOS, Oracle Linux, Red Hat Linux](#centos-oracle-linux-red-hat-linux) + * [BSD](#freebsd) 3. [Initial Configuration](#configuration) 4. [Installing Ruby 2.1.2 using RVM](#installing-ruby-2.1.2-using-rvm) 5. [Running with Docker](#running-with-docker) @@ -32,6 +33,7 @@ Oxidized is a network device configuration backup tool. It's a RANCID replacemen * [Source: Mysql](#source-mysql) * [Source: HTTP](#source-http) * [Output: GIT](#output-git) + * [Output: GIT-Crypt](#output-git-crypt) * [Output: HTTP](#output-http) * [Output: File](#output-file) * [Output types](#output-types) @@ -64,6 +66,9 @@ Oxidized is a network device configuration backup tool. It's a RANCID replacemen * [C4CMTS](lib/oxidized/model/c4cmts.rb) * Aruba * [AOSW](lib/oxidized/model/aosw.rb) + * Avaya + * [VOSS (VSP Operating System Software)](lib/oxidized/model/voss.rb) + * [BOSS (Baystack Operating System Software)](lib/oxidized/model/boss.rb) * Brocade * [FabricOS](lib/oxidized/model/fabricos.rb) * [Ironware](lib/oxidized/model/ironware.rb) @@ -115,6 +120,8 @@ Oxidized is a network device configuration backup tool. It's a RANCID replacemen * [PRIMERGY Blade switch 1/10Gbe](lib/oxidized/model/fujitsupy.rb) * Hatteras * [Hatteras](lib/oxidized/model/hatteras.rb) + * Hirschmann + * [HiOS](lib/oxidized/model/hirschmann.rb) * HP * [Comware (HP A-series, H3C, 3Com)](lib/oxidized/model/comware.rb) * [Procurve](lib/oxidized/model/procurve.rb) @@ -126,6 +133,7 @@ Oxidized is a network device configuration backup tool. It's a RANCID replacemen * [ScreenOS (Netscreen)](lib/oxidized/model/screenos.rb) * Mellanox * [MLNX-OS](lib/oxidized/model/mlnxos.rb) + * [Voltaire](lib/oxidized/model/voltaire.rb) * Mikrotik * [RouterOS](lib/oxidized/model/routeros.rb) * Motorola @@ -133,6 +141,8 @@ Oxidized is a network device configuration backup tool. It's a RANCID replacemen * MRV * [MasterOS](lib/oxidized/model/masteros.rb) * [FiberDriver](lib/oxidized/model/fiberdriver.rb) + * Netgear + * [Netgear](lib/oxidized/model/netgear.rb) * Netonix * [WISP Switch (As Netonix)](lib/oxidized/model/netonix.rb) * Nokia (formerly TiMetra, Alcatel, Alcatel-Lucent) @@ -147,6 +157,8 @@ Oxidized is a network device configuration backup tool. It's a RANCID replacemen * [pfSense](lib/oxidized/model/pfsense.rb) * Quanta * [Quanta / VxWorks 6.6 (1.1.0.8)](lib/oxidized/model/quantaos.rb) + * Siklu + * [EtherHaul](lib/oxidized/model/siklu.rb) * Supermicro * [Supermicro](lib/oxidized/model/supermicro.rb) * Trango Systems @@ -193,6 +205,19 @@ gem install oxidized gem install oxidized-script oxidized-web ``` +## FreeBSD +Use RVM to install Ruby v2.1.2 + +Install all required packages and gems. + +```shell +pkg install cmake pkgconf +gem install oxidized +gem install oxidized-script oxidized-web +``` + + + ## Build from Git ```shell git clone https://github.com/ytti/oxidized.git @@ -236,7 +261,7 @@ Oxidized supports ```CSV```, ```SQLite``` and ```HTTP``` as source backends. The ## Outputs -Possible outputs are either ```file``` or ```git```. The file backend takes a destination directory as argument and will keep a file per device, with most recent running version of a device. The GIT backend (recommended) will initialize an empty GIT repository in the specified path and create a new commit on every configuration change. Take a look at the [Cookbook](#cookbook) for more details. +Possible outputs are either ```file```, ```git``` or ```git-crypt```. The file backend takes a destination directory as argument and will keep a file per device, with most recent running version of a device. The GIT backend (recommended) will initialize an empty GIT repository in the specified path and create a new commit on every configuration change. The GIT-Crypt backend will also initialize a GIT repository but every configuration push to it will be encrypted on the fly by using ```git-crypt``` tool. Take a look at the [Cookbook](#cookbook) for more details. Maps define how to map a model's fields to model [model fields](https://github.com/ytti/oxidized/tree/master/lib/oxidized/model). Most of the settings should be self explanatory, log is ignored if `use_syslog`(requires Ruby >= 2.0) is set to `true`. @@ -362,6 +387,12 @@ If you want to have the config automatically reloaded (e.g. when using a http so docker run -v /etc/oxidized:/root/.config/oxidized -p 8888:8888/tcp -e CONFIG_RELOAD_INTERVAL=3600 -t oxidized/oxidized:latest ``` +If you need to use an internal CA (e.g. to connect to an private github instance) + +``` +docker run -v /etc/oxidized:/root/.config/oxidized -v /path/to/MY-CA.crt:/usr/local/share/ca-certificates/MY-CA.crt -p 8888:8888/tcp -e UPDATE_CA_CERTIFICATES=true -t oxidized/oxidized:latest +``` + ## Cookbook ### Debugging In case a model plugin doesn't work correctly (ios, procurve, etc.), you can enable live debugging of SSH/Telnet sessions. Just add a ```debug``` option containing the value true to the ```input``` section. The log files will be created depending on the parent directory of the logfile option. @@ -604,6 +635,72 @@ output: ``` +### Output: Git-Crypt + +This uses the gem git and system git-crypt interfaces. Have a look at [GIT-Crypt](https://www.agwa.name/projects/git-crypt/) documentation to know how to install it. +Additionally to user and email informations, you have to provide the users ID that can be a key ID, a full fingerprint, an email address, or anything else that uniquely identifies a public key to GPG (see "HOW TO SPECIFY A USER ID" in the gpg man page). + + +For a single repositories for all devices: + +``` yaml +output: + default: gitcrypt + gitcrypt: + user: Oxidized + email: o@example.com + repo: "/var/lib/oxidized/devices" + users: + - "0x0123456789ABCDEF" + - "<user@example.com>" +``` + +And for groups repositories: + +``` yaml +output: + default: gitcrypt + gitcrypt: + user: Oxidized + email: o@example.com + repo: "/var/lib/oxidized/git-repos/default" + users: + - "0xABCDEF0123456789" + - "0x0123456789ABCDEF" +``` + +Oxidized will create a repository for each group in the same directory as the `default`. For +example: + +``` csv +host1:ios:first +host2:nxos:second +``` + +This will generate the following repositories: + +``` bash +$ ls /var/lib/oxidized/git-repos + +default.git first.git second.git +``` + +If you would like to use groups and a single repository, you can force this with the `single_repo` config. + +``` yaml +output: + default: gitcrypt + gitcrypt: + single_repo: true + repo: "/var/lib/oxidized/devices" + users: + - "0xABCDEF0123456789" + - "0x0123456789ABCDEF" + +``` + +Please note that user list is only updated once at creation. + ### Output: Http POST a config to the specified URL @@ -747,6 +844,22 @@ map: name: 1 group: 2 ``` +For model specific credentials + +``` +models: + junos: + username: admin + password: password + ironware: + username: admin + password: password + vars: + enable: enablepassword + apc_aos: + username: apc + password: password +``` ### Triggered backups @@ -781,6 +894,7 @@ Command is executed with the following environment: ``` OX_EVENT OX_NODE_NAME +OX_NODE_IP OX_NODE_FROM OX_NODE_MSG OX_NODE_GROUP diff --git a/extra/update-ca-certificates.runit b/extra/update-ca-certificates.runit new file mode 100755 index 0000000..53efdb9 --- /dev/null +++ b/extra/update-ca-certificates.runit @@ -0,0 +1,7 @@ +#!/bin/bash + +if [ "$UPDATE_CA_CERTIFICATES" == "true" ]; then + update-ca-certificates +fi + +sleep infinity diff --git a/lib/oxidized/config.rb b/lib/oxidized/config.rb index aba8b63..23af9c2 100644 --- a/lib/oxidized/config.rb +++ b/lib/oxidized/config.rb @@ -31,6 +31,7 @@ module Oxidized asetus.default.next_adds_job = false # if true, /next adds job, so device is fetched immmeiately asetus.default.vars = {} # could be 'enable'=>'enablePW' asetus.default.groups = {} # group level configuration + asetus.default.models = {} # model level configuration asetus.default.pid = File.join(Oxidized::Config::Root, 'pid') asetus.default.input.default = 'ssh, telnet' diff --git a/lib/oxidized/config/vars.rb b/lib/oxidized/config/vars.rb index 09f9781..d926ee2 100644 --- a/lib/oxidized/config/vars.rb +++ b/lib/oxidized/config/vars.rb @@ -8,8 +8,12 @@ module Oxidized::Config::Vars r ||= Oxidized.config.groups[@node.group].vars[name.to_s] end end + if Oxidized.config.models.has_key?(@node.model.class.name.to_s.downcase) + if Oxidized.config.models[@node.model.class.name.to_s.downcase].vars.has_key?(name.to_s) + r ||= Oxidized.config.models[@node.model.class.name.to_s.downcase].vars[name.to_s] + end + end r ||= Oxidized.config.vars[name.to_s] if Oxidized.config.vars.has_key?(name.to_s) r end end - diff --git a/lib/oxidized/hook/exec.rb b/lib/oxidized/hook/exec.rb index 58d6fd5..fa8aff1 100644 --- a/lib/oxidized/hook/exec.rb +++ b/lib/oxidized/hook/exec.rb @@ -67,6 +67,7 @@ class Exec < Oxidized::Hook if ctx.node env.merge!( "OX_NODE_NAME" => ctx.node.name.to_s, + "OX_NODE_IP" => ctx.node.ip.to_s, "OX_NODE_FROM" => ctx.node.from.to_s, "OX_NODE_MSG" => ctx.node.msg.to_s, "OX_NODE_GROUP" => ctx.node.group.to_s, diff --git a/lib/oxidized/hook/slackdiff.rb b/lib/oxidized/hook/slackdiff.rb index b37c9c4..61f1743 100644 --- a/lib/oxidized/hook/slackdiff.rb +++ b/lib/oxidized/hook/slackdiff.rb @@ -10,7 +10,11 @@ class SlackDiff < Oxidized::Hook if ctx.node if ctx.event.to_s == "post_store" log "Connecting to slack" - client = Slack::Client.new token: cfg.token + Slack.configure do |config| + config.token = cfg.token + config.proxy = cfg.proxy if cfg.has_key?('proxy') + end + client = Slack::Client.new client.auth_test log "Connected" gitoutput = ctx.node.output.new diff --git a/lib/oxidized/model/airos.rb b/lib/oxidized/model/airos.rb index 775005f..7d82956 100644 --- a/lib/oxidized/model/airos.rb +++ b/lib/oxidized/model/airos.rb @@ -1,14 +1,19 @@ class Airos < Oxidized::Model # Ubiquiti AirOS circa 5.x - + prompt /^[^#]+# / - + comment '# ' + cmd 'cat /etc/board.info' do |cfg| cfg.split("\n").map { |line| "# #{line}" }.join("\n") + "\n" end - + + cmd 'cat /etc/version' do |cfg| + comment "airos version: #{cfg}" + end + cmd 'sort /tmp/system.cfg' - + cmd :secret do |cfg| cfg.gsub! /^(users\.\d+\.password|snmp\.community)=.+/, "# \\1=<hidden>" cfg diff --git a/lib/oxidized/model/boss.rb b/lib/oxidized/model/boss.rb new file mode 100644 index 0000000..02201a1 --- /dev/null +++ b/lib/oxidized/model/boss.rb @@ -0,0 +1,76 @@ +class Boss < Oxidized::Model + # Avaya Baystack Operating System Software(BOSS) + # Created by danielcoxman@gmail.com + # May 15, 2017 + # This was tested on ers3510, ers5530, ers4850, ers5952 + # ssh and telnet were tested with banner and without + + comment '! ' + + prompt /^[^\s#>]+[#>]$/ + + # Handle the banner + # to disable the banner on BOSS the configuration parameter is "banner disabled" + expect /Enter Ctrl-Y to begin\./ do |data, re| + send "\cY" + data.sub re, '' + end + + # Handle the Failed retries since last login + # no known way to disable other than to implement radius authentication + expect /Press ENTER to continue/ do |data, re| + send "\n" + data.sub re, '' + end + + # Handle the menu on the older BOSS example ers55xx, ers56xx + # to disable them menu on BOSS the configuration parameter is "cmd-interface cli" + expect /ommand Line Interface\.\.\./ do |data, re| + send "c" + data.sub re, '' + end + + # needed for proper formatting + cmd('') { |cfg| comment "#{cfg}\n" } + + # Do a sys-info and check and see if it supports stack + cmd 'show sys-info' do |cfg| + @stack = true if cfg.match /Stack/ + cfg.gsub! /(^((.*)sysUpTime(.*))$)/, 'removed sysUpTime' + cfg.gsub! /(^((.*)sysNtpTime(.*))$)/, 'removed sysNtpTime' + cfg.gsub! /(^((.*)sysRtcTime(.*))$)/, 'removed sysNtpTime' + # remove timestamp + cfg.gsub! /\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} .*/, '' + comment "#{cfg}\n" + end + + # if a stack then collect the stacking information + cmd 'show stack-info' do |cfg| + if @stack + # remove timestamp + cfg.gsub! /\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} .*/, '' + comment "#{cfg}\n" + end + end + + cmd 'show running-config' do |cfg| + cfg.gsub! /^show running-config/, '! show running-config' + # remove timestamp + cfg.gsub! /\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} .*/, '' + cfg.gsub! /^[^\s#>]+[#>]$/, '' + cfg.gsub! /^! clock set.*/, '! removed clock set' + cfg + end + + cfg :telnet do + username /Username: / + password /Password: / + end + + cfg :telnet, :ssh do + pre_logout 'logout' + post_login 'terminal length 0' + post_login 'terminal width 132' + end + +end diff --git a/lib/oxidized/model/firewareos.rb b/lib/oxidized/model/firewareos.rb index f2bef4c..f456c60 100644 --- a/lib/oxidized/model/firewareos.rb +++ b/lib/oxidized/model/firewareos.rb @@ -1,12 +1,18 @@ class FirewareOS < Oxidized::Model - prompt /^\[?\w*\]?\w*<?\w*>?#\s*$/ + prompt /^([\w.@-]+[#>]\s?)$/ comment '-- ' cmd :all do |cfg| cfg.each_line.to_a[1..-2].join end + # Handle Logon Disclaimer added in XTM 11.9.3 + expect /^I have read and accept the Logon Disclaimer message. \(yes or no\)\? $/ do |data, re| + send "yes\n" + data.sub re, '' + end + cmd 'show sysinfo' do |cfg| # avoid commits due to uptime cfg = cfg.each_line.select { |line| not line.match /(.*time.*)|(.*memory.*)|(.*cpu.*)/ } diff --git a/lib/oxidized/model/fortios.rb b/lib/oxidized/model/fortios.rb index 2415537..0249933 100644 --- a/lib/oxidized/model/fortios.rb +++ b/lib/oxidized/model/fortios.rb @@ -15,7 +15,7 @@ class FortiOS < Oxidized::Model end cmd :secret do |cfg| - cfg.gsub! /(set (?:passwd|password|psksecret)).*/, '\\1 <configuration removed>' + cfg.gsub! /(set (?:passwd|password|psksecret|secret|key ENC)).*/, '\\1 <configuration removed>' cfg.gsub! /(set private-key).*-+END ENCRYPTED PRIVATE KEY-*"$/m , '\\1 <configuration removed>' cfg end @@ -38,6 +38,7 @@ class FortiOS < Oxidized::Model #do not include if variable "show_autoupdate" is set to false if defined?(vars(:fortios_autoupdate)).nil? || vars(:fortios_autoupdate) cfg << cmd('diagnose autoupdate version') do |cfg| + cfg.gsub! /(FDS Address\n---------\n).*/, '\\1IP Address removed' comment cfg.each_line.reject { |line| line.match /Last Update|Result/ }.join end end diff --git a/lib/oxidized/model/hirschmann.rb b/lib/oxidized/model/hirschmann.rb new file mode 100644 index 0000000..82432d7 --- /dev/null +++ b/lib/oxidized/model/hirschmann.rb @@ -0,0 +1,41 @@ +class Hirschmann < Oxidized::Model + + prompt /^[\(\w\s\w\)]+\s[>|#]+?$/ + + comment '## ' + + #Handle pager + expect /^--More--.*$/ do |data, re| + send 'a' + data.sub re, '' + end + + cmd :all do |cfg| + cfg.each_line.to_a[1..-2].join + end + + cmd 'show sysinfo' do |cfg| + cfg.gsub! /^System Up Time.*\n/, "" + cfg.gsub! /^System Date and Time.*\n/, "" + cfg.gsub! /^CPU Utilization.*\n/, "" + cfg.gsub! /^Memory.*\n/, "" + cfg.gsub! /^Average CPU Utilization.*\n/, "" + comment cfg + end + + cmd 'show running-config' do |cfg| + cfg.gsub! /^users.*\n/, "" + cfg + end + + cfg :telnet do + username /^User:/ + password /^Password:/ + end + + cfg :telnet, :ssh do + post_login 'enable' + pre_logout 'logout' + end + +end diff --git a/lib/oxidized/model/ios.rb b/lib/oxidized/model/ios.rb index 94a3ab0..c3d5543 100644 --- a/lib/oxidized/model/ios.rb +++ b/lib/oxidized/model/ios.rb @@ -19,24 +19,96 @@ class IOS < Oxidized::Model cmd :all do |cfg| #cfg.gsub! /\cH+\s{8}/, '' # example how to handle pager #cfg.gsub! /\cH+/, '' # example how to handle pager + # get rid of errors for commands that don't work on some devices + cfg.gsub! /^% Invalid input detected at '\^' marker\.$|^\s+\^$/, '' cfg.each_line.to_a[1..-2].join end cmd :secret do |cfg| cfg.gsub! /^(snmp-server community).*/, '\\1 <configuration removed>' - cfg.gsub! /username (\S+) privilege (\d+) (\S+).*/, '<secret hidden>' - cfg.gsub! /^username \S+ password \d \S+/, '<secret hidden>' - cfg.gsub! /^username \S+ secret \d \S+/, '<secret hidden>' - cfg.gsub! /^enable (password|secret) \d \S+/, '<secret hidden>' + cfg.gsub! /^(username \S+ privilege \d+) (\S+).*/, '\\1 <secret hidden>' + cfg.gsub! /^(username \S+ password \d) (\S+)/, '\\1 <secret hidden>' + cfg.gsub! /^(username \S+ secret \d) (\S+)/, '\\1 <secret hidden>' + cfg.gsub! /^(enable (password|secret) \d) (\S+)/, '\\1 <secret hidden>' cfg.gsub! /^(\s+(?:password|secret)) (?:\d )?\S+/, '\\1 <secret hidden>' - cfg.gsub! /wpa-psk ascii \d \S+/, '<secret hidden>' - cfg.gsub! /key 7 \d.+/, '<secret hidden>' - cfg.gsub! /^tacacs-server key \d \S+/, '<secret hidden>' + cfg.gsub! /^(.*wpa-psk ascii \d) (\S+)/, '\\1 <secret hidden>' + cfg.gsub! /^(.*key 7) (\d.+)/, '\\1 <secret hidden>' + cfg.gsub! /^(tacacs-server key \d) (\S+)/, '\\1 <secret hidden>' + cfg.gsub! /^(crypto isakmp key) (\S+) (.*)/, '\\1 <secret hidden> \\3' cfg end cmd 'show version' do |cfg| - comment cfg.lines.first + comments = [] + comments << cfg.lines.first + lines = cfg.lines + lines.each_with_index do |line,i| + slave = '' + slaveslot = '' + + if line.match /^Slave in slot (\d+) is running/ + slave = " Slave:"; + slaveslot = ", slot #{$1}"; + end + + if line.match /^Compiled (.*)$/ + comments << "Image:#{slave} Compiled: #{$1}" + end + + if line.match /^(?:Cisco )?IOS .* Software,? \(([A-Za-z0-9_-]*)\), .*Version\s+(.*)$/ + comments << "Image:#{slave} Software: #{$1}, #{$2}" + end + + if line.match /^ROM: (IOS \S+ )?(System )?Bootstrap.*(Version.*)$/ + comments << "ROM Bootstrap: #{$3}" + end + + if line.match /^BOOTFLASH: .*(Version.*)$/ + comments << "BOOTFLASH: #{$1}" + end + + if line.match /^(\d+[kK]) bytes of (non-volatile|NVRAM)/ + comments << "Memory: nvram #{$1}" + end + + if line.match /^(\d+[kK]) bytes of (flash memory|flash internal|processor board System flash|ATA CompactFlash)/i + comments << "Memory: flash #{$1}" + end + + if line.match (/^(\d+[kK]) bytes of (Flash|ATA)?.*PCMCIA .*(slot|disk) ?(\d)/i) + comments << "Memory: pcmcia #{$2} #{$3}#{$4} #{$1}"; + end + + if line.match /(\S+(?:\sseries)?)\s+(?:\((\S+)\)\s+processor|\(revision[^)]+\)).*\s+with (\S+k) bytes/i + sproc = $1 + cpu = $2 + mem = $3 + cpuxtra = '' + comments << "Chassis type:#{slave} #{sproc}"; + comments << "Memory:#{slave} main #{mem}"; + # check the next two lines for more CPU info + if cfg.lines[i+1].match /processor board id (\S+)/i + comments << "Processor ID: #{$1}"; + end + if cfg.lines[i+2].match /(cpu at |processor: |#{cpu} processor,)/i + # change implementation to impl and prepend comma + cpuxtra = cfg.lines[i+2].gsub(/implementation/,'impl').gsub(/^/,', ').chomp; + end + comments << "CPU:#{slave} #{cpu}#{cpuxtra}#{slaveslot}"; + end + + if line.match /^System image file is "([^\"]*)"$/ + comments << "Image: #{$1}" + end + end + comments << "\n" + comment comments.join "\n" + end + + cmd 'show vtp status' do |cfg| + cfg.gsub! /^$\n/, '' + cfg.gsub! /^/, 'VTP: ' if (!cfg.empty?) + comment "#{cfg}\n" end cmd 'show inventory' do |cfg| diff --git a/lib/oxidized/model/mlnxos.rb b/lib/oxidized/model/mlnxos.rb index 97e7098..49f3369 100644 --- a/lib/oxidized/model/mlnxos.rb +++ b/lib/oxidized/model/mlnxos.rb @@ -42,6 +42,6 @@ class MLNXOS < Oxidized::Model cfg :ssh do password /^Password:\s*/ - pre_logout 'exit' + pre_logout "\nexit" end end diff --git a/lib/oxidized/model/netgear.rb b/lib/oxidized/model/netgear.rb new file mode 100644 index 0000000..08c64d3 --- /dev/null +++ b/lib/oxidized/model/netgear.rb @@ -0,0 +1,32 @@ +class Netgear < Oxidized::Model + + comment '!' + prompt /^(\([\w-]+\)\s[#>])$/ + + cmd :secret do |cfg| + cfg.gsub!(/password (\S+)/, 'password <hidden>') + cfg + end + + cfg :telnet, :ssh do + if vars :enable + post_login do + cmd 'enable' + # Interpret enable: true as meaning we won't be prompted for a password + unless vars(:enable).is_a? TrueClass + expect /[pP]assword:\s?$/ + cmd vars(:enable) + "\n" + end + expect /^.+[#]$/ + end + end + post_login 'terminal length 0' + pre_logout 'exit' + pre_logout 'quit' + end + + cmd 'show running-config' do |cfg| + cfg.gsub! /^(!.*Time).*$/, '\1' + end + +end diff --git a/lib/oxidized/model/pfsense.rb b/lib/oxidized/model/pfsense.rb index 782969e..5849160 100644 --- a/lib/oxidized/model/pfsense.rb +++ b/lib/oxidized/model/pfsense.rb @@ -3,7 +3,7 @@ class PfSense < Oxidized::Model # use other use than 'admin' user, 'admin' user cannot get ssh/exec. See issue #535 cmd :all do |cfg| - cfg.each_line.to_a[1..-2].join + cfg.each_line.to_a[1..-1].join end cmd 'cat /cf/conf/config.xml' do |cfg| diff --git a/lib/oxidized/model/powerconnect.rb b/lib/oxidized/model/powerconnect.rb index ac36c26..618f05a 100644 --- a/lib/oxidized/model/powerconnect.rb +++ b/lib/oxidized/model/powerconnect.rb @@ -67,6 +67,7 @@ class PowerConnect < Oxidized::Model end out << line.strip end + out = out.select { |line| not line[/Up\sTime/] } out = comment out.join "\n" out << "\n" end diff --git a/lib/oxidized/model/quantaos.rb b/lib/oxidized/model/quantaos.rb index 274440d..5197aa8 100644 --- a/lib/oxidized/model/quantaos.rb +++ b/lib/oxidized/model/quantaos.rb @@ -12,7 +12,7 @@ class QuantaOS < Oxidized::Model end cfg :telnet do - username /^Username:/ + username /^User(name)?:/ password /^Password:/ end diff --git a/lib/oxidized/model/routeros.rb b/lib/oxidized/model/routeros.rb index 5717100..31650c7 100644 --- a/lib/oxidized/model/routeros.rb +++ b/lib/oxidized/model/routeros.rb @@ -10,6 +10,10 @@ class RouterOS < Oxidized::Model comment cfg end + cmd '/system history print' do |cfg| + comment cfg + end + cmd '/export' do |cfg| cfg.gsub! /\x1B\[([0-9]{1,3}((;[0-9]{1,3})*)?)?[m|K]/, '' # strip ANSI colours cfg.gsub! /\\\r\n\s+/, '' # strip new line @@ -22,6 +26,10 @@ class RouterOS < Oxidized::Model password /^Password:/ end + cfg :telnet, :ssh do + pre_logout 'quit' + end + cfg :ssh do exec true end diff --git a/lib/oxidized/model/siklu.rb b/lib/oxidized/model/siklu.rb new file mode 100644 index 0000000..2bdfbc3 --- /dev/null +++ b/lib/oxidized/model/siklu.rb @@ -0,0 +1,19 @@ +class Siklu < Oxidized::Model + + # Siklu EtherHaul # + + prompt /^[\w-]+>$/ + + cmd 'copy startup-configuration display' do |cfg| + cfg.each_line.to_a[2..2].join + end + + cmd 'copy running-configuration display' do |cfg| + cfg.each_line.to_a[3..-2].join + end + + cfg :ssh do + pre_logout 'exit' + end + +end diff --git a/lib/oxidized/model/voltaire.rb b/lib/oxidized/model/voltaire.rb new file mode 100644 index 0000000..1e7fad2 --- /dev/null +++ b/lib/oxidized/model/voltaire.rb @@ -0,0 +1,56 @@ +class VOLTAIRE < Oxidized::Model + + prompt /([\w.@()-\[:\s\]]+[#>]\s|(One or more tests have failed.*))$/ + comment '## ' + + # Pager Handling + expect /.+lines\s\d+\-\d+([\s]|\/\d+\s\(END\)\s).+$/ do |data, re| + send ' ' + data.sub re, '' + end + + + cmd :all do |cfg| + cfg.gsub! /\[\?1h=\r/, '' # Pager Handling + cfg.gsub! /\r\[K/,'' # Pager Handling + cfg.gsub! /\s/, '' # Linebreak Handling + cfg.gsub! /^CPU\ load\ averages\:\s.+/, '' # Omit constantly changing CPU info + cfg.gsub! /^System\ memory\:\s.+/, '' # Omit constantly changing memory info + cfg.gsub! /^Uptime\:\s.+/, '' # Omit constantly changing uptime info + cfg.gsub! /.+Generated\ at\s\d+.+/, '' # Omit constantly changing generation time info + cfg = cfg.lines.to_a[2..-3].join + end + + cmd :secret do |cfg| + cfg.gsub! /(snmp-server community).*/, ' <snmp-server community configuration removed>' + cfg.gsub! /username (\S+) password (\d+) (\S+).*/, '<secret hidden>' + cfg + end + + + cmd 'version show' do |cfg| + comment cfg + end + + cmd 'firmware-version show' do |cfg| + comment cfg + end + + cmd 'remote show' do |cfg| + cfg + end + + cmd 'sm-info show' do |cfg| + cfg + end + + cmd ' show' do |cfg| + cfg + end + + cfg :ssh do + post_login "no\n" + password /^Password:\s*/ + pre_logout 'exit' + end +end diff --git a/lib/oxidized/model/voss.rb b/lib/oxidized/model/voss.rb new file mode 100644 index 0000000..599462a --- /dev/null +++ b/lib/oxidized/model/voss.rb @@ -0,0 +1,36 @@ +class Voss < Oxidized::Model + # Avaya VSP Operating System Software(VOSS) + # Created by danielcoxman@gmail.com + # May 15, 2017 + # This was tested on vsp4k and vsp8k + + comment '# ' + + prompt /^[^\s#>]+[#>]$/ + + # needed for proper formatting after post_login + cmd('') { |cfg| comment "#{cfg}\n" } + # get some general information about switch + cmd('show sys-info card') { |cfg| comment "#{cfg}\n" } + cmd('show sys-info fan') { |cfg| comment "#{cfg}\n" } + cmd('show sys-info power') { |cfg| comment "#{cfg}\n" } + + # more the config rather than doing a show run + cmd 'more config.cfg' do |cfg| + cfg + cfg.gsub! /^[^\s#>]+[#>]$/, '' + cfg.gsub! /^more config.cfg/, '# more config.cfg' + end + + cfg :telnet do + username /Login: $/ + password /Password: $/ + end + + cfg :telnet, :ssh do + pre_logout 'exit' + post_login 'enable' + post_login 'terminal more disable' + end + +end diff --git a/lib/oxidized/model/vrp.rb b/lib/oxidized/model/vrp.rb index 12a9ca3..98229c3 100644 --- a/lib/oxidized/model/vrp.rb +++ b/lib/oxidized/model/vrp.rb @@ -4,6 +4,12 @@ class VRP < Oxidized::Model prompt /^(<[\w.-]+>)$/ comment '# ' + cmd :secret do |cfg| + cfg.gsub! /(pin verify (?:auto|)).*/, '\\1 <PIN hidden>' + cfg.gsub! /(%\^%#.*%\^%#)/, '<secret hidden>' + cfg + end + cmd :all do |cfg| cfg.each_line.to_a[1..-2].join end diff --git a/lib/oxidized/node.rb b/lib/oxidized/node.rb index 6f89b56..8d0d0af 100644 --- a/lib/oxidized/node.rb +++ b/lib/oxidized/node.rb @@ -166,18 +166,32 @@ module Oxidized end def resolve_repo opt - return unless is_git? opt - - remote_repo = Oxidized.config.output.git.repo - - if remote_repo.is_a?(::String) - if Oxidized.config.output.git.single_repo? || @group.nil? - remote_repo + if is_git? opt + remote_repo = Oxidized.config.output.git.repo + + if remote_repo.is_a?(::String) + if Oxidized.config.output.git.single_repo? || @group.nil? + remote_repo + else + File.join(File.dirname(remote_repo), @group + '.git') + end + else + remote_repo[@group] + end + elsif is_gitcrypt? opt + remote_repo = Oxidized.config.output.gitcrypt.repo + + if remote_repo.is_a?(::String) + if Oxidized.config.output.gitcrypt.single_repo? || @group.nil? + remote_repo + else + File.join(File.dirname(remote_repo), @group + '.git') + end else - File.join(File.dirname(remote_repo), @group + '.git') + remote_repo[@group] end else - remote_repo[@group] + return end end @@ -202,6 +216,14 @@ module Oxidized end end + #model + if Oxidized.config.models.has_key?(@model.class.name.to_s.downcase) + if Oxidized.config.models[@model.class.name.to_s.downcase].has_key?(key_str) + value = Oxidized.config.models[@model.class.name.to_s.downcase][key_str] + Oxidized.logger.debug "node.rb: setting node key '#{key}' to value '#{value}' from model" + end + end + #node value = opt[key_sym] || value Oxidized.logger.debug "node.rb: returning node key '#{key}' with value '#{value}'" @@ -212,5 +234,9 @@ module Oxidized (opt[:output] || Oxidized.config.output.default) == 'git' end + def is_gitcrypt? opt + (opt[:output] || Oxidized.config.output.default) == 'gitcrypt' + end + end end diff --git a/lib/oxidized/output/gitcrypt.rb b/lib/oxidized/output/gitcrypt.rb new file mode 100644 index 0000000..b0d80f2 --- /dev/null +++ b/lib/oxidized/output/gitcrypt.rb @@ -0,0 +1,244 @@ +module Oxidized + class GitCrypt < Output + class GitCryptError < OxidizedError; end + begin + require 'git' + rescue LoadError + raise OxidizedError, 'git not found: sudo gem install ruby-git' + end + + attr_reader :commitref + + def initialize + @cfg = Oxidized.config.output.gitcrypt + @gitcrypt_cmd = "/usr/bin/git-crypt" + @gitcrypt_init = @gitcrypt_cmd + " init" + @gitcrypt_unlock = @gitcrypt_cmd + " unlock" + @gitcrypt_lock = @gitcrypt_cmd + " lock" + @gitcrypt_adduser = @gitcrypt_cmd + " add-gpg-user --trusted " + end + + def setup + if @cfg.empty? + Oxidized.asetus.user.output.gitcrypt.user = 'Oxidized' + Oxidized.asetus.user.output.gitcrypt.email = 'o@example.com' + Oxidized.asetus.user.output.gitcrypt.repo = File.join(Config::Root, 'oxidized.git') + Oxidized.asetus.save :user + raise NoConfig, 'no output git config, edit ~/.config/oxidized/config' + end + + if @cfg.repo.respond_to?(:each) + @cfg.repo.each do |group, repo| + @cfg.repo["#{group}="] = File.expand_path repo + end + else + @cfg.repo = File.expand_path @cfg.repo + end + end + + def crypt_init repo + repo.chdir do + system(@gitcrypt_init) + @cfg.users.each do |user| + system("#{@gitcrypt_adduser} #{user}") + end + File.write(".gitattributes", "* filter=git-crypt diff=git-crypt\n.gitattributes !filter !diff") + repo.add(".gitattributes") + repo.commit("Initial commit: crypt all config files") + end + end + + def lock repo + repo.chdir do + system(@gitcrypt_lock) + end + end + + def unlock repo + repo.chdir do + system(@gitcrypt_unlock) + end + end + + def store file, outputs, opt={} + @msg = opt[:msg] + @user = (opt[:user] or @cfg.user) + @email = (opt[:email] or @cfg.email) + @opt = opt + @commitref = nil + repo = @cfg.repo + + outputs.types.each do |type| + type_cfg = '' + type_repo = File.join(File.dirname(repo), type + '.git') + outputs.type(type).each do |output| + (type_cfg << output; next) if not output.name + type_file = file + '--' + output.name + if @cfg.type_as_directory? + type_file = type + '/' + type_file + type_repo = repo + end + update type_repo, type_file, output + end + update type_repo, file, type_cfg + end + + update repo, file, outputs.to_cfg + end + + + def fetch node, group + begin + repo, path = yield_repo_and_path(node, group) + repo = Git.open repo + unlock repo + index = repo.index + # Empty repo ? + empty = File.exists? index.path + if empty + raise 'Empty git repo' + else + File.read path + end + lock repo + rescue + 'node not found' + end + end + + # give a hash of all oid revision for the given node, and the date of the commit + def version node, group + begin + repo, path = yield_repo_and_path(node, group) + + repo = Git.open repo + unlock repo + walker = repo.log.path(path) + i = -1 + tab = [] + walker.each do |commit| + hash = {} + hash[:date] = commit.date.to_s + hash[:oid] = commit.objectish + hash[:author] = commit.author + hash[:message] = commit.message + tab[i += 1] = hash + end + walker.reset + tab + rescue + 'node not found' + end + end + + #give the blob of a specific revision + def get_version node, group, oid + begin + repo, path = yield_repo_and_path(node, group) + repo = Git.open repo + unlock repo + repo.gtree(oid).files[path].contents + rescue + 'version not found' + ensure + lock repo + end + end + + #give a hash with the patch of a diff between 2 revision and the stats (added and deleted lines) + def get_diff node, group, oid1, oid2 + begin + diff_commits = nil + repo, path = yield_repo_and_path(node, group) + repo = Git.open repo + unlock repo + commit = repo.gcommit(oid1) + + if oid2 + commit_old = repo.gcommit(oid2) + diff = repo.diff(commit_old, commit) + stats = [diff.stats[:files][node.name][:insertions], diff.stats[:files][node.name][:deletions]] + diff.each do |patch| + if /#{node.name}\s+/.match(patch.patch.to_s.lines.first) + diff_commits = {:patch => patch.patch.to_s, :stat => stats} + break + end + end + else + stat = commit.parents[0].diff(commit).stats + stat = [stat[:files][node.name][:insertions],stat[:files][node.name][:deletions]] + patch = commit.parents[0].diff(commit).patch + diff_commits = {:patch => patch, :stat => stat} + end + lock repo + diff_commits + rescue + 'no diffs' + ensure + lock repo + end + end + + private + + def yield_repo_and_path(node, group) + repo, path = node.repo, node.name + + if group and @cfg.single_repo? + path = "#{group}/#{node.name}" + end + + [repo, path] + end + + def update repo, file, data + return if data.empty? + + if @opt[:group] + if @cfg.single_repo? + file = File.join @opt[:group], file + else + repo = if repo.is_a?(::String) + File.join File.dirname(repo), @opt[:group] + '.git' + else + repo[@opt[:group]] + end + end + end + + begin + update_repo repo, file, data, @msg, @user, @email + rescue Git::GitExecuteError, ArgumentError => open_error + Oxidized.logger.debug "open_error #{open_error} #{file}" + begin + grepo = Git.init repo + crypt_init grepo + rescue => create_error + raise GitCryptError, "first '#{open_error.message}' was raised while opening git repo, then '#{create_error.message}' was while trying to create git repo" + end + retry + end + end + + def update_repo repo, file, data, msg, user, email + grepo = Git.open repo + grepo.config('user.name', user) + grepo.config('user.email', email) + grepo.chdir do + unlock grepo + File.write(file, data) + grepo.add(file) + if grepo.status[file].nil? + grepo.commit(msg) + @commitref = grepo.log(1).first.objectish + true + elsif !grepo.status[file].type.nil? + grepo.commit(msg) + @commitref = grepo.log(1).first.objectish + true + end + lock grepo + end + end + end +end diff --git a/lib/oxidized/version.rb b/lib/oxidized/version.rb index 073aae9..221cffa 100644 --- a/lib/oxidized/version.rb +++ b/lib/oxidized/version.rb @@ -1,3 +1,3 @@ module Oxidized - VERSION = '0.19.0' + VERSION = '0.20.0' end diff --git a/oxidized.gemspec b/oxidized.gemspec index 78324d4..ea088e7 100644 --- a/oxidized.gemspec +++ b/oxidized.gemspec @@ -33,4 +33,5 @@ Gem::Specification.new do |s| s.add_development_dependency 'rake', '~> 10.0' s.add_development_dependency 'minitest', '~> 5.8' s.add_development_dependency 'mocha', '~> 1.1' + s.add_development_dependency 'git', '~> 1' end |