Merge pull request #117 from ukfast/http-source

Added support for http as a source

2 files changed, 79 insertions, 2 deletions

diff --git a/README.md b/README.md
index 1f7622d..2e40556 100644
--- a/README.md
+++ b/README.md
@@ -27,6 +27,7 @@ Oxidized is a network device configuration backup tool. It's a RANCID replacment
     * [Privileged mode](#privileged-mode)
     * [Source: CSV](#source-csv)
     * [Source: SQLite](#source-sqlite)
+    * [Source: HTTP](#source-http)
     * [Output: GIT](#output-git)
     * [Output: File](#output-file)
     * [Advanced Configuration](#advanced-configuration)
@@ -100,7 +101,7 @@ To initialize a default configuration in your home directory ```~/.config/oxidiz
 
 ## Source
 
-Oxidized supports ```CSV``` and ```SQLite``` as source backends. The CSV backend reads nodes from a rancid compatible router.db file. The SQLite backend will fire queries against a database and map certain fields to model items. Take a look at the [Cookbook](#cookbook) for more details.
+Oxidized supports ```CSV```, ```SQLite``` and ```HTTP``` as source backends. The CSV backend reads nodes from a rancid compatible router.db file. The SQLite backend will fire queries against a database and map certain fields to model items. The HTTP backend will fire queries against a http/https url. Take a look at the [Cookbook](#cookbook) for more details.
 
 ## Outputs
 
@@ -221,6 +222,28 @@ source:
       enable: enable
 ```
 
+### Source: HTTP
+
+One object per device.
+
+```
+source:
+  default: http
+  http:
+    url: https://url/api
+    scheme: https
+    delimiter: !ruby/regexp /:/
+    map:
+      name: hostname
+      model: os
+      username: username
+      password: password
+    vars_map:
+      enable: enable
+    headers:
+      X-Auth-Token: 'somerandomstring'
+```
+
 ### Output: File
 
 Parent directory needs to be created manually, one file per device, with most recent running config.
@@ -314,7 +337,7 @@ The following objects exist in Oxidized.
    * input - method to acquire config, loaded dynamically as needed (Also default in config file)
    * output - method to store config, loaded dynamically as needed (Also default in config file)
    * prompt - prompt used for node (Also default in config file, can be specified in model too)
- * 'sql' and 'csv' (supports any format with single entry per line, like router.db)
+ * 'sql', 'csv' and 'http' (supports any format with single entry per line, like router.db)
 
 ## Model
  * lists commands to gather from given device model
diff --git a/lib/oxidized/source/http.rb b/lib/oxidized/source/http.rb
new file mode 100644
index 0000000..d2e3ea6
--- /dev/null
+++ b/lib/oxidized/source/http.rb
@@ -0,0 +1,54 @@
+module Oxidized
+class HTTP < Source
+  def initialize
+    @cfg = CFG.source.http
+    super
+  end
+
+  def setup
+    if @cfg.url.empty?
+      raise NoConfig, 'no source http url config, edit ~/.config/oxidized/config'
+    end
+  end
+
+  require "net/http"
+  require "uri"
+  require "json"
+
+  def load
+    nodes = []
+    uri = URI.parse(@cfg.url)
+    http = Net::HTTP.new(uri.host, uri.port)
+    http.use_ssl = true if uri.scheme == 'https'
+
+    # map headers
+    headers = {}
+    @cfg.headers.each do |header, value|
+      headers[header] = value
+    end
+
+    request = Net::HTTP::Get.new(uri.request_uri, headers)
+
+    response = http.request(request)
+    data = JSON.parse(response.body)
+    data.each do |line|
+      next if line.empty?
+      # map node parameters
+      keys = {}
+      @cfg.map.each do |key, position|
+        keys[key.to_sym] = line[position]
+      end
+      keys[:model] = map_model keys[:model] if keys.key? :model
+
+      # map node specific vars, empty value is considered as nil
+      vars = {}
+      @cfg.vars_map.each { |key, position| vars[key.to_sym] = line[position].to_s.empty? ? nil : line[position] }
+      keys[:vars] = vars unless vars.empty?
+
+      nodes << keys
+    end
+    nodes
+  end
+
+end
+end