From 570bbf6a1312f3c236a0ac8113fd4cd27c3fc51e Mon Sep 17 00:00:00 2001 From: Steve Kemp Date: Thu, 17 Dec 2015 18:14:11 +0200 Subject: Treat our Redis queue as a set. This means that tests will only ever be enqueued once, regardless of how many times they are parsed. In the past we could have a configuration file that read: test1 .. test2 .. test3 .. Parsing/adding this file would result in a queue looking like so: test1 .. test2 .. test3 .. test1 .. test2 .. test3 .. test1 .. test2 .. test3 .. Now the queue will *ALWAYS* look like this: test1 .. test2 .. test3 .. In the normal course of events this won't matter, as teh processing loop will look like so: * Add new jobs every minute. * Worker runs the jobs. In the case of a failing job though the test might take 2.5 minutes and that will cause the queue to backup. (2.5 minutes because a test is repeated 5 times before a fail is announced, and the timeout is 30 seconds. These values can and should be tweaked.) With the new method even if the queue is slowly draining the queue will never grow to containu hundreds of events it will just be "topped up" not "overflowing". Thanks to James Hannah for the suggestion, and James Lawrie for the patience. --- lib/custodian/queue.rb | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/lib/custodian/queue.rb b/lib/custodian/queue.rb index bb67c8e..89266df 100644 --- a/lib/custodian/queue.rb +++ b/lib/custodian/queue.rb @@ -96,7 +96,7 @@ module Custodian loop do - _foo, job = @redis.blpop('queue', :timeout => timeout) + job = @redis.spop('queue') if job return job @@ -112,7 +112,7 @@ module Custodian # Add a new job to the queue. # def add(job_string) - @redis.rpush('queue', job_string) + @redis.sadd('queue', job_string) end @@ -120,7 +120,7 @@ module Custodian # How many jobs in the queue? # def size? - @redis.llen('queue') + @redis.scard('queue') end -- cgit v1.2.1