summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorSteve Kemp <steve@steve.org.uk>2015-12-17 18:14:11 +0200
committerSteve Kemp <steve@steve.org.uk>2015-12-17 18:14:11 +0200
commit570bbf6a1312f3c236a0ac8113fd4cd27c3fc51e (patch)
tree4cc34f0ff23b5a8f1002c09223fb9491cd10a4e0
parent1b0548485335df7735abc2c66280b31aa98d866c (diff)
Treat our Redis queue as a set.
This means that tests will only ever be enqueued once, regardless of how many times they are parsed. In the past we could have a configuration file that read: test1 .. test2 .. test3 .. Parsing/adding this file would result in a queue looking like so: test1 .. test2 .. test3 .. test1 .. test2 .. test3 .. test1 .. test2 .. test3 .. Now the queue will *ALWAYS* look like this: test1 .. test2 .. test3 .. In the normal course of events this won't matter, as teh processing loop will look like so: * Add new jobs every minute. * Worker runs the jobs. In the case of a failing job though the test might take 2.5 minutes and that will cause the queue to backup. (2.5 minutes because a test is repeated 5 times before a fail is announced, and the timeout is 30 seconds. These values can and should be tweaked.) With the new method even if the queue is slowly draining the queue will never grow to containu hundreds of events it will just be "topped up" not "overflowing". Thanks to James Hannah for the suggestion, and James Lawrie for the patience.
-rw-r--r--lib/custodian/queue.rb6
1 files changed, 3 insertions, 3 deletions
diff --git a/lib/custodian/queue.rb b/lib/custodian/queue.rb
index bb67c8e..89266df 100644
--- a/lib/custodian/queue.rb
+++ b/lib/custodian/queue.rb
@@ -96,7 +96,7 @@ module Custodian
loop do
- _foo, job = @redis.blpop('queue', :timeout => timeout)
+ job = @redis.spop('queue')
if job
return job
@@ -112,7 +112,7 @@ module Custodian
# Add a new job to the queue.
#
def add(job_string)
- @redis.rpush('queue', job_string)
+ @redis.sadd('queue', job_string)
end
@@ -120,7 +120,7 @@ module Custodian
# How many jobs in the queue?
#
def size?
- @redis.llen('queue')
+ @redis.scard('queue')
end