summaryrefslogtreecommitdiff
path: root/lib/custodian/queue.rb
AgeCommit message (Collapse)Author
2015-12-17Treat our Redis queue as a set.Steve Kemp
This means that tests will only ever be enqueued once, regardless of how many times they are parsed. In the past we could have a configuration file that read: test1 .. test2 .. test3 .. Parsing/adding this file would result in a queue looking like so: test1 .. test2 .. test3 .. test1 .. test2 .. test3 .. test1 .. test2 .. test3 .. Now the queue will *ALWAYS* look like this: test1 .. test2 .. test3 .. In the normal course of events this won't matter, as teh processing loop will look like so: * Add new jobs every minute. * Worker runs the jobs. In the case of a failing job though the test might take 2.5 minutes and that will cause the queue to backup. (2.5 minutes because a test is repeated 5 times before a fail is announced, and the timeout is 30 seconds. These values can and should be tweaked.) With the new method even if the queue is slowly draining the queue will never grow to containu hundreds of events it will just be "topped up" not "overflowing". Thanks to James Hannah for the suggestion, and James Lawrie for the patience.
2015-04-16Mark variables as unused.Steve Kemp
We've now prefixed our transient/unused variables with "_" to stop any future errors about variables not being used.
2015-03-09Removed trailing whitespace from the codeSteve Kemp
2015-03-09Remove spaces inside blocks.Steve Kemp
2015-03-09Whitespace fixups.Steve Kemp
These were all identified and suggested by rubocop.
2015-03-09Use def with parentheses when there are parameters.Steve Kemp
2015-03-09Removed spaces inside parenthesis.Steve Kemp
2015-03-09Prefer single-quotes when you don't need interpolation.Steve Kemp
So "foo" is less good than 'foo'.
2015-03-09Don't use parenthesis aroudn conditions in an if.Steve Kemp
2015-03-09Do not use parentheses for method calls with no arguments.Steve Kemp
This is neater. Flagged by rubocop
2015-03-09Omit the parentheses in defs when the method doesn't accept any arguments.Steve Kemp
This is neater.
2015-03-09Avoid arguments we're not using.Steve Kemp
Having methods take arguments which are ignored is a misleading thing, prefix with "_" to make that explicit, or remove.
2015-02-18Prefer "while true" to "while 1"Steve Kemp
Also only sleep between fetches if we couldn't find a job because the queue was empty.
2015-02-18Minor formatting and commenting update.Steve Kemp
2015-02-18Updated the way we wait for jobs.Steve Kemp
The new approach uses the redis gems timeout functionality and ensures we never return a null-job. Instead we timeout and repeat with a stalling-sleep in the way. This closes #9553.
2015-02-17fix the redis queue not to melt the cpu when polling redis for a jobPatrick J Cherry
2015-02-05Simplify code by removing configuration.Steve Kemp
We now set the queue-address via $QUEUE_ADDRESS, otherwise we default to localhost. This works for both redis & beanstalkd.
2015-02-05Simplify the flushing of queues.Steve Kemp
Now that we've moved to using redis by default the handling of queue-flushing needs to change. We can simply get rid of the busy-wait and run a redis "del" operation. With that in mind we've moved the flushing logic to our queue abstraction layer, and simplified our queue-helper script.
2015-01-29Don't output messages unless $DEBUG=1Steve Kemp
2015-01-19Updated to use our queue-abstraction.Steve Kemp
This means we can queue/dequeue to either Redis or Beanstalkd.
2015-01-13Added queue-abstraction layer.Steve Kemp
This is not-yet used, but it will-be shortly. The intention is that we can seamlessly swap out the queue implemention in the near future so that we'll be able to use Redis.
2012-11-20 RemovedSteve Kemp
2012-11-20 New stub interface.Steve Kemp