| Age | Commit message (Collapse) | Author |
|---|
|
The beanstalkd queue used to be used in the past, and we later
added support for Redis via a simple abstraction layer. But now
we've no longer tested and used beanstalkd for over a year, and
the client-libraries are no longer available as native Debian
packages.
With that in mind we've excised the code, although left the
abstraction-class in-place.
|
|
|
|
This ensures that all tests always run, and we have an ordering.
|
|
This means that tests will only ever be enqueued once, regardless
of how many times they are parsed.
In the past we could have a configuration file that read:
test1 ..
test2 ..
test3 ..
Parsing/adding this file would result in a queue looking like so:
test1 ..
test2 ..
test3 ..
test1 ..
test2 ..
test3 ..
test1 ..
test2 ..
test3 ..
Now the queue will *ALWAYS* look like this:
test1 ..
test2 ..
test3 ..
In the normal course of events this won't matter, as teh processing
loop will look like so:
* Add new jobs every minute.
* Worker runs the jobs.
In the case of a failing job though the test might take 2.5 minutes
and that will cause the queue to backup. (2.5 minutes because a test
is repeated 5 times before a fail is announced, and the timeout is
30 seconds. These values can and should be tweaked.)
With the new method even if the queue is slowly draining the queue
will never grow to containu hundreds of events it will just be "topped
up" not "overflowing".
Thanks to James Hannah for the suggestion, and James Lawrie for
the patience.
|
|
We've now prefixed our transient/unused variables with "_" to stop
any future errors about variables not being used.
|
|
|
|
|
|
These were all identified and suggested by rubocop.
|
|
|
|
|
|
So "foo" is less good than 'foo'.
|
|
|
|
This is neater. Flagged by rubocop
|
|
This is neater.
|
|
Having methods take arguments which are ignored is a misleading thing,
prefix with "_" to make that explicit, or remove.
|
|
Also only sleep between fetches if we couldn't find a job because
the queue was empty.
|
|
|
|
The new approach uses the redis gems timeout functionality
and ensures we never return a null-job. Instead we timeout
and repeat with a stalling-sleep in the way.
This closes #9553.
|
|
|
|
We now set the queue-address via $QUEUE_ADDRESS, otherwise
we default to localhost. This works for both redis & beanstalkd.
|
|
Now that we've moved to using redis by default the handling of queue-flushing
needs to change. We can simply get rid of the busy-wait and run a redis
"del" operation.
With that in mind we've moved the flushing logic to our queue abstraction
layer, and simplified our queue-helper script.
|
|
|
|
This means we can queue/dequeue to either Redis or Beanstalkd.
|
|
This is not-yet used, but it will-be shortly. The intention is
that we can seamlessly swap out the queue implemention in the
near future so that we'll be able to use Redis.
|
|
|
|
|
