abelanger5 7b7fbe3668 fix: update Requeue and Reassign logic to fix performance degradation when many events are queued (#310) ...

Logic for requeueing and reassigning did not limit the number of step runs to requeue, so when events accumulate with no worker present it causes memory to spike along with a very high query latency on the database. This commit limits the number of step runs returned in the requeue and reassign queries, and also properly locks step run rows for these queries so only a step run in a PENDING or PENDING_ASSIGNMENT state can be requeued.

It also improves performance of the `AssignStepRunToWorker` query and ensures that `maxRuns` on workers are always respected through the introduction of a `WorkerSemaphore` model. The value gets decremented when a step run is assigned and incremented when a step run is in a final state. 

Co-authored-by: Luca Steeb <contact@luca-steeb.com>

* Update controller.go

---------

Co-authored-by: steebchen <contact@luca-steeb.com>

2024-04-01 12:33:18 -04:00

..

rabbitmq_test.go

fix: deadlocks on workers and tickers (#241)

2024-03-12 00:45:18 -04:00

rabbitmq.go

fix: update Requeue and Reassign logic to fix performance degradation when many events are queued (#310)

2024-04-01 12:33:18 -04:00