From b1738fff13648331718c33c89face833bf6666ad Mon Sep 17 00:00:00 2001 From: Klaas van Schelven Date: Tue, 9 Jan 2024 19:20:52 +0100 Subject: [PATCH] Document a few thoughts on celery --- DESIGN-process-model.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 DESIGN-process-model.md diff --git a/DESIGN-process-model.md b/DESIGN-process-model.md new file mode 100644 index 0000000..b10cd5b --- /dev/null +++ b/DESIGN-process-model.md @@ -0,0 +1,26 @@ + + +Just a few thoughts on the model we use for multi-processes, multi-threads, and how this relates to DB locks. + + +Basic idea: if we run a single (gunicorn) process, locks can't ever become a problem. + + +### Celery + +For now, we use Celery. +It's the most out-of-the-box solution for async stuff (which we need for e.g. emails) + +I wonder if we'll still use this once we do a serious cutting up between ingest/digest, but we'll cross that bridge when +we get there. + +For now I'm just running in always-eager mode to facilitate running debugserver. This will probably change at some point. + +If you'd need a worker, this is how you'd start it: + +``` +celery -A bugsink worker -l INFO +``` + +(I wonder: if single-worker is such a key concept, this may need to be applied on celery too... but that depends +entirely on what we let celery do)