Triggered by issue_event_list being more than 5s on "emu" (my 1,500,000 event
test-machine). Reason: sorting those events on non-indexed field. Switching
to a field-with-index solved it.
I then analysed (grepped) for "ordering" and "order_by" and set indexes
accordingly and more or less indiscriminately (i.e. even on tables that are
assumed to have relatively few rows, such as Project & Team).
As discussed in #11, there are scenarios (e.g. misconfiguration) where snappea
does not pick up the tasks. Events not showing up in Bugsink, w/o further
indication why that may be, leaves people confused. Better to warn explicitly
in that case.
Using a pid-file that's implied by the ingestion directory.
We do this in `get_pc_registry`, i.e. on the first request. This means failure is
in the first request on the 2nd process.
Why not on startup? Because we don't have a configtest or generic on-startup location
(yet). Making _that_ could be another source of fragility, and getting e.g. the nr
of processes might be non-trivial / config-dependent.
exposed when playing around with arbitrary Tasks in a shell; this created
workers I could not run, which would put the foreman in a 'waiting for available threads'
mode.
I briefly looked at the rest of that loop to see whether more exception handling
is necessary, but TBH I don't think we can reasonably recover from e.g. task.delete()
failing (or at least I don't want to think about it now)
* recommend to just run in the home dir
* don't use private tmp
The troubles were: when set up using private tmp files, the 2 processes
cannot communicate with each other
Stackoverflow said "In Python, everything is shared, except for function-local
variables (because each function call gets its own set of locals, and threads
are always separate function calls.) "
whatever was meant exactly by that, it's not true for us, because our functions
are called on-import, and the vars-in-closure are shared between threads. i.e.:
>>> from threading import Thread
>>> def closure():
... l = []
... def inner():
... l.append("x")
... print(len(l))
... return inner
...
>>> inner = closure()
>>> thread = Thread(target=inner)
>>> thread.start()
1
>>> thread = Thread(target=inner)
>>> thread.start()
2
actually: not autodiscover on-bootup, but just try to import the tasks
when you actually need them. This makes it so that we can avoid the whole
of the Django app-loading magical machinery
Klaas van Schelven