Klaas van Schelven
82d40e3741
Push get_pc_registry into snappea.foreman init
2024-05-23 10:12:24 +02:00
Klaas van Schelven
5af1d2384e
Performance logging of Snappea task create/delete
2024-05-22 17:45:28 +02:00
Klaas van Schelven
151af98559
Performance logging: push into development.py (i.e. remove from non-development servers)
2024-05-22 17:06:21 +02:00
Klaas van Schelven
b09cfb21c3
Configure runsnappea to be rebooted every day
...
a way to limit memory leaks
also: deal with SIGTERM 'correctly' (i.e. as demanded by systemd)
2024-05-22 12:44:58 +02:00
Klaas van Schelven
f150c839dc
Recommended setup: fix tmpfile troubles in 2 ways
...
* recommend to just run in the home dir
* don't use private tmp
The troubles were: when set up using private tmp files, the 2 processes
cannot communicate with each other
2024-05-22 08:37:52 +02:00
Klaas van Schelven
89dba6e6e5
Fix typo
2024-05-17 15:52:43 +02:00
Klaas van Schelven
5ff2623112
Add checksnappea command
2024-05-17 12:03:40 +02:00
Klaas van Schelven
46220f97ea
Snappea: 'ensure' it is running as a singleton
2024-04-27 21:59:20 +02:00
Klaas van Schelven
8d23239526
Be more strict about usage of TransactionTestCase in tests
2024-04-27 20:35:35 +02:00
Klaas van Schelven
2e6be5cbe7
Wrap Task.objects.all() call in a transaction
2024-04-27 11:07:23 +02:00
Klaas van Schelven
52f720bf4f
Add missing file
2024-04-24 12:30:25 +02:00
Klaas van Schelven
0855299c97
Robustness of Task -> actual running thread mechanism
2024-04-23 15:28:21 +02:00
Klaas van Schelven
8ca2388c45
Factor out wakeup_server
2024-04-23 15:27:57 +02:00
Klaas van Schelven
1e16f80236
Snappea: minor textual in logging
2024-04-23 13:45:09 +02:00
Klaas van Schelven
ab7b7c467b
Snappea: file locations configurable
2024-04-23 13:42:36 +02:00
Klaas van Schelven
9adf971a14
Snappea: settings configurable
2024-04-23 13:32:33 +02:00
Klaas van Schelven
1ef7458619
Snappea: Only do another Task query when the previous result was perhaps limited by the TASK_QS_SIZE
2024-04-23 12:07:03 +02:00
Klaas van Schelven
a7d484f070
Snappea: logging
2024-04-23 11:28:07 +02:00
Klaas van Schelven
08330dd274
Use proper thread-local for wakeup_file uuid part
...
Stackoverflow said "In Python, everything is shared, except for function-local
variables (because each function call gets its own set of locals, and threads
are always separate function calls.) "
whatever was meant exactly by that, it's not true for us, because our functions
are called on-import, and the vars-in-closure are shared between threads. i.e.:
>>> from threading import Thread
>>> def closure():
... l = []
... def inner():
... l.append("x")
... print(len(l))
... return inner
...
>>> inner = closure()
>>> thread = Thread(target=inner)
>>> thread.start()
1
>>> thread = Thread(target=inner)
>>> thread.start()
2
2024-04-23 09:42:24 +02:00
Klaas van Schelven
c7d96c362e
Bind wakeup_filename to the decorator to avoid flooding the associated dir with wakeup signals
2024-04-22 22:56:39 +02:00
Klaas van Schelven
1fc1da1ceb
Implement the registry's 'autodiscover'
...
actually: not autodiscover on-bootup, but just try to import the tasks
when you actually need them. This makes it so that we can avoid the whole
of the Django app-loading magical machinery
2024-04-22 22:37:47 +02:00
Klaas van Schelven
5597e423ea
Document roads-not-taken with the wake-up calls
2024-04-22 15:52:34 +02:00
Klaas van Schelven
4fbc283f3e
Convert snappea 'wakeup' signal to inotify-based
...
see https://github.com/python/cpython/issues/118143
2024-04-22 15:34:33 +02:00
Klaas van Schelven
46f34370f4
Snappea: Document stuff (and factor a few bits out)
2024-04-19 23:14:19 +02:00
Klaas van Schelven
41a4913299
Implement SNAPPEA_TASK_ALWAYS_EAGER
2024-04-19 21:41:42 +02:00
Klaas van Schelven
cb3b236106
Pea -> Task
2024-04-19 21:29:04 +02:00
Klaas van Schelven
8d2713e09d
Logs (slightly) less noisy
2024-04-19 21:24:15 +02:00
Klaas van Schelven
0467e1d316
Snappea: implement graceful shutdown
2024-04-19 21:16:41 +02:00
Klaas van Schelven
3c106521bc
snappea WIP commit
2024-04-19 16:21:42 +02:00
Klaas van Schelven
318a26526d
Yet another WIP
2024-04-19 14:01:06 +02:00
Klaas van Schelven
14fd545f79
WIP: Example with actual workers and waiting for those
2024-04-19 13:12:01 +02:00
Klaas van Schelven
4e484ff3d7
Another midway checkin; demonstrating semaphore usage and how it is robust for multiple signals
2024-04-19 12:11:25 +02:00
Klaas van Schelven
94d0fda6ab
Midway checking to preserve state
2024-04-19 11:58:20 +02:00
