471b69e956
Stress test: ability to generate random event types:
2024-06-27 10:49:25 +02:00
c5df10e9cf
Stress test: ability to use multiple dsns (projects)
2024-06-27 09:52:10 +02:00
fe6c955465
never_evict events that are a Historic Turning Point
...
Both for technical (foreign keys) and business reasons (these are events you
care about)
2024-06-24 22:50:00 +02:00
ea6aa9bbca
Retention/quotas: something that 'seems to work' (doesn't immediately crash)
2024-06-21 11:50:13 +02:00
c2b821589d
Retention, WIP (yesterday)
2024-06-21 09:28:04 +02:00
7cce0c58ab
Simplify code
...
by moving the updating of the denormalized fields up, we can remove an assymmetry
2024-06-20 09:12:24 +02:00
a1e842fee1
Remove no-longer-true/relevant comment
...
The shaving-off of queries that's discussed in the comment is no longer
relevant because the associated branch is fully seen as ValidationError
these days.
2024-06-19 16:58:21 +02:00
8ad6059722
Complete migration reset
2024-06-14 10:29:10 +02:00
fb66b04be9
Document playground.bugsink.com performance findings
2024-05-23 14:24:04 +02:00
4647c1b498
Stress test: better stats
2024-05-23 12:36:14 +02:00
f0c255a346
Force envelope use in stress tests
2024-05-23 11:02:44 +02:00
23d7f2172d
Add showstat (for snappea queue size)
2024-05-23 08:37:48 +02:00
78a85087a2
'performance' info for event bytes count
2024-05-22 20:43:10 +02:00
c92e3ec772
Add INGEST_STORE_BASE_DIR
2024-05-20 09:44:38 +02:00
53353d2d8c
Add some more percentiles
2024-05-19 15:36:27 +02:00
12acd2c950
Add test for digest-non-immediate
2024-04-27 21:39:56 +02:00
ab791941ed
Add test for envelope ingestion
2024-04-27 21:24:25 +02:00
7e32893b78
Fix the store endpoint (but always DIGEST_IMMEDIATELY)
2024-04-27 21:13:29 +02:00
2ddb773c6b
Remove temp files after delayed digestion
2024-04-27 20:48:43 +02:00
4b509ac13b
Stress test: print some stats
2024-04-27 20:45:59 +02:00
4a6196c267
Leave TODO about pc_registry and multi-process setups
2024-04-26 20:56:56 +02:00
61d5615d5d
Add stress-test command (using copy/paste from send_json)
2024-04-26 16:11:38 +02:00
c87582fba6
Don't fsync
...
It's (very, in the order of 10ms) costly.
As I understand it now: the reason is for durability in case of power outages. But that's not
what we're looking for here. We merely want "available in another process". Which I think should
just be the case after a flush().
2024-04-26 15:26:41 +02:00
58435754e0
Digest immediately implemented
2024-04-26 14:48:07 +02:00
7bfad36376
Implement brotli decompression
2024-04-25 15:56:11 +02:00
41e5331f35
Comment about relative performance of some key steps in Envelope unpacking
2024-04-25 15:34:51 +02:00
f3b090a859
Document why we do streaming parsing
2024-04-25 15:25:15 +02:00
d3ac2d0568
(configurable) max header size (both envelope and item header)
2024-04-25 15:07:25 +02:00
234008bee3
Don't subject non-event types to event-type size limit
2024-04-25 14:59:24 +02:00
58d448350a
Implement MAX sizes for event & envelope
2024-04-25 14:56:11 +02:00
387234b847
Read the Envelope specification once more, implement some more details
2024-04-25 13:39:52 +02:00
fb2ddc718c
Log non-event items
2024-04-25 13:09:07 +02:00
4d996e0c51
Use my own impl. of conent_encoding based decompress
...
In the process: make it read-read rather than read-write
2024-04-25 12:10:15 +02:00
c3af388461
envelope endpoint: use StreamingEnvelopeParser
...
in the process, the DRF was removed (it's non-streaming)
2024-04-25 11:39:19 +02:00
d099d43418
send_json: ability to send envelopes
2024-04-25 11:38:29 +02:00
1ad14e72fd
Check that an item is present when a header is given
...
Refactor: NewlineFinder always accepts EOF as an implicit ending
2024-04-25 10:41:43 +02:00
13913c37c4
Implement EnvelopeParser (at least all examples implemented)
2024-04-25 10:20:45 +02:00
5d874f7a46
EnvelopeParser's read_until: reimplement using output-stream; write tests
2024-04-25 08:54:01 +02:00
5d5c49899e
WIP: implement envelope parsing myself, and actually according to spec
2024-04-24 21:46:33 +02:00
5a0195c533
Add 'deflate' as an option to send_json command
2024-04-24 15:44:02 +02:00
483aca6802
send_json: ability to send gzipped
2024-04-24 15:29:39 +02:00
41a4913299
Implement SNAPPEA_TASK_ALWAYS_EAGER
2024-04-19 21:41:42 +02:00
318a26526d
Yet another WIP
2024-04-19 14:01:06 +02:00
059f831816
Document thoughts on the model for write-transactions
2024-04-19 09:23:13 +02:00
6bb7ce89a1
Factor out delay_on_commit
2024-04-19 08:18:19 +02:00
c50780ab4e
Use atomic transactions in views
2024-04-18 13:15:46 +02:00
1e1f7c4a6d
Fix tests (for recent timezone change)
2024-04-18 08:39:29 +02:00
8c7e1f1cf2
Configure sqlite WAL and related PRAGMAs
2024-04-17 21:50:39 +02:00
5ae0a8227f
Implement bookkeeping of events_at
...
(I didn't think the effects on regressions through, but this will at least
manifest itself because you cannot mark an issue as "fixed in" a release
in which it occurs. It will also show up once we start displaying "events_at"
in the UI, which should be "soon")
2024-04-16 13:25:18 +02:00
d89e3d4dd5
Add 'next-materialized historic annotation
2024-04-16 09:31:12 +02:00
Klaas van Schelven