in IMMEDIATE mode, project was passed from an (implicit) read transaction to
the immediate_atomic-wrapped code; this meant that it was possible to miscount
project.digested_event_count
I'm sure this can be useful, but since I validate before storing, this is just
a ReallyBadIdea(TM). For one, it makes it so that running with different settings
of VALIDATE_ON_DIGEST leads to different data in the DB
we've observed that this is sent-in-practice. we don't want to
actually remove it from the stored data, so we remove it only
from the thing we validate (not the actual data)
* As per the spec 'takes precendence'
* Also fixes the reported bug on Laravel, which apparently doesn't send event_id
as part of the event payload.
* Fixes the envelope tests (they were doing nothing when I moved the
data samples around recently)
* Adds a 'no event_id in data, but yes in envelope' test to that test.
* Adds handling to send_json such that we can send envelopes when the event_id
is missing from the event data.
This reverts course on 4201fbd778, and restores event.schema.json from that
commit. In that commit we said: 'this is not used'. Not true: it's used in a
test, though this test used the validity check to silently skip.
In this commit:
1. Do _not_ just silently skip invalid samples. Since we have a way of properly
validating, let's use that so that we know how useful the samples that we have
actually are.
2. Deal with "_meta", a field that we sometimes see in the "private samples" (data
that ultimately comes from running a somewhat recent python-sdk against my
actual codebase). The need for this was exposed by [1]
3. Add a test for the up-to-date-ness of event.json.schema
4. remove special-cased attribute-checks in `is_valid`; `send_json` was, at the
time, an opportunistic way to just get my hands on some sample data. the
approach at validation reflected that: I just did some tests on the existence
of certain attributes to determine which json files were even events. But in
the end I did a full validation using an API schema, which kinda made the
whole business useless. This commit cleans up the individual checks.
In the previous commit I put the code for a small performance-experiment.
The results are (very) obvious: don't do this. Response times go through
the roof, and more importantly, the server becomes unreliable. Reason:
time-outs caused by waiting for the write-lock.
