I've done a full grep on Issue.objects, Project.objects and get_object_or_404
equivelents, and applying some common sense. The goal: avoid having
confusing/half-broken pages in the UI.
On index-usage: I've decided not to update the indexes. The assumption is:
`is_deleted` items will be a tiny minority of items in general, making the
cost/benefit analysis not turn out favorably (just scanning them out as a final
step is more efficient). Also: sqlite is able to use the correct index without
adding a special one, proof:
```
EXPLAIN QUERY PLAN SELECT [..] WHERE ("issues_issue"."project_id" = 1 AND "issues_issue"."is_muted" = (0) AND "issues_issue"."is_resolved" = (0)) ORDER BY "issues_issue"."last_seen" DESC LIMIT 250;
QUERY PLAN
`--SEARCH issues_issue USING INDEX issue_list_open (project_id=? AND is_resolved=? AND is_muted=?)
EXPLAIN QUERY PLAN SELECT [..] WHERE ("issues_issue"."project_id" = 1 AND "issues_issue"."is_muted" = (0) AND "issues_issue"."is_resolved" = (0) AND "issues_issue"."is_deleted" = 0) ORDER BY "issues_issue"."last_seen" DESC LIMIT 250;
QUERY PLAN
`--SEARCH issues_issue USING INDEX issue_list_open (project_id=? AND is_resolved=? AND is_muted=?)
```
See #139 for the 0/1 notation in the above.
(Project-indexes: not an issue, the scale is "below relevance for indexes")
Implemented using a batch-wise dependency-scanner in delayed
(snappea) style.
* no real point-of-entry in the (regular, non-admin) UI yet.
* no hiding of Projects which are delete-in-progress from the UI
* lack of DRY
* some unnessary work (needed in the Issue-context, but not here)
is still being done.
See #50
"possibly expensive" turned out to be "actually expensive". On 'emu', with 1.5M
events, the counts take 85 and 154 ms for Project and Issue respectively;
bottlenecking our digestion to ~3 events/s.
Note: this is single-issue, single-project (presumably, the cost would be lower
for more spread-out cases)
Note on indexes: Event already has indexes for both Project & Issue (though as
the first item in a multi-column index). Without checking further: that appears
to not "magically solve counting".
This commit also optimizes the .count() on the issue-detail event list (via
Paginator).
This commit also slightly changes the value passed as `stored_event_count` to
be used for `get_random_irrelevance` to be the post-evication value. That won't
matter much in practice, but is slightly more correct IMHO.
An event always has a single (automatically calculated) Grouping associated with it.
We add this info to the Event model (we'll soon display it in the UI, and as per the
now-removed comment it's simply the consistent thing to do)
In 59372aba33 a lazily evaluated BASE_URL tool was introduced.
I found 2 more cases in which BASE_URL was not "collapsed into a
string" magically by `__add__`, one of which actually causing
an `Object of type TenantBaseURL is not JSON serializable`
As had been noted on some of the commands, 'ingest' was not the best place for
them. However, [project-level apps are not supported in
Django](https://forum.djangoproject.com/t/allow-project-to-have-management-commands/5220/2)
So just create a 'main' app. I want to qualify it as 'myproject-main' though, to avoid
further unqualified global namespace pollution. And I want to avoid prefixing with 'bugsink'
b/c that's annoying for tab-completion. So 'bs' it is.
I've moved all commands over; even though a case could be made that the "feeding" commands
(raise_exception, send_json, stress_test) are somewhat related to ingestion, that's not
a very good case :-)
We don't use fastjsonschema's exception info, falling back to jsonschema for that. Not using the info is faster.
Possible because https://github.com/horejsek/python-fastjsonschema/issues/187 was completed.
I did not at present do any speed/memory checks, but the diff is quite promising :-)
in IMMEDIATE mode, project was passed from an (implicit) read transaction to
the immediate_atomic-wrapped code; this meant that it was possible to miscount
project.digested_event_count
I'm sure this can be useful, but since I validate before storing, this is just
a ReallyBadIdea(TM). For one, it makes it so that running with different settings
of VALIDATE_ON_DIGEST leads to different data in the DB
