Commit Graph

184 Commits

Author SHA1 Message Date
Klaas van Schelven
6b8d912e1a Store remote_addr on the event
Fix #165
2025-07-25 21:54:32 +02:00
Klaas van Schelven
c4fe1c1292 Debug IDs for missing sourcemaps: show them right in the stacktrace
See #158
2025-07-23 10:57:30 +02:00
Klaas van Schelven
1983633b38 Sourcemap Images IDs: show those in event_details
See #158
2025-07-23 10:38:45 +02:00
Klaas van Schelven
99f782f4e3 add vacuum_files command
Fix #129
2025-07-17 09:05:16 +02:00
Klaas van Schelven
403e28adb4 Remove 'choices' from Event.model
See https://github.com/getsentry/sentry-docs/pull/14331

See https://code.djangoproject.com/ticket/22837 for the migration-clobbering
2025-07-16 10:38:06 +02:00
Klaas van Schelven
b2769d7202 cleanup_eventstorage command: be more clear when no event_storage is actually configured 2025-07-14 15:48:47 +02:00
Klaas van Schelven
0d8d55a451 Fix: don't do 'contains' check on the error message
probably worked in 'some cases'; probably caused by accepting
a copilot suggestion a bit too eagerly
2025-07-14 15:35:44 +02:00
Klaas van Schelven
7419c924c1 make_consistent: handling of Issue/Project .is_deleted
as an 'enqueue again', per the comment

See #50
2025-07-07 16:43:47 +02:00
Klaas van Schelven
cb3168133e Fix Event-deletion from the admin
b/c of the introduction of on_delete=DO_NOTHING, this was broken.
this is the quick & dirty fix (no tests)
2025-07-04 17:33:02 +02:00
Klaas van Schelven
28b2ce0eaf Various models: .project SET_NULL => DO_NOTHING
Like e45c61d6f0, but for .project.

I originally thought `SET_NULL` would be a good way to "do stuff later", but
that's only so the degree that [1] updates are cheaper than deletes and [2]
2nd-order effects (further deletes in the dep-tree) are avoided.

Now that we have explicit Project-deletion (deps-first, delayed, properly batched)
the SET_NULL behavior is always a no-op (but with cost in queries).

As a result, in the test for project deletion (which has deletes for many
of the altered models), the following 12 queries are no longer done:

```
SELECT "projects_project"."id", [..many fields..] FROM "projects_project" WHERE "projects_project"."id" = 1
DELETE FROM "projects_projectmembership" WHERE "projects_projectmembership"."project_id" IN (1)
DELETE FROM "alerts_messagingserviceconfig" WHERE "alerts_messagingserviceconfig"."project_id" IN (1)
UPDATE "releases_release" SET "project_id" = NULL WHERE "releases_release"."project_id" IN (1)
UPDATE "issues_issue" SET "project_id" = NULL WHERE "issues_issue"."project_id" IN (1)
UPDATE "issues_grouping" SET "project_id" = NULL WHERE "issues_grouping"."project_id" IN (1)
UPDATE "events_event" SET "project_id" = NULL WHERE "events_event"."project_id" IN (1)
UPDATE "tags_tagkey" SET "project_id" = NULL WHERE "tags_tagkey"."project_id" IN (1)
UPDATE "tags_tagvalue" SET "project_id" = NULL WHERE "tags_tagvalue"."project_id" IN (1)
UPDATE "tags_eventtag" SET "project_id" = NULL WHERE "tags_eventtag"."project_id" IN (1)
UPDATE "tags_issuetag" SET "project_id" = NULL WHERE "tags_issuetag"."project_id" IN (1)
```
2025-07-03 21:49:49 +02:00
Klaas van Schelven
ad2aa08e0a Rename local var. for understanding 2025-07-03 11:44:01 +02:00
Klaas van Schelven
3b3ce782c5 Fix the tests for prev. commit
tests were broken b/c not respecting constraints / not properly using the factories.
2025-07-03 11:33:58 +02:00
Klaas van Schelven
e45c61d6f0 Various models: .issue and .grouping; SET_NULL => DO_NOTHING
I originally thought `SET_NULL` would be a good way to "do stuff later", but
that's only so the degree that [1] updates are cheaper than deletes and [2]
2nd-order effects (further deletes in the dep-tree) are avoided.

Now that we have explicit Issue-deletion (deps-first, delayed, properly batched)
the SET_NULL behavior is always a no-op (but with cost in queries).

As a result, in the test for issue deletion (which has deletes for many
of the altered models), the following 8 queries are no longer done:

```
SELECT "issues_grouping"."id", [..many fields..] FROM "issues_grouping" WHERE "issues_grouping"."id" IN (1)
UPDATE "events_event" SET "grouping_id" = NULL WHERE "events_event"."grouping_id" IN (1)

[.. a few moments later..]

SELECT "issues_issue"."id", [..many fields..] FROM "issues_issue" WHERE "issues_issue"."id" = 'uuid'
UPDATE "issues_grouping" SET "issue_id" = NULL WHERE "issues_grouping"."issue_id" IN ('uuid')
UPDATE "issues_turningpoint" SET "issue_id" = NULL WHERE "issues_turningpoint"."issue_id" IN ('uuid')
UPDATE "events_event" SET "issue_id" = NULL WHERE "events_event"."issue_id" IN ('uuid')
UPDATE "tags_eventtag" SET "issue_id" = NULL WHERE "tags_eventtag"."issue_id" IN ('uuid')
UPDATE "tags_issuetag" SET "issue_id" = NULL WHERE "tags_issuetag"."issue_id" IN ('uuid')
```

(breaks the tests b/c of constraints and not always using factories; will fix next)
2025-07-03 11:33:58 +02:00
Klaas van Schelven
4ca15c7159 fix make_consistent on mysql
Problem: on mysql `make_consistent` cannot always clean up `Event`s, because
`EventTag` objects still point to them, leading to an integrityerror.
The problem does not happen for `sqlite`, because sqlite does FK-checks
on-commit. And the offending `EventTag` objects are "eventually cleaned up" (in
the same transaction, in make_consistent)

This is the "mostly works" solution, for the scenario we've encountered.
Namely: remove EventTags which have no issue before removing Events. This works
in practice because of the way Events-to-cleanup were created in the UI in
practice, namely by removal of some Issue in the admin, triggering a `SET_NULL`
on the `issue_id`. Removal of issue implies an analagous `SET_NULL` on the
`EventTag`'s `issue_id`, and by removing those `EventTag`s before proceeding
with the `Event`s, you avoid the FK constraint triggering.

We don't want to fully reimplement `CASCADE` (as in Django) here, and the
values of `on_delete` are "Design Decision Needed" and non-homogonous anyway,
and we might soon implement proper deletions (see #50) anyway, so the "mostly
works" solution will have to do for now.

Fixes #132
2025-06-26 10:56:31 +02:00
Klaas van Schelven
a835449bef flake8 2025-05-28 22:24:19 +02:00
Klaas van Schelven
46c9f3d7cc Merge pull request #87 from bpeschier/fix/sourcemap-with-multiple-sources
* Apply source mapping when sourcemap contains multiple sources
* Propagate `filename` and `function` from the sourcemap.
2025-05-28 22:18:02 +02:00
Klaas van Schelven
f817b3ba2f Lookup by debug_id in dicts: use UUID
in absence of a clean reproducer, just doing proper lookups on a
normalized thing seems the more stable thing to do.

Fix #105 (in absence of test: I think)
2025-05-28 22:03:51 +02:00
Klaas van Schelven
a5401414db Fix deprecationwarnings (about Django 5.0) in the tests
by changing an import
2025-04-26 10:52:04 +02:00
Bas Peschier
bc3daf7662 Apply source mapping when sourcemap contains mappings for multiple sources. 2025-04-24 15:45:28 +02:00
Klaas van Schelven
dd339f73de One more fix for psycopg<3.0.17
see ce9d0e4941
2025-04-14 11:49:41 +02:00
Klaas van Schelven
ce9d0e4941 Fix for psycopg<3.0.17 2025-04-14 11:29:12 +02:00
Klaas van Schelven
712eb63faa Merge pull request #80 from bugsink/sourcemaps-spike
Sourcemaps: first version
2025-04-14 11:08:19 +02:00
Klaas van Schelven
f95927ffd6 annotate_with_meta: add a case for (typically Filtered) string 2025-04-11 22:40:51 +02:00
Klaas van Schelven
87130043e3 PoC of using sourcemaps in the UI
See #19
2025-04-10 15:52:34 +02:00
Klaas van Schelven
a097e25310 issue.stored_event_count: consequences for 'irrelevance'
document & assert
2025-03-31 15:25:59 +02:00
Klaas van Schelven
9b1911aded Fix issue.stored_event_count for eviction/retention 2025-03-31 14:51:58 +02:00
Klaas van Schelven
3c35ea5398 Eviction: note on newest-first behavior 2025-03-19 16:00:38 +01:00
Klaas van Schelven
6948f3a2e1 Dead code removal
'allowed as pass-in' but in fact we always pass-in
2025-03-19 15:53:51 +01:00
Klaas van Schelven
72ab0c68ef Log message: no need to mention 'include_never_evict'
because when you reach that point, it's always True
2025-03-19 15:43:05 +01:00
Klaas van Schelven
a2f3ad900b eviction-target not reached handling changes
this error has shown up for one of our users; I can't reproduce yet, but I can
make it better:

* log-don't-crash: not worth failing for this (drops the event, and also
  rolls back the transaction such that nothing is achieved regarding eviction)
* provide more info on-error (various counts)

NB: I've also changed the < into a <=, and combined it with a check on "loop
not done". I _think_ they are functionally equivalent, and that the new version
is simply more clear as well as slightly more efficient.

In my understanding: the old version simply looped one more time before giving
up (because it was < it needed one more iteration, and because there was no
explicit check on 'loop done' that inefficiency was needed in the old formulation).
I say "I think" because I don't have a test specific to the edge-case.
2025-03-19 15:32:39 +01:00
Klaas van Schelven
fc4aae2dea Retention tests: hit even more edge-cases 2025-03-19 14:49:29 +01:00
Klaas van Schelven
1d0c0c65ff Retention tests/clarification: filter_for_work 2025-03-19 14:33:00 +01:00
Klaas van Schelven
d3c6627556 Add a more complicated case to the retention tests
this one tests at least multiple epochs and irrelevances
2025-03-19 14:18:28 +01:00
Klaas van Schelven
1b7865d3b9 Eviction: Tests and rewrite-for-understanding of epoch_bounds_with_irrelevance 2025-03-19 11:56:55 +01:00
Klaas van Schelven
f548eab778 Merge branch 'main' into tag-search 2025-03-10 09:09:40 +01:00
Klaas van Schelven
4b079487ca make_consistent: explain the use of 'dangling fk cleanup' 2025-03-07 16:36:10 +01:00
Klaas van Schelven
0ade3c0f86 Add a comment about DB-CASCADE 2025-03-07 16:35:36 +01:00
Klaas van Schelven
24b0c32281 make_consistent: tag-related models added 2025-03-07 13:57:22 +01:00
Klaas van Schelven
96e07c4dc3 Tags: delete EventTag when Events are evicted
and document related things
2025-03-07 13:50:10 +01:00
Klaas van Schelven
97f03a8951 Rewrite 'eviction_target' comment 2025-03-06 14:03:51 +01:00
Klaas van Schelven
f76d3f4f40 Merge branch 'main' into tag-search 2025-03-05 16:05:17 +01:00
Klaas van Schelven
f25d693804 Add postgres to GitHub CI
A first step towards (experimental) postgres support, see #21
2025-03-05 12:20:51 +01:00
Klaas van Schelven
b98c1d4f44 MySQL recover from IntegrityError
as we do for the sqlite case

the need for this was revealed by running the testsuite against mysql
2025-03-05 11:04:41 +01:00
Klaas van Schelven
4cde74d7cb Event search: first version 2025-03-04 13:51:56 +01:00
Klaas van Schelven
a00a815261 Merge branch 'main' into tag-search 2025-03-03 15:02:13 +01:00
Klaas van Schelven
c8ecf508de Tags: on event details page show calculated tags
(not just the explicitly provided ones)
2025-03-03 11:29:07 +01:00
Klaas van Schelven
adf92f6b1b make_consistent: update has_releases when needed
See #50
2025-03-03 09:06:31 +01:00
Klaas van Schelven
e10c1bf7ca Remove 'store_events' command
this command to store all events on the local filesystem was useful
while 'scaffolding'; getting my hands on some initial event-data in
the early days of Bugsink, but it was never meant as a permanent tool
2025-02-27 15:14:05 +01:00
Klaas van Schelven
7a19e2d277 Tags; deducing tags; search on tags; WIP 2025-02-27 13:12:49 +01:00
Klaas van Schelven
4b7ed8f4ec Rename get_contexts_enriched_with_ua
more closely match what's going on
2025-02-26 18:20:52 +01:00