Commit Graph

1584 Commits

Author SHA1 Message Date
Klaas van Schelven
e861280c1f Merge pull request #137 from bugsink/project-delete
Project delete
2025-07-04 10:11:39 +02:00
Klaas van Schelven
4900f0447e Project-deletion: slight optimization
Removes the following 2 redundant queries from the deletion process:

```
SELECT "tags_tagkey"."id" FROM "tags_tagkey" WHERE "tags_tagkey"."project_id" IN (1) ORDER BY "tags_tagkey"."project_id" ASC, "tags_tagkey"."id" ASC LIMIT 498
UPDATE "projects_project" SET "stored_event_count" = ("projects_project"."stored_event_count" - 1) WHERE "projects_project"."id" = 1
```
2025-07-03 22:04:51 +02:00
Klaas van Schelven
28b2ce0eaf Various models: .project SET_NULL => DO_NOTHING
Like e45c61d6f0, but for .project.

I originally thought `SET_NULL` would be a good way to "do stuff later", but
that's only so the degree that [1] updates are cheaper than deletes and [2]
2nd-order effects (further deletes in the dep-tree) are avoided.

Now that we have explicit Project-deletion (deps-first, delayed, properly batched)
the SET_NULL behavior is always a no-op (but with cost in queries).

As a result, in the test for project deletion (which has deletes for many
of the altered models), the following 12 queries are no longer done:

```
SELECT "projects_project"."id", [..many fields..] FROM "projects_project" WHERE "projects_project"."id" = 1
DELETE FROM "projects_projectmembership" WHERE "projects_projectmembership"."project_id" IN (1)
DELETE FROM "alerts_messagingserviceconfig" WHERE "alerts_messagingserviceconfig"."project_id" IN (1)
UPDATE "releases_release" SET "project_id" = NULL WHERE "releases_release"."project_id" IN (1)
UPDATE "issues_issue" SET "project_id" = NULL WHERE "issues_issue"."project_id" IN (1)
UPDATE "issues_grouping" SET "project_id" = NULL WHERE "issues_grouping"."project_id" IN (1)
UPDATE "events_event" SET "project_id" = NULL WHERE "events_event"."project_id" IN (1)
UPDATE "tags_tagkey" SET "project_id" = NULL WHERE "tags_tagkey"."project_id" IN (1)
UPDATE "tags_tagvalue" SET "project_id" = NULL WHERE "tags_tagvalue"."project_id" IN (1)
UPDATE "tags_eventtag" SET "project_id" = NULL WHERE "tags_eventtag"."project_id" IN (1)
UPDATE "tags_issuetag" SET "project_id" = NULL WHERE "tags_issuetag"."project_id" IN (1)
```
2025-07-03 21:49:49 +02:00
Klaas van Schelven
6b9e4d8011 Project.delete_deferred(): first version (WIP)
Implemented using a batch-wise dependency-scanner in delayed
(snappea) style.

* no real point-of-entry in the (regular, non-admin) UI yet.
* no hiding of Projects which are delete-in-progress from the UI

* lack of DRY
* some unnessary work (needed in the Issue-context, but not here)
  is still being done.

See #50
2025-07-03 21:01:28 +02:00
Klaas van Schelven
76dc2ede16 Merge pull request #136 from bugsink/issue-delete
Issue delete (with cleanup async)
2025-07-03 14:52:48 +02:00
Klaas van Schelven
99f71b8a47 Fix the tests (query-counting across DBs) 2025-07-03 14:48:51 +02:00
Klaas van Schelven
eb5e2d2a48 Fix 'delay_on_commit' calls for project-id passing-around 2025-07-03 13:46:40 +02:00
Klaas van Schelven
6a75216c8f Document issue-deletion batch size budget 2025-07-03 13:46:16 +02:00
Klaas van Schelven
5637e04cec Assert no automatic cascades happen in our manual dep-deletion 2025-07-03 13:23:08 +02:00
Klaas van Schelven
2baa4446fd Issue deletion: event pre-deletes (storage, project-counts) 2025-07-03 13:18:28 +02:00
Klaas van Schelven
ad2aa08e0a Rename local var. for understanding 2025-07-03 11:44:01 +02:00
Klaas van Schelven
3b3ce782c5 Fix the tests for prev. commit
tests were broken b/c not respecting constraints / not properly using the factories.
2025-07-03 11:33:58 +02:00
Klaas van Schelven
e45c61d6f0 Various models: .issue and .grouping; SET_NULL => DO_NOTHING
I originally thought `SET_NULL` would be a good way to "do stuff later", but
that's only so the degree that [1] updates are cheaper than deletes and [2]
2nd-order effects (further deletes in the dep-tree) are avoided.

Now that we have explicit Issue-deletion (deps-first, delayed, properly batched)
the SET_NULL behavior is always a no-op (but with cost in queries).

As a result, in the test for issue deletion (which has deletes for many
of the altered models), the following 8 queries are no longer done:

```
SELECT "issues_grouping"."id", [..many fields..] FROM "issues_grouping" WHERE "issues_grouping"."id" IN (1)
UPDATE "events_event" SET "grouping_id" = NULL WHERE "events_event"."grouping_id" IN (1)

[.. a few moments later..]

SELECT "issues_issue"."id", [..many fields..] FROM "issues_issue" WHERE "issues_issue"."id" = 'uuid'
UPDATE "issues_grouping" SET "issue_id" = NULL WHERE "issues_grouping"."issue_id" IN ('uuid')
UPDATE "issues_turningpoint" SET "issue_id" = NULL WHERE "issues_turningpoint"."issue_id" IN ('uuid')
UPDATE "events_event" SET "issue_id" = NULL WHERE "events_event"."issue_id" IN ('uuid')
UPDATE "tags_eventtag" SET "issue_id" = NULL WHERE "tags_eventtag"."issue_id" IN ('uuid')
UPDATE "tags_issuetag" SET "issue_id" = NULL WHERE "tags_issuetag"."issue_id" IN ('uuid')
```

(breaks the tests b/c of constraints and not always using factories; will fix next)
2025-07-03 11:33:58 +02:00
Klaas van Schelven
e58be0018f Tag models: no CASCADE
CASCADE was defined for keys & values, but in practice those are never directly
deleted except in the very case in which it has been established that they are
'orphaned', i.e. no longer being referrred to. That's exactly the case in which
CASCADE is superfluous.

As a result, in the test for issue deletion (which contains a prune of
tagvalue), the following 3 queries are no longer done:

```
SELECT "tags_tagvalue"."id", "tags_tagvalue"."project_id", "tags_tagvalue"."key_id", "tags_tagvalue"."value" FROM "tags_tagvalue" WHERE "tags_tagvalue"."id" IN (1)
DELETE FROM "tags_eventtag" WHERE "tags_eventtag"."value_id" IN (1)
DELETE FROM "tags_issuetag" WHERE "tags_issuetag"."value_id" IN (1)
```
2025-07-03 11:33:58 +02:00
Klaas van Schelven
428011ff9b Prune orphans (TagValue, via IssueTag) inline
See #135
2025-07-03 11:33:53 +02:00
Klaas van Schelven
a68a30bcad Diagram in comment for delete_deps 2025-07-03 08:28:29 +02:00
Klaas van Schelven
29238ece43 Test for model-topo overrides 2025-07-02 23:48:10 +02:00
Klaas van Schelven
f1878da154 Fix the test
tests were green but IssueTag was missing from the assertions (and some others doubled)
2025-07-02 23:35:23 +02:00
Klaas van Schelven
ebd9cceb6d Issue cleanup: hardcode order of visited models 2025-07-02 23:32:31 +02:00
Klaas van Schelven
ee9add5e5f Vacuum Tags command
See #135
2025-07-02 21:43:51 +02:00
Klaas van Schelven
aed19e70d3 Issue deletion test: actually test the Issue itself 2025-07-02 16:41:42 +02:00
Klaas van Schelven
20b14ed69c Document index-usage 2025-07-02 16:30:59 +02:00
Klaas van Schelven
1601aa0b1f Remove 'order_by' from delete_with_deps logic
I _think_ I added this in the first version for [1] some idea around
determinism and [2] with the idea that by "grouping" around fk_name,
you'd get deletions close-to-root sooner, but I don't think either of
those are valuable goals (in both cases: as long as it's gone eventually).
The drawback of ordering is that it forces further thinking about indexes
(esp. when ordering on 2 items) so I'd rather avoid that.
2025-07-02 16:21:56 +02:00
Klaas van Schelven
cb765dae30 simplify query
a previous version (prob. not committed) of this function
used both values, but the current version only uses 'pk'
so that's all we fetch
2025-07-02 16:19:08 +02:00
Klaas van Schelven
f8345eb919 Rename & inline for clarity 2025-07-02 16:15:50 +02:00
Klaas van Schelven
7996ed773e README: condense verbose wordings 2025-07-01 13:51:58 +02:00
Klaas van Schelven
0e3bbd9ab5 1.6.3 CHANGELOG 1.6.3 2025-06-27 13:01:46 +02:00
Klaas van Schelven
6e9defedb4 Simple IntegrationTest for issue deletion 2025-06-27 12:57:51 +02:00
Klaas van Schelven
38397bf2f2 Remove superfluous comment 2025-06-27 12:57:24 +02:00
Klaas van Schelven
e5dbeae514 Issue.delete_deferred(): first version (WIP)
Implemented using a batch-wise dependency-scanner in delayed
(snappea) style.

* no tests yet.
* no real point-of-entry in the (regular, non-admin) UI yet.
* no hiding of Issues which are delete-in-progress from the UI
* file storage not yet cleaned up
* project issue counts not yet updated
* dangling tag values: no cleanup mechanism yet.

See #50
2025-06-27 12:52:59 +02:00
Klaas van Schelven
b5ffa154c1 Support the combination (delay_on_commit, immediate_atomic, TASK_ALWAYS_EAGER)
As per the comment.

Additionally, the original of _start_transaction_under_autocommit
is stored on the contextprocessor now (which is the place with the
best match of the actual call-stack anyway). (Was needed for nested
invocations)
2025-06-27 11:09:58 +02:00
Klaas van Schelven
4ca15c7159 fix make_consistent on mysql
Problem: on mysql `make_consistent` cannot always clean up `Event`s, because
`EventTag` objects still point to them, leading to an integrityerror.
The problem does not happen for `sqlite`, because sqlite does FK-checks
on-commit. And the offending `EventTag` objects are "eventually cleaned up" (in
the same transaction, in make_consistent)

This is the "mostly works" solution, for the scenario we've encountered.
Namely: remove EventTags which have no issue before removing Events. This works
in practice because of the way Events-to-cleanup were created in the UI in
practice, namely by removal of some Issue in the admin, triggering a `SET_NULL`
on the `issue_id`. Removal of issue implies an analagous `SET_NULL` on the
`EventTag`'s `issue_id`, and by removing those `EventTag`s before proceeding
with the `Event`s, you avoid the FK constraint triggering.

We don't want to fully reimplement `CASCADE` (as in Django) here, and the
values of `on_delete` are "Design Decision Needed" and non-homogonous anyway,
and we might soon implement proper deletions (see #50) anyway, so the "mostly
works" solution will have to do for now.

Fixes #132
2025-06-26 10:56:31 +02:00
Klaas van Schelven
70e0f147b5 Tags in event_data can be lists; deal with that
Fix #130
2025-06-26 09:06:21 +02:00
Klaas van Schelven
6c8624f683 Development settings: email using os.getenv 2025-06-24 22:16:20 +02:00
Klaas van Schelven
072ed70681 1.6.2 CHANGELOG 1.6.2 2025-06-19 20:44:27 +02:00
Klaas van Schelven
9135e88495 Wrap one more BASE_URL usages with str()
In 59372aba33 a lazily evaluated BASE_URL tool was introduced.
I found 1 more case in which BASE_URL was not "collapsed into a
string" magically by `__add__`, causing an
`AttributeError: 'TenantBaseURL' object has no attribute 'decode'`
2025-06-19 09:44:55 +02:00
Fabien LEFEBVRE (d1ceward)
9cec248ad8 Add dark theme 2025-06-16 15:37:37 +02:00
Klaas van Schelven
8ad9fe33ca Too many quotes in local-vars display
Fix #119
2025-06-11 16:57:43 +02:00
Klaas van Schelven
f84af74a7c Remove hard-coded webhook_url from test_message
as pointed out by Slack's own bots
1.6.1
2025-06-11 08:42:06 +02:00
Klaas van Schelven
b53b1c6ce2 1.6.0 CHANGELOG 1.6.0 2025-06-10 22:32:12 +02:00
Klaas van Schelven
fac5b19966 Slack Alerts
Fix #3
2025-06-10 22:00:37 +02:00
Klaas van Schelven
33d5865579 Clarifying comments on logmessage display 2025-06-03 15:47:46 +02:00
Klaas van Schelven
5d316332d0 Display formatted log message when available
Fix #111
2025-06-03 15:44:36 +02:00
Klaas van Schelven
6822ec0194 Add 2 env variables to compose-sample.yaml
See #110
2025-06-03 15:31:02 +02:00
Klaas van Schelven
462b4f131a Merge pull request #108 from bugsink/hijacked-pr-84
feat: add delete functionality for users
2025-05-29 17:21:53 +02:00
Animesh Agrawal
febcabc9f3 feat: add delete functionality for users
Implement delete functionality with confirmation modals for users. Ensure
proper authorization checks are in place before deletion. Add corresponding
JavaScript files to handle modal interactions and form submissions.

Based on #84

Signed-off-by: Animesh Agrawal <animesh@flick2know.com>
2025-05-29 17:14:54 +02:00
Klaas van Schelven
da77957870 in development: env-level configuration of the NPM_BIN_PATH
As asked by @anime-shed on #78
2025-05-29 16:51:45 +02:00
Klaas van Schelven
4c104714e3 Fix the tests
update key phrases according to the recent improvements;
specifically: test also for the fact that function-name now shows up

See #87
2025-05-29 11:11:45 +02:00
Klaas van Schelven
ef5c203d78 Merge branch 'sourcemaps-multi-source-test' 2025-05-29 11:01:37 +02:00
Klaas van Schelven
1a60cbcd3c Add test for multi-file sourcemaps
See #87 for the code that fixes this
2025-05-29 11:01:10 +02:00