Commit Graph

1352 Commits

Author SHA1 Message Date
Klaas van Schelven 9b0f0e04f4 CREATE_SUPERUSER shortcut: robust for ':' in password 2025-05-07 09:21:21 +02:00
Klaas van Schelven 0dfd01db9b Performance: don't evaluate all project issues to show/hide checkbox
Doing so is expensive (to the point of timeout) for largish numbers
of issues.
2025-05-06 23:01:09 +02:00
Klaas van Schelven aad0f624f9 Fix: issue-list indexes must have project first
because we always filter by project before ordering;

the now-removed first_seen index was simply unused
2025-05-06 22:19:31 +02:00
Klaas van Schelven 699f6e587d fix: _get_runtime_limit must apply to the right alias
instead of just returning the value for the default alias

that this wasn't done was a bug generally (the per-alias limits
simply didn't apply) but was exposed when running the CI against
non-sqlite databases, because in that setup the main database
would not get a limit set, but because the getter fell back
to the default (which wasn't set) there was an exception.
2025-05-06 21:45:41 +02:00
Klaas van Schelven 474c677f12 limit_runtime: log non-interruptions 2025-05-06 16:57:06 +02:00
Klaas van Schelven b33d559d44 1.5.2 CHANGELOG 1.5.2 2025-05-06 12:55:06 +02:00
Klaas van Schelven 49e6700d4a Grouping.grouping_key: hash it for the index 2025-05-06 11:32:19 +02:00
Klaas van Schelven d5a449020d Stress test: more fat-tailed randomness
direct cause: the ability to create test-data for many-Issue setups
2025-05-06 10:42:18 +02:00
Klaas van Schelven cc2b22f08e Digest: check Grouping.exists only once (save a query) 2025-05-06 10:30:51 +02:00
Klaas van Schelven 17fb9cc850 Remove open_issue_count from homepage; it's too expensive 2025-05-06 10:27:16 +02:00
Klaas van Schelven 3783661054 Issue Paginator: don't attempt to count the Issues
Counting incurs looking at all records which is too expensive if you have e.g.
1_000_000 issues.

Note that we take a different approach than the one for Events (where we
count-with-timeout). Reason for switching:
https://sqlite.org/forum/forumpost/fa65709226

For Events we have a known count for the non-query case (denormalized/counted
value), so we preserve what we had there. For Issues the trouble of keeping
counts right for muted/etc. is not (currently) worth it.
2025-05-06 10:13:06 +02:00
Klaas van Schelven 392f5a30be Add index for Grouping.grouping_key (and project) 2025-05-05 22:45:33 +02:00
Klaas van Schelven c292d9f87c Merge pull request #88 from bugsink/dependabot/pip/python-packages-e08a0a53e9
Update sentry-sdk requirement from ==2.26.* to ==2.27.* in the python-packages group
2025-04-26 10:56:57 +02:00
Klaas van Schelven a5401414db Fix deprecationwarnings (about Django 5.0) in the tests
by changing an import
2025-04-26 10:52:04 +02:00
Klaas van Schelven c51051dc2e format_exception in capture_or_log_exception: python 3.9 compatible 2025-04-26 10:47:16 +02:00
Klaas van Schelven 4b396446b0 Fix github workflows to include sentry_sdk_extensions in tests
it's the other non-Django-app with tests that we have now
2025-04-26 10:43:35 +02:00
Klaas van Schelven ebd7d8075b Add test for test_capture_or_log_exception
this test is assumed to surface problems on Python<3.10
2025-04-26 10:24:49 +02:00
dependabot[bot] a0d3da6034 Update sentry-sdk requirement in the python-packages group
Updates the requirements on [sentry-sdk](https://github.com/getsentry/sentry-python) to permit the latest version.

Updates `sentry-sdk` to 2.27.0
- [Release notes](https://github.com/getsentry/sentry-python/releases)
- [Changelog](https://github.com/getsentry/sentry-python/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-python/compare/2.26.0...2.27.0)

---
updated-dependencies:
- dependency-name: sentry-sdk
  dependency-version: 2.27.0
  dependency-type: direct:production
  dependency-group: python-packages
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-25 15:24:45 +00:00
Klaas van Schelven f41f8262c1 EMAIL_USE_SSL: not EMAIL_USE_TLS by default
avoids crashing on "both true" when only `EMAIL_USE_TLS` is explicitly configured

> In your setup those are both true because the former is true by default,
> and the latter is what you have specified. Set the first to false and you
> should be good.

See #86
1.5.1
2025-04-24 21:18:53 +02:00
Klaas van Schelven 7c3c19b6c8 EMAL_USE_SSL should be default False in the Docker config
this is what makes sense given that:

* it's the lesser used of the to EMAIL_USE_xxx variables
* the two settings are mutually exclusive

This was simply wrong-when-introduced in 921d5bd4a3

See #86 (recent discussion of someone running into a related problem).
2025-04-24 21:11:05 +02:00
Klaas van Schelven cddd4f2c02 Fix Header/Grouper for Log Messages using deprecated SDKs
Fix #85
2025-04-24 20:59:54 +02:00
Klaas van Schelven 82bae6f24c Note on limit_runtime in the context of COUNT queries 2025-04-24 20:21:29 +02:00
Klaas van Schelven 934fd4a93a 1.5.1 CHANGELOG 2025-04-24 09:09:16 +02:00
Klaas van Schelven 53d4be8183 Fix 'different_runtime_limit' race conditions
This commit fixes 3 related issues with the way runtime_limit was administered;
which could lead to race conditions (and hence: the wrong runtime_limit
applying at some point in time). Post-fix, the folllowing holds:

1. We use thread_locals to store this info, since there are at least 2 sources of
    threaded code that touch this (snappea's workers and the django debugserver)

2. We distinguish between the "from connection settings" timeout and the
    "temporarily overridden" ones, since we cannot assume
    connection-initialization happens first (as per the comment in base.py)

3. We store runtime-limits per alias ('using'). Needed for [2] (each connection
    may have a different moment-of-initialization, clobbering CM-set values from
    the other connection) and also needed once you realize there may be
    different defaults for the timeouts.

General context: I've recently started introducing the 'different runtime'
helper quite a bit more; and across connections (snappea!), which created more
and more doubts as to it actually working as advertised.

Thoughts on "using" being required. I used to think "you can reason about a
global timeout value, and the current transaction makes clear what you're
actually doing", but as per the notes above that doesn't really work.

Thoughts on reproducing:
A few thoughts/notes on reproducing problems with race conditions. Basic note:
that's always hairy. So in the end I settled on a solution that's hopefully
easy to reason about, even if it's verbose.

When I started work on this commit, I focussed on thread-safety; "proving the
problem" consisted of F5/^R on a web page with 2 context managers with different
timeouts, hoping to show that the stack unrolling didn't work properly.
However, during those "tests" I noticed quite a few resets-to-5s (from the
connection defaults), which prompted fix [2] from above.
2025-04-22 22:08:53 +02:00
Klaas van Schelven 5c0e45a16d Explain tailwind usage during development & vendoring step 2025-04-19 22:11:30 +02:00
Klaas van Schelven 9b6fbe523f Snappea foreman: on catastrophic errors, wait for workers 2025-04-18 14:57:08 +02:00
Klaas van Schelven 366d22f295 Snappea stats: fix for when no tasks remain 2025-04-18 14:37:44 +02:00
Klaas van Schelven 420f20df24 Transaction-level logging: clarify 'using' 2025-04-18 14:36:57 +02:00
Klaas van Schelven 70493206fd Better 'yellow bar' for snappea warnings (using Stat info) 2025-04-18 10:11:11 +02:00
Klaas van Schelven 20f339965c Remove 'real time' from the tagline in the README 2025-04-17 22:03:28 +02:00
Klaas van Schelven 5895253803 Count view: async slow counts
when you count, it's usually because there are many, so this
extra complication is probaly going to be required
2025-04-17 22:03:23 +02:00
Klaas van Schelven 90b80cdc32 Parentheses fix
botched in 6ff303858b
2025-04-17 12:48:11 +02:00
Klaas van Schelven d3de2e6d5b Add '--window' param to showstat 2025-04-17 12:20:11 +02:00
Klaas van Schelven 21dfa6dbbc Show stat command: add column for 'write saturation 2025-04-17 12:18:09 +02:00
Klaas van Schelven 2a871b06d4 conf_templates: STATS_RETENTION_MINUTES configurable 2025-04-17 11:00:43 +02:00
Klaas van Schelven 731d446a3d snappea task warning should itself never slow down the site
(max at 0.1s)
2025-04-17 11:00:43 +02:00
Klaas van Schelven 5379f9a168 Simple page for object counts (superuser) 2025-04-17 11:00:42 +02:00
Klaas van Schelven 7616b0ea77 Document timing of task.create/delete in code 2025-04-17 10:16:43 +02:00
Klaas van Schelven 89927c7ab4 Snappea stats: never bring down snappea 2025-04-17 10:13:19 +02:00
Klaas van Schelven 6500548168 Snappea Stats: document the need for separate table 2025-04-17 09:41:03 +02:00
Klaas van Schelven abd05b7269 Snappea stats: silently ignore backwards clock drift 2025-04-17 09:38:42 +02:00
Klaas van Schelven b3c99fda72 Add 'munin' command
(initial target market: me)
2025-04-17 09:01:42 +02:00
Klaas van Schelven fb45a8ccc9 Show stats command for snappea 2025-04-16 21:48:41 +02:00
Klaas van Schelven 4cedffc1b7 Snappea stats: configurable retention 2025-04-16 17:10:15 +02:00
Klaas van Schelven 06342010dd Turn off KEEP_ENVELOPES in development
as per the documented caveats, I'd rather not do this when not
necessary (when not actively looking for malformed envelopes)
2025-04-16 16:59:15 +02:00
Klaas van Schelven e27439ab7b snappea stats: log cost of stats themselves 2025-04-16 16:57:53 +02:00
Klaas van Schelven 94338051ef Snappea Stats: first version 2025-04-16 16:40:28 +02:00
Klaas van Schelven 67f769d3e5 immediate_semaphore: implement using
Given what immediate_semaphore _is_, the principled need for this is obvious.
In practice, this wasn't needed yet (immediate_atomic is not used for
any non-default DB). But better fix it before we do decide on that.
2025-04-16 13:55:22 +02:00
Klaas van Schelven f8db5edf82 immediate atomic 'get-write-lock' performance logging: fix it
it was reporting the actual sqlite get-write-lock timings, but since the
introduction of the immediate_semaphore those were 'basically never' blocking
since all the blocking was done in the semaphore (ignoring cross
gunicorn/snappea details)
2025-04-16 12:53:42 +02:00
Klaas van Schelven 8c87d0fe13 When is the email_system_warning shown? change & document
as per the comment
2025-04-16 09:55:41 +02:00