[UPSTREAM CHANGES] latest changes as of Mon May 18 2026 01:10:10 GMT+0000 (Coordinated Universal Time) - #229
Open
github-actions[bot] wants to merge 5505 commits into
Open
[UPSTREAM CHANGES] latest changes as of
Mon May 18 2026 01:10:10 GMT+0000 (Coordinated Universal Time)#229github-actions[bot] wants to merge 5505 commits into
github-actions[bot] wants to merge 5505 commits into
Conversation
Oversized payloads on the /store/ endpoint raised RequestDataTooBig during ninja's parameter parsing, which bubbled up as an unhandled error event. The middleware already logs a warning with diagnostic details. Adding a ninja exception handler returns a clean 413 response without generating noise in error tracking. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
chore(deps): update dependency boto3 to v1.42.77 See merge request glitchtip/glitchtip-backend!2247
chore(deps): update dependency ruff to v0.15.8 See merge request glitchtip/glitchtip-backend!2248
Concurrent INSERT ... ON CONFLICT DO UPDATE statements on uptime_uptimecheckhourlystatistic could deadlock when acquiring row locks in different orders. Sorting the batch by organization_id ensures consistent lock ordering. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ubscriptions from stripe sync or webhook.
DuckDB defaults to one thread per HOST CPU core. In Kubernetes, it reads the node's cores (e.g. 64) rather than the pod's CPU limit (e.g. 2). Each thread allocates scan buffers outside the memory_limit setting, causing VmPeak to hit 8+ GB and triggering OOM pod recycling. Changes: - Set threads on ALL connections (was only set on write connections). Default: 2, configurable via DUCKDB_THREADS env var. - Lower default memory_limit cap from 1024 MB to 256 MB. The memory_limit only bounds DuckDB's internal buffer pool — thread stacks, mmap'd regions, and jemalloc overhead are additional. A 256 MB pool with 2 threads typically peaks at ~400-500 MB total process impact. - Both settings are configurable for dedicated analytics workloads. Benchmark proof (8-core machine, 500k rows across 10 Parquet files): BEFORE (8 threads, 1024MB): VmPeak 985 MB, RSS +147 MB AFTER (2 threads, 256MB): VmPeak 320 MB, RSS +45 MB Query time impact: +4% (negligible) On a 64-core Kubernetes node the VmPeak reduction would be proportionally larger — consistent with the 8.1 GB VmPeak observed in production. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DUCKDB_THREADS now reads cgroup v2 cpu.max (Kubernetes/Docker) to use the pod's actual CPU limit instead of a hardcoded default. Falls back to min(os.cpu_count(), 4) on bare-metal. Also refactors the cgroup reading into shared helpers: - _get_cgroup_memory_bytes(): reads /sys/fs/cgroup/memory.max - _get_cgroup_cpu_count(): reads /sys/fs/cgroup/cpu.max Both settings auto-detect but accept env var overrides: - DUCKDB_MEMORY_LIMIT: auto from cgroup memory (25%, cap 256MB) - DUCKDB_THREADS: auto from cgroup CPU quota (or cap at 4) - DUCKDB_TEMP_DIRECTORY: defaults to /tmp Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add billing cycle dates to created subscriptions See merge request glitchtip/glitchtip-backend!2253
fix: cap DuckDB threads and memory to prevent VmPeak explosion See merge request glitchtip/glitchtip-backend!2254
fix: sort uptime stats upsert by org_id to prevent deadlock See merge request glitchtip/glitchtip-backend!2251
fix: handle RequestDataTooBig with 413 instead of unhandled exception See merge request glitchtip/glitchtip-backend!2250
chore(deps): update dependency google-cloud-logging to v3.15.0 See merge request glitchtip/glitchtip-backend!2249
chore(deps): update dependency model-bakery to v1.23.4 See merge request glitchtip/glitchtip-backend!2256
chore(deps): update dependency boto3 to v1.42.78 See merge request glitchtip/glitchtip-backend!2255
chore(deps): update dependency aiohttp to v3.13.4 See merge request glitchtip/glitchtip-backend!2258
chore(deps): update dependency boto3 to v1.42.79 See merge request glitchtip/glitchtip-backend!2259
Left over after switching ingest test invocations to ``run_async_closing``. Disclosure: this MR was prepared with assistance from Claude. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The lock file kept the workspace member entry after pyproject.toml lost it; CI failed with "Distribution not found at .../gt_rust". Regenerated with ``uv lock``. Disclosure: this MR was prepared with assistance from Claude. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The ingest hot path was a mix of native async cursors (raw SQL via async-backend) and Django ORM convenience methods (abulk_create, aupdate, aexists, async-iter on a sync QuerySet) that thread-pool through the sync connection. The latter group made the hot path double-counted across two pools and opaque to async-aware test assertions. Convert every remaining sync_to_async-wrapped DB call in process_issue_events / process_transaction_events to either raw async SQL via async_db.fetchall/execute helpers or async-backend's AsyncQuerySet (debug bundles, where ORM model construction with select_related is the right tool). Issue + IssueHash creation now runs inside async_atomic with raw INSERTs; the IntegrityError race-recovery path reads issue_id back via a SELECT. Replace assertNumQueries on the affected hot-path tests with a new AsyncQueryCounter (glitchtip/test_utils/async_query_counter.py) that patches AsyncCursorWrapper at the class level, so it observes queries across whatever task the sync test client dispatches into. Sync queries from setUp fixtures are now ignored by design. test_store_api.setUp gains cache.clear() so TransactionTestCase's RESTART IDENTITY truncation can't race the auth/throttle cache between tests in the class.
Project baseline is Python 3.12+; PEP 604 / PEP 585 are native and PEP 563 deferred evaluation isn't relied on. Use a forward-reference string for the self-referential return annotation instead.
Vulture flagged exc_type/exc_value/traceback as unused. Replace with *_exc_info — the leading underscore tells vulture the variable is intentionally unused. Also expand the module docstring to spell out why AsyncQueryCounter exists and what would have to change before AsyncCaptureQueriesContext from django-async-backend can replace it (every middleware in MIDDLEWARE has to be async_capable, and a recalibration pass to absorb the BEGIN/COMMIT delta in queries_log). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a prominent docstring banner and an import-time DeprecationWarning to glitchtip/wsgi.py. GlitchTip is async-first now: WSGI bridges every async view (including the entire ingest hot path) through async_to_sync, leaks a task-local async-backend DB connection per request, and bypasses the glitchtip.ingest_asgi fast-path dispatcher entirely. Custom deployments that bypass our scripts and import glitchtip.wsgi directly will now see the warning in their logs and have a clear pointer to glitchtip.asgi:application under Granian. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat(ingest): native async DB cursors via django-async-backend See merge request glitchtip/glitchtip-backend!2343
The release Docker build previously ran on a single amd64 runner and
emulated arm64 layers via QEMU (`multiarch/qemu-user-static`). Emulated
ARM builds are 5-10x slower than native, dominating the wall time of
every release tag.
GitLab now exposes native aarch64 SaaS runners (tag
`saas-linux-medium-arm64`), so split the multi-arch build into two
parallel native jobs (one per arch) that push arch-suffixed tags, plus a
small `release_manifest_merge` job that uses
`docker buildx imagetools create` to publish the user-facing tags
(`latest`, `X.Y.Z`, `X.Y`, `X`) as a manifest list referencing both
arches. A `release_prepare_assets` job downloads the frontend
`dist/` artifact once and shares it with both arch builds.
Also add a `validate_build_{amd64,arm64}` matrix that runs on MRs into
the default branch and on master pushes. It builds the production
Dockerfile natively for each arch (no push) and runs a cheap
`python -c "import django, glitchtip"` smoke test against the resulting
image. This catches "build succeeds but image won't run on this arch"
regressions before they reach a release tag, where the QEMU build was
the first time we discovered them.
Triggers for the release jobs are unchanged: protected vX.Y.Z tags
only, never on schedules or MRs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop verbose intros and the QEMU/legacy-job references; keep only the non-obvious "why" (smoke test purpose, no-DinD note for imagetools). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ci: native multi-arch builds + per-MR build validation See merge request glitchtip/glitchtip-backend!2349
Bulk INSERT/UPDATE in event ingest, log ingest, and the transaction pipeline previously expanded one ``(%s, %s, ...)`` tuple per row into a ``VALUES`` list via ``execute_mogrified_values``. That has three costs that grow with batch size: a per-row mogrify round-trip in Python, a distinct statement shape per batch length (poor plan-cache reuse), and the 65535 bind-parameter ceiling Postgres enforces, which clamps batches on wide schemas (the 13-column ``issue_events_issueevent`` insert tops out around 5000 rows). Pass per-column arrays to ``unnest(%s::T[], %s::T[], ...)`` instead. One parameter per column regardless of batch size; one statement shape; no bind-param ceiling. Synthetic benchmark (Postgres 18, unlogged tables, 30 rounds, local docker): issue_events_issueevent 13 cols 500 rows: 26% faster (45ms -> 33ms median) issue_events_issuetag 6 cols 2000 rows: 52% faster (70ms -> 34ms median) Converted call sites: - apps/event_ingest/process_event.py: issueevent INSERT, issuetag INSERT, update_issues UPDATE, update_statistics, update_org_statistics, transactiongroup INSERT, spanstaging INSERT, _update_transaction_group_stats Phase 2, and the two IN(VALUES) lookups (_fetch_issue_hashes_raw, _fetch_transaction_groups), now rewritten as JOIN unnest(). - apps/logs/process_logs.py: logevent INSERT, log_statistics upsert, resource_lookup upsert. Left as VALUES (not worth converting): the small release/environment fanout (tiny batches), and ``_update_transaction_group_stats`` Phase 1 which carries an int[] histogram per row that doesn't trivially unnest. Adds two thin helpers in apps/shared/async_db.py — ``execute_unnest`` and ``fetchall_unnest`` — that transpose row-major data to per-column lists before executing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
perf(ingest): switch hot-path bulk writes from VALUES to UNNEST See merge request glitchtip/glitchtip-backend!2350
Same conversion as the prior commit on this branch but for the uptime hot path: the upsert previously mogrified one ``(%s,%s,%s)`` tuple per organization in a Python loop, joined them into a ``VALUES`` list, and ran the whole thing through ``sync_to_async``. Switch to async-native ``execute_unnest`` with ``async_atomic`` so we drop both the per-row Python work and the sync_to_async thread hop on every check flush (every 1 s or 100 checks, whichever comes first). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
perf(uptime): switch update_uptime_statistics from VALUES to UNNEST See merge request glitchtip/glitchtip-backend!2351
The Stripe webhook handler was hitting unrecoverable failures on transient 429 lock_timeout responses from the Stripe API, which Stripe itself flags as retryable via the Stripe-Should-Retry header. Two related issues: 1. stripe_get/stripe_post had no retry logic — any non-200 response immediately raised. Now they share a retry helper that backs off exponentially on 429/5xx, honoring the Stripe-Should-Retry header when present. 2. The webhook view marked event IDs as processed via cache.aadd before actually processing them. If processing raised, Stripe's webhook retry would arrive, hit the dedup cache, and be silently dropped. The mark is now released on failure so retries can reprocess. 🤖 Generated with Claude Code
fix(stripe): retry transient API errors and release dedup key on failure See merge request glitchtip/glitchtip-backend!2353
`unix_to_datetime` used `datetime.fromtimestamp(ts)` (no `tz` arg), which returns a naive datetime in the process's local time, then attached `settings.TIME_ZONE` via `make_aware`. This only produced correct UTC values because the standard container TZ is UTC — naive UTC tagged as UTC happens to round-trip correctly. If a deployment sets `TZ` to a non-UTC zone (e.g. for log readability), `fromtimestamp` would return a naive local datetime and `make_aware` would mislabel it as UTC, silently shifting every Stripe timestamp into the database by the local offset. Pass `tz=timezone.utc` explicitly so the conversion no longer depends on process-local time. No behavior change on UTC hosts. Regression test forces `TZ=America/New_York` and `TZ=Asia/Tokyo` via `time.tzset()` to confirm the result is the same in any local timezone. 🤖 Generated with Claude Code
fix(stripe): redirect cancel_url to subscription page instead of home See merge request glitchtip/glitchtip-backend!2355
fix(stripe): make unix_to_datetime timezone-independent See merge request glitchtip/glitchtip-backend!2354
The HTTP path returns django-async-backend's per-asyncio-Task connection wrapper to the pool via the close_async_connections middleware. The task worker has no equivalent. When an async task uses async_connections (e.g. an ingest task reading the read replica), the wrapper's close() is never called -- the asyncio.Task ends, the wrapper is garbage collected, and psycopg.AsyncConnection.__del__ runs synchronously, which can't perform an async close handshake cleanly. Wire receivers on django_vtasks.signals.task_finished and task_failure that call async_connections.close_all() inside the same asyncio.Task that ran the user task. Mirrors the HTTP middleware semantics, including the asyncio.shield to keep cleanup running through cancellation. vtasks 2.1.1 emits exactly one of task_finished/task_failure per task (success, exception, or cancellation), so the two receivers cover every completion path. Bump the dependency floor to require it. Disclosure: prepared with assistance from Claude. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Django signal receivers receive sender/signal/extra kwargs they don't necessarily use. Rename **kwargs to **_kwargs so vulture's 100% confidence sweep skips it.
fix: close async DB connections after each background task See merge request glitchtip/glitchtip-backend!2356
django.contrib.postgres wires register_type_handlers to the connection_created signal so psycopg can auto-decode hstore columns. That handler runs `SELECT ... FROM pg_type WHERE typname = 'hstore'` on every new connection. GlitchTip uses jsonb instead of hstore, so the extra round-trip is wasted work — and the receiver uses the sync ORM, which raises SynchronousOnlyOperation when fired from inside an asyncio task. The result is that async DB connections in worker tasks raise on their first open after !2356 returned the connection-wrapper lifecycle to a healthy state. django.contrib.postgres has to stay in INSTALLED_APPS because Django's postgres.E005 check requires it for ArrayField and SearchVectorField, both of which we use. The fix is to disconnect the single offending receiver after PostgresConfig.ready() runs. Also swap the e2e test from sentry-cli to glitchtip-cli. sentry-cli is no longer open source and pulling from sentry.io is incompatible with the clean-room rules in CLAUDE.md. glitchtip-cli is a drop-in for the send-event command we use, accepts SENTRY_DSN for compatibility, and prints the event id as the last token on stdout, so the existing parser keeps working. 🤖 Generated with Claude Opus 4.7. AI-assisted; please review carefully. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fix: skip hstore OID lookup on every connection_created See merge request glitchtip/glitchtip-backend!2357
fix: make_sample_* failing because partitions in the past didn't exist See merge request glitchtip/glitchtip-backend!2358
… into upstream-changes-2026-05-18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is auto-generated by
actions/github-script.