[UPSTREAM CHANGES] latest changes as of Mon Jul 06 2026 00:59:06 GMT+0000 (Coordinated Universal Time) - #236
Open
github-actions[bot] wants to merge 5710 commits into
Open
[UPSTREAM CHANGES] latest changes as of
Mon Jul 06 2026 00:59:06 GMT+0000 (Coordinated Universal Time)#236github-actions[bot] wants to merge 5710 commits into
github-actions[bot] wants to merge 5710 commits into
Conversation
…seline Python 3.12 is the project's minimum (`requires-python = ">=3.12"`), so PEP 604 (`X | Y`) and PEP 585 (`list[X]`) syntax is native and the `from __future__ import annotations` import added in `glitchtip/oidc_discovery.py` was a no-op. Note this in AGENTS.md so future AI-drafted contributions don't reintroduce it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GlitchTip is async-first; the `a` prefix is only meaningful in libraries that expose parallel sync and async APIs in the same namespace (Django ORM, allauth, etc.). Rename `aget_openid_config` and `aget_authorize_url` to drop the prefix and document the convention in AGENTS.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
expose is_public on StripePrice for canonical-price selection See merge request glitchtip/glitchtip-backend!2342
feat: filter Stripe products by hosted product_type metadata See merge request glitchtip/glitchtip-backend!2307
perf(api): cache OIDC discovery doc instead of fetching per request See merge request glitchtip/glitchtip-backend!2344
Refactor JavaScript sourcemap processing to remap stacktraces per exception instead of flattening all frames first. This keeps `raw_stacktrace` unset when sourcemap resolution is a no-op for an exception, while still preserving the original stacktrace before the first successful frame remap.
fix: persist raw JS stacktrace before sourcemap remap See merge request glitchtip/glitchtip-backend!2340
The hosted product_type filter merged in !2307 was added at the same time !2306 introduced test_sync_product_round_trips_price_is_public. The filter MR updated test_sync_product to set product_type=hosted in metadata, but the round-trip test (developed in parallel) was missed. After both merged the new test's products were filtered out, no StripePrice rows were created, and aget(stripe_id="price_pub") raised DoesNotExist on master CI. Generated with assistance from Claude Code (Opus 4.7).
fix(stripe): add product_type=hosted to round-trip test products See merge request glitchtip/glitchtip-backend!2345
add test and test data for ios event context;
Fix ios event context v2 See merge request glitchtip/glitchtip-backend!2346
GlitchTip's hot ingest paths (event store, log ingest, transaction ingest, auth) now use ``async_connections`` directly instead of wrapping sync cursors in ``sync_to_async``. The default ``ENGINE`` swaps to ``django_async_backend.db.backends.postgresql`` so async cursors are always available; psycopg3 still does the actual I/O. Why --- Threadpool-wrapped sync queries serialise on the GIL during the await chain. With granian + asyncio, native async cursors release the GIL during PG round-trips, letting concurrent ingest waves overlap I/O. What ---- - ``apps.shared.async_db`` exposes ``fetchall`` / ``execute`` / ``fetchall_mogrified_values`` / ``execute_mogrified_values`` on top of ``async_connections``. - ``get_project_auth_info_row`` is async-only; auth path uses it directly. - Default DB ENGINE: ``django_async_backend.db.backends.postgresql``. ``close_async_connections`` middleware is always in the chain so async cursors return to the pool per request. - ``apps.event_ingest.process_event`` and ``apps.logs.process_logs`` use the async helpers throughout the hot path. - Synthetic ``/api/_probe/async/`` endpoint for concurrency benchmarks. - Concurrency benchmark scaffolding in ``benchmarks/``. Tests ----- ``async_connections`` opens its own PG sessions, so ``TestCase``-style rollback can't roll back uncommitted fixture data — the async session doesn't see it. Hot-path test classes (event_ingest, log ingest, transaction ingest, sourcecode workflow, alert regression) now subclass ``TransactionTestCase`` (via a new ``GlitchTipTransactionTestCase``). Fixtures get committed and truncated between tests instead of rolled back, which the async session sees correctly. The remaining ~870 tests stay on fast ``TestCase``. Other test-runner adjustments: - ``TimedTestRunner.teardown_databases`` terminates lingering async-pool sessions before ``DROP DATABASE`` so test teardown doesn't fail on task-local connections. - ``compose.yml`` raises Postgres ``max_connections`` to 300 to absorb the connection accumulation across ``TransactionTestCase`` classes before runner teardown. - ``EventIngestTestCase.tearDown`` closes async connections per test. 1024 tests pass, 0 fail. Disclosure: this MR was prepared with assistance from Claude. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
``psycopg.AsyncConnection.close()`` is async, so connections opened in a task that has gone out of scope keep their socket open until process exit. Over a 1024-test suite that piles up enough leaked sessions to trip Postgres' default ``max_connections``. Two-part fix: 1. ``run_async_closing`` helper (apps/event_ingest/tests/utils.py) wraps ``async_to_sync`` for direct ingest invocations, closing each alias's task-local async connection in the same task that opened it — the same lifecycle ``close_async_connections`` middleware provides for HTTP requests. Used by tests that drive ``process_issue_events`` / ``process_log_events`` / ``process_transaction_events`` directly. 2. ``TimedTestRunner.teardown_databases`` runs ``pg_terminate_backend`` on each test DB before stock teardown. Catches the residual leaks from paths we can't reach (Django's test client async-view dispatch, ``django_vtasks`` immediate batch flushes) so the final ``DROP DATABASE`` succeeds. 3. ``compose.yml`` raises ``max_connections`` to 300 — pure test-infrastructure headroom while we run with leaks. Production doesn't have this pattern (each request goes through middleware which closes cleanly). Disclosure: this MR was prepared with assistance from Claude. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Left over after switching ingest test invocations to ``run_async_closing``. Disclosure: this MR was prepared with assistance from Claude. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The lock file kept the workspace member entry after pyproject.toml lost it; CI failed with "Distribution not found at .../gt_rust". Regenerated with ``uv lock``. Disclosure: this MR was prepared with assistance from Claude. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The ingest hot path was a mix of native async cursors (raw SQL via async-backend) and Django ORM convenience methods (abulk_create, aupdate, aexists, async-iter on a sync QuerySet) that thread-pool through the sync connection. The latter group made the hot path double-counted across two pools and opaque to async-aware test assertions. Convert every remaining sync_to_async-wrapped DB call in process_issue_events / process_transaction_events to either raw async SQL via async_db.fetchall/execute helpers or async-backend's AsyncQuerySet (debug bundles, where ORM model construction with select_related is the right tool). Issue + IssueHash creation now runs inside async_atomic with raw INSERTs; the IntegrityError race-recovery path reads issue_id back via a SELECT. Replace assertNumQueries on the affected hot-path tests with a new AsyncQueryCounter (glitchtip/test_utils/async_query_counter.py) that patches AsyncCursorWrapper at the class level, so it observes queries across whatever task the sync test client dispatches into. Sync queries from setUp fixtures are now ignored by design. test_store_api.setUp gains cache.clear() so TransactionTestCase's RESTART IDENTITY truncation can't race the auth/throttle cache between tests in the class.
Project baseline is Python 3.12+; PEP 604 / PEP 585 are native and PEP 563 deferred evaluation isn't relied on. Use a forward-reference string for the self-referential return annotation instead.
Vulture flagged exc_type/exc_value/traceback as unused. Replace with *_exc_info — the leading underscore tells vulture the variable is intentionally unused. Also expand the module docstring to spell out why AsyncQueryCounter exists and what would have to change before AsyncCaptureQueriesContext from django-async-backend can replace it (every middleware in MIDDLEWARE has to be async_capable, and a recalibration pass to absorb the BEGIN/COMMIT delta in queries_log). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a prominent docstring banner and an import-time DeprecationWarning to glitchtip/wsgi.py. GlitchTip is async-first now: WSGI bridges every async view (including the entire ingest hot path) through async_to_sync, leaks a task-local async-backend DB connection per request, and bypasses the glitchtip.ingest_asgi fast-path dispatcher entirely. Custom deployments that bypass our scripts and import glitchtip.wsgi directly will now see the warning in their logs and have a clear pointer to glitchtip.asgi:application under Granian. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat(ingest): native async DB cursors via django-async-backend See merge request glitchtip/glitchtip-backend!2343
The release Docker build previously ran on a single amd64 runner and
emulated arm64 layers via QEMU (`multiarch/qemu-user-static`). Emulated
ARM builds are 5-10x slower than native, dominating the wall time of
every release tag.
GitLab now exposes native aarch64 SaaS runners (tag
`saas-linux-medium-arm64`), so split the multi-arch build into two
parallel native jobs (one per arch) that push arch-suffixed tags, plus a
small `release_manifest_merge` job that uses
`docker buildx imagetools create` to publish the user-facing tags
(`latest`, `X.Y.Z`, `X.Y`, `X`) as a manifest list referencing both
arches. A `release_prepare_assets` job downloads the frontend
`dist/` artifact once and shares it with both arch builds.
Also add a `validate_build_{amd64,arm64}` matrix that runs on MRs into
the default branch and on master pushes. It builds the production
Dockerfile natively for each arch (no push) and runs a cheap
`python -c "import django, glitchtip"` smoke test against the resulting
image. This catches "build succeeds but image won't run on this arch"
regressions before they reach a release tag, where the QEMU build was
the first time we discovered them.
Triggers for the release jobs are unchanged: protected vX.Y.Z tags
only, never on schedules or MRs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop verbose intros and the QEMU/legacy-job references; keep only the non-obvious "why" (smoke test purpose, no-DinD note for imagetools). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ci: native multi-arch builds + per-MR build validation See merge request glitchtip/glitchtip-backend!2349
Bulk INSERT/UPDATE in event ingest, log ingest, and the transaction pipeline previously expanded one ``(%s, %s, ...)`` tuple per row into a ``VALUES`` list via ``execute_mogrified_values``. That has three costs that grow with batch size: a per-row mogrify round-trip in Python, a distinct statement shape per batch length (poor plan-cache reuse), and the 65535 bind-parameter ceiling Postgres enforces, which clamps batches on wide schemas (the 13-column ``issue_events_issueevent`` insert tops out around 5000 rows). Pass per-column arrays to ``unnest(%s::T[], %s::T[], ...)`` instead. One parameter per column regardless of batch size; one statement shape; no bind-param ceiling. Synthetic benchmark (Postgres 18, unlogged tables, 30 rounds, local docker): issue_events_issueevent 13 cols 500 rows: 26% faster (45ms -> 33ms median) issue_events_issuetag 6 cols 2000 rows: 52% faster (70ms -> 34ms median) Converted call sites: - apps/event_ingest/process_event.py: issueevent INSERT, issuetag INSERT, update_issues UPDATE, update_statistics, update_org_statistics, transactiongroup INSERT, spanstaging INSERT, _update_transaction_group_stats Phase 2, and the two IN(VALUES) lookups (_fetch_issue_hashes_raw, _fetch_transaction_groups), now rewritten as JOIN unnest(). - apps/logs/process_logs.py: logevent INSERT, log_statistics upsert, resource_lookup upsert. Left as VALUES (not worth converting): the small release/environment fanout (tiny batches), and ``_update_transaction_group_stats`` Phase 1 which carries an int[] histogram per row that doesn't trivially unnest. Adds two thin helpers in apps/shared/async_db.py — ``execute_unnest`` and ``fetchall_unnest`` — that transpose row-major data to per-column lists before executing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
perf(ingest): switch hot-path bulk writes from VALUES to UNNEST See merge request glitchtip/glitchtip-backend!2350
Same conversion as the prior commit on this branch but for the uptime hot path: the upsert previously mogrified one ``(%s,%s,%s)`` tuple per organization in a Python loop, joined them into a ``VALUES`` list, and ran the whole thing through ``sync_to_async``. Switch to async-native ``execute_unnest`` with ``async_atomic`` so we drop both the per-row Python work and the sync_to_async thread hop on every check flush (every 1 s or 100 checks, whichever comes first). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
perf(uptime): switch update_uptime_statistics from VALUES to UNNEST See merge request glitchtip/glitchtip-backend!2351
The Stripe webhook handler was hitting unrecoverable failures on transient 429 lock_timeout responses from the Stripe API, which Stripe itself flags as retryable via the Stripe-Should-Retry header. Two related issues: 1. stripe_get/stripe_post had no retry logic — any non-200 response immediately raised. Now they share a retry helper that backs off exponentially on 429/5xx, honoring the Stripe-Should-Retry header when present. 2. The webhook view marked event IDs as processed via cache.aadd before actually processing them. If processing raised, Stripe's webhook retry would arrive, hit the dedup cache, and be silently dropped. The mark is now released on failure so retries can reprocess. 🤖 Generated with Claude Code
chore(deps): update dependency symbolic to ~=13.6.1 See merge request glitchtip/glitchtip-backend!2426
Chore/implement async organization methods See merge request glitchtip/glitchtip-backend!2429
pyproject.toml pins `symbolic~=13.6.1`, but uv.lock was still resolved at 13.5.0 — a stale lock that does not satisfy the constraint (~=13.6.1 means >=13.6.1,<13.7), so `uv lock --locked` failed on master. The renovate bump that updated pyproject never regenerated the lock. Regenerate so symbolic resolves to 13.6.1; lock and pyproject are now consistent. 🤖 Generated with AI (Claude Code) — human review required. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fix(ingest): bucket transaction usage by received time, not client time See merge request glitchtip/glitchtip-backend!2404
chore(deps): relock symbolic to 13.6.1 to match pyproject See merge request glitchtip/glitchtip-backend!2431
Replace regex-based 33-char Breakpad appendix stripping with symbolic's normalize_debug_id, which handles all format variants including non-zero appendix values that the regex approach missed. 💘 Generated with Crush Assisted-by: Deepseek V4 Pro via Crush <crush@charm.land>
💘 Generated with Crush Assisted-by: Deepseek V4 Pro via Crush <crush@charm.land>
FileBlobs become orphaned when multi-chunk uploads are concatenated into a combined blob, when Files are replaced during source-map re-uploads, or when uploads are abandoned before assembly. Add cleanup_orphaned_file_blobs() to the daily maintenance task. It deletes FileBlobs with zero File references that are older than 24 hours (grace period to protect in-flight uploads).
chore(deps): update dependency symbolic to ~=13.7.0 See merge request glitchtip/glitchtip-backend!2433
apps/organizations_ext/permissions.py defined three DRF-style ScopedPermission subclasses left over from before the migration to django-ninja. It imported `from glitchtip.permissions import ScopedPermission`, but that module no longer exists — importing this file would raise ImportError. Nothing references the module or its classes anywhere in the codebase, so it is dead code. Authorization is now enforced via glitchtip/api/permissions.py (has_permission) plus per-endpoint role checks; this file played no part. AI-assisted change (Claude Code). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pulls in glitchtip-rust 0.3.1, whose gt_rust.django_backend now honors libpq/psycopg verify-ca semantics (CA chain checked, hostname not), so the Rust Postgres driver can connect through the staging pgbouncer pooler whose serving cert is issued for the CNPG cluster service names, not the pooler. uv lock also corrects a latent lockfile inconsistency: pyproject already pins symbolic~=13.7.0 but the lock still had 13.6.1; relocking updates it to 13.7.0 to satisfy the existing constraint. No other packages change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…_debug_id 💘 Generated with Crush Assisted-by: Deepseek V4 Pro via Crush <crush@charm.land>
chore(deps): bump glitchtip-rust to >=0.3.1 (verify-ca hostname fix) See merge request glitchtip/glitchtip-backend!2436
- Merge orphan cleanup into cleanup_old_files instead of a separate maintenance step - Use aiterator() instead of sync_to_async(list) for real async - Delete redundant cleanup_orphaned_fileblobs management command
fix: fix support for mattermost webhooks glitchtip-backend#485 See merge request glitchtip/glitchtip-backend!2434
Accept Breakpad-format debug_id in NativeDebugImage See merge request glitchtip/glitchtip-backend!2420
feat(files): periodic cleanup of orphaned FileBlobs See merge request glitchtip/glitchtip-backend!2432
Follow-ups to the orphaned-FileBlob cleanup: - Delete DB rows before storage, filtering through the original queryset so the pass conditions are re-evaluated inside the delete. Uploads dedupe on checksum via get_or_create, so a blob fetched as orphaned can gain a File reference before the batch delete runs; previously that File would be cascaded away. Row-first ordering means a skipped blob keeps both its row and its storage bytes — a crash mid-batch can only leak an unreferenced storage object, never leave a File whose backing bytes are gone. - Pass save=False to blob.delete() so it no longer issues an UPDATE blanking the blob field on an already-deleted row, halving sync-connection writes per blob. - Count only FileBlob rows against MAX_DELETIONS_PER_RUN instead of the cascade total, so the cap is a blob budget. - Log storage-delete failures with exc_info so a missing file is distinguishable from a storage misconfiguration. - Assert storage-file existence in the cleanup tests. The ORM stays on the async API throughout; only the storage delete hops threads, since django-storages has no async API. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
perf(files): streamline FileBlob cleanup deletes See merge request glitchtip/glitchtip-backend!2439
chore(deps): update dependency symbolic to ~=13.8.0 See merge request glitchtip/glitchtip-backend!2437
django-organizations 2.7.0 ships async versions of the core membership utilities. Bump to ~=2.7 and drop GlitchTip's local reimplementation of the create + user_added signal mechanics: - Organization.add_user/aadd_user now delegate org-user creation and the user_added signal to upstream via super(), keeping only the role-based first-user-becomes-owner logic (upstream's abstract variant is is_admin-based, which GlitchTip does not use). - The create-organization API calls aadd_user instead of hand-rolling the org user and owner records and sending the sync user_added signal from an async view. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Assert the creating user gets the OWNER role and an OrganizationOwner record, since that behavior now lives in aadd_user's first-user branch rather than inline in the endpoint. Also document that user_added fires before the owner record exists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
refactor(organizations): use django-organizations 2.7 async core utils See merge request glitchtip/glitchtip-backend!2440
… into upstream-changes-2026-07-06
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is auto-generated by
actions/github-script.