[UPSTREAM CHANGES] latest changes as of Mon Mar 30 2026 00:52:45 GMT+0000 (Coordinated Universal Time) - #222
Open
github-actions[bot] wants to merge 5276 commits into
Open
[UPSTREAM CHANGES] latest changes as of
Mon Mar 30 2026 00:52:45 GMT+0000 (Coordinated Universal Time)#222github-actions[bot] wants to merge 5276 commits into
github-actions[bot] wants to merge 5276 commits into
Conversation
fix: use DETACH PARTITION CONCURRENTLY to avoid deadlocks See merge request glitchtip/glitchtip-backend!2151
Issue search direct-hit only checked issueevent.id (UUIDv7), so searching by client-provided sentry SDK event_id (UUIDv4) returned no results. Now checks both id and event_id via is_uuid7() dispatch: UUIDv7 queries prune to a single time-range partition, while UUIDv4 queries scan the event_id index with organization_id to prune hash sub-partitions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fix: search issues by sentry SDK event_id See merge request glitchtip/glitchtip-backend!2152
The event detail endpoints only looked up by `issueevent.id` (UUIDv7), so when the frontend redirects with a client-provided sentry SDK event_id (UUIDv4), the lookup 404'd. Now sniffs for UUIDv7 first (fast partition-pruned lookup), and falls back to `event_id` field with `organization_id` for hash sub-partition pruning when the UUID is not v7. Applies to all three event detail endpoints: get_issue_event, get_project_issue_event, get_event_json. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fix: event detail endpoints fall back to sentry SDK event_id See merge request glitchtip/glitchtip-backend!2153
- Skip model instantiation: use values_list("id", "blob") +
default_storage.delete() instead of loading full FileBlob instances.
Cuts one query per batch.
- Move count() to --dry-run only — avoids a full-table anti-join that
serves no purpose during actual cleanup.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Vulture flags code after `while True:` as unreachable (doesn't understand `break`). Refactor to `while batch:` pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The event detail endpoints only looked up by `issueevent.id` (UUIDv7), so when the frontend redirects with a client-provided sentry SDK event_id (UUIDv4), the lookup 404'd. Now sniffs for UUIDv7 first (fast partition-pruned lookup), and falls back to `event_id` field with `organization_id` for hash sub-partition pruning when the UUID is not v7. Applies to all three event detail endpoints: get_issue_event, get_project_issue_event, get_event_json. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When events have been archived to parquet, MCP get_latest_event() and get_event() now fall back to DuckDB cold storage instead of returning None. Mirrors the pattern already used in the REST API. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Accept optional organization_slug for partition pruning in both Postgres and cold storage queries - Split Postgres query into separate id/event_id branches (avoids OR across partitions when type is known) - Add UUIDv4 cold storage fallback via query_cold_events with event_id filter and 90-day time window - Extract _attach_issue helper for defense-in-depth access check - Add event_id filter support to query_cold_events Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously the memory limit was only applied when DUCKDB_TEMP_DIRECTORY was explicitly set and writable. Without it, DuckDB ran with unbounded memory, causing OOM kills during cold storage archival of large partitions. Changes: - Always set memory_limit (default 128MB) regardless of temp directory - Auto-detect writable temp directory via tempfile.gettempdir() - DUCKDB_TEMP_DIRECTORY env var still works as an explicit override This prevents OOM kills in workers during archival. Without a temp dir, DuckDB queries exceeding the memory limit will error instead of silently consuming all available memory.
Previously the memory limit was only applied when DUCKDB_TEMP_DIRECTORY was explicitly set and writable. Without it, DuckDB ran with unbounded memory, causing OOM kills during cold storage archival of large partitions. Changes: - Always set memory_limit (default 128MB) regardless of temp directory - Auto-detect writable temp directory via tempfile.gettempdir() - DUCKDB_TEMP_DIRECTORY env var still works as an explicit override This prevents OOM kills in workers during archival. Without a temp dir, DuckDB queries exceeding the memory limit will error instead of silently consuming all available memory.
CSV archival of issue events fails when data::text contains JSON with
backslash-escaped quotes (\") that conflict with standard CSV
double-quote escaping (""). DuckDB's CSV parser can't resolve the
ambiguous \\"" sequences.
Fix by using SOH (\x01) as the CSV quote/escape character for both
Postgres COPY TO STDOUT and DuckDB read_csv. This control character
never appears in real text/JSON data, making parsing unambiguous.
Also fix deadlocks during DETACH PARTITION CONCURRENTLY by:
- Retrying up to 3 times with exponential backoff on deadlock
- Falling back to non-concurrent DETACH if all retries fail
- Properly distinguishing "already detached" from real errors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fix: always set DuckDB memory limit and auto-detect temp directory See merge request glitchtip/glitchtip-backend!2156
fix: use SOH quote char for CSV archival and retry deadlocked detach See merge request glitchtip/glitchtip-backend!2157
Uses `uv run` so hooks use the same tool versions from pyproject.toml that CI uses. No extra installs needed beyond `uv sync`. Setup: `pre-commit install` (requires pre-commit, e.g. `uv tool install pre-commit`) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fix: add cold storage fallback to MCP event lookups See merge request glitchtip/glitchtip-backend!2155
Replace DuckDB's CSV→Parquet write path with arro3 (Rust Arrow/Parquet via PyO3). DuckDB remains for all read/query operations. Key changes: - _flush_csv_to_parquet now uses arro3.io.read_csv + write_parquet instead of DuckDB's read_csv → COPY TO PARQUET - No temp files needed — reads CSV from BytesIO, writes Parquet directly - Removed DuckDB write connection from archive_partition_per_org - DELTA_BYTE_ARRAY encoding for high-cardinality string columns (IDs, body, JSON data) with dictionary encoding for low-cardinality columns (service, environment, host) - Per-table dictionary_columns config in logs and issue_events wrappers Performance (50K rows, realistic JSON): - Write speed: ~12x faster than DuckDB - Peak memory: ~0.1 MB vs ~4.5 MB (Python-tracked) - File size: 35% smaller than DuckDB (170 KB vs 262 KB) thanks to DELTA_BYTE_ARRAY + ZSTD compression Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
django-storages' save() appends random suffixes when a file exists, which would make archived Parquet files unreadable by the cold storage query path. Also update stale docstrings referencing DuckDB writes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
chore(deps): update dependency ruff to v0.15.5 See merge request glitchtip/glitchtip-backend!2160
chore(deps): update dependency boto3 to v1.42.62 See merge request glitchtip/glitchtip-backend!2148
…les/dsyms/) Closes #459
- Tighten arro3 dependency pins from >=0.8 to ~=0.8 (compatible release) - Update settings.py comment: DuckDB controls reads, arro3 handles writes - Fix _duckdb_type_to_arrow docstring (arro3 is now a hard dependency) - Log S3 delete failures at debug level instead of silent pass Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fix: auto-set GRANIAN_WORKERS_MAX_RSS to bound memory fragmentation See merge request glitchtip/glitchtip-backend!2243
perf: convert hot-path ORM queries to raw SQL in event ingest See merge request glitchtip/glitchtip-backend!2242
check_all_organizations_throttle() used .aiterator() which creates a server-side cursor, incompatible with PgBouncer transaction pooling. Replace with keyset pagination (id__gt) in batches of 500 to keep memory bounded without requiring server-side cursors. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- wizard: setup_wizard and setup_wizard_delete now use async cache ops - api fallback 404: async to avoid unnecessary thread in ASGI - prometheus metrics: inline generate_latest instead of sync_to_async wrapping the django-prometheus view (no I/O, just CPU formatting) django-ninja uses sync Operation for sync views, which Django's ASGI handler wraps in sync_to_async — creating a thread for no reason. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix: convert remaining sync views to async See merge request glitchtip/glitchtip-backend!2244
fix: replace server-side cursor with batched iteration in throttle check See merge request glitchtip/glitchtip-backend!2246
Oversized payloads on the /store/ endpoint raised RequestDataTooBig during ninja's parameter parsing, which bubbled up as an unhandled error event. The middleware already logs a warning with diagnostic details. Adding a ninja exception handler returns a clean 413 response without generating noise in error tracking. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
chore(deps): update dependency boto3 to v1.42.77 See merge request glitchtip/glitchtip-backend!2247
chore(deps): update dependency ruff to v0.15.8 See merge request glitchtip/glitchtip-backend!2248
Concurrent INSERT ... ON CONFLICT DO UPDATE statements on uptime_uptimecheckhourlystatistic could deadlock when acquiring row locks in different orders. Sorting the batch by organization_id ensures consistent lock ordering. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ubscriptions from stripe sync or webhook.
DuckDB defaults to one thread per HOST CPU core. In Kubernetes, it reads the node's cores (e.g. 64) rather than the pod's CPU limit (e.g. 2). Each thread allocates scan buffers outside the memory_limit setting, causing VmPeak to hit 8+ GB and triggering OOM pod recycling. Changes: - Set threads on ALL connections (was only set on write connections). Default: 2, configurable via DUCKDB_THREADS env var. - Lower default memory_limit cap from 1024 MB to 256 MB. The memory_limit only bounds DuckDB's internal buffer pool — thread stacks, mmap'd regions, and jemalloc overhead are additional. A 256 MB pool with 2 threads typically peaks at ~400-500 MB total process impact. - Both settings are configurable for dedicated analytics workloads. Benchmark proof (8-core machine, 500k rows across 10 Parquet files): BEFORE (8 threads, 1024MB): VmPeak 985 MB, RSS +147 MB AFTER (2 threads, 256MB): VmPeak 320 MB, RSS +45 MB Query time impact: +4% (negligible) On a 64-core Kubernetes node the VmPeak reduction would be proportionally larger — consistent with the 8.1 GB VmPeak observed in production. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DUCKDB_THREADS now reads cgroup v2 cpu.max (Kubernetes/Docker) to use the pod's actual CPU limit instead of a hardcoded default. Falls back to min(os.cpu_count(), 4) on bare-metal. Also refactors the cgroup reading into shared helpers: - _get_cgroup_memory_bytes(): reads /sys/fs/cgroup/memory.max - _get_cgroup_cpu_count(): reads /sys/fs/cgroup/cpu.max Both settings auto-detect but accept env var overrides: - DUCKDB_MEMORY_LIMIT: auto from cgroup memory (25%, cap 256MB) - DUCKDB_THREADS: auto from cgroup CPU quota (or cap at 4) - DUCKDB_TEMP_DIRECTORY: defaults to /tmp Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add billing cycle dates to created subscriptions See merge request glitchtip/glitchtip-backend!2253
fix: cap DuckDB threads and memory to prevent VmPeak explosion See merge request glitchtip/glitchtip-backend!2254
fix: sort uptime stats upsert by org_id to prevent deadlock See merge request glitchtip/glitchtip-backend!2251
fix: handle RequestDataTooBig with 413 instead of unhandled exception See merge request glitchtip/glitchtip-backend!2250
chore(deps): update dependency google-cloud-logging to v3.15.0 See merge request glitchtip/glitchtip-backend!2249
chore(deps): update dependency model-bakery to v1.23.4 See merge request glitchtip/glitchtip-backend!2256
chore(deps): update dependency boto3 to v1.42.78 See merge request glitchtip/glitchtip-backend!2255
chore(deps): update dependency aiohttp to v3.13.4 See merge request glitchtip/glitchtip-backend!2258
… into upstream-changes-2026-03-30
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is auto-generated by
actions/github-script.