[UPSTREAM CHANGES] latest changes as of Mon Mar 16 2026 00:51:37 GMT+0000 (Coordinated Universal Time) - #220
Open
github-actions[bot] wants to merge 5133 commits into
Open
[UPSTREAM CHANGES] latest changes as of
Mon Mar 16 2026 00:51:37 GMT+0000 (Coordinated Universal Time)#220github-actions[bot] wants to merge 5133 commits into
github-actions[bot] wants to merge 5133 commits into
Conversation
- Add computed errorRate and throughput fields to TransactionGroupSchema
and MCP serializer
- Add query_n_plus_one_patterns() DuckDB query that detects repeated
spans per transaction (spans_per_txn > threshold)
- Add query_transaction_trend() DuckDB query for daily performance stats
- Add detect_n_plus_one and get_transaction_trend MCP tools
- Add GET .../n-plus-one/ and .../transaction-groups/{id}/trend/ REST endpoints
- Fix MCP OAuth discovery: rewrite /.well-known/openid-configuration to
/.well-known/oauth-authorization-server in ASGI dispatcher
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The performance MCP tools (transaction groups, spans, N+1 detection, trends) were gated behind ENABLE_OBSERVABILITY_API which controls Prometheus metrics — unrelated to performance monitoring. These tools should always be registered when MCP is enabled. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ut, rename trend fields - Add optional start/end ISO 8601 params to list_transaction_spans, list_span_groups, detect_n_plus_one, and get_transaction_trend MCP tools (previously hardcoded to 7-day lookback) - Move error_rate and throughput computation to TransactionGroup model properties, removing duplication between schema and MCP serializer - Rename trend fields from span-centric (spanCount, avgSpanDuration, totalSpanTime) to consistent naming (count, avgDuration, totalTime) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Transaction and log hourly stat partitions were being dropped using GLITCHTIP_EVENT_RETENTION_DAYS instead of their own retention settings. This could cause stats to be dropped too early if transaction or log retention is set longer than event retention. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Transactions now generate significant I/O (span staging, Parquet promotion, cold storage) so the old 0.1x weight no longer reflects actual resource cost. Logs remain at 0.1x. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Change promotion DELETE from range-based (id >= min AND id <= max) to exact match (id = ANY(...)) to prevent over-deleting rows inserted by concurrent workers between SELECT and DELETE - Change count and error_count from PositiveIntegerField (max ~2.1B) to PositiveBigIntegerField to prevent overflow on high-traffic endpoints that accumulate counts over their lifetime Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Model property tests: error_rate, throughput, BigInteger overflow safety - End-to-end ingest tests: TransactionGroup stats accumulation, histogram merge across batches, p50/p95 computation, error_count from trace status, SpanStaging population, SQL parameterization - Serializer tests: MCP serialize_transaction_group output, Pydantic schema computed_field delegation, camelCase alias verification Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix compaction crash safety: _enumerate_parquet_files now skips chunk subdirectories when a compacted flat file exists for the same date - Make promotion partition-aware: iterate per-org so PostgreSQL can prune both RANGE (UUIDv7) and HASH (organization_id) sub-partitions - Remove duplicate _build_histogram_increment, use merge_durations from histogram module - Document p50/p95 race condition in _update_transaction_group_stats - Add TODO for windowed throughput computation - Fix test_data.py histogram percentile bug (count vs sample_size) - Add 13 cold storage integration tests (promotion, compaction, queries) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix ImportError: tasks.py imported nonexistent BATCH_LIMIT; promote_spans now returns (count, truncated) tuple, task re-enqueues on truncation - Fix chunk file cleanup gap: cleanup_cold_storage_for_org now deletes both flat files and chunk subdirectories for expired dates - Add thread-local DuckDB connection caching: get_duckdb_read_connection() reuses connections per thread for read-only cold storage queries - Eliminate duplicate ORM query: query_span_groups_for_transaction accepts transaction_name directly instead of re-fetching the TransactionGroup Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The two early-exit paths (DuckDB unavailable, no storage backend) returned bare int 0 instead of (0, False) tuple, causing ValueError in the scheduled task that destructures the return value. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…aping - M1: Invalidate thread-cached DuckDB connection on query errors to prevent poisoned connections from failing subsequent queries - M2: Add cross-organization isolation test verifying user in org B cannot access org A's transaction groups (list, detail, spans, trend) - M3: Escape single quotes in DuckDB path interpolation via duckdb_quote_path() helper to prevent SQL injection from paths - m1: Expose limit parameter on span-groups and n-plus-one REST endpoints - m2: Type TransactionTrendSchema.date as datetime instead of str - m4: Use freezegun in test_promote_skips_recent_rows for deterministic timing instead of relying on wall clock Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…nd races - Escape all DuckDB parquet paths with duckdb_quote_path() consistently - On promotion DELETE failure, remove the written chunk to prevent duplicates - Crash-safe compaction via temp file + atomic rename on filesystem - Skip last 2 days in compaction to prevent race with concurrent promotion - Wrap stats update phases in transaction.atomic() to prevent stale p50/p95 - Use time.time_ns() for chunk filenames to avoid collisions - Use CREATE OR REPLACE for DuckDB temp tables defensively Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ilures Only re-enqueue immediately when progress was actually made (promoted > 0). If all parquet writes fail (e.g. DuckDB extension missing, disk full), the task now waits for the next scheduled run instead of tight-looping. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
chore(deps): update dependency boto3 to v1.42.55 See merge request glitchtip/glitchtip-backend!2104
Replace cursor.fetchall() with cursor.fetchmany() batching in archive_partition_per_org to bound peak memory usage. Previously the entire partition was loaded into Python memory, normalized into a second copy, then inserted into DuckDB as a third copy — ~1.5-2GB+ for a 550MB partition, exceeding the 768Mi worker memory limit. Now streams in ARCHIVE_BATCH_SIZE (5000) row batches, keeping peak memory to ~batch_size × 2 copies. DuckDB table and output directory are lazily created on the first non-empty batch. Extract _normalize_row() helper from the inline normalization loop. AI-assisted (Claude). Human review required. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…, conn reuse) - Make archive_partition_per_org idempotent: skip orgs whose Parquet file already exists (1 S3 HEAD vs full re-export on crash restart) - Guard detach_partition with try/except for already-detached partitions - Reuse single DuckDB connection across orgs in archive loop (avoids 9,999 DuckDB startups at 10K orgs scale) - Switch query_cold_parquet_files to cached read connection - Revert promote_spans .iterator() — server-side cursors break PgBouncer - Document PgBouncer/server-side cursor gotcha in CLAUDE.md Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- P0: Use shared detach_partition() in logs delete_old_hot_partitions (was inline DETACH without try/except guard) - P1: Switch get_event_from_cold to cached read connection, avoiding full DuckDB startup on every single-event lookup - P1: Add missing duckdb_quote_path() in get_event_from_cold SQL - P2: Replace .iterator() with values_list() in cleanup_all_cold_storage (server-side cursors break PgBouncer transaction pooling) - Remove dead re-exports from apps/logs/cold_storage.py, update callers to import directly from glitchtip.cold_storage Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…write - Switch _get_log_from_cold to cached read connection + duckdb_quote_path (same fix applied to get_event_from_cold in previous commit) - Reuse single DuckDB connection across file loop in rewrite_parquet_excluding_project (was creating one per file) - Extract is_missing_file_error() helper to deduplicate the "file not found" check repeated in 3 call sites Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Free-tier orgs (no subscription or price=0) should have data deleted at hot_days rather than archived to cold storage. The existing isnull=False check missed the $0 subscription case where a user downgrades but retains a Stripe presence. Added price__price__gt=0 filter to both archive_partition_per_org (archival eligibility) and cleanup_all_cold_storage (cleanup scope). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Recreate TransactionGroup as PARTITION BY HASH (organization_id) for multi-tenant query pruning. API queries always filter by organization, so each query now scans 1/N partitions instead of the full table. - Composite PK (id, organization_id) required by PG for hash partitioning - Unique constraint now includes organization_id - Migration uses SeparateDatabaseAndState with raw SQL for table swap - Hash bucket count follows settings.PARTITION_HASH_BUCKETS - Backfill from old table is conditional (handles edge cases) - Removed Django admin registration (incompatible with CompositePrimaryKey) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
managed=False tables are excluded from Django's TRUNCATE during test flush. DB-level FK constraints from TransactionGroup to projects_project and organizations_ext_organization caused "cannot truncate a table referenced in a foreign key constraint" errors. Remove DB FKs and use DO_NOTHING on the Django model, matching the SpanStaging pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fix: stream archive_partition_per_org to prevent OOM See merge request glitchtip/glitchtip-backend!2106
promote-spans (every 5 min) and compact-span-chunks (daily) were scheduled unconditionally, wasting worker cycles when DuckDB cold storage is not configured. process_transaction_events also wrote SpanStaging rows that would never be promoted. - Make promote-spans/compact-span-chunks schedule entries conditional on GLITCHTIP_ENABLE_DUCKDB=true in settings - Skip span extraction in process_transaction_events when DuckDB is disabled (TransactionGroup stats still collected) - Remove backfill from migration 0021 SQL — upgraders start fresh - Remove managed=False from TransactionGroup and SpanStaging to match IssueEvent/LogEvent/MonitorCheck pattern (SeparateDatabaseAndState already prevents Django from managing the DDL) - Add @override_settings to span-specific tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fix: gate span ingestion and tasks on GLITCHTIP_ENABLE_DUCKDB See merge request glitchtip/glitchtip-backend!2107
Move `is_duckdb_available` import in process_event.py and four cold_storage imports in performance/api.py to top-level. These modules are lightweight with no circular dependency risk. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GLITCHTIP_ENABLE_DUCKDB=true without a configured storage backend (bucket, dir, or STORAGES["cold"]) previously returned True, causing SpanStaging rows to be written even though they could never be promoted. Now returns False unless a backend is actually available. Result is cached at module level and automatically invalidated by Django's setting_changed signal for test compatibility. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add default="" to TransactionGroup.method and SpanStaging.description to match SQL's DEFAULT '' - Add CHECK >= 0 constraints on count/error_count in SQL to match PositiveBigIntegerField's DB-level expectation - Fix misleading "managed=False" comment in SQL (actual reason is partitioned tables can't be FK targets + test flush issues) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When an issue is resolved in a specific release and a new event arrives without a release, skip reopening. Only reopen when the event has a release that differs from resolved_in_release. Also add clarifying comment on TransactionGroup.id db_default. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…catch Flask is not installed in the production Docker image, so importing sentry_sdk.integrations.flask raises DidNotEnable (not ImportError). Remove Flask from the list since it can never auto-activate in production. Broaden except to catch all Exception types including DidNotEnable. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The sentry-dotnet SDK sends debug images with type "pe_dotnet" for managed .NET modules. These were previously falling through to OtherDebugImage and generating noisy "Unknown debug image type" warnings in production logs. Also adds the debug_checksum field to NativeDebugImage, which the .NET SDK includes for PDB verification. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
chore(deps): update dependency boto3 to v1.42.66 See merge request glitchtip/glitchtip-backend!2184
django-vcache v2.0.0 includes django-valkey as a dependency, so the explicit django-valkey dependency is no longer needed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
cleanup_old_issues ran 3 NOT EXISTS subqueries against partitioned tables (IssueEvent, IssueAggregate, IssueTag) without organization_id, forcing Postgres to scan every partition. This exceeded the 45s statement_timeout on the readwrite role. Since maintain_partitions already drops old partitions before cleanup runs, replace the subqueries with a 7-day buffer beyond retention to account for weekly partition lag. archive_partition_per_org and get_partitions_older_than used Django's default connection (readwrite, 45s) for SELECT/COPY operations. Thread db_alias through so they use the maintenance connection (30min timeout) alongside the DDL operations that already did. AI was used to help write this merge request. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Standardize all maintenance cleanup functions to use MAINTENANCE_DATABASE_ALIAS for consistency. This routes maintenance queries through the maintainer role (longer statement_timeout) instead of the readwrite role (45s, designed for web requests). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fix: resolve maintenance statement timeouts See merge request glitchtip/glitchtip-backend!2185
django-valkey was removed as a dependency but two files still imported from it. Use caches["default"].get_raw_client() from django-vcache instead, with eval_sync() for Valkey set operations via Lua scripts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
chore(deps): update dependency ruff to v0.15.6 See merge request glitchtip/glitchtip-backend!2188
chore(deps): update dependency django-vcache to v2.0.4 See merge request glitchtip/glitchtip-backend!2183
Downgrade from v2.x to latest v1.x releases to work around a regression. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
v2's Rust driver handles both sync and async natively, so the separate ValkeyWSGICache is no longer needed. Connection pool OPTIONS are also removed — the Rust driver manages its own multiplexed connections. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v2's Rust driver handles both sync and async natively, so the separate ValkeyWSGICache is no longer needed. Connection pool OPTIONS are also removed — the Rust driver manages its own multiplexed connections. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
vtasks v2.0 is compatible with vcache v2's Rust driver (no more async_client kwarg on get_raw_client). Convert CACHE_SENTINEL_URL to sentinel:// URL format for vcache v2. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
chore(deps): update dependency django-vcache to v2.0.9 See merge request glitchtip/glitchtip-backend!2190
CREATE TABLE IF NOT EXISTS ... PARTITION OF takes AccessExclusiveLock on the root table and ALL sibling partitions, even when the table already exists. This blocks concurrent INSERTs and causes deadlocks under load. The fix: - Check partition existence via pg_class catalog queries before executing DDL (catalog queries only take AccessShareLock, no conflict) - Skip DDL entirely for partitions that already exist - Wrap remaining DDL in transaction.atomic() so new partitions are created atomically with minimal lock duration For the common case (6 of 7 daily partitions already exist), this eliminates all AccessExclusiveLock contention. For new partitions, the lock is held only for milliseconds. AI was used in the creation of this MR. Human review is required. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
fix: avoid deadlock during partition maintenance with concurrent INSERTs See merge request glitchtip/glitchtip-backend!2192
Fixes ContextVar.reset(token) ValueError in allauth AccountMiddleware on async ASGI requests. vcache 2.0.10 preserves contextvars.Context through the async callback mode, matching asyncio.Future's contract. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… into upstream-changes-2026-03-16
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is auto-generated by
actions/github-script.