Import rspec-mergify into clients/rspec-mergify with its git history - #83
Closed
kozlek wants to merge 39 commits into
Closed
Import rspec-mergify into clients/rspec-mergify with its git history#83kozlek wants to merge 39 commits into
kozlek wants to merge 39 commits into
Conversation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Switch to single-quoted strings throughout - Migrate rubocop-rspec from require to plugins syntax - Sort Gemfile gems and gemspec dependencies alphabetically - Add rubygems_mfa_required metadata to gemspec - Wrap long description line in gemspec Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements Mergify::RSpec::Utils as the foundational utility layer: strtobool, env_truthy?, in_ci?, ci_provider, repository_name_from_url, split_full_repo_name, git, get_attributes, and repository_name, matching the Python pytest-mergify utils.py reference implementation. Also adds rspec gem to Gemfile (needed for rspec-expectations/rspec-mocks in tests) and the full TDD test suite (38 examples, 0 failures). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… Mergify, and RSpec Implements 6 resource detector modules under Mergify::RSpec::Resources, each with a .detect method returning OpenTelemetry::SDK::Resources::Resource with provider-specific attributes matching the Python pytest-mergify implementation. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements the central CI Insights orchestrator that sets up OpenTelemetry tracing with resource detection, supports debug/test/production modes, and a custom batch span processor that queues spans and exports them all on force_flush to avoid HTTP requests during test execution. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…al report Implements Mergify::RSpec::Formatter as a purely observational RSpec formatter that creates OpenTelemetry spans for session and per-example tracing, sets attributes for test results and flaky metadata, and prints a terminal CI report with MERGIFY_TEST_RUN_ID on stop. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…Mergify API Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…get constraints Implements FlakyDetector and TestMetrics classes that manage flaky test detection in "new" and "unhealthy" modes, with budget-based deadline tracking, per-test metrics, and a text report generator. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wire everything together: create Configuration module that registers RSpec hooks for quarantine (before/after) and flaky detection (around), add the formatter in CI mode, and update the entry point to auto- initialize CIInsights and register hooks on require. Fix ci_insights.rb load_flaky_detector (wrong filename), load_quarantine (was stub), and mark_test_as_quarantined_if_needed (was always false). Expose FlakyDetector#mode via attr_reader. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add end-to-end integration specs that verify CIInsights creates a working InMemorySpanExporter in test mode and that Quarantine correctly fetches, matches, and reports quarantined tests via a stubbed API. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… gracefully - SynchronousBatchSpanProcessor#on_finish now calls span.to_span_data before queuing, matching what the OTLP exporter expects - Formatter#flush_and_shutdown rescues errors during force_flush and shutdown, printing them as warnings instead of crashing rspec Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: Ic92ea473d5701890ba34aa17dc3f37c735fd0461 Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: I2483e63c1542db3648d57add7597dd5bd4a4b123 Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
- version.rb uses git describe to derive version from tags (fallback 0.0.0.dev) - Release workflow uses rubygems/release-gem with OIDC trusted publishing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: I3caf7f9fa99cc6c2edd6e62f1edbdd1d363ddc62 Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
- Pass GEM_VERSION from release tag so version.rb resolves correctly - Strip v prefix from tag name in version.rb - Override rake release:source_control_push and release:guard_clean to skip git operations (handled by GitHub Releases) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: I42b311ad9ad062585c18451be5acedb53450060b Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: I5ac698e45678a2eec459fd656b19175b073686e5 Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
…ibility - Add troubleshooting hints when trace export fails (token, CI Insights, docs) - Fix version.rb to produce valid RubyGems versions from git describe (strip v prefix, convert "0.1.0-3-gabc123" to "0.1.0.dev3") Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: Id88c1975428b05dce47eee9bdff6150e0a93b8d0 Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
…detection - Add before(:suite) hook to call prepare_for_session with all example IDs so the detector knows which tests to track and budget is calculated - Feed metrics from each example's initial run into fill_metrics_from_report so rerun decisions work correctly Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: If005d1f3924e1b72d7e639838d893f4a49840204 Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
- Extract print_export_error method to reduce flush_and_shutdown length - Nest SynchronousBatchSpanProcessor and CIInsights specs under single top-level describe to fix RSpec/MultipleDescribes - Replace test doubles with Object.new for span mocks - Fix rubocop autocorrectable offenses (receive_messages, indentation) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: Ic211a2ec198cebf3198b375d3a1f119e7614b775 Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
- Test flaky_detector loaded when _MERGIFY_TEST_NEW_FLAKY_DETECTION is true - Test error handling when flaky detection API fails or times out - Test quarantine loaded when branch_name is available - Test mark_test_as_quarantined_if_needed with real quarantine data Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: Ic0c8fd1cef108a6cb4e9d8a331c233fffa7176b7 Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: I8582111961301b31185dbe99407790a2dc68e677 Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
…check These attributes should be set on all flaky detection candidates, even tests that are too slow to rerun, matching the Python behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: I2f12c53ac35ca16e9d20e9516cebdb0f54af4fcd Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
…ectly Without this, TestMetrics.rerun_count stayed at 1 after the initial run because fill_metrics_from_report was never called for reruns. This caused last_rerun_for_test? to never trigger on max_test_execution_count, and make_report to show incorrect rerun counts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: Idb72ecf5323dcb00ee3208f5941f039ef3db923d Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
…utes FlakyDetector tests: - rerun_count increments on each call phase (initial=1, then +1 per rerun) - total_duration accumulates across all phases and reruns - initial_call_duration only set from first call - last_rerun_for_test? triggers at max_test_execution_count via metrics - last_rerun_for_test? stays false before max count Formatter tests: - cicd.test.flaky_detection set from mergify_flaky_detection metadata - cicd.test.new set from mergify_new_test metadata - cicd.test.rerun_count set from mergify_rerun_count metadata - cicd.test.flaky set from mergify_flaky metadata - None of these attributes set when metadata is nil Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: I83fc4620dcea983f5f3130f878a49e672a11aebd Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
…o warnings - Test that CIInsights creates no tracer when not in CI - Test warning output when MERGIFY_TOKEN is missing - Test warning output when repository name is missing - Update .rubocop.yml to exclude integration test sandbox patterns - Fix rubocop offenses across all spec files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: I7a6d8b740e0b8f6f868ac45ddb6b0726cb0e4069 Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
Use vcs.ref.base.name (GITHUB_BASE_REF) to decide between "new" and "unhealthy" mode. Base branch is only set in PR context, so: - PR runs → "new" mode (detect flaky new tests) - Push/schedule runs → "unhealthy" mode (monitor known flaky tests) Previously used branch_name which includes vcs.ref.head.name fallback, meaning push events on main would incorrectly use "new" mode. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: I8f40103703249870354acc3726c1e384094fd91b Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
GITHUB_BASE_REF is set to "" (empty string) on push/schedule events. In Python empty string is falsy, but in Ruby it's truthy, causing vcs.ref.base.name to be set to "" and flaky detection to incorrectly pick "new" mode instead of "unhealthy" on non-PR runs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: I20ba67103fdb3b9ee59b4520a08dc24875942a12 Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
RSpec >= 3.12 uses ThreadsafeMemoized for @__memoized which doesn't have a clear method. Access the inner @memoized hash instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: Ife38ba6762460f042e6fd4150c307cee20107c0f Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
Use NullReporter when re-running examples during flaky detection to prevent each rerun failure from being reported as a separate test failure. Previously, a test rerun 200 times would show 200 failures in the output instead of a single result. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: I01fa9265df3f857c31aaa783a19abe5859659327 Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: I6097d83030a2c20d40d4873bb520091df05f4d2d Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Change-Id: I8f024baad23a866662f5abe4a5cee081a78d9534 Claude-Session-Id: 42cf8738-224f-43c9-8cec-0306f468649f
Brings get_attributes under the Metrics/MethodLength limit (11 → 7). Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Detect Buildkite environment and extract OpenTelemetry resource attributes: pipeline name, job name, build ID/URL, retry count, agent name, branch, base branch, commit SHA, and repository. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use BUILDKITE_PIPELINE_SLUG (not BUILDKITE_PIPELINE_NAME) for cicd.pipeline.name and report cicd.pipeline.run.attempt as 1-based (BUILDKITE_RETRY_COUNT + 1) to match pytest-mergify and mergify-ci-plugins-ts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Mergify API kept the unpaginated `GET /v1/ci/{owner}/repositories/{repo}/quarantines` shortcut for backward compatibility but its OpenAPI now states the shortcut "is expected to go away once all callers paginate." Repositories with large quarantine sets (e.g. PSPDFKit at 2,463 entries) trigger statement timeouts on the unpaginated path. Walking RFC 5988 `next` links bounds each request and lets the server drop the shortcut.
References: MRGFY-7391
…#24) ## What Remove the private `_MERGIFY_TEST_NEW_FLAKY_DETECTION` gate; the client now always requests the flaky-detection context and lets the server decide. | Server response | Client behavior | |---|---| | `200` + context (baseline may be empty) | opted in → run detection | | `404` | not opted in → **skip silently** | | `401` / `403` / `5xx` / timeout | real error → surface the banner | The empty-baseline guard (new mode, no recorded tests) now also skips silently instead of erroring; it still prevents rerunning the whole suite. Both cases raise a new `FlakyDetectionDisabledError`, caught at the load site so the detector stays `nil` with no error message. ## Why `_MERGIFY_TEST_NEW_FLAKY_DETECTION` was private and undocumented — no customer could enable client-side flaky detection. Moving the decision server-side lets customers opt in from the dashboard. ##⚠️ Rollout dependency Do not release to customers until the server returns `404` for repositories that have not opted in. Rollout is coordinated in MRGFY-7580. ## Also - Drops the defunct flag from the specs and the shared sandbox helper. The gem's own CI never set it, so no workflow change was needed. - Specs that set up a CI-like environment now stub the context endpoint (404) so flaky detection skips deterministically instead of hitting the network. - `docs/decisions.md` still mentions the flag; those are historical decision-log entries and were left untouched on purpose. ## Test plan - `bundle exec rubocop` — 38 files, no offenses ✓ - `bundle exec rspec` — 215 examples, 0 failures ✓ - `gem build rspec-mergify.gemspec` — builds ✓ - New: 404 → silent skip (unit + client level); empty baseline → silent skip. Part of MRGFY-7580. Mirrors Mergifyio/pytest-mergify#440. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Graft the standalone rspec-mergify repository's history -- rewritten so its files sit under clients/rspec-mergify/ at the monorepo layout (the lib/ gem, spec/, docs/decisions.md, Gemfile/Rakefile/gemspec and the rubocop/rspec/ ruby-version config) -- so git log and git blame on the gem trace back through its original commits. Repo-infra (.github, .mergify.yml, .claude) is dropped from the rewritten history and commits left empty by that rewrite are pruned, mirroring the pytest-mergify (#27) and TypeScript (#46) imports; that pruning is what takes the 64 upstream commits down to 37, since most of the dropped ones only ever bumped a pinned GitHub Action. Files are imported as-is, GPL LICENSE included; relicensing to the repo's Apache-2.0 and pinning the version are the next commit. Related to MRGFY-8639 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Change-Id: I6acb95ad581039e4ef0d29dfde7c9d0e2a4eb9d3
Merge Protections🔴 3 of 6 protections blocking · waiting on 👀 reviews and 🙋 you
🔴 👀 Review RequirementsWaiting for
This rule is failing.
🔴 Enforce conventional commitWaiting for
This rule is failing.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
🔴 🔎 ReviewsWaiting for
This rule is failing.
Show 3 satisfied protections🟢 🤖 Continuous Integration
🟢 📕 PR description
🟢 🚦 Auto-queueWhen all merge protections are satisfied, this pull request will be queued automatically. |
The gem arrived GPL-3.0-only, but every client here is Apache-2.0 under the single root LICENSE (#60), so drop the gem-local LICENSE and switch the gemspec. `gem build` then finds no LICENSE to package; the release workflow will stage the root one next to the gem, the way build-pytest-mergify-wheels.yml already does for maturin. version.rb computed VERSION by shelling out to `git describe --tags` at load time, which only ever gave the right answer inside a checkout of the standalone repo. Here it would read the monorepo's namespaced tags; in an installed gem it read whatever repository the user's suite happened to be running in, and reported that as the instrumentation-library version on every span. Pin it to the same 0.0.0 placeholder the other two clients carry and let the release workflow stamp the real value. The Rakefile's release:source_control_push / release:guard_clean overrides stay, against what the issue anticipated: they exist because GitHub Releases owns the tagging, not to serve the git-describe scheme, so they outlive it. Fixes MRGFY-8639 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Change-Id: Ib333e9b3e45717d3d67f47b23471fe458877c56c
kozlek
force-pushed
the
thomasberdy/mrgfy-8639-import-rspec-mergify-into-the-monorepo-with-git-history
branch
from
August 12, 2026 16:26
de5ffc7 to
2f3c1d4
Compare
kozlek
deleted the
thomasberdy/mrgfy-8639-import-rspec-mergify-into-the-monorepo-with-git-history
branch
August 14, 2026 11:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third and last client to move in, after pytest-mergify (#27) and the TypeScript workspace (#46). Same two-commit shape as both: a history graft, then the wiring the graft can't carry.
Import
git filter-reporewrites the standalone repo's history underclients/rspec-mergify/, thengit merge --allow-unrelated-historiesgrafts it on, sogit log --followandgit blamekeep working through the move. Repo-infra (.github,.mergify.yml,.claude) is dropped and the commits left empty by that are pruned — which is why 64 upstream commits arrive as 37, most of the dropped ones having only bumped a pinned GitHub Action.Relicense + version
Two things could not come across as-is:
version.rbcomputedVERSIONby shelling out togit describe --tagsat load time. That only ever worked inside a checkout of the standalone repo — here it would read the monorepo's namespaced tags, and in an installed gem it read whatever repository the user's suite happened to run in, reporting that as the instrumentation-library version on every span. Now the same0.0.0placeholder the other two clients carry, for the release workflow to stamp.One deviation from the issue: the Rakefile's
release:source_control_push/release:guard_cleanoverrides stay. They exist because GitHub Releases owns the tagging, not to serve the git-describe scheme, so they outlive it.Verification
bundle exec rspec→ 215 examples, 0 failures.bundle exec rubocop→ 38 files, no offenses.gem buildclean. Run on Ruby 3.4.4, matching the gem's.ruby-version.Nothing else in the repo changes behavior: no CI job runs this gem yet and it still releases from the standalone repo, both of which land in MRGFY-8640.