Skip to content

CI observability: publish hosted semantic-lane timing and resource diagnostics #181

Description

@Konan69

Context

PR #180 records a local improvement from 975.62 seconds to 330.70 seconds wall time, but hosted runner timing and resource behavior remain unproven. The semantic runner already emits lane summaries and slow-test durations.

Outcome

Make hosted backend CI performance observable and regression-detectable without introducing cross-job artifacts, hidden routing, or test sampling.

Scope

  • Publish a credential-free GitHub step summary for every semantic lane.
  • Record collection count, setup time, test time, lane wall time, exit status, and coverage-combine time.
  • Distinguish PostgreSQL migration/setup cost from pytest execution cost.
  • Record runner CPU and memory observations at bounded intervals without environment dumps.
  • Define a reviewed regression threshold based on multiple successful hosted runs.
  • Rebalance only by semantic dependency ownership when evidence shows a persistent bottleneck.

Acceptance criteria

  • Exact-commit hosted timing is visible in the Backend / test job summary.
  • Metrics contain no URLs, credentials, tokens, environment dumps, or mutable database names.
  • Failed lanes remain failed; telemetry cannot compensate for tests or coverage.
  • No GitHub matrix, cross-job coverage fan-in, or timing-based test omission is introduced.
  • The summary reports all discovered modules and confirms exact-once assignment.
  • A regression threshold and escalation policy are documented from at least five successful runs.
  • Semantic lane changes require inventory tests and before/after evidence.

Non-goals

  • Arbitrary weighted sharding.
  • Test result caching.
  • Lowering coverage or skipping slow tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/opsOperations, pilot setup, onboarding, reportingenhancementNew feature or requestpriority/P2Later or explicitly deferredstatus/needs-scopeNeeds maintainer scoping before implementation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions