Skip to content

fix: return CollectionStats from collect_results, unify zero-output check (#95)#103

Merged
Muizzkolapo merged 2 commits intomainfrom
fix/collection-stats-zero-output-check
Apr 2, 2026
Merged

fix: return CollectionStats from collect_results, unify zero-output check (#95)#103
Muizzkolapo merged 2 commits intomainfrom
fix/collection-stats-zero-output-check

Conversation

@Muizzkolapo
Copy link
Copy Markdown
Owner

Summary

  • First action failing all items (e.g., invalid API key → 401) was incorrectly marked completed_with_failures (PARTIAL) instead of failed (ERROR) because the initial pipeline had no zero-output check
  • Root cause: collect_results() computed success/failure counts internally but discarded them — callers had to re-derive the information independently, and the initial pipeline didn't
  • Fix: collect_results() now returns (output, CollectionStats) — both pipelines use the same stats to make the same decision

Changes

  • result_collector.py: Added CollectionStats dataclass, changed return type to tuple[list[dict], CollectionStats]
  • pipeline.py: Uses stats.failed instead of re-scanning results list for FAILED items
  • initial_pipeline.py: Adds zero-output check using stats (was completely missing — this was the bug)
  • Tests: Updated ~20 call sites to unpack new return type

Root cause analysis

Two code paths both call collect_results():

  1. StandardStrategypipeline.py — had a zero-output check (PR feat: pre-flight validation and dependency-aware execution guards #67)
  2. InitialStrategyinitial_pipeline.pyhad no check at all

When the first action (initial strategy) failed all items with 401, it wrote empty output and returned successfully. The executor called _handle_run_success_resolve_completion_status found failed dispositions → set completed_with_failures. The circuit breaker didn't fire because completed_with_failures is not a failure status.

Mitigates #95 — downstream actions no longer execute after the first action fails entirely.

Test plan

  • ruff check . — clean
  • ruff format --check — clean
  • pytest — 4300 passed, 2 skipped, 0 failures
  • Manual: first action with invalid API key → status failed → circuit breaker skips all downstream → tally shows 0 OK | 0 PARTIAL | N SKIP | 1 ERROR

Muizzkolapo and others added 2 commits April 2, 2026 15:39
…heck

Root cause: the zero-output check (raise when 0/N items succeed) only
existed in pipeline.py. The initial pipeline (first action) had no
equivalent — so when all items failed with 401, it wrote empty output
and "completed" instead of failing. The circuit breaker never fired.

Fix: collect_results() now returns (output, CollectionStats) instead of
just output. Both pipeline.py and initial_pipeline.py use stats.failed
to make the same decision. No more ad-hoc re-scanning of results.

Changes:
- Added CollectionStats dataclass to result_collector.py
- collect_results() returns tuple[list[dict], CollectionStats]
- pipeline.py: uses stats.failed instead of re-scanning results list
- initial_pipeline.py: adds zero-output check using stats (was missing)
- Updated ~20 test call sites to unpack the new return type

Mitigates #95 — invalid API key on first action now properly fails
instead of silently completing with empty output.
@Muizzkolapo Muizzkolapo merged commit 3bc68af into main Apr 2, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant