fix(browserstack-service): close orphaned cucumber hooks that inflate build duration (SDK-7167) - #120
Open
osho-20 wants to merge 2 commits into
Open
fix(browserstack-service): close orphaned cucumber hooks that inflate build duration (SDK-7167)#120osho-20 wants to merge 2 commits into
osho-20 wants to merge 2 commits into
Conversation
… build duration (SDK-7167)
A cucumber hook (typically AFTER_EACH) that emitted HookRunStarted but never
its HookRunFinished stayed open on the Test Observability backend until the
project's hook timeout (2h), inflating the build duration shown on the new
dashboard (customer saw 4h35m for a 2h42m build). Customer SDK debug logs
confirmed the drop is client-side: 525 hook starts vs 521 finishes triggered,
zero upload failures.
Three complementary fixes:
- Extend the teardown sweep (previously mocha-only, documented known gap) to
cucumber: hook meta is tagged kind/name/hookType/testRunId at start,
scenario meta is tagged in beforeScenario and stamped finished in
afterScenario, and sweepUnfinished now emits terminal HookRunFinished /
TestRunFinished for any started-but-unfinished cucumber entity before the
worker's event queue shuts down.
- Journal open hook runs like open test runs, so when the worker is killed
outright mid-hook (Ctrl-C / CI cancellation) the exit cleanup finalizes the
orphaned hook with a HookRunFinished (hook_run envelope) instead of only
finalizing the test run.
- Guard the cucumber hook 'after' path against a missing start record (skip
with a warning instead of emitting an unmatched finish / TypeError), and
reset in-flight step state at scenario start so an aborted step can no
longer silently drop every later AFTER_EACH hook's events.
Verified end-to-end on Automate: interrupting a run mid-After-hook with the
published 9.33.0 leaves the hook open (only the test run is finalized);
with this fix the exit cleanup finalizes both ("Finalized 2 orphaned
test/hook run(s)").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
🔴 SDK PR Review gate is red. Pending:
It turns green once the latest SDK PR Review Agent run reports GTG on the current head commit. A native reviewer approval is separately required by branch protection before merge. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is this about?
Fixes incorrect (inflated) build durations on the new Test Observability dashboard for WebdriverIO + Cucumber runs ([Aya] customer report: build showed 4h 35m on the new dashboard vs 2h 42m on the old one).
Root cause: some Cucumber
AFTER_EACHhooks emitHookRunStartedbut theirHookRunFinishedis never sent. The backend holds the hook open until the project's hook timeout (2h) and the build watchdog inflates the reported duration. Customer SDK debug logs prove the drop is client-side:hookEvents { started: 525, finished: 521 }with zero upload failures — 4 finish events were never created.Three complementary fixes:
kind/name/hookType/testRunIdat start, scenario meta is tagged inbeforeScenarioand stamped finished inafterScenario, andsweepUnfinished()emits a terminalHookRunFinished/TestRunFinishedfor any started-but-unfinished entity before the worker's event queue shuts down.HookRunFinished(hook_runenvelope). Previously only orphaned test runs were finalized.afterpath skips (with a warning) when no start was recorded instead of emitting an unmatched finish / throwing aTypeError, and in-flight step state is reset at scenario start so an aborted step can no longer silently drop every laterAFTER_EACHhook's events for the rest of the worker.End-to-end verification on Automate (minimal WDIO 9 + Cucumber project, run interrupted via SIGINT while the
Afterhook sleeps):Finalized 1 orphaned test run(s)→ repro build: https://automation.browserstack.com/builds/m4ohrcqiqwnflmfrino0snwsnlfmunul9d5pv5bvFinalized 2 orphaned test/hook run(s)— hook closed → build: https://automation.browserstack.com/builds/pyyrun48cfjll4xcfxo1fgornzrovs8xrklibwhyUnit tests: 9 new tests (cucumber sweep, hook journaling/finalization, after-path guard, step-state reset). Full vitest suite shows the identical 70 pre-existing environmental failures as clean
main— zero regressions. Build + eslint clean.Related Jira task/s
Release (mandatory for every PR — required for the
ready-for-reviewlabel)Version bump: (required — tick exactly one)
Release notes type: (optional)
Release notes (customer-facing): (optional but encouraged)
Release notes (internal): (required — engineer-facing; what actually changed / why)
sweepUnfinished()teardown safety net to Cucumber: hook/scenario meta is tagged withkind+ identity at start,afterScenariostampsfinishedAt, and the sweep emits terminalHookRunFinished/TestRunFinishedfor started-but-unfinished cucumber entities (SDK-7167 — orphanedAFTER_EACHstarts held hooks open until the 2h backend hook timeout, inflating build duration on the new dashboard).listener.hookStarted/hookFinishednow record/clear the open-runs journal, andfinalizeOrphanedRuns()emitsHookRunFinished(hook_runenvelope) for journaled hook entries — so a worker killed outright mid-hook gets its hook finalized by the exit cleanup, not just its test run.afterevents with no recorded start are skipped with a warning (no unmatched finish, noTypeError);_cucumberData.stepsis reset per scenario so a stuck in-flight step can't misclassify laterAFTER_EACHhooks as step-level and silently drop their events.Checklist
PR Validations
Run Tests: Comment RUN_TESTS to trigger sanity tests.
🤖 Generated with Claude Code