Python: Enforce package coverage by lifecycle

[Eduard van Valkenburg (eavanvalkenburg)]

Motivation & Context

Python coverage enforcement used a manually maintained target allowlist, allowing many Beta packages to fall below the documented 85% threshold without failing CI. This makes package maturity and coverage policy drift apart as packages are added or promoted.

Description & Review Guide

• What are the major changes? Coverage enforcement now derives package policy from each distribution's Development Status classifier, aggregates Cobertura results by distribution directory, and requires 85% line coverage for non-exempt Beta and stable packages. Alpha packages remain report-only; DevUI and Lab remain explicit exemptions. The checker moved into .github/scripts, gained focused automation tests, and package tests were expanded where the new gate exposed coverage gaps.
• What is the impact of these changes? New and promoted Python packages automatically enter coverage enforcement, missing enforced packages fail clearly, and all currently enforced packages meet the threshold. Contributor guidance now documents the lifecycle policy and prefers extending existing test files for related behavior.
• What do you want reviewers to focus on? Please focus on lifecycle discovery, distribution-level coverage aggregation, and the explicit DevUI/Lab exemption behavior.

Related Issue

Fixes #7260

Contribution Checklist

• The code builds clean without any errors or warnings
• All unit tests pass, and I have added new tests where possible
• The PR follows the Contribution Guidelines
• This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
• This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

[github-actions]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/chatkit/agent_framework_chatkit
_converter.py	135	0	100%
packages/core/agent_framework/_workflows
_workflow.py	351	24	93%	61, 63, 68, 92, 97, 158, 194, 411–413, 415–416, 440, 474, 641, 893, 914, 962, 974, 980, 985, 1005–1007
_workflow_builder.py	207	13	93%	221, 498–502, 504, 607, 622, 651, 667, 675, 710
TOTAL	45112	4474	90%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
9328	33 💤	0 ❌	0 🔥	2m 30s ⏱️

[github-actions]

Automated Code Review

Reviewers: 4 | Confidence: 90% | Result: All clear

Reviewed: Correctness, Security Reliability, Test Coverage, Failure Modes

---

Automated review by eavanvalkenburg's agents

[Copilot]

Pull request overview

This PR updates the Python coverage gate to enforce the documented 85% line-coverage threshold based on each distribution’s lifecycle (Development Status classifier), rather than a manually maintained allowlist. It also updates CI wiring/docs and expands tests across multiple Python packages to meet the newly enforced policy.

Changes:

• Replaced the coverage allowlist gate with a lifecycle-driven checker (.github/scripts/python_check_coverage.py) that aggregates Cobertura coverage by python/packages/<dist>/... directory and enforces 85% for non-exempt Beta+ packages.
• Updated GitHub Actions workflows to run the new checker and added focused automation tests for the checker itself.
• Added/expanded unit tests across several Python packages (tools, redis, monty, hyperlight, durabletask, chatkit, bedrock) to close gaps exposed by the new enforcement.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
python/packages/tools/tests/test_shell_resolve.py	Adds more unit coverage for shell resolution (env override, platform defaults, helper functions).
python/packages/tools/tests/test_shell_killtree.py	New tests for process-tree termination helpers across psutil/stdlib paths.
python/packages/tools/tests/test_local_shell_tool.py	Expands LocalShellTool coverage (env handling, stateless runner behavior, Windows process group kwargs).
python/packages/tools/tests/test_docker_shell_tool.py	Adds broader DockerShellTool unit tests around mode behavior, policy denial, session/container lifecycle, and timeouts.
python/packages/redis/tests/test_context_provider_edges.py	Adds edge-case coverage for RedisContextProvider (schema, filtering, pagination, validation).
python/packages/monty/tests/monty/test_monty_codeact.py	Expands coverage for instruction builders, mount helpers, file capture, truncation behavior, and provider delegation.
python/packages/hyperlight/tests/hyperlight/test_hyperlight_codeact.py	Adds tests for normalizers/shape guards, instruction builders, and provider delegation.
python/packages/durabletask/tests/test_workflow_runner_context.py	New tests for CapturingRunnerContext behavior and reset semantics.
python/packages/durabletask/tests/test_workflow_orchestrator_helpers.py	New unit tests for orchestrator helper functions (task prep/routing, HITL serialization/dispatch, fan-in).
python/packages/durabletask/tests/test_workflow_dt_context.py	New tests for DurableTaskWorkflowContext adapter behavior.
python/packages/durabletask/tests/test_async_bridge.py	New tests for the durabletask async bridge loop/thread lifecycle and coroutine execution.
python/packages/chatkit/tests/test_converter.py	Expands ThreadItemConverter coverage across more ChatKit item variants (quoted text, tags, widgets, tool calls, dispatch).
python/packages/bedrock/tests/test_bedrock_client.py	Adds significant coverage for BedrockChatClient session creation, request prep, tool-result handling, and parsing helpers.
python/packages/azurefunctions/tests/test_workflow_af_context.py	New tests for Azure Functions workflow-context adapter and wrapper orchestration entrypoint.
python/packages/azurefunctions/tests/test_func_utils.py	Adds extra unit coverage for CapturingRunnerContext no-ops and request-info tracking.
python/DEV_SETUP.md	Documents the lifecycle-based 85% enforcement policy and exemptions.
python/.github/skills/python-testing/SKILL.md	Updates testing guidance to reflect lifecycle-based enforcement and recommends extending existing test files.
.github/workflows/python-test-coverage.yml	Wires coverage checking to the new script and updates the workflow’s path filters/step naming.
.github/workflows/python-check-coverage.py	Removes the old allowlist-based coverage gate script.
.github/workflows/github-automation-tests.yml	Adds a Python job step to run the new checker’s unit tests in CI.
.github/tests/test_python_check_coverage.py	Adds automation tests validating lifecycle policy loading, aggregation, and pass/fail behavior.
.github/scripts/python_check_coverage.py	Introduces the lifecycle-driven coverage enforcement implementation used by CI.

[Tao Chen (TaoChenOSU)]

Approving but the workflow builder changes need Evan Mattson (@moonbox3) to greenlight