` | Query a specific test case |
| `teaforge mermaid validate --code ` | Validate Mermaid syntax |
-| `teaforge mermaid generate --source --function --code --output-dir ` | Persist and render a Mermaid diagram |
-| `teaforge coverage generate --path --output --function --diagram-dir ` | Generate coverage report with flowcharts |
+| `teaforge mermaid generate --source --function --diagram-type --code --output-dir ` | Validate and persist a typed Mermaid source file |
+| `teaforge coverage generate --path --output --function --sequence --diagram-dir --min-c0 --min-c1 ` | Generate coverage report with flowchart/sequence pages and optional file-level gates |
| `teaforge coverage generate --framework jest --path --output --function --diagram-dir ` | Generate a Jest / TypeScript coverage report |
## Procedure
@@ -83,28 +97,29 @@ Cover every execution path through the function:
3. Map every branch (`if/else`, `try/except`, early return) in the function.
4. Write a baseline test for the normal path with normal parameters.
5. Derive variant tests — change exactly one input or trigger exactly one different branch per test.
-6. Verify C1 coverage reaches 100% using `teaforge coverage generate`.
+6. Inspect the C1 result from `teaforge coverage generate` and add tests until all intended branches are covered. In CI, pass `--min-c0` and `--min-c1`; an unmet gate preserves the reports and exits with code `3`.
### Choosing the Framework
1. Use the default `pytest` flow for Python tests.
2. Use `--framework jest` for Node.js / TypeScript Jest tests.
3. Keep the framework choice consistent between PCL generation and coverage generation.
+4. Before automation, run `teaforge doctor --framework --path --json` and stop if `ready` is false.
### Generating Coverage Reports (End-to-End)
-Coverage reports require Mermaid flowcharts for selected business functions. The AI is responsible for **choosing which functions need flowcharts**, **writing the Mermaid code**, and **invoking the CLI**. Follow these steps:
+Coverage reports accept Mermaid flowcharts and sequence diagrams for selected business functions. The AI is responsible for choosing the useful diagram type, writing the Mermaid code, and invoking the CLI.
1. **Read the source file** under test to understand all function signatures and bodies.
-2. **Select business functions** that need flowcharts (see Flowchart Selection below).
-3. **Write Mermaid flowchart code** for each selected function (see Writing Mermaid Flowcharts below).
+2. **Select business functions** that need flowcharts, sequence diagrams, or both.
+3. **Write Mermaid code** for each selected function.
4. **Validate** each diagram: `teaforge mermaid validate --code ""`.
5. **Persist** each diagram: `teaforge mermaid generate --source --function --code "" --output-dir `.
6. **Generate the report**:
- pytest: `teaforge coverage generate --path --output --function --function --diagram-dir `
- jest: `teaforge coverage generate --framework jest --path --output --function --function --diagram-dir `
-The `--function` flag is repeatable — pass it once per function that has a flowchart.
+The `--function` flag adds flowchart pages and `--sequence` adds sequence-diagram pages. Both are repeatable and may target the same function.
### Jest-Specific Notes
@@ -112,6 +127,15 @@ The `--function` flag is repeatable — pass it once per function that has a flo
- The AI should still follow the same Japanese PCL principle: derive neighboring boundary / abnormal cases from one normal baseline.
- TeaForge currently expects Jest row data to be statically readable array literals.
- If the parser cannot prove the business source file under test, PCL generation may still fallback for display, but coverage generation will fail fast.
+- Prefer runtime evidence for older or dynamically written tests whose variables cannot be resolved statically. It requires an already-installed project-local Jest and intentionally never downloads one.
+- A completed Jest run with failing tests still writes the evidence report and returns exit code `2`. Do not treat it as a TeaForge generation error; either fix the test or deliberately pass `--allow-test-failures`.
+- Use `--runtime-timeout` to bound Jest execution. For pytest coverage in another virtual environment, pass `--python-executable `.
+
+### Sequence Diagram Selection
+
+Use a sequence diagram when the important fact is interaction order across callers, business logic, persistence, or external systems. Keep flowcharts for internal branches and exits. TeaForge expects the AI to provide `sequenceDiagram` Mermaid; assertion values alone are not sufficient to infer a trustworthy call trace.
+
+Generate it with `--diagram-type sequence`, then include it in coverage with `--sequence `.
### Flowchart Selection
diff --git a/src/teaforge/__init__.py b/src/teaforge/__init__.py
index 84b6217..ae3175a 100644
--- a/src/teaforge/__init__.py
+++ b/src/teaforge/__init__.py
@@ -1,4 +1,3 @@
__all__ = ["__version__"]
-__version__ = "0.1.0"
-
+__version__ = "0.2.0"
diff --git a/src/teaforge/angular/coverage.py b/src/teaforge/angular/coverage.py
index 392d9e4..6b98343 100644
--- a/src/teaforge/angular/coverage.py
+++ b/src/teaforge/angular/coverage.py
@@ -2,14 +2,11 @@
from __future__ import annotations
-import json
-import subprocess
-import tempfile
from pathlib import Path
from typing import TYPE_CHECKING
from teaforge.angular.parser import parse_angular_documents
-from teaforge.jest.coverage import _extract_snapshots, parse_jest_source_functions
+from teaforge.jest.coverage import _analyze_jest_coverage, parse_jest_source_functions
if TYPE_CHECKING:
from teaforge.coverage.analyzer import SourceCoverageSnapshot, SourceFunction
@@ -43,53 +40,16 @@ def parse_angular_source_functions(source_path: Path) -> list[SourceFunction]:
def analyze_angular_coverage(
test_path: Path,
source_paths: list[Path],
+ *,
+ timeout_seconds: int = 120,
) -> dict[Path, SourceCoverageSnapshot]:
"""Run Jest/Istanbul coverage for Angular page-level tests."""
- resolved_test_path = test_path.resolve()
- resolved_sources = [path.resolve() for path in source_paths]
-
- with tempfile.TemporaryDirectory(prefix="teaforge-angular-coverage-") as temp_dir:
- temp_root = Path(temp_dir)
- coverage_dir = temp_root / "coverage"
- coverage_dir.mkdir(parents=True, exist_ok=True)
- coverage_file = coverage_dir / "coverage-final.json"
-
- try:
- run_result = subprocess.run(
- [
- "npx",
- "jest",
- "--coverage",
- "--coverageReporters=json",
- "--coverageDirectory",
- str(coverage_dir),
- "--runInBand",
- str(resolved_test_path),
- ],
- capture_output=True,
- text=True,
- check=False,
- )
- except FileNotFoundError as exc:
- raise RuntimeError(
- "Angular coverage execution requires `npx` and Jest. Install Node.js and Jest before running coverage reports."
- ) from exc
-
- if run_result.returncode != 0:
- detail = (run_result.stderr or run_result.stdout).strip()
- raise RuntimeError(
- "Angular/Jest failed while collecting coverage data. "
- f"Exit code: {run_result.returncode}. {detail}"
- )
-
- if not coverage_file.exists():
- raise RuntimeError(
- "Angular/Jest did not emit coverage-final.json. Ensure Istanbul JSON output is available."
- )
-
- payload = json.loads(coverage_file.read_text(encoding="utf-8"))
-
- return _extract_snapshots(payload, resolved_sources)
+ return _analyze_jest_coverage(
+ test_path,
+ source_paths,
+ timeout_seconds=timeout_seconds,
+ operation="Angular/Jest coverage collection",
+ )
def _document_uses_test_file_as_source(document) -> bool:
diff --git a/src/teaforge/angular/parser.py b/src/teaforge/angular/parser.py
index 532a4d7..36c95de 100644
--- a/src/teaforge/angular/parser.py
+++ b/src/teaforge/angular/parser.py
@@ -8,8 +8,8 @@
from urllib.parse import parse_qsl, urlsplit
from teaforge.jest.parser import (
- JSImport,
JestTestBlock,
+ JSImport,
SubjectLocation,
_collect_test_files,
_extract_imports,
@@ -19,9 +19,10 @@
_split_top_level,
_stringify_value,
)
+from teaforge.pcl.assembly import build_input_rows, build_output_rows, merge_documents_by_subject
from teaforge.pcl.classification import classify_type
from teaforge.pcl.models import PCLDocument, PCLTestCase
-from teaforge.pcl.parser import _build_input_rows, _build_output_rows, _display_path, _merge_documents_by_subject
+from teaforge.pcl.parser import _display_path
def parse_angular_documents(test_path: Path) -> list[PCLDocument]:
@@ -34,13 +35,20 @@ def parse_angular_documents(test_path: Path) -> list[PCLDocument]:
for file_path in files:
content = file_path.read_text(encoding="utf-8")
imports = _extract_imports(file_path, content)
- for index, block in enumerate(_extract_test_blocks(content), start=1):
+ for index, block in enumerate(
+ _extract_test_blocks(
+ content,
+ suffix=file_path.suffix,
+ source_name=str(file_path),
+ ),
+ start=1,
+ ):
raw_documents.append(_build_document_for_block(file_path, block, imports, index))
if not raw_documents:
raise ValueError(f"No supported Angular tests found under: {test_path}")
- return _merge_documents_by_subject(raw_documents)
+ return merge_documents_by_subject(raw_documents)
def _build_document_for_block(
@@ -83,10 +91,11 @@ def _build_document_for_block(
navigation_outputs=navigation_outputs,
verification_mode="unit",
output_checks=output_checks,
+ test_full_name=block.full_name or block.title,
)
)
- document.input_rows = _build_input_rows(document.testcases)
- document.output_rows = _build_output_rows(document.testcases)
+ document.input_rows = build_input_rows(document.testcases)
+ document.output_rows = build_output_rows(document.testcases)
return document
diff --git a/src/teaforge/artifacts.py b/src/teaforge/artifacts.py
new file mode 100644
index 0000000..412e0b3
--- /dev/null
+++ b/src/teaforge/artifacts.py
@@ -0,0 +1,44 @@
+"""Persist generated artifacts without exposing partially written files."""
+
+from __future__ import annotations
+
+import json
+import os
+import tempfile
+from pathlib import Path
+from typing import Any
+
+
+def json_for_html_script(payload: Any, *, indent: int | None = 2) -> str:
+ """Serialize JSON without allowing values to terminate an HTML script element."""
+ serialized = json.dumps(payload, ensure_ascii=False, indent=indent)
+ return (
+ serialized.replace("&", "\\u0026")
+ .replace("<", "\\u003c")
+ .replace(">", "\\u003e")
+ .replace("\u2028", "\\u2028")
+ .replace("\u2029", "\\u2029")
+ )
+
+
+def write_text_atomic(path: Path, content: str, *, encoding: str = "utf-8") -> None:
+ """Write and fsync a sibling temporary file before atomically replacing the target."""
+ path.parent.mkdir(parents=True, exist_ok=True)
+ temporary_path: Path | None = None
+ try:
+ with tempfile.NamedTemporaryFile(
+ mode="w",
+ encoding=encoding,
+ dir=path.parent,
+ prefix=f".{path.name}.",
+ suffix=".tmp",
+ delete=False,
+ ) as handle:
+ temporary_path = Path(handle.name)
+ handle.write(content)
+ handle.flush()
+ os.fsync(handle.fileno())
+ os.replace(temporary_path, path)
+ finally:
+ if temporary_path is not None and temporary_path.exists():
+ temporary_path.unlink()
diff --git a/src/teaforge/cli.py b/src/teaforge/cli.py
index 3496f0f..4365161 100644
--- a/src/teaforge/cli.py
+++ b/src/teaforge/cli.py
@@ -2,14 +2,17 @@
from __future__ import annotations
+import json
from pathlib import Path
+from typing import Any
-import click
import typer
from typer.main import get_command as get_typer_command
+from teaforge import __version__
from teaforge.coverage.mermaid import save_mermaid_diagram, validate_mermaid_with_renderer
-from teaforge.coverage.service import generate_coverage_reports
+from teaforge.coverage.service import CoverageThresholdError, generate_coverage_reports
+from teaforge.doctor import format_doctor_report, inspect_capabilities
from teaforge.pcl.pdf import export_pdf_from_html
from teaforge.pcl.service import generate_pcl, get_testcase_description, load_pcl_document
@@ -22,6 +25,61 @@
app.add_typer(coverage_app, name="coverage")
+def _error_code(error: Exception) -> str:
+ """Map public failures to stable codes without leaking implementation types."""
+ explicit_code = getattr(error, "code", None)
+ if isinstance(explicit_code, str) and explicit_code:
+ return explicit_code
+ if isinstance(error, FileNotFoundError):
+ return "file-not-found"
+ if isinstance(error, ValueError):
+ return "invalid-input"
+ return "runtime-error"
+
+
+def _emit_cli_error(error: Exception, *, json_output: bool = False) -> None:
+ """Keep human errors concise and JSON-mode failures machine readable."""
+ if json_output:
+ typer.echo(
+ json.dumps(
+ {
+ "schema_version": 1,
+ "teaforge_version": __version__,
+ "ready": False,
+ "error": {
+ "code": _error_code(error),
+ "message": str(error),
+ },
+ },
+ ensure_ascii=False,
+ indent=2,
+ )
+ )
+ return
+ typer.echo(f"Error: {error}", err=True)
+
+
+def _version_callback(value: bool) -> bool:
+ """Print the installed TeaForge version before command dispatch."""
+ if value:
+ typer.echo(__version__)
+ raise typer.Exit()
+ return value
+
+
+@app.callback()
+def root_callback(
+ version: bool = typer.Option(
+ False,
+ "--version",
+ callback=_version_callback,
+ is_eager=True,
+ help="show the TeaForge version and exit",
+ ),
+) -> None:
+ """Generate auditable test specifications and coverage reports."""
+
+
@app.command("help")
def help_command(
command_path: list[str] | None = typer.Argument(
@@ -33,11 +91,77 @@ def help_command(
try:
command, info_name = _resolve_help_target(command_path or [])
except ValueError as exc:
- typer.echo(f"Error: {exc}", err=True)
+ _emit_cli_error(exc)
raise typer.Exit(code=1) from exc
- context = click.Context(command, info_name=info_name)
- typer.echo(command.get_help(context))
+ with command.make_context(info_name, [], resilient_parsing=True) as context:
+ help_text = command.get_help(context).rstrip()
+ option_names = [
+ option
+ for parameter in command.params
+ for option in getattr(parameter, "opts", ())
+ if option.startswith("--")
+ ]
+ typer.echo(help_text)
+ if option_names:
+ typer.echo("\nOption names: " + ", ".join(dict.fromkeys(option_names)))
+
+
+@app.command("doctor")
+def doctor_command(
+ framework: str = typer.Option(
+ "pytest",
+ "--framework",
+ help="workflow to inspect: pytest, jest, angular, or playwright",
+ ),
+ path: Path | None = typer.Option(
+ None,
+ "--path",
+ help="optional target test file or directory for project discovery",
+ ),
+ require_mermaid: bool = typer.Option(
+ False,
+ "--require-mermaid",
+ help="treat a missing Mermaid renderer as an error",
+ ),
+ require_pdf: bool = typer.Option(
+ False,
+ "--require-pdf",
+ help="treat an unavailable PDF renderer as an error",
+ ),
+ json_output: bool = typer.Option(
+ False,
+ "--json",
+ help="emit a machine-readable readiness report",
+ ),
+ timeout: int = typer.Option(
+ 10,
+ "--timeout",
+ min=1,
+ help="maximum seconds for each version/capability check",
+ ),
+ python_executable: Path | None = typer.Option(
+ None,
+ "--python-executable",
+ help="Python interpreter from the target project environment (pytest only)",
+ ),
+) -> None:
+ """Check dependencies and target-project discovery without running tests."""
+ try:
+ report = inspect_capabilities(
+ framework=framework,
+ test_path=path,
+ require_mermaid=require_mermaid,
+ require_pdf=require_pdf,
+ timeout_seconds=timeout,
+ python_executable=python_executable,
+ )
+ except (OSError, RuntimeError, ValueError) as exc:
+ _emit_cli_error(exc, json_output=json_output)
+ raise typer.Exit(code=1) from exc
+ typer.echo(report.to_json() if json_output else format_doctor_report(report))
+ if not report.ready:
+ raise typer.Exit(code=1)
@pcl_app.command("generate")
@@ -59,6 +183,22 @@ def pcl_generate(
"--template",
help="optional custom html template path",
),
+ evidence_mode: str = typer.Option(
+ "static",
+ "--evidence-mode",
+ help="Jest assertion evidence: static, runtime, or auto fallback",
+ ),
+ runtime_timeout: int = typer.Option(
+ 120,
+ "--runtime-timeout",
+ min=1,
+ help="maximum seconds for Jest runtime evidence collection",
+ ),
+ allow_test_failures: bool = typer.Option(
+ False,
+ "--allow-test-failures",
+ help="return exit code 0 after writing a report that contains failing Jest tests",
+ ),
) -> None:
"""Generate PCL HTML and JSON from pytest, Jest, Angular, or Playwright tests."""
try:
@@ -68,13 +208,26 @@ def pcl_generate(
json_output=json_output,
template_path=template,
framework=framework,
+ evidence_mode=evidence_mode,
+ runtime_timeout=runtime_timeout,
)
- except (FileNotFoundError, ValueError) as exc:
- typer.echo(f"Error: {exc}", err=True)
+ except (FileNotFoundError, RuntimeError, ValueError) as exc:
+ _emit_cli_error(exc)
raise typer.Exit(code=1) from exc
for html_path, json_path in generated_files:
typer.echo(f"Generated HTML: {html_path}")
typer.echo(f"Generated JSON: {json_path}")
+ runtime_failed = any(
+ load_pcl_document(json_path).runtime_tests_passed is False
+ for _, json_path in generated_files
+ )
+ if runtime_failed and not allow_test_failures:
+ typer.echo(
+ "Warning: Jest tests failed. Reports were generated with failure evidence; "
+ "TeaForge is returning exit code 2. Use --allow-test-failures to accept this result.",
+ err=True,
+ )
+ raise typer.Exit(code=2)
@app.command("export")
@@ -86,7 +239,7 @@ def export_command(
try:
export_pdf_from_html(path, output)
except (RuntimeError, FileNotFoundError) as exc:
- typer.echo(f"Error: {exc}", err=True)
+ _emit_cli_error(exc)
raise typer.Exit(code=1) from exc
typer.echo(f"Generated PDF: {output}")
@@ -101,7 +254,7 @@ def get_command(
document = load_pcl_document(path)
description = get_testcase_description(document, testcase)
except (FileNotFoundError, ValueError) as exc:
- typer.echo(f"Error: {exc}", err=True)
+ _emit_cli_error(exc)
raise typer.Exit(code=1) from exc
typer.echo(description)
@@ -118,7 +271,7 @@ def mermaid_validate(
try:
validate_mermaid_with_renderer(code)
except (RuntimeError, ValueError) as exc:
- typer.echo(f"Error: {exc}", err=True)
+ _emit_cli_error(exc)
raise typer.Exit(code=1) from exc
typer.echo("Mermaid syntax looks valid.")
@@ -141,6 +294,11 @@ def mermaid_generate(
"--output-dir",
help="directory used to store .mmd files",
),
+ diagram_type: str = typer.Option(
+ "auto",
+ "--diagram-type",
+ help="diagram type: auto, flowchart, or sequence",
+ ),
) -> None:
"""Validate Mermaid text and save it as a per-business-function .mmd file."""
try:
@@ -149,9 +307,10 @@ def mermaid_generate(
source_path=source,
function_name=function,
output_dir=output_dir,
+ diagram_type=diagram_type,
)
except (FileNotFoundError, RuntimeError, ValueError) as exc:
- typer.echo(f"Error: {exc}", err=True)
+ _emit_cli_error(exc)
raise typer.Exit(code=1) from exc
typer.echo(f"Generated Mermaid file: {mmd_path}")
@@ -170,6 +329,11 @@ def coverage_generate(
"--function",
help="repeat to include a business function flowchart page; only key business functions need diagrams",
),
+ sequence: list[str] | None = typer.Option(
+ None,
+ "--sequence",
+ help="repeat to include a business function sequence-diagram page",
+ ),
diagram_dir: Path = typer.Option(
Path("output/files"),
"--diagram-dir",
@@ -180,6 +344,31 @@ def coverage_generate(
"--template",
help="optional custom coverage report html template path",
),
+ runtime_timeout: int = typer.Option(
+ 120,
+ "--runtime-timeout",
+ min=1,
+ help="maximum seconds for test coverage collection",
+ ),
+ python_executable: Path | None = typer.Option(
+ None,
+ "--python-executable",
+ help="Python interpreter from the target project environment (pytest coverage only)",
+ ),
+ min_c0: float | None = typer.Option(
+ None,
+ "--min-c0",
+ min=0.0,
+ max=100.0,
+ help="minimum required file-level C0 percentage; writes reports and exits 3 on failure",
+ ),
+ min_c1: float | None = typer.Option(
+ None,
+ "--min-c1",
+ min=0.0,
+ max=100.0,
+ help="minimum required file-level C1 percentage; writes reports and exits 3 on failure",
+ ),
) -> None:
"""Generate a file-level C0/C1 report and optional flowchart pages for selected business functions."""
try:
@@ -188,26 +377,37 @@ def coverage_generate(
html_output=output,
diagram_dir=diagram_dir,
diagram_functions=function,
+ sequence_functions=sequence,
template_path=template,
framework=framework,
+ runtime_timeout=runtime_timeout,
+ python_executable=python_executable,
+ min_c0=min_c0,
+ min_c1=min_c1,
)
+ except CoverageThresholdError as exc:
+ for html_path in exc.outputs:
+ typer.echo(f"Generated coverage HTML: {html_path}")
+ typer.echo(f"Warning: {exc}", err=True)
+ raise typer.Exit(code=3) from exc
except (FileNotFoundError, RuntimeError, ValueError) as exc:
- typer.echo(f"Error: {exc}", err=True)
+ _emit_cli_error(exc)
raise typer.Exit(code=1) from exc
for html_path in generated_files:
typer.echo(f"Generated coverage HTML: {html_path}")
-def _resolve_help_target(command_path: list[str]) -> tuple[click.Command, str]:
+def _resolve_help_target(command_path: list[str]) -> tuple[Any, str]:
"""Resolve a nested command path into the Click command that owns its help text."""
- command: click.Command = get_typer_command(app)
+ command = get_typer_command(app)
resolved_path: list[str] = []
for part in command_path:
- if not isinstance(command, click.Group):
+ commands = getattr(command, "commands", None)
+ if not isinstance(commands, dict):
current_path = " ".join(resolved_path) or "teaforge"
raise ValueError(f"Command path '{current_path}' does not accept subcommands.")
- next_command = command.commands.get(part)
+ next_command = commands.get(part)
if next_command is None:
requested_path = " ".join([*resolved_path, part])
raise ValueError(f"Unknown command path: {requested_path}")
diff --git a/src/teaforge/coverage/analyzer.py b/src/teaforge/coverage/analyzer.py
index fbddd55..5b254e2 100644
--- a/src/teaforge/coverage/analyzer.py
+++ b/src/teaforge/coverage/analyzer.py
@@ -5,13 +5,16 @@
import ast
import importlib.util
import json
-import subprocess
import sys
import tempfile
from dataclasses import dataclass
from pathlib import Path
from teaforge.pcl.backends import normalize_pcl_framework, parse_documents_for_framework
+from teaforge.process import WorkflowDeadline, bounded_process_detail, run_process
+
+CoverageBranch = tuple[int, int] | tuple[int, int, str]
+PYTEST_PROJECT_MARKERS = ("pyproject.toml", "pytest.ini", "setup.cfg", "tox.ini")
@dataclass(slots=True)
@@ -30,8 +33,8 @@ class SourceCoverageSnapshot:
c1_total: int
executed_lines: set[int]
missing_lines: set[int]
- executed_branches: set[tuple[int, int]]
- missing_branches: set[tuple[int, int]]
+ executed_branches: set[CoverageBranch]
+ missing_branches: set[CoverageBranch]
def discover_source_files(test_path: Path, framework: str = "pytest") -> list[Path]:
@@ -114,64 +117,93 @@ def analyze_coverage(
test_path: Path,
source_paths: list[Path],
framework: str = "pytest",
+ *,
+ timeout_seconds: int = 120,
+ python_executable: Path | None = None,
) -> dict[Path, SourceCoverageSnapshot]:
"""Run the selected test framework and map coverage payloads back to each source file."""
normalized = normalize_pcl_framework(framework)
if normalized == "jest":
from teaforge.jest.coverage import analyze_jest_coverage
- return analyze_jest_coverage(test_path, source_paths)
+ return analyze_jest_coverage(
+ test_path, source_paths, timeout_seconds=timeout_seconds
+ )
if normalized == "angular":
from teaforge.angular.coverage import analyze_angular_coverage
- return analyze_angular_coverage(test_path, source_paths)
+ return analyze_angular_coverage(
+ test_path, source_paths, timeout_seconds=timeout_seconds
+ )
if normalized == "playwright":
raise ValueError("Coverage reports do not support framework: playwright")
- return _analyze_pytest_coverage(test_path, source_paths)
+ selected_python = (python_executable or Path(sys.executable)).expanduser().absolute()
+ if not selected_python.is_file():
+ raise FileNotFoundError(
+ f"Python executable does not exist: {selected_python}"
+ )
+ return _analyze_pytest_coverage(
+ test_path,
+ source_paths,
+ timeout_seconds=timeout_seconds,
+ python_executable=selected_python,
+ )
def _analyze_pytest_coverage(
pytest_path: Path,
source_paths: list[Path],
+ *,
+ timeout_seconds: int,
+ python_executable: Path,
) -> dict[Path, SourceCoverageSnapshot]:
"""Run pytest through coverage.py and map the JSON payload back to each source file."""
_ensure_runtime_dependencies()
resolved_pytest_path = pytest_path.resolve()
resolved_sources = [path.resolve() for path in source_paths]
+ source_roots = ",".join(
+ dict.fromkeys(str(path.parent) for path in resolved_sources)
+ )
+ project_root = discover_pytest_project_root(resolved_pytest_path)
+ deadline = WorkflowDeadline.start(
+ "pytest coverage workflow",
+ timeout_seconds,
+ )
with tempfile.TemporaryDirectory(prefix="teaforge-coverage-") as temp_dir:
temp_root = Path(temp_dir)
data_file = temp_root / ".coverage"
json_output = temp_root / "coverage.json"
- run_result = subprocess.run(
+ run_result = run_process(
[
- sys.executable,
+ str(python_executable),
"-m",
"coverage",
"run",
"--branch",
+ f"--source={source_roots}",
f"--data-file={data_file}",
"-m",
"pytest",
str(resolved_pytest_path),
],
- capture_output=True,
- text=True,
- check=False,
+ operation="pytest coverage collection",
+ timeout_seconds=deadline.remaining_seconds(),
+ cwd=project_root,
)
if run_result.returncode != 0:
- detail = (run_result.stderr or run_result.stdout).strip()
+ detail = bounded_process_detail(run_result.stderr, run_result.stdout)
raise RuntimeError(
"pytest failed while collecting coverage data. "
f"Exit code: {run_result.returncode}. {detail}"
)
- include_arg = ",".join(str(path) for path in resolved_sources)
- json_result = subprocess.run(
+ include_arg = _coverage_include_argument(resolved_sources, project_root)
+ json_result = run_process(
[
- sys.executable,
+ str(python_executable),
"-m",
"coverage",
"json",
@@ -180,19 +212,49 @@ def _analyze_pytest_coverage(
str(json_output),
f"--include={include_arg}",
],
- capture_output=True,
- text=True,
- check=False,
+ operation="coverage.py JSON export",
+ timeout_seconds=deadline.remaining_seconds(),
+ cwd=project_root,
)
if json_result.returncode != 0:
- detail = (json_result.stderr or json_result.stdout).strip()
+ detail = bounded_process_detail(json_result.stderr, json_result.stdout)
raise RuntimeError(
"coverage.py failed to export JSON data. "
f"Exit code: {json_result.returncode}. {detail}"
)
payload = json.loads(json_output.read_text(encoding="utf-8"))
- return _extract_snapshots(payload, resolved_sources)
+ return _extract_snapshots(
+ payload,
+ resolved_sources,
+ project_root=project_root,
+ )
+
+
+def discover_pytest_project_root(test_path: Path) -> Path:
+ """Find the nearest pytest configuration root, with a local-path fallback."""
+ resolved = test_path.expanduser().resolve()
+ start = resolved if resolved.is_dir() else resolved.parent
+ for candidate in (start, *start.parents):
+ if any((candidate / marker).is_file() for marker in PYTEST_PROJECT_MARKERS):
+ return candidate
+ return start
+
+
+def _coverage_include_argument(source_paths: list[Path], project_root: Path) -> str:
+ """Match coverage.py data whether it stores absolute or project-relative names."""
+ patterns: list[str] = []
+ for source_path in source_paths:
+ absolute = str(source_path)
+ if absolute not in patterns:
+ patterns.append(absolute)
+ try:
+ relative = str(source_path.relative_to(project_root))
+ except ValueError:
+ continue
+ if relative not in patterns:
+ patterns.append(relative)
+ return ",".join(patterns)
def _ensure_runtime_dependencies() -> None:
@@ -210,11 +272,14 @@ def _ensure_runtime_dependencies() -> None:
def _extract_snapshots(
payload: dict,
source_paths: list[Path],
+ *,
+ project_root: Path | None = None,
) -> dict[Path, SourceCoverageSnapshot]:
"""Convert coverage.json payloads into per-source snapshot dataclasses."""
- # coverage.py may emit relative paths; resolving them once keeps matching deterministic.
+ # coverage.py commonly emits paths relative to the target project's execution root.
+ resolution_root = (project_root or Path.cwd()).resolve()
file_entries = {
- Path(file_name).resolve(): file_payload
+ _resolve_coverage_entry_path(file_name, resolution_root): file_payload
for file_name, file_payload in payload.get("files", {}).items()
}
@@ -244,6 +309,13 @@ def _extract_snapshots(
return snapshots
+def _resolve_coverage_entry_path(file_name: str, project_root: Path) -> Path:
+ candidate = Path(file_name)
+ if not candidate.is_absolute():
+ candidate = project_root / candidate
+ return candidate.resolve()
+
+
def _match_file_payload(file_entries: dict[Path, dict], source_path: Path) -> dict:
"""Find the exact coverage payload for one resolved source file."""
if source_path in file_entries:
diff --git a/src/teaforge/coverage/mermaid.py b/src/teaforge/coverage/mermaid.py
index 999647c..4f80991 100644
--- a/src/teaforge/coverage/mermaid.py
+++ b/src/teaforge/coverage/mermaid.py
@@ -2,12 +2,14 @@
from __future__ import annotations
+import os
import shutil
-import subprocess
import tempfile
from pathlib import Path
+from teaforge.artifacts import write_text_atomic
from teaforge.naming import folder_name
+from teaforge.process import bounded_process_detail, run_process
MERMAID_EXAMPLE = """flowchart TD
A[Start] --> B{Input valid?}
@@ -28,6 +30,9 @@
"mindmap",
"timeline",
)
+DIAGRAM_TYPES = ("flowchart", "sequence")
+DEFAULT_RENDER_TIMEOUT_SECONDS = 120
+PUPPETEER_CONFIG_ENV = "TEAFORGE_MERMAID_PUPPETEER_CONFIG"
def mmdc_install_message() -> str:
@@ -68,14 +73,56 @@ def validate_mermaid_with_renderer(code: str) -> str:
return normalized
-def diagram_file_stem(source_path: Path, function_name: str) -> str:
+def detect_mermaid_diagram_type(code: str) -> str:
+ """Classify the Mermaid source into a report-supported diagram type."""
+ normalized = validate_mermaid(code)
+ first_line = next(line.strip() for line in normalized.splitlines() if line.strip())
+ if first_line.startswith("sequenceDiagram"):
+ return "sequence"
+ if first_line.startswith(("flowchart ", "graph ")):
+ return "flowchart"
+ raise ValueError(
+ "TeaForge reports currently support Mermaid flowcharts and sequence diagrams only."
+ )
+
+
+def normalize_diagram_type(diagram_type: str, code: str | None = None) -> str:
+ normalized = diagram_type.strip().lower()
+ if normalized == "auto":
+ if code is None:
+ raise ValueError("Mermaid code is required when diagram type is auto.")
+ return detect_mermaid_diagram_type(code)
+ if normalized not in DIAGRAM_TYPES:
+ raise ValueError(
+ f"Unsupported diagram type: {diagram_type}. Supported values: auto, "
+ + ", ".join(DIAGRAM_TYPES)
+ )
+ if code is not None and detect_mermaid_diagram_type(code) != normalized:
+ raise ValueError(
+ f"Mermaid source does not match --diagram-type {normalized}."
+ )
+ return normalized
+
+
+def diagram_file_stem(
+ source_path: Path,
+ function_name: str,
+ diagram_type: str = "flowchart",
+) -> str:
"""Keep Mermaid source and SVG names aligned with coverage report naming."""
- return f"{folder_name(source_path.stem)}_{folder_name(function_name)}_coverage_report"
+ base = f"{folder_name(source_path.stem)}_{folder_name(function_name)}"
+ suffix = "sequence_diagram" if diagram_type == "sequence" else "coverage_report"
+ return f"{base}_{suffix}"
-def diagram_output_paths(diagram_dir: Path, source_path: Path, function_name: str) -> tuple[Path, Path]:
+def diagram_output_paths(
+ diagram_dir: Path,
+ source_path: Path,
+ function_name: str,
+ diagram_type: str = "flowchart",
+) -> tuple[Path, Path]:
"""Return the expected Mermaid source path and rendered SVG path for one function."""
- stem = diagram_file_stem(source_path, function_name)
+ stem = diagram_file_stem(source_path, function_name, diagram_type)
return diagram_dir / f"{stem}.mmd", diagram_dir / f"{stem}.svg"
@@ -85,12 +132,18 @@ def save_mermaid_diagram(
source_path: Path,
function_name: str,
output_dir: Path,
+ diagram_type: str = "auto",
) -> Path:
"""Persist validated Mermaid text as the source-of-truth diagram artifact."""
+ if not source_path.is_file():
+ raise FileNotFoundError(f"Tested source file does not exist: {source_path}")
+ resolved_type = normalize_diagram_type(diagram_type, code)
normalized = validate_mermaid_with_renderer(code)
output_dir.mkdir(parents=True, exist_ok=True)
- mmd_path, _ = diagram_output_paths(output_dir, source_path, function_name)
- mmd_path.write_text(normalized, encoding="utf-8")
+ mmd_path, _ = diagram_output_paths(
+ output_dir, source_path, function_name, resolved_type
+ )
+ write_text_atomic(mmd_path, normalized)
return mmd_path
@@ -107,14 +160,23 @@ def _run_mmdc(mmd_path: Path, svg_path: Path) -> None:
raise RuntimeError(mmdc_install_message())
svg_path.parent.mkdir(parents=True, exist_ok=True)
- result = subprocess.run(
- [renderer, "-i", str(mmd_path), "-o", str(svg_path), "-b", "transparent"],
- capture_output=True,
- text=True,
- check=False,
+ command = [renderer, "-i", str(mmd_path), "-o", str(svg_path), "-b", "transparent"]
+ puppeteer_config = os.environ.get(PUPPETEER_CONFIG_ENV)
+ if puppeteer_config:
+ config_path = Path(puppeteer_config).expanduser()
+ if not config_path.is_file():
+ raise FileNotFoundError(
+ f"Mermaid Puppeteer config does not exist: {config_path}"
+ )
+ command.extend(["--puppeteerConfigFile", str(config_path)])
+ result = run_process(
+ command,
+ operation=f"Mermaid rendering for {mmd_path.name}",
+ timeout_seconds=DEFAULT_RENDER_TIMEOUT_SECONDS,
)
+
if result.returncode != 0:
- detail = (result.stderr or result.stdout).strip()
+ detail = bounded_process_detail(result.stderr, result.stdout)
if "Parse error" in detail:
raise ValueError(
"Invalid Mermaid syntax. Please fix the diagram and try again.\n"
diff --git a/src/teaforge/coverage/models.py b/src/teaforge/coverage/models.py
index 3756355..8577ec8 100644
--- a/src/teaforge/coverage/models.py
+++ b/src/teaforge/coverage/models.py
@@ -6,6 +6,8 @@
from datetime import UTC, datetime
from typing import Any
+COVERAGE_SCHEMA_VERSION = 2
+
@dataclass(slots=True)
class CoverageMetric:
@@ -13,6 +15,19 @@ class CoverageMetric:
total: int
percent: float
missing: int = 0
+ applicable: bool = field(init=False)
+
+ def __post_init__(self) -> None:
+ self.applicable = self.total > 0
+
+
+@dataclass(slots=True)
+class DiagramArtifact:
+ kind: str
+ page_number: int
+ mermaid_path: str
+ svg_path: str
+ svg_data_uri: str
@dataclass(slots=True)
@@ -29,6 +44,7 @@ class FunctionCoverage:
mermaid_path: str = ""
svg_path: str = ""
svg_data_uri: str = ""
+ diagrams: list[DiagramArtifact] = field(default_factory=list)
@dataclass(slots=True)
@@ -41,6 +57,10 @@ class CoverageDocument:
page_count: int
c0: CoverageMetric
c1: CoverageMetric
+ evidence_source: str
+ c0_definition: str
+ c1_definition: str
+ schema_version: int = COVERAGE_SCHEMA_VERSION
requested_functions: list[str] = field(default_factory=list)
functions: list[FunctionCoverage] = field(default_factory=list)
@@ -55,17 +75,29 @@ def create(
c1: CoverageMetric,
requested_functions: list[str],
functions: list[FunctionCoverage],
+ evidence_source: str,
+ c0_definition: str,
+ c1_definition: str,
) -> "CoverageDocument":
- """Create a report document with a stable page count for summary and flowchart pages."""
+ """Create a report document with a stable page count for summary and diagram pages."""
+ diagram_pages = sum(len(function.diagrams) for function in functions)
+ if diagram_pages == 0:
+ diagram_pages = sum(
+ 1 for function in functions if function.page_number is not None
+ )
return cls(
title="Coverage Report",
file=file,
source_path=source_path,
test_path=test_path,
generated_at=datetime.now(UTC).isoformat(),
- page_count=max(1, 1 + sum(1 for function in functions if function.page_number is not None)),
+ page_count=max(1, 1 + diagram_pages),
c0=c0,
c1=c1,
+ evidence_source=evidence_source,
+ c0_definition=c0_definition,
+ c1_definition=c1_definition,
+ schema_version=COVERAGE_SCHEMA_VERSION,
requested_functions=requested_functions,
functions=functions,
)
diff --git a/src/teaforge/coverage/render.py b/src/teaforge/coverage/render.py
index eec4e7d..5561d8c 100644
--- a/src/teaforge/coverage/render.py
+++ b/src/teaforge/coverage/render.py
@@ -2,17 +2,21 @@
from __future__ import annotations
-import json
+from importlib import resources
from pathlib import Path
from jinja2 import Environment, FileSystemLoader, select_autoescape
+from teaforge.artifacts import json_for_html_script
+
from .models import CoverageDocument
def default_template_path() -> Path:
"""Return the built-in coverage report template path."""
- return Path(__file__).resolve().parents[3] / "templates" / "coverage_report.html"
+ return Path(
+ str(resources.files("teaforge.templates").joinpath("coverage_report.html"))
+ )
def render_coverage_html(
@@ -20,14 +24,25 @@ def render_coverage_html(
template_path: Path | None = None,
) -> str:
"""Render one coverage document and embed the JSON payload for downstream tooling."""
- template_file = template_path or default_template_path()
- env = Environment(
- loader=FileSystemLoader(str(template_file.parent)),
- autoescape=select_autoescape(["html", "xml"]),
- )
- template = env.get_template(template_file.name)
+ if template_path is None:
+ env = Environment(autoescape=select_autoescape(["html", "xml"]))
+ template = env.from_string(
+ resources.files("teaforge.templates")
+ .joinpath("coverage_report.html")
+ .read_text(encoding="utf-8")
+ )
+ else:
+ if not template_path.is_file():
+ raise FileNotFoundError(
+ f"Coverage template does not exist: {template_path}"
+ )
+ env = Environment(
+ loader=FileSystemLoader(str(template_path.parent)),
+ autoescape=select_autoescape(["html", "xml"]),
+ )
+ template = env.get_template(template_path.name)
payload = document.to_dict()
- embedded_json = json.dumps(payload, ensure_ascii=False, indent=2)
+ embedded_json = json_for_html_script(payload)
return template.render(
document=payload,
embedded_json=embedded_json,
diff --git a/src/teaforge/coverage/service.py b/src/teaforge/coverage/service.py
index f6e99ee..07bb772 100644
--- a/src/teaforge/coverage/service.py
+++ b/src/teaforge/coverage/service.py
@@ -4,9 +4,11 @@
import base64
import re
+from dataclasses import dataclass
from pathlib import Path
-from teaforge.naming import folder_name
+from teaforge.artifacts import write_text_atomic
+from teaforge.naming import folder_name, source_folder_names
from .analyzer import (
SourceCoverageSnapshot,
@@ -16,77 +18,192 @@
parse_source_functions,
)
from .mermaid import diagram_output_paths, render_mermaid_svg
-from .models import CoverageDocument, CoverageMetric, FunctionCoverage
+from .models import CoverageDocument, CoverageMetric, DiagramArtifact, FunctionCoverage
from .render import render_coverage_html
+@dataclass(slots=True, frozen=True)
+class CoverageThresholdViolation:
+ output_path: Path
+ metric: str
+ actual: float
+ required: float
+
+ def message(self) -> str:
+ return (
+ f"{self.output_path}: {self.metric} {self.actual:.1f}% is below "
+ f"the required {self.required:.1f}%"
+ )
+
+
+class CoverageThresholdError(RuntimeError):
+ """Signal that reports were written but one or more coverage gates failed."""
+
+ def __init__(
+ self,
+ outputs: list[Path],
+ violations: list[CoverageThresholdViolation],
+ ) -> None:
+ self.outputs = tuple(outputs)
+ self.violations = tuple(violations)
+ details = "\n".join(f"- {item.message()}" for item in violations)
+ super().__init__(f"Coverage thresholds were not met:\n{details}")
+
+
def generate_coverage_reports(
*,
pytest_path: Path,
html_output: Path,
diagram_dir: Path,
diagram_functions: list[str] | None = None,
+ sequence_functions: list[str] | None = None,
template_path: Path | None = None,
framework: str = "pytest",
+ runtime_timeout: int = 120,
+ python_executable: Path | None = None,
+ min_c0: float | None = None,
+ min_c1: float | None = None,
) -> list[Path]:
"""Generate one report per source file, plus optional function flowchart pages."""
if not pytest_path.exists():
raise FileNotFoundError(f"Input path does not exist: {pytest_path}")
+ _validate_threshold("min_c0", min_c0)
+ _validate_threshold("min_c1", min_c1)
source_paths = discover_source_files(pytest_path, framework=framework)
function_index = {
source_path: parse_source_functions(source_path, framework=framework) for source_path in source_paths
}
_ensure_functions(function_index)
- selected_names = _normalize_requested_functions(diagram_functions)
- resolved_selected_names = _resolve_selected_names(function_index, selected_names)
- selected_index = _build_selected_index(function_index, resolved_selected_names)
- _ensure_diagrams(selected_index, diagram_dir)
-
- snapshots = analyze_coverage(pytest_path, source_paths, framework=framework)
+ flowchart_names = _resolve_selected_names(
+ function_index, _normalize_requested_functions(diagram_functions)
+ )
+ sequence_names = _resolve_selected_names(
+ function_index, _normalize_requested_functions(sequence_functions)
+ )
+ flowchart_index = _build_selected_index(function_index, flowchart_names)
+ sequence_index = _build_selected_index(function_index, sequence_names)
+ _ensure_diagrams(flowchart_index, diagram_dir, "flowchart")
+ _ensure_diagrams(sequence_index, diagram_dir, "sequence")
+
+ snapshots = analyze_coverage(
+ pytest_path,
+ source_paths,
+ framework=framework,
+ timeout_seconds=runtime_timeout,
+ python_executable=python_executable,
+ )
html_output.parent.mkdir(parents=True, exist_ok=True)
- outputs: list[Path] = []
+ prepared_outputs: list[tuple[Path, str, CoverageDocument]] = []
multiple_outputs = len(source_paths) > 1
+ source_folders = source_folder_names(source_paths)
for source_path in source_paths:
snapshot = snapshots[source_path.resolve()]
functions = function_index[source_path]
- selected_for_source = {function.name for function in selected_index[source_path]}
+ flowcharts_for_source = {
+ function.name for function in flowchart_index[source_path]
+ }
+ sequences_for_source = {
+ function.name for function in sequence_index[source_path]
+ }
page_number = 2
sections: list[FunctionCoverage] = []
for function in functions:
- include_diagram = function.name in selected_for_source
+ diagram_types: list[str] = []
+ if function.name in flowcharts_for_source:
+ diagram_types.append("flowchart")
+ if function.name in sequences_for_source:
+ diagram_types.append("sequence")
+ page_numbers = list(range(page_number, page_number + len(diagram_types)))
sections.append(
_build_function_section(
source_path=source_path,
function=function,
snapshot=snapshot,
diagram_dir=diagram_dir,
- page_number=page_number if include_diagram else None,
- include_diagram=include_diagram,
+ diagram_types=diagram_types,
+ page_numbers=page_numbers,
)
)
- if include_diagram:
- page_number += 1
+ page_number += len(diagram_types)
document = CoverageDocument.create(
file=source_path.name,
source_path=str(source_path),
test_path=str(pytest_path.resolve()),
c0=_build_metric(snapshot.c0_covered, snapshot.c0_total),
c1=_build_metric(snapshot.c1_covered, snapshot.c1_total),
- requested_functions=[function.name for function in selected_index[source_path]],
+ requested_functions=sorted(flowcharts_for_source | sequences_for_source),
functions=sections,
+ evidence_source=_coverage_evidence_source(framework),
+ c0_definition="covered statement lines / measurable statement lines",
+ c1_definition="executed branch destinations / measurable branch destinations",
)
- html_path = _resolve_output_path(source_path, html_output, multiple_outputs)
- html_path.parent.mkdir(parents=True, exist_ok=True)
- html_path.write_text(
- render_coverage_html(document, template_path=template_path),
- encoding="utf-8",
+ html_path = _resolve_output_path(
+ source_path,
+ html_output,
+ multiple_outputs,
+ source_folders[source_path.resolve()],
)
+ prepared_outputs.append(
+ (
+ html_path,
+ render_coverage_html(document, template_path=template_path),
+ document,
+ )
+ )
+
+ outputs: list[Path] = []
+ for html_path, html_content, _document in prepared_outputs:
+ write_text_atomic(html_path, html_content)
outputs.append(html_path)
+ violations = find_coverage_threshold_violations(
+ [(path, document) for path, _html, document in prepared_outputs],
+ min_c0=min_c0,
+ min_c1=min_c1,
+ )
+ if violations:
+ raise CoverageThresholdError(outputs, violations)
return outputs
+def find_coverage_threshold_violations(
+ reports: list[tuple[Path, CoverageDocument]],
+ *,
+ min_c0: float | None,
+ min_c1: float | None,
+) -> list[CoverageThresholdViolation]:
+ """Evaluate file-level coverage metrics for deterministic CI gating."""
+ violations: list[CoverageThresholdViolation] = []
+ for output_path, document in reports:
+ for metric_name, metric, required in (
+ ("C0", document.c0, min_c0),
+ ("C1", document.c1, min_c1),
+ ):
+ if required is not None and metric.applicable and metric.percent < required:
+ violations.append(
+ CoverageThresholdViolation(
+ output_path=output_path,
+ metric=metric_name,
+ actual=metric.percent,
+ required=required,
+ )
+ )
+ return violations
+
+
+def _validate_threshold(name: str, value: float | None) -> None:
+ if value is not None and not 0 <= value <= 100:
+ raise ValueError(f"{name} must be between 0 and 100: {value}")
+
+
+def _coverage_evidence_source(framework: str) -> str:
+ normalized = framework.strip().lower()
+ if normalized == "pytest":
+ return "coverage.py JSON"
+ return "Jest/Istanbul coverage-final.json"
+
+
def _ensure_functions(function_index: dict[Path, list[SourceFunction]]) -> None:
"""Fail early when a resolved source file contains no measurable functions."""
empty_sources = [str(source_path) for source_path, functions in function_index.items() if not functions]
@@ -160,7 +277,11 @@ def _build_selected_index(
}
-def _ensure_diagrams(function_index: dict[Path, list[SourceFunction]], diagram_dir: Path) -> None:
+def _ensure_diagrams(
+ function_index: dict[Path, list[SourceFunction]],
+ diagram_dir: Path,
+ diagram_type: str,
+) -> None:
"""Require Mermaid source files for every function selected for diagram pages."""
if not any(functions for functions in function_index.values()):
return
@@ -169,7 +290,9 @@ def _ensure_diagrams(function_index: dict[Path, list[SourceFunction]], diagram_d
for source_path, functions in function_index.items():
missing_functions = []
for function in functions:
- mmd_path, _ = _resolve_diagram_paths(diagram_dir, source_path, function.name)
+ mmd_path, _ = _resolve_diagram_paths(
+ diagram_dir, source_path, function.name, diagram_type
+ )
if not mmd_path.exists():
missing_functions.append(function.name)
if missing_functions:
@@ -178,17 +301,29 @@ def _ensure_diagrams(function_index: dict[Path, list[SourceFunction]], diagram_d
if not missing:
return
- lines = ["Missing Mermaid flowcharts for the coverage report:"]
+ diagram_label = "flowcharts" if diagram_type == "flowchart" else "sequence diagrams"
+ lines = [f"Missing Mermaid {diagram_label} for the coverage report:"]
for source_path, function_names in missing.items():
lines.append(f"- {source_path}: {', '.join(function_names)}")
first_source, first_functions = next(iter(missing.items()))
lines.append("")
lines.append("Generate only the required business-function diagrams before creating the report. Example:")
+ example_code = (
+ "sequenceDiagram\\n participant Client\\n participant Target\\n"
+ " Client->>Target: Call business function\\n"
+ " Target-->>Client: Return result"
+ if diagram_type == "sequence"
+ else "flowchart TD\\n A[Start] --> B{Valid?}\\n"
+ " B -- Yes --> C[Execute business logic]\\n"
+ " B -- No --> D[Return validation error]\\n"
+ " C --> E[Return result]"
+ )
lines.append(
"teaforge mermaid generate "
f"--source {first_source} "
f"--function {first_functions[0]} "
- '--code "flowchart TD\\n A[Start] --> B{Valid?}\\n B -- Yes --> C[Execute business logic]\\n B -- No --> D[Return validation error]\\n C --> E[Return result]" '
+ f"--diagram-type {diagram_type} "
+ f'--code "{example_code}" '
f"--output-dir {diagram_dir}"
)
raise ValueError("\n".join(lines))
@@ -200,8 +335,8 @@ def _build_function_section(
function: SourceFunction,
snapshot: SourceCoverageSnapshot,
diagram_dir: Path,
- page_number: int | None,
- include_diagram: bool,
+ diagram_types: list[str],
+ page_numbers: list[int],
) -> FunctionCoverage:
"""Build the per-function coverage section rendered in the final HTML report."""
# Function-level metrics are derived from the source line span already parsed from AST.
@@ -219,38 +354,54 @@ def _build_function_section(
branch for branch in snapshot.missing_branches if _branch_in_function(branch, function)
}
- mermaid_path = ""
- svg_path = ""
- svg_data_uri = ""
- if include_diagram:
- mermaid_file, svg_file = _resolve_diagram_paths(diagram_dir, source_path, function.name)
+ diagrams: list[DiagramArtifact] = []
+ for diagram_type, diagram_page_number in zip(
+ diagram_types, page_numbers, strict=True
+ ):
+ mermaid_file, svg_file = _resolve_diagram_paths(
+ diagram_dir, source_path, function.name, diagram_type
+ )
render_mermaid_svg(mermaid_file, svg_file)
- mermaid_path = str(mermaid_file)
- svg_path = str(svg_file)
- svg_data_uri = _load_svg_data_uri(svg_file)
+ diagrams.append(
+ DiagramArtifact(
+ kind=diagram_type,
+ page_number=diagram_page_number,
+ mermaid_path=str(mermaid_file),
+ svg_path=str(svg_file),
+ svg_data_uri=_load_svg_data_uri(svg_file),
+ )
+ )
+
+ primary_diagram = diagrams[0] if diagrams else None
return FunctionCoverage(
name=function.name,
lineno=function.lineno,
end_lineno=function.end_lineno,
- page_number=page_number,
+ page_number=primary_diagram.page_number if primary_diagram else None,
c0=_build_metric(covered_lines, len(relevant_lines)),
c1=_build_metric(len(executed_branches), len(executed_branches | missing_branches)),
missing_lines=missing_lines,
missing_branches=[_format_branch(branch) for branch in sorted(missing_branches)],
- diagram_included=include_diagram,
- mermaid_path=mermaid_path,
- svg_path=svg_path,
- svg_data_uri=svg_data_uri,
+ diagram_included=bool(diagrams),
+ mermaid_path=primary_diagram.mermaid_path if primary_diagram else "",
+ svg_path=primary_diagram.svg_path if primary_diagram else "",
+ svg_data_uri=primary_diagram.svg_data_uri if primary_diagram else "",
+ diagrams=diagrams,
)
-def _resolve_output_path(source_path: Path, base_html: Path, multiple_outputs: bool) -> Path:
+def _resolve_output_path(
+ source_path: Path,
+ base_html: Path,
+ multiple_outputs: bool,
+ output_folder: str,
+) -> Path:
"""Choose either the user path or a per-source report path for multi-file runs."""
if not multiple_outputs:
return base_html
- output_dir = base_html.parent / folder_name(source_path.stem)
+ output_dir = base_html.parent / output_folder
output_dir.mkdir(parents=True, exist_ok=True)
return output_dir / f"{folder_name(source_path.stem)}_coverage_report{base_html.suffix}"
@@ -263,16 +414,20 @@ def _build_metric(covered: int, total: int) -> CoverageMetric:
return CoverageMetric(covered=covered, total=total, percent=percent, missing=missing)
-def _branch_in_function(branch: tuple[int, int], function: SourceFunction) -> bool:
+def _branch_in_function(
+ branch: tuple[int, int] | tuple[int, int, str],
+ function: SourceFunction,
+) -> bool:
"""Check whether a branch record starts inside the function line range."""
start_line = branch[0]
return function.lineno <= start_line <= function.end_lineno
-def _format_branch(branch: tuple[int, int]) -> str:
+def _format_branch(branch: tuple[int, int] | tuple[int, int, str]) -> str:
"""Render a branch tuple into a readable report label."""
- start, end = branch
- return f"{start} -> {'exit' if end < 0 else end}"
+ start, end = branch[:2]
+ identity = f" [{branch[2]}]" if len(branch) == 3 else ""
+ return f"{start} -> {'exit' if end < 0 else end}{identity}"
def _load_svg_data_uri(svg_path: Path) -> str:
@@ -283,14 +438,23 @@ def _load_svg_data_uri(svg_path: Path) -> str:
return f"data:image/svg+xml;base64,{encoded}"
-def _resolve_diagram_paths(diagram_dir: Path, source_path: Path, function_name: str) -> tuple[Path, Path]:
+def _resolve_diagram_paths(
+ diagram_dir: Path,
+ source_path: Path,
+ function_name: str,
+ diagram_type: str = "flowchart",
+) -> tuple[Path, Path]:
"""Resolve a Mermaid file path, allowing short-name fallback for class methods."""
- exact_paths = diagram_output_paths(diagram_dir, source_path, function_name)
+ exact_paths = diagram_output_paths(
+ diagram_dir, source_path, function_name, diagram_type
+ )
if exact_paths[0].exists() or "." not in function_name:
return exact_paths
short_name = function_name.rsplit(".", 1)[1]
- short_paths = diagram_output_paths(diagram_dir, source_path, short_name)
+ short_paths = diagram_output_paths(
+ diagram_dir, source_path, short_name, diagram_type
+ )
if short_paths[0].exists():
return short_paths
return exact_paths
diff --git a/src/teaforge/discovery.py b/src/teaforge/discovery.py
new file mode 100644
index 0000000..19c39c9
--- /dev/null
+++ b/src/teaforge/discovery.py
@@ -0,0 +1,56 @@
+"""Discover target-project files without descending into generated dependency trees."""
+
+from __future__ import annotations
+
+import os
+from collections.abc import Callable, Collection
+from pathlib import Path
+
+DEFAULT_EXCLUDED_DIRECTORY_NAMES = frozenset(
+ {
+ ".git",
+ ".hg",
+ ".mypy_cache",
+ ".nox",
+ ".pytest_cache",
+ ".ruff_cache",
+ ".svn",
+ ".tox",
+ ".venv",
+ "__pycache__",
+ "build",
+ "dist",
+ "node_modules",
+ "venv",
+ }
+)
+
+
+def discover_files(
+ path: Path,
+ *,
+ is_candidate: Callable[[Path], bool],
+ excluded_directory_names: Collection[str] = DEFAULT_EXCLUDED_DIRECTORY_NAMES,
+) -> list[Path]:
+ """Return deterministic candidates while pruning known generated directories."""
+ if path.is_file():
+ return [path] if is_candidate(path) else []
+ if not path.is_dir():
+ return []
+
+ excluded = frozenset(excluded_directory_names)
+ files: list[Path] = []
+ for current_root, directory_names, file_names in os.walk(
+ path,
+ topdown=True,
+ followlinks=False,
+ ):
+ directory_names[:] = sorted(
+ name for name in directory_names if name not in excluded
+ )
+ root = Path(current_root)
+ for file_name in sorted(file_names):
+ candidate = root / file_name
+ if is_candidate(candidate):
+ files.append(candidate)
+ return files
diff --git a/src/teaforge/doctor.py b/src/teaforge/doctor.py
new file mode 100644
index 0000000..36393e1
--- /dev/null
+++ b/src/teaforge/doctor.py
@@ -0,0 +1,354 @@
+"""Inspect whether TeaForge can execute a requested workflow before generation."""
+
+from __future__ import annotations
+
+import importlib.util
+import io
+import json
+import shutil
+import sys
+from contextlib import redirect_stderr, redirect_stdout
+from dataclasses import asdict, dataclass
+from importlib import resources
+from pathlib import Path
+
+from teaforge import __version__
+from teaforge.jest.project import JestProject
+from teaforge.pcl.backends import normalize_pcl_framework
+from teaforge.process import ProcessTimeoutError, bounded_process_detail, run_process
+
+
+@dataclass(slots=True, frozen=True)
+class CapabilityCheck:
+ name: str
+ status: str
+ detail: str
+ hint: str = ""
+
+
+@dataclass(slots=True)
+class DoctorReport:
+ framework: str
+ checks: list[CapabilityCheck]
+
+ @property
+ def ready(self) -> bool:
+ return all(check.status != "error" for check in self.checks)
+
+ def to_dict(self) -> dict:
+ return {
+ "schema_version": 1,
+ "teaforge_version": __version__,
+ "framework": self.framework,
+ "ready": self.ready,
+ "checks": [asdict(check) for check in self.checks],
+ }
+
+ def to_json(self) -> str:
+ return json.dumps(self.to_dict(), ensure_ascii=False, indent=2)
+
+
+def inspect_capabilities(
+ *,
+ framework: str,
+ test_path: Path | None = None,
+ require_mermaid: bool = False,
+ require_pdf: bool = False,
+ timeout_seconds: int = 10,
+ python_executable: Path | None = None,
+) -> DoctorReport:
+ """Return a non-mutating readiness report for one TeaForge workflow."""
+ normalized = normalize_pcl_framework(framework)
+ if python_executable is not None and normalized != "pytest":
+ raise ValueError("--python-executable is supported only for the pytest workflow.")
+ checks = [
+ CapabilityCheck(
+ name="teaforge",
+ status="ok",
+ detail=f"TeaForge {__version__} on Python {sys.version.split()[0]}",
+ )
+ ]
+ checks.extend(_resource_checks())
+ checks.append(_javascript_syntax_check())
+
+ if test_path is not None:
+ resolved_test_path = test_path.expanduser().resolve()
+ if resolved_test_path.exists():
+ checks.append(
+ CapabilityCheck("test-path", "ok", str(resolved_test_path))
+ )
+ else:
+ checks.append(
+ CapabilityCheck(
+ "test-path",
+ "error",
+ f"Path does not exist: {resolved_test_path}",
+ "Move/copy the tests into place or pass the correct --path.",
+ )
+ )
+ else:
+ resolved_test_path = None
+ checks.append(
+ CapabilityCheck(
+ "test-path",
+ "warning",
+ "No target test path was supplied; project discovery was skipped.",
+ "Pass --path to verify a real project.",
+ )
+ )
+
+ if normalized == "pytest":
+ checks.extend(
+ _python_test_checks(
+ python_executable=python_executable,
+ timeout_seconds=timeout_seconds,
+ )
+ )
+ elif normalized in {"jest", "angular"}:
+ checks.extend(
+ _jest_checks(resolved_test_path, timeout_seconds=timeout_seconds)
+ )
+
+ checks.append(
+ _command_check(
+ "mermaid",
+ "mmdc",
+ required=require_mermaid,
+ install_hint="Install Mermaid CLI: npm install -g @mermaid-js/mermaid-cli",
+ timeout_seconds=timeout_seconds,
+ )
+ )
+ checks.append(_pdf_check(required=require_pdf))
+ return DoctorReport(framework=normalized, checks=checks)
+
+
+def format_doctor_report(report: DoctorReport) -> str:
+ """Render a concise terminal view while preserving hints for agent callers."""
+ labels = {"ok": "OK", "warning": "WARN", "error": "ERROR"}
+ lines = [f"TeaForge doctor ({report.framework})"]
+ for check in report.checks:
+ lines.append(f"[{labels[check.status]}] {check.name}: {check.detail}")
+ if check.hint:
+ lines.append(f" Hint: {check.hint}")
+ lines.append(f"Ready: {'yes' if report.ready else 'no'}")
+ return "\n".join(lines)
+
+
+def _resource_checks() -> list[CapabilityCheck]:
+ required_resources = (
+ ("pcl-template", "teaforge.templates", "pcl.html"),
+ ("coverage-template", "teaforge.templates", "coverage_report.html"),
+ ("jest-listener", "teaforge.jest.assets", "runtime-listener.cjs"),
+ )
+ checks: list[CapabilityCheck] = []
+ for name, package, filename in required_resources:
+ present = resources.files(package).joinpath(filename).is_file()
+ checks.append(
+ CapabilityCheck(
+ name,
+ "ok" if present else "error",
+ f"Packaged resource {'found' if present else 'missing'}: {filename}",
+ "Reinstall TeaForge from a valid wheel." if not present else "",
+ )
+ )
+ return checks
+
+
+def _javascript_syntax_check() -> CapabilityCheck:
+ """Verify that the packaged JS/TS grammars can build structural evidence."""
+ try:
+ from teaforge.javascript.syntax import extract_test_cases
+
+ cases = extract_test_cases(
+ "test('probe', () => expect(1).toBe(1));",
+ suffix=".js",
+ source_name="doctor-probe.js",
+ )
+ if len(cases) != 1 or cases[0].title != "probe":
+ raise RuntimeError("the parser returned unexpected probe evidence")
+ except Exception as exc:
+ return CapabilityCheck(
+ "javascript-syntax",
+ "error",
+ f"Tree-sitter JavaScript/TypeScript parser is unavailable: {exc}",
+ "Reinstall TeaForge from a valid wheel with its base dependencies.",
+ )
+ return CapabilityCheck(
+ "javascript-syntax",
+ "ok",
+ "Tree-sitter JavaScript/TypeScript grammars parsed the probe source.",
+ )
+
+
+def _python_test_checks(
+ *,
+ python_executable: Path | None,
+ timeout_seconds: int,
+) -> list[CapabilityCheck]:
+ selected_python = (
+ python_executable or Path(sys.executable)
+ ).expanduser().absolute()
+ if not selected_python.is_file():
+ return [
+ CapabilityCheck(
+ "python-executable",
+ "error",
+ f"Python executable does not exist: {selected_python}",
+ "Pass the Python launcher inside the target project's virtual environment.",
+ )
+ ]
+ checks: list[CapabilityCheck] = []
+ for module_name, install_name in (("pytest", "pytest"), ("coverage", "coverage")):
+ try:
+ result = run_process(
+ [
+ str(selected_python),
+ "-c",
+ (
+ f"import {module_name}; "
+ f"print(getattr({module_name}, '__version__', 'available'))"
+ ),
+ ],
+ operation=f"{module_name} capability check",
+ timeout_seconds=timeout_seconds,
+ )
+ present = result.returncode == 0
+ version = result.stdout.strip() if present else ""
+ failure = bounded_process_detail(result.stderr, result.stdout)
+ except (OSError, ProcessTimeoutError) as exc:
+ present = False
+ version = ""
+ failure = str(exc)
+ checks.append(
+ CapabilityCheck(
+ module_name,
+ "ok" if present else "error",
+ (
+ f"{module_name} {version or 'available'} via {selected_python}"
+ if present
+ else f"{module_name} is unavailable via {selected_python}: {failure}"
+ ),
+ (
+ f"Install it in the target environment: {selected_python} -m pip install {install_name}"
+ if not present
+ else ""
+ ),
+ )
+ )
+ return checks
+
+
+def _jest_checks(
+ test_path: Path | None,
+ *,
+ timeout_seconds: int,
+) -> list[CapabilityCheck]:
+ checks = [
+ _command_check(
+ "node",
+ "node",
+ required=True,
+ install_hint="Install a supported Node.js release.",
+ timeout_seconds=timeout_seconds,
+ )
+ ]
+ if test_path is None or not test_path.exists():
+ return checks
+ try:
+ project = JestProject.discover(test_path)
+ version_result = project.run(
+ ["--version"],
+ timeout_seconds=timeout_seconds,
+ operation="Jest version check",
+ )
+ if version_result.returncode != 0:
+ detail = bounded_process_detail(
+ version_result.stderr, version_result.stdout
+ )
+ raise RuntimeError(f"Jest --version failed: {detail}")
+ version = version_result.stdout.strip() or "unknown version"
+ checks.append(
+ CapabilityCheck(
+ "jest",
+ "ok",
+ f"{version} at {project.executable} (root: {project.root})",
+ )
+ )
+ checks.append(
+ CapabilityCheck(
+ "jest-tests",
+ "ok",
+ f"Discovered {len(project.test_files)} exact test file(s).",
+ )
+ )
+ except RuntimeError as exc:
+ checks.append(
+ CapabilityCheck(
+ "jest",
+ "error",
+ str(exc),
+ "Run the target project's package-manager install command first.",
+ )
+ )
+ return checks
+
+
+def _command_check(
+ name: str,
+ executable: str,
+ *,
+ required: bool,
+ install_hint: str,
+ timeout_seconds: int,
+) -> CapabilityCheck:
+ path = shutil.which(executable)
+ if path is None:
+ return CapabilityCheck(
+ name,
+ "error" if required else "warning",
+ f"Executable not found on PATH: {executable}",
+ install_hint,
+ )
+ try:
+ result = run_process(
+ [path, "--version"],
+ operation=f"{name} version check",
+ timeout_seconds=timeout_seconds,
+ )
+ except (OSError, ProcessTimeoutError) as exc:
+ return CapabilityCheck(
+ name,
+ "error" if required else "warning",
+ f"Could not execute {path}: {exc}",
+ install_hint,
+ )
+ detail = (result.stdout or result.stderr).strip().splitlines()
+ version = detail[0] if detail else "version unavailable"
+ status = "ok" if result.returncode == 0 else ("error" if required else "warning")
+ return CapabilityCheck(name, status, f"{version} at {path}", install_hint if status != "ok" else "")
+
+
+def _pdf_check(*, required: bool) -> CapabilityCheck:
+ if importlib.util.find_spec("weasyprint") is None:
+ return CapabilityCheck(
+ "pdf",
+ "error" if required else "warning",
+ "WeasyPrint is not installed.",
+ "Install TeaForge with the PDF extra: pip install 'teaforge[pdf]'",
+ )
+ try:
+ diagnostics = io.StringIO()
+ with redirect_stdout(diagnostics), redirect_stderr(diagnostics):
+ from weasyprint import HTML
+
+ payload = HTML(string="TeaForge").write_pdf()
+ if not payload.startswith(b"%PDF-"):
+ raise RuntimeError("WeasyPrint returned an invalid PDF payload.")
+ except (OSError, RuntimeError) as exc:
+ return CapabilityCheck(
+ "pdf",
+ "error" if required else "warning",
+ f"WeasyPrint is installed but not operational: {exc}",
+ "Install the required Pango/Cairo system libraries.",
+ )
+ return CapabilityCheck("pdf", "ok", "WeasyPrint produced a valid PDF payload.")
diff --git a/src/teaforge/javascript/__init__.py b/src/teaforge/javascript/__init__.py
new file mode 100644
index 0000000..e65da5e
--- /dev/null
+++ b/src/teaforge/javascript/__init__.py
@@ -0,0 +1,27 @@
+"""Structural JavaScript/TypeScript evidence used by framework adapters."""
+
+from .syntax import (
+ JavaScriptCall,
+ JavaScriptFunction,
+ JavaScriptImport,
+ JavaScriptSyntaxError,
+ JavaScriptTestCase,
+ extract_imports,
+ extract_test_cases,
+ parse_calls,
+ parse_source_functions,
+ source_defines_symbol,
+)
+
+__all__ = [
+ "JavaScriptCall",
+ "JavaScriptFunction",
+ "JavaScriptImport",
+ "JavaScriptSyntaxError",
+ "JavaScriptTestCase",
+ "extract_imports",
+ "extract_test_cases",
+ "parse_calls",
+ "parse_source_functions",
+ "source_defines_symbol",
+]
diff --git a/src/teaforge/javascript/syntax.py b/src/teaforge/javascript/syntax.py
new file mode 100644
index 0000000..3d85ef3
--- /dev/null
+++ b/src/teaforge/javascript/syntax.py
@@ -0,0 +1,947 @@
+"""Build structural Static Evidence without executing or modifying target projects."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from functools import lru_cache
+from typing import Iterator
+
+import tree_sitter_javascript
+import tree_sitter_typescript
+from tree_sitter import Language, Node, Parser, Tree
+
+JAVASCRIPT_SUFFIXES = {".js", ".jsx", ".mjs", ".cjs"}
+TYPESCRIPT_SUFFIXES = {".ts", ".mts", ".cts"}
+TSX_SUFFIXES = {".tsx"}
+SUPPORTED_SUFFIXES = JAVASCRIPT_SUFFIXES | TYPESCRIPT_SUFFIXES | TSX_SUFFIXES
+TEST_ROOTS = {"test", "it"}
+DESCRIBE_ROOTS = {"describe"}
+
+
+class JavaScriptSyntaxError(ValueError):
+ """Report a source location that could not be represented as safe Static Evidence."""
+
+ code = "javascript-syntax-error"
+
+
+@dataclass(slots=True, frozen=True)
+class JavaScriptImport:
+ module: str
+ local_name: str
+ imported_name: str
+
+
+@dataclass(slots=True, frozen=True)
+class JavaScriptTestCase:
+ title: str
+ full_name: str
+ body: str
+ context: dict[str, object]
+ start_byte: int
+
+
+@dataclass(slots=True, frozen=True)
+class JavaScriptCall:
+ callee: str
+ argument_text: str
+ arguments: tuple[str, ...]
+ start_byte: int
+
+
+@dataclass(slots=True, frozen=True)
+class JavaScriptFunction:
+ name: str
+ start_line: int
+ end_line: int
+
+
+@dataclass(slots=True, frozen=True)
+class _ParsedSource:
+ source: bytes
+ tree: Tree
+ suffix: str
+ source_name: str
+
+ def text(self, node: Node) -> str:
+ return self.source[node.start_byte : node.end_byte].decode(
+ "utf-8", errors="replace"
+ )
+
+
+@dataclass(slots=True, frozen=True)
+class _Callback:
+ node: Node
+ body: Node
+ parameter_names: tuple[str, ...]
+
+
+@dataclass(slots=True, frozen=True)
+class _Invocation:
+ title_node: Node
+ callback: _Callback
+ rows_node: Node | None
+
+
+_UNRESOLVED = object()
+
+
+def extract_imports(
+ source: str,
+ *,
+ suffix: str,
+ source_name: str = "",
+) -> list[JavaScriptImport]:
+ """Return ESM and statically provable CommonJS bindings in source order."""
+ parsed = _parse(source, suffix=suffix, source_name=source_name)
+ imports: list[JavaScriptImport] = []
+ for statement in parsed.tree.root_node.named_children:
+ if statement.type == "import_statement":
+ imports.extend(_esm_imports(parsed, statement))
+ elif statement.type in {"lexical_declaration", "variable_declaration"}:
+ imports.extend(_commonjs_imports(parsed, statement))
+ return imports
+
+
+def extract_test_cases(
+ source: str,
+ *,
+ suffix: str,
+ source_name: str = "",
+) -> list[JavaScriptTestCase]:
+ """Return literal Jest test cases with lexical describe identity and each rows."""
+ parsed = _parse(source, suffix=suffix, source_name=source_name)
+ cases: list[JavaScriptTestCase] = []
+ _visit_test_scope(
+ parsed,
+ parsed.tree.root_node,
+ suite_titles=(),
+ inherited_context={},
+ output=cases,
+ )
+ cases.sort(key=lambda case: case.start_byte)
+ return cases
+
+
+def parse_calls(
+ source: str,
+ *,
+ suffix: str,
+ source_name: str = "",
+) -> list[JavaScriptCall]:
+ """Return structurally parsed call expressions in lexical source order."""
+ parsed = _parse(source, suffix=suffix, source_name=source_name)
+ calls: list[JavaScriptCall] = []
+ for node in _walk(parsed.tree.root_node):
+ if node.type != "call_expression":
+ continue
+ function = node.child_by_field_name("function")
+ arguments_node = node.child_by_field_name("arguments")
+ if function is None or arguments_node is None:
+ continue
+ callee = _callee_path(parsed, function)
+ if not callee:
+ continue
+ arguments = tuple(
+ parsed.text(argument) for argument in arguments_node.named_children
+ )
+ argument_text = parsed.source[
+ arguments_node.start_byte + 1 : arguments_node.end_byte - 1
+ ].decode("utf-8", errors="replace")
+ calls.append(
+ JavaScriptCall(
+ callee=callee,
+ argument_text=argument_text,
+ arguments=arguments,
+ start_byte=node.start_byte,
+ )
+ )
+ calls.sort(key=lambda call: call.start_byte)
+ return calls
+
+
+def parse_source_functions(
+ source: str,
+ *,
+ suffix: str,
+ source_name: str = "",
+) -> list[JavaScriptFunction]:
+ """Return top-level functions and class methods from one structural index."""
+ parsed = _parse(source, suffix=suffix, source_name=source_name)
+ functions: list[JavaScriptFunction] = []
+ for statement in parsed.tree.root_node.named_children:
+ declaration = _unwrap_export(statement)
+ if declaration is None:
+ continue
+ if declaration.type in {"function_declaration", "generator_function_declaration"}:
+ name_node = declaration.child_by_field_name("name")
+ if name_node is not None:
+ functions.append(_function_fact(parsed, parsed.text(name_node), declaration))
+ continue
+ if declaration.type in {"lexical_declaration", "variable_declaration"}:
+ for declarator in declaration.named_children:
+ if declarator.type != "variable_declarator":
+ continue
+ name_node = declarator.child_by_field_name("name")
+ value_node = declarator.child_by_field_name("value")
+ if (
+ name_node is not None
+ and name_node.type == "identifier"
+ and value_node is not None
+ and value_node.type
+ in {"arrow_function", "function_expression", "generator_function"}
+ ):
+ functions.append(
+ _function_fact(parsed, parsed.text(name_node), declaration)
+ )
+ continue
+ if declaration.type in {"class_declaration", "abstract_class_declaration"}:
+ functions.extend(_class_functions(parsed, declaration))
+ continue
+ if declaration.type == "expression_statement":
+ assignment = next(
+ (
+ child
+ for child in declaration.named_children
+ if child.type == "assignment_expression"
+ ),
+ None,
+ )
+ if assignment is not None:
+ assigned = _assigned_function(parsed, assignment)
+ if assigned is not None:
+ functions.append(assigned)
+ unique = {
+ (function.name, function.start_line, function.end_line): function
+ for function in functions
+ }
+ return sorted(unique.values(), key=lambda item: (item.start_line, item.name))
+
+
+def source_defines_symbol(
+ source: str,
+ symbol_name: str,
+ *,
+ suffix: str,
+ source_name: str = "",
+) -> bool:
+ """Prove that a module declares or explicitly exports a requested symbol."""
+ parsed = _parse(source, suffix=suffix, source_name=source_name)
+ symbols: set[str] = set()
+ for statement in parsed.tree.root_node.named_children:
+ if statement.type == "export_statement":
+ prefix = parsed.text(statement).lstrip()
+ if prefix.startswith("export default"):
+ symbols.add("default")
+ symbols.update(_exported_names(parsed, statement))
+ declaration = _unwrap_export(statement)
+ if declaration is not None and not prefix.startswith("export default"):
+ symbols.update(_declaration_names(parsed, declaration))
+ continue
+ elif statement.type == "expression_statement":
+ symbols.update(_commonjs_export_names(parsed, statement))
+ return symbol_name in symbols
+
+
+def _parse(source: str, *, suffix: str, source_name: str) -> _ParsedSource:
+ normalized_suffix = _normalize_suffix(suffix)
+ source_bytes = source.encode("utf-8")
+ parser = Parser(_language(normalized_suffix))
+ tree = parser.parse(source_bytes)
+ parsed = _ParsedSource(
+ source=source_bytes,
+ tree=tree,
+ suffix=normalized_suffix,
+ source_name=source_name,
+ )
+ if tree.root_node.has_error:
+ problem = next(
+ (
+ node
+ for node in _walk(tree.root_node)
+ if node.is_error or node.is_missing
+ ),
+ tree.root_node,
+ )
+ line = problem.start_point.row + 1
+ column = problem.start_point.column + 1
+ excerpt = parsed.text(problem).strip().replace("\n", " ")[:120]
+ detail = f" near {excerpt!r}" if excerpt else ""
+ raise JavaScriptSyntaxError(
+ f"Could not parse JavaScript/TypeScript Static Evidence in "
+ f"{source_name}:{line}:{column}{detail}."
+ )
+ return parsed
+
+
+def _normalize_suffix(suffix: str) -> str:
+ normalized = suffix.lower()
+ if not normalized.startswith("."):
+ normalized = f".{normalized}"
+ if normalized not in SUPPORTED_SUFFIXES:
+ raise ValueError(
+ f"Unsupported JavaScript/TypeScript suffix: {suffix}. "
+ f"Supported values: {', '.join(sorted(SUPPORTED_SUFFIXES))}"
+ )
+ return normalized
+
+
+@lru_cache(maxsize=None)
+def _language(suffix: str) -> Language:
+ if suffix in JAVASCRIPT_SUFFIXES:
+ return Language(tree_sitter_javascript.language())
+ if suffix in TSX_SUFFIXES:
+ return Language(tree_sitter_typescript.language_tsx())
+ return Language(tree_sitter_typescript.language_typescript())
+
+
+def _walk(node: Node) -> Iterator[Node]:
+ yield node
+ for child in node.named_children:
+ yield from _walk(child)
+
+
+def _esm_imports(parsed: _ParsedSource, statement: Node) -> list[JavaScriptImport]:
+ source_node = statement.child_by_field_name("source")
+ if source_node is None:
+ return []
+ module = _static_string(parsed, source_node, {})
+ if module is None:
+ return []
+ clause = next(
+ (child for child in statement.named_children if child.type == "import_clause"),
+ None,
+ )
+ if clause is None:
+ return []
+ result: list[JavaScriptImport] = []
+ for child in clause.named_children:
+ if child.type == "identifier":
+ result.append(JavaScriptImport(module, parsed.text(child), "default"))
+ elif child.type == "namespace_import":
+ identifier = next(
+ (item for item in child.named_children if item.type == "identifier"),
+ None,
+ )
+ if identifier is not None:
+ result.append(JavaScriptImport(module, parsed.text(identifier), "*"))
+ elif child.type == "named_imports":
+ for specifier in child.named_children:
+ if specifier.type != "import_specifier":
+ continue
+ name = specifier.child_by_field_name("name")
+ alias = specifier.child_by_field_name("alias")
+ if name is not None:
+ result.append(
+ JavaScriptImport(
+ module,
+ parsed.text(alias or name),
+ parsed.text(name),
+ )
+ )
+ return result
+
+
+def _commonjs_imports(
+ parsed: _ParsedSource,
+ declaration: Node,
+) -> list[JavaScriptImport]:
+ result: list[JavaScriptImport] = []
+ for declarator in declaration.named_children:
+ if declarator.type != "variable_declarator":
+ continue
+ name = declarator.child_by_field_name("name")
+ value = declarator.child_by_field_name("value")
+ if name is None or value is None:
+ continue
+ require = _require_value(parsed, value)
+ if require is None:
+ continue
+ module, member = require
+ if name.type == "identifier":
+ result.append(
+ JavaScriptImport(module, parsed.text(name), member or "*")
+ )
+ elif name.type == "object_pattern":
+ for binding in name.named_children:
+ if binding.type == "pair_pattern":
+ key = binding.child_by_field_name("key")
+ local = binding.child_by_field_name("value")
+ if key is not None and local is not None:
+ result.append(
+ JavaScriptImport(
+ module,
+ parsed.text(local),
+ parsed.text(key),
+ )
+ )
+ elif binding.type == "shorthand_property_identifier_pattern":
+ binding_name = parsed.text(binding)
+ result.append(
+ JavaScriptImport(module, binding_name, binding_name)
+ )
+ return result
+
+
+def _require_value(parsed: _ParsedSource, node: Node) -> tuple[str, str | None] | None:
+ member: str | None = None
+ call = node
+ if node.type == "member_expression":
+ call = node.child_by_field_name("object")
+ property_node = node.child_by_field_name("property")
+ if property_node is not None:
+ member = parsed.text(property_node)
+ if call is None or call.type != "call_expression":
+ return None
+ function = call.child_by_field_name("function")
+ arguments = call.child_by_field_name("arguments")
+ if (
+ function is None
+ or parsed.text(function) != "require"
+ or arguments is None
+ or len(arguments.named_children) != 1
+ ):
+ return None
+ module = _static_string(parsed, arguments.named_children[0], {})
+ return (module, member) if module is not None else None
+
+
+def _visit_test_scope(
+ parsed: _ParsedSource,
+ node: Node,
+ *,
+ suite_titles: tuple[str, ...],
+ inherited_context: dict[str, object],
+ output: list[JavaScriptTestCase],
+) -> None:
+ if node.type == "call_expression":
+ suite_invocation = _decode_invocation(parsed, node, DESCRIBE_ROOTS)
+ if suite_invocation is not None:
+ for rendered_title, row_context in _expand_invocation(
+ parsed, suite_invocation, inherited_context
+ ):
+ merged_context = {**inherited_context, **row_context}
+ _visit_test_scope(
+ parsed,
+ suite_invocation.callback.body,
+ suite_titles=(*suite_titles, rendered_title),
+ inherited_context=merged_context,
+ output=output,
+ )
+ return
+
+ test_invocation = _decode_invocation(parsed, node, TEST_ROOTS)
+ if test_invocation is not None:
+ for rendered_title, row_context in _expand_invocation(
+ parsed, test_invocation, inherited_context
+ ):
+ merged_context = {**inherited_context, **row_context}
+ output.append(
+ JavaScriptTestCase(
+ title=rendered_title,
+ full_name=" ".join((*suite_titles, rendered_title)).strip(),
+ body=_callback_body_text(parsed, test_invocation.callback.body),
+ context=merged_context,
+ start_byte=node.start_byte,
+ )
+ )
+ return
+
+ for child in node.named_children:
+ _visit_test_scope(
+ parsed,
+ child,
+ suite_titles=suite_titles,
+ inherited_context=inherited_context,
+ output=output,
+ )
+
+
+def _decode_invocation(
+ parsed: _ParsedSource,
+ call: Node,
+ roots: set[str],
+) -> _Invocation | None:
+ function = call.child_by_field_name("function")
+ arguments = call.child_by_field_name("arguments")
+ if function is None or arguments is None:
+ return None
+ rows_node: Node | None = None
+ callee = _callee_path(parsed, function)
+ if callee is None and function.type == "call_expression":
+ each_function = function.child_by_field_name("function")
+ each_arguments = function.child_by_field_name("arguments")
+ each_callee = (
+ _callee_path(parsed, each_function) if each_function is not None else None
+ )
+ if (
+ each_callee
+ and _root_name(each_callee) in roots
+ and each_callee.split(".")[-1] == "each"
+ and each_arguments is not None
+ and each_arguments.named_children
+ ):
+ callee = each_callee
+ rows_node = each_arguments.named_children[0]
+ if callee is None or _root_name(callee) not in roots:
+ return None
+ if callee.split(".")[-1] == "each" and rows_node is None:
+ return None
+ invocation_arguments = arguments.named_children
+ if len(invocation_arguments) < 2:
+ return None
+ callback = _callback(parsed, invocation_arguments[1])
+ if callback is None:
+ return None
+ return _Invocation(
+ title_node=invocation_arguments[0],
+ callback=callback,
+ rows_node=rows_node,
+ )
+
+
+def _callback(parsed: _ParsedSource, node: Node) -> _Callback | None:
+ if node.type not in {"arrow_function", "function_expression", "function"}:
+ return None
+ body = node.child_by_field_name("body")
+ if body is None:
+ return None
+ parameters = node.child_by_field_name("parameters")
+ parameter_names = tuple(_pattern_names(parsed, parameters)) if parameters else ()
+ return _Callback(node=node, body=body, parameter_names=parameter_names)
+
+
+def _pattern_names(parsed: _ParsedSource, node: Node) -> list[str]:
+ if node.type in {
+ "identifier",
+ "shorthand_property_identifier_pattern",
+ }:
+ return [parsed.text(node)]
+ if node.type in {"required_parameter", "optional_parameter"}:
+ pattern = node.child_by_field_name("pattern")
+ return _pattern_names(parsed, pattern) if pattern is not None else []
+ if node.type == "assignment_pattern":
+ left = node.child_by_field_name("left")
+ return _pattern_names(parsed, left) if left is not None else []
+ names: list[str] = []
+ for child in node.named_children:
+ if child.type in {"type_annotation", "predefined_type", "type_identifier"}:
+ continue
+ names.extend(_pattern_names(parsed, child))
+ return names
+
+
+def _expand_invocation(
+ parsed: _ParsedSource,
+ invocation: _Invocation,
+ inherited_context: dict[str, object],
+) -> list[tuple[str, dict[str, object]]]:
+ if invocation.rows_node is None:
+ title = _static_string(parsed, invocation.title_node, inherited_context)
+ return [(title, {})] if title is not None else []
+
+ rows = _literal_value(parsed, invocation.rows_node, inherited_context)
+ if not isinstance(rows, list):
+ return []
+ expanded: list[tuple[str, dict[str, object]]] = []
+ for index, row in enumerate(rows, start=1):
+ context = _row_context(invocation.callback.parameter_names, row)
+ combined = {**inherited_context, **context}
+ title = _static_string(parsed, invocation.title_node, combined)
+ if title is None:
+ continue
+ expanded.append((_render_each_title(title, row, context, index), context))
+ return expanded
+
+
+def _row_context(parameter_names: tuple[str, ...], row: object) -> dict[str, object]:
+ if isinstance(row, dict):
+ return dict(row)
+ if isinstance(row, list):
+ return {
+ name: row[index]
+ for index, name in enumerate(parameter_names)
+ if index < len(row)
+ }
+ if len(parameter_names) == 1:
+ return {parameter_names[0]: row}
+ return {}
+
+
+def _render_each_title(
+ title: str,
+ row: object,
+ context: dict[str, object],
+ index: int,
+) -> str:
+ rendered = title
+ values = row if isinstance(row, list) else list(context.values())
+ pending = [_display_value(value) for value in values]
+ for token in ("%s", "%d", "%i", "%f", "%p", "%j", "%o"):
+ while token in rendered and pending:
+ rendered = rendered.replace(token, pending.pop(0), 1)
+ for name, value in context.items():
+ rendered = rendered.replace(f"${name}", _display_value(value))
+ if rendered == title:
+ rendered = f"{title} case {index:02d}"
+ return rendered
+
+
+def _static_string(
+ parsed: _ParsedSource,
+ node: Node,
+ context: dict[str, object],
+) -> str | None:
+ if node.type == "string":
+ raw = parsed.text(node)
+ return _decode_quoted_string(raw)
+ if node.type == "template_string":
+ parts: list[str] = []
+ for child in node.named_children:
+ if child.type == "string_fragment":
+ parts.append(parsed.text(child))
+ elif child.type == "escape_sequence":
+ parts.append(_decode_escape(parsed.text(child)))
+ elif child.type == "template_substitution":
+ expression = next(iter(child.named_children), None)
+ value = (
+ _literal_value(parsed, expression, context)
+ if expression is not None
+ else _UNRESOLVED
+ )
+ if value is _UNRESOLVED:
+ return None
+ parts.append(_display_value(value))
+ return "".join(parts)
+ return None
+
+
+def _literal_value(
+ parsed: _ParsedSource,
+ node: Node,
+ context: dict[str, object],
+) -> object:
+ if node.type in {"string", "template_string"}:
+ value = _static_string(parsed, node, context)
+ return value if value is not None else _UNRESOLVED
+ if node.type == "number":
+ raw = parsed.text(node).replace("_", "")
+ try:
+ return float(raw) if any(char in raw for char in ".eE") else int(raw, 0)
+ except ValueError:
+ return _UNRESOLVED
+ if node.type in {"true", "false"}:
+ return node.type == "true"
+ if node.type == "null":
+ return "null"
+ if node.type == "undefined":
+ return "undefined"
+ if node.type == "identifier":
+ return context.get(parsed.text(node), _UNRESOLVED)
+ if node.type == "array":
+ values: list[object] = []
+ for child in node.named_children:
+ value = _literal_value(parsed, child, context)
+ if value is _UNRESOLVED:
+ return _UNRESOLVED
+ values.append(value)
+ return values
+ if node.type == "object":
+ result: dict[str, object] = {}
+ for child in node.named_children:
+ if child.type == "pair":
+ key_node = child.child_by_field_name("key")
+ value_node = child.child_by_field_name("value")
+ if key_node is None or value_node is None:
+ return _UNRESOLVED
+ key = _property_name(parsed, key_node, context)
+ value = _literal_value(parsed, value_node, context)
+ if key is None or value is _UNRESOLVED:
+ return _UNRESOLVED
+ result[key] = value
+ elif child.type == "shorthand_property_identifier":
+ key = parsed.text(child)
+ value = context.get(key, _UNRESOLVED)
+ if value is _UNRESOLVED:
+ return _UNRESOLVED
+ result[key] = value
+ else:
+ return _UNRESOLVED
+ return result
+ if node.type in {
+ "parenthesized_expression",
+ "as_expression",
+ "satisfies_expression",
+ "type_assertion",
+ }:
+ expression = node.child_by_field_name("expression") or next(
+ (
+ child
+ for child in node.named_children
+ if child.type not in {"type_annotation", "type_identifier"}
+ ),
+ None,
+ )
+ return (
+ _literal_value(parsed, expression, context)
+ if expression is not None
+ else _UNRESOLVED
+ )
+ if node.type == "unary_expression":
+ argument = node.child_by_field_name("argument")
+ if argument is None:
+ return _UNRESOLVED
+ value = _literal_value(parsed, argument, context)
+ operator = parsed.text(node)[: argument.start_byte - node.start_byte].strip()
+ if isinstance(value, (int, float)) and operator in {"+", "-"}:
+ return value if operator == "+" else -value
+ if node.type == "member_expression":
+ object_node = node.child_by_field_name("object")
+ property_node = node.child_by_field_name("property")
+ if object_node is not None and property_node is not None:
+ owner = _literal_value(parsed, object_node, context)
+ key = parsed.text(property_node)
+ if isinstance(owner, dict) and key in owner:
+ return owner[key]
+ return _UNRESOLVED
+
+
+def _property_name(
+ parsed: _ParsedSource,
+ node: Node,
+ context: dict[str, object],
+) -> str | None:
+ if node.type in {"property_identifier", "identifier", "number"}:
+ return parsed.text(node)
+ value = _static_string(parsed, node, context)
+ return value
+
+
+def _decode_quoted_string(raw: str) -> str:
+ if len(raw) < 2:
+ return raw
+ inner = raw[1:-1]
+ result: list[str] = []
+ cursor = 0
+ while cursor < len(inner):
+ if inner[cursor] == "\\" and cursor + 1 < len(inner):
+ escape = inner[cursor : cursor + 2]
+ result.append(_decode_escape(escape))
+ cursor += 2
+ else:
+ result.append(inner[cursor])
+ cursor += 1
+ return "".join(result)
+
+
+def _decode_escape(raw: str) -> str:
+ escapes = {
+ "\\n": "\n",
+ "\\r": "\r",
+ "\\t": "\t",
+ "\\b": "\b",
+ "\\f": "\f",
+ "\\v": "\v",
+ "\\0": "\0",
+ "\\\\": "\\",
+ "\\\"": '"',
+ "\\'": "'",
+ "\\`": "`",
+ }
+ return escapes.get(raw, raw[1:] if raw.startswith("\\") else raw)
+
+
+def _display_value(value: object) -> str:
+ if isinstance(value, bool):
+ return "true" if value else "false"
+ if isinstance(value, dict):
+ return "{" + ", ".join(
+ f"{key}: {_display_value(inner)}" for key, inner in value.items()
+ ) + "}"
+ if isinstance(value, list):
+ return "[" + ", ".join(_display_value(inner) for inner in value) + "]"
+ return str(value)
+
+
+def _callback_body_text(parsed: _ParsedSource, body: Node) -> str:
+ if body.type == "statement_block":
+ return parsed.source[body.start_byte + 1 : body.end_byte - 1].decode(
+ "utf-8", errors="replace"
+ ).strip()
+ return parsed.text(body).strip()
+
+
+def _callee_path(parsed: _ParsedSource, node: Node) -> str | None:
+ if node.type in {"identifier", "property_identifier"}:
+ return parsed.text(node)
+ if node.type in {"member_expression", "optional_chain"}:
+ owner = node.child_by_field_name("object")
+ property_node = node.child_by_field_name("property")
+ if owner is None or property_node is None:
+ return None
+ owner_path = _callee_path(parsed, owner)
+ return (
+ f"{owner_path}.{parsed.text(property_node)}" if owner_path else None
+ )
+ return None
+
+
+def _root_name(callee: str) -> str:
+ return callee.split(".", 1)[0]
+
+
+def _unwrap_export(statement: Node) -> Node | None:
+ if statement.type != "export_statement":
+ return statement
+ return statement.child_by_field_name("declaration") or next(
+ (
+ child
+ for child in statement.named_children
+ if child.type
+ in {
+ "function_declaration",
+ "generator_function_declaration",
+ "class_declaration",
+ "abstract_class_declaration",
+ "lexical_declaration",
+ "variable_declaration",
+ "expression_statement",
+ }
+ ),
+ None,
+ )
+
+
+def _function_fact(
+ parsed: _ParsedSource,
+ name: str,
+ node: Node,
+) -> JavaScriptFunction:
+ return JavaScriptFunction(
+ name=name,
+ start_line=node.start_point.row + 1,
+ end_line=node.end_point.row + 1,
+ )
+
+
+def _class_functions(
+ parsed: _ParsedSource,
+ declaration: Node,
+) -> list[JavaScriptFunction]:
+ name_node = declaration.child_by_field_name("name")
+ body = declaration.child_by_field_name("body")
+ if name_node is None or body is None:
+ return []
+ class_name = parsed.text(name_node)
+ functions: list[JavaScriptFunction] = []
+ for member in body.named_children:
+ if member.type == "method_definition":
+ method_name = member.child_by_field_name("name")
+ if method_name is None or parsed.text(method_name) == "constructor":
+ continue
+ functions.append(
+ _function_fact(
+ parsed,
+ f"{class_name}.{parsed.text(method_name)}",
+ member,
+ )
+ )
+ elif member.type == "public_field_definition":
+ field_name = member.child_by_field_name("name")
+ value = member.child_by_field_name("value")
+ if (
+ field_name is not None
+ and value is not None
+ and value.type in {"arrow_function", "function_expression"}
+ ):
+ functions.append(
+ _function_fact(
+ parsed,
+ f"{class_name}.{parsed.text(field_name)}",
+ member,
+ )
+ )
+ return functions
+
+
+def _assigned_function(
+ parsed: _ParsedSource,
+ assignment: Node,
+) -> JavaScriptFunction | None:
+ left = assignment.child_by_field_name("left")
+ right = assignment.child_by_field_name("right")
+ if (
+ left is None
+ or right is None
+ or right.type not in {"function_expression", "arrow_function"}
+ ):
+ return None
+ name = _member_property(parsed, left)
+ return _function_fact(parsed, name, assignment) if name else None
+
+
+def _member_property(parsed: _ParsedSource, node: Node) -> str | None:
+ if node.type != "member_expression":
+ return None
+ property_node = node.child_by_field_name("property")
+ return parsed.text(property_node) if property_node is not None else None
+
+
+def _exported_names(parsed: _ParsedSource, statement: Node) -> set[str]:
+ names: set[str] = set()
+ for node in _walk(statement):
+ if node.type != "export_specifier":
+ continue
+ name = node.child_by_field_name("name")
+ alias = node.child_by_field_name("alias")
+ if name is not None:
+ names.add(parsed.text(alias or name))
+ return names
+
+
+def _declaration_names(parsed: _ParsedSource, declaration: Node) -> set[str]:
+ names: set[str] = set()
+ if declaration.type in {
+ "function_declaration",
+ "generator_function_declaration",
+ "class_declaration",
+ "abstract_class_declaration",
+ }:
+ name_node = declaration.child_by_field_name("name")
+ if name_node is not None:
+ names.add(parsed.text(name_node))
+ elif declaration.type in {"lexical_declaration", "variable_declaration"}:
+ for declarator in declaration.named_children:
+ if declarator.type != "variable_declarator":
+ continue
+ name_node = declarator.child_by_field_name("name")
+ if name_node is not None and name_node.type == "identifier":
+ names.add(parsed.text(name_node))
+ return names
+
+
+def _commonjs_export_names(parsed: _ParsedSource, statement: Node) -> set[str]:
+ names: set[str] = set()
+ for node in _walk(statement):
+ if node.type != "assignment_expression":
+ continue
+ left = node.child_by_field_name("left")
+ right = node.child_by_field_name("right")
+ if left is None:
+ continue
+ left_text = parsed.text(left)
+ if left_text in {"module.exports", "exports"} and right is not None:
+ if right.type == "object":
+ for child in right.named_children:
+ if child.type == "shorthand_property_identifier":
+ names.add(parsed.text(child))
+ elif child.type == "pair":
+ key = child.child_by_field_name("key")
+ if key is not None:
+ rendered = _property_name(parsed, key, {})
+ if rendered:
+ names.add(rendered)
+ elif left_text.startswith(("exports.", "module.exports.")):
+ property_name = _member_property(parsed, left)
+ if property_name:
+ names.add(property_name)
+ return names
diff --git a/src/teaforge/jest/assets/__init__.py b/src/teaforge/jest/assets/__init__.py
new file mode 100644
index 0000000..bc9b2cb
--- /dev/null
+++ b/src/teaforge/jest/assets/__init__.py
@@ -0,0 +1 @@
+"""Runtime assets injected into Jest projects by TeaForge."""
diff --git a/src/teaforge/jest/assets/runtime-listener.cjs b/src/teaforge/jest/assets/runtime-listener.cjs
new file mode 100644
index 0000000..b6535db
--- /dev/null
+++ b/src/teaforge/jest/assets/runtime-listener.cjs
@@ -0,0 +1,211 @@
+"use strict";
+
+const fs = require("fs");
+
+const evidencePath = process.env.TEAFORGE_JEST_EVIDENCE_PATH;
+const originalExpect = global.expect;
+const MAX_VALUE_LENGTH = 4096;
+const DEFAULT_MAX_ASSERTIONS = 5000;
+const DEFAULT_MAX_EVIDENCE_BYTES = 5 * 1024 * 1024;
+const WARNING_RESERVE_BYTES = 512;
+
+const positiveInteger = (value, fallback) => {
+ const parsed = Number.parseInt(value || "", 10);
+ return Number.isSafeInteger(parsed) && parsed > 0 ? parsed : fallback;
+};
+
+const maxAssertions = positiveInteger(
+ process.env.TEAFORGE_JEST_MAX_ASSERTIONS,
+ DEFAULT_MAX_ASSERTIONS,
+);
+const maxEvidenceBytes = positiveInteger(
+ process.env.TEAFORGE_JEST_MAX_EVIDENCE_BYTES,
+ DEFAULT_MAX_EVIDENCE_BYTES,
+);
+const sensitiveKey = /(?:password|passwd|secret|token|authorization|cookie|api[_-]?key|session)/i;
+const secretTextPatterns = [
+ /\bBearer\s+[A-Za-z0-9._~+\/-]+=*/gi,
+ /\bsk-[A-Za-z0-9_-]{16,}\b/g,
+ /\bAKIA[A-Z0-9]{16}\b/g,
+ /\beyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\b/g,
+];
+
+if (evidencePath && typeof originalExpect === "function") {
+ let assertionCount = 0;
+ let limitWarningWritten = false;
+
+ const redactText = (value) => {
+ let redacted = value;
+ for (const pattern of secretTextPatterns) {
+ redacted = redacted.replace(pattern, "[REDACTED]");
+ }
+ return redacted;
+ };
+
+ const render = (value) => {
+ const seen = new WeakSet();
+ let rendered;
+ try {
+ rendered = JSON.stringify(value, (key, nested) => {
+ if (key && sensitiveKey.test(key)) return "[REDACTED]";
+ if (typeof nested === "bigint") return `${nested.toString()}n`;
+ if (typeof nested === "function") return `[Function ${nested.name || "anonymous"}]`;
+ if (typeof nested === "symbol") return nested.toString();
+ if (typeof nested === "string") return redactText(nested);
+ if (nested instanceof Error) {
+ return { name: nested.name, message: nested.message };
+ }
+ if (nested && typeof nested === "object") {
+ if (seen.has(nested)) return "[Circular]";
+ seen.add(nested);
+ }
+ return nested;
+ });
+ } catch (_error) {
+ rendered = redactText(String(value));
+ }
+ if (rendered === undefined) rendered = redactText(String(value));
+ if (rendered.length > MAX_VALUE_LENGTH) {
+ return `${rendered.slice(0, MAX_VALUE_LENGTH)}…[truncated]`;
+ }
+ return rendered;
+ };
+
+ const currentEvidenceBytes = () => {
+ try {
+ return fs.statSync(evidencePath).size;
+ } catch (error) {
+ if (error && error.code === "ENOENT") return 0;
+ throw error;
+ }
+ };
+
+ const appendPayload = (payload, reserveBytes = 0) => {
+ const line = `${JSON.stringify(payload)}\n`;
+ const lineBytes = Buffer.byteLength(line, "utf8");
+ if (currentEvidenceBytes() + lineBytes + reserveBytes > maxEvidenceBytes) {
+ return false;
+ }
+ fs.appendFileSync(evidencePath, line, "utf8");
+ return true;
+ };
+
+ const writeLimitWarning = (message) => {
+ if (limitWarningWritten) return;
+ limitWarningWritten = true;
+ appendPayload({
+ schema_version: 1,
+ kind: "warning",
+ code: "evidence-limit-reached",
+ message,
+ });
+ };
+
+ const writeEvidence = ({ actual, expected, matcher, modifiers, passed, error }) => {
+ if (assertionCount >= maxAssertions) {
+ writeLimitWarning(
+ `Jest evidence was capped at ${maxAssertions} assertion records.`,
+ );
+ return;
+ }
+ const state = typeof originalExpect.getState === "function" ? originalExpect.getState() : {};
+ const payload = {
+ schema_version: 1,
+ kind: "assertion",
+ test_name: state.currentTestName || "",
+ test_path: state.testPath || "",
+ matcher,
+ expected: expected.map(render),
+ actual: render(actual),
+ passed,
+ negated: modifiers.includes("not"),
+ promise_mode: modifiers.includes("resolves")
+ ? "resolves"
+ : modifiers.includes("rejects")
+ ? "rejects"
+ : "",
+ error: error ? `${error.name || "Error"}: matcher failed; inspect expected and actual evidence.` : "",
+ };
+ if (!appendPayload(payload, WARNING_RESERVE_BYTES)) {
+ writeLimitWarning(
+ `Jest evidence reached its ${maxEvidenceBytes}-byte limit and was truncated.`,
+ );
+ return;
+ }
+ assertionCount += 1;
+ };
+
+ const observedPromiseValue = async (actual, modifiers) => {
+ if (modifiers.includes("resolves")) return Promise.resolve(actual);
+ if (modifiers.includes("rejects")) {
+ try {
+ return await Promise.resolve(actual);
+ } catch (error) {
+ return error;
+ }
+ }
+ return actual;
+ };
+
+ const wrapExpectation = (expectation, actual, modifiers = []) => new Proxy(expectation, {
+ get(target, property, receiver) {
+ const value = Reflect.get(target, property, receiver);
+ if (typeof value === "function") {
+ const matcher = String(property);
+ return (...expected) => {
+ let observedActual = actual;
+ let matcherTarget = target;
+ let matcherFunction = value;
+ if (typeof actual === "function" && matcher.startsWith("toThrow")) {
+ const observedFunction = (...args) => {
+ try {
+ const result = actual(...args);
+ observedActual = result;
+ return result;
+ } catch (error) {
+ observedActual = error;
+ throw error;
+ }
+ };
+ matcherTarget = originalExpect(observedFunction);
+ for (const modifier of modifiers) matcherTarget = matcherTarget[modifier];
+ matcherFunction = matcherTarget[property];
+ }
+ try {
+ const result = Reflect.apply(matcherFunction, matcherTarget, expected);
+ if (result && typeof result.then === "function") {
+ return Promise.resolve(result).then(
+ async (resolved) => {
+ const observed = await observedPromiseValue(observedActual, modifiers);
+ writeEvidence({ actual: observed, expected, matcher, modifiers, passed: true });
+ return resolved;
+ },
+ async (error) => {
+ const observed = await observedPromiseValue(observedActual, modifiers);
+ writeEvidence({ actual: observed, expected, matcher, modifiers, passed: false, error });
+ throw error;
+ },
+ );
+ }
+ writeEvidence({ actual: observedActual, expected, matcher, modifiers, passed: true });
+ return result;
+ } catch (error) {
+ writeEvidence({ actual: observedActual, expected, matcher, modifiers, passed: false, error });
+ throw error;
+ }
+ };
+ }
+ if (value && typeof value === "object") {
+ return wrapExpectation(value, actual, [...modifiers, String(property)]);
+ }
+ return value;
+ },
+ });
+
+ global.expect = new Proxy(originalExpect, {
+ apply(target, thisArg, args) {
+ const expectation = Reflect.apply(target, thisArg, args);
+ return wrapExpectation(expectation, args[0]);
+ },
+ });
+}
diff --git a/src/teaforge/jest/coverage.py b/src/teaforge/jest/coverage.py
index 6f13ca3..be01a41 100644
--- a/src/teaforge/jest/coverage.py
+++ b/src/teaforge/jest/coverage.py
@@ -3,11 +3,13 @@
from __future__ import annotations
import json
-import subprocess
from pathlib import Path
from typing import TYPE_CHECKING
+from teaforge.javascript.syntax import parse_source_functions as parse_javascript_source_functions
from teaforge.jest.parser import parse_jest_documents
+from teaforge.jest.project import JestProject
+from teaforge.process import bounded_process_detail
if TYPE_CHECKING:
from teaforge.coverage.analyzer import SourceCoverageSnapshot, SourceFunction
@@ -38,20 +40,44 @@ def parse_jest_source_functions(source_path: Path) -> list[SourceFunction]:
from teaforge.coverage.analyzer import SourceFunction
source = source_path.read_text(encoding="utf-8")
- functions: list[SourceFunction] = []
- functions.extend(_parse_top_level_functions(source))
- functions.extend(_parse_arrow_functions(source))
- functions.extend(_parse_class_methods(source))
- functions.sort(key=lambda function: (function.lineno, function.name))
- return functions
+ return [
+ SourceFunction(
+ name=function.name,
+ lineno=function.start_line,
+ end_lineno=function.end_line,
+ )
+ for function in parse_javascript_source_functions(
+ source,
+ suffix=source_path.suffix,
+ source_name=str(source_path),
+ )
+ ]
def analyze_jest_coverage(
test_path: Path,
source_paths: list[Path],
+ *,
+ timeout_seconds: int = 120,
) -> dict[Path, SourceCoverageSnapshot]:
"""Run Jest with Istanbul JSON coverage output and map it to resolved sources."""
- resolved_test_path = test_path.resolve()
+ return _analyze_jest_coverage(
+ test_path,
+ source_paths,
+ timeout_seconds=timeout_seconds,
+ operation="Jest coverage collection",
+ )
+
+
+def _analyze_jest_coverage(
+ test_path: Path,
+ source_paths: list[Path],
+ *,
+ timeout_seconds: int,
+ operation: str,
+) -> dict[Path, SourceCoverageSnapshot]:
+ """Execute the project-local Jest once and parse its Istanbul payload."""
+ project = JestProject.discover(test_path)
resolved_sources = [path.resolve() for path in source_paths]
import tempfile
@@ -62,31 +88,24 @@ def analyze_jest_coverage(
coverage_dir.mkdir(parents=True, exist_ok=True)
coverage_file = coverage_dir / "coverage-final.json"
- try:
- run_result = subprocess.run(
- [
- "npx",
- "jest",
- "--coverage",
- "--coverageReporters=json",
- "--coverageDirectory",
- str(coverage_dir),
- "--runInBand",
- str(resolved_test_path),
- ],
- capture_output=True,
- text=True,
- check=False,
- )
- except FileNotFoundError as exc:
- raise RuntimeError(
- "Jest coverage execution requires `npx`. Install Node.js and Jest before running coverage reports."
- ) from exc
+ run_result = project.run(
+ [
+ "--coverage",
+ "--coverageReporters=json",
+ "--coverageDirectory",
+ str(coverage_dir),
+ "--runInBand",
+ "--runTestsByPath",
+ *[str(path) for path in project.test_files],
+ ],
+ timeout_seconds=timeout_seconds,
+ operation=operation,
+ )
if run_result.returncode != 0:
- detail = (run_result.stderr or run_result.stdout).strip()
+ detail = bounded_process_detail(run_result.stderr, run_result.stdout)
raise RuntimeError(
- "Jest failed while collecting coverage data. "
+ f"{operation} failed. "
f"Exit code: {run_result.returncode}. {detail}"
)
@@ -158,17 +177,17 @@ def _collect_line_sets(
def _collect_branch_sets(
branches: dict,
branch_hits: dict,
-) -> tuple[set[tuple[int, int]], set[tuple[int, int]]]:
+) -> tuple[set[tuple[int, int, str]], set[tuple[int, int, str]]]:
"""Collect covered and missing branch edges from Istanbul branch maps."""
- executed_branches: set[tuple[int, int]] = set()
- missing_branches: set[tuple[int, int]] = set()
+ executed_branches: set[tuple[int, int, str]] = set()
+ missing_branches: set[tuple[int, int, str]] = set()
for branch_id, branch in branches.items():
branch_line = int(branch.get("line") or branch.get("loc", {}).get("start", {}).get("line", 0))
locations = branch.get("locations", [])
hits = branch_hits.get(branch_id, [])
for index, location in enumerate(locations):
destination = int(location.get("start", {}).get("line", branch_line))
- edge = (branch_line, destination)
+ edge = (branch_line, destination, f"{branch_id}:{index}")
if index < len(hits) and int(hits[index]) > 0:
executed_branches.add(edge)
else:
@@ -178,147 +197,10 @@ def _collect_branch_sets(
return executed_branches, missing_branches
-def _parse_top_level_functions(source: str) -> list[SourceFunction]:
- """Parse top-level function declarations from JS/TS source text."""
- from teaforge.coverage.analyzer import SourceFunction
-
- functions: list[SourceFunction] = []
- import re
-
- pattern = re.compile(r"(?:export\s+)?(?:async\s+)?function\s+(?P[A-Za-z_$][\w$]*)\s*\([^)]*\)\s*\{")
- for match in pattern.finditer(source):
- open_brace = match.end() - 1
- close_brace = _find_matching_brace(source, open_brace)
- if close_brace == -1:
- continue
- functions.append(
- SourceFunction(
- name=match.group("name"),
- lineno=_line_number_at(source, match.start()),
- end_lineno=_line_number_at(source, close_brace),
- )
- )
- return functions
-
-
-def _parse_arrow_functions(source: str) -> list[SourceFunction]:
- """Parse top-level arrow-function variable declarations from JS/TS source text."""
- from teaforge.coverage.analyzer import SourceFunction
-
- functions: list[SourceFunction] = []
- import re
-
- pattern = re.compile(
- r"(?:export\s+)?(?:const|let|var)\s+(?P[A-Za-z_$][\w$]*)\s*=\s*(?:async\s*)?\([^)]*\)\s*=>\s*\{"
- )
- for match in pattern.finditer(source):
- open_brace = match.end() - 1
- close_brace = _find_matching_brace(source, open_brace)
- if close_brace == -1:
- continue
- functions.append(
- SourceFunction(
- name=match.group("name"),
- lineno=_line_number_at(source, match.start()),
- end_lineno=_line_number_at(source, close_brace),
- )
- )
- return functions
-
-
-def _parse_class_methods(source: str) -> list[SourceFunction]:
- """Parse class method declarations from JS/TS source text."""
- from teaforge.coverage.analyzer import SourceFunction
-
- methods: list[SourceFunction] = []
- import re
-
- class_pattern = re.compile(r"(?:export\s+)?class\s+(?P[A-Za-z_$][\w$]*)[^\{]*\{")
- method_pattern = re.compile(r"^\s*(?:async\s+)?(?P[A-Za-z_$][\w$]*)\s*\([^)]*\)\s*\{", re.MULTILINE)
- for class_match in class_pattern.finditer(source):
- class_name = class_match.group("name")
- class_open = class_match.end() - 1
- class_close = _find_matching_brace(source, class_open)
- if class_close == -1:
- continue
- body = source[class_open + 1 : class_close]
- body_offset = class_open + 1
- for method_match in method_pattern.finditer(body):
- method_name = method_match.group("name")
- if method_name == "constructor":
- continue
- method_start = body_offset + method_match.start()
- method_open = body_offset + method_match.end() - 1
- method_close = _find_matching_brace(source, method_open)
- if method_close == -1 or method_close > class_close:
- continue
- methods.append(
- SourceFunction(
- name=f"{class_name}.{method_name}",
- lineno=_line_number_at(source, method_start),
- end_lineno=_line_number_at(source, method_close),
- )
- )
- return methods
-
-
-def _find_matching_brace(source: str, open_index: int) -> int:
- """Find the matching closing brace in a JS/TS source block."""
- depth = 0
- cursor = open_index
- while cursor < len(source):
- char = source[cursor]
- if char in {'"', "'", "`"}:
- cursor = _skip_string_literal(source, cursor)
- if cursor == -1:
- return -1
- continue
- if source.startswith("//", cursor):
- newline = source.find("\n", cursor)
- if newline == -1:
- return -1
- cursor = newline + 1
- continue
- if source.startswith("/*", cursor):
- end_comment = source.find("*/", cursor + 2)
- if end_comment == -1:
- return -1
- cursor = end_comment + 2
- continue
- if char == "{":
- depth += 1
- elif char == "}":
- depth -= 1
- if depth == 0:
- return cursor
- cursor += 1
- return -1
-
-
-def _skip_string_literal(source: str, cursor: int) -> int:
- """Skip past one string literal, returning the next cursor position."""
- quote = source[cursor]
- cursor += 1
- while cursor < len(source):
- char = source[cursor]
- if char == "\\":
- cursor += 2
- continue
- if char == quote:
- return cursor + 1
- cursor += 1
- return -1
-
-
-def _line_number_at(source: str, index: int) -> int:
- """Translate a character offset into a 1-based source line number."""
- return source.count("\n", 0, index) + 1
-
-
def _document_uses_test_file_as_source(document) -> bool:
"""Detect parser fallbacks where the test file could not be resolved to production code."""
return (
document.source_path == document.test_source_path
and document.file == document.test_file
and document.method == document.test_method
- )
\ No newline at end of file
+ )
diff --git a/src/teaforge/jest/parser.py b/src/teaforge/jest/parser.py
index db644d8..aeb8c1d 100644
--- a/src/teaforge/jest/parser.py
+++ b/src/teaforge/jest/parser.py
@@ -8,16 +8,36 @@
from functools import lru_cache
from pathlib import Path
+from teaforge.discovery import discover_files
+from teaforge.javascript.syntax import (
+ extract_imports as extract_javascript_imports,
+)
+from teaforge.javascript.syntax import (
+ extract_test_cases as extract_javascript_test_cases,
+)
+from teaforge.javascript.syntax import (
+ parse_calls as parse_javascript_calls,
+)
+from teaforge.javascript.syntax import (
+ source_defines_symbol,
+)
+from teaforge.pcl.assembly import build_input_rows, build_output_rows, merge_documents_by_subject
from teaforge.pcl.classification import classify_type
from teaforge.pcl.models import PCLDocument, PCLTestCase
from teaforge.pcl.parser import (
- _build_input_rows,
- _build_output_rows,
_display_path,
- _merge_documents_by_subject,
)
-TEST_FILE_SUFFIXES = (".test.ts", ".spec.ts", ".test.js", ".spec.js")
+TEST_FILE_SUFFIXES = (
+ ".test.ts",
+ ".spec.ts",
+ ".test.tsx",
+ ".spec.tsx",
+ ".test.js",
+ ".spec.js",
+ ".test.jsx",
+ ".spec.jsx",
+)
DECLARATION_PREFIXES = ("const", "let", "var")
@@ -40,6 +60,8 @@ class JestTestBlock:
body: str
identifier: str
context: dict[str, object]
+ suffix: str = ".ts"
+ full_name: str = ""
@dataclass(slots=True, frozen=True)
@@ -59,13 +81,20 @@ def parse_jest_documents(test_path: Path) -> list[PCLDocument]:
for file_path in files:
content = file_path.read_text(encoding="utf-8")
imports = _extract_imports(file_path, content)
- for index, block in enumerate(_extract_test_blocks(content), start=1):
+ for index, block in enumerate(
+ _extract_test_blocks(
+ content,
+ suffix=file_path.suffix,
+ source_name=str(file_path),
+ ),
+ start=1,
+ ):
raw_documents.append(_build_document_for_block(file_path, block, imports, index))
if not raw_documents:
raise ValueError(f"No supported Jest tests found under: {test_path}")
- return _merge_documents_by_subject(raw_documents)
+ return merge_documents_by_subject(raw_documents)
def _build_document_for_block(
@@ -85,7 +114,13 @@ def _build_document_for_block(
test_source_path=_display_path(file_path),
)
- inputs = _extract_case_inputs(block.body, imports, subject.method, block.context)
+ inputs = _extract_case_inputs(
+ block.body,
+ imports,
+ subject.method,
+ block.context,
+ suffix=block.suffix,
+ )
output_checks = _extract_output_expectations(block.body, block.context)
output = "; ".join(output_checks) if output_checks else "No explicit expect() assertion found"
document.testcases.append(
@@ -97,22 +132,18 @@ def _build_document_for_block(
output=output,
type=classify_type(block.title, block.identifier, output),
output_checks=output_checks,
+ test_full_name=block.full_name or block.title,
+ test_source_path=_display_path(file_path),
)
)
- document.input_rows = _build_input_rows(document.testcases)
- document.output_rows = _build_output_rows(document.testcases)
+ document.input_rows = build_input_rows(document.testcases)
+ document.output_rows = build_output_rows(document.testcases)
return document
def _collect_test_files(path: Path) -> list[Path]:
"""Expand a file or directory input into supported Jest-style test files."""
- if path.is_file():
- return [path] if _is_test_file(path) else []
- patterns = ("*.test.ts", "*.spec.ts", "*.test.js", "*.spec.js")
- files: list[Path] = []
- for pattern in patterns:
- files.extend(path.rglob(pattern))
- return sorted(set(files))
+ return discover_files(path, is_candidate=_is_test_file)
def _is_test_file(path: Path) -> bool:
@@ -121,41 +152,18 @@ def _is_test_file(path: Path) -> bool:
def _extract_imports(file_path: Path, content: str) -> dict[str, JSImport]:
- """Map imported bindings in a Jest file to local source files when resolvable."""
- imports: dict[str, JSImport] = {}
-
- named_pattern = re.compile(r"import\s*\{(?P.*?)\}\s*from\s*[\"'](?P[^\"']+)[\"']", re.DOTALL)
- namespace_pattern = re.compile(r"import\s*\*\s*as\s*(?P[A-Za-z_$][\w$]*)\s*from\s*[\"'](?P[^\"']+)[\"']")
- default_pattern = re.compile(r"import\s+(?P[A-Za-z_$][\w$]*)\s+from\s*[\"'](?P[^\"']+)[\"']")
-
- for match in named_pattern.finditer(content):
- module_path = _resolve_module_path(file_path, match.group("module"))
- for raw_name in match.group("names").split(","):
- name = raw_name.strip()
- if not name:
- continue
- original_name, bind_name = _split_import_alias(name)
- imports[bind_name] = JSImport(file_path=module_path, original_name=original_name)
-
- for match in namespace_pattern.finditer(content):
- alias = match.group("alias")
- module_path = _resolve_module_path(file_path, match.group("module"))
- imports[alias] = JSImport(file_path=module_path, original_name="*")
-
- for match in default_pattern.finditer(content):
- alias = match.group("alias")
- module_path = _resolve_module_path(file_path, match.group("module"))
- imports.setdefault(alias, JSImport(file_path=module_path, original_name="default"))
-
- return imports
-
-
-def _split_import_alias(token: str) -> tuple[str, str]:
- """Split a named import token into original and locally bound names."""
- parts = [part.strip() for part in token.split(" as ", 1)]
- if len(parts) == 2:
- return parts[0], parts[1]
- return token.strip(), token.strip()
+ """Map structural ESM/CommonJS bindings to resolvable local source files."""
+ return {
+ binding.local_name: JSImport(
+ file_path=_resolve_module_path(file_path, binding.module),
+ original_name=binding.imported_name,
+ )
+ for binding in extract_javascript_imports(
+ content,
+ suffix=file_path.suffix,
+ source_name=str(file_path),
+ )
+ }
@lru_cache(maxsize=None)
@@ -183,198 +191,31 @@ def _resolve_module_path(test_file: Path, module_name: str) -> Path | None:
return None
-def _extract_test_blocks(content: str) -> list[JestTestBlock]:
- """Extract supported `test()` and `it()` blocks from one Jest file."""
- blocks: list[JestTestBlock] = []
- pattern = re.compile(r"\b(?:it|test)(?P(?:\.(?:each|only|skip))*)\s*\(")
- cursor = 0
- counter = 1
- while True:
- match = pattern.search(content, cursor)
- if match is None:
- break
- open_paren = content.find("(", match.start())
- close_paren = _find_matching(content, open_paren, "(", ")")
- if close_paren == -1:
- break
- modifiers = match.group("modifiers") or ""
- if ".each" in modifiers:
- invocation_open = content.find("(", close_paren + 1)
- if invocation_open == -1:
- break
- invocation_close = _find_matching(content, invocation_open, "(", ")")
- if invocation_close == -1:
- break
- each_blocks = _parse_each_invocation(
- rows_expression=content[open_paren + 1 : close_paren],
- invocation=content[invocation_open + 1 : invocation_close],
- start_index=counter,
- )
- blocks.extend(each_blocks)
- counter += len(each_blocks)
- cursor = invocation_close + 1
- continue
-
- invocation = content[open_paren + 1 : close_paren]
- parsed = _parse_test_invocation(invocation)
- if parsed is not None:
- title, body = parsed
- identifier = _normalize_test_identifier(title, counter)
- blocks.append(
- JestTestBlock(
- title=title,
- body=body,
- identifier=identifier,
- context={},
- )
- )
- counter += 1
- cursor = close_paren + 1
- return blocks
-
-
-def _parse_each_invocation(
- rows_expression: str,
- invocation: str,
- start_index: int,
+def _extract_test_blocks(
+ content: str,
+ *,
+ suffix: str = ".ts",
+ source_name: str = "",
) -> list[JestTestBlock]:
- """Expand one `test.each(...)` invocation into one block per row."""
- title, cursor = _parse_title_and_cursor(invocation)
- if title is None:
- return []
-
- callback = _parse_callback_signature(invocation, cursor)
- if callback is None:
- return []
-
- parameter_names, body = callback
- rows = _parse_each_rows(rows_expression)
- blocks: list[JestTestBlock] = []
- for offset, row in enumerate(rows, start=0):
- context = _build_each_context(parameter_names, row)
- rendered_title = _render_each_title(title, parameter_names, row, context, start_index + offset)
- blocks.append(
- JestTestBlock(
- title=rendered_title,
- body=body,
- identifier=_normalize_test_identifier(rendered_title, start_index + offset),
- context=context,
- )
+ """Extract Jest tests from structural syntax evidence."""
+ return [
+ JestTestBlock(
+ title=case.title,
+ body=case.body,
+ identifier=_normalize_test_identifier(case.title, index),
+ context=case.context,
+ suffix=suffix,
+ full_name=case.full_name,
)
- return blocks
-
-
-def _parse_title_and_cursor(invocation: str) -> tuple[str | None, int]:
- """Parse the first title string from a Jest invocation payload."""
- start = _skip_whitespace(invocation, 0)
- string_result = _parse_string_literal(invocation, start)
- if string_result is None:
- return None, start
- title, cursor = string_result
- return title.strip(), cursor
-
-
-def _parse_callback_signature(invocation: str, cursor: int) -> tuple[list[str], str] | None:
- """Parse callback parameters and body from an arrow-function invocation."""
- cursor = _skip_spaces(invocation, cursor)
- if cursor >= len(invocation) or invocation[cursor] != ",":
- return None
- cursor = _skip_spaces(invocation, cursor + 1)
-
- if cursor < len(invocation) and invocation[cursor] == "(":
- params_end = _find_matching(invocation, cursor, "(", ")")
- if params_end == -1:
- return None
- parameter_names = [
- name.strip() for name in _split_top_level(invocation[cursor + 1 : params_end], ",") if name.strip()
- ]
- body = _extract_callback_body(invocation, params_end + 1)
- if body is None:
- return None
- return parameter_names, body.strip()
-
- arrow_index = invocation.find("=>", cursor)
- if arrow_index == -1:
- return None
- parameter_name = invocation[cursor:arrow_index].strip()
- body = _extract_callback_body(invocation, arrow_index)
- if body is None:
- return None
- return ([parameter_name] if parameter_name else []), body.strip()
-
-
-def _parse_each_rows(rows_expression: str) -> list[object]:
- """Parse a supported `test.each` rows expression into row payloads."""
- parsed = _parse_literal_expression(rows_expression)
- if isinstance(parsed, list):
- return parsed
- return []
-
-
-def _build_each_context(parameter_names: list[str], row: object) -> dict[str, object]:
- """Map one parsed `test.each` row to callback parameter names."""
- if isinstance(row, dict):
- return dict(row)
- if isinstance(row, list):
- return {
- name: row[index] for index, name in enumerate(parameter_names) if index < len(row)
- }
- if len(parameter_names) == 1:
- return {parameter_names[0]: row}
- return {}
-
-
-def _render_each_title(
- title: str,
- parameter_names: list[str],
- row: object,
- context: dict[str, object],
- index: int,
-) -> str:
- """Render `%` and `$name` placeholders for one `test.each` row."""
- rendered = title
- if isinstance(row, list):
- row_values = [_stringify_value(value) for value in row]
- else:
- row_values = [_stringify_value(context[name]) for name in parameter_names if name in context]
-
- for token in ("%s", "%d", "%i", "%f", "%p", "%j", "%o"):
- while token in rendered and row_values:
- rendered = rendered.replace(token, row_values.pop(0), 1)
-
- for name, value in context.items():
- rendered = rendered.replace(f"${name}", _stringify_value(value))
-
- if rendered == title:
- rendered = f"{title} case {index:02d}"
- return rendered
-
-
-def _parse_test_invocation(invocation: str) -> tuple[str, str] | None:
- """Parse the title and callback body from one Jest invocation payload."""
- start = _skip_whitespace(invocation, 0)
- string_result = _parse_string_literal(invocation, start)
- if string_result is None:
- return None
- title, cursor = string_result
- callback_body = _extract_callback_body(invocation, cursor)
- if callback_body is None:
- return None
- return title.strip(), callback_body.strip()
-
-
-def _skip_whitespace(text: str, cursor: int) -> int:
- """Advance past whitespace and commas."""
- while cursor < len(text) and text[cursor] in " \t\r\n,":
- cursor += 1
- return cursor
-
-
-def _skip_spaces(text: str, cursor: int) -> int:
- """Advance past whitespace without consuming punctuation."""
- while cursor < len(text) and text[cursor] in " \t\r\n":
- cursor += 1
- return cursor
+ for index, case in enumerate(
+ extract_javascript_test_cases(
+ content,
+ suffix=suffix,
+ source_name=source_name,
+ ),
+ start=1,
+ )
+ ]
def _parse_string_literal(text: str, cursor: int) -> tuple[str, int] | None:
@@ -397,22 +238,6 @@ def _parse_string_literal(text: str, cursor: int) -> tuple[str, int] | None:
return None
-def _extract_callback_body(invocation: str, cursor: int) -> str | None:
- """Extract the callback block body from an arrow/function callback."""
- arrow_index = invocation.find("=>", cursor)
- function_index = invocation.find("function", cursor)
- if arrow_index == -1 and function_index == -1:
- return None
- search_from = arrow_index + 2 if arrow_index != -1 else function_index + len("function")
- open_brace = invocation.find("{", search_from)
- if open_brace == -1:
- return None
- close_brace = _find_matching(invocation, open_brace, "{", "}")
- if close_brace == -1:
- return None
- return invocation[open_brace + 1 : close_brace]
-
-
def _find_matching(text: str, open_index: int, open_char: str, close_char: str) -> int:
"""Find the matching closing delimiter while skipping strings and comments."""
depth = 0
@@ -460,12 +285,12 @@ def _infer_subject_location(
imports: dict[str, JSImport],
) -> SubjectLocation:
"""Infer the tested production file and function from imported function calls."""
- for call in _iter_calls_in_order(block.body):
+ for call in _iter_calls_in_order(block.body, suffix=block.suffix):
subject = _subject_from_call(call, imports)
if subject is not None:
return subject
- for nested_call in _iter_calls_in_order(call.arg_text):
+ for nested_call in _iter_calls_in_order(call.arg_text, suffix=block.suffix):
subject = _subject_from_call(nested_call, imports)
if subject is not None:
return subject
@@ -489,17 +314,27 @@ def _subject_from_call(call: CallMatch, imports: dict[str, JSImport]) -> Subject
imported = imports.get(call.callee)
if imported is None or imported.file_path is None:
return None
- symbol_name = imported.original_name if imported.original_name != "default" else call.callee
- return _resolve_symbol_subject(imported.file_path, symbol_name)
+ if imported.original_name == "default":
+ return _resolve_symbol_subject(
+ imported.file_path,
+ "default",
+ method_name=call.callee,
+ )
+ return _resolve_symbol_subject(imported.file_path, imported.original_name)
-def _resolve_symbol_subject(module_path: Path, symbol_name: str) -> SubjectLocation | None:
+def _resolve_symbol_subject(
+ module_path: Path,
+ symbol_name: str,
+ *,
+ method_name: str | None = None,
+) -> SubjectLocation | None:
"""Build a subject descriptor when the imported module really defines the symbol."""
if not _module_defines_symbol(module_path, symbol_name):
return None
return SubjectLocation(
file=module_path.name,
- method=symbol_name,
+ method=method_name or symbol_name,
source_path=_display_path(module_path),
)
@@ -510,34 +345,28 @@ def _module_defines_symbol(module_path: Path, symbol_name: str) -> bool:
if not module_path.exists() or module_path.suffix not in {".ts", ".tsx", ".js", ".jsx"}:
return False
source = module_path.read_text(encoding="utf-8")
- patterns = (
- rf"\bexport\s+(?:async\s+)?function\s+{re.escape(symbol_name)}\b",
- rf"\b(?:export\s+)?(?:const|let|var)\s+{re.escape(symbol_name)}\s*=",
- rf"\bexport\s+class\s+{re.escape(symbol_name)}\b",
- rf"\bclass\s+{re.escape(symbol_name)}\b",
+ return source_defines_symbol(
+ source,
+ symbol_name,
+ suffix=module_path.suffix,
+ source_name=str(module_path),
)
- return any(re.search(pattern, source) for pattern in patterns)
-def _iter_calls_in_order(body: str) -> list[CallMatch]:
+def _iter_calls_in_order(body: str, *, suffix: str = ".ts") -> list[CallMatch]:
"""Return function calls in source order for subject inference and input extraction."""
- calls: list[CallMatch] = []
- pattern = re.compile(r"\b(?:(?P[A-Za-z_$][\w$]*)\.)?(?P[A-Za-z_$][\w$]*)\s*\(")
- cursor = 0
- while True:
- match = pattern.search(body, cursor)
- if match is None:
- break
- open_paren = body.find("(", match.start())
- close_paren = _find_matching(body, open_paren, "(", ")")
- if close_paren == -1:
- break
- namespace = match.group("namespace")
- name = match.group("name")
- callee = f"{namespace}.{name}" if namespace else name
- calls.append(CallMatch(callee=callee, arg_text=body[open_paren + 1 : close_paren], start=match.start()))
- cursor = close_paren + 1
- return calls
+ return [
+ CallMatch(
+ callee=call.callee,
+ arg_text=call.argument_text,
+ start=call.start_byte,
+ )
+ for call in parse_javascript_calls(
+ body,
+ suffix=suffix,
+ source_name="Jest callback",
+ )
+ ]
def _extract_case_inputs(
@@ -545,9 +374,11 @@ def _extract_case_inputs(
imports: dict[str, JSImport],
subject_method: str,
context: dict[str, object],
+ *,
+ suffix: str = ".ts",
) -> dict[str, str]:
"""Extract one testcase input dictionary from the first matching subject call."""
- for call in _iter_calls_in_order(body):
+ for call in _iter_calls_in_order(body, suffix=suffix):
if call.callee == subject_method or call.callee.endswith(f".{subject_method}"):
return _render_inputs_from_call(body, call.arg_text, context)
if call.callee in imports:
@@ -555,7 +386,7 @@ def _extract_case_inputs(
if imported.original_name == subject_method:
return _render_inputs_from_call(body, call.arg_text, context)
- for nested_call in _iter_calls_in_order(call.arg_text):
+ for nested_call in _iter_calls_in_order(call.arg_text, suffix=suffix):
if nested_call.callee == subject_method or nested_call.callee.endswith(
f".{subject_method}"
):
@@ -764,4 +595,4 @@ def _format_expectation(
return [f"{actual} == {rendered_expected}"]
if matcher == "toThrow":
return [f"raises {rendered_expected or 'Error'}"]
- return []
\ No newline at end of file
+ return []
diff --git a/src/teaforge/jest/project.py b/src/teaforge/jest/project.py
new file mode 100644
index 0000000..c88dc1c
--- /dev/null
+++ b/src/teaforge/jest/project.py
@@ -0,0 +1,136 @@
+"""Discover and execute the target project's own Jest installation safely."""
+
+from __future__ import annotations
+
+import json
+import os
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Mapping, Sequence
+
+from teaforge.discovery import discover_files
+from teaforge.jest.parser import TEST_FILE_SUFFIXES
+from teaforge.process import (
+ ProcessResult,
+ bounded_process_detail,
+ run_process,
+)
+
+
+@dataclass(slots=True, frozen=True)
+class JestProject:
+ """Resolved Jest execution context for one test path."""
+
+ root: Path
+ executable: Path
+ test_files: tuple[Path, ...]
+
+ @classmethod
+ def discover(cls, test_path: Path) -> "JestProject":
+ """Resolve test files, config root, and the nearest installed Jest binary."""
+ test_files = tuple(path.resolve() for path in collect_jest_test_files(test_path))
+ if not test_files:
+ raise RuntimeError(f"No executable Jest test files found under: {test_path}")
+ root = discover_jest_project_root(test_path)
+ return cls(
+ root=root,
+ executable=find_local_jest(root),
+ test_files=test_files,
+ )
+
+ def run(
+ self,
+ arguments: Sequence[str],
+ *,
+ timeout_seconds: float,
+ operation: str,
+ environment: Mapping[str, str] | None = None,
+ ) -> ProcessResult:
+ """Run Jest with stable cwd, timeout, and actionable process errors."""
+ command = [str(self.executable), *arguments]
+ try:
+ return run_process(
+ command,
+ cwd=self.root,
+ environment=(
+ dict(environment) if environment is not None else os.environ.copy()
+ ),
+ timeout_seconds=timeout_seconds,
+ operation=operation,
+ )
+ except FileNotFoundError as exc:
+ raise RuntimeError(
+ f"The project-local Jest executable disappeared: {self.executable}. "
+ "Reinstall the target project's dependencies."
+ ) from exc
+
+ def setup_files_after_env(self, *, timeout_seconds: float) -> list[str]:
+ """Read resolved setup files so listener injection preserves project behavior."""
+ result = self.run(
+ ["--showConfig", "--json"],
+ timeout_seconds=timeout_seconds,
+ operation="Reading the Jest configuration",
+ )
+ if result.returncode != 0:
+ detail = bounded_process_detail(result.stderr, result.stdout)
+ raise RuntimeError(
+ "TeaForge could not read the Jest configuration before injecting its listener. "
+ f"Exit code: {result.returncode}. {detail}"
+ )
+ try:
+ payload = json.loads(result.stdout)
+ except json.JSONDecodeError as exc:
+ raise RuntimeError(
+ "Jest --showConfig did not return valid JSON; TeaForge refused to replace "
+ "the project's setupFilesAfterEnv configuration."
+ ) from exc
+
+ setup_files: list[str] = []
+ for config in payload.get("configs", []):
+ for setup_file in config.get("setupFilesAfterEnv", []):
+ normalized = str(setup_file)
+ if normalized and normalized not in setup_files:
+ setup_files.append(normalized)
+ return setup_files
+
+
+def discover_jest_project_root(test_path: Path) -> Path:
+ """Find the nearest directory that owns Jest/package configuration."""
+ start = test_path.resolve() if test_path.is_dir() else test_path.resolve().parent
+ for candidate in (start, *start.parents):
+ if any(
+ (candidate / marker).exists()
+ for marker in (
+ "package.json",
+ "jest.config.js",
+ "jest.config.cjs",
+ "jest.config.mjs",
+ "jest.config.ts",
+ )
+ ):
+ return candidate
+ raise RuntimeError(
+ f"Could not find a Jest project root for: {test_path}. "
+ "TeaForge looks for package.json or jest.config.* in parent directories."
+ )
+
+
+def find_local_jest(project_root: Path) -> Path:
+ """Find the nearest ancestor-installed Jest without invoking npx downloads."""
+ executable_name = "jest.cmd" if os.name == "nt" else "jest"
+ for candidate in (project_root, *project_root.parents):
+ executable = candidate / "node_modules" / ".bin" / executable_name
+ if executable.exists() and executable.is_file():
+ return executable
+ raise RuntimeError(
+ f"No project-local Jest executable found from: {project_root}. "
+ "Install the target project's dependencies before running TeaForge."
+ )
+
+
+def collect_jest_test_files(test_path: Path) -> list[Path]:
+ """Expand a file or directory into exact Jest test paths."""
+ return discover_files(
+ test_path,
+ is_candidate=lambda path: path.name.endswith(TEST_FILE_SUFFIXES),
+ )
diff --git a/src/teaforge/jest/runtime.py b/src/teaforge/jest/runtime.py
new file mode 100644
index 0000000..cb32128
--- /dev/null
+++ b/src/teaforge/jest/runtime.py
@@ -0,0 +1,267 @@
+"""Capture observed Jest assertion evidence without modifying the target project."""
+
+from __future__ import annotations
+
+import json
+import os
+from dataclasses import dataclass
+from importlib import resources
+from pathlib import Path
+
+from teaforge.jest.project import JestProject
+from teaforge.pcl.assembly import build_output_rows
+from teaforge.pcl.models import PCLAssertionEvidence, PCLDocument, PCLTestCase
+from teaforge.process import WorkflowDeadline, bounded_process_detail
+
+EVIDENCE_MODES = ("static", "runtime", "auto")
+EVIDENCE_SCHEMA_VERSION = 1
+MAX_EVIDENCE_FILE_BYTES = 6 * 1024 * 1024
+MAX_EVIDENCE_RECORDS = 6000
+
+
+@dataclass(slots=True, frozen=True)
+class JestAssertionEvidence:
+ test_name: str
+ test_path: str
+ matcher: str
+ expected: list[str]
+ actual: str
+ passed: bool
+ negated: bool = False
+ promise_mode: str = ""
+ error: str = ""
+
+
+@dataclass(slots=True, frozen=True)
+class JestEvidenceRun:
+ evidence: list[JestAssertionEvidence]
+ exit_code: int
+ stdout: str = ""
+ stderr: str = ""
+ warnings: tuple[str, ...] = ()
+
+ @property
+ def tests_passed(self) -> bool:
+ return self.exit_code == 0
+
+
+class JestRuntimeEvidenceUnavailableError(RuntimeError):
+ """Runtime evidence cannot start or the target uses unsupported assertion hooks."""
+
+ code = "jest-runtime-evidence-unavailable"
+
+
+def normalize_evidence_mode(mode: str) -> str:
+ normalized = mode.strip().lower()
+ if normalized not in EVIDENCE_MODES:
+ supported = ", ".join(EVIDENCE_MODES)
+ raise ValueError(f"Unsupported evidence mode: {mode}. Supported values: {supported}")
+ return normalized
+
+
+def capture_jest_assertions(
+ test_path: Path,
+ *,
+ timeout_seconds: int = 120,
+) -> list[JestAssertionEvidence]:
+ """Compatibility wrapper returning only assertions from one Jest evidence run."""
+ return capture_jest_run(
+ test_path,
+ timeout_seconds=timeout_seconds,
+ ).evidence
+
+
+def capture_jest_run(
+ test_path: Path,
+ *,
+ timeout_seconds: int = 120,
+) -> JestEvidenceRun:
+ """Execute project-local Jest and preserve both evidence and process status."""
+ try:
+ project = JestProject.discover(test_path)
+ except RuntimeError as exc:
+ raise JestRuntimeEvidenceUnavailableError(str(exc)) from exc
+ deadline = WorkflowDeadline.start(
+ "Jest runtime evidence workflow",
+ timeout_seconds,
+ )
+ setup_files = project.setup_files_after_env(
+ timeout_seconds=deadline.remaining_seconds()
+ )
+
+ import tempfile
+
+ with tempfile.TemporaryDirectory(prefix="teaforge-jest-evidence-") as temp_dir:
+ evidence_path = Path(temp_dir) / "assertions.jsonl"
+ listener_resource = resources.files("teaforge.jest.assets").joinpath(
+ "runtime-listener.cjs"
+ )
+ with resources.as_file(listener_resource) as listener_path:
+ command = [
+ "--runInBand",
+ "--runTestsByPath",
+ *[str(path) for path in project.test_files],
+ ]
+ for setup_file in [*setup_files, str(listener_path)]:
+ command.extend(["--setupFilesAfterEnv", setup_file])
+ environment = os.environ.copy()
+ environment["TEAFORGE_JEST_EVIDENCE_PATH"] = str(evidence_path)
+ result = project.run(
+ command,
+ timeout_seconds=deadline.remaining_seconds(),
+ operation="Jest runtime evidence",
+ environment=environment,
+ )
+
+ evidence, warnings = load_jest_evidence(evidence_path)
+ if evidence:
+ return JestEvidenceRun(
+ evidence=evidence,
+ exit_code=result.returncode,
+ stdout=result.stdout,
+ stderr=result.stderr,
+ warnings=tuple(warnings),
+ )
+
+ detail = bounded_process_detail(result.stderr, result.stdout)
+ if result.returncode != 0:
+ raise RuntimeError(
+ "Jest failed before TeaForge captured assertion evidence. "
+ f"Exit code: {result.returncode}. {detail}"
+ )
+ raise JestRuntimeEvidenceUnavailableError(
+ "Jest completed but no supported expect() assertions were observed. "
+ "Use --evidence-mode static for tests that import a separate expect implementation."
+ )
+
+
+def enrich_documents_with_runtime_evidence(
+ documents: list[PCLDocument],
+ evidence: list[JestAssertionEvidence],
+) -> int:
+ """Attach observed assertions to statically parsed testcases without replacing design intent."""
+ matched_count = 0
+ for document in documents:
+ document_changed = False
+ for case in document.testcases:
+ matches = [
+ item
+ for item in evidence
+ if _evidence_matches_case(item, case, document.test_file)
+ ]
+ if not matches:
+ continue
+ case.assertion_evidence = [
+ PCLAssertionEvidence(
+ matcher=item.matcher,
+ expected=item.expected,
+ actual=item.actual,
+ passed=item.passed,
+ negated=item.negated,
+ promise_mode=item.promise_mode,
+ error=item.error,
+ )
+ for item in matches
+ ]
+ case.execution_status = (
+ "passed" if all(item.passed for item in matches) else "failed"
+ )
+ case.runtime_test_name = matches[0].test_name
+ case.runtime_test_path = matches[0].test_path
+ document_changed = True
+ matched_count += 1
+ if document_changed:
+ document.output_rows = build_output_rows(document.testcases)
+ return matched_count
+
+
+def load_jest_assertions(path: Path) -> list[JestAssertionEvidence]:
+ """Load assertion records while preserving the historical list-only API."""
+ evidence, _warnings = load_jest_evidence(path)
+ return evidence
+
+
+def load_jest_evidence(
+ path: Path,
+) -> tuple[list[JestAssertionEvidence], list[str]]:
+ """Load bounded, versioned assertion evidence and producer warnings."""
+ if not path.exists():
+ return [], []
+ file_size = path.stat().st_size
+ if file_size > MAX_EVIDENCE_FILE_BYTES:
+ raise RuntimeError(
+ "Jest evidence file is too large to load safely: "
+ f"{file_size} bytes (limit: {MAX_EVIDENCE_FILE_BYTES})."
+ )
+ evidence: list[JestAssertionEvidence] = []
+ warnings: list[str] = []
+ with path.open(encoding="utf-8") as evidence_file:
+ for line_number, raw_line in enumerate(evidence_file, start=1):
+ if line_number > MAX_EVIDENCE_RECORDS:
+ raise RuntimeError(
+ "Jest evidence contains too many records to load safely "
+ f"(limit: {MAX_EVIDENCE_RECORDS})."
+ )
+ if not raw_line.strip():
+ continue
+ try:
+ payload = json.loads(raw_line)
+ except json.JSONDecodeError as exc:
+ raise RuntimeError(
+ f"Invalid Jest evidence JSON on line {line_number}: {exc.msg}"
+ ) from exc
+ if payload.get("schema_version") != EVIDENCE_SCHEMA_VERSION:
+ raise RuntimeError(
+ "Unsupported Jest evidence schema on line "
+ f"{line_number}: {payload.get('schema_version')!r}."
+ )
+ kind = payload.get("kind")
+ if kind == "warning":
+ message = str(payload.get("message", "Jest evidence was truncated."))
+ warnings.append(message)
+ continue
+ if kind != "assertion":
+ raise RuntimeError(
+ f"Unsupported Jest evidence record kind on line {line_number}: {kind!r}."
+ )
+ expected = payload.get("expected", [])
+ if not isinstance(expected, list):
+ raise RuntimeError(
+ f"Invalid Jest evidence expected values on line {line_number}: list required."
+ )
+ evidence.append(
+ JestAssertionEvidence(
+ test_name=str(payload.get("test_name", "")),
+ test_path=str(payload.get("test_path", "")),
+ matcher=str(payload.get("matcher", "")),
+ expected=[str(value) for value in expected],
+ actual=str(payload.get("actual", "")),
+ passed=bool(payload.get("passed", False)),
+ negated=bool(payload.get("negated", False)),
+ promise_mode=str(payload.get("promise_mode", "")),
+ error=str(payload.get("error", "")),
+ )
+ )
+ return evidence, warnings
+
+
+def _evidence_matches_case(
+ evidence: JestAssertionEvidence,
+ testcase: PCLTestCase,
+ document_test_file: str,
+) -> bool:
+ if testcase.test_full_name:
+ name_matches = evidence.test_name == testcase.test_full_name
+ else:
+ name_matches = evidence.test_name.endswith(testcase.testcase)
+ if not name_matches:
+ return False
+ if not evidence.test_path:
+ return True
+ if testcase.test_source_path:
+ return Path(evidence.test_path).resolve() == Path(
+ testcase.test_source_path
+ ).resolve()
+ if document_test_file.startswith("複数"):
+ return True
+ return Path(evidence.test_path).name == document_test_file
diff --git a/src/teaforge/naming.py b/src/teaforge/naming.py
index 2b0326d..5581711 100644
--- a/src/teaforge/naming.py
+++ b/src/teaforge/naming.py
@@ -2,7 +2,12 @@
from __future__ import annotations
+import hashlib
+import os
import re
+from collections import defaultdict
+from pathlib import Path
+from typing import Iterable
def slugify(value: str) -> str:
@@ -15,3 +20,37 @@ def folder_name(value: str) -> str:
"""Create snake_case names for grouping output directories."""
name = re.sub(r"[^a-zA-Z0-9_]+", "_", value).strip("_").lower()
return name or "item"
+
+
+def source_folder_names(source_paths: Iterable[Path]) -> dict[Path, str]:
+ """Build collision-free, mostly human-readable folders for source artifacts."""
+ paths = list(dict.fromkeys(path.resolve() for path in source_paths))
+ grouped: defaultdict[str, list[Path]] = defaultdict(list)
+ for path in paths:
+ grouped[folder_name(path.stem)].append(path)
+
+ result: dict[Path, str] = {}
+ for base_name, related in grouped.items():
+ if len(related) == 1:
+ result[related[0]] = base_name
+ continue
+ try:
+ common_root = Path(os.path.commonpath([str(path) for path in related]))
+ if common_root.suffix:
+ common_root = common_root.parent
+ except ValueError:
+ common_root = None
+
+ used: set[str] = set()
+ for path in related:
+ if common_root is not None:
+ relative = path.with_suffix("").relative_to(common_root)
+ candidate = folder_name("_".join(relative.parts))
+ else:
+ candidate = folder_name(f"{path.parent.name}_{path.stem}")
+ if candidate in used:
+ digest = hashlib.sha256(str(path).encode("utf-8")).hexdigest()[:8]
+ candidate = f"{candidate}_{digest}"
+ used.add(candidate)
+ result[path] = candidate
+ return result
diff --git a/src/teaforge/pcl/__init__.py b/src/teaforge/pcl/__init__.py
index a3c119c..a519861 100644
--- a/src/teaforge/pcl/__init__.py
+++ b/src/teaforge/pcl/__init__.py
@@ -1,7 +1,8 @@
-from .models import PCLDocument, PCLMatrixRow, PCLTestCase
+from .models import PCLAssertionEvidence, PCLDocument, PCLMatrixRow, PCLTestCase
from .service import generate_pcl, get_testcase_description, load_pcl_document
__all__ = [
+ "PCLAssertionEvidence",
"PCLDocument",
"PCLMatrixRow",
"PCLTestCase",
@@ -9,4 +10,3 @@
"load_pcl_document",
"get_testcase_description",
]
-
diff --git a/src/teaforge/pcl/assembly.py b/src/teaforge/pcl/assembly.py
new file mode 100644
index 0000000..e359ffd
--- /dev/null
+++ b/src/teaforge/pcl/assembly.py
@@ -0,0 +1,125 @@
+"""Assemble framework-neutral test evidence into PCL documents and matrices."""
+
+from __future__ import annotations
+
+from collections import OrderedDict
+from dataclasses import replace
+
+from .models import PCLDocument, PCLMatrixRow, PCLTestCase
+
+
+def merge_documents_by_subject(documents: list[PCLDocument]) -> list[PCLDocument]:
+ """Group intermediate documents by their inferred production subject."""
+ grouped: OrderedDict[tuple[str, str, str], list[PCLDocument]] = OrderedDict()
+ for document in documents:
+ key = (document.file, document.method, document.source_path)
+ grouped.setdefault(key, []).append(document)
+
+ return [
+ related[0] if len(related) == 1 else _merge_related_documents(related)
+ for related in grouped.values()
+ ]
+
+
+def build_input_rows(cases: list[PCLTestCase]) -> list[PCLMatrixRow]:
+ """Build the PCL input matrix rows from testcase input values."""
+ row_hits: OrderedDict[tuple[str, str], dict[str, str]] = OrderedDict()
+ for case in cases:
+ for item, value in case.inputs.items():
+ key = (item, value)
+ row_hits.setdefault(key, {})
+ row_hits[key][case.testcasecode] = "○"
+
+ return _build_rows("input", cases, row_hits)
+
+
+def build_output_rows(cases: list[PCLTestCase]) -> list[PCLMatrixRow]:
+ """Build the PCL output matrix rows from testcase assertions."""
+ row_hits: OrderedDict[tuple[str, str], dict[str, str]] = OrderedDict()
+ for case in cases:
+ checks = case.output_checks or ([case.output] if case.output else [])
+ for check in checks:
+ item, value = _split_output_check(check)
+ key = (item, value)
+ row_hits.setdefault(key, {})
+ row_hits[key][case.testcasecode] = "○"
+
+ return _build_rows("output", cases, row_hits)
+
+
+def _merge_related_documents(documents: list[PCLDocument]) -> PCLDocument:
+ """Merge test documents that target the same production subject."""
+ first = documents[0]
+ merged = PCLDocument.create(
+ file=first.file,
+ method=first.method,
+ source_path=first.source_path,
+ test_file=_merge_source_labels([document.test_file for document in documents]),
+ test_method=_merge_source_labels([document.test_method for document in documents]),
+ test_source_path=_merge_source_labels(
+ [document.test_source_path for document in documents]
+ ),
+ )
+ merged.generated_at = first.generated_at
+
+ case_idx = 1
+ for document in documents:
+ for case in document.testcases:
+ merged.testcases.append(replace(case, testcasecode=f"TC-{case_idx:03d}"))
+ case_idx += 1
+
+ merged.input_rows = build_input_rows(merged.testcases)
+ merged.output_rows = build_output_rows(merged.testcases)
+ return merged
+
+
+def _merge_source_labels(values: list[str]) -> str:
+ unique_values = list(dict.fromkeys(value for value in values if value))
+ if not unique_values:
+ return ""
+ if len(unique_values) == 1:
+ return unique_values[0]
+ return f"複数 ({len(unique_values)} 件)"
+
+
+def _build_rows(
+ category: str,
+ cases: list[PCLTestCase],
+ row_hits: OrderedDict[tuple[str, str], dict[str, str]],
+) -> list[PCLMatrixRow]:
+ rows: list[PCLMatrixRow] = []
+ for (item, value), hits in _group_keys_by_item(row_hits).items():
+ rows.append(
+ PCLMatrixRow(
+ category=category,
+ item=item,
+ value=value,
+ values={case.testcasecode: hits.get(case.testcasecode, "") for case in cases},
+ )
+ )
+ return rows
+
+
+def _split_output_check(check: str) -> tuple[str, str]:
+ if check.startswith("raises "):
+ return ("raises", check.removeprefix("raises ").strip())
+ if " == " in check:
+ left, right = check.split(" == ", 1)
+ return (left.strip(), right.strip())
+ return ("assertion", check)
+
+
+def _group_keys_by_item(
+ row_hits: OrderedDict[tuple[str, str], dict[str, str]],
+) -> OrderedDict[tuple[str, str], dict[str, str]]:
+ item_groups: OrderedDict[
+ str, list[tuple[tuple[str, str], dict[str, str]]]
+ ] = OrderedDict()
+ for key, hits in row_hits.items():
+ item_groups.setdefault(key[0], []).append((key, hits))
+
+ grouped: OrderedDict[tuple[str, str], dict[str, str]] = OrderedDict()
+ for entries in item_groups.values():
+ for key, hits in entries:
+ grouped[key] = hits
+ return grouped
diff --git a/src/teaforge/pcl/backends.py b/src/teaforge/pcl/backends.py
index 4c56bd1..3a22902 100644
--- a/src/teaforge/pcl/backends.py
+++ b/src/teaforge/pcl/backends.py
@@ -19,9 +19,17 @@ def normalize_pcl_framework(framework: str) -> str:
return normalized
-def parse_documents_for_framework(test_path: Path, framework: str) -> list[PCLDocument]:
+def parse_documents_for_framework(
+ test_path: Path,
+ framework: str,
+ *,
+ evidence_mode: str = "static",
+ runtime_timeout: int = 120,
+) -> list[PCLDocument]:
"""Route one test path through the selected framework parser."""
normalized = normalize_pcl_framework(framework)
+ if evidence_mode != "static" and normalized != "jest":
+ raise ValueError("Runtime assertion evidence is currently supported only for Jest.")
if normalized == "pytest":
return parse_pytest_documents(test_path)
@@ -36,5 +44,43 @@ def parse_documents_for_framework(test_path: Path, framework: str) -> list[PCLDo
return parse_playwright_documents(test_path)
from teaforge.jest.parser import parse_jest_documents
+ from teaforge.jest.runtime import (
+ JestRuntimeEvidenceUnavailableError,
+ capture_jest_run,
+ enrich_documents_with_runtime_evidence,
+ normalize_evidence_mode,
+ )
- return parse_jest_documents(test_path)
\ No newline at end of file
+ mode = normalize_evidence_mode(evidence_mode)
+ documents = parse_jest_documents(test_path)
+ if mode == "static":
+ return documents
+
+ try:
+ run = capture_jest_run(test_path, timeout_seconds=runtime_timeout)
+ matched_count = enrich_documents_with_runtime_evidence(
+ documents, run.evidence
+ )
+ if matched_count == 0:
+ raise JestRuntimeEvidenceUnavailableError(
+ "Jest produced assertion evidence, but it could not be matched to parsed testcases."
+ )
+ except JestRuntimeEvidenceUnavailableError as exc:
+ if mode == "runtime":
+ raise
+ for document in documents:
+ document.evidence_mode = "static-fallback"
+ document.evidence_warnings.append(str(exc))
+ return documents
+
+ for document in documents:
+ document.evidence_mode = "runtime"
+ document.runtime_exit_code = run.exit_code
+ document.runtime_tests_passed = run.tests_passed
+ document.evidence_warnings.extend(run.warnings)
+ if not run.tests_passed:
+ document.evidence_warnings.append(
+ f"Jest completed with failing tests (exit code {run.exit_code}); "
+ "the report was generated from the captured failure evidence."
+ )
+ return documents
diff --git a/src/teaforge/pcl/classification.py b/src/teaforge/pcl/classification.py
index 07885f1..25c1278 100644
--- a/src/teaforge/pcl/classification.py
+++ b/src/teaforge/pcl/classification.py
@@ -2,7 +2,6 @@
from __future__ import annotations
-
NORMAL_HINTS = ("normal", "success", "valid", "ok", "happy")
BOUNDARY_HINTS = ("boundary", "edge", "limit", "min", "max", "minlength", "maxlength")
EXCEPTION_HINTS = (
diff --git a/src/teaforge/pcl/models.py b/src/teaforge/pcl/models.py
index ac47f3f..90d1051 100644
--- a/src/teaforge/pcl/models.py
+++ b/src/teaforge/pcl/models.py
@@ -6,6 +6,8 @@
from datetime import UTC, datetime
from typing import Any
+PCL_SCHEMA_VERSION = 1
+
@dataclass(slots=True)
class PCLMatrixRow:
@@ -15,6 +17,18 @@ class PCLMatrixRow:
values: dict[str, str]
+@dataclass(slots=True)
+class PCLAssertionEvidence:
+ matcher: str
+ expected: list[str]
+ actual: str
+ passed: bool
+ negated: bool = False
+ promise_mode: str = ""
+ source: str = "jest-runtime"
+ error: str = ""
+
+
@dataclass(slots=True)
class PCLTestCase:
testcase: str
@@ -31,6 +45,12 @@ class PCLTestCase:
navigation_outputs: list[str] = field(default_factory=list)
verification_mode: str = ""
output_checks: list[str] = field(default_factory=list)
+ test_full_name: str = ""
+ test_source_path: str = ""
+ assertion_evidence: list[PCLAssertionEvidence] = field(default_factory=list)
+ execution_status: str = ""
+ runtime_test_name: str = ""
+ runtime_test_path: str = ""
executed_date: str = ""
bug_number: str = ""
@@ -45,6 +65,11 @@ class PCLDocument:
test_method: str
test_source_path: str
generated_at: str
+ schema_version: int = PCL_SCHEMA_VERSION
+ evidence_mode: str = "static"
+ evidence_warnings: list[str] = field(default_factory=list)
+ runtime_exit_code: int | None = None
+ runtime_tests_passed: bool | None = None
sheet_number: int = 1
sheet_count: int = 1
testcases: list[PCLTestCase] = field(default_factory=list)
@@ -72,6 +97,11 @@ def create(
test_method=test_method or method,
test_source_path=test_source_path or source_path,
generated_at=datetime.now(UTC).isoformat(),
+ schema_version=PCL_SCHEMA_VERSION,
+ evidence_mode="static",
+ evidence_warnings=[],
+ runtime_exit_code=None,
+ runtime_tests_passed=None,
sheet_number=1,
sheet_count=1,
)
@@ -83,6 +113,12 @@ def to_dict(self) -> dict[str, Any]:
@classmethod
def from_dict(cls, payload: dict[str, Any]) -> "PCLDocument":
"""Rebuild the nested dataclasses from JSON payloads embedded in reports."""
+ schema_version = int(payload.get("schema_version", 0))
+ if schema_version not in (0, PCL_SCHEMA_VERSION):
+ raise ValueError(
+ f"Unsupported PCL schema version: {schema_version}. "
+ f"This TeaForge release supports version {PCL_SCHEMA_VERSION}."
+ )
return cls(
title=payload["title"],
file=payload["file"],
@@ -92,6 +128,11 @@ def from_dict(cls, payload: dict[str, Any]) -> "PCLDocument":
test_method=payload.get("test_method", payload["method"]),
test_source_path=payload.get("test_source_path", payload["source_path"]),
generated_at=payload["generated_at"],
+ schema_version=PCL_SCHEMA_VERSION,
+ evidence_mode=payload.get("evidence_mode", "static"),
+ evidence_warnings=payload.get("evidence_warnings", []),
+ runtime_exit_code=payload.get("runtime_exit_code"),
+ runtime_tests_passed=payload.get("runtime_tests_passed"),
sheet_number=payload.get("sheet_number", 1),
sheet_count=payload.get("sheet_count", 1),
testcases=[
@@ -110,6 +151,24 @@ def from_dict(cls, payload: dict[str, Any]) -> "PCLDocument":
navigation_outputs=row.get("navigation_outputs", []),
verification_mode=row.get("verification_mode", ""),
output_checks=row.get("output_checks", []),
+ test_full_name=row.get("test_full_name", ""),
+ test_source_path=row.get("test_source_path", ""),
+ assertion_evidence=[
+ PCLAssertionEvidence(
+ matcher=evidence.get("matcher", ""),
+ expected=evidence.get("expected", []),
+ actual=evidence.get("actual", ""),
+ passed=bool(evidence.get("passed", False)),
+ negated=bool(evidence.get("negated", False)),
+ promise_mode=evidence.get("promise_mode", ""),
+ source=evidence.get("source", "jest-runtime"),
+ error=evidence.get("error", ""),
+ )
+ for evidence in row.get("assertion_evidence", [])
+ ],
+ execution_status=row.get("execution_status", ""),
+ runtime_test_name=row.get("runtime_test_name", ""),
+ runtime_test_path=row.get("runtime_test_path", ""),
executed_date=row.get("executed_date", ""),
bug_number=row.get("bug_number", ""),
)
diff --git a/src/teaforge/pcl/parser.py b/src/teaforge/pcl/parser.py
index f1999b7..92d1d4c 100644
--- a/src/teaforge/pcl/parser.py
+++ b/src/teaforge/pcl/parser.py
@@ -10,8 +10,11 @@
from pathlib import Path
from typing import Any
+from teaforge.discovery import discover_files
+
+from .assembly import build_input_rows, build_output_rows, merge_documents_by_subject
from .classification import classify_type
-from .models import PCLDocument, PCLMatrixRow, PCLTestCase
+from .models import PCLDocument, PCLTestCase
KNOWN_FIXTURE_NAMES = {
"cache",
@@ -58,14 +61,14 @@ def parse_pytest_documents(pytest_path: Path) -> list[PCLDocument]:
raw_documents: list[PCLDocument] = []
for file_path in files:
tree = ast.parse(file_path.read_text(encoding="utf-8"), filename=str(file_path))
- imports = _extract_imports(tree)
+ imports = _extract_imports(tree, file_path)
for node in tree.body:
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and node.name.startswith(
"test_"
):
raw_documents.append(_build_document_for_function(file_path, node, imports))
- return _merge_documents_by_subject(raw_documents)
+ return merge_documents_by_subject(raw_documents)
def parse_pytest_path(pytest_path: Path) -> PCLDocument:
@@ -85,8 +88,8 @@ def parse_pytest_path(pytest_path: Path) -> PCLDocument:
merged.testcases.append(replace(case, testcasecode=f"TC-{case_idx:03d}"))
case_idx += 1
- merged.input_rows = _build_input_rows(merged.testcases)
- merged.output_rows = _build_output_rows(merged.testcases)
+ merged.input_rows = build_input_rows(merged.testcases)
+ merged.output_rows = build_output_rows(merged.testcases)
return merged
@@ -119,61 +122,11 @@ def _build_document_for_function(
)
)
- document.input_rows = _build_input_rows(document.testcases)
- document.output_rows = _build_output_rows(document.testcases)
+ document.input_rows = build_input_rows(document.testcases)
+ document.output_rows = build_output_rows(document.testcases)
return document
-def _merge_documents_by_subject(documents: list[PCLDocument]) -> list[PCLDocument]:
- """Group intermediate documents by the inferred production subject."""
- grouped: OrderedDict[tuple[str, str, str], list[PCLDocument]] = OrderedDict()
- for document in documents:
- key = (document.file, document.method, document.source_path)
- grouped.setdefault(key, []).append(document)
-
- merged_documents: list[PCLDocument] = []
- for related_documents in grouped.values():
- if len(related_documents) == 1:
- merged_documents.append(related_documents[0])
- continue
- merged_documents.append(_merge_related_documents(related_documents))
- return merged_documents
-
-
-def _merge_related_documents(documents: list[PCLDocument]) -> PCLDocument:
- """Merge multiple pytest functions that target the same production function."""
- first = documents[0]
- merged = PCLDocument.create(
- file=first.file,
- method=first.method,
- source_path=first.source_path,
- test_file=_merge_source_labels([document.test_file for document in documents]),
- test_method=_merge_source_labels([document.test_method for document in documents]),
- test_source_path=_merge_source_labels([document.test_source_path for document in documents]),
- )
- merged.generated_at = first.generated_at
-
- case_idx = 1
- for document in documents:
- for case in document.testcases:
- merged.testcases.append(replace(case, testcasecode=f"TC-{case_idx:03d}"))
- case_idx += 1
-
- merged.input_rows = _build_input_rows(merged.testcases)
- merged.output_rows = _build_output_rows(merged.testcases)
- return merged
-
-
-def _merge_source_labels(values: list[str]) -> str:
- """Collapse multiple test source labels into one display string."""
- unique_values = _dedupe_preserve_order(value for value in values if value)
- if not unique_values:
- return ""
- if len(unique_values) == 1:
- return unique_values[0]
- return f"複数 ({len(unique_values)} 件)"
-
-
def _infer_subject_location(
file_path: Path,
node: ast.FunctionDef | ast.AsyncFunctionDef,
@@ -304,9 +257,7 @@ def _iter_calls_in_order(
def _collect_test_files(path: Path) -> list[Path]:
"""Expand a file or directory input into pytest-style test files."""
- if path.is_file():
- return [path] if _is_test_file(path) else []
- return sorted(file for file in path.rglob("*.py") if _is_test_file(file))
+ return discover_files(path, is_candidate=_is_test_file)
def _is_test_file(path: Path) -> bool:
@@ -315,26 +266,38 @@ def _is_test_file(path: Path) -> bool:
return name.startswith("test_") or name.endswith("_test.py")
-def _extract_imports(tree: ast.Module) -> dict[str, ImportedSymbol]:
+def _extract_imports(
+ tree: ast.Module,
+ test_file: Path,
+) -> dict[str, ImportedSymbol]:
"""Map imported names in a test module to resolvable source files."""
imports: dict[str, ImportedSymbol] = {}
+ search_roots = _import_search_roots(test_file)
for node in tree.body:
if isinstance(node, ast.Import):
for alias in node.names:
bind_name = alias.asname or alias.name.split(".")[0]
imports[bind_name] = ImportedSymbol(
- file_path=_resolve_module_path(alias.name),
+ file_path=_resolve_module_path(alias.name, search_roots),
original_name=alias.name.rsplit(".", 1)[-1],
)
if isinstance(node, ast.ImportFrom):
module_name = node.module or ""
- module_path = _resolve_module_path(module_name) if module_name else None
+ module_roots = (
+ (_relative_import_root(test_file, node.level),)
+ if node.level
+ else search_roots
+ )
+ module_path = _resolve_module_path(module_name, module_roots)
for alias in node.names:
if alias.name == "*":
continue
bind_name = alias.asname or alias.name
symbol_module_name = f"{module_name}.{alias.name}" if module_name else alias.name
- file_path = _resolve_module_path(symbol_module_name) or module_path
+ file_path = (
+ _resolve_module_path(symbol_module_name, module_roots)
+ or module_path
+ )
imports[bind_name] = ImportedSymbol(
file_path=file_path,
original_name=alias.name,
@@ -343,20 +306,38 @@ def _extract_imports(tree: ast.Module) -> dict[str, ImportedSymbol]:
@lru_cache(maxsize=None)
-def _resolve_module_path(module_name: str) -> Path | None:
- """Resolve an import string to a local module or package path in the workspace."""
- if not module_name:
- return None
- base_path = Path.cwd() / Path(*module_name.split("."))
- module_file = base_path.with_suffix(".py")
- if module_file.exists():
- return module_file
- package_init = base_path / "__init__.py"
- if package_init.exists():
- return package_init
+def _resolve_module_path(
+ module_name: str,
+ search_roots: tuple[Path, ...],
+) -> Path | None:
+ """Resolve imports against the tested project instead of TeaForge's process cwd."""
+ module_parts = tuple(part for part in module_name.split(".") if part)
+ for root in search_roots:
+ base_path = root.joinpath(*module_parts)
+ module_file = base_path.with_suffix(".py") if module_parts else None
+ if module_file is not None and module_file.exists():
+ return module_file.resolve()
+ package_init = base_path / "__init__.py"
+ if package_init.exists():
+ return package_init.resolve()
return None
+def _import_search_roots(test_file: Path) -> tuple[Path, ...]:
+ """Return deterministic roots that can own absolute imports in the target project."""
+ test_parent = test_file.resolve().parent
+ candidates = [test_parent, *test_parent.parents, Path.cwd().resolve()]
+ return tuple(dict.fromkeys(candidates))
+
+
+def _relative_import_root(test_file: Path, level: int) -> Path:
+ """Resolve the filesystem root represented by an ImportFrom relative level."""
+ root = test_file.resolve().parent
+ for _ in range(max(0, level - 1)):
+ root = root.parent
+ return root
+
+
def _resolve_symbol_subject(module_path: Path, symbol_name: str) -> SubjectLocation | None:
"""Build a subject descriptor only when the target module really defines the symbol."""
if not _module_defines_symbol(module_path, symbol_name):
@@ -677,77 +658,6 @@ def _extract_raises_expectation(items: list[ast.withitem]) -> list[str]:
return expectations
-def _build_input_rows(cases: list[PCLTestCase]) -> list[PCLMatrixRow]:
- """Build the PCL input matrix rows from testcase input values."""
- row_hits: OrderedDict[tuple[str, str], dict[str, str]] = OrderedDict()
- for case in cases:
- for item, value in case.inputs.items():
- key = (item, value)
- row_hits.setdefault(key, {})
- row_hits[key][case.testcasecode] = "○"
-
- rows: list[PCLMatrixRow] = []
- for (item, value), hits in _group_keys_by_item(row_hits).items():
- rows.append(
- PCLMatrixRow(
- category="input",
- item=item,
- value=value,
- values={case.testcasecode: hits.get(case.testcasecode, "") for case in cases},
- )
- )
- return rows
-
-
-def _build_output_rows(cases: list[PCLTestCase]) -> list[PCLMatrixRow]:
- """Build the PCL output matrix rows from testcase assertions."""
- row_hits: OrderedDict[tuple[str, str], dict[str, str]] = OrderedDict()
- for case in cases:
- checks = case.output_checks or ([case.output] if case.output else [])
- for check in checks:
- item, value = _split_output_check(check)
- key = (item, value)
- row_hits.setdefault(key, {})
- row_hits[key][case.testcasecode] = "○"
-
- rows: list[PCLMatrixRow] = []
- for (item, value), hits in _group_keys_by_item(row_hits).items():
- rows.append(
- PCLMatrixRow(
- category="output",
- item=item,
- value=value,
- values={case.testcasecode: hits.get(case.testcasecode, "") for case in cases},
- )
- )
- return rows
-
-
-def _split_output_check(check: str) -> tuple[str, str]:
- """Split one textual expectation into a matrix item/value pair."""
- if check.startswith("raises "):
- return ("raises", check.removeprefix("raises ").strip())
- if " == " in check:
- left, right = check.split(" == ", 1)
- return (left.strip(), right.strip())
- return ("assertion", check)
-
-
-def _group_keys_by_item(
- row_hits: OrderedDict[tuple[str, str], dict[str, str]],
-) -> OrderedDict[tuple[str, str], dict[str, str]]:
- """Preserve insertion order while grouping matrix rows by item name."""
- item_groups: OrderedDict[str, list[tuple[tuple[str, str], dict[str, str]]]] = OrderedDict()
- for key, hits in row_hits.items():
- item_groups.setdefault(key[0], []).append((key, hits))
-
- grouped: OrderedDict[tuple[str, str], dict[str, str]] = OrderedDict()
- for entries in item_groups.values():
- for key, hits in entries:
- grouped[key] = hits
- return grouped
-
-
def _evaluate_node(node: ast.AST, context: dict[str, Any]) -> Any:
"""Evaluate a limited subset of AST nodes into plain Python values."""
if isinstance(node, ast.Constant):
@@ -847,6 +757,7 @@ def _to_title(test_name: str) -> str:
def _display_path(path: Path) -> str:
"""Prefer workspace-relative paths when reporting resolved source locations."""
try:
- return str(path.relative_to(Path.cwd()))
+ display_path = path.relative_to(Path.cwd())
except ValueError:
- return str(path)
+ display_path = path
+ return display_path.as_posix()
diff --git a/src/teaforge/pcl/pdf.py b/src/teaforge/pcl/pdf.py
index cc52cf5..dd1cd47 100644
--- a/src/teaforge/pcl/pdf.py
+++ b/src/teaforge/pcl/pdf.py
@@ -2,13 +2,19 @@
from __future__ import annotations
+import io
+from contextlib import redirect_stderr, redirect_stdout
from pathlib import Path
def export_pdf_from_html(html_path: Path, output_pdf: Path) -> None:
"""Export one HTML report to PDF and surface dependency errors as runtime guidance."""
+ if not html_path.is_file():
+ raise FileNotFoundError(f"Input HTML does not exist: {html_path}")
try:
- from weasyprint import HTML
+ diagnostics = io.StringIO()
+ with redirect_stdout(diagnostics), redirect_stderr(diagnostics):
+ from weasyprint import HTML
except ImportError as exc:
raise RuntimeError(
"WeasyPrint is required for PDF export. Install with: pip install -e '.[pdf]'"
diff --git a/src/teaforge/pcl/render.py b/src/teaforge/pcl/render.py
index 1e0b6ef..715ef37 100644
--- a/src/teaforge/pcl/render.py
+++ b/src/teaforge/pcl/render.py
@@ -2,12 +2,14 @@
from __future__ import annotations
-import json
+from importlib import resources
from pathlib import Path
from typing import Any
from jinja2 import Environment, FileSystemLoader, select_autoescape
+from teaforge.artifacts import json_for_html_script
+
from .models import PCLDocument
CASE_SLOTS_PER_SHEET = 25
@@ -15,19 +17,28 @@
def default_template_path() -> Path:
"""Return the built-in PCL template path."""
- return Path(__file__).resolve().parents[3] / "templates" / "pcl.html"
+ return Path(str(resources.files("teaforge.templates").joinpath("pcl.html")))
def render_pcl_html(document: PCLDocument, template_path: Path | None = None) -> str:
"""Render one PCL document and embed its JSON payload for later lookup."""
- template_file = template_path or default_template_path()
- env = Environment(
- loader=FileSystemLoader(str(template_file.parent)),
- autoescape=select_autoescape(["html", "xml"]),
- )
- template = env.get_template(template_file.name)
+ if template_path is None:
+ env = Environment(autoescape=select_autoescape(["html", "xml"]))
+ template = env.from_string(
+ resources.files("teaforge.templates")
+ .joinpath("pcl.html")
+ .read_text(encoding="utf-8")
+ )
+ else:
+ if not template_path.is_file():
+ raise FileNotFoundError(f"PCL template does not exist: {template_path}")
+ env = Environment(
+ loader=FileSystemLoader(str(template_path.parent)),
+ autoescape=select_autoescape(["html", "xml"]),
+ )
+ template = env.get_template(template_path.name)
payload = document.to_dict()
- embedded_json = json.dumps(payload, ensure_ascii=False, indent=2)
+ embedded_json = json_for_html_script(payload)
case_columns = [
{
"index": index,
@@ -42,6 +53,11 @@ def render_pcl_html(document: PCLDocument, template_path: Path | None = None) ->
"ui_outputs": case.get("ui_outputs", []),
"navigation_outputs": case.get("navigation_outputs", []),
"verification_mode": case.get("verification_mode", ""),
+ "test_full_name": case.get("test_full_name", ""),
+ "assertion_evidence": case.get("assertion_evidence", []),
+ "execution_status": case.get("execution_status", ""),
+ "runtime_test_name": case.get("runtime_test_name", ""),
+ "runtime_test_path": case.get("runtime_test_path", ""),
"executed_date": case.get("executed_date", ""),
"bug_number": case.get("bug_number", ""),
}
@@ -64,6 +80,7 @@ def render_pcl_html(document: PCLDocument, template_path: Path | None = None) ->
or case["navigation_outputs"]
or case["verification_mode"]
]
+ runtime_cases = [case for case in case_columns if case["assertion_evidence"]]
return template.render(
document=payload,
embedded_json=embedded_json,
@@ -72,6 +89,7 @@ def render_pcl_html(document: PCLDocument, template_path: Path | None = None) ->
input_rows=input_rows,
output_rows=output_rows,
frontend_cases=frontend_cases,
+ runtime_cases=runtime_cases,
)
diff --git a/src/teaforge/pcl/service.py b/src/teaforge/pcl/service.py
index aed90ea..3c966f2 100644
--- a/src/teaforge/pcl/service.py
+++ b/src/teaforge/pcl/service.py
@@ -8,7 +8,8 @@
from math import ceil
from pathlib import Path
-from teaforge.naming import folder_name, slugify
+from teaforge.artifacts import write_text_atomic
+from teaforge.naming import slugify, source_folder_names
from .backends import parse_documents_for_framework
from .models import PCLDocument, PCLMatrixRow
@@ -23,31 +24,49 @@ def generate_pcl(
json_output: Path | None = None,
template_path: Path | None = None,
framework: str = "pytest",
+ evidence_mode: str = "static",
+ runtime_timeout: int = 120,
) -> list[tuple[Path, Path]]:
"""Generate HTML and JSON outputs for each logical PCL document."""
if not test_path.exists():
raise FileNotFoundError(f"Input path does not exist: {test_path}")
- documents = parse_documents_for_framework(test_path, framework)
+ documents = parse_documents_for_framework(
+ test_path,
+ framework,
+ evidence_mode=evidence_mode,
+ runtime_timeout=runtime_timeout,
+ )
sheet_documents = _split_documents(documents, MAX_CASES_PER_SHEET)
html_output.parent.mkdir(parents=True, exist_ok=True)
- outputs: list[tuple[Path, Path]] = []
+ prepared_outputs: list[tuple[Path, Path, str, str]] = []
multiple_outputs = len(sheet_documents) > 1
+ source_folders = source_folder_names(
+ Path(document.source_path) for document in sheet_documents
+ )
for document in sheet_documents:
html_path, json_path = _resolve_output_paths(
document=document,
base_html=html_output,
base_json=json_output,
multiple_outputs=multiple_outputs,
+ output_folder=source_folders[Path(document.source_path).resolve()],
)
html_content = render_pcl_html(document, template_path=template_path)
- html_path.write_text(html_content, encoding="utf-8")
- json_path.parent.mkdir(parents=True, exist_ok=True)
- json_path.write_text(
- json.dumps(document.to_dict(), ensure_ascii=False, indent=2),
- encoding="utf-8",
+ json_content = json.dumps(
+ document.to_dict(),
+ ensure_ascii=False,
+ indent=2,
+ )
+ prepared_outputs.append(
+ (html_path, json_path, html_content, json_content)
)
+
+ outputs: list[tuple[Path, Path]] = []
+ for html_path, json_path, html_content, json_content in prepared_outputs:
+ write_text_atomic(html_path, html_content)
+ write_text_atomic(json_path, json_content)
outputs.append((html_path, json_path))
return outputs
@@ -105,6 +124,11 @@ def _split_documents(documents: list[PCLDocument], chunk_size: int) -> list[PCLD
test_method=document.test_method,
test_source_path=document.test_source_path,
generated_at=document.generated_at,
+ schema_version=document.schema_version,
+ evidence_mode=document.evidence_mode,
+ evidence_warnings=list(document.evidence_warnings),
+ runtime_exit_code=document.runtime_exit_code,
+ runtime_tests_passed=document.runtime_tests_passed,
sheet_number=sheet_number,
sheet_count=total_sheets,
testcases=[replace(case) for case in cases],
@@ -138,6 +162,7 @@ def _resolve_output_paths(
base_html: Path,
base_json: Path | None,
multiple_outputs: bool,
+ output_folder: str,
) -> tuple[Path, Path]:
"""Resolve the HTML and JSON output paths for one generated PCL document."""
# Multi-document runs are grouped by production file so one tested module owns one folder.
@@ -147,12 +172,12 @@ def _resolve_output_paths(
json_path.parent.mkdir(parents=True, exist_ok=True)
return html_path, json_path
- output_dir = base_html.parent / folder_name(Path(document.file).stem)
+ output_dir = base_html.parent / output_folder
output_dir.mkdir(parents=True, exist_ok=True)
suffix = _build_output_suffix(document)
html_path = output_dir / f"{base_html.stem}-{suffix}{base_html.suffix}"
if base_json is not None:
- json_dir = base_json.parent / folder_name(Path(document.file).stem)
+ json_dir = base_json.parent / output_folder
json_dir.mkdir(parents=True, exist_ok=True)
json_path = json_dir / f"{base_json.stem}-{suffix}{base_json.suffix}"
else:
@@ -202,5 +227,14 @@ def _format_case_description(case) -> str:
lines.append(f"遷移確認: {'; '.join(case.navigation_outputs)}")
if getattr(case, "verification_mode", ""):
lines.append(f"確認方法: {case.verification_mode}")
+ if getattr(case, "assertion_evidence", None):
+ lines.append(f"実行結果: {case.execution_status}")
+ for evidence in case.assertion_evidence:
+ expected = ", ".join(evidence.expected) or "(none)"
+ lines.append(
+ "実行証拠: "
+ f"{evidence.matcher} expected={expected} actual={evidence.actual} "
+ f"passed={str(evidence.passed).lower()}"
+ )
lines.append(f"期待結果: {case.output}")
return "\n".join(lines)
diff --git a/src/teaforge/playwright/parser.py b/src/teaforge/playwright/parser.py
index 8899e26..fd0ee68 100644
--- a/src/teaforge/playwright/parser.py
+++ b/src/teaforge/playwright/parser.py
@@ -6,10 +6,18 @@
from pathlib import Path
from urllib.parse import parse_qsl, urlsplit
-from teaforge.jest.parser import JestTestBlock, _collect_test_files, _extract_test_blocks, _find_matching, _parse_literal_expression, _stringify_value
+from teaforge.jest.parser import (
+ JestTestBlock,
+ _collect_test_files,
+ _extract_test_blocks,
+ _find_matching,
+ _parse_literal_expression,
+ _stringify_value,
+)
+from teaforge.pcl.assembly import build_input_rows, build_output_rows, merge_documents_by_subject
from teaforge.pcl.classification import classify_type
from teaforge.pcl.models import PCLDocument, PCLTestCase
-from teaforge.pcl.parser import _build_input_rows, _build_output_rows, _display_path, _merge_documents_by_subject
+from teaforge.pcl.parser import _display_path
def parse_playwright_documents(test_path: Path) -> list[PCLDocument]:
@@ -21,13 +29,20 @@ def parse_playwright_documents(test_path: Path) -> list[PCLDocument]:
raw_documents: list[PCLDocument] = []
for file_path in files:
content = file_path.read_text(encoding="utf-8")
- for index, block in enumerate(_extract_test_blocks(content), start=1):
+ for index, block in enumerate(
+ _extract_test_blocks(
+ content,
+ suffix=file_path.suffix,
+ source_name=str(file_path),
+ ),
+ start=1,
+ ):
raw_documents.append(_build_document_for_block(file_path, block, index))
if not raw_documents:
raise ValueError(f"No supported Playwright tests found under: {test_path}")
- return _merge_documents_by_subject(raw_documents)
+ return merge_documents_by_subject(raw_documents)
def _build_document_for_block(file_path: Path, block: JestTestBlock, index: int) -> PCLDocument:
@@ -61,10 +76,11 @@ def _build_document_for_block(file_path: Path, block: JestTestBlock, index: int)
navigation_outputs=navigation_outputs,
verification_mode="playwright",
output_checks=output_checks,
+ test_full_name=block.full_name or block.title,
)
)
- document.input_rows = _build_input_rows(document.testcases)
- document.output_rows = _build_output_rows(document.testcases)
+ document.input_rows = build_input_rows(document.testcases)
+ document.output_rows = build_output_rows(document.testcases)
return document
diff --git a/src/teaforge/process.py b/src/teaforge/process.py
new file mode 100644
index 0000000..c8807f3
--- /dev/null
+++ b/src/teaforge/process.py
@@ -0,0 +1,454 @@
+"""Run external tools with bounded output, deadlines, and process-tree cleanup."""
+
+from __future__ import annotations
+
+import os
+import re
+import signal
+import subprocess
+import threading
+import time
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Callable, Mapping, Sequence
+
+DEFAULT_PROCESS_DETAIL_LIMIT = 8_000
+DEFAULT_PROCESS_OUTPUT_LIMIT = 4 * 1024 * 1024
+DEFAULT_TERMINATE_GRACE_SECONDS = 2.0
+_OUTPUT_MARKER_RESERVE = 128
+_ANSI_ESCAPE = re.compile(r"\x1b(?:\[[0-?]*[ -/]*[@-~]|\][^\x07]*(?:\x07|\x1b\\))")
+
+
+@dataclass(slots=True, frozen=True)
+class ProcessResult:
+ """One completed external process with memory-bounded captured streams."""
+
+ args: tuple[str, ...]
+ returncode: int
+ stdout: str
+ stderr: str
+ duration_seconds: float
+ stdout_truncated: bool = False
+ stderr_truncated: bool = False
+
+
+class ProcessTimeoutError(RuntimeError):
+ """A workflow deadline expired after TeaForge cleaned up the process tree."""
+
+ code = "process-timeout"
+
+ def __init__(
+ self,
+ *,
+ operation: str,
+ timeout_seconds: float,
+ result: ProcessResult,
+ ) -> None:
+ self.operation = operation
+ self.timeout_seconds = timeout_seconds
+ self.result = result
+ super().__init__(f"{operation} timed out after {timeout_seconds:g} seconds.")
+
+
+@dataclass(slots=True, frozen=True)
+class WorkflowDeadline:
+ """Share one monotonic timeout budget across every process in a workflow."""
+
+ operation: str
+ timeout_seconds: float
+ started_at: float
+ _clock: Callable[[], float] = field(repr=False, compare=False)
+
+ @classmethod
+ def start(
+ cls,
+ operation: str,
+ timeout_seconds: float,
+ *,
+ clock: Callable[[], float] | None = None,
+ ) -> "WorkflowDeadline":
+ if timeout_seconds <= 0:
+ raise ValueError("Workflow timeout must be greater than zero seconds.")
+ selected_clock = clock or time.monotonic
+ return cls(
+ operation=operation,
+ timeout_seconds=timeout_seconds,
+ started_at=selected_clock(),
+ _clock=selected_clock,
+ )
+
+ def remaining_seconds(self) -> float:
+ """Return the remaining budget or fail before another process is started."""
+ elapsed = max(self._clock() - self.started_at, 0.0)
+ remaining = self.timeout_seconds - elapsed
+ if remaining > 0:
+ return remaining
+ raise ProcessTimeoutError(
+ operation=self.operation,
+ timeout_seconds=self.timeout_seconds,
+ result=ProcessResult(
+ args=(),
+ returncode=-1,
+ stdout="",
+ stderr="",
+ duration_seconds=elapsed,
+ ),
+ )
+
+
+class _BoundedByteCapture:
+ """Drain a byte stream while retaining only a diagnostic head and tail."""
+
+ def __init__(self, limit: int) -> None:
+ if limit < 1024:
+ raise ValueError("Process output limit must be at least 1024 bytes.")
+ storage_limit = limit - _OUTPUT_MARKER_RESERVE
+ self._limit = limit
+ self._head_limit = storage_limit // 3
+ self._tail_limit = storage_limit - self._head_limit
+ self._head = bytearray()
+ self._tail = bytearray()
+ self._total = 0
+
+ @property
+ def truncated(self) -> bool:
+ return self._total > self._head_limit + self._tail_limit
+
+ def feed(self, chunk: bytes) -> None:
+ if not chunk:
+ return
+ self._total += len(chunk)
+ head_space = self._head_limit - len(self._head)
+ if head_space > 0:
+ self._head.extend(chunk[:head_space])
+ chunk = chunk[head_space:]
+ if chunk:
+ self._tail.extend(chunk)
+ if len(self._tail) > self._tail_limit:
+ del self._tail[: len(self._tail) - self._tail_limit]
+
+ def text(self) -> str:
+ if not self.truncated:
+ captured = bytes(self._head + self._tail)
+ else:
+ omitted = self._total - len(self._head) - len(self._tail)
+ marker = f"\n... [process output truncated {omitted} bytes] ...\n".encode()
+ captured = bytes(self._head) + marker + bytes(self._tail)
+ return (
+ captured[: self._limit]
+ .decode("utf-8", errors="replace")
+ .replace("\r\n", "\n")
+ .replace("\r", "\n")
+ )
+
+
+def run_process(
+ command: Sequence[str | os.PathLike[str]],
+ *,
+ operation: str,
+ timeout_seconds: float,
+ cwd: Path | str | None = None,
+ environment: Mapping[str, str] | None = None,
+ output_limit: int = DEFAULT_PROCESS_OUTPUT_LIMIT,
+ terminate_grace_seconds: float = DEFAULT_TERMINATE_GRACE_SECONDS,
+) -> ProcessResult:
+ """Run one command without a shell and clean up its process tree on timeout."""
+ if not command:
+ raise ValueError("Process command must not be empty.")
+ if timeout_seconds <= 0:
+ raise ValueError("Process timeout must be greater than zero seconds.")
+ if terminate_grace_seconds < 0:
+ raise ValueError("Process terminate grace period must not be negative.")
+
+ normalized_command = tuple(os.fspath(part) for part in command)
+ stdout_capture = _BoundedByteCapture(output_limit)
+ stderr_capture = _BoundedByteCapture(output_limit)
+ popen_options: dict[str, object] = {}
+ if os.name == "nt":
+ popen_options["creationflags"] = subprocess.CREATE_NEW_PROCESS_GROUP
+ else:
+ popen_options["start_new_session"] = True
+
+ started = time.monotonic()
+ process = subprocess.Popen(
+ normalized_command,
+ cwd=os.fspath(cwd) if cwd is not None else None,
+ env=dict(environment) if environment is not None else None,
+ stdin=subprocess.DEVNULL,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ **popen_options,
+ )
+ job_handle = _assign_windows_kill_job(process) if os.name == "nt" else None
+ readers = (
+ threading.Thread(
+ target=_drain_stream,
+ args=(process.stdout, stdout_capture),
+ name="teaforge-stdout",
+ daemon=True,
+ ),
+ threading.Thread(
+ target=_drain_stream,
+ args=(process.stderr, stderr_capture),
+ name="teaforge-stderr",
+ daemon=True,
+ ),
+ )
+ for reader in readers:
+ reader.start()
+
+ timed_out = False
+ deadline = started + timeout_seconds
+ try:
+ process.wait(timeout=timeout_seconds)
+ except subprocess.TimeoutExpired:
+ timed_out = True
+ if not timed_out:
+ for reader in readers:
+ reader.join(timeout=max(deadline - time.monotonic(), 0.0))
+ timed_out = any(reader.is_alive() for reader in readers)
+
+ if timed_out:
+ _terminate_process_tree(process, job_handle)
+ _wait_for_exit(process, terminate_grace_seconds)
+ for reader in readers:
+ reader.join(timeout=terminate_grace_seconds)
+ if process.poll() is None or any(reader.is_alive() for reader in readers):
+ _kill_process_tree(process, job_handle)
+ _wait_for_exit(process, terminate_grace_seconds)
+ for stream in (process.stdout, process.stderr):
+ if stream is not None and not stream.closed:
+ stream.close()
+ for reader in readers:
+ reader.join(timeout=terminate_grace_seconds)
+
+ _close_windows_job(job_handle)
+
+ result = ProcessResult(
+ args=normalized_command,
+ returncode=process.returncode,
+ stdout=stdout_capture.text(),
+ stderr=stderr_capture.text(),
+ duration_seconds=time.monotonic() - started,
+ stdout_truncated=stdout_capture.truncated,
+ stderr_truncated=stderr_capture.truncated,
+ )
+ if timed_out:
+ raise ProcessTimeoutError(
+ operation=operation,
+ timeout_seconds=timeout_seconds,
+ result=result,
+ )
+ return result
+
+
+def _drain_stream(stream, capture: _BoundedByteCapture) -> None:
+ if stream is None:
+ return
+ try:
+ while True:
+ chunk = stream.read(64 * 1024)
+ if not chunk:
+ return
+ capture.feed(chunk)
+ finally:
+ stream.close()
+
+
+def _wait_for_exit(process: subprocess.Popen[bytes], timeout_seconds: float) -> None:
+ if process.poll() is not None:
+ return
+ try:
+ process.wait(timeout=timeout_seconds)
+ except subprocess.TimeoutExpired:
+ pass
+
+
+def _terminate_process_tree(process: subprocess.Popen[bytes], job_handle) -> None:
+ if os.name == "nt":
+ if job_handle is not None:
+ _terminate_windows_job(job_handle, exit_code=1)
+ return
+ if process.poll() is not None:
+ return
+ if _taskkill_windows_tree(process.pid):
+ return
+ try:
+ process.terminate()
+ except OSError:
+ pass
+ return
+ try:
+ os.killpg(process.pid, signal.SIGTERM)
+ except ProcessLookupError:
+ pass
+
+
+def _kill_process_tree(process: subprocess.Popen[bytes], job_handle) -> None:
+ if os.name == "nt" and job_handle is not None:
+ _terminate_windows_job(job_handle, exit_code=1)
+ return
+ if os.name == "nt":
+ if process.poll() is not None:
+ return
+ if _taskkill_windows_tree(process.pid):
+ return
+ try:
+ process.kill()
+ except OSError:
+ pass
+ return
+ try:
+ os.killpg(process.pid, signal.SIGKILL)
+ except ProcessLookupError:
+ pass
+
+
+def _taskkill_windows_tree(process_id: int) -> bool:
+ """Use Windows' built-in tree termination when Job Object assignment is unavailable."""
+ if os.name != "nt":
+ return False
+ try:
+ cleanup = subprocess.Popen(
+ ["taskkill", "/PID", str(process_id), "/T", "/F"],
+ stdin=subprocess.DEVNULL,
+ stdout=subprocess.DEVNULL,
+ stderr=subprocess.DEVNULL,
+ creationflags=subprocess.CREATE_NO_WINDOW,
+ )
+ return cleanup.wait(timeout=5) == 0
+ except (OSError, subprocess.TimeoutExpired):
+ return False
+
+
+def _assign_windows_kill_job(process: subprocess.Popen[bytes]):
+ """Assign a Windows process to a kill-on-close Job Object when available."""
+ if os.name != "nt":
+ return None
+ try:
+ import ctypes
+ from ctypes import wintypes
+
+ kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
+ kernel32.CreateJobObjectW.argtypes = [ctypes.c_void_p, wintypes.LPCWSTR]
+ kernel32.CreateJobObjectW.restype = wintypes.HANDLE
+ kernel32.SetInformationJobObject.argtypes = [
+ wintypes.HANDLE,
+ ctypes.c_int,
+ ctypes.c_void_p,
+ wintypes.DWORD,
+ ]
+ kernel32.SetInformationJobObject.restype = wintypes.BOOL
+ kernel32.AssignProcessToJobObject.argtypes = [
+ wintypes.HANDLE,
+ wintypes.HANDLE,
+ ]
+ kernel32.AssignProcessToJobObject.restype = wintypes.BOOL
+ kernel32.CloseHandle.argtypes = [wintypes.HANDLE]
+ kernel32.CloseHandle.restype = wintypes.BOOL
+ job = kernel32.CreateJobObjectW(None, None)
+ if not job:
+ return None
+
+ class JOBOBJECT_BASIC_LIMIT_INFORMATION(ctypes.Structure):
+ _fields_ = [
+ ("PerProcessUserTimeLimit", ctypes.c_longlong),
+ ("PerJobUserTimeLimit", ctypes.c_longlong),
+ ("LimitFlags", wintypes.DWORD),
+ ("MinimumWorkingSetSize", ctypes.c_size_t),
+ ("MaximumWorkingSetSize", ctypes.c_size_t),
+ ("ActiveProcessLimit", wintypes.DWORD),
+ ("Affinity", ctypes.c_size_t),
+ ("PriorityClass", wintypes.DWORD),
+ ("SchedulingClass", wintypes.DWORD),
+ ]
+
+ class IO_COUNTERS(ctypes.Structure):
+ _fields_ = [(name, ctypes.c_ulonglong) for name in (
+ "ReadOperationCount",
+ "WriteOperationCount",
+ "OtherOperationCount",
+ "ReadTransferCount",
+ "WriteTransferCount",
+ "OtherTransferCount",
+ )]
+
+ class JOBOBJECT_EXTENDED_LIMIT_INFORMATION(ctypes.Structure):
+ _fields_ = [
+ ("BasicLimitInformation", JOBOBJECT_BASIC_LIMIT_INFORMATION),
+ ("IoInfo", IO_COUNTERS),
+ ("ProcessMemoryLimit", ctypes.c_size_t),
+ ("JobMemoryLimit", ctypes.c_size_t),
+ ("PeakProcessMemoryUsed", ctypes.c_size_t),
+ ("PeakJobMemoryUsed", ctypes.c_size_t),
+ ]
+
+ information = JOBOBJECT_EXTENDED_LIMIT_INFORMATION()
+ information.BasicLimitInformation.LimitFlags = 0x00002000
+ configured = kernel32.SetInformationJobObject(
+ job,
+ 9,
+ ctypes.byref(information),
+ ctypes.sizeof(information),
+ )
+ assigned = kernel32.AssignProcessToJobObject(job, int(process._handle))
+ if not configured or not assigned:
+ kernel32.CloseHandle(job)
+ return None
+ return job
+ except Exception:
+ return None
+
+
+def _close_windows_job(job_handle) -> None:
+ if os.name != "nt" or job_handle is None:
+ return
+ try:
+ import ctypes
+ from ctypes import wintypes
+
+ kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
+ kernel32.CloseHandle.argtypes = [wintypes.HANDLE]
+ kernel32.CloseHandle.restype = wintypes.BOOL
+ kernel32.CloseHandle(job_handle)
+ except Exception:
+ pass
+
+
+def _terminate_windows_job(job_handle, *, exit_code: int) -> None:
+ if os.name != "nt" or job_handle is None:
+ return
+ try:
+ import ctypes
+ from ctypes import wintypes
+
+ kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
+ kernel32.TerminateJobObject.argtypes = [wintypes.HANDLE, wintypes.UINT]
+ kernel32.TerminateJobObject.restype = wintypes.BOOL
+ kernel32.TerminateJobObject(
+ job_handle,
+ exit_code,
+ )
+ except Exception:
+ pass
+
+
+def bounded_process_detail(
+ stderr: str | None,
+ stdout: str | None,
+ *,
+ limit: int = DEFAULT_PROCESS_DETAIL_LIMIT,
+) -> str:
+ """Select stderr-first diagnostics, remove terminal control text, and cap size."""
+ if limit < 128:
+ raise ValueError("Process detail limit must be at least 128 characters.")
+ detail = (stderr or stdout or "").strip().replace("\x00", "")
+ detail = _ANSI_ESCAPE.sub("", detail)
+ if len(detail) <= limit:
+ return detail
+
+ marker = f"\n... [truncated {len(detail) - limit} characters] ...\n"
+ remaining = max(limit - len(marker), 2)
+ head_length = remaining // 3
+ tail_length = remaining - head_length
+ return f"{detail[:head_length]}{marker}{detail[-tail_length:]}"
diff --git a/src/teaforge/templates/__init__.py b/src/teaforge/templates/__init__.py
new file mode 100644
index 0000000..4b2ed4b
--- /dev/null
+++ b/src/teaforge/templates/__init__.py
@@ -0,0 +1 @@
+"""Bundled HTML templates for TeaForge reports."""
diff --git a/src/teaforge/templates/coverage_report.html b/src/teaforge/templates/coverage_report.html
new file mode 100644
index 0000000..ecde7ab
--- /dev/null
+++ b/src/teaforge/templates/coverage_report.html
@@ -0,0 +1,392 @@
+
+
+
+
+
+
+ {{ document.title }} - {{ document.file }}
+
+
+
+
+
+
+ {{ document.title }}
+ Design by TeaForge.🍵
+
+
+
+
+
+ 対象ファイル
+ {{ document.file }}
+ ページ
+ 1 / {{ document.page_count }}
+
+
+ ソース
+ {{ document.source_path }}
+ テスト入力
+ {{ document.test_path }}
+
+
+ 生成日時
+ {{ document.generated_at }}
+ 証拠ソース
+ {{ document.evidence_source }}
+
+
+
+
+
+
+
+ C0 定義
+ {{ document.c0_definition }}
+
+
+ C1 定義
+ {{ document.c1_definition }}
+
+
+
+
+ カバレッジ概要
+
+
+
+ 指標
+ 割合
+ Covered
+ Total
+ Missing
+
+
+
+
+ C0 (行カバレッジ)
+ {% if document.c0.applicable %}{{ "%.1f"|format(document.c0.percent) }}%{% else %}N/A{% endif %}
+ {{ document.c0.covered }}
+ {{ document.c0.total }}
+ {{ document.c0.missing }}
+
+
+ C1 (分岐カバレッジ)
+ {% if document.c1.applicable %}{{ "%.1f"|format(document.c1.percent) }}%{% else %}N/A{% endif %}
+ {{ document.c1.covered }}
+ {{ document.c1.total }}
+ {{ document.c1.missing }}
+
+
+
+
+ 関数別ページ一覧
+
+
+
+ No.
+ 関数
+ 行範囲
+ C0
+ C1
+ ページ
+ Mermaid
+
+
+
+ {% for function in document.functions %}
+
+ {{ loop.index }}
+ {{ function.name }}
+ {{ function.lineno }} - {{ function.end_lineno }}
+ {% if function.c0.applicable %}{{ "%.1f"|format(function.c0.percent) }}%{% else %}N/A{% endif %}
+ {% if function.c1.applicable %}{{ "%.1f"|format(function.c1.percent) }}%{% else %}N/A{% endif %}
+
+ {% if function.diagrams %}
+ {% for diagram in function.diagrams %}{{ diagram.page_number }}{% if not loop.last %}, {% endif %}{% endfor %}
+ {% elif function.page_number %}{{ function.page_number }}{% else %}-{% endif %}
+
+
+ {% if function.diagrams %}
+ {% for diagram in function.diagrams %}{{ diagram.kind }}: {{ diagram.mermaid_path }}{% if not loop.last %}
{% endif %}{% endfor %}
+ {% elif function.diagram_included %}{{ function.mermaid_path }}{% else %}Not requested{% endif %}
+
+
+ {% endfor %}
+
+
+
+ {% if not document.requested_functions %}
+ 業務図
+
+
+
+ 状態
+ このレポートはカバレッジ概要のみです。必要な業務関数だけ `--function` を指定してフロー図を、`--sequence` を指定してシーケンス図を追加してください。
+
+
+
+ {% endif %}
+
+
+
+
+ {% for function in document.functions %}
+ {% for diagram in function.diagrams %}
+ {% if diagram.kind == "sequence" %}
+
+ {% else %}
+
+ {% endif %}
+
+ {{ document.title }} - {{ function.name.split('.')[-1] }}
+ Design by TeaForge.🍵
+
+
+
+
+
+ 対象ファイル
+ {{ document.file }}
+ ページ
+ {{ diagram.page_number }} / {{ document.page_count }}
+
+
+ 関数
+ {{ function.name }}
+ 行範囲
+ {{ function.lineno }} - {{ function.end_lineno }}
+
+
+ Mermaid
+ {{ diagram.mermaid_path }}
+ SVG
+ {{ diagram.svg_path }}
+
+
+
+
+ 関数別カバレッジ
+
+
+
+ 指標
+ 割合
+ Covered
+ Total
+ Missing
+
+
+
+
+ C0 (行カバレッジ)
+ {% if function.c0.applicable %}{{ "%.1f"|format(function.c0.percent) }}%{% else %}N/A{% endif %}
+ {{ function.c0.covered }}
+ {{ function.c0.total }}
+ {{ function.c0.missing }}
+
+
+ C1 (分岐カバレッジ)
+ {% if function.c1.applicable %}{{ "%.1f"|format(function.c1.percent) }}%{% else %}N/A{% endif %}
+ {{ function.c1.covered }}
+ {{ function.c1.total }}
+ {{ function.c1.missing }}
+
+
+
+
+ 未カバー詳細
+
+
+
+ 未カバー行
+
+ {% if function.missing_lines %}
+ {{ function.missing_lines|join(", ") }}
+ {% else %}
+ なし
+ {% endif %}
+
+
+
+ 未カバー分岐
+
+ {% if function.missing_branches %}
+ {{ function.missing_branches|join(", ") }}
+ {% else %}
+ なし
+ {% endif %}
+
+
+
+
+
+ {% if diagram.kind == "sequence" %}シーケンス図{% else %}ロジックフロー図{% endif %}
+
+
+
+
+
+
+ {% endfor %}
+ {% endfor %}
+
+
+
+
+
diff --git a/src/teaforge/templates/pcl.html b/src/teaforge/templates/pcl.html
new file mode 100644
index 0000000..5b00223
--- /dev/null
+++ b/src/teaforge/templates/pcl.html
@@ -0,0 +1,460 @@
+
+
+
+
+
+
+ {{ document.title }}
+
+
+
+
+
+
+ {{ document.title }}
+ Design by TeaForge.🍵
+
+
+
+
+
+ 対象ファイル
+ {{ document.file }}
+ 対象メソッド
+ {{ document.method }}
+
+
+ 生成元
+ {{ document.source_path }}
+ シート
+ {{ document.sheet_number }} / {{ document.sheet_count }}
+
+
+ 生成日時
+ {{ document.generated_at }}
+
+
+ 証拠モード
+ {{ document.evidence_mode }}
+ 警告
+ {{ document.evidence_warnings|join("; ") }}
+
+ {% if document.runtime_exit_code is not none %}
+
+ Jest 終了コード
+ {{ document.runtime_exit_code }}
+ テスト実行
+ {% if document.runtime_tests_passed %}PASS{% else %}FAIL{% endif %}
+
+ {% endif %}
+
+
+
+ テストケース一覧
+
+
+
+ 番号
+ コード
+ 区分
+ テストケース
+ テスト名
+ 実施日
+ Bug No.
+
+
+
+ {% for case in case_columns %}
+
+ {{ case.index }}
+ {{ case.testcasecode }}
+ {{ case.type }}
+ {{ case.testcase }}
+ {{ case.testname }}
+ {{ case.executed_date }}
+ {{ case.bug_number }}
+
+ {% endfor %}
+
+
+
+ {% if runtime_cases %}
+ Jest 実行証拠
+
+
+
+ コード
+ 実行テスト名
+ 状態
+ Matcher
+ 期待値
+ 実際値
+ エラー
+
+
+
+ {% for case in runtime_cases %}
+ {% for evidence in case.assertion_evidence %}
+
+ {{ case.testcasecode }}
+ {{ case.runtime_test_name }}
+
+ {% if evidence.passed %}PASS{% else %}FAIL{% endif %}
+
+ {% if evidence.negated %}not.{% endif %}{{ evidence.matcher }}
+ {{ evidence.expected|join(", ") }}
+ {{ evidence.actual }}
+ {{ evidence.error }}
+
+ {% endfor %}
+ {% endfor %}
+
+
+ {% endif %}
+
+ {% if frontend_cases %}
+ 前端ページUT補足
+
+
+
+ 番号
+ コード
+ 画面名
+ 入口URL
+ 確認方法
+ 前提条件
+ 操作入力
+ 画面出力
+ 遷移出力
+
+
+
+ {% for case in frontend_cases %}
+
+ {{ case.index }}
+ {{ case.testcasecode }}
+ {{ case.screen_name }}
+ {{ case.entry_url }}
+ {% if case.verification_mode %}{{ case.verification_mode }}{% endif %}
+
+ {% if case.preconditions %}
+
+ {% for item in case.preconditions %}
+ - {{ item }}
+ {% endfor %}
+
+ {% endif %}
+
+
+ {% if case.input_actions %}
+
+ {% for item in case.input_actions %}
+ - {{ item }}
+ {% endfor %}
+
+ {% endif %}
+
+
+ {% if case.ui_outputs %}
+
+ {% for item in case.ui_outputs %}
+ - {{ item }}
+ {% endfor %}
+
+ {% endif %}
+
+
+ {% if case.navigation_outputs %}
+
+ {% for item in case.navigation_outputs %}
+ - {{ item }}
+ {% endfor %}
+
+ {% endif %}
+
+
+ {% endfor %}
+
+
+ {% endif %}
+
+ 判定表
+
+
+
+ テスト項目番号
+ {% for case in decision_columns %}
+ {{ loop.index }}
+ {% endfor %}
+
+
+
+ {% for row in input_rows %}
+
+ {% if loop.first %}
+ 条件
+ {% endif %}
+ {% if row.show_item %}
+ {{ row.item }}
+ {% endif %}
+ {{ row.value }}
+ {% for case in decision_columns %}
+ {% if case and row["values"].get(case.testcasecode) %}◯{% endif %}
+ {% endfor %}
+
+ {% endfor %}
+
+ {% for row in output_rows %}
+
+ {% if loop.first %}
+ 結果
+ {% endif %}
+ {% if row.show_item %}
+ {{ row.item }}
+ {% endif %}
+ {{ row.value }}
+ {% for case in decision_columns %}
+ {% if case and row["values"].get(case.testcasecode) %}◯{% endif %}
+ {% endfor %}
+
+ {% endfor %}
+
+
+
+ 記入欄
+
+
+
+ 実施日
+
+ 担当者
+
+
+
+ レビュー
+
+ 備考
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/tests/conftest.py b/tests/conftest.py
new file mode 100644
index 0000000..46701aa
--- /dev/null
+++ b/tests/conftest.py
@@ -0,0 +1,2 @@
+# This directory is an external target-project fixture with its own pytest root.
+collect_ignore = ["fixtures/coverage_project/tests/test_app.py"]
diff --git a/tests/fixtures/coverage_project/app.py b/tests/fixtures/coverage_project/app.py
new file mode 100644
index 0000000..8f777bc
--- /dev/null
+++ b/tests/fixtures/coverage_project/app.py
@@ -0,0 +1,2 @@
+def double(value: int) -> int:
+ return value * 2
diff --git a/tests/fixtures/coverage_project/pyproject.toml b/tests/fixtures/coverage_project/pyproject.toml
new file mode 100644
index 0000000..80abd51
--- /dev/null
+++ b/tests/fixtures/coverage_project/pyproject.toml
@@ -0,0 +1,6 @@
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+
+[tool.coverage.run]
+relative_files = true
+source = ["unrelated_target"]
diff --git a/tests/fixtures/coverage_project/tests/test_app.py b/tests/fixtures/coverage_project/tests/test_app.py
new file mode 100644
index 0000000..58fb70b
--- /dev/null
+++ b/tests/fixtures/coverage_project/tests/test_app.py
@@ -0,0 +1,5 @@
+from app import double
+
+
+def test_double():
+ assert double(3) == 6
diff --git a/tests/test_artifacts.py b/tests/test_artifacts.py
new file mode 100644
index 0000000..2ba2d81
--- /dev/null
+++ b/tests/test_artifacts.py
@@ -0,0 +1,44 @@
+from pathlib import Path
+
+import pytest
+
+import teaforge.artifacts as artifacts_module
+from teaforge.artifacts import json_for_html_script, write_text_atomic
+
+
+def test_json_for_html_script_preserves_data_without_closing_script_element():
+ serialized = json_for_html_script(
+ {"test": "", "separator": "\u2028"}
+ )
+
+ assert "" not in serialized
+ assert "\\u003c/script\\u003e" in serialized
+ assert "\\u2028" in serialized
+
+
+def test_atomic_write_replaces_complete_artifact(tmp_path):
+ target = tmp_path / "report.json"
+ target.write_text("old", encoding="utf-8")
+
+ write_text_atomic(target, "new content")
+
+ assert target.read_text(encoding="utf-8") == "new content"
+ assert list(tmp_path.glob(".report.json.*.tmp")) == []
+
+
+def test_atomic_write_preserves_old_artifact_when_replace_fails(
+ tmp_path, monkeypatch
+):
+ target = tmp_path / "report.html"
+ target.write_text("valid old report", encoding="utf-8")
+
+ def fail_replace(source: Path, destination: Path) -> None:
+ raise OSError("disk refused replacement")
+
+ monkeypatch.setattr(artifacts_module.os, "replace", fail_replace)
+
+ with pytest.raises(OSError, match="disk refused replacement"):
+ write_text_atomic(target, "partial new report")
+
+ assert target.read_text(encoding="utf-8") == "valid old report"
+ assert list(tmp_path.glob(".report.html.*.tmp")) == []
diff --git a/tests/test_cli_integration.py b/tests/test_cli_integration.py
index 386f489..2f18272 100644
--- a/tests/test_cli_integration.py
+++ b/tests/test_cli_integration.py
@@ -1,12 +1,27 @@
+import json
from pathlib import Path
+import pytest
from typer.testing import CliRunner
+from teaforge import __version__
from teaforge.cli import app
+from teaforge.coverage.service import (
+ CoverageThresholdError,
+ CoverageThresholdViolation,
+)
+from teaforge.pcl.models import PCLDocument
runner = CliRunner()
+def test_version_option_matches_package_version():
+ result = runner.invoke(app, ["--version"])
+
+ assert result.exit_code == 0
+ assert result.stdout.strip() == __version__
+
+
def test_generate_and_get_commands(tmp_path):
html_path = tmp_path / "pcl.html"
source = Path("tests/fixtures/test_sample_pytest.py")
@@ -170,3 +185,141 @@ def test_generate_pcl_rejects_unknown_framework(tmp_path):
assert result.exit_code == 1
assert "Unsupported framework: unknown" in result.stderr
+
+
+def test_generate_pcl_reports_missing_custom_template_without_traceback(tmp_path):
+ missing_template = tmp_path / "missing-template.html"
+
+ result = runner.invoke(
+ app,
+ [
+ "pcl",
+ "generate",
+ "--path",
+ "tests/fixtures/test_sample_pytest.py",
+ "--output",
+ str(tmp_path / "pcl.html"),
+ "--template",
+ str(missing_template),
+ ],
+ )
+
+ assert result.exit_code == 1
+ assert "PCL template does not exist" in result.stderr
+ assert "Traceback" not in result.stderr
+
+
+def test_runtime_report_returns_distinct_exit_code_when_jest_tests_failed(
+ tmp_path, monkeypatch
+):
+ html_path = tmp_path / "pcl.html"
+ json_path = tmp_path / "pcl.json"
+ document = PCLDocument.create(
+ file="user.js",
+ method="createUser",
+ source_path="src/user.js",
+ )
+ document.evidence_mode = "runtime"
+ document.runtime_exit_code = 1
+ document.runtime_tests_passed = False
+ html_path.write_text("", encoding="utf-8")
+ json_path.write_text(
+ json.dumps(document.to_dict(), ensure_ascii=False),
+ encoding="utf-8",
+ )
+ monkeypatch.setattr(
+ "teaforge.cli.generate_pcl",
+ lambda **kwargs: [(html_path, json_path)],
+ )
+
+ result = runner.invoke(
+ app,
+ [
+ "pcl",
+ "generate",
+ "--framework",
+ "jest",
+ "--evidence-mode",
+ "runtime",
+ "--path",
+ "tests/fixtures/test_sample_jest.test.ts",
+ "--output",
+ str(html_path),
+ ],
+ )
+
+ assert result.exit_code == 2
+ assert "Reports were generated with failure evidence" in result.stderr
+
+ accepted = runner.invoke(
+ app,
+ [
+ "pcl",
+ "generate",
+ "--framework",
+ "jest",
+ "--evidence-mode",
+ "runtime",
+ "--allow-test-failures",
+ "--path",
+ "tests/fixtures/test_sample_jest.test.ts",
+ "--output",
+ str(html_path),
+ ],
+ )
+
+ assert accepted.exit_code == 0
+
+
+def test_pcl_document_rejects_unknown_future_schema():
+ document = PCLDocument.create(
+ file="user.py",
+ method="create_user",
+ source_path="src/user.py",
+ )
+ payload = document.to_dict()
+ payload["schema_version"] = 999
+
+ with pytest.raises(ValueError, match="Unsupported PCL schema version: 999"):
+ PCLDocument.from_dict(payload)
+
+
+def test_coverage_gate_preserves_report_and_returns_exit_code_three(
+ tmp_path, monkeypatch
+):
+ output = tmp_path / "coverage.html"
+ output.write_text("coverage report", encoding="utf-8")
+
+ def fail_gate(**_kwargs):
+ raise CoverageThresholdError(
+ [output],
+ [
+ CoverageThresholdViolation(
+ output_path=output,
+ metric="C1",
+ actual=75.0,
+ required=100.0,
+ )
+ ],
+ )
+
+ monkeypatch.setattr("teaforge.cli.generate_coverage_reports", fail_gate)
+
+ result = runner.invoke(
+ app,
+ [
+ "coverage",
+ "generate",
+ "--path",
+ "demo/fastapi_crud/tests",
+ "--output",
+ str(output),
+ "--min-c1",
+ "100",
+ ],
+ )
+
+ assert result.exit_code == 3
+ assert output.exists()
+ assert f"Generated coverage HTML: {output}" in result.stdout
+ assert "C1 75.0% is below the required 100.0%" in result.stderr
diff --git a/tests/test_coverage_report.py b/tests/test_coverage_report.py
index d496b16..7d320ab 100644
--- a/tests/test_coverage_report.py
+++ b/tests/test_coverage_report.py
@@ -7,13 +7,193 @@
from typer.testing import CliRunner
from teaforge.cli import app
+from teaforge.coverage import mermaid as mermaid_module
from teaforge.coverage import service as coverage_service
-from teaforge.coverage.analyzer import parse_source_functions
+from teaforge.coverage.analyzer import analyze_coverage
from teaforge.coverage.mermaid import save_mermaid_diagram
+from teaforge.coverage.models import CoverageDocument, CoverageMetric
+from teaforge.coverage.render import render_coverage_html
+from teaforge.coverage.service import find_coverage_threshold_violations
+from teaforge.jest.coverage import _collect_branch_sets
+from teaforge.jest.project import JestProject
runner = CliRunner()
+def test_coverage_embedded_json_cannot_close_its_script_element():
+ document = CoverageDocument.create(
+ file=".py",
+ source_path="src/app.py",
+ test_path="tests/test_app.py",
+ c0=CoverageMetric(covered=1, total=1, percent=100.0),
+ c1=CoverageMetric(covered=0, total=0, percent=0.0),
+ requested_functions=[],
+ functions=[],
+ evidence_source="coverage.py JSON",
+ c0_definition="covered statements / measurable statements",
+ c1_definition="executed branches / measurable branches",
+ )
+
+ html = render_coverage_html(document)
+ embedded_json = html.split(
+ '", 1)[0]
+
+ assert "