Feat/failure objective enhancements

.codegraph/.gitignore

@@ -0,0 +1,5 @@
+# CodeGraph data files — local to each machine, not for committing.
+# Ignore everything in .codegraph/ except this file itself, so transient
+# files (the database, daemon.pid, sockets, logs) never show up in git.
+*
+!.gitignore

.github/workflows/ci.yml

@@ -0,0 +1,63 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+concurrency:
+  group: ci-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  test:
+    name: test (py${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+
+      - name: Set up Python ${{ matrix.python-version }}
+        run: uv python install ${{ matrix.python-version }}
+
+      - name: Install project (dev extras)
+        run: uv pip install --system --python ${{ matrix.python-version }} -e ".[dev]"
+
+      - name: Run tests
+        run: python -m pytest -q --cov=app --cov-report=term-missing
+
+  lint:
+    name: lint (ruff)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+      - name: Ruff check
+        run: uvx ruff check app benchmarks
+
+  types:
+    # Non-blocking: HELIOS carries known typing debt outside the contracts/core
+    # boundary. This job surfaces regressions without gating merges. Remove
+    # `continue-on-error` once the debt is burned down.
+    name: types (mypy, informational)
+    runs-on: ubuntu-latest
+    continue-on-error: true
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+      - name: Set up Python
+        run: uv python install 3.11
+      - name: Install project (dev extras)
+        run: uv pip install --system -e ".[dev]"
+      - name: Mypy
+        run: python -m mypy app

.gitignore

@@ -7,6 +7,7 @@ data/orchestrator.db
 data/orchestrator.db-shm
 data/orchestrator.db-wal
 data/object_store/
+data/scientific_ledger/
 .env
 
 # macOS
@@ -36,6 +37,8 @@ docs/*
 !docs/adaptive_campaign_substrate.md
 !docs/development_progress.md
 !docs/agent_architecture.md
+!docs/scientific_decision_ledger.md
+!docs/scientific_evidence_loop.md
 !docs/plans/
 docs/plans/*
 !docs/plans/2026-07-06-helios-experiment-planner-and-hybrid-switching-plan.md
@@ -59,6 +62,9 @@ tests/**/__pycache__/
 # Node modules
 node_modules/
 
+# Benchmark run artifacts (studies are reproducible from tracked configs)
+/benchmark_results/
+
 # Runtime logs
 logs/
 *.log
@@ -98,3 +104,4 @@ ot2-nlp-agent/lab_automation/plugins/potentiostat/adapters/squidstat.py
 
 # git worktrees (isolated workspaces)
 .worktrees/
+.gstack/

README.md

@@ -53,29 +53,99 @@ The live path is conservative by design: rule-based, auditable, and bounded by e
 ## Core Capabilities
 
 - **Orchestrator-agnostic campaign decision layer** — keeps campaign-level decision authority separate from any downstream backend.
-- **Context-aware policy** — uses objective hierarchy, proxy-gap state, failure attribution, backend memory, Nexus diagnostics, candidate/failure-zone memory, and bandit/learned-policy signals.
+- **Context-aware policy** — uses objective hierarchy, proxy-gap state, failure attribution, backend memory, Nexus diagnostics, BO MCP availability, candidate/failure-zone memory, and bandit/learned-policy signals.
 - **Dynamic action vocabulary** — represents optimization, validation, calibration, failure diagnosis, context seeking, human observation, safety-constraint tightening, stopping, and future scale/fidelity choices.
-- **Candidate and backend arbitration** — combines local baselines, Nexus signals, candidate pools, safety gates, and provenance into a traceable portfolio.
+- **Candidate and backend arbitration** — combines local baselines, Nexus/BO MCP signals, candidate pools, safety gates, and provenance into a traceable portfolio.
 - **Failure-aware recovery** — separates scientific negative evidence from measurement, backend, constraint, and downstream tool failures.
 - **Trace, reward, and replay** — records `StrategyTrace`, `StrategyEvidence`, `StrategyOutcome`, `StrategyReward`, typed `FailureEvent`, and replay summaries.
+- **Scientific evidence loop** — tracks falsifiable claims, updates posterior odds only from independent auditable likelihood ratios, ranks hypothesis-discrimination experiments by robust information gain, and blocks live promotion behind prospective evidence and explicit approval.
+- **Scientific Decision Ledger** — projects every live campaign decision into deterministic, redacted Markdown Decision Cards with evidence, alternatives, outcome, reward, failures, recovery, policy/Nexus versions, exact-text memory search, and typed RLVR export.
 - **LLM boundary discipline** — LLMs can help translate intent, gather context, or generate review notes; they do not steer the live optimization loop.
 
 ---
 
+## Scientific Decision Ledger
+
+HELIOS keeps two deliberately separate truths:
+
+- Typed DTOs and SQLite rows are the transactional runtime truth.
+- Markdown and optional campaign-local Git are the readable, reviewable, auditable scientific truth.
+
+A decision is written before execution with `Outcome: Pending`, then finalized in place after analysis with its observed outcome, deterministic verifier scores, reward, failures, and recovery episode. A campaign is projected under `data/scientific_ledger/campaigns/<campaign-id>/`:
+
+```text
+campaign.md                  # objective, metadata, decision index
+index.md                     # navigable artifact index
+summary.md                   # aggregate decisions and rewards
+trajectory.md                # Mermaid decision trajectory
+policy.md                    # current decision policy snapshot
+policy_versions/<version>.md # immutable first snapshot per policy version
+nexus.md                     # Nexus contract/version and diagnostics
+training_dataset.md          # human-reviewable RLVR projection
+evidence/
+  index.md                   # scientific claims and discrimination plans
+  claims/<claim-id>.md       # posterior, falsifiers, evidence, promotion gate
+  plans/<plan-id>.md         # robust information-gain ranking for review
+rounds/001/
+  objective.md
+  observations.md
+  decision_001.md
+  strategy.md
+  evidence.md
+  failure.md
+  recovery.md
+  summary.md
+```
+
+Decision Cards contain the question, scientific context, evidence, ranked candidate actions, selected action/backend, rationale, confidence/expected gain, outcome, reward/verifiers, failure/recovery counts, and reproducibility provenance. Values are deterministically rendered and recursively redacted before they reach Markdown.
+
+The read-only API exposes:
+
+- `GET /api/v1/memory/scientific/search?q=pipette%20offset`
+- `GET /api/v1/memory/scientific/{campaign_id}/artifact?path=rounds/001/decision_001.md`
+- `GET /api/v1/memory/scientific/{campaign_id}/rlvr`
+
+RLVR JSONL is generated from the typed `decision_trajectories` store, not by scraping Markdown. Optional Git history is one repository per campaign, stages exact Markdown paths only, and never pushes or modifies the HELIOS source repository. See [Scientific Decision Ledger](docs/scientific_decision_ledger.md) for lifecycle, schemas, safety properties, and operations.
+
+The scientific evidence loop is deliberately separate from the operational reward loop. A successful execution does not increase a scientific claim posterior. Only evidence carrying an auditable likelihood ratio can do that; descriptive evidence remains visible without being numerically counted. See [Scientific Evidence Loop](docs/scientific_evidence_loop.md).
+
+---
+
 ## Architecture
 
 | Surface | Responsibility | Representative modules |
 |---------|----------------|------------------------|
 | **Contract and context** | Typed campaign goal, objectives, constraints, budget, safety, and round context | `app/contracts/`, `app/services/round_context.py`, `app/services/objective_state.py` |
 | **Campaign policy** | Decide next campaign-level action and strategy mode | `app/services/strategy_selector.py`, `app/services/strategy_actions.py`, `app/services/decision_layer.py` |
-| **Evidence and memory** | Attach diagnostics, prior-campaign evidence, failure history, and backend memory | `app/services/decision_trace.py`, `app/services/backend_memory.py`, `app/optimization/candidate_memory.py`, `app/optimization/failure_zone_memory.py` |
+| **Evidence and memory** | Track scientific claims/posteriors, discrimination plans, diagnostics, prior-campaign evidence, failure history, and backend memory | `app/services/scientific_evidence.py`, `app/services/hypothesis_experiment_planner.py`, `app/services/scientific_ledger.py`, `app/services/backend_memory.py` |
 | **Candidate/backend arbitration** | Build, gate, score, and explain candidate/backend choices | `app/optimization/service.py`, `app/optimization/pool_service.py`, `app/optimization/decision_policy.py`, `app/optimization/provenance.py` |
 | **Adaptive substrate** | Shadow-only scientific activity mode, dynamic action space, and value-of-information assessment | `app/services/adaptive_campaign_substrate.py`, `app/services/campaign_mode.py`, `app/services/dynamic_action_space.py`, `app/services/value_of_information.py` |
 | **Outcome and replay** | Evaluate decision quality, reward components, and replay summaries | `app/services/decision_outcome.py`, `app/services/verifiable_reward.py`, `app/services/decision_replay.py`, `app/services/policy_evaluation.py` |
 
 ### Adaptive Campaign Decision Layer
 
-HELIOS uses scientific context, objective hierarchy, typed failure attribution, backend performance memory, candidate/failure-zone memory, Nexus diagnostics and bandit/learned-policy signals to decide which campaign-level action should happen next. Today that includes `CampaignIntent`, `OptimizationMode`, and candidate-generation backend selection; the same layer owns validation, failure-aware recovery, context acquisition, human/LLM query, dynamic objective/constraint handling, and future scale/fidelity-aware decisions.
+HELIOS uses scientific context, objective hierarchy, typed failure attribution, backend performance memory, candidate/failure-zone memory, Nexus diagnostics, BO MCP availability, and bandit/learned-policy signals to decide which campaign-level action should happen next. Today that includes `CampaignIntent`, `OptimizationMode`, and candidate-generation backend selection; the same layer owns validation, failure-aware recovery, context acquisition, human/LLM query, dynamic objective/constraint handling, and future scale/fidelity-aware decisions.
+
+The default runtime still records contextual campaign decisions in shadow mode.
+When `CAMPAIGN_DECISION_AUTHORITY_ENABLED=true`, the orchestrator promotes the
+decision envelope into a bounded pre-candidate gate: `STOP_CAMPAIGN` terminates
+before more candidates, while validation, recovery, context, objective, and
+constraint actions defer the current round, persist the requested campaign
+state update, and leave candidate generation untouched for later rounds. The
+gate never executes hardware or auto-applies objective/space changes.
+
+### Experimental-node active learning
+
+For campaigns whose alternatives are materially different experimental nodes
+(for example, different synthesis routes), `experimental_route_graph` declares
+the nodes, transitions, execution mapping, capability requirements, cost, and
+safety metadata. Nexus `/api/experimental-routes/analyze` supplies versioned,
+`advisory_only` evidence. HELIOS then scores every reachable option and enforces
+local capability, safety, budget, operator-approval, and executable-protocol
+gates. `NEXUS_EXPERIMENTAL_ROUTES_ENABLED` enables characterization in shadow;
+the separate `EXPERIMENTAL_ROUTE_AUTHORITY_ENABLED` gate is required to change
+the live node. Each decision and route-labelled outcome is checkpointed in the
+campaign context and included in the Scientific Decision Ledger trajectory.
 
 ### Optimization Code Map
 
@@ -141,14 +211,26 @@ pytest \
 | `LLM_PROVIDER` | `mock` | LLM provider for language/knowledge-boundary tasks only |
 | `LLM_MODEL` | provider default | Model ID passed to the configured provider |
 | `CONTEXTUAL_DECISION_SHADOW_ENABLED` | `false` | Record the legacy contextual decision shadow trace per round |
+| `CAMPAIGN_DECISION_AUTHORITY_ENABLED` | `false` | Promote contextual campaign decisions into a bounded live pre-candidate gate |
 | `ADAPTIVE_SUBSTRATE_SHADOW_ENABLED` | `false` | Record the adaptive campaign substrate shadow snapshot per round |
 | `ENABLE_CANDIDATE_ARBITRATION` | `false` | Enable deep candidate-pool arbitration instead of legacy generation fallback |
+| `NEXUS_EXPERIMENTAL_ROUTES_ENABLED` | `false` | Request advisory experimental-route characterization from Nexus each round |
+| `EXPERIMENTAL_ROUTE_AUTHORITY_ENABLED` | `false` | Allow HELIOS to apply a route selected by its local safety/budget/approval policy |
+| `NEXUS_URL` | `http://localhost:8000/api` | Base URL for optional Nexus REST advisory endpoints |
+| `NEXUS_API_KEY` | empty | Optional `X-API-Key` sent to Nexus REST endpoints |
+| `NEXUS_TIMEOUT_SECONDS` | `10` | Nexus REST request timeout |
+| `SCIENTIFIC_LEDGER_ENABLED` | `true` | Persist live Decision Cards and typed outcome/reward accounting; fail-open with respect to campaign routing |
+| `SCIENTIFIC_LEDGER_ROOT` | `data/scientific_ledger` | Root for campaign Markdown artifacts |
+| `SCIENTIFIC_LEDGER_GIT_ENABLED` | `false` | Commit changed Markdown artifacts to each campaign's local Git repository |
+| `SCIENTIFIC_LEDGER_GIT_AUTO_INIT` | `true` | Initialize a missing campaign-local repository when Git recording is enabled |
+| `SCIENTIFIC_LEDGER_GIT_AUTHOR_NAME` | `HELIOS Scientific Ledger` | Local ledger commit author name |
+| `SCIENTIFIC_LEDGER_GIT_AUTHOR_EMAIL` | `helios-ledger@localhost` | Local ledger commit author email |
 
 ---
 
 ## Validation Evidence
 
-HELIOS is framed as an orchestrator-agnostic adaptive campaign decision layer. The product boundary is campaign-level decision authority rather than ownership of downstream automation or presentation surfaces. The live campaign policy remains rule-based and auditable by default; Nexus is optimization advisor/backend/tool paths, not campaign decision authorities. Learned policy and self-evolution paths are offline, shadow, canary, and approval-gated.
+HELIOS is framed as an orchestrator-agnostic adaptive campaign decision layer. The product boundary is campaign-level decision authority rather than ownership of downstream automation or presentation surfaces. The live campaign policy remains rule-based and auditable by default; Nexus and BO MCP are optimization advisor/backend/tool paths, not campaign decision authorities. Learned policy and self-evolution paths are offline, shadow, canary, and approval-gated; their metadata does not change default BO MCP/Nexus/backend behavior.
 
 The architecture validation report is version-controlled at [docs/HELIOS_ARCHITECTURE_VALIDATION.md](docs/HELIOS_ARCHITECTURE_VALIDATION.md). It is a static evidence pack for the current validation boundary.
 
@@ -188,6 +270,7 @@ HELIOS/
 ├── docs/
 │   ├── HELIOS_ARCHITECTURE_VALIDATION.md
 │   ├── adaptive_campaign_substrate.md
+│   ├── scientific_decision_ledger.md
 │   └── development_progress.md
 ├── tests/                   # Pytest coverage for policy, replay, validation, and evidence layers
 ├── benchmarks/              # Offline method and policy evaluation harnesses
@@ -198,6 +281,25 @@ HELIOS/
 
 ---
 
+## External Decision Inputs
+
+| Integration | Role in HELIOS |
+|-------------|----------------|
+| **Nexus** | Optimization diagnostics, profiling, and backend/candidate evidence |
+| **BO MCP / Ax / local BO** | Optimization backend signals and candidate proposals |
+| **Anthropic / OpenAI** | Language/knowledge-boundary tasks such as intent parsing, context requests, and review notes |
+| **Campaign memory** | Similar-campaign priors, backend history, failure zones, and replay evidence |
+
+---
+
+## Contributing
+
+1. Keep decision authority explicit: backends advise, HELIOS decides.
+2. Preserve typed traces, evidence, outcomes, rewards, and replay records for every new decision path.
+3. Keep learned policies gated by replay, shadow/canary evidence, and explicit promotion controls.
+4. Add tests beside changes to policy, arbitration, reward, replay, or validation logic.
+
+---
 
 ## License