Feat/failure objective enhancements - #11
Open
SissiFeng wants to merge 10 commits into
Open
Conversation
… foundation Turn the reward from an opaque scalar into a per-signal, auditable, RLVR-ready record, and start persisting decision trajectories. This is the foundation for the self-evolving evaluation loop (Phases B/C). - New app/services/verifiable_reward.py: single source of truth for the reward components both calculators duplicated. Each signal exposes a RewardVerification (name/passed/score/evidence/verifier_type). Tri-state passed; verifier errors recorded, never swallowed. process/outcome split. - CampaignDecisionRewardCalculator + LoopRewardCalculator now delegate to the shared core (DRY). Loop keeps its no-positive-clamp objective via positive_clamp=False — reward values are bit-for-bit unchanged (regression guarded). - CampaignDecisionReward + LoopReward gain rubric_version, process_reward, outcome_reward, verifications (backward-compatible defaults). - New decision_trajectories table (+ trajectory_schema_version) and app/services/decision_trajectory.py: append-only persistence + JSONL export. - DetailedEventEmitter.emit_decision_reward streams the per-signal verifier report so the inner evaluation loop is visible in /lab. Tests: 893 passed, ruff clean, mypy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eedback, retrospective + offline eval) Make "what counts as a good decision" itself evolve, on top of Phase A's verifiable reward. All of B operates over the recorded verifications, so a decision can be re-scored under any rubric without rerunning the campaign. - B1 app/services/rubric.py: versioned per-signal weight multipliers. v0.1_static is the identity rubric (reproduces Phase A exactly); v0.2_campaign_aware reweights by CampaignMode (optimization values objective, safety-tightening values safety, etc.). rescore() applies a rubric to verifications. - B2 app/services/scientist_feedback.py: constrained feedback taxonomy (mechanistic_value, safety_concern, ...) mapped to signals; folds a feedback batch into a derived v0.3_feedback_adaptive rubric (human supervision reshapes the evaluator). - B3 app/services/decision_evaluation.py retrospective_audit(): re-score stored trajectories under a hindsight rubric, flag decisions the immediate reward under-/over-estimated. Explicit trigger only. - B4 offline_policy_evaluation(): compare rubrics over accumulated trajectories, gated at >=50 to avoid noise. Tests: 908 passed, ruff clean, mypy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…verlay Break fixed-objective, fixed-boundary BO: an advisor proposes reframing the objective or widening the search space, as a first-class, provenance-tracked, safety-gated overlay — without mutating the versioned TaskContract. - C3 app/services/space_overlay.py: ObjectiveOverlay / BoundaryOverlay / SpaceOverlay. derive_contract() applies an overlay to a NEW contract, base never mutated, lineage recorded in contract_id + migrated_from. - C2 review_space_change(): gates a deliberate space change — must reference real dimensions, may only WIDEN bounds (never silently shrink), escalates to a human on large expansion, low confidence, or require_human_approval. Categorically distinct from decision_policy._bounds_violation (which rejects hallucinated out-of-bounds candidates in the current space). - C1 app/services/space_evolution.py SpaceEvolutionAdvisor: deterministic, mock-safe proposals from a Nexus fingerprint (widen a plateaued dimension; add a secondary KPI on proxy mismatch). - C4 group_relative_rank(): GRPO-style advantage-over-group-mean ranking of a round's proposals/candidates. Tests: 923 passed, ruff clean, mypy clean (new files). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tion metrics Wrapper-injected experimental pathologies (noise drift, spatially correlated execution failure, censoring, proxy-gap shift, objective shift) over unmodified runner/problems; ground-truth event ledger; adaptation metrics (recovery efficiency, adaptation lag, constraint latency, decision quality) with family-scoped Holm-corrected paired bootstrap; dynamic + epoch-local regret rulers for objective shifts; backend-owned helios_full ablation variants; reproducible study artifacts (full config snapshot, dirty-tree + dependency provenance, overwrite protection); canonical preregistered study v1 and pathology-study CLI subcommand.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.