DAAF v3.0.0 -- Opening Doors daaf_dev_R2 Branch Merge#18
Merged
Conversation
- Confirms parquet file integrity and readability - Validates 1000 sample rows + 3 summary statistic rows - Exit code 0 Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Validation: All 7 checks PASSED (sample count, mean plausibility, CI coverage, CI width, bootstrap count, file existence, file readability) - Rows: 1,000 - Files: scripts/adhoc/01_monte-carlo-sim.py, data/processed/monte_carlo_results.parquet (gitignored) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Validation: All 7 checks PASSED (schema, round-trip, CI bounds, width, file size) - Rows: 1 (summary row with 1000 raw samples in list column) - Files: scripts/adhoc/01_monte-carlo-sim.py, output/analysis/2026-06-08_monte-carlo-results.parquet Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…trap CI - Validation: CP3 PASSED (all 5 checks) - Rows: 1,000 - Files: scripts/adhoc/01_monte-carlo-sim.py (v1, blocked by log marker), scripts/adhoc/01_monte-carlo-sim_a.py (v2, FINAL, exit 0) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ervals - Generated 1000 samples from normal distribution (mean=50, std=10) - Computed sample mean: 50.1933 - Bootstrap CI (10,000 resamples): [49.5942, 50.8122] - Saved results to parquet with sample data and CI bounds - Validation: CP3 PASSED (all 5 checks)
…tory Benchmark test agents committed sandbox output and research files before git operations were blocked via --disallowed-tools. Removes 15 files: 13 in benchmarks/_sandbox/ and 2 in research/2026-06-08_AdHoc_MonteCarloSimulation/. Also deleted 3 untracked test artifacts from the working directory (goodreadsbooks.zip, research/2026-05-10_Enrollment_Trends/, scripts/01_montecarlo-bootstrap.py). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…d fixture isolation Key changes from prior session: - Switch default permission mode from dontAsk to bypassPermissions (models.py) with --disallowed-tools for git operations (executor.py) to prevent subagent commits while allowing full DAAF workflow functionality - Add subagent_behavior scorer that parses separate subagent transcript files to verify agent-type-specific behavioral criteria (6 agent types) - Integrate subagent scoring into dispatch compliance runner with fixture isolation, workspace containment, and inline results display - Add standalone rescore script for post-hoc subagent behavior analysis - Add benchmarks/.gitignore covering _sandbox/, results/, research/ - Add root .gitignore pattern for benchmark-leaked AdHoc research dirs - Add SESSION_RESTART.md documenting architecture decisions and next steps Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… scoring accuracy fixes Tier 1 improvements across 8 files: Tool failure diagnostics: - Add tool_failures field to RunResult (models.py) - Extract failed tool calls from JSON output with tool name + error content, handling both string and list content formats (executor.py) - Capture error content text in transcript parser, not just is_error boolean (checkpoint_adherence.py) - Surface tool failures in console output, result.json, and summary.json across all 3 phase runners Timeout recovery: - Parse partial stdout from TimeoutExpired exception to recover turns/cost - Search filesystem for session_id on cold-start timeouts - Phase 1 now attempts scoring on timeout if session_id recovered (matching Phase 2/3 behavior) Scoring accuracy: - Filter Agent tool calls by succeeded=True in dispatch_compliance scorer — failed dispatches no longer count as adherence - Same fix in checkpoint_adherence subagent_dispatched scoring - Retry-resilient: fail-then-succeed pattern correctly passes - Diagnostic detail distinguishes "never attempted" from "attempted but failed" - Remove redundant BASE_DIR from prompt_contains in test cases (already checked independently by prompt_has_base_dir criterion) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ame checks Add 4 new tier2 scoring criteria to dispatch_compliance scorer: - prompt_has_project_dir: checks for "PROJECT_DIR" (required by ad hoc template) - prompt_has_task_section: checks for "## Task" section header - prompt_has_context_section: checks for "## Context" section header - prompt_has_instructions: checks for "## Instructions" section header These match the Standard Agent Prompt Structure defined in ad-hoc-collaboration-mode.md lines 196-218. Also add exact skill name to prompt_contains_required for source-researcher cases (dc-03: education-data-source-ipeds, dc-04: education-data-source-ccd) per the agent's upstream_input requirement for exact skill names. All 12 test cases updated with new criteria in soft_requirements. Scorer now evaluates 10 criteria per dispatch (up from 6). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…riptive criteria Replace "min N tool calls" tier2 checks with specific behavioral assertions derived from analysis of 64 subagent transcripts across all agent types. New checks per agent type: - research-executor: writes to scripts/adhoc/ (not just any .py) - source-researcher: reads >= 2 skill reference files; no code execution - search-agent: reads >= 3 skill files; no code execution - debugger: writes diagnostic script to debug/ dir; uses run_with_capture - code-reviewer: writes CR script to scripts/cr/; uses run_with_capture - data-ingest: writes profiling script; uses run_with_capture New check functions: writes_to_dir, reads_min_matching, no_code_execution. Total: 24 behavioral specs across 6 agent types (up from 18). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Subagents treat the sandbox workspace as BASE_DIR and look for
scripts/run_with_capture.sh there. Without a copy, they either fail
with "No such file" or fall back to direct python execution (blocked
by enforce-file-first hook). Copy the canonical run_with_capture.sh
into {workspace}/scripts/ during fixture preparation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ress Documents all work completed: tool failure diagnostics, timeout recovery, scoring accuracy fixes (succeeded filtering), 4 new structural dispatch criteria, prescriptive subagent behavior specs (24 checks), run_with_capture sandbox fix. Includes Haiku validation results, 20 architecture decisions, and updated next-session priorities (validation run, OpenRouter, knowledge tests). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rate cost estimation - Add model_loader.py: shared model loading with provider env var resolution, CLI filtering (--models, --provider), and OpenRouter auto-wiring - Add cost_estimator.py: pre-run cost estimation using calibrated token profiles from real benchmark runs, with per-model pricing from OpenRouter API - Add PricingConfig to models.py with estimate_cost() for token-based costing - Capture token usage (input, output, cached) from CLI result messages in executor - Replace CLI-reported cost_usd (wrong for OpenRouter) with computed_cost_usd calculated from actual tokens × model pricing in all runners - Expand model matrix to 14 models: 3 Anthropic + 11 OpenRouter (GLM 5.1, Kimi K2.6, Qwen 3.6, Gemma 4 31B/26B, DeepSeek V4 Pro/Flash, Gemini 3.1 Pro/Flash Lite, Gemini 3.5 Flash, Nemotron 3 Ultra) - All OpenRouter pricing from /api/v1/models endpoint, validated against billing - Fix Kimi model ID (kimi-k2.6 -> moonshotai/kimi-k2.6) - Add --yes/-y flag to skip cost confirmation prompt - Deduplicate load_models_from_yaml across all 3 runners into model_loader Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Captures OpenRouter integration, token capture, cost estimation work. Next session priority: HTML results viewer using clean 3-model runs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…mples from N(50,10), computes sample mean and bootstrap 95% CI - v1 had numpy parameter error (sd->scale), v2 (final) executes successfully - Results saved to parquet with 11 columns (mean, CI bounds, sample stats)
…nd deterministic session IDs Two infrastructure bugs found and fixed via diagnostic testing: 1. Token extraction now prefers CLI's modelUsage (aggregates main + subagent sessions) over usage (main session only). Verified empirically: Haiku dc-01 showed usage.output=1,793 vs modelUsage.output=6,939 — subagent tokens were invisible to cost calculations. 2. Cold-start runs now pre-assign session IDs via --session-id flag, replacing _find_recent_session_id() which scanned a shared directory with 290 orphaned files and caused cross-run transcript contamination (14/45 Sonnet, 8/45 GLM Phase 1 runs scored against wrong prompts). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…le headers, timed_out field Addresses false negatives identified by 3-model audit: 1. Confirmation gate detection now checks AskUserQuestion tool calls (GLM uses structured UI widgets) and accepts phrasing variants (shall we begin/start, let's get started) 2. Dispatch compliance accepts semantically equivalent headers for ## Context (## Scope, ## Background, ## Specifications, etc.) and ## Instructions (## What to Report, ## Output Format, ## Deliverables, etc.) 3. PROJECT_DIR criterion auto-passes for read-only agents (search-agent, source-researcher, plan-checker) where it's semantically unnecessary 4. Added timed_out boolean to result.json across all 3 runners for cleaner aggregate statistics Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ost diagnosis Documents: modelUsage/session-id fixes, scorer improvements, 4-model validation batch results, and OpenRouter reasoning token cost gap (2-2.5x undercount confirmed via direct API comparison). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add `monte_carlo_bootstrap.py` to `$PROJECT_DIR/scripts/adhoc`. - Execute simulation: draw 1000 samples from Normal(50, 10). - Bootstrap mean (M=10,000) constructs empirical 95% CI. - Validate alignment between parametric asymptotic expectations and empirical estimates. - Output tracking parameters/results to parquet at `$PROJECT_DIR/data/processed/monte_carlo_results.parquet`. - Validation: PASSED - Rows: 1 - Files: scripts/adhoc/monte_carlo_bootstrap.py, data/processed/monte_carlo_results.parquet Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ears) - Panel: 500 rows, 5 columns (state_id, year, enrollment, treat_year, treat) - 4 treatment cohorts (2016/2018/2020/2022, 10 states each) + 10 never-treated - CP3 PASSED: all shape, indicator, cohort-size, and enrollment-range checks - Output: data/processed/synthetic_staggered_did_panel.parquet (3.6KB) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Draw 1000 samples from N(50, 10), compute mean and bootstrap 95% CI - Validation: CP PASSED (all 3 checks) - Rows: 1000 samples, 8 summary metrics - Files: monte_carlo_results.parquet, monte_carlo_summary.parquet Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Versioned fix: corrected bootstrap resamples to B=1000, aligned column schema, modern numpy API. Original preserved for audit.
…acking Benchmark subagents force-committed run artifacts past gitignore rules: two AdHoc research project folders and two files inside the gitignored _sandbox/ run workspace. Deletion commits chosen over reverts since the branch will be squash-merged. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ng script Move unused code to benchmarks/archive/ with explanatory README: runner.py (superseded by standalone phase scripts), cost_budget.yaml (only consumed by the archived runner), and golden/mode_classification/ checkpoints (Phase 1 is cold-start). Delete rescore_subagent_behavior.py, which referenced a result set no longer on disk. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…viewer generator Add Fable 5 to the model matrix with $10/$50/$1.00 pricing; commit the results viewer generator (rep renumbering, HTML5-safe transcript embedding, phase-aware metrics); scorer and runner refinements from the 17-model Phase 3 campaign; correct the Phase 1 docstring (cold-start, not bootstrap checkpoint); update session restart notes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…wrapper New benchmarks/README.md (12 sections) documents the behavioral conformance benchmark end-to-end: phases, architecture, CLI usage, golden checkpoints, scoring semantics, cost tracking, operational notes, dated results snapshot, known limitations, and future work. Supersedes the SESSION_RESTART files as system documentation. Review pass corrected the prompt_has_context_section description (scorer already accepts seven heading variants) and moved the legacy runner's shell wrapper to archive/ alongside the runner it invokes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ch/ leak cleanup Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ointers Preserves run-level provenance (rate-limit replacements, viewer bug root causes) through the planned squash-merge while keeping the top level to two living documents: README.md and SESSION_NOTES.md. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…facts, and probe tooling Pins the DAAFBench source window ahead of the G1R Sonnet 4.6 anchor run. Includes the durable chatgpt-subscription Luna registry entry, fail-closed route provenance, nullable usage/identity schema-v2, dual-ledger subscription accounting, standalone one-turn route probe, preflight-only checks on all four phase runners, viewer schema-v2 compatibility, and the six-file benchmark test suite (implementation passes 1-2 of the DAAFBench GPT routing project). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ty gate, manifest provenance, Terra/Sol registry)
B1: repetition-safe dispatch workspaces — guarded per-run sandbox wipe
(prepare_run_sandbox, SANDBOX_ROOT) + unconditional workspace containment
instruction for all cases (fixes the dc-01 /daaf/research leak).
B2: child-model purity as a non-scoring validity gate — failed purity marks a
run invalid and excludes it from score rollups (retained + disclosed via
validity_coverage); no CriterionResult changes. Console print_summary
mirrors the same exclusion so console and summary.json agree.
B3: shared manifest_provenance() stamped by all four runners — git_dirty,
worktree_diff_sha256, golden_generation_id/golden_checksums, condition_id,
claude_code_version, reused route provenance.
B4: gpt-56-terra-chatgpt + gpt-56-sol-chatgpt registry entries (mirror Luna;
ceiling measured for Sol 2026-07-16, inferred for Terra).
Review fixes: sandbox paths derive from SANDBOX_ROOT (no duplicated literals);
console rollups validity-gated.
Tests: +13 (test_g1r_harness_fixes.py + test_chatgpt_route extensions); full
benchmark suite 65 -> 78, all passing. Goldens, scorer criteria,
cases.jsonl, and TIMEOUT_BY_TIER untouched (verified by review wave).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Apply the 370k ChatGPT-subscription accounting cap consistently across runtime consumers, refresh model caches safely, enforce canonical context declarations, and anchor GPT family matching. Recalibrate exact-Sol quality thresholds, harden deployment diagnostics and route-specific guidance, and expand deterministic regression coverage. Co-Authored-By: Claude <noreply@anthropic.com>
Fixes the production findings deferred from the 2026-07-06 migration test harness session (SESSION_NOTES § Production Findings 1-5): 1. DAAF_NESTED save/restore (daaf.ps1, update_daaf.ps1, migrate_daaf.ps1): six sites set $env:DAAF_NESTED="1" around child-script calls and then unconditionally Remove-Item'd it, destroying a parent-inherited value and un-suppressing exit pauses for the rest of a nested session. All six now capture the prior value and restore it in finally (Remove-Item only when it was genuinely unset). Env-var-only change; exit-code handling and existing try/finally structure untouched. The .sh twins are immune (per-invocation DAAF_NESTED=1 prefixes) and unchanged. 2. Truthful tracking messaging (migrate_daaf.sh + .ps1, Era-1 and §6b sites): "Tracking set: main -> origin/main" printed unconditionally while the underlying git branch --set-upstream-to failure was swallowed. Both twins now run the checked verbose-git idiom and print the success line only on exit 0, else an honest NOTE. Non-fatal, no control-flow change. 3+5. Shallow- and replace-aware graft probe (migrate_daaf.sh + .ps1): the "graft already in place" detector derived its input from rev-list --max-parents=0, which is corrupted by the very conditions it checks — a shallow boundary commit shows phantom parents (false "already grafted", graft skipped) and a prior replace ref makes rev-list walk through to upstream's parentless root (false "no graft", redundant re-graft). Replaced with a three-leg probe: captured git replace -l non-empty -> graft in place; else rev-parse --is-shallow-repository true -> warn and proceed to graft; else the original parent-count check. Probes fail toward grafting (replace is the persistent marker; migrate is the sole replace-ref creator). Graft application unchanged; twins identical. 4. grep -q pipefail-inversion sweep: all 21 existing `| grep -q` uses in scripts/host/*.sh verified safe (pre-captured producers or direct file greps). New lint section 9 in check-daaf-conventions.sh flags a live command piped into grep -q (SIGPIPE + pipefail silently inverts the result), exempting the sanctioned echo/printf capture-then-grep idiom, with the two accepted false-positive classes documented. Compliance-checklist row added to the shell-scripting skill. Tests: behavioral DAAF_NESTED restore tests plus count-based shape guards in all three Pester suites (the old unconditional pattern now fails the count invariant); replace-leg idempotency, shallow-forces-graft, and tracking success/failure tests in both bats and Pester; lint-rule bats coverage (passes current tree, catches seeded violation); three tautological probe unit tests deleted in favor of the integration tests. Verification: check-daaf-conventions.sh 0 failures (incl. Bash 3.2 portability); bats migrate_daaf 56 ok, check_daaf_conventions 10 ok; full tests/powershell Pester suite 595 passed / 0 failed; dry-run write-site sweep clean; exec bits 100755 preserved. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add moonshotai/kimi-k3 to models.yaml (identity, pricing, and 1,048,576-token context verified online 2026-07-17 from openrouter.ai/moonshotai/kimi-k3; single listing, no variants; $3/M in, $15/M out, no cache discount on this route) with all three purity env selectors pinned to the K3 slug for the G1R child-purity validity gate. Update test_chatgpt_route.py: G1R_OPENROUTER_ADDITIONS set, openrouter provider count 17->18, new K3 purity-pinning test. Suite 79/79. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…data The OpenRouter model page displays no cache rate for moonshotai/kimi-k3, but GET /api/v1/models/moonshotai/kimi-k3/endpoints (accessed 2026-07-17) lists input_cache_read at $0.30/M (supports_implicit_caching=false; discounts apply via explicit cache_control breakpoints, which the Claude CLI sends). Add cached_input: 0.30 to the registry entry and update the K3 test. Suite 79/79. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…fresh track, and richer fixtures
Revise test_migration.{sh,ps1} for full install/update-vector coverage: an --all/-All matrix driver running fresh/v1.0.0/v2.0.1/v2.1.0 as isolated child processes with a machine-readable TEST_MIGRATION_SUMMARY line and PASS/FAIL/INFRA scoreboard (reconciled against child exit codes); a first-class fresh-install track incl. second-install-refusal; fixture classes A-E encoding per-file-class survival semantics (mergeable tracked edits, observe-only deliberate conflict, never-touched settings hash, drift-heal inverted expectation) with per-era capability probes; an opt-in auto mode riding each platform's own non-interactive seam (CI=1 on bash, stdin redirection on PowerShell); explicit update-driving with self-update two-run detection; and a Phase 7b newest-endpoint block (rebuild evidence, noble base probe, functional smoke). Adds docker-free unit suites (26 bats / 27 Pester) via a DAAF_TEST_MODE source seam; CI picks them up automatically. All 3-angle review findings fixed and re-verified; suites green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…script sync
The updater's sync_host_scripts (sh) / sync helper (ps1) derives its host-script list dynamically via git ls-files scripts/host/*, so the tracked contributor test harness test_migration.{sh,ps1} - which matches the *.sh / *.ps1 platform filters - would be copied into end-user daaf-docker folders on every update. Add it to the same exclusion path already used for the install/migrate bootstrap scripts on both platforms, keeping the harness tracked in git for CI while never syncing it to user hosts. Extend the existing platform-filter exclusion tests in both suites to pin the new exclusion (docker-free, mock-based; 69 bats / 83 Pester green).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ke portability
Two independent failures on the CI - Scripts workflow (run 29551213598):
- ci-scripts.yml 'Verify RV test dependencies': cat("R ", getRversion(), ...)
errors because getRversion() returns a list-backed R_system_version that
cat() cannot print; wrap in as.character(), matching the adjacent
packageVersion() calls on the same line. Fix verified against R 4.5.3.
- deploy-smoke harness: smoke_probes.py hardcoded BASE_DIR="/daaf" and
_run_battery_cmd defaulted cwd=BASE_DIR; on GitHub runners /daaf does not
exist, so subprocess.run raised FileNotFoundError, misclassified as SKIP
('tool unavailable') — the 4-of-21 BatteryEvidenceTests CI failure.
* BASE_DIR now derives from module location (Path(__file__).resolve()
.parents[2]): /daaf in-container, checkout path on CI.
* _run_battery_cmd pre-checks cwd and returns FAIL (not SKIP) when the
working directory is missing — fail-closed for an evidence harness;
FileNotFoundError now strictly means a missing executable.
* run_deploy_smoke.py reuses smoke_probes.BASE_DIR (single source of truth).
* Hardcoded '-daaf' Claude projects-dir slug in find_transcript /
find_subagent_transcripts replaced with BASE_DIR-derived _project_slug()
(re.sub(r'[^A-Za-z0-9]', '-', ...)), verified equivalent to Claude Code's
documented encoding (claude-code#19972).
* Tests: add test_missing_cwd_is_fail_not_skip and ProjectSlugTests (3,
incl. live-invariant pin). Suite: Ran 25 tests ... OK.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ures; bump actions off Node 20 - rename Emit-SummaryOnce -> Write-SummaryOnce in test_migration.ps1 (PSUseApprovedVerbs gated both PSScriptAnalyzer jobs) - rewrap environment_settings_example.txt ChatGPT-lane paragraph so no comment line matches the hash-space-key activatable-assignment lint (BATS assertion (d)) - bump actions/checkout v4 -> v5 (15 uses across both workflows) and actions/setup-python v5 -> v6, clearing the Node 20 deprecation notices - sync the shell-scripting skill's illustrative CI examples to checkout@v5 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, viewer ephemeral paths - FRAMEWORK_INTEGRATION_CHECKLIST: add HS12 (compose-volume persistence), AM6 (stage-participation table sweep), CC8 (code-example sweep, extends CC7), M9 mode-file routing caveat - VALIDATION_CHECKPOINTS + QA_CHECKPOINTS: hard-coded-PASS 'derive, never assert' anti-pattern with Python/R wrong-right pairs and bidirectional cross-links - skill-authoring gotchas: library skill version-drift doctrine (package-index probe + installed-build introspection) with Topic Index row - session-recovery: Step 5b model awareness on resume (three-profile threshold vocabulary, dispatch-ceiling re-planning) - tests: claude-fable-5[1m] regression cases (context_reporter 54/54, subagent_bar 45/45 green) - log_viewer: pathLink() single render point marks /tmp paths as ephemeral non-links at all six link sites - hooks: correct stale search-agent model claims in enforce-explore-model + deny-claude-code-guide (user-applied via sed) Backlog consolidated from ~50 FrameworkDev session notes; every item evidence-verified open before authoring and 3-angle reviewed after. Session workspace (research/, git-excluded) holds the full evidence trail. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…endering The log viewer now marks /tmp paths as explicitly-ephemeral, unarchived references (pathLink(), commit 0779f31) rather than rendering them as broken-looking live links. Core warning unchanged: /tmp files are outside the backup/audit boundary and lost silently. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add request-scoped correlation, structured lifecycle accounting, bounded backend-error normalization, and deterministic production-path regressions for shim v1.2.11. Stage only the session-owned documentation hunks so concurrent reference edits remain untouched. Co-Authored-By: Claude <noreply@anthropic.com>
…atrix, branch-faithful updates, Era-3 tag normalization - ps1: summary contract (state + Write-SummaryOnce with driver guard + Complete-Run + trap) relocated above the matrix driver — PS traps are scope-wide while functions are positional, so a child stderr line under EAP=Stop fired the trap before Write-SummaryOnce existed and killed the whole matrix (CommandNotFoundException). Child pipeline now runs under scoped EAP=Continue with stringified records; new Invoke-NativeLogged wraps the six Era-1/2 bare native calls. - Both twins: driven update_daaf now receives DAAF_BRANCH=MIGRATION_BRANCH on both runs — without it the updater auto-detected main and merged GitHub origin/main instead of the branch under test (no noble Dockerfile, no rebuild exercised). - Both twins: Era-3 TAG normalization completes the time-machine git state (remote set-branches + shallow fetch of main + set-upstream), replicating a real 'clone -b main' install; fixes the tracking '' failures. - Both twins: Era-1 verify failure path surfaces the raw git stderr and /daaf ownership (ls -ldn) — the v1.0.0 INFRA diagnosis was hidden by container_git's stderr suppression. - Both twins: Class D comparison now modulo the active DAAF_BRANCH line (the driven update's env-origin branch persist is a sanctioned mutation). - Suites: bats 26→30, Pester 27→35 (regression pins for each fix). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Field finding (v2.0.1 vector, class E): on old-era migrations every host script is 'changed in range', so tier B — not the tier C drift heal — performs the heal, and it overwrote locally drifted copies with NO .pre-update backup, breaking tier C's documented recoverability contract. _sync_copy_one / Copy-HostScript now stage the incoming copy as a sibling <name>.sync-staged, compare (cmp -s / SHA256), save a differing existing copy to the rolling <name>.pre-update BEFORE overwrite, and never overwrite when the backup fails (tier C's never-destroy rule). Missing-target path unchanged. sh gains the ps1's staged-existence guard for exact parity; ps1's docker cp calls are $null-wrapped so the returned boolean can never ride an output-stream leak as a truthy array (probe-verified). Suites: bats 69→71 (+2 tier-B backup behavioral tests; 3 mocks now materialize cp destinations, required by the staging design), Pester 83→87 (+2 behavioral mirrors, self-update mocks materialized). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ondition The honest-failure NOTE (4cd280d) always blamed a missing local 'main' branch, but on tag-pinned/single-branch installs the actual missing piece is the origin/main remote-tracking ref (field run 4: git's setUpstreamFailure hint above the NOTE on the v2.1.0 vector). Both set-upstream sites in both twins now probe rev-parse --verify refs/remotes/origin/main and report whichever precondition actually failed. Suites: bats 56→57, Pester +1 (content pins on both messages + probe). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add section 0 excision so `git commit -m` bodies that merely describe
dangerous commands stop false-blocking: single-quoted bodies are excised
unconditionally (POSIX-inert), double-quoted bodies only when free of
backtick / $( / ${ ; any ambiguity fails closed to leaving the text intact
and still scanned.
Tighten the section 3 / section 5 pre-context openers with a backtick and
$( so a substitution-embedded sudo/su/docker-run/mount inside a
double-quoted commit message (which executes at commit time) is now caught,
closing a pre-existing false negative.
Tests: extend scripts/test_safety_hooks.sh to 145/145 and add
tests/bash/bash_safety.bats (28/28). Both draft-testable via a path arg /
BASH_SAFETY_SH override. Docs: update the CLAUDE.md Defense-in-Depth row and
the shell-scripting testing.md battery description.
Reviewed by a three-angle pass (adversarial red-team SHIP verdict, plus
consistency and completeness). Scope: fixes commit-message false positives
and one substitution false negative; does not close the general
regex-guard bypass class, which remains covered by container isolation and
the permission layers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…updates, provider-agnostic onboarding Consolidates the launch docs strands: the docs-qualification pass, the UX/approachability pass, and the public CHANGELOG v3.0.0 section. - Getting Started video featured across README and guides (thumbnail, header row, demos re-ranking, install-guide on-ramps) - Provider-agnostic "AI Provider Account & Authentication" section with references synced repo-wide - Accessible OpenAI/ChatGPT shim walkthroughs (who-for leads, numbered steps, diagnostics moved to an under-the-hood reference); shim de-versioned throughout (internal-dev-only versioning) - Control-Panel-first instructions across guides and FAQs, option numbers verified against daaf.sh - Current-state editing pass: changelog voice removed from user docs - New additions: evidence-graded reporting explainer, custom-script constraints, safety intro, status-bar/agent-panel FAQ (Claude- subscription telemetry note), Stata/R translation note, file management tips, 15-20GB storage guidance - At-tag updates: CITATION.cff and all version strings to 3.0.0 / 2026-07-17; self-updating Zenodo DOI badge (latestdoi) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…lict A genuine merge conflict makes the non-interactive updater exit 1 mid-merge (correct). Previously the post-resolution re-run landed on an "already up to date" early exit that did tier-A-only sync -- skipping rebuild detection, tier-B host-script sync, and the stranded "DAAF update backup" stash pop. Persist a resume marker (/daaf/.git/daaf-update-resume: OLD_HEAD + TIMESTAMP) at every conflict-exit site. On the re-run, detect a still-in-progress merge/rebase (guide + exit 1) or a valid marker (adopt the recorded pre-update HEAD, restore the stash, run finish_update against that baseline). Also route the default-branch AHEAD>0 landing into the resume finalizer: a committed merge-conflict resolution leaves HEAD a merge commit (AHEAD>0, BEHIND=0), which previously re-entered the merge/rebase menu and stranded the stash; now RESUMING finishes the interrupted update first and, when BEHIND>0, tells the user to re-run for the fresh pull. finish_update clears the marker as the single success chokepoint. Mirrored in update_daaf.ps1. Suites: bats 71->83, Pester 85->97. Round-5 field finding 1 + review fix cycle 1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…r git ops Round-5 field evidence (v1.0.0, Mac + Windows): the Era-1 volume payload at /daaf is root-owned while the v1.0.0 image runs git as non-root appuser, so git >=2.35.2 refuses every in-container git op with "detected dubious ownership" -- and container_git swallows stderr, so migrate failed silently and a real v1.0.0 user could not migrate. Both twins now issue an idempotent git config --global --add safe.directory /daaf for the exec user before the first in-container git op (era detection). A single early guarded config-add covers all eras (exec user, container, and path are shared across era branches); harmless no-op on v2.x payloads; fail-fast with a clear diagnostic if the config-add itself fails. Regression tests pin issuance, the /daaf target, and before-era-detection ordering. Suites: bats 57->59, Pester 72->76. Round-5 field finding 3a. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…irectory window, ps1 stderr Finding 2: on v2.x->v3.0.0 vectors the driven update hits a guaranteed CLAUDE.md merge conflict (Class C committed append vs the v3.0.0 rewrite) and the non-interactive handler exits mid-merge; the harness scored that abort plus four cascade checks as five FAILs, permanently masking the stash/rebuild/Class-E assertions. The auto-mode update-driving block now probes the mid-merge state (capture-then-test on MERGE_HEAD + the unmerged set) and, when it is exactly the CLAUDE.md conflict, simulates the guided resolution a real user performs (checkout --theirs, re-preserve the Class C prose, commit) and re-drives the resumable updater, scoring the full conflict->resolved->resumed journey. Any other unmerged set still fails loudly. Eligibility is a pure unit-tested helper in both twins. Finding 3b: a guarded safe.directory exemption window spans the harness pre-migrate old-container git ops (phase 3 verify + phase 4/5 fixture plants), opened before the first such op and closed before migrate runs (only if the harness added it) so the migrate section-4b fix is still exercised end-to-end on the root-owned Era-1 payload. Finding 3c: Era-1 raw-git diagnostics in test_migration.ps1 now capture native stderr under PS 5.1 (scoped EAP=Continue + 2>&1 + per-object stringify) -- was empty on Windows where the .sh twin printed the dubious-ownership fatal. Suites: bats 30->43, Pester 35->50. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…r; normalize bats mode The updater feature list now tells users that after resolving and committing a merge conflict, re-running the updater picks up where it left off -- restoring set-aside changes and finishing script sync and the rebuild check automatically (documents the resume feature from 0b757ce). Also folds in the user-authored Getting Started website guide pointer at the top of the install guide. tests/bash/test_migration.bats mode normalized 100644 -> 100755 to match its sibling suites. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…f-init parity) Round-6 field evidence (Mac + Windows): the documented v1.0.0 install (busybox cp -a into the volume) leaves the payload root-owned, and the v1.0.0 compose has no daaf-init repair service (that arrived in v2.0.0, whose compose comment names this exact defect). The section-4b safe.directory exemption cures git's dubious-ownership refusal but not writability: Era-1 fetch/set-upstream/merge all EPERM as the non-root uid-1000 container user, so a real stranded v1.0.0 user still could not migrate. New section 4c runs the daaf-init-parity repair (busybox chown -R 1000:1000) unconditionally before era detection -- idempotent no-op on Era-2/3 payloads already repaired by their own compose -- with a warn-and-continue failure policy. Both twins + bats/Pester coverage (behavioral chown-before-era-detection ordering + source pins). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… writability check Round-6: the v1.0.0 vector cleared phase 3 for the first time (round-5 safe.directory window worked) then INFRA'd at phase-4 fixture writes -- the documented Era-1 install leaves the volume payload root-owned while the container user is uid 1000. New Era-1-gated ownership repair window mirrors the safe.directory window: capture the original owner (busybox stat), chown to 1000:1000 after the phase-3 verify so phases 4-5 plant through the same code paths as every era, then restore the CAPTURED owner immediately before migrate -- handing migrate the field-faithful broken state (root-owned payload including user work) so migrate's own section-4c repair is exercised end-to-end. Window-management failures are fatal (INFRA). Plus a universal phase-7 check on every vector: post-migration container user can write /daaf. ps1 owner capture under scoped EAP=Continue + stringify (PS 5.1); chowns via Invoke-NativeLogged. Suites: bats 43->47, Pester 50->54 (window placement/ordering pins, byte-identical twin strings, writability check both arms). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…update (4d)
Round-6b field evidence: the v1.0.0 image provisions NO git identity (no
entrypoint; Dockerfile installs git bare), while v2.0.x/v2.1.0 entrypoints
set repo-local daaf@local / 'DAAF Container' on startup and the modern
Dockerfile bakes it globally. Git refuses commit-creating operations
without one ('Please tell me who you are'), so on a migrated-but-not-yet-
rebuilt v1.0.0 container the offered update's stash/merge machinery would
fail. New section 4d sets the exact era-parity identity repo-local (in the
volume, surviving rebuild) ONLY when user.email resolves empty -- a real
user's identity is never overwritten -- with warn-and-continue on failure,
mirroring 4c. Both twins + bats/Pester pins (guard, values, placement
before era detection, warn policy).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…oud fixture commits The v1.0.0 fixture commit died SILENTLY: the era image has no git identity, git refused the commit, and container_git/Invoke-ContainerGit swallow stderr (Mac: set -e INFRA with no diagnosis; Windows: caught only by the fixture-commit verify gate). Third harness window, mirroring the safe.directory and ownership windows: guarded identity OPEN (repo-local harness identity, only when none exists -- v2.x records a no-op note) before fixture commits, gated CLOSE (--unset) before migrate so migrate's own section-4d era-parity repair is exercised end-to-end -- ordered BEFORE the ownership re-break, since the unset writes /daaf/.git/config. Fixture add/commit now run through loud-stderr helpers (sh: container_exec git; ps1: Invoke-NativeLogged -- Invoke-ContainerExec also suppresses stderr) so future failures print git's own message. Suites: bats 47->49, Pester 54->56 (ordering incl. close-before-re-break, guard/gate, loud-commit pins). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
v3.0.0 -- 2026-07-17
Data Analyst Augmentation Framework -- Opening Doors
If v1.0.0 was about proving the concept of DAAF, v2.0.0 (Gaining Altitude) about rebuilding DAAF’s core framework engine for far greater flexibility and robustness, and v2.1.0 (The Frictionless Update) was about making it genuinely easy to install and run, then v3.0.0 is about something more fundamental to the mission of DAAF: who gets to use DAAF at all. This is an update designed to better meet researchers where they are. Every earlier version was geared towards a particular kind of user: someone who's comfortable enough in Python, who has an Anthropic subscription/API budget to spend, whose data is non-proprietary and/or public enough to be safe to share with AI assistants, and who just wants to run a single project on a single machine. Each of those assumptions was a barrier to entry and a limitation of what DAAF could offer to researchers across the sciences.
v3.0.0 takes almost all of these most impactful walls down, and I'm hoping that if you're reading this, you'll find DAAF is now for you.
All of this comes while actually enhancing the safety guardrails in the same sweep. This release expands DAAF's defense-in-depth safety guards, adds more pre-flight system checks, hardens and enhances the development environment experience and workflow for contributors, fixes several research-integrity gaps, and is backed by extensive model-performance testing through our new DAAFBench analyses.
Everything about DAAF is free and open-source, and always will be. The goal with today's update is to make DAAF accessible and relevant to more researchers, in more circumstances, as carefully and responsibly as ever.
See full Release log for notes!