Preserve OpenHands context budget

[Yiminnn]

What changed

• generate a complete OpenHands Agent settings file instead of only the minimal LLM route
• pass explicit input/output limits when BENCHFLOW_OPENHANDS_CONTEXT_LIMIT and BENCHFLOW_OPENHANDS_OUTPUT_LIMIT are set
• enable LLMSummarizingCondenser with BENCHFLOW_OPENHANDS_CONTEXT_RESERVE (default 4096)
• install the settings writer during the root-owned agent install phase, then invoke it through a host-owned launch override
• keep install_cmd and launch_cmd byte-identical to the external agent manifest by modeling the compatibility hooks as shim-only fields
• add regression coverage for generated settings, root-owned installation, and the real rollout launch seam

Root cause

BenchFlow wrote only llm.model, llm.api_key, and optional provider fields to agent_settings.json. That bypassed the OpenHands CLI's default agent construction and left condenser=None plus no explicit input/output limits. A long rollout reached 262,216 input tokens against a 262,144-token endpoint and ended in a provider context-window rejection instead of condensing first.

Behavior

With context 262144, output 32768, and the default 4096 reserve, condensation starts at 225280 tokens. Existing provider URL and optional LiteLLM boolean flags are preserved. When limits are omitted, the writer still restores the normal summarizing condenser without inventing a token threshold.

Validation

• Ruff passes
• 259 targeted tests pass
• 12/12 tests pass against the live benchflow-ai/agents@main manifests
• GitHub test, manifest-parity, and real OpenHands rollout-smoke checks pass
• live Daytona read-back verified Agent, input 262144, output 32768, LLMSummarizingCondenser, trigger 225280
• 24 publishable post-patch rollouts / 540 assistant turns: zero context rejections, malformed tool calls, raw tool XML leaks, or <think> leaks

The production experiment patch was deployed only to newly launched jobs; existing active jobs were not restarted.

[bingran-you]

Daily scan hygiene (2026-07-13): added rollout/eval tracking labels while this remains a draft PR.

Current state:

• Head ff29b1637b97f30f025ad544df79e51d0aac3a01 has green core checks (test, pip-audit, manifest-parity, integration-light / rollout-smoke).
• Not merge-safe today because the PR is still draft and has no non-author human approval.

Labels: enhancement, P1, status:in-progress, area:rollout, area:eval.

[bingran-you]

Users Simulation automation review (2026-07-13): behavior-ready, but still draft/in-progress on head ff29b1637b97f30f025ad544df79e51d0aac3a01.

Scope checked: OpenHands settings generation, context/output budget propagation, summarizing condenser setup, manifest-neutral install/launch behavior, CLI/SDK regression risk, and artifact health.

Evidence:

• Shared base smoke passed: uv sync --extra dev --extra sandbox-daytona --locked, ruff check ., ty check src, CLI help/list/check surfaces, and SDK Scene.single(agent="oracle") + RolloutConfig.
• PR worktree checks passed: uv sync --extra dev --locked; pytest tests/test_openhands_context_budget.py tests/test_registry_invariants.py tests/agents/test_manifest_parity.py tests/test_runtime_config_wired.py tests/test_source_adapters.py -q -> 213 passed, 2 skipped; focused ruff; ty check src; CLI + SDK smoke.
• Direct writer smoke with BENCHFLOW_OPENHANDS_CONTEXT_LIMIT=262144, BENCHFLOW_OPENHANDS_OUTPUT_LIMIT=32768, and reserve 4096 produced Agent settings with LLMSummarizingCondenser, agent/condenser usage IDs, max input/output limits, and condenser max_tokens=225280.
• GitHub current-head checks are green: test, pip-audit, manifest-parity / parity, and integration-light / rollout-smoke passed.

Artifact health:

• Downloaded run 29217686724 and validated it with benchflow-experiment-review: 1/1 healthy, ACP + LLM trajectories present, results.jsonl training-ready, reward 1.0, 18/18 LLM exchanges with provider usage, 309,996 tokens, 16 tool calls, timing total 296.1s.
• Caveat: GLM user-endpoint cost accounting is still unpriced (agent_result.cost_usd=null, price_source=null) even though token/timing telemetry is complete.

Thermo-nuclear review: no blocker from this diff. The PR edits an already-large registry file, but the new settings writer is cohesive and avoids scattering context-budget branches across rollout code. I left labels as draft status:in-progress; do not merge until it is marked ready and gets non-author human review.