Restore OpenHands xhigh propagation after PR #911#920
Conversation
bingran-you
temporarily deployed
to
pypi-internal-preview
GitHub Actions
Inactive
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cc784ef31b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| 'if [ -n "$LLM_REASONING_EFFORT" ]; then ' | ||
| 'printf \',"reasoning_effort":"%s",' | ||
| '"litellm_extra_body":{"reasoning_effort":"%s"}\' ' | ||
| '"$LLM_REASONING_EFFORT" "$LLM_REASONING_EFFORT"; fi; ' |
There was a problem hiding this comment.
When OpenHands runs are configured through BenchFlow's reasoning_effort field/--reasoning-effort, the value is passed separately to connect_acp(..., reasoning_effort=...) and is not added to agent_env as LLM_REASONING_EFFORT; because OpenHands has no acp_effort_config_id, _configure_acp_session then fails closed instead of reaching this launcher branch. This means the new xhigh settings only work for callers who manually duplicate the value as --agent-env LLM_REASONING_EFFORT=xhigh, so the normal experiment/config path for OpenHands xhigh remains broken.
Useful? React with 👍 / 👎.
1 participant
Summary
LLM_REASONING_EFFORTin the OpenHands settings JSONhighfrom overriding requestedxhighProduction evidence
A 72-row GPT-5.6-SOL/OpenHands batch recorded
reasoning.effort=highon every audited provider response even though both xhigh env variables were present. The only material difference from the known-good canaries was that those canaries ran before commit9c114d3removed the OpenHands-side setting. Direct Azure probes up to concurrency 16 returned xhigh on every successful response, isolating the downgrade to the agent/proxy request path rather than the provider.Validation
uv run pytest -q tests/test_agent_registry.py(26 passed)uv run ruff check src/benchflow/agents/registry.py tests/test_agent_registry.pygit diff --checkA fresh Daytona xhigh canary will gate resuming official experiment rows.