Skip to content

Propagate OpenHands MAX reasoning effort#54

Merged
bingran-you merged 6 commits into
mainfrom
bry/openhands-max-reasoning
Jul 14, 2026
Merged

Propagate OpenHands MAX reasoning effort#54
bingran-you merged 6 commits into
mainfrom
bry/openhands-max-reasoning

Conversation

@bingran-you

Copy link
Copy Markdown
Collaborator

Summary

  • keep OpenHands typed reasoning_effort for values through xhigh
  • route MAX through litellm_extra_body.reasoning.effort because OpenHands SDK 1.22.1 rejects typed max
  • restore byte-identical parity with BenchFlow PR #921

Validation

  • changed OpenHands manifest loads against the contract schema
  • BenchFlow live manifest parity: 12 passed
  • git diff --check

Note: the agents repo broad contract suite currently has an unrelated existing corust-agent acp_model_format=prefixed schema failure on main.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 93b3c55e5c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread acp/openhands/manifest.toml Outdated
description = "OpenHands agent via ACP (multi-model, Python-based)"
install_cmd = "export DEBIAN_FRONTEND=noninteractive && export PATH=\"$HOME/.local/bin:$PATH\" && ( command -v curl >/dev/null 2>&1 && command -v git >/dev/null 2>&1 || if command -v apt-get >/dev/null 2>&1; then ( attempt=1; while [ \"$attempt\" -le 3 ]; do rm -rf /var/lib/apt/lists/*; apt-get clean; if apt-get -o Acquire::Retries=3 update -qq && apt-get -o Acquire::Retries=3 install -y -qq curl ca-certificates git; then exit 0; fi; case \"$attempt\" in 1) sleep 2; attempt=2 ;; 2) sleep 4; attempt=3 ;; *) sleep 6; attempt=4 ;; esac; done; exit 1 ); elif command -v dnf >/dev/null 2>&1; then dnf -y --allowerasing install curl ca-certificates git >/dev/null 2>&1; elif command -v apk >/dev/null 2>&1; then apk add --no-cache curl ca-certificates git >/dev/null 2>&1; else echo 'OpenHands GitHub install requires curl and git' >&2; exit 127; fi ) && ( UV_OK=0; if command -v uv >/dev/null 2>&1; then UV_VER=$(uv --version 2>/dev/null | awk '{print $2}'); if [ -n \"$UV_VER\" ] && [ \"$(printf '%s\\n' 0.11.6 \"$UV_VER\" | sort -V | head -n1)\" = \"0.11.6\" ]; then UV_OK=1; fi; fi; if [ \"$UV_OK\" = 0 ]; then curl -LsSf https://astral.sh/uv/install.sh | sh >/dev/null 2>&1 && export PATH=\"$HOME/.local/bin:$PATH\"; fi && printf 'openhands-sdk==1.22.1\\nopenhands-tools==1.22.1\\n' > /tmp/oh-sdk-overrides.txt && uv tool install --force --refresh --overrides /tmp/oh-sdk-overrides.txt --from 'git+https://github.com/OpenHands/OpenHands-CLI.git@3ca17446c5d9c1e35e054803478a3501ec251ecf' openhands --python 3.12 && uv tool list | grep -q '^openhands\\b' ) && chmod o+x /root /root/.local /root/.local/share /root/.local/share/uv /root/.local/share/uv/tools 2>/dev/null; mkdir -p ~/.openhands && echo '{\"llm\":{\"model\":\"placeholder\",\"api_key\":\"placeholder\"}}' > ~/.openhands/agent_settings.json && command -v openhands >/dev/null 2>&1"
launch_cmd = "export PATH=\"$HOME/.local/bin:$PATH\" && mkdir -p ~/.openhands && { printf '{\"llm\":{\"model\":\"%s\",\"api_key\":\"%s\"' \"$LLM_MODEL\" \"$LLM_API_KEY\"; if [ -n \"$LLM_BASE_URL\" ]; then printf ',\"base_url\":\"%s\"' \"$LLM_BASE_URL\"; fi; if [ -n \"$LLM_API_VERSION\" ]; then printf ',\"api_version\":\"%s\"' \"$LLM_API_VERSION\"; fi; printf '}}'; } > ~/.openhands/agent_settings.json && openhands acp --always-approve --override-with-envs"
launch_cmd = "export PATH=\"$HOME/.local/bin:$PATH\" && mkdir -p ~/.openhands && { printf '{\"llm\":{\"model\":\"%s\",\"api_key\":\"%s\"' \"$LLM_MODEL\" \"$LLM_API_KEY\"; if [ -n \"$LLM_BASE_URL\" ]; then printf ',\"base_url\":\"%s\"' \"$LLM_BASE_URL\"; fi; if [ -n \"$LLM_API_VERSION\" ]; then printf ',\"api_version\":\"%s\"' \"$LLM_API_VERSION\"; fi; if [ \"$LLM_REASONING_EFFORT\" = \"max\" ]; then printf ',\"litellm_extra_body\":{\"reasoning\":{\"effort\":\"max\"}}'; elif [ -n \"$LLM_REASONING_EFFORT\" ]; then printf ',\"reasoning_effort\":\"%s\",\"litellm_extra_body\":{\"reasoning_effort\":\"%s\"}' \"$LLM_REASONING_EFFORT\" \"$LLM_REASONING_EFFORT\"; fi; printf '}}'; } > ~/.openhands/agent_settings.json && openhands acp --always-approve --override-with-envs"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clear the SDK's default high effort for MAX

When LLM_REASONING_EFFORT=max, this branch omits the typed reasoning_effort field, but the pinned OpenHands SDK 1.22.1 defaults that field to high and forwards it on both chat and responses calls. As a result MAX runs still send a conflicting typed high effort alongside the litellm_extra_body override, so gateways/providers that honor the typed field or reject duplicate effort fields will not actually run at MAX. Explicitly clear the typed field for this branch (for example by writing it as null) before relying on the extra body.

Useful? React with 👍 / 👎.

@bingran-you

Copy link
Copy Markdown
Collaborator Author

Updated the branch onto current agents/main and aligned the OpenHands launch command with the final BenchFlow typed-effort routing:

  • max -> litellm_extra_body.reasoning.effort=max
  • none|low|medium|high|xhigh -> typed reasoning_effort plus provider extra body
  • provider-only values -> extra body only
  • LLM_TIMEOUT remains numeric and validated

Validation against the refreshed BenchFlow PR #921 runtime (including companion #924): manifest parity 12/12 passed. The agents registry suite is 79 passed with 2 pre-existing current-main failures (qwen-code shadowing and empty re-registration), unrelated to this manifest change.

This update was generated by Codex on behalf of Bingran You.

@bingran-you

Copy link
Copy Markdown
Collaborator Author

OpenHands manifest sync update (2026-07-13): pushed 3428796 to match BenchFlow PR921 commit 4d8cf1f4f.

  • OpenHands CLI pin: 2df8a2835d3f1bd2f2eadf5a7a2e1ad0dfb0d271
  • openhands-sdk==1.28.1
  • openhands-tools==1.28.1

Validation:

  • agents PR CI: Ruff and Markdown link checks passed
  • BenchFlow live parity against this branch: 12/12 passed
  • matching BenchFlow runtime passed 4,891 tests and a scored, independently audited GPT-5.6-SOL MAX Daytona SSH canary

This manifest update is the external parity dependency for benchflow-ai/benchflow#921.

This update was generated by Codex on behalf of Bingran You.

@bingran-you bingran-you merged commit 5466d21 into main Jul 14, 2026
2 checks passed
@bingran-you bingran-you deleted the bry/openhands-max-reasoning branch July 14, 2026 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant