Skip to content

Tell the agent about the connected apps in the system prompt - #36

Merged
milind-soni merged 2 commits into
milind-soni:mainfrom
guilimasp:fix/composio-in-system-prompt
Aug 16, 2026
Merged

Tell the agent about the connected apps in the system prompt#36
milind-soni merged 2 commits into
milind-soni:mainfrom
guilimasp:fix/composio-in-system-prompt

Conversation

@guilimasp

@guilimasp guilimasp commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

When a Composio key is configured the harness mounts the Connect MCP server and pre-allows its tools, but the persona never mentions it — the computer and agents integrations each get a line, Composio gets none.

The result is that the model does not link a request like "can you read my email?" to the mcp__composio__* tools it is holding. It answers that it has no access, or starts hunting for an unrelated integration to authorize, while a working Gmail connection sits one tool call away.

Asked the same question before and after this line: before, the bot said it had no access and suggested authorizing a different connector; after, it searched the toolkits, ran the tool and answered from the mailbox.

One line, in the same shape as the two beside it. pnpm typecheck && pnpm test pass.

Summary by CodeRabbit

  • New Features

    • Connected Composio apps are now available through supported providers as HTTP MCP integrations.
    • Supported providers advertise whether Composio integrations are available.
    • Connected-app guidance is shown only when the integration is active.
  • Bug Fixes

    • Prevented Composio integrations from being mounted for providers that do not support them.
  • Tests

    • Added coverage for integration mounting, authorization, API key forwarding, and capability reporting.

@milind-soni milind-soni left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intent is good, but this hint is currently gated only on integrations.composio. startTurn sets that integration for every provider whenever a key exists, while server/drivers/claude.ts is the only driver that mounts the Composio MCP server; Codex, Grok, and ACP drivers ignore it. That means this change would tell non-Claude bots to call tools they do not have. Please gate both the integration/prompt through an explicit capability (preferred), or at minimum restrict this prompt to the Claude driver until other drivers mount Composio.

When a Composio key is configured the Claude driver mounts the Connect
MCP server and pre-allows its tools, but the persona never mentions it —
so the model does not link "can you read my email?" to the composio tools
it is holding, and answers that it has no access.

Add the line, gated the way the computer and agents hints already are: a
composioMcp capability the driver declares, which gates the integration
itself. A key in the config says the user has those connections, not that
this engine can reach them — Codex, Grok and the ACP drivers ignore
integrations.composio, and must not be told otherwise.

Tests: Claude mounts and claims it; ACP claims neither.
@guilimasp
guilimasp force-pushed the fix/composio-in-system-prompt branch from 522d7ca to 595be3a Compare August 14, 2026 14:29
@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@milind-soni, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 34 minutes

Limit details: You’ve used all 3 included reviews currently available under your plan.

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 39f2a57b-be00-4c20-bc16-921729b9f7cd

📥 Commits

Reviewing files that changed from the base of the PR and between 595be3a and ca4ac9e.

📒 Files selected for processing (3)
  • server/drivers/acp/acp.test.ts
  • server/drivers/claude.ts
  • server/index.ts

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b6f5fc30-2247-46a7-9cf7-31efd01d838f

📥 Commits

Reviewing files that changed from the base of the PR and between 13a1bb7 and 595be3a.

📒 Files selected for processing (5)
  • server/contracts.ts
  • server/drivers/acp/acp.test.ts
  • server/drivers/claude.test.ts
  • server/drivers/claude.ts
  • server/index.ts

📝 Walkthrough

Walkthrough

The provider capability contract now identifies Composio MCP support. The Claude driver enables this capability, ACP coverage confirms it remains disabled, and server wiring gates Composio credentials and prompt text on the selected adapter’s capability.

Changes

Composio MCP capability gating

Layer / File(s) Summary
Capability contract and driver metadata
server/contracts.ts, server/drivers/claude.ts, server/drivers/claude.test.ts, server/drivers/acp/acp.test.ts
The adapter contract adds composioMcp. Claude enables the capability and tests verify its MCP configuration. ACP tests verify that unsupported drivers do not enable it.
Conditional Composio integration wiring
server/index.ts
Composio credentials and the system-prompt hint are added only when the selected adapter supports Composio MCP.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 595be

This localized prompt change makes the existing connected-app tools discoverable to the agent without altering permissions or runtime behavior; no actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: milind-soni, clewkord, aivsomkar

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the primary change: informing the agent about connected apps in the system prompt.
Description check ✅ Passed The description explains what changed, why it changed, and how it was verified; omitted template headings and checklist items are non-critical.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@guilimasp

Copy link
Copy Markdown
Contributor Author

You are right, and thank you — gating on cfg.composio.key would have told Codex, Grok and every ACP bot to call tools they were never handed. Took the preferred route.

  • New capabilities.composioMcp, documented next to agentsMcp and computerMcp and following the same rule: the flag means this driver mounts the integration, not the user has connections.
  • claude.ts declares it (it is the only driver that mounts the Connect MCP server today). Nothing else does.
  • startTurn now gates integrations.composio on the capability, so a non-Claude bot gets neither the tools nor the hint. The prompt line hangs off integrations.composio rather than the key, the way the agents hint already hangs off integrations.agents — so the two can never drift apart.

Tests: claude.test.ts asserts it both mounts the server (pre-allowing mcp__composio) and claims the capability; acp.test.ts asserts ACP does not claim what it cannot honour. Any future driver that mounts Composio flips one flag and picks up the prompt with it.

pnpm typecheck && pnpm test pass. Rebased onto current main.

@milind-soni milind-soni left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The requested provider-capability gating is now implemented: Composio configuration and prompt guidance are exposed only through adapters that declare composioMcp support. I refreshed the branch on current main while preserving both ACP/Droid test coverage, then verified the full test/build suite and the hosted macOS, Ubuntu, Windows, and Linux package-smoke matrix.

@milind-soni
milind-soni merged commit 51de669 into milind-soni:main Aug 16, 2026
5 checks passed
milind-soni added a commit that referenced this pull request Aug 16, 2026
The system-prompt hint added in #36 tells every Composio-enabled bot to
"run it with COMPOSIO_EXECUTE_TOOL". That tool does not exist. Queried
the live endpoint the driver mounts (connect.composio.dev/mcp) with the
repo's own key: tools/list returns seven tools, and calling
COMPOSIO_EXECUTE_TOOL answers `-32602: Tool COMPOSIO_EXECUTE_TOOL not
found`. The executor is COMPOSIO_MULTI_EXECUTE_TOOL, and the real flow
has a middle step the hint skipped — a tool's arguments come from
COMPOSIO_GET_TOOL_SCHEMAS.

So the hint sent bots to a dead name on the one path it was written to
unblock: a bot that trusts it burns its turn on a -32602 and then tells
the user it has no access, which is exactly the outcome #36 set out to
prevent. Now: SEARCH_TOOLS -> GET_TOOL_SCHEMAS -> MULTI_EXECUTE_TOOL.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@milind-soni

Copy link
Copy Markdown
Owner

Heads-up after this merged: the prompt line named COMPOSIO_EXECUTE_TOOL, which doesn't exist on the endpoint the driver mounts. I checked tools/list against connect.composio.dev/mcp with the repo's own key — there are seven tools, and calling that name returns -32602: Tool COMPOSIO_EXECUTE_TOOL not found. The executor is COMPOSIO_MULTI_EXECUTE_TOOL, and arguments come from COMPOSIO_GET_TOOL_SCHEMAS, so search→execute alone can't build a valid call.

Fixed in #126 (merged): the hint now says SEARCH_TOOLS → GET_TOOL_SCHEMAS → MULTI_EXECUTE_TOOL. The idea was right — a bot that doesn't know it has connected apps tells the user it has no access — it just needed the real tool names.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants