LocalConversation.fork(agent=...) can clobber the source prompt_cache_key

[enyst]

Summary

I verified the conversation.fork() behavior for the prompt_cache_key change in PR #2907.

What works today

The default fork path is correct:

• LocalConversation.fork() generates a new conversation ID when conversation_id is omitted.
• It builds the fork via LocalConversation(...), which repins agent.llm._prompt_cache_key to the fork's new state.id.
• The source conversation keeps its original key.
• The agent-server / remote fork path uses this same default behavior server-side, so normal remote forks inherit the same good behavior.

I verified this locally on branch 2904-responses-api-prompt_cache_key-never-sent-cache-shard-routing-causes-oscillating-cache-miss-per-turn:

• source ID/key: 41176d21-79e0-412a-b974-02a87b41dd47
• fork ID/key: 2539d579-9efc-4a8e-998e-892246b9b294
• source key after fork remained 41176d21-79e0-412a-b974-02a87b41dd47

The footgun

LocalConversation.fork(agent=...) can overwrite the source conversation's cache key if the caller passes an agent that aliases the source agent or shares the same LLM object.

Why:

• LocalConversation.__init__ mutates the provided LLM in place with:
  • self.agent.llm._prompt_cache_key = str(self._state.id)
• fork(agent=...) uses the supplied agent object directly.
• So conversation.fork(agent=conversation.agent) repins the shared LLM to the fork ID and clobbers the source conversation's key.

I verified that locally too:

• source ID/key before: 41176d21-79e0-412a-b974-02a87b41dd47
• fork(agent=conv.agent) created fork ID/key: 398100de-cf1f-4029-b758-ab5755de9a8a
• source key after the fork also became 398100de-cf1f-4029-b758-ab5755de9a8a

That means both conversations can now send requests with the fork's cache key, which defeats the intended one-conversation/one-cache-shard mapping.

Recommendation

I recommend making fork(agent=...) defensive against aliasing:

1. Preferred: clone the supplied agent before pinning if it shares identity with the source agent or shares the same llm object.
2. Acceptable alternative: reject aliased/shared agents with a clear ValueError.
3. Add regression tests for:
   • default fork() keeps distinct keys for source and fork
   • fork(agent=source.agent) cannot silently clobber the source key

The default no-arg fork() behavior already looks good; the risk is specifically the customizable agent= path.

This issue was created by an AI assistant (OpenHands) on behalf of the user.