Python: [Bug]: InMemoryCheckpointStorage returns its stored checkpoint objects, unlike the other backends

[dev404ai]

Description

InMemoryCheckpointStorage returns its stored checkpoint objects directly, so a caller that mutates a checkpoint it loaded also mutates what the storage holds. The other two in-tree implementations of the same CheckpointStorage protocol do not behave that way: FileCheckpointStorage and the Azure Cosmos backend rebuild every returned checkpoint from its serialized form, so reads are isolated by construction.

All three read paths are affected, because the load, list_checkpoints and get_latest methods hand out the instances kept in the internal dictionary, while save already stores copy.deepcopy(checkpoint). Isolation on write is therefore deliberate, and isolation on read is missing.

This matters mostly because InMemoryCheckpointStorage is the backend tests and samples reach for first. A workflow that mutates restored state passes against the in-memory backend and behaves differently against a persistent one, or the reverse, and the two cannot both be right for the same protocol.

I could not find an existing issue or pull request covering this. It is unrelated to the per-step checkpoint save serialization in #7647 and #7676, which changes when saves happen rather than what a read returns.

I have a fix ready with tests and will open a pull request alongside this issue.

Code Sample

import asyncio
import tempfile

from agent_framework._workflows._checkpoint import (
    FileCheckpointStorage,
    InMemoryCheckpointStorage,
    WorkflowCheckpoint,
)


async def probe(storage, label: str) -> None:
    checkpoint = WorkflowCheckpoint(
        workflow_name="probe-workflow",
        graph_signature_hash="probe-signature",
        state={"answer": "original"},
    )
    checkpoint_id = await storage.save(checkpoint)

    first = await storage.load(checkpoint_id)
    first.state["answer"] = "MUTATED"

    second = await storage.load(checkpoint_id)
    print(f"{label}: {second.state['answer']!r}")


async def main() -> None:
    await probe(InMemoryCheckpointStorage(), "InMemoryCheckpointStorage")
    with tempfile.TemporaryDirectory() as tmp:
        await probe(FileCheckpointStorage(storage_path=tmp), "FileCheckpointStorage")


asyncio.run(main())


Output:


InMemoryCheckpointStorage: 'MUTATED'
FileCheckpointStorage: 'original'


Expected: both print `'original'`.

Error Messages / Stack Traces

Package Versions

agent-framework-core: 1.13.0

Python Version

Python 3.13.7

Additional Context

The protocol docstrings do not currently state whether a returned checkpoint belongs to the caller, which is why the two behaviors were able to drift apart. My pull request adds that sentence to the CheckpointStorage protocol and makes the in-memory backend copy on read, so all three implementations agree.

[github-actions]

🤖 Automated triage reproduction notes (agent-authored — trust but verify)

Agent analysis

Bug is in python/packages/core/agent_framework/_workflows/_checkpoint.py::InMemoryCheckpointStorage. The load() (line 217), list_checkpoints() (line 225), and get_latest() (line 240) methods return stored objects directly without copying, while save() already does copy.deepcopy. Minimal repro: save a checkpoint, load it, mutate state, load again — the mutation is visible in the second load. Fix: wrap each return with copy.deepcopy().

• Failing test: /tmp/test_checkpoint_isolation.py
• Files examined: python/packages/core/agent_framework/_workflows/_checkpoint.py, python/packages/core/pyproject.toml
• Tests run: /tmp/test_checkpoint_isolation.py (standalone async script testing load, list_checkpoints, get_latest isolation)
• Reported version: 1.13.0
• Current version: 1.14.0