bug: concurrent Memory append silently drops valid FTS search hits

[thunguo]

Describe the bug

A Memory FTS search can silently return a false negative when another request appends a memory entry concurrently.
The search first reads and validates head revision N. If an append then commits revision N+1 before the FTS query executes, the mutable head index projection is replaced with rows belonging to revision N+1. The pending search queries those new rows but still filters them using revision N, causing otherwise valid hits to be discarded.
The response therefore reports revision N with an empty hit list, even though the searched fact exists in both revisions N and N+1. No exception, retry signal, or stale-result indication is returned.
This is a deterministic TOCTOU race between head validation and index lookup, and can cause intermittent memory recall failures in applications that search while appending memories.

Steps to reproduce

1. Use a PowerContext checkout with its dependencies installed.
2. Save the following as repro_memory_search_race.py:

#!/usr/bin/env python3

import asyncio
import json
from tempfile import TemporaryDirectory
from types import MethodType
from typing import Any

from powercontext.builtin.artifacts.memory import MemoryEntryInput
from powercontext.builtin.persistence.sqlite import SQLiteConfig
from powercontext.builtin.runtime import (
    BuiltinConfig,
    RememberMemoryRequest,
    SearchMemoryRequest,
    open_builtin_runtime,
)


async def main() -> None:
    with TemporaryDirectory() as directory:
        database = SQLiteConfig(url=f"sqlite+aiosqlite:///{directory}/repro.db")
        async with open_builtin_runtime(BuiltinConfig(database=database)) as runtime:
            memory = runtime.memory.for_scope("race-repro")
            await memory.remember(
                RememberMemoryRequest(entries=(MemoryEntryInput(kind="fact", text="Stable searchable fact."),))
            )

            provider: Any = runtime._provider
            fts = provider.index.indexes[0]
            original_search = fts.search
            paused, resume = asyncio.Event(), asyncio.Event()

            async def pause_search(self: Any, connection: Any, scope: str, request: Any) -> Any:
                paused.set()
                await resume.wait()
                return await original_search(connection, scope, request)

            fts.search = MethodType(pause_search, fts)
            query = SearchMemoryRequest(query="stable searchable", mode="fts")
            try:
                pending_search = asyncio.create_task(memory.search(query))
                await asyncio.wait_for(paused.wait(), 2)
                new_head = await memory.remember(
                    RememberMemoryRequest(entries=(MemoryEntryInput(kind="fact", text="Unrelated appended fact."),))
                )
                resume.set()
                raced = await asyncio.wait_for(pending_search, 2)
            finally:
                resume.set()
                fts.search = original_search

            normal = await memory.search(query)
            observed = {
                "search_revision": raced.memory_ref.revision if raced.memory_ref else None,
                "new_head_revision": new_head.memory_ref.revision,
                "raced_hits": [hit.text for hit in raced.hits],
                "post_race_hits": [hit.text for hit in normal.hits],
            }

    print(json.dumps(observed, sort_keys=True))
    expected = {
        "search_revision": 1,
        "new_head_revision": 2,
        "raced_hits": [],
        "post_race_hits": ["Stable searchable fact."],
    }
    if observed != expected:
        raise SystemExit(1)
    print("0")


if __name__ == "__main__":
    asyncio.run(main())

3. Run the reproducer:

python repro_memory_search_race.py

4. Observe that the search associated with revision 1 returns no hits, while the same query immediately succeeds after the concurrent append.

The method wrapper only introduces a deterministic scheduling barrier. It delegates to the original FTS implementation without modifying the request, indexed data, or returned results.

Expected behavior

The search should operate against one consistent revision.
Either:

• it completes against revision 1 and returns Stable searchable fact., or
• it detects that the head advanced, retries against revision 2, and returns the same fact.

Because the concurrent append adds an unrelated fact and does not remove the existing one, the searched fact is present in both revisions. The search must not silently return a successful empty result.

Actual behavior

The reproducer consistently prints:

{"new_head_revision": 2, "post_race_hits": ["Stable searchable fact."], "raced_hits": [], "search_revision": 1}
0

The raced search reports revision 1 but returns:

"raced_hits": []

The identical query immediately afterward returns:

"post_race_hits": ["Stable searchable fact."]

This demonstrates that the empty result is not caused by missing data, query mismatch, or FTS tokenization. It is caused by the search combining the old revision with the new head projection.
The final 0 and process exit code 0 mean that the reproducer observed the known buggy state.

Environment

• PowerContext version: 0.0.1
• Commit: 72b9417
• Python version: 3.14.3
• OS: macOS 26.2, arm64
• aiosqlite: 0.22.1
• SQLAlchemy: 2.0.51
• Storage backend: local SQLite
• External services: none

Are you willing to submit a PR to fix this bug?

• Yes, I would like to submit a PR.

[Teingi]

Thanks for the detailed report and reproducer. This is a valid TOCTOU bug: a search can validate revision N but query the updated N+1 projection, resulting in false empty hits. The fix should ensure a consistent revision or detect and retry the head change. Please also check the vector/hybrid paths and OceanBase. Feel free to submit a PR—thanks!

[thunguo]

Thanks for the detailed report and reproducer. This is a valid TOCTOU bug: a search can validate revision N but query the updated N+1 projection, resulting in false empty hits. The fix should ensure a consistent revision or detect and retry the head change. Please also check the vector/hybrid paths and OceanBase. Feel free to submit a PR—thanks!

Okay, I've fixed it locally. After verifying the fix through testing, I'll submit a PR to resolve the issue.