Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!-- SPECKIT START -->
For additional context about technologies to be used, project structure,
shell commands, and other important information, read the current plan:
[specs/004-eval-kit/plan.md](../specs/004-eval-kit/plan.md)
[specs/005-async-fastapi/plan.md](../specs/005-async-fastapi/plan.md)
<!-- SPECKIT END -->
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
- name: Install
run: |
python -m pip install --upgrade pip
pip install -e "./sdk-python[dev]"
pip install -e "./sdk-python[dev,async]"
- name: Validate OpenAPI spec (Principle I)
run: omp-validate-spec
- name: Run conformance suite (overall ≥85%)
Expand Down
2 changes: 1 addition & 1 deletion .specify/feature.json
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"feature_directory":"specs/004-eval-kit"}
{"feature_directory":"specs/005-async-fastapi"}
52 changes: 52 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,58 @@

All notable changes to this project will be documented in this file.

## [0.4.0] — Unreleased (M3.2 PR-A — `AsyncMemory`)

### Added

- **`openmem.AsyncMemory`** — async/await-native facade mirroring
`openmem.Memory`. Constructor performs zero blocking I/O; pools and
HTTP clients are built lazily on first verb call (data-model
AM-INV-3 / C-LIFE-1). Routes to native or threadwrap backends per
provider:
- `postgres` → `AsyncPostgresAdapter` on top of `asyncpg` + pgvector
(native cancel; SQL placeholders translated `%s` → `$1` from the
shared `_postgres_sql` module so sync/async share one source of
truth).
- `passthrough` → `AsyncPassthroughAdapter` on top of
`httpx.AsyncClient` (native cancel; identical timeout / retry
headers to sync).
- `mem0` / `supermemory` / `letta` → `AsyncThreadwrapAdapter` over a
per-instance `ThreadPoolExecutor` (best-effort cancel; orphan
completion logged at DEBUG).
- **Three-tier cancellation contract** (`contracts/async-memory.md` §3):
- C-CAN-1: native awaiter receives `CancelledError` ≤ 50 ms.
- C-CAN-2: native pool/socket released ≤ 500 ms post-cancel.
- C-CAN-3: Postgres server-side query aborted ≤ 1 s.
- C-CAN-4: threadwrap awaiter cancels immediately; worker thread
finishes in the background.
- C-CAN-5: subsequent verbs on the same `AsyncMemory` succeed after
cancellation (no pool corruption).
- **`[async]` install extra** (`asyncpg>=0.29`, `httpx>=0.27`).
`from openmem import AsyncMemory` raises a clear `ImportError`
containing `pip install 'openmem[async]'` when the extra is missing
(FR-026 / C-EXT-1..3). Importing bare `openmem` triggers no async
dependency resolution.
- **Cross-loop guard** — `AsyncMemory` captures the running loop id on
the first verb call and raises `RuntimeError` *before* any backend
call if a later verb is awaited on a different loop (C-LOOP-1).
- **Sync `Memory` signature regression test**
(`tests/async/test_memory_signatures.py`) commits a JSON snapshot of
every public verb's signature; any drift fails the gate (SC-008 /
FR-011).

### Changed

- `sdk-python/pyproject.toml`: bumped `version` to `0.4.0`; declared
the new `[async]` extra; `[server]` extra now depends on
`openmem[async]`.

### Preserved

- The sync `openmem.Memory` class is **byte-identical** — no public
signature, default value, or behavior changed. The full sync test
suite still passes (365 passed / 21 skipped at branch-off baseline).

## [0.2.1] — Unreleased (M2.1 — Live-API bridges)

### Added
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ print(ctx.text)
## Tooling

- `openmem-eval` — manual benchmark harness comparing recall, MRR, and latency across configured providers. **Never runs in CI.** Default invocation is a dry-run that makes zero network calls. See [specs/004-eval-kit/quickstart.md](specs/004-eval-kit/quickstart.md) for usage and [docs/eval/](docs/eval/README.md) for a sample report and trace from a real postgres run.
- `openmem.AsyncMemory` — async/await-native facade mirroring `openmem.Memory`. Postgres + passthrough adapters use native async clients (asyncpg, httpx); mem0/supermemory/letta are wrapped with a per-instance thread pool. Cancellation propagates within 50 ms on the native tier. Install with `pip install 'openmem[async]'` and see [specs/005-async-fastapi/quickstart.md](specs/005-async-fastapi/quickstart.md) §1–§4.

## License

Expand Down
30 changes: 30 additions & 0 deletions sdk-python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,38 @@ Protocol](../spec/OMP-0.1.md) v0.1.
pip install -e . # core
pip install -e ".[dev]" # core + tests
pip install -e ".[openai]" # core + OpenAIEmbedder
pip install -e ".[async]" # core + AsyncMemory (asyncpg)
```

## Async usage

`openmem.AsyncMemory` is the async/await-native mirror of
`openmem.Memory`. Method names, parameters, and error semantics match
the sync class — the only difference is that every verb is awaitable.
Postgres + passthrough run on native async clients (asyncpg / httpx);
mem0, supermemory, and letta are wrapped with a per-instance
`ThreadPoolExecutor`.

```python
import asyncio
from openmem import AsyncMemory # requires: pip install 'openmem[async]'

async def main():
async with AsyncMemory(provider="postgres",
url="postgresql://postgres:postgres@localhost:5432/postgres") as mem:
rec = await mem.add(content="user prefers dark mode", user_id="u1")
hits = await mem.search("dark mode", user_id="u1")
print(hits[0].memory.content)

asyncio.run(main())
```

Cancellation propagates within 50 ms on the native tier (postgres,
passthrough); the threadwrap tier returns immediately to the awaiter
while the worker thread completes in the background. See
[../specs/005-async-fastapi/quickstart.md](../specs/005-async-fastapi/quickstart.md)
for the full contract.

## Environment variables

| Var | Purpose |
Expand Down
38 changes: 37 additions & 1 deletion sdk-python/openmem/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,12 @@
SearchResult,
)

__version__ = "0.1.0"
__version__ = "0.4.0"

__all__ = [
"__version__",
"Memory",
"AsyncMemory",
"MemoryRecord",
"MemoryInput",
"MemoryUpdate",
Expand All @@ -66,3 +67,38 @@
"ProviderError",
"UnsupportedProviderError",
]


# ---------------------------------------------------------------------------
# Lazy `AsyncMemory` import (T019 / FR-026 / contracts §C-EXT-1..3).
#
# `from openmem import AsyncMemory` works iff the `[async]` extra is
# installed (asyncpg + httpx). Without it, the import raises a clear
# `ImportError` whose message contains the exact remediation string.
# Importing `openmem` itself MUST NOT trigger any async dependency
# resolution (C-EXT-3) — that is why this lives in `__getattr__`.
# ---------------------------------------------------------------------------


def __getattr__(name: str): # noqa: D401 - module-level hook
if name == "AsyncMemory":
# FR-026 / C-EXT-1..3: the user must learn *now* that the
# `[async]` extras are missing — not deep inside a backend
# call. Eagerly probe `asyncpg` (the only async-only runtime
# dep; httpx is a base requirement) before exposing the class.
try:
import asyncpg # noqa: F401
except ImportError as exc:
raise ImportError(
"openmem.AsyncMemory requires the async extras. "
"Install with: pip install 'openmem[async]'"
) from exc
try:
from .async_memory import AsyncMemory as _AsyncMemory
except ImportError as exc: # pragma: no cover - import-time path
raise ImportError(
"openmem.AsyncMemory requires the async extras. "
"Install with: pip install 'openmem[async]'"
) from exc
return _AsyncMemory
raise AttributeError(f"module 'openmem' has no attribute {name!r}")
153 changes: 153 additions & 0 deletions sdk-python/openmem/adapters/_postgres_sql.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
"""Shared postgres SQL fragments and helpers (T007 / M3.2).

Extracted from ``openmem.adapters.postgres`` so the upcoming
``AsyncPostgresAdapter`` (T015, asyncpg-based) can reuse the **exact same**
SQL strings as the sync adapter — eliminating any drift between the two
backends and keeping `Memory` byte-identical with `AsyncMemory` (SC-008).

Behavior **must not change** — every existing sync test must still pass.
"""

from __future__ import annotations

import base64
from datetime import datetime
from typing import Any

from ulid import ULID

# ---------------------------------------------------------------------------
# Constants
# ---------------------------------------------------------------------------

STD_FIELDS = {
"id",
"content",
"user_id",
"scope",
"tags",
"source",
"confidence",
"valid_from",
"valid_to",
"supersedes",
"embedding_model",
"created_at",
"updated_at",
}

# Static query templates (psycopg / %s placeholders).

GET_MEMORY_SQL = "SELECT * FROM memories WHERE id = %s;"

DELETE_MEMORY_SQL = "DELETE FROM memories WHERE id = %s;"

INSERT_MEMORY_SQL = """
INSERT INTO memories (
id, content, user_id, scope, tags, source, confidence,
valid_from, valid_to, supersedes, embedding_model,
embedding, extensions, created_at
) VALUES (
%s, %s, %s, %s, %s, %s, %s,
%s, %s, %s, %s,
%s::vector, %s, %s
)
RETURNING *;
"""

CREATE_EXTENSION_SQL = "CREATE EXTENSION IF NOT EXISTS vector;"

INDEX_USER_SCOPE_SQL = (
"CREATE INDEX IF NOT EXISTS idx_memories_user_scope ON memories(user_id, scope);"
)

INDEX_TAGS_SQL = (
"CREATE INDEX IF NOT EXISTS idx_memories_tags ON memories USING GIN(tags);"
)

INDEX_CREATED_AT_SQL = (
"CREATE INDEX IF NOT EXISTS idx_memories_created_at "
"ON memories(created_at DESC, id DESC);"
)


def make_create_table_sql(dim: int) -> str:
"""Return the CREATE TABLE statement parameterized by embedding dim."""
return f"""
CREATE TABLE IF NOT EXISTS memories (
id TEXT PRIMARY KEY,
content TEXT NOT NULL,
user_id TEXT NOT NULL,
scope TEXT,
tags TEXT[],
source JSONB,
confidence REAL,
valid_from TIMESTAMPTZ,
valid_to TIMESTAMPTZ,
supersedes TEXT[],
embedding_model TEXT,
embedding VECTOR({dim}),
extensions JSONB,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ
);
"""


# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------


def new_id() -> str:
"""Generate a new ULID-suffixed memory id."""
return f"mem_{ULID()}"


def vector_literal(vec: list[float]) -> str:
"""Render a Python list as a pgvector literal."""
return "[" + ",".join(repr(float(x)) for x in vec) + "]"


def scope_glob_to_sql_like(scope: str | None) -> str | None:
"""Translate an OMP scope glob (``*``) to SQL ``LIKE`` (``%``)."""
if scope is None:
return None
return scope.replace("*", "%")


def encode_cursor(created_at: datetime, id_: str) -> str:
"""Encode a ``(created_at, id)`` pair as a stable opaque cursor."""
raw = f"{created_at.isoformat()}|{id_}".encode()
return base64.urlsafe_b64encode(raw).decode("ascii")


def decode_cursor(cursor: str) -> tuple[datetime, str]:
"""Inverse of :func:`encode_cursor`."""
raw = base64.urlsafe_b64decode(cursor.encode("ascii")).decode()
ts, id_ = raw.split("|", 1)
return datetime.fromisoformat(ts), id_


def split_extensions(extra: dict[str, Any]) -> dict[str, Any]:
"""Extract ``x-<provider>`` keys from a free-form dict (Principle V)."""
return {k: v for k, v in extra.items() if k.startswith("x-")}


__all__ = [
"STD_FIELDS",
"GET_MEMORY_SQL",
"DELETE_MEMORY_SQL",
"INSERT_MEMORY_SQL",
"CREATE_EXTENSION_SQL",
"INDEX_USER_SCOPE_SQL",
"INDEX_TAGS_SQL",
"INDEX_CREATED_AT_SQL",
"make_create_table_sql",
"new_id",
"vector_literal",
"scope_glob_to_sql_like",
"encode_cursor",
"decode_cursor",
"split_extensions",
]
26 changes: 26 additions & 0 deletions sdk-python/openmem/adapters/_validation.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
"""Cross-adapter validation helpers (T008 / M3.2).

Centralizes input checks that previously lived inline in each sync
adapter so both sync and the upcoming async adapters share one
implementation.
"""

from __future__ import annotations

from ..errors import InvalidRequestError


def require_user_id(user_id: str | None, *, provider: str) -> str:
"""Raise ``InvalidRequestError`` if ``user_id`` is missing or whitespace.

Returns the (unchanged) ``user_id`` for ergonomic chaining.

Cross-user broadening defence: every adapter MUST refuse a search/list
that omits ``user_id`` BEFORE issuing any upstream call.
"""
if user_id is None or not str(user_id).strip():
raise InvalidRequestError("user_id is required", provider=provider)
return user_id


__all__ = ["require_user_id"]
Loading
Loading