Skip to content

feat: memory.ingestMedia() — multimodal ingestion (v0.5.0) - #15

Merged
rrader26 merged 1 commit into
mainfrom
feat/ingest-media
Jul 8, 2026
Merged

feat: memory.ingestMedia() — multimodal ingestion (v0.5.0)#15
rrader26 merged 1 commit into
mainfrom
feat/ingest-media

Conversation

@rrader26

@rrader26 rrader26 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Adds memory.ingestMedia() — the client half of multimodal ingestion (engine PR #58, shell PR #56).

const res = await tf.memory.ingestMedia({
  media: fs.readFileSync('receipt.png'),   // Uint8Array | Buffer | base64 string
  mimeType: 'image/png',
  source: 'receipt.png',
})
res.extractedText   // what the model read
res.saved           // MemoryItem[] created from it
res.modality        // 'image' | 'audio' | 'document'
res.blobUri         // stored-media location (if retained)
  • IngestMediaRequest/IngestMediaResult types.
  • MemoryResource.ingestMedia — reuses the cross-platform toBase64, POSTs to /memory/media.
  • Exported from the barrel; v0.4.1 → v0.5.0 (new feature).

Verified: tsc --noEmit clean, tsup build clean, check:proto passes (ingestMedia now covered, 42/47 RPCs). Republish to npm after merge.

🤖 Generated with Claude Code

Adds ingestMedia(media, mimeType, ...) — POSTs base64 media to
/memory/media, which the shell proxies to the engine's IngestMedia RPC
(LiteLLM extraction -> Observe pipeline). Returns the created memories +
extracted text + modality + blob uri.

- src/types/memory.ts: IngestMediaRequest/IngestMediaResult.
- src/resources/memory.ts: MemoryResource.ingestMedia (reuses toBase64).
- src/index.ts: export the new types.
- bump 0.4.1 -> 0.5.0.

typecheck + build clean; proto-drift check passes (ingestMedia covered).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@rrader26
rrader26 merged commit 3ddf197 into main Jul 8, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants