Self-hosted file uploads fail extraction: /files prefix is passed as local storage key

[Konan69]

Summary

Self-hosted supermemory-server v0.0.3 accepts POST /v3/documents/file, stores the uploaded file, and serves it at /files/:key, but the async extraction workflow fails immediately with:

[Extraction] Preprocessor supermemory-bucket failed: Failed to fetch from bucket: invalid local file storage key
[Workflow] Extract-content step failed: Failed to fetch from bucket: invalid local file storage key

This means file uploads can remain failed even though the raw file exists and is retrievable from /files/:key.

Environment

• Release: server-v0.0.3
• Binary: supermemory-server-linux-x64
• Original binary SHA256: 4036486514bd3511099e8f8642e1c56ecc58550c5e194442e88a5438992c0957
• OS: Linux/WSL
• Local file storage, no S3/custom storage configured
• SUPERMEMORY_DATA_DIR set
• PORT / SUPERMEMORY_PORT set to 6767
• At least one provider key configured

Reproduction

Start the self-hosted binary with default file storage config, then upload a plain text file:

printf 'hello from self-hosted file upload\n' >/tmp/sm-test.txt

curl -sS -X POST http://127.0.0.1:6767/v3/documents/file \
  -H 'Authorization: Bearer <api-key>' \
  -F 'file=@/tmp/sm-test.txt;type=text/plain' \
  -F 'containerTag=debug:file-upload' \
  -F 'customId=debug-file-upload-1'

The API returns a queued document, for example:

{"id":"nssp1UF5AC2jiPMHXborYV","status":"queued"}

The server logs show the raw file was stored:

File uploaded to bucket {"url":"http://localhost:6767/files/v6QR4xqXX3UJpHyLgEFQgT.txt","storageKey":"v6QR4xqXX3UJpHyLgEFQgT.txt"}
mimeType text/plain

The file route works:

curl http://127.0.0.1:6767/files/v6QR4xqXX3UJpHyLgEFQgT.txt
# returns the uploaded text

But extraction fails:

[Extraction] Preprocessor supermemory-bucket failed: Failed to fetch from bucket: invalid local file storage key
[Workflow] Extract-content step failed: Failed to fetch from bucket: invalid local file storage key

Suspected cause

The self-hosted default storagePublicUrl appears to include /files:

http://localhost:${PORT}/files

So uploaded documents store content URLs like:

http://localhost:6767/files/<storageKey>

In the bundled supermemory-bucket extractor in server-v0.0.3, the storage key appears to be derived from the URL with:

e.pathname.slice(1)

For a URL path like:

/files/v6QR4xqXX3UJpHyLgEFQgT.txt

that produces:

files/v6QR4xqXX3UJpHyLgEFQgT.txt

The local encrypted file store rejects keys containing /, which causes:

invalid local file storage key

The extractor should fetch the local store with only the actual storage key:

v6QR4xqXX3UJpHyLgEFQgT.txt

Verification / workaround tested

If I run the original unpatched binary with:

STORAGE_PUBLIC_URL=http://localhost:6767

then the document URL becomes:

http://localhost:6767/<storageKey>

and pathname.slice(1) returns only <storageKey>, so extraction proceeds:

[Text Extractor] Decoded ArrayBuffer to text
[Extraction] Extracted with text

However, that workaround makes the stored document URL point to /<storageKey>, while the server only serves files at /files/:key, so it fixes ingestion but produces a non-serving file URL.

Expected behavior

With default self-hosted config, file uploads served at /files/:key should be extracted successfully.

Suggested fix

When running self-hosted/local file storage, the supermemory-bucket extractor should strip the configured public URL path prefix before calling getSelfHostedFile, e.g. strip /files/ from /files/:key and pass only :key to local storage.

A regression test could upload a text file through /v3/documents/file using default self-hosted config and assert the document reaches done instead of failed.

[linear-code]

ENG-887