Skip to content

Unify file creation and signed upload sessions #6264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
TheodoreSpeaks merged 4 commits into from
Aug 5, 2026
Merged

Unify file creation and signed upload sessions #6264

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6fb1776
feat(uploads): unify signed upload sessions
TheodoreSpeaks Aug 4, 2026
b1d4d63
fix(uploads): preserve attachment storage semantics
TheodoreSpeaks Aug 4, 2026
c03fc3d
feat(files): add authored file creation
TheodoreSpeaks Aug 4, 2026
db668c6
fix(uploads): omit hoisted S3 metadata headers
TheodoreSpeaks Aug 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions apps/docs/content/docs/en/platform/self-hosting/object-storage.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,26 @@ AZURE_STORAGE_OG_IMAGES_CONTAINER_NAME=og-images
AZURE_STORAGE_WORKSPACE_LOGOS_CONTAINER_NAME=workspace-logos
```

Direct browser uploads require a Blob service CORS rule on the storage account. Allow your exact
Sim origin, `GET` and `PUT`, the `Content-Type` header, and the `x-ms-*` prefix used by signed blob
and metadata headers:

```bash
az storage cors add \
--services b \
--methods GET PUT \
--origins https://sim.yourdomain.com \
--allowed-headers content-type 'x-ms-*' \
--exposed-headers ETag \
--max-age 3600 \
--account-name mystorageaccount \
--account-key '<account-key>'
```

If you authenticate with a connection string, replace the last two options with
`--connection-string "$AZURE_CONNECTION_STRING"`. CORS is configured once for the account's Blob
service and applies to all of its containers.

A full Helm example lives at `helm/sim/examples/values-azure.yaml`.

## Set up Google Cloud Storage
Expand Down Expand Up @@ -276,11 +296,13 @@ cat > /tmp/cors.json <<'EOF'
"responseHeader": [
"Content-Type",
"ETag",
"x-goog-meta-uploadid",
"x-goog-meta-originalname",
"x-goog-meta-uploadedat",
"x-goog-meta-purpose",
"x-goog-meta-userid",
"x-goog-meta-workspaceid",
"x-goog-meta-knowledgebaseid",
"x-goog-meta-folderid",
"x-goog-meta-workflowid",
"x-goog-meta-executionid"
Expand Down Expand Up @@ -445,6 +467,27 @@ The same browser-reachability and CORS requirements apply.
</Tab>
</Tabs>

## Configure temporary upload cleanup

Sim stages every direct upload under the `upload-sessions/` prefix before promoting it to its final,
immutable object key. Apply the cleanup policy to **every** purpose-specific bucket or container
configured above:

- On AWS S3 and Google Cloud Storage, expire objects under `upload-sessions/` after two days and
abort incomplete multipart uploads after two days.
- On Azure Blob, expire committed blobs under `upload-sessions/` after two days. Azure automatically
removes uncommitted blocks after seven days.
- For an S3-compatible provider, configure both rules when its lifecycle implementation supports
them. Check the provider's documentation because lifecycle feature support varies.

The two-day window exceeds the 24-hour upload-token lifetime and leaves time to retry completion.
Do not apply this prefix rule to final objects outside `upload-sessions/`.

<Callout type="warning">
Configure both expiration and incomplete-multipart cleanup where available. Expiring staged
objects alone does not necessarily remove abandoned multipart parts.
</Callout>

## Verify it works

After restarting with the new configuration:
Expand Down
159 changes: 63 additions & 96 deletions apps/docs/openapi-v2-files-audit.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,56 +169,75 @@
}
}
},
"x-removed-buffered-post": {
"operationId": "uploadFile",
"summary": "Upload File",
"description": "Upload a file to a workspace as `multipart/form-data` with a single `file` field. The workspace — and the optional target `folderId` — are supplied as query parameters (not form fields) so authorization runs before the request body is buffered. Maximum file size is 100MB. A name already taken in the destination folder is **not** an error: the name is auto-suffixed (`data.csv` -> `data (1).csv`), matching the in-app uploader, so a `201` can come back with a `name` different from the one you sent — always read `name` from the response rather than assuming it. `409` is returned only if a unique name cannot be allocated after several attempts. Use `PATCH /api/v2/files/{fileId}` if you need a specific name to be exact-or-fail. Returns `201 Created`.\n\nPresigned upload is not part of the public API: it debits the storage quota only in a separate register step, so a caller that never registers would leave unaccounted bytes in storage. This buffered path debits inside the upload transaction.",
"post": {
"operationId": "createFile",
"summary": "Create File",
"description": "Create an authored workspace file, either empty or with initial inline content. Use this endpoint for files whose bytes are already available as UTF-8 text or base64 and are at most 50 MiB after decoding. Use the upload-session endpoints for streamed or larger files. A live file with the same name in the same folder is rejected with `409`.",
"tags": ["Files"],
"x-codeSamples": [
{
"id": "curl",
"label": "cURL",
"lang": "bash",
"source": "curl -X POST \\\n \"https://www.sim.ai/api/v2/files?workspaceId=YOUR_WORKSPACE_ID\" \\\n -H \"X-API-Key: YOUR_API_KEY\" \\\n -F \"file=@/path/to/file.csv\""
}
],
"parameters": [
{
"$ref": "#/components/parameters/WorkspaceIdQuery"
},
{
"name": "folderId",
"in": "query",
"required": false,
"description": "Target file folder. Omit to upload to the workspace root. Supplied as a query parameter, like `workspaceId`, so authorization runs before the multipart body is buffered.",
"schema": {
"type": "string",
"example": "fold_9Kq2mZ7pR4tLxWc0Ye3Nu"
}
"source": "curl -X POST \\\n \"https://www.sim.ai/api/v2/files\" \\\n -H \"X-API-Key: YOUR_API_KEY\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"workspaceId\": \"YOUR_WORKSPACE_ID\", \"name\": \"notes.md\"}'"
}
],
"requestBody": {
"required": true,
"description": "The file to upload, sent as multipart/form-data.",
"content": {
"multipart/form-data": {
"application/json": {
"schema": {
"type": "object",
"required": ["file"],
"additionalProperties": false,
"required": ["workspaceId", "name"],
"properties": {
"file": {
"workspaceId": {
"type": "string",
"minLength": 1,
"description": "Workspace in which to create the file."
},
"name": {
"type": "string",
"minLength": 1,
"maxLength": 255,
"description": "File name, including its extension. Path separators and dot segments are rejected."
},
"contentType": {
"type": "string",
"minLength": 1,
"maxLength": 255,
"description": "MIME type. When omitted, it is inferred from the file extension."
},
"folderId": {
"type": "string",
"minLength": 1,
"maxLength": 128,
"description": "Destination folder. Omit to create the file at the workspace root."
},
"content": {
"type": "string",
"maxLength": 70000000,
"default": "",
"description": "Initial file content. Omit or send an empty string to create a zero-byte file."
},
"encoding": {
"type": "string",
"format": "binary",
"description": "The file to upload. Maximum size is 100MB."
"enum": ["utf-8", "base64"],
"default": "utf-8",
"description": "Encoding of `content`."
}
}
},
"example": {
"workspaceId": "a91c4b2e-6d3f-4e8a-b5c7-0d9e2f1a8c64",
"name": "notes.md"
}
}
}
},
"responses": {
"201": {
"description": "The file was uploaded successfully.",
"description": "The created file.",
"headers": {
"X-RateLimit-Limit": {
"$ref": "#/components/headers/X-RateLimit-Limit"
Expand All @@ -238,12 +257,12 @@
"example": {
"data": {
"id": "wf_V1StGXR8z5jdHi6BmyT91",
"name": "data.csv",
"size": 1024,
"type": "text/csv",
"key": "workspace/a91c4b2e-6d3f-4e8a-b5c7-0d9e2f1a8c64/1709571234-xyz-data.csv",
"folderId": "fold_9Kq2mZ7pR4tLxWc0Ye3Nu",
"folderPath": "Reports/Q1",
"name": "notes.md",
"size": 0,
"type": "text/markdown",
"key": "workspace/a91c4b2e-6d3f-4e8a-b5c7-0d9e2f1a8c64/1709571234-xyz-notes.md",
"folderId": null,
"folderPath": null,
"uploadedBy": "user_abc123",
"uploadedAt": "2026-01-15T10:30:00Z",
"updatedAt": "2026-01-15T10:30:00Z"
Expand All @@ -252,74 +271,22 @@
}
}
},
"400": {
"description": "The request was malformed: an invalid `workspaceId` query parameter, a body that is not valid multipart form data, or a missing `file` form field.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
},
"example": {
"error": {
"code": "BAD_REQUEST",
"message": "file form field is required"
}
}
}
}
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"403": {
"$ref": "#/components/responses/Forbidden"
},
"409": {
"description": "A unique filename could not be allocated in the destination folder after several attempts. An ordinary name collision is auto-suffixed instead, not rejected.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
},
"example": {
"error": {
"code": "CONFLICT",
"message": "A file named \"data.csv\" already exists in this workspace"
}
}
}
}
},
"413": {
"description": "The upload exceeds the 100MB file size limit, or the workspace storage limit would be exceeded.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/V2Error"
},
"example": {
"error": {
"code": "PAYLOAD_TOO_LARGE",
"message": "File size exceeds 100MB limit (142.30MB)"
}
}
}
}
},
"429": {
"$ref": "#/components/responses/RateLimited"
},
"500": {
"$ref": "#/components/responses/InternalError"
}
"400": { "$ref": "#/components/responses/BadRequest" },
"401": { "$ref": "#/components/responses/Unauthorized" },
"403": { "$ref": "#/components/responses/Forbidden" },
"404": { "$ref": "#/components/responses/NotFound" },
"409": { "$ref": "#/components/responses/Conflict" },
"413": { "$ref": "#/components/responses/PayloadTooLarge" },
"429": { "$ref": "#/components/responses/RateLimited" },
"500": { "$ref": "#/components/responses/InternalError" }
}
}
},
"/api/v2/files/uploads": {
"post": {
"operationId": "createFileUpload",
"summary": "Create File Upload",
"description": "Create a stateless multipart upload session and signed upload token. Every file uses this flow; a small file is a single part. The maximum file size is 5 GB.",
"description": "Create an upload session and signed control token. Empty files and files up to and including 50 MiB receive a single signed PUT URL; larger files receive multipart transfer instructions. The maximum file size is 5 GB.",
"tags": ["Files"],
"requestBody": {
"required": true,
Expand Down Expand Up @@ -405,7 +372,7 @@
"post": {
"operationId": "completeFileUpload",
"summary": "Complete File Upload",
"description": "Verify every part, assemble the object, and atomically register the workspace file.",
"description": "Verify the single PUT or assemble every multipart part, then atomically register the workspace file.",
"tags": ["Files"],
"parameters": [
{
Expand Down Expand Up @@ -1681,7 +1648,7 @@
"name": "upload-token",
"in": "header",
"required": true,
"description": "The signed token returned when the multipart upload was created.",
"description": "The signed control token returned when the upload session was created.",
"schema": { "type": "string", "minLength": 1 }
},
"FileIdPath": {
Expand Down
Loading
Loading