Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,24 @@ All four expose the _same_ host functions and types. Only the data behind
`HostEnv` differs. That's what lets a plugin built once run identically in tests,
the dev server, and production.

`storage.sql` is the one place where the host, not just the data behind it,
differs. The three non-production hosts share `host-runtime/sqlstore`, which
gives each plugin a private in-memory SQLite database and runs every request
through the same `plugins.SQLRunner` Owncast uses, so request validation,
parameter typing, the call timeout, atomic `exec`, and the row, value, result,
and database-size limits all match. It uses `modernc.org/sqlite` rather than the
cgo `mattn/go-sqlite3` driver Owncast uses, because these binaries are
cross-compiled for every release target with `CGO_ENABLED=0`.

Owncast additionally installs a SQLite authorizer to deny `ATTACH`, `DETACH`,
every `PRAGMA`, and temp-schema DDL, and the pure-Go driver has no equivalent.
That difference is not left visible to plugins: those statements are refused
above the driver by `plugins.DeniedSQLReason`, which every host applies at the
host-function boundary, so a plugin gets the same refusal locally that it gets
on a real server. `plugins.DeniedSQLStatementExamples` is the fixture both
repositories test against, this one through the Go check and Owncast through
the authorizer, which is what keeps the two in step.

## The plugin API contract

The plugin-facing API exists in three representations that must agree:
Expand Down
37 changes: 34 additions & 3 deletions docs/PLUGIN_AUTHOR_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -359,7 +359,8 @@ Each method requires the matching permission in your manifest:
| `owncast.auth.grantSession({userId, ttl?})` / `owncast.auth.endSession()` | `auth.gate` |
| `owncast.kv.get(key)` / `.set(key, value)` (+ `.getJSON` / `.setJSON`) | `storage.kv` |
| `owncast.storage.upload(name, bytes)`, returns `{url}` | `storage.upload` |
| `owncast.fs.read/readText/write/list/delete/exists(...)`, sandboxed disk | `storage.fs` |
| `owncast.fs.read/readText/write/list/delete/exists(...)`, sandboxed disk. `write` and `delete` return `{}` on success or `{error}` on failure. | `storage.fs` |
| `owncast.sql.exec/query/queryRow(sql, params?)`, private SQLite database | `storage.sql` |
| `owncast.http.fetch(url, opts?)` | `network.fetch` |
| `owncast.events.emit(eventType, payload)` | `events.emit` |
| `owncast.stream.current()`, live stream state | `server.read` |
Expand All @@ -370,7 +371,7 @@ Each method requires the matching permission in your manifest:
| `owncast.server.federation()`, `{enabled, username, isPrivate}` | `server.read` |
| `owncast.server.tags()`, `[string]` | `server.read` |
| `owncast.videoConfig.read()`, `{latencyLevel, codec, variants}` | `videoconfig.read` |
| `owncast.videoConfig.write({latencyLevel?, codec?, variants?})`, partial update | `videoconfig.write` |
| `owncast.videoConfig.write({latencyLevel?, codec?, variants?})`, partial update, throws on failure | `videoconfig.write` |
| `owncast.notifications.discord(text)` | `notifications.send` |
| `owncast.notifications.browserPush({title, body, url?})` | `notifications.send` |
| `owncast.notifications.fediverse({type, body, image?, link?})` | `notifications.send` |
Expand All @@ -382,6 +383,27 @@ Each method requires the matching permission in your manifest:

Calling an API without its permission throws a clear error.

### SQL database

`storage.sql` gives your plugin one SQLite database of its own, private to your plugin and separate from Owncast's database. `owncast.sql.exec(sql, params?)` runs statements and reports `rowsAffected` and `lastInsertId`, `owncast.sql.query(sql, params?)` returns rows as objects keyed by column name, and `owncast.sql.queryRow(sql, params?)` returns the first row or `null`. Python is the same API with `query_row` and dicts. A failed statement throws (Python raises). The SDK treats a result object with no `error` field as success and rejects a missing or non-object host response.

Each `exec` call is atomic: a multi-statement batch either commits whole or leaves the database untouched, so a schema migration can't half-apply. You can't hold a transaction open across calls, which also means you never have to clean one up.

`query` never hands you a silently short answer. A query returning more than the row cap, or a result larger than the result budget (see [Limits](#limits)), fails with an error asking for a `LIMIT`. Write the `LIMIT` yourself when a table can grow without bound, and use `queryRow` when you only need one row out of a big table.

Your database has its own size cap, independent of the `storage.fs` quota, so writing files never shrinks the room your tables have, and the other way around. Two things to design around: plugin databases are **not** part of Owncast's database backups, so treat the contents as rebuildable or export what matters yourself, and the data is kept when an admin uninstalls your plugin, exactly like your config and your `storage.fs` files, so a reinstall finds its tables where it left them. An admin who wants the space back deletes one directory, `data/plugin-storage/<slug>/`, which holds both your database and your `storage.fs` files.

Ordinary SQL is all available: DDL, DML, indexes, views, triggers, `ORDER BY`, recursive CTEs, subqueries, `UNION`, and the json1 functions. Refused in every host: `ATTACH`, `DETACH`, every `PRAGMA` (reads included), temporary-schema DDL both as keywords (`CREATE TEMP TABLE` / `INDEX` / `TRIGGER` / `VIEW`) and schema-qualified (`CREATE TABLE temp.x`), `load_extension()`, `VACUUM` and `VACUUM INTO`, and transaction controls (`BEGIN`, `COMMIT`, `END`, `ROLLBACK`, `SAVEPOINT`, and `RELEASE`). Each `exec` call already owns the transaction around the whole batch.

`owncast-plugin-test` and `owncast-plugin-serve` give your plugin a real SQLite database, so you can develop and test against `owncast.sql` without a running Owncast. It is in-memory, so every scenario and every restart of the dev server starts clean. Every limit and every refusal above applies there too, including the statements listed in the previous paragraph, so a passing scenario test means the same SQL is accepted on a real server.

A worked example ships with the SDK: `examples/js/chat-leaderboard` and
`examples/python/chat-leaderboard` rank chatters by message count, covering
schema creation in one atomic `exec`, an `ON CONFLICT` upsert, a bounded ranked
`query`, and a single-row read.

> **JavaScript integers.** Parameters and results cross the host boundary as JSON. Python can bind and read exact 64-bit SQLite integers. JavaScript loses unsafe integers before `JSON.stringify` on writes and during `JSON.parse` on reads. Store values above `Number.MAX_SAFE_INTEGER` (2^53 - 1) as TEXT when a JavaScript plugin needs them to remain exact.

### Chat identity

Every plugin has **exactly one chat identity**, the auto-bot Owncast provisions when your plugin is installed. The display name is your plugin's `name` (e.g. `echo-bot`), with `IsBot: true`. `owncast.chat.send(text)` and `owncast.chat.sendAction(text)` both post as this identity, through Owncast's normal chat pipeline (filters, rate limits, persistence, moderation, same as any user).
Expand Down Expand Up @@ -511,7 +533,8 @@ in the built-in help listing.
| `auth.gate` | Be the site's viewer-authentication gate: `owncast.auth.grantSession` / `.endSession` plus the `onAuthCheck` hook. Only one gate plugin can be enabled at a time. See [Viewer authentication gates](#viewer-authentication-gates). |
| `storage.kv` | Per-plugin namespaced key/value store |
| `storage.upload` | `owncast.storage.upload`, upload files, get a public URL |
| `storage.fs` | `owncast.fs.*`, private sandboxed disk under `data/plugin-data/<slug>/` (server-side only, never served over HTTP) |
| `storage.fs` | `owncast.fs.*`, private sandboxed disk under `data/plugin-storage/<slug>/files/` (server-side only, never served over HTTP). `write` and `delete` return `{}` on success or `{error}` on failure. |
| `storage.sql` | `owncast.sql.*`, private per-plugin SQLite database under `data/plugin-storage/<slug>/db/`, separate from the `storage.fs` sandbox and not included in Owncast backups |
| `network.fetch` | Outbound HTTP, also requires `network.allowedHosts` (see below) |
| `events.emit` | Emit custom events for other plugins to subscribe to |
| `http.serve` | Serve HTTP at `/plugins/<your-name>/*` |
Expand Down Expand Up @@ -560,6 +583,14 @@ The host enforces these caps per plugin. They're generous for normal use. Size p
| Pending timers | 64 | `owncast.timer.setTimeout`/`setInterval` outstanding at once |
| Timer delay | 100 ms to 24 h | clamped into this range |
| SSE connections | 64 | concurrent browser clients on your event stream |
| `storage.fs` footprint | 256 MiB | every file your plugin writes through `owncast.fs.*` |
| `storage.sql` database | 128 MiB | your whole SQLite database, counted separately from `storage.fs` |
| SQL request | 64 KiB | one `owncast.sql` call, statement text plus parameters |
| SQL bound parameters | 64 | `params` in one `owncast.sql` call |
| SQL column value | 1 MiB | a single value in a returned row |
| SQL query result | 1 MiB | the whole encoded result of one `query` |
| SQL rows returned | 10000 | one `query`, and passing it is an error rather than a truncated result |
| SQL call runtime | 2 s | each `exec` or `query` |

`owncast.kv` values have no hard size cap, but it's a config/state store, not a blob store, so keep values small (use `storage.upload` or `storage.fs` for large data). Timeouts mean a handler that blocks (a slow `owncast.http.fetch`, a tight loop) is cancelled, so keep event/filter work quick and push slow work elsewhere.

Expand Down
98 changes: 93 additions & 5 deletions docs/WIRE_PROTOCOL.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,14 +87,102 @@ Because all plugins of a language share one engine, the engine imports the **ful

### `storage.fs`

Sandboxed per-plugin filesystem under `data/plugin-data/<slug>/`. The host confines every path to the plugin's own directory.
Sandboxed per-plugin filesystem under `data/plugin-storage/<slug>/files/`. The host confines every path to the plugin's own directory.

- `owncast_fs_read(pathPtr: PTR): PTR`, returns the file's raw bytes, or 0-offset when missing/unreadable
- `owncast_fs_write(pathPtr: PTR, dataPtr: PTR): PTR`, returns JSON `{ok, error?}`
- `owncast_fs_write(pathPtr: PTR, dataPtr: PTR): PTR`, returns JSON `FSResult` (`{error?}`). An empty object means success.
- `owncast_fs_list(dirPtr: PTR): PTR`, returns JSON `string[]` of entry names (missing dir → empty)
- `owncast_fs_delete(pathPtr: PTR): PTR`, returns JSON `{ok, error?}`
- `owncast_fs_delete(pathPtr: PTR): PTR`, returns JSON `FSResult` (`{error?}`). An empty object means success.
- `owncast_fs_exists(pathPtr: PTR): I32`, returns 1 if the path exists, 0 otherwise

### `storage.sql`

The host opens one private SQLite database per plugin at
`data/plugin-storage/<slug>/db/plugin.db`. The `storage.fs` sandbox is rooted at
`data/plugin-storage/<slug>/files/`, so `db/` is not a path the `storage.fs` API
refuses, it is one that API cannot express, and the two quotas stay independent
because the filesystem quota walk covers `files/` only. A plugin's database is
capped at 128 MiB, separate from the 256 MiB `storage.fs` quota. Plugin
databases are not included in Owncast's database backups. The host keeps a
plugin's SQL data when the plugin is uninstalled, the same way it keeps its
config and its `storage.fs` files, and an operator reclaims all of a plugin's
space by deleting the one directory `data/plugin-storage/<slug>/`.

- `owncast_sql_exec(requestPtr: PTR): PTR`, returns JSON `SQLExecResult`
- `owncast_sql_query(requestPtr: PTR): PTR`, returns JSON `SQLQueryResult`

Both take the same request JSON. `params` is an optional array of scalar values,
and `maxRows` is optional:

```json
{ "sql": "SELECT name FROM viewers WHERE seen > ?", "params": [1730000000], "maxRows": 100 }
```

A parameter is `null`, a boolean, a number, or a string. Any other JSON type
fails the call.

`SQLExecResult` is `{error?, rowsAffected, lastInsertId}`, both counters 64-bit.
`SQLQueryResult` is
`{error?, columns: string[], rows: any[][], truncated?}`, one `rows` entry per
row with values in `columns` order. Use SQL column aliases when selecting
duplicate column names. Absence of `error` means success. A failed operation
sets `error`.

The SDKs reject a missing or non-object host response instead of treating it as
success.

`maxRows` omitted or 0 means no caller limit, and the host never silently
returns a short result for an unbounded query: once the result passes the row cap
or the result-size budget, the call fails with an error telling the author to add
a `LIMIT`. A value from 1 through 10000 is caller intent, so the host returns at
most that many rows and sets `truncated` true when more rows matched. Values
below 0 or above 10000 are invalid. Reading one row out of a large table is the
bounded case: the SDK's `queryRow` sends `maxRows: 1`.

Limits the host applies to every `exec` and `query`:

- request JSON: 64 KiB, which bounds the statement text along with it
- bound parameters: 64
- one returned column value: 1 MiB
- whole encoded query result: 1 MiB
- rows returned: 10000
- one call: 2 seconds, including time spent waiting for the plugin's serialized connection

Each `exec` call runs as one host-owned transaction: a multi-statement batch
either commits whole or leaves the database untouched, and a plugin cannot leave
a transaction open across calls.

These operations are refused, in every host:

- `ATTACH` and `DETACH`
- every `PRAGMA`, reads included
- temporary-schema DDL, both the keyword forms (`CREATE TEMP TABLE` / `INDEX` /
`TRIGGER` / `VIEW`) and the schema-qualified ones (`CREATE TABLE temp.x`),
which SQLite reports as ordinary DDL against the `temp` schema
- `load_extension()`
- `VACUUM` and `VACUUM INTO`
- transaction controls: `BEGIN`, `COMMIT`, `END`, `ROLLBACK`, `SAVEPOINT`, and
`RELEASE`

Owncast refuses them twice. A SQLite authorizer on each connection is the gate,
because it sees the compiled statement: every statement in a multi-statement
string runs, so an operation smuggled in behind a permitted one would defeat a
check on the text alone. In front of that, the host runtime refuses the same
list in Go before the statement reaches a driver. That second check is what lets
a host without an authorizer (the SDK's test runner and dev server, which use a
pure-Go SQLite driver so they can cross-compile) reach the same verdict, so a
plugin that passes its scenario tests is not about to fail on a real server.

Those refusals cost ordinary SQL nothing: DDL, DML, indexes, views, triggers,
`ORDER BY` sorts, recursive CTEs, subqueries, `UNION`, and the json1 functions
all work, as do identifiers that merely begin with `temp`.

An integral JSON parameter binds as a SQLite INTEGER exactly, including values
beyond 2^53 when the guest language can represent them. Python can bind and
read exact 64-bit integers. JavaScript loses unsafe integers before
`JSON.stringify` on writes and during `JSON.parse` on reads, so a JavaScript
plugin should store values above `Number.MAX_SAFE_INTEGER` (2^53 - 1) as TEXT.

### `events.emit`

- `owncast_emit_event(eventTypePtr: PTR, payloadPtr: PTR): void`, payload is a JSON-encoded value
Expand All @@ -115,7 +203,7 @@ Sandboxed per-plugin filesystem under `data/plugin-data/<slug>/`. The host confi

### `videoconfig.write`

- `owncast_video_config_write(configPtr: PTR): PTR`, applies a partial `VideoConfigUpdate`. Returns JSON `{ok, error?}`
- `owncast_video_config_write(configPtr: PTR): PTR`, applies a partial `VideoConfigUpdate`. Returns JSON `VideoConfigWriteResult` (`{error?}`). An empty object means success.

### `notifications.send`

Expand Down Expand Up @@ -154,7 +242,7 @@ The access boundary is not part of the plugin wire protocol. The operator
selects one cumulative, host-owned mode: website only, website and stream, or
website, stream, and status. A plugin cannot read or change that selection.

- `owncast_auth_grant_session(reqPtr: PTR): PTR`, JSON `GrantSessionRequest` in, JSON `{error?}` out. Mints a session for the named `userId` (which the same plugin must have registered via `users.register`) and attaches the signed cookie to the in-flight response.
- `owncast_auth_grant_session(reqPtr: PTR): PTR`, JSON `GrantSessionRequest` in, JSON `GrantSessionResult` (`{error?}`) out. An empty object means success. Mints a session for the named `userId` (which the same plugin must have registered via `users.register`) and attaches the signed cookie to the in-flight response.
- `owncast_auth_end_session(): void`, clears the session cookie on the in-flight response (logout).

The optional `on_auth_check` export (see [Exports](#exports-plugin--host)) lets
Expand Down
4 changes: 4 additions & 0 deletions engines/build_py.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,10 @@
("owncast_fs_delete", "path: str", "str"),
("owncast_fs_exists", "path: str", "int"),
],
"storage.sql": [
("owncast_sql_exec", "request: str", "str"),
("owncast_sql_query", "request: str", "str"),
],
"events.emit": [
("owncast_emit_event", "event_type: str, payload: str"),
],
Expand Down
2 changes: 2 additions & 0 deletions engines/javascript/engine.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ declare module 'extism:host' {
owncast_fs_list(dirPtr: PTR): PTR;
owncast_fs_delete(pathPtr: PTR): PTR;
owncast_fs_exists(pathPtr: PTR): I64;
owncast_sql_exec(requestPtr: PTR): PTR;
owncast_sql_query(requestPtr: PTR): PTR;
owncast_fediverse_post(textPtr: PTR): PTR;
owncast_kv_get(keyPtr: PTR): PTR;
owncast_kv_set(keyPtr: PTR, valPtr: PTR): void;
Expand Down
1 change: 1 addition & 0 deletions examples/js/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ One self-contained npm project per directory. Each has its own `README.md` with
| [echo-bot](./echo-bot/) | Posts a reply to every chat message via `owncast.chat.send`. |
| [mod-commands](./mod-commands/) | Declarative `commands` table: custom prefix, aliases, moderator gating, cooldowns, ordinary chat-handler composition, and `!help` metadata. |
| [message-counter](./message-counter/) | Per-user message counter persisted in the plugin's namespaced config. |
| [chat-leaderboard](./chat-leaderboard/) | `storage.sql`, a private SQLite database: schema in one atomic `exec`, an `ON CONFLICT` upsert, a bounded ranked `query`, and `queryRow` for a single row. |
| [profanity-filter](./profanity-filter/) | `filter.modify(payload)`, rewrites flagged words to asterisks. |
| [slow-mode](./slow-mode/) | `filter.drop(reason)`, rate-limits per user, with plugin-config-backed state. |
| [buggy-filter](./buggy-filter/) | Always throws, exercises the host's fail-open + strike system. |
Expand Down
1 change: 1 addition & 0 deletions examples/js/all-permissions-test/plugin.manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
"storage.kv",
"storage.upload",
"storage.fs",
"storage.sql",
"chat.send",
"chat.history",
"chat.moderate",
Expand Down
Loading
Loading