diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 0d9e75c..536da49 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -130,6 +130,24 @@ All four expose the _same_ host functions and types. Only the data behind `HostEnv` differs. That's what lets a plugin built once run identically in tests, the dev server, and production. +`storage.sql` is the one place where the host, not just the data behind it, +differs. The three non-production hosts share `host-runtime/sqlstore`, which +gives each plugin a private in-memory SQLite database and runs every request +through the same `plugins.SQLRunner` Owncast uses, so request validation, +parameter typing, the call timeout, atomic `exec`, and the row, value, result, +and database-size limits all match. It uses `modernc.org/sqlite` rather than the +cgo `mattn/go-sqlite3` driver Owncast uses, because these binaries are +cross-compiled for every release target with `CGO_ENABLED=0`. + +Owncast additionally installs a SQLite authorizer to deny `ATTACH`, `DETACH`, +every `PRAGMA`, and temp-schema DDL, and the pure-Go driver has no equivalent. +That difference is not left visible to plugins: those statements are refused +above the driver by `plugins.DeniedSQLReason`, which every host applies at the +host-function boundary, so a plugin gets the same refusal locally that it gets +on a real server. `plugins.DeniedSQLStatementExamples` is the fixture both +repositories test against, this one through the Go check and Owncast through +the authorizer, which is what keeps the two in step. + ## The plugin API contract The plugin-facing API exists in three representations that must agree: diff --git a/docs/PLUGIN_AUTHOR_GUIDE.md b/docs/PLUGIN_AUTHOR_GUIDE.md index 1d38d47..a29166f 100644 --- a/docs/PLUGIN_AUTHOR_GUIDE.md +++ b/docs/PLUGIN_AUTHOR_GUIDE.md @@ -359,7 +359,8 @@ Each method requires the matching permission in your manifest: | `owncast.auth.grantSession({userId, ttl?})` / `owncast.auth.endSession()` | `auth.gate` | | `owncast.kv.get(key)` / `.set(key, value)` (+ `.getJSON` / `.setJSON`) | `storage.kv` | | `owncast.storage.upload(name, bytes)`, returns `{url}` | `storage.upload` | -| `owncast.fs.read/readText/write/list/delete/exists(...)`, sandboxed disk | `storage.fs` | +| `owncast.fs.read/readText/write/list/delete/exists(...)`, sandboxed disk. `write` and `delete` return `{}` on success or `{error}` on failure. | `storage.fs` | +| `owncast.sql.exec/query/queryRow(sql, params?)`, private SQLite database | `storage.sql` | | `owncast.http.fetch(url, opts?)` | `network.fetch` | | `owncast.events.emit(eventType, payload)` | `events.emit` | | `owncast.stream.current()`, live stream state | `server.read` | @@ -370,7 +371,7 @@ Each method requires the matching permission in your manifest: | `owncast.server.federation()`, `{enabled, username, isPrivate}` | `server.read` | | `owncast.server.tags()`, `[string]` | `server.read` | | `owncast.videoConfig.read()`, `{latencyLevel, codec, variants}` | `videoconfig.read` | -| `owncast.videoConfig.write({latencyLevel?, codec?, variants?})`, partial update | `videoconfig.write` | +| `owncast.videoConfig.write({latencyLevel?, codec?, variants?})`, partial update, throws on failure | `videoconfig.write` | | `owncast.notifications.discord(text)` | `notifications.send` | | `owncast.notifications.browserPush({title, body, url?})` | `notifications.send` | | `owncast.notifications.fediverse({type, body, image?, link?})` | `notifications.send` | @@ -382,6 +383,27 @@ Each method requires the matching permission in your manifest: Calling an API without its permission throws a clear error. +### SQL database + +`storage.sql` gives your plugin one SQLite database of its own, private to your plugin and separate from Owncast's database. `owncast.sql.exec(sql, params?)` runs statements and reports `rowsAffected` and `lastInsertId`, `owncast.sql.query(sql, params?)` returns rows as objects keyed by column name, and `owncast.sql.queryRow(sql, params?)` returns the first row or `null`. Python is the same API with `query_row` and dicts. A failed statement throws (Python raises). The SDK treats a result object with no `error` field as success and rejects a missing or non-object host response. + +Each `exec` call is atomic: a multi-statement batch either commits whole or leaves the database untouched, so a schema migration can't half-apply. You can't hold a transaction open across calls, which also means you never have to clean one up. + +`query` never hands you a silently short answer. A query returning more than the row cap, or a result larger than the result budget (see [Limits](#limits)), fails with an error asking for a `LIMIT`. Write the `LIMIT` yourself when a table can grow without bound, and use `queryRow` when you only need one row out of a big table. + +Your database has its own size cap, independent of the `storage.fs` quota, so writing files never shrinks the room your tables have, and the other way around. Two things to design around: plugin databases are **not** part of Owncast's database backups, so treat the contents as rebuildable or export what matters yourself, and the data is kept when an admin uninstalls your plugin, exactly like your config and your `storage.fs` files, so a reinstall finds its tables where it left them. An admin who wants the space back deletes one directory, `data/plugin-storage//`, which holds both your database and your `storage.fs` files. + +Ordinary SQL is all available: DDL, DML, indexes, views, triggers, `ORDER BY`, recursive CTEs, subqueries, `UNION`, and the json1 functions. Refused in every host: `ATTACH`, `DETACH`, every `PRAGMA` (reads included), temporary-schema DDL both as keywords (`CREATE TEMP TABLE` / `INDEX` / `TRIGGER` / `VIEW`) and schema-qualified (`CREATE TABLE temp.x`), `load_extension()`, `VACUUM` and `VACUUM INTO`, and transaction controls (`BEGIN`, `COMMIT`, `END`, `ROLLBACK`, `SAVEPOINT`, and `RELEASE`). Each `exec` call already owns the transaction around the whole batch. + +`owncast-plugin-test` and `owncast-plugin-serve` give your plugin a real SQLite database, so you can develop and test against `owncast.sql` without a running Owncast. It is in-memory, so every scenario and every restart of the dev server starts clean. Every limit and every refusal above applies there too, including the statements listed in the previous paragraph, so a passing scenario test means the same SQL is accepted on a real server. + +A worked example ships with the SDK: `examples/js/chat-leaderboard` and +`examples/python/chat-leaderboard` rank chatters by message count, covering +schema creation in one atomic `exec`, an `ON CONFLICT` upsert, a bounded ranked +`query`, and a single-row read. + +> **JavaScript integers.** Parameters and results cross the host boundary as JSON. Python can bind and read exact 64-bit SQLite integers. JavaScript loses unsafe integers before `JSON.stringify` on writes and during `JSON.parse` on reads. Store values above `Number.MAX_SAFE_INTEGER` (2^53 - 1) as TEXT when a JavaScript plugin needs them to remain exact. + ### Chat identity Every plugin has **exactly one chat identity**, the auto-bot Owncast provisions when your plugin is installed. The display name is your plugin's `name` (e.g. `echo-bot`), with `IsBot: true`. `owncast.chat.send(text)` and `owncast.chat.sendAction(text)` both post as this identity, through Owncast's normal chat pipeline (filters, rate limits, persistence, moderation, same as any user). @@ -511,7 +533,8 @@ in the built-in help listing. | `auth.gate` | Be the site's viewer-authentication gate: `owncast.auth.grantSession` / `.endSession` plus the `onAuthCheck` hook. Only one gate plugin can be enabled at a time. See [Viewer authentication gates](#viewer-authentication-gates). | | `storage.kv` | Per-plugin namespaced key/value store | | `storage.upload` | `owncast.storage.upload`, upload files, get a public URL | -| `storage.fs` | `owncast.fs.*`, private sandboxed disk under `data/plugin-data//` (server-side only, never served over HTTP) | +| `storage.fs` | `owncast.fs.*`, private sandboxed disk under `data/plugin-storage//files/` (server-side only, never served over HTTP). `write` and `delete` return `{}` on success or `{error}` on failure. | +| `storage.sql` | `owncast.sql.*`, private per-plugin SQLite database under `data/plugin-storage//db/`, separate from the `storage.fs` sandbox and not included in Owncast backups | | `network.fetch` | Outbound HTTP, also requires `network.allowedHosts` (see below) | | `events.emit` | Emit custom events for other plugins to subscribe to | | `http.serve` | Serve HTTP at `/plugins//*` | @@ -560,6 +583,14 @@ The host enforces these caps per plugin. They're generous for normal use. Size p | Pending timers | 64 | `owncast.timer.setTimeout`/`setInterval` outstanding at once | | Timer delay | 100 ms to 24 h | clamped into this range | | SSE connections | 64 | concurrent browser clients on your event stream | +| `storage.fs` footprint | 256 MiB | every file your plugin writes through `owncast.fs.*` | +| `storage.sql` database | 128 MiB | your whole SQLite database, counted separately from `storage.fs` | +| SQL request | 64 KiB | one `owncast.sql` call, statement text plus parameters | +| SQL bound parameters | 64 | `params` in one `owncast.sql` call | +| SQL column value | 1 MiB | a single value in a returned row | +| SQL query result | 1 MiB | the whole encoded result of one `query` | +| SQL rows returned | 10000 | one `query`, and passing it is an error rather than a truncated result | +| SQL call runtime | 2 s | each `exec` or `query` | `owncast.kv` values have no hard size cap, but it's a config/state store, not a blob store, so keep values small (use `storage.upload` or `storage.fs` for large data). Timeouts mean a handler that blocks (a slow `owncast.http.fetch`, a tight loop) is cancelled, so keep event/filter work quick and push slow work elsewhere. diff --git a/docs/WIRE_PROTOCOL.md b/docs/WIRE_PROTOCOL.md index 2f4b866..fe28e8c 100644 --- a/docs/WIRE_PROTOCOL.md +++ b/docs/WIRE_PROTOCOL.md @@ -87,14 +87,102 @@ Because all plugins of a language share one engine, the engine imports the **ful ### `storage.fs` -Sandboxed per-plugin filesystem under `data/plugin-data//`. The host confines every path to the plugin's own directory. +Sandboxed per-plugin filesystem under `data/plugin-storage//files/`. The host confines every path to the plugin's own directory. - `owncast_fs_read(pathPtr: PTR): PTR`, returns the file's raw bytes, or 0-offset when missing/unreadable -- `owncast_fs_write(pathPtr: PTR, dataPtr: PTR): PTR`, returns JSON `{ok, error?}` +- `owncast_fs_write(pathPtr: PTR, dataPtr: PTR): PTR`, returns JSON `FSResult` (`{error?}`). An empty object means success. - `owncast_fs_list(dirPtr: PTR): PTR`, returns JSON `string[]` of entry names (missing dir → empty) -- `owncast_fs_delete(pathPtr: PTR): PTR`, returns JSON `{ok, error?}` +- `owncast_fs_delete(pathPtr: PTR): PTR`, returns JSON `FSResult` (`{error?}`). An empty object means success. - `owncast_fs_exists(pathPtr: PTR): I32`, returns 1 if the path exists, 0 otherwise +### `storage.sql` + +The host opens one private SQLite database per plugin at +`data/plugin-storage//db/plugin.db`. The `storage.fs` sandbox is rooted at +`data/plugin-storage//files/`, so `db/` is not a path the `storage.fs` API +refuses, it is one that API cannot express, and the two quotas stay independent +because the filesystem quota walk covers `files/` only. A plugin's database is +capped at 128 MiB, separate from the 256 MiB `storage.fs` quota. Plugin +databases are not included in Owncast's database backups. The host keeps a +plugin's SQL data when the plugin is uninstalled, the same way it keeps its +config and its `storage.fs` files, and an operator reclaims all of a plugin's +space by deleting the one directory `data/plugin-storage//`. + +- `owncast_sql_exec(requestPtr: PTR): PTR`, returns JSON `SQLExecResult` +- `owncast_sql_query(requestPtr: PTR): PTR`, returns JSON `SQLQueryResult` + +Both take the same request JSON. `params` is an optional array of scalar values, +and `maxRows` is optional: + +```json +{ "sql": "SELECT name FROM viewers WHERE seen > ?", "params": [1730000000], "maxRows": 100 } +``` + +A parameter is `null`, a boolean, a number, or a string. Any other JSON type +fails the call. + +`SQLExecResult` is `{error?, rowsAffected, lastInsertId}`, both counters 64-bit. +`SQLQueryResult` is +`{error?, columns: string[], rows: any[][], truncated?}`, one `rows` entry per +row with values in `columns` order. Use SQL column aliases when selecting +duplicate column names. Absence of `error` means success. A failed operation +sets `error`. + +The SDKs reject a missing or non-object host response instead of treating it as +success. + +`maxRows` omitted or 0 means no caller limit, and the host never silently +returns a short result for an unbounded query: once the result passes the row cap +or the result-size budget, the call fails with an error telling the author to add +a `LIMIT`. A value from 1 through 10000 is caller intent, so the host returns at +most that many rows and sets `truncated` true when more rows matched. Values +below 0 or above 10000 are invalid. Reading one row out of a large table is the +bounded case: the SDK's `queryRow` sends `maxRows: 1`. + +Limits the host applies to every `exec` and `query`: + +- request JSON: 64 KiB, which bounds the statement text along with it +- bound parameters: 64 +- one returned column value: 1 MiB +- whole encoded query result: 1 MiB +- rows returned: 10000 +- one call: 2 seconds, including time spent waiting for the plugin's serialized connection + +Each `exec` call runs as one host-owned transaction: a multi-statement batch +either commits whole or leaves the database untouched, and a plugin cannot leave +a transaction open across calls. + +These operations are refused, in every host: + +- `ATTACH` and `DETACH` +- every `PRAGMA`, reads included +- temporary-schema DDL, both the keyword forms (`CREATE TEMP TABLE` / `INDEX` / + `TRIGGER` / `VIEW`) and the schema-qualified ones (`CREATE TABLE temp.x`), + which SQLite reports as ordinary DDL against the `temp` schema +- `load_extension()` +- `VACUUM` and `VACUUM INTO` +- transaction controls: `BEGIN`, `COMMIT`, `END`, `ROLLBACK`, `SAVEPOINT`, and + `RELEASE` + +Owncast refuses them twice. A SQLite authorizer on each connection is the gate, +because it sees the compiled statement: every statement in a multi-statement +string runs, so an operation smuggled in behind a permitted one would defeat a +check on the text alone. In front of that, the host runtime refuses the same +list in Go before the statement reaches a driver. That second check is what lets +a host without an authorizer (the SDK's test runner and dev server, which use a +pure-Go SQLite driver so they can cross-compile) reach the same verdict, so a +plugin that passes its scenario tests is not about to fail on a real server. + +Those refusals cost ordinary SQL nothing: DDL, DML, indexes, views, triggers, +`ORDER BY` sorts, recursive CTEs, subqueries, `UNION`, and the json1 functions +all work, as do identifiers that merely begin with `temp`. + +An integral JSON parameter binds as a SQLite INTEGER exactly, including values +beyond 2^53 when the guest language can represent them. Python can bind and +read exact 64-bit integers. JavaScript loses unsafe integers before +`JSON.stringify` on writes and during `JSON.parse` on reads, so a JavaScript +plugin should store values above `Number.MAX_SAFE_INTEGER` (2^53 - 1) as TEXT. + ### `events.emit` - `owncast_emit_event(eventTypePtr: PTR, payloadPtr: PTR): void`, payload is a JSON-encoded value @@ -115,7 +203,7 @@ Sandboxed per-plugin filesystem under `data/plugin-data//`. The host confi ### `videoconfig.write` -- `owncast_video_config_write(configPtr: PTR): PTR`, applies a partial `VideoConfigUpdate`. Returns JSON `{ok, error?}` +- `owncast_video_config_write(configPtr: PTR): PTR`, applies a partial `VideoConfigUpdate`. Returns JSON `VideoConfigWriteResult` (`{error?}`). An empty object means success. ### `notifications.send` @@ -154,7 +242,7 @@ The access boundary is not part of the plugin wire protocol. The operator selects one cumulative, host-owned mode: website only, website and stream, or website, stream, and status. A plugin cannot read or change that selection. -- `owncast_auth_grant_session(reqPtr: PTR): PTR`, JSON `GrantSessionRequest` in, JSON `{error?}` out. Mints a session for the named `userId` (which the same plugin must have registered via `users.register`) and attaches the signed cookie to the in-flight response. +- `owncast_auth_grant_session(reqPtr: PTR): PTR`, JSON `GrantSessionRequest` in, JSON `GrantSessionResult` (`{error?}`) out. An empty object means success. Mints a session for the named `userId` (which the same plugin must have registered via `users.register`) and attaches the signed cookie to the in-flight response. - `owncast_auth_end_session(): void`, clears the session cookie on the in-flight response (logout). The optional `on_auth_check` export (see [Exports](#exports-plugin--host)) lets diff --git a/engines/build_py.py b/engines/build_py.py index 1372a5a..7dffa12 100644 --- a/engines/build_py.py +++ b/engines/build_py.py @@ -58,6 +58,10 @@ ("owncast_fs_delete", "path: str", "str"), ("owncast_fs_exists", "path: str", "int"), ], + "storage.sql": [ + ("owncast_sql_exec", "request: str", "str"), + ("owncast_sql_query", "request: str", "str"), + ], "events.emit": [ ("owncast_emit_event", "event_type: str, payload: str"), ], diff --git a/engines/javascript/engine.d.ts b/engines/javascript/engine.d.ts index 42afd4b..0a374f2 100644 --- a/engines/javascript/engine.d.ts +++ b/engines/javascript/engine.d.ts @@ -45,6 +45,8 @@ declare module 'extism:host' { owncast_fs_list(dirPtr: PTR): PTR; owncast_fs_delete(pathPtr: PTR): PTR; owncast_fs_exists(pathPtr: PTR): I64; + owncast_sql_exec(requestPtr: PTR): PTR; + owncast_sql_query(requestPtr: PTR): PTR; owncast_fediverse_post(textPtr: PTR): PTR; owncast_kv_get(keyPtr: PTR): PTR; owncast_kv_set(keyPtr: PTR, valPtr: PTR): void; diff --git a/examples/js/README.md b/examples/js/README.md index 7f5c650..5a5585b 100644 --- a/examples/js/README.md +++ b/examples/js/README.md @@ -9,6 +9,7 @@ One self-contained npm project per directory. Each has its own `README.md` with | [echo-bot](./echo-bot/) | Posts a reply to every chat message via `owncast.chat.send`. | | [mod-commands](./mod-commands/) | Declarative `commands` table: custom prefix, aliases, moderator gating, cooldowns, ordinary chat-handler composition, and `!help` metadata. | | [message-counter](./message-counter/) | Per-user message counter persisted in the plugin's namespaced config. | +| [chat-leaderboard](./chat-leaderboard/) | `storage.sql`, a private SQLite database: schema in one atomic `exec`, an `ON CONFLICT` upsert, a bounded ranked `query`, and `queryRow` for a single row. | | [profanity-filter](./profanity-filter/) | `filter.modify(payload)`, rewrites flagged words to asterisks. | | [slow-mode](./slow-mode/) | `filter.drop(reason)`, rate-limits per user, with plugin-config-backed state. | | [buggy-filter](./buggy-filter/) | Always throws, exercises the host's fail-open + strike system. | diff --git a/examples/js/all-permissions-test/plugin.manifest.json b/examples/js/all-permissions-test/plugin.manifest.json index 95fa176..390a9b9 100644 --- a/examples/js/all-permissions-test/plugin.manifest.json +++ b/examples/js/all-permissions-test/plugin.manifest.json @@ -9,6 +9,7 @@ "storage.kv", "storage.upload", "storage.fs", + "storage.sql", "chat.send", "chat.history", "chat.moderate", diff --git a/examples/js/chat-leaderboard/INSTRUCTIONS.md b/examples/js/chat-leaderboard/INSTRUCTIONS.md new file mode 100644 index 0000000..3b11b02 --- /dev/null +++ b/examples/js/chat-leaderboard/INSTRUCTIONS.md @@ -0,0 +1,38 @@ +# Example Chat Leaderboard + +Keeps a running count of how many messages each chatter has sent and posts the +standings in chat on request. + +Counting starts as soon as you enable the plugin. Messages that begin with `!` +are not counted, so asking for the leaderboard does not raise your own score. +Chatters are tracked by their account, not their name, so someone who renames +keeps their history and shows up under their new name. + +## Commands + +Enable the plugin in **Admin → Plugins**, then type these in chat. + +| Command | Who can use it | What it does | +| --- | --- | --- | +| `!top` | anyone | Posts the five most active chatters with their message counts. Says `No messages counted yet.` on an empty board. | +| `!rank` | anyone | Posts the sender's own position and message count, or tells them they have not been counted yet. | +| `!resetleaderboard` | moderators only | Clears the standings and records that a reset happened. Non-moderator invocations are silent. | + +## Where the data lives + +The counts are kept in a small database that belongs to this plugin alone, on +your server under `data/plugin-storage/chat-leaderboard/db/`. Nothing else can +read it, and it is not served over the web. + +Two things worth knowing before you rely on it: + +- It is **not** included in Owncast's database backups. If the standings matter + to you, copy that directory yourself. +- Uninstalling the plugin leaves the data in place, so reinstalling picks the + standings back up. To start clean, delete that directory while the plugin is + disabled. + +## Permissions + +- **storage.sql** gives the plugin its own private database for the counts. +- **chat.send** lets the bot post the standings. diff --git a/examples/js/chat-leaderboard/README.md b/examples/js/chat-leaderboard/README.md new file mode 100644 index 0000000..398a06d --- /dev/null +++ b/examples/js/chat-leaderboard/README.md @@ -0,0 +1,63 @@ +# chat-leaderboard + +Counts how many messages each chatter has sent and ranks them, using the +plugin's own private SQLite database at +`data/plugin-storage/chat-leaderboard/db/`. `!top` shows the standings, `!rank` +shows the sender's own position, and a moderator can clear the board with +`!resetleaderboard`. + +**Demonstrates:** the `storage.sql` permission end to end. `owncast.sql.exec` +for schema creation, an `ON CONFLICT` upsert, and a two-statement atomic batch; +`owncast.sql.query` for a bounded, database-sorted result set; and +`owncast.sql.queryRow` for a single row (returning `null` when nothing matches). + +## Why SQL and not `storage.kv` + +[message-counter](../message-counter/) keeps the same per-user counts in the +key-value store, and that is the right choice when you only ever read a value +back by its key. It cannot answer "who are the top five", because ranking means +sorting across every key, and the plugin would have to pull all of them into +memory to do it. + +Here the database does the sorting and only the rows that will be shown cross +the host boundary. The trade is that you own a schema. + +## What the host enforces + +- **`exec` is one transaction.** A multi-statement batch commits whole or leaves + the database untouched. The schema, and the reset that clears the standings + while writing its audit row, both rely on this. +- **`query` never truncates silently.** A query that returns more than 10000 + rows, or more than 1 MiB of encoded results, fails and asks for a `LIMIT`. + `!top` passes its bound as a parameter. Use `queryRow` when one row will do. +- **The database is private and capped.** 128 MiB, separate from the + `storage.fs` quota. Plugins cannot reach each other's databases, and + `ATTACH`, `PRAGMA`, temporary tables, and `load_extension` are all refused. +- **Not in Owncast's backups.** Treat the contents as rebuildable, or export + what matters yourself. + +## Run it + +```sh +npm install +npm test # build + run the scenarios in __tests__/ +``` + +The test runner gives the plugin a real in-memory SQLite database, so the +scenarios exercise the actual SQL without a running Owncast. Each run starts +from an empty schema. + +`npm run serve` drives chat too, but the dev server does not dispatch chat +commands yet, so `!top`, `!rank`, and `!resetleaderboard` only answer under +`npm test` or on a real Owncast instance. Plain messages still reach +`onChatMessage` and are counted: + +```sh +curl -XPOST localhost:8080/_dev/chat -d '{"user":"alice","body":"hello"}' +``` + +## Permissions + +- **storage.sql** for the private database. +- **chat.send** posts the bot's replies. The moderator gating on + `!resetleaderboard` needs no extra permission. diff --git a/examples/js/chat-leaderboard/__tests__/chat-leaderboard.test.json b/examples/js/chat-leaderboard/__tests__/chat-leaderboard.test.json new file mode 100644 index 0000000..b5ee8c3 --- /dev/null +++ b/examples/js/chat-leaderboard/__tests__/chat-leaderboard.test.json @@ -0,0 +1,126 @@ +[ + { + "name": "!top reports an empty leaderboard before anyone has chatted", + "events": [ + { + "event": "chat.message.received", + "payload": { "id": "m1", "user": { "id": "u-alice", "displayName": "alice" }, "body": "!top", "timestamp": "2026-01-01T00:00:00Z" } + } + ], + "expect": { "chatSends": ["No messages counted yet."] } + }, + { + "name": "counting is silent, and !top ranks chatters by message count", + "events": [ + { + "event": "chat.message.received", + "payload": { "id": "m1", "user": { "id": "u-alice", "displayName": "alice" }, "body": "hello", "timestamp": "2026-01-01T00:00:00Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m2", "user": { "id": "u-bob", "displayName": "bob" }, "body": "hi", "timestamp": "2026-01-01T00:00:01Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m3", "user": { "id": "u-alice", "displayName": "alice" }, "body": "again", "timestamp": "2026-01-01T00:00:02Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m4", "user": { "id": "u-bob", "displayName": "bob" }, "body": "!top", "timestamp": "2026-01-01T00:00:03Z" } + } + ], + "expect": { "chatSends": ["Top chatters: 1. alice (2), 2. bob (1)"] } + }, + { + "name": "a rename follows the same row instead of splitting the history", + "events": [ + { + "event": "chat.message.received", + "payload": { "id": "m1", "user": { "id": "u-alice", "displayName": "alice" }, "body": "first", "timestamp": "2026-01-01T00:00:00Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m2", "user": { "id": "u-alice", "displayName": "alice-renamed" }, "body": "second", "timestamp": "2026-01-01T00:00:01Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m3", "user": { "id": "u-alice", "displayName": "alice-renamed" }, "body": "!top", "timestamp": "2026-01-01T00:00:02Z" } + } + ], + "expect": { "chatSends": ["Top chatters: 1. alice-renamed (2)"] } + }, + { + "name": "!rank uses the same tie ordering as !top", + "events": [ + { + "event": "chat.message.received", + "payload": { "id": "m1", "user": { "id": "u-alice", "displayName": "alice" }, "body": "one", "timestamp": "2026-01-01T00:00:00Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m2", "user": { "id": "u-bob", "displayName": "bob" }, "body": "one", "timestamp": "2026-01-01T00:00:01Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m3", "user": { "id": "u-bob", "displayName": "bob" }, "body": "!rank", "timestamp": "2026-01-01T00:00:02Z" } + } + ], + "expect": { "chatSends": ["bob is #2 with 1 message(s)."] } + }, + { + "name": "!rank says so when the sender has not been counted yet", + "events": [ + { + "event": "chat.message.received", + "payload": { "id": "m1", "user": { "id": "u-newcomer", "displayName": "newcomer" }, "body": "!rank", "timestamp": "2026-01-01T00:00:00Z" } + } + ], + "expect": { "chatSends": ["You have not sent any messages yet."] } + }, + { + "name": "a moderator can clear the leaderboard, and the audit row commits with it", + "events": [ + { + "event": "chat.message.received", + "payload": { "id": "m1", "user": { "id": "u-alice", "displayName": "alice" }, "body": "hello", "timestamp": "2026-01-01T00:00:00Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m2", "user": { "id": "u-mod", "displayName": "mod", "scopes": ["MODERATOR"] }, "body": "!resetleaderboard", "timestamp": "2026-01-01T00:00:01Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m3", "user": { "id": "u-mod", "displayName": "mod", "scopes": ["MODERATOR"] }, "body": "!top", "timestamp": "2026-01-01T00:00:02Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m4", "user": { "id": "u-mod", "displayName": "mod", "scopes": ["MODERATOR"] }, "body": "!resetleaderboard", "timestamp": "2026-01-01T00:00:03Z" } + } + ], + "expect": { + "chatSends": [ + "Leaderboard cleared. Times reset: 1.", + "No messages counted yet.", + "Leaderboard cleared. Times reset: 2." + ] + } + }, + { + "name": "!resetleaderboard is silent for a non-moderator", + "events": [ + { + "event": "chat.message.received", + "payload": { "id": "m1", "user": { "id": "u-alice", "displayName": "alice" }, "body": "hello", "timestamp": "2026-01-01T00:00:00Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m2", "user": { "id": "u-alice", "displayName": "alice" }, "body": "!resetleaderboard", "timestamp": "2026-01-01T00:00:01Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m3", "user": { "id": "u-alice", "displayName": "alice" }, "body": "!top", "timestamp": "2026-01-01T00:00:02Z" } + } + ], + "expect": { "chatSends": ["Top chatters: 1. alice (1)"] } + } +] diff --git a/examples/js/chat-leaderboard/package-lock.json b/examples/js/chat-leaderboard/package-lock.json new file mode 100644 index 0000000..0dab1b2 --- /dev/null +++ b/examples/js/chat-leaderboard/package-lock.json @@ -0,0 +1,35 @@ +{ + "name": "chat-leaderboard", + "version": "0.1.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "chat-leaderboard", + "version": "0.1.0", + "dependencies": { + "@owncast/plugin-sdk": "file:../../../sdks/js" + } + }, + "../../../sdks/js": { + "name": "@owncast/plugin-sdk", + "version": "0.10.1", + "hasInstallScript": true, + "license": "MIT", + "dependencies": { + "esbuild": "^0.24.0", + "jszip": "^3.10.1" + }, + "bin": { + "owncast-plugin": "bin/owncast-plugin.js" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@owncast/plugin-sdk": { + "resolved": "../../../sdks/js", + "link": true + } + } +} diff --git a/examples/js/chat-leaderboard/package.json b/examples/js/chat-leaderboard/package.json new file mode 100644 index 0000000..baa117e --- /dev/null +++ b/examples/js/chat-leaderboard/package.json @@ -0,0 +1,13 @@ +{ + "name": "chat-leaderboard", + "version": "0.1.0", + "private": true, + "scripts": { + "build": "owncast-plugin build", + "test": "owncast-plugin build && owncast-plugin test", + "serve": "owncast-plugin build && owncast-plugin serve" + }, + "dependencies": { + "@owncast/plugin-sdk": "file:../../../sdks/js" + } +} diff --git a/examples/js/chat-leaderboard/plugin.manifest.json b/examples/js/chat-leaderboard/plugin.manifest.json new file mode 100644 index 0000000..822708d --- /dev/null +++ b/examples/js/chat-leaderboard/plugin.manifest.json @@ -0,0 +1,15 @@ +{ + "api": "1", + "name": "Example Chat Leaderboard", + "slug": "chat-leaderboard", + "version": "0.1.0", + "description": "Ranks chatters by message count in the plugin's own SQL database. This example was written in JavaScript.", + "category": "analytics", + "permissions": [ + "storage.sql", + "chat.send" + ], + "bot": { + "displayName": "Example Leaderboard" + } +} diff --git a/examples/js/chat-leaderboard/src/plugin.js b/examples/js/chat-leaderboard/src/plugin.js new file mode 100644 index 0000000..6432301 --- /dev/null +++ b/examples/js/chat-leaderboard/src/plugin.js @@ -0,0 +1,121 @@ +const { definePlugin, owncast } = require("@owncast/plugin-sdk"); + +const TOP_N = 5; + +// The plugin gets one private SQLite database. There is no init hook, so the +// schema is created on first use. +// +// Both statements go in a single exec call, which the host runs as one +// transaction: either the whole schema is there or none of it is. That is why a +// half-applied migration is not a state this plugin can end up in. +let schemaReady = false; + +function ensureSchema() { + if (schemaReady) return; + owncast.sql.exec(` + CREATE TABLE IF NOT EXISTS chatters ( + user_id TEXT PRIMARY KEY, + display_name TEXT NOT NULL, + messages INTEGER NOT NULL DEFAULT 0 + ); + CREATE TABLE IF NOT EXISTS resets ( + reset_at TEXT NOT NULL + ); + `); + schemaReady = true; +} + +module.exports = definePlugin({ + // Count real chat, not command invocations, so `!top` does not inflate the + // score of whoever asked for it. + onChatMessage(msg) { + const body = msg.body || ""; + if (!msg.user || body.startsWith("!")) return; + ensureSchema(); + + // Keyed on the stable user id. The display name is stored alongside it and + // refreshed on every message, so a rename shows up in the standings without + // splitting anyone's history. `excluded` is the row the INSERT tried to add. + owncast.sql.exec( + `INSERT INTO chatters (user_id, display_name, messages) VALUES (?, ?, 1) + ON CONFLICT (user_id) DO UPDATE SET + messages = messages + 1, + display_name = excluded.display_name`, + [msg.user.id, msg.user.displayName || "someone"], + ); + }, + + commands: { + top: { + description: "Show the most active chatters", + run(ctx) { + ensureSchema(); + // Ranking is the reason this plugin uses SQL rather than the key-value + // store: the database does the sorting, and only the rows that will be + // shown cross into the plugin. + // + // The LIMIT is not optional. An unbounded query over a table that grows + // with your audience is refused by the host rather than quietly + // truncated, so write the bound you actually want. + const rows = owncast.sql.query( + `SELECT display_name, messages FROM chatters + ORDER BY messages DESC, display_name ASC, user_id ASC + LIMIT ?`, + [TOP_N], + ); + if (rows.length === 0) { + ctx.reply("No messages counted yet."); + return; + } + const standings = rows + .map((row, i) => `${i + 1}. ${row.display_name} (${row.messages})`) + .join(", "); + ctx.reply(`Top chatters: ${standings}`); + }, + }, + + rank: { + description: "Show your own position on the leaderboard", + run(ctx) { + ensureSchema(); + const userId = ctx.user ? ctx.user.id : ""; + // queryRow asks the host for a single row, so this stays cheap no matter + // how many chatters the table holds. It returns null when nothing + // matches, which here means the sender has not been counted yet. + const row = owncast.sql.queryRow( + `SELECT mine.display_name, + mine.messages, + (SELECT count(*) FROM chatters AS other + WHERE other.messages > mine.messages + OR (other.messages = mine.messages AND other.display_name < mine.display_name) + OR (other.messages = mine.messages AND other.display_name = mine.display_name AND other.user_id < mine.user_id)) + 1 AS position + FROM chatters AS mine + WHERE mine.user_id = ?`, + [userId], + ); + if (!row) { + ctx.reply("You have not sent any messages yet."); + return; + } + ctx.reply(`${row.display_name} is #${row.position} with ${row.messages} message(s).`); + }, + }, + + resetleaderboard: { + description: "Clear the leaderboard (moderators only)", + modOnly: true, + run(ctx) { + ensureSchema(); + // Two statements, one exec, one transaction. The audit row cannot be + // written without the standings being cleared, and the standings cannot + // be cleared without the audit row. + owncast.sql.exec( + `DELETE FROM chatters; + INSERT INTO resets (reset_at) VALUES (datetime('now'))`, + ); + const total = owncast.sql.queryRow("SELECT count(*) AS resets FROM resets"); + ctx.reply(`Leaderboard cleared. Times reset: ${total.resets}.`); + }, + }, + }, +}); diff --git a/examples/js/file-manager/INSTRUCTIONS.md b/examples/js/file-manager/INSTRUCTIONS.md index 54eea28..27d3a22 100644 --- a/examples/js/file-manager/INSTRUCTIONS.md +++ b/examples/js/file-manager/INSTRUCTIONS.md @@ -5,7 +5,7 @@ A small admin tool for the files in this plugin's private storage sandbox. ## What this plugin does - Adds a **Files** admin page. -- Lists the files in the plugin's sandbox (`data/plugin-data/file-manager/`). +- Lists the files in the plugin's sandbox (`data/plugin-storage/file-manager/files/`). - Lets you upload new files and delete existing ones. - Lets you download any file back to your machine. @@ -18,5 +18,5 @@ A small admin tool for the files in this plugin's private storage sandbox. ## Try it Enable the plugin, open the **Files** admin page, and upload a file. It's -written into `data/plugin-data/file-manager/` on the server. Refresh the page -and you'll see it listed. Download or delete it from there. +written into `data/plugin-storage/file-manager/files/` on the server. Refresh +the page and you'll see it listed. Download or delete it from there. diff --git a/examples/js/file-manager/README.md b/examples/js/file-manager/README.md index eee680d..bf26e70 100644 --- a/examples/js/file-manager/README.md +++ b/examples/js/file-manager/README.md @@ -1,7 +1,7 @@ # file-manager An admin page that browses, uploads, and deletes files in the plugin's own -private sandbox at `data/plugin-data/file-manager/`, all through the +private sandbox at `data/plugin-storage/file-manager/files/`, all through the `owncast.fs.*` API. The admin routes are gated by the host before the plugin sees them, so the handler never checks auth itself. diff --git a/examples/js/file-manager/__tests__/files.test.json b/examples/js/file-manager/__tests__/files.test.json index 75ffc95..092661c 100644 --- a/examples/js/file-manager/__tests__/files.test.json +++ b/examples/js/file-manager/__tests__/files.test.json @@ -126,6 +126,24 @@ } ] }, + { + "name": "deleting a missing file reports the host error", + "events": [ + { + "http": { + "method": "POST", + "path": "/admin/api/files/delete", + "headers": { "Content-Type": "application/json" }, + "body": "{\"name\":\"missing.txt\"}", + "authenticated": true, + "expect": { + "status": 500, + "body": "{\"ok\":false,\"error\":\"remove missing.txt: file does not exist\"}" + } + } + } + ] + }, { "name": "upload rejects an invalid file name", "events": [ diff --git a/examples/js/file-manager/public/admin/index.html b/examples/js/file-manager/public/admin/index.html index e25211b..6a07aa4 100644 --- a/examples/js/file-manager/public/admin/index.html +++ b/examples/js/file-manager/public/admin/index.html @@ -53,7 +53,7 @@

File Manager

These files live in this plugin's private sandbox - (data/plugin-data/file-manager/) via the + (data/plugin-storage/file-manager/files/) via the storage.fs permission. They are server-side only and never served publicly.

diff --git a/examples/js/file-manager/src/plugin.js b/examples/js/file-manager/src/plugin.js index 7302104..5250e93 100644 --- a/examples/js/file-manager/src/plugin.js +++ b/examples/js/file-manager/src/plugin.js @@ -1,9 +1,10 @@ // file-manager, a worked example of the storage.fs permission. // // It serves an admin-only page that lists the files in this plugin's -// private sandbox (data/plugin-data/file-manager/), lets you upload new -// ones, and delete existing ones. Everything goes through the owncast.fs.* -// API, so the host confines every path to the plugin's own directory. +// private sandbox (data/plugin-storage/file-manager/files/), lets you +// upload new ones, and delete existing ones. Everything goes through the +// owncast.fs.* API, so the host confines every path to the plugin's own +// directory. // // Routes (all the /admin/* ones are auth-gated by the host before the // plugin ever sees them, so the handler never checks auth itself): @@ -91,8 +92,8 @@ function uploadFile(req) { // owncast.fs.exists lets us tell the admin whether they replaced a file. const replaced = owncast.fs.exists(name); const result = owncast.fs.write(name, b64decode(dataBase64 || "")); - if (!result.ok) { - return json(500, { ok: false, error: result.error || "write failed" }); + if (result.error) { + return json(500, { ok: false, error: result.error }); } return json(200, { ok: true, replaced }); } @@ -108,8 +109,8 @@ function deleteFile(req) { return json(400, { ok: false, error: "invalid file name" }); } const result = owncast.fs.delete(parsed.name); - if (!result.ok) { - return json(500, { ok: false, error: result.error || "delete failed" }); + if (result.error) { + return json(500, { ok: false, error: result.error }); } return json(200, { ok: true }); } diff --git a/examples/python/README.md b/examples/python/README.md index 542c6df..f33c382 100644 --- a/examples/python/README.md +++ b/examples/python/README.md @@ -9,6 +9,7 @@ One self-contained plugin per directory, authored in Python and compiled to wasm | [echo-bot](./echo-bot/) | Posts a reply to every chat message via `owncast.chat.send`. | | [mod-commands](./mod-commands/) | Declarative `plugin.commands()` table: custom prefix, aliases, moderator gating, cooldowns, ordinary chat-handler composition, and `!help` metadata. | | [message-counter](./message-counter/) | Per-user message counter persisted in the plugin's namespaced config. | +| [chat-leaderboard](./chat-leaderboard/) | `storage.sql`, a private SQLite database: schema in one atomic `exec`, an `ON CONFLICT` upsert, a bounded ranked `query`, and `query_row` for a single row. | | [profanity-filter](./profanity-filter/) | `filter.modify(payload)`, rewrites flagged words to asterisks. | | [slow-mode](./slow-mode/) | `filter.drop(reason)`, rate-limits per user, with in-memory state. | | [buggy-filter](./buggy-filter/) | Always raises, exercises the host's fail-open + strike system. | diff --git a/examples/python/all-permissions-test/plugin.manifest.json b/examples/python/all-permissions-test/plugin.manifest.json index 78e7370..039fa6c 100644 --- a/examples/python/all-permissions-test/plugin.manifest.json +++ b/examples/python/all-permissions-test/plugin.manifest.json @@ -9,6 +9,7 @@ "storage.kv", "storage.upload", "storage.fs", + "storage.sql", "chat.send", "chat.history", "chat.moderate", diff --git a/examples/python/chat-leaderboard/INSTRUCTIONS.md b/examples/python/chat-leaderboard/INSTRUCTIONS.md new file mode 100644 index 0000000..3b11b02 --- /dev/null +++ b/examples/python/chat-leaderboard/INSTRUCTIONS.md @@ -0,0 +1,38 @@ +# Example Chat Leaderboard + +Keeps a running count of how many messages each chatter has sent and posts the +standings in chat on request. + +Counting starts as soon as you enable the plugin. Messages that begin with `!` +are not counted, so asking for the leaderboard does not raise your own score. +Chatters are tracked by their account, not their name, so someone who renames +keeps their history and shows up under their new name. + +## Commands + +Enable the plugin in **Admin → Plugins**, then type these in chat. + +| Command | Who can use it | What it does | +| --- | --- | --- | +| `!top` | anyone | Posts the five most active chatters with their message counts. Says `No messages counted yet.` on an empty board. | +| `!rank` | anyone | Posts the sender's own position and message count, or tells them they have not been counted yet. | +| `!resetleaderboard` | moderators only | Clears the standings and records that a reset happened. Non-moderator invocations are silent. | + +## Where the data lives + +The counts are kept in a small database that belongs to this plugin alone, on +your server under `data/plugin-storage/chat-leaderboard/db/`. Nothing else can +read it, and it is not served over the web. + +Two things worth knowing before you rely on it: + +- It is **not** included in Owncast's database backups. If the standings matter + to you, copy that directory yourself. +- Uninstalling the plugin leaves the data in place, so reinstalling picks the + standings back up. To start clean, delete that directory while the plugin is + disabled. + +## Permissions + +- **storage.sql** gives the plugin its own private database for the counts. +- **chat.send** lets the bot post the standings. diff --git a/examples/python/chat-leaderboard/README.md b/examples/python/chat-leaderboard/README.md new file mode 100644 index 0000000..4f6f21c --- /dev/null +++ b/examples/python/chat-leaderboard/README.md @@ -0,0 +1,64 @@ +# chat-leaderboard + +Counts how many messages each chatter has sent and ranks them, using the +plugin's own private SQLite database at +`data/plugin-storage/chat-leaderboard/db/`. `!top` shows the standings, `!rank` +shows the sender's own position, and a moderator can clear the board with +`!resetleaderboard`. + +**Demonstrates:** the `storage.sql` permission end to end. `owncast.sql.exec` +for schema creation, an `ON CONFLICT` upsert, and a two-statement atomic batch; +`owncast.sql.query` for a bounded, database-sorted result set (rows arrive as +dicts keyed by column name); and `owncast.sql.query_row` for a single row +(returning `None` when nothing matches). A failed statement raises +`RuntimeError`. + +## Why SQL and not `storage.kv` + +[message-counter](../message-counter/) keeps the same per-user counts in the +key-value store, and that is the right choice when you only ever read a value +back by its key. It cannot answer "who are the top five", because ranking means +sorting across every key, and the plugin would have to pull all of them into +memory to do it. + +Here the database does the sorting and only the rows that will be shown cross +the host boundary. The trade is that you own a schema. + +## What the host enforces + +- **`exec` is one transaction.** A multi-statement batch commits whole or leaves + the database untouched. The schema, and the reset that clears the standings + while writing its audit row, both rely on this. +- **`query` never truncates silently.** A query that returns more than 10000 + rows, or more than 1 MiB of encoded results, raises and asks for a `LIMIT`. + `!top` passes its bound as a parameter. Use `query_row` when one row will do. +- **The database is private and capped.** 128 MiB, separate from the + `storage.fs` quota. Plugins cannot reach each other's databases, and + `ATTACH`, `PRAGMA`, temporary tables, and `load_extension` are all refused. +- **Not in Owncast's backups.** Treat the contents as rebuildable, or export + what matters yourself. + +## Run it + +```bash +owncast-plugin-py test # build + run the tests +``` + +The test runner gives the plugin a real in-memory SQLite database, so the +scenarios exercise the actual SQL without a running Owncast. Each run starts +from an empty schema. + +`owncast-plugin-py serve` drives chat too, but the dev server does not dispatch +chat commands yet, so `!top`, `!rank`, and `!resetleaderboard` only answer under +`owncast-plugin-py test` or on a real Owncast instance. Plain messages still +reach the `@plugin.on_chat_message` handler and are counted: + +```bash +curl -XPOST localhost:8080/_dev/chat -d '{"user":"alice","body":"hello"}' +``` + +## Permissions + +- **storage.sql** for the private database. +- **chat.send** posts the bot's replies. The moderator gating on + `!resetleaderboard` needs no extra permission. diff --git a/examples/python/chat-leaderboard/__tests__/chat-leaderboard.test.json b/examples/python/chat-leaderboard/__tests__/chat-leaderboard.test.json new file mode 100644 index 0000000..b5ee8c3 --- /dev/null +++ b/examples/python/chat-leaderboard/__tests__/chat-leaderboard.test.json @@ -0,0 +1,126 @@ +[ + { + "name": "!top reports an empty leaderboard before anyone has chatted", + "events": [ + { + "event": "chat.message.received", + "payload": { "id": "m1", "user": { "id": "u-alice", "displayName": "alice" }, "body": "!top", "timestamp": "2026-01-01T00:00:00Z" } + } + ], + "expect": { "chatSends": ["No messages counted yet."] } + }, + { + "name": "counting is silent, and !top ranks chatters by message count", + "events": [ + { + "event": "chat.message.received", + "payload": { "id": "m1", "user": { "id": "u-alice", "displayName": "alice" }, "body": "hello", "timestamp": "2026-01-01T00:00:00Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m2", "user": { "id": "u-bob", "displayName": "bob" }, "body": "hi", "timestamp": "2026-01-01T00:00:01Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m3", "user": { "id": "u-alice", "displayName": "alice" }, "body": "again", "timestamp": "2026-01-01T00:00:02Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m4", "user": { "id": "u-bob", "displayName": "bob" }, "body": "!top", "timestamp": "2026-01-01T00:00:03Z" } + } + ], + "expect": { "chatSends": ["Top chatters: 1. alice (2), 2. bob (1)"] } + }, + { + "name": "a rename follows the same row instead of splitting the history", + "events": [ + { + "event": "chat.message.received", + "payload": { "id": "m1", "user": { "id": "u-alice", "displayName": "alice" }, "body": "first", "timestamp": "2026-01-01T00:00:00Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m2", "user": { "id": "u-alice", "displayName": "alice-renamed" }, "body": "second", "timestamp": "2026-01-01T00:00:01Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m3", "user": { "id": "u-alice", "displayName": "alice-renamed" }, "body": "!top", "timestamp": "2026-01-01T00:00:02Z" } + } + ], + "expect": { "chatSends": ["Top chatters: 1. alice-renamed (2)"] } + }, + { + "name": "!rank uses the same tie ordering as !top", + "events": [ + { + "event": "chat.message.received", + "payload": { "id": "m1", "user": { "id": "u-alice", "displayName": "alice" }, "body": "one", "timestamp": "2026-01-01T00:00:00Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m2", "user": { "id": "u-bob", "displayName": "bob" }, "body": "one", "timestamp": "2026-01-01T00:00:01Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m3", "user": { "id": "u-bob", "displayName": "bob" }, "body": "!rank", "timestamp": "2026-01-01T00:00:02Z" } + } + ], + "expect": { "chatSends": ["bob is #2 with 1 message(s)."] } + }, + { + "name": "!rank says so when the sender has not been counted yet", + "events": [ + { + "event": "chat.message.received", + "payload": { "id": "m1", "user": { "id": "u-newcomer", "displayName": "newcomer" }, "body": "!rank", "timestamp": "2026-01-01T00:00:00Z" } + } + ], + "expect": { "chatSends": ["You have not sent any messages yet."] } + }, + { + "name": "a moderator can clear the leaderboard, and the audit row commits with it", + "events": [ + { + "event": "chat.message.received", + "payload": { "id": "m1", "user": { "id": "u-alice", "displayName": "alice" }, "body": "hello", "timestamp": "2026-01-01T00:00:00Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m2", "user": { "id": "u-mod", "displayName": "mod", "scopes": ["MODERATOR"] }, "body": "!resetleaderboard", "timestamp": "2026-01-01T00:00:01Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m3", "user": { "id": "u-mod", "displayName": "mod", "scopes": ["MODERATOR"] }, "body": "!top", "timestamp": "2026-01-01T00:00:02Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m4", "user": { "id": "u-mod", "displayName": "mod", "scopes": ["MODERATOR"] }, "body": "!resetleaderboard", "timestamp": "2026-01-01T00:00:03Z" } + } + ], + "expect": { + "chatSends": [ + "Leaderboard cleared. Times reset: 1.", + "No messages counted yet.", + "Leaderboard cleared. Times reset: 2." + ] + } + }, + { + "name": "!resetleaderboard is silent for a non-moderator", + "events": [ + { + "event": "chat.message.received", + "payload": { "id": "m1", "user": { "id": "u-alice", "displayName": "alice" }, "body": "hello", "timestamp": "2026-01-01T00:00:00Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m2", "user": { "id": "u-alice", "displayName": "alice" }, "body": "!resetleaderboard", "timestamp": "2026-01-01T00:00:01Z" } + }, + { + "event": "chat.message.received", + "payload": { "id": "m3", "user": { "id": "u-alice", "displayName": "alice" }, "body": "!top", "timestamp": "2026-01-01T00:00:02Z" } + } + ], + "expect": { "chatSends": ["Top chatters: 1. alice (1)"] } + } +] diff --git a/examples/python/chat-leaderboard/plugin.manifest.json b/examples/python/chat-leaderboard/plugin.manifest.json new file mode 100644 index 0000000..6de3f25 --- /dev/null +++ b/examples/python/chat-leaderboard/plugin.manifest.json @@ -0,0 +1,15 @@ +{ + "api": "1", + "name": "Example Chat Leaderboard", + "slug": "chat-leaderboard", + "version": "0.1.0", + "description": "Ranks chatters by message count in the plugin's own SQL database. This example was written in Python.", + "category": "analytics", + "permissions": [ + "storage.sql", + "chat.send" + ], + "bot": { + "displayName": "Example Leaderboard" + } +} diff --git a/examples/python/chat-leaderboard/src/plugin.py b/examples/python/chat-leaderboard/src/plugin.py new file mode 100644 index 0000000..4856402 --- /dev/null +++ b/examples/python/chat-leaderboard/src/plugin.py @@ -0,0 +1,140 @@ +from owncast_plugin import plugin, owncast + +TOP_N = 5 + +# The plugin gets one private SQLite database. There is no init hook, so the +# schema is created on first use. +# +# Both statements go in a single exec call, which the host runs as one +# transaction: either the whole schema is there or none of it is. That is why a +# half-applied migration is not a state this plugin can end up in. +_schema_ready = False + + +def _ensure_schema(): + global _schema_ready + if _schema_ready: + return + owncast.sql.exec( + """ + CREATE TABLE IF NOT EXISTS chatters ( + user_id TEXT PRIMARY KEY, + display_name TEXT NOT NULL, + messages INTEGER NOT NULL DEFAULT 0 + ); + CREATE TABLE IF NOT EXISTS resets ( + reset_at TEXT NOT NULL + ); + """ + ) + _schema_ready = True + + +# Count real chat, not command invocations, so !top does not inflate the score +# of whoever asked for it. +@plugin.on_chat_message +def _count(msg): + body = msg.body or "" + if not msg.user or body.startswith("!"): + return + _ensure_schema() + + # Keyed on the stable user id. The display name is stored alongside it and + # refreshed on every message, so a rename shows up in the standings without + # splitting anyone's history. `excluded` is the row the INSERT tried to add. + owncast.sql.exec( + """ + INSERT INTO chatters (user_id, display_name, messages) VALUES (?, ?, 1) + ON CONFLICT (user_id) DO UPDATE SET + messages = messages + 1, + display_name = excluded.display_name + """, + [msg.user.id, msg.user.display_name or "someone"], + ) + + +def _top(ctx): + _ensure_schema() + # Ranking is the reason this plugin uses SQL rather than the key-value + # store: the database does the sorting, and only the rows that will be shown + # cross into the plugin. + # + # The LIMIT is not optional. An unbounded query over a table that grows with + # your audience is refused by the host rather than quietly truncated, so + # write the bound you actually want. + rows = owncast.sql.query( + """ + SELECT display_name, messages FROM chatters + ORDER BY messages DESC, display_name ASC, user_id ASC + LIMIT ? + """, + [TOP_N], + ) + if not rows: + ctx.reply("No messages counted yet.") + return + standings = ", ".join( + f"{i + 1}. {row['display_name']} ({row['messages']})" + for i, row in enumerate(rows) + ) + ctx.reply(f"Top chatters: {standings}") + + +def _rank(ctx): + _ensure_schema() + user_id = ctx.user.id if ctx.user else "" + # query_row asks the host for a single row, so this stays cheap no matter how + # many chatters the table holds. It returns None when nothing matches, which + # here means the sender has not been counted yet. + row = owncast.sql.query_row( + """ + SELECT mine.display_name, + mine.messages, + (SELECT count(*) FROM chatters AS other + WHERE other.messages > mine.messages + OR (other.messages = mine.messages AND other.display_name < mine.display_name) + OR (other.messages = mine.messages AND other.display_name = mine.display_name AND other.user_id < mine.user_id)) + 1 AS position + FROM chatters AS mine + WHERE mine.user_id = ? + """, + [user_id], + ) + if not row: + ctx.reply("You have not sent any messages yet.") + return + ctx.reply( + f"{row['display_name']} is #{row['position']} " + f"with {row['messages']} message(s)." + ) + + +def _reset(ctx): + _ensure_schema() + # Two statements, one exec, one transaction. The audit row cannot be written + # without the standings being cleared, and the standings cannot be cleared + # without the audit row. + owncast.sql.exec( + """ + DELETE FROM chatters; + INSERT INTO resets (reset_at) VALUES (datetime('now')) + """ + ) + total = owncast.sql.query_row("SELECT count(*) AS resets FROM resets") + ctx.reply(f"Leaderboard cleared. Times reset: {total['resets']}.") + + +plugin.commands({ + "top": { + "description": "Show the most active chatters", + "run": _top, + }, + "rank": { + "description": "Show your own position on the leaderboard", + "run": _rank, + }, + "resetleaderboard": { + "description": "Clear the leaderboard (moderators only)", + "mod_only": True, + "run": _reset, + }, +}) diff --git a/examples/python/file-manager/INSTRUCTIONS.md b/examples/python/file-manager/INSTRUCTIONS.md index 54eea28..27d3a22 100644 --- a/examples/python/file-manager/INSTRUCTIONS.md +++ b/examples/python/file-manager/INSTRUCTIONS.md @@ -5,7 +5,7 @@ A small admin tool for the files in this plugin's private storage sandbox. ## What this plugin does - Adds a **Files** admin page. -- Lists the files in the plugin's sandbox (`data/plugin-data/file-manager/`). +- Lists the files in the plugin's sandbox (`data/plugin-storage/file-manager/files/`). - Lets you upload new files and delete existing ones. - Lets you download any file back to your machine. @@ -18,5 +18,5 @@ A small admin tool for the files in this plugin's private storage sandbox. ## Try it Enable the plugin, open the **Files** admin page, and upload a file. It's -written into `data/plugin-data/file-manager/` on the server. Refresh the page -and you'll see it listed. Download or delete it from there. +written into `data/plugin-storage/file-manager/files/` on the server. Refresh +the page and you'll see it listed. Download or delete it from there. diff --git a/examples/python/file-manager/README.md b/examples/python/file-manager/README.md index 47acded..6dbd7a9 100644 --- a/examples/python/file-manager/README.md +++ b/examples/python/file-manager/README.md @@ -1,6 +1,6 @@ # file-manager -An admin page that browses, uploads, and deletes files in the plugin's own private sandbox at `data/plugin-data/file-manager/`, all through the `owncast.fs.*` API. The admin routes are gated by the host before the plugin sees them, so the handler never checks auth itself. +An admin page that browses, uploads, and deletes files in the plugin's own private sandbox at `data/plugin-storage/file-manager/files/`, all through the `owncast.fs.*` API. The admin routes are gated by the host before the plugin sees them, so the handler never checks auth itself. **Demonstrates:** the `storage.fs` permission end to end: `owncast.fs.list` (browse), `owncast.fs.write` + `owncast.fs.exists` (upload, reporting whether a file was replaced), `owncast.fs.read` (download), and `owncast.fs.delete` (remove). Binary files cross the string-typed HTTP body base64-encoded. The plugin uses Python's stdlib `base64`, tolerating missing padding to mirror the JS codec's leniency. diff --git a/examples/python/file-manager/__tests__/files.test.json b/examples/python/file-manager/__tests__/files.test.json index 75ffc95..092661c 100644 --- a/examples/python/file-manager/__tests__/files.test.json +++ b/examples/python/file-manager/__tests__/files.test.json @@ -126,6 +126,24 @@ } ] }, + { + "name": "deleting a missing file reports the host error", + "events": [ + { + "http": { + "method": "POST", + "path": "/admin/api/files/delete", + "headers": { "Content-Type": "application/json" }, + "body": "{\"name\":\"missing.txt\"}", + "authenticated": true, + "expect": { + "status": 500, + "body": "{\"ok\":false,\"error\":\"remove missing.txt: file does not exist\"}" + } + } + } + ] + }, { "name": "upload rejects an invalid file name", "events": [ diff --git a/examples/python/file-manager/public/admin/index.html b/examples/python/file-manager/public/admin/index.html index e25211b..6a07aa4 100644 --- a/examples/python/file-manager/public/admin/index.html +++ b/examples/python/file-manager/public/admin/index.html @@ -53,7 +53,7 @@

File Manager

These files live in this plugin's private sandbox - (data/plugin-data/file-manager/) via the + (data/plugin-storage/file-manager/files/) via the storage.fs permission. They are server-side only and never served publicly.

diff --git a/examples/python/file-manager/src/plugin.py b/examples/python/file-manager/src/plugin.py index 75a9fd3..8a26884 100644 --- a/examples/python/file-manager/src/plugin.py +++ b/examples/python/file-manager/src/plugin.py @@ -1,9 +1,10 @@ # file-manager, a worked example of the storage.fs permission. # # It serves an admin-only page that lists the files in this plugin's -# private sandbox (data/plugin-data/file-manager/), lets you upload new -# ones, and delete existing ones. Everything goes through the owncast.fs.* -# API, so the host confines every path to the plugin's own directory. +# private sandbox (data/plugin-storage/file-manager/files/), lets you +# upload new ones, and delete existing ones. Everything goes through the +# owncast.fs.* API, so the host confines every path to the plugin's own +# directory. # # Routes (all the /admin/* ones are auth-gated by the host before the # plugin ever sees them, so the handler never checks auth itself): @@ -35,9 +36,8 @@ def upload_file(req): # owncast.fs.exists lets us tell the admin whether they replaced a file. replaced = owncast.fs.exists(name) result = owncast.fs.write(name, b64decode(parsed.get("dataBase64") or "")) - if not (result and result.get("ok")): - err = (result or {}).get("error") or "write failed" - return json_resp(500, {"ok": False, "error": err}) + if result.get("error"): + return json_resp(500, {"ok": False, "error": result["error"]}) return json_resp(200, {"ok": True, "replaced": replaced}) @@ -49,9 +49,8 @@ def delete_file(req): if bad_name(parsed.get("name")): return json_resp(400, {"ok": False, "error": "invalid file name"}) result = owncast.fs.delete(parsed.get("name")) - if not (result and result.get("ok")): - err = (result or {}).get("error") or "delete failed" - return json_resp(500, {"ok": False, "error": err}) + if result.get("error"): + return json_resp(500, {"ok": False, "error": result["error"]}) return json_resp(200, {"ok": True}) diff --git a/host-runtime/cmd/owncast-plugin-serve/main.go b/host-runtime/cmd/owncast-plugin-serve/main.go index 0e5df47..3b2f748 100644 --- a/host-runtime/cmd/owncast-plugin-serve/main.go +++ b/host-runtime/cmd/owncast-plugin-serve/main.go @@ -33,6 +33,7 @@ import ( "time" extism "github.com/extism/go-sdk" + "github.com/owncast/owncast-plugin-sdk/host-runtime/sqlstore" plugin "github.com/owncast/owncast/services/plugins" "github.com/owncast/owncast/services/plugins/kv" ) @@ -71,7 +72,7 @@ func main() { if info, statErr := os.Stat(abs); statErr == nil && !info.IsDir() { devDataBase = filepath.Dir(abs) } - devDataRoot := filepath.Join(devDataBase, ".owncast-dev-data", "plugin-data") + devDataRoot := filepath.Join(devDataBase, ".owncast-dev-data", "plugin-storage") ctx := context.Background() extism.SetLogLevel(extism.LogLevelError) @@ -277,6 +278,12 @@ func main() { GetRequestUser: devRequestUser, } + // storage.sql, on the same terms as the dev key-value store: a private + // in-memory database per plugin, so a restart starts clean. + sqlStore := sqlstore.NewMemory() + defer sqlStore.Close() + env.SQLExec = sqlStore.Exec + env.SQLQuery = sqlStore.Query loaded, name, staticDescription := loadTarget(ctx, env, abs) defer loaded.Close(ctx) @@ -561,12 +568,12 @@ func devRequestUser(r *http.Request) *plugin.HostUser { } // fsSandboxPath maps a plugin-supplied relative path to an absolute path -// inside root/ and refuses to escape it. Mirrors the production host's -// storage.fs sandboxing (rel is rooted at "/" then cleaned, so "../" and -// absolute paths collapse back inside) so `owncast-plugin serve` behaves like -// real Owncast. +// inside root//files and refuses to escape it. Mirrors the production +// host's storage.fs sandbox (data/plugin-storage//files/, with rel +// rooted at "/" then cleaned, so "../" and absolute paths collapse back +// inside) so `owncast-plugin serve` behaves like real Owncast. func fsSandboxPath(root, pluginName, rel string) (string, error) { - sandbox, err := filepath.Abs(filepath.Join(root, pluginName)) + sandbox, err := filepath.Abs(filepath.Join(root, pluginName, "files")) if err != nil { return "", err } diff --git a/host-runtime/go.mod b/host-runtime/go.mod index 6d61119..0f076b6 100644 --- a/host-runtime/go.mod +++ b/host-runtime/go.mod @@ -5,17 +5,27 @@ go 1.26.2 require ( github.com/extism/go-sdk v1.7.1 github.com/gobwas/glob v0.2.3 - github.com/owncast/owncast v0.2.6-0.20260714223726-763564636ead + github.com/owncast/owncast v0.2.6-0.20260801053819-b13a53cdb0d6 + modernc.org/sqlite v1.53.0 ) require github.com/sirupsen/logrus v1.9.4 // indirect require ( + github.com/dustin/go-humanize v1.0.1 // indirect github.com/dylibso/observe-sdk/go v0.0.0-20240819160327-2d926c5d788a // indirect + github.com/google/uuid v1.6.0 // indirect github.com/ianlancetaylor/demangle v0.0.0-20240805132620-81f5be970eca // indirect + github.com/mattn/go-isatty v0.0.21 // indirect + github.com/ncruces/go-strftime v1.0.0 // indirect + github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect github.com/tetratelabs/wabin v0.0.0-20230304001439-f6f874872834 // indirect github.com/tetratelabs/wazero v1.12.0 // indirect go.opentelemetry.io/proto/otlp v1.3.1 // indirect golang.org/x/sys v0.47.0 // indirect + golang.org/x/tools v0.47.0 // indirect google.golang.org/protobuf v1.36.11 // indirect + modernc.org/libc v1.73.4 // indirect + modernc.org/mathutil v1.7.1 // indirect + modernc.org/memory v1.11.0 // indirect ) diff --git a/host-runtime/go.sum b/host-runtime/go.sum index 8680c33..f313acb 100644 --- a/host-runtime/go.sum +++ b/host-runtime/go.sum @@ -1,5 +1,7 @@ github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= +github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= github.com/dylibso/observe-sdk/go v0.0.0-20240819160327-2d926c5d788a h1:UwSIFv5g5lIvbGgtf3tVwC7Ky9rmMFBp0RMs+6f6YqE= github.com/dylibso/observe-sdk/go v0.0.0-20240819160327-2d926c5d788a/go.mod h1:C8DzXehI4zAbrdlbtOByKX6pfivJTBiV9Jjqv56Yd9Q= github.com/extism/go-sdk v1.7.1 h1:lWJos6uY+tRFdlIHR+SJjwFDApY7OypS/2nMhiVQ9Sw= @@ -8,12 +10,26 @@ github.com/gobwas/glob v0.2.3 h1:A4xDbljILXROh+kObIiy5kIaPYD8e96x1tgBhUI5J+Y= github.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJAkT8= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs= +github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= +github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= github.com/ianlancetaylor/demangle v0.0.0-20240805132620-81f5be970eca h1:T54Ema1DU8ngI+aef9ZhAhNGQhcRTrWxVeG07F+c/Rw= github.com/ianlancetaylor/demangle v0.0.0-20240805132620-81f5be970eca/go.mod h1:gx7rwoVhcfuVKG5uya9Hs3Sxj7EIvldVofAWIUtGouw= -github.com/owncast/owncast v0.2.6-0.20260714223726-763564636ead h1:69RvI4bgOf/SsLPmItMozKys8cM5E9M6pZcER2MFP9k= -github.com/owncast/owncast v0.2.6-0.20260714223726-763564636ead/go.mod h1:Z99fFjsXBLPapY7EzYHEqd6sPIsxjH5Af2fYpIrnalU= +github.com/mattn/go-isatty v0.0.21 h1:xYae+lCNBP7QuW4PUnNG61ffM4hVIfm+zUzDuSzYLGs= +github.com/mattn/go-isatty v0.0.21/go.mod h1:ZXfXG4SQHsB/w3ZeOYbR0PrPwLy+n6xiMrJlRFqopa4= +github.com/mattn/go-sqlite3 v1.14.47 h1:jOBI62gS7nKeZv+as1oGEy0+1qISgXwH/QBlR6KbfIo= +github.com/mattn/go-sqlite3 v1.14.47/go.mod h1:6JTjA44L93a0QCyJef5YvlPoKXntQPjzWv5gtm9sB6w= +github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w= +github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= +github.com/owncast/owncast v0.2.6-0.20260801053819-b13a53cdb0d6 h1:PnPcT+aVkSmHyu3l4f41DAcu4oyKdL/4XNp98i8TvVQ= +github.com/owncast/owncast v0.2.6-0.20260801053819-b13a53cdb0d6/go.mod h1:Z99fFjsXBLPapY7EzYHEqd6sPIsxjH5Af2fYpIrnalU= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= +github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= github.com/sirupsen/logrus v1.9.4 h1:TsZE7l11zFCLZnZ+teH4Umoq5BhEIfIzfRDZ1Uzql2w= github.com/sirupsen/logrus v1.9.4/go.mod h1:ftWc9WdOfJ0a92nsE2jF5u5ZwH8Bv2zdeOC42RjbV2g= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= @@ -24,9 +40,43 @@ github.com/tetratelabs/wazero v1.12.0 h1:DuWcpNu/FzgEXgGBDp8J1Spc+CWOvvtvVyjKlaZ github.com/tetratelabs/wazero v1.12.0/go.mod h1:LvKtzl2RqO4gyF27BiXU+nKAjcV8f38U+kP/q2vgxh0= go.opentelemetry.io/proto/otlp v1.3.1 h1:TrMUixzpM0yuc/znrFTP9MMRh8trP93mkCiDVeXrui0= go.opentelemetry.io/proto/otlp v1.3.1/go.mod h1:0X1WI4de4ZsLrrJNLAQbFeLCm3T7yBkR0XqQ7niQU+8= +golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ= +golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0= +golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek= +golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= +golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q= +golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA= google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE= google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +modernc.org/cc/v4 v4.28.4 h1:Hd/4Es+MBj+/7hSdZaisNyu6bv3V0Dp2MdllyfqaH+c= +modernc.org/cc/v4 v4.28.4/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI= +modernc.org/ccgo/v4 v4.34.4 h1:OVnSOWQjVKOYkFxoHYB+qQmSHK5gqMqARM+K9DpR/Ws= +modernc.org/ccgo/v4 v4.34.4/go.mod h1:qdKqE8FNIYyysougB1RX9MxCzp5oJOcQXSobANJ4TuE= +modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM= +modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU= +modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI= +modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito= +modernc.org/gc/v3 v3.1.3 h1:6QAplYyVO+KdPW3pGnqmJDUxtkec8ooEWvks/hhU3lc= +modernc.org/gc/v3 v3.1.3/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY= +modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks= +modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI= +modernc.org/libc v1.73.4 h1:+ra4Ui8ngyt8HDcO1FTDPWlkAh6yOdaO2yAoh8MddQA= +modernc.org/libc v1.73.4/go.mod h1:DXZ3eO8qMCNn2SnmTNCiC71nJ9Rcq3PsnpU6Vc4rWK8= +modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU= +modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg= +modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI= +modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw= +modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg= +modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns= +modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w= +modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE= +modernc.org/sqlite v1.53.0 h1:20WG8N9q4ji/dEqGk4uiI0c6OPjSeLTNYGFCc3+7c1M= +modernc.org/sqlite v1.53.0/go.mod h1:xoEpOIpGrgT48H5iiyt/YXPCZPEzlfmfFwtk8Lklw8s= +modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0= +modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A= +modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y= +modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM= diff --git a/host-runtime/main.go b/host-runtime/main.go index 1cb7ddd..0c55215 100644 --- a/host-runtime/main.go +++ b/host-runtime/main.go @@ -12,13 +12,14 @@ import ( "time" extism "github.com/extism/go-sdk" - "github.com/owncast/owncast/services/plugins/kv" + "github.com/owncast/owncast-plugin-sdk/host-runtime/sqlstore" plugin "github.com/owncast/owncast/services/plugins" + "github.com/owncast/owncast/services/plugins/kv" ) // demoFS is the in-memory storage.fs backing for the demo host, keyed by // plugin slug then cleaned path. Production wires this to a real sandboxed -// directory under data/plugin-data//. +// directory under data/plugin-storage//files/. var demoFS = map[string]map[string][]byte{} // demoFSClean normalizes a plugin path the way the real sandbox does: rooted @@ -40,10 +41,10 @@ func isAuthenticatedHeader(r *http.Request) bool { } type ChatMessage struct { - ID string `json:"id"` + ID string `json:"id"` User *plugin.HostUser `json:"user,omitempty"` - Body string `json:"body"` - Timestamp time.Time `json:"timestamp"` + Body string `json:"body"` + Timestamp time.Time `json:"timestamp"` } // chatUser builds the nested ChatUser object the host sends with every chat @@ -137,7 +138,7 @@ func main() { // storage.fs: in-memory sandbox for the demo, keyed by plugin slug // then cleaned path, so owncast.fs.* round-trips work without writing - // to disk. Production wires this to data/plugin-data//. + // to disk. Production wires this to data/plugin-storage//files/. FSRead: func(pluginName, p string) ([]byte, error) { data, ok := demoFS[pluginName][demoFSClean(p)] if !ok { @@ -214,6 +215,12 @@ func main() { }, } + // storage.sql, matching this host's in-memory key-value store: a private + // database per plugin, discarded when the demo exits. + sqlStore := sqlstore.NewMemory() + defer sqlStore.Close() + env.SQLExec = sqlStore.Exec + env.SQLQuery = sqlStore.Query mgr := plugin.NewManager(pluginsDir, env) if err := mgr.Start(ctx); err != nil { log.Fatalf("start plugin manager: %v", err) diff --git a/host-runtime/plugin/testing/mocks.go b/host-runtime/plugin/testing/mocks.go index 02aa328..0611fd9 100644 --- a/host-runtime/plugin/testing/mocks.go +++ b/host-runtime/plugin/testing/mocks.go @@ -11,6 +11,7 @@ import ( "sync" "github.com/gobwas/glob" + "github.com/owncast/owncast-plugin-sdk/host-runtime/sqlstore" plugin "github.com/owncast/owncast/services/plugins" "github.com/owncast/owncast/services/plugins/kv" ) @@ -122,7 +123,10 @@ type MockHost struct { // plugin slug then by cleaned relative path. Lets scenario tests // exercise owncast.fs.* round-trips (write then read/list/delete) // without touching the real disk. - fsFiles map[string]map[string][]byte + fsFiles map[string]map[string][]byte + // sql holds the scenario's per-plugin SQLite databases, opened on first use + // so a scenario that never touches storage.sql pays nothing for it. + sql *sqlstore.Store fediversePosts []RecordedFediverse fediverseOutbox []string chatTo []RecordedChatTo @@ -441,6 +445,12 @@ func (m *MockHost) HostEnv() *plugin.HostEnv { _, ok := m.fsFiles[pluginName][mockFSClean(p)] return ok, nil }, + SQLExec: func(ctx context.Context, pluginName string, req plugin.SQLRequest) plugin.SQLExecResult { + return m.sqlExec(ctx, pluginName, req) + }, + SQLQuery: func(ctx context.Context, pluginName string, req plugin.SQLRequest) plugin.SQLQueryResult { + return m.sqlQuery(ctx, pluginName, req) + }, } } diff --git a/host-runtime/plugin/testing/runner.go b/host-runtime/plugin/testing/runner.go index a07f5a1..f38b31a 100644 --- a/host-runtime/plugin/testing/runner.go +++ b/host-runtime/plugin/testing/runner.go @@ -46,6 +46,7 @@ func RunFile(ctx context.Context, wasmPath, manifestPath, path string) ([]Result // here fails to install on a real Owncast server. func LoadCheck(ctx context.Context, wasmPath, manifestPath string) error { mock := NewMockHost() + defer mock.closeSQL() origTransport := http.DefaultClient.Transport http.DefaultClient.Transport = mock.HTTPTransport() defer func() { http.DefaultClient.Transport = origTransport }() @@ -62,6 +63,7 @@ func runOne(ctx context.Context, wasmPath, manifestPath, file string, sc Scenari res := Result{File: file, Scenario: sc.Name} mock := NewMockHost() + defer mock.closeSQL() // Install the mock HTTP transport on http.DefaultClient (which Extism's // built-in http_request uses). Restore on exit so scenarios are isolated. diff --git a/host-runtime/plugin/testing/sql.go b/host-runtime/plugin/testing/sql.go new file mode 100644 index 0000000..5d90524 --- /dev/null +++ b/host-runtime/plugin/testing/sql.go @@ -0,0 +1,41 @@ +package testing + +import ( + "context" + + "github.com/owncast/owncast-plugin-sdk/host-runtime/sqlstore" + plugin "github.com/owncast/owncast/services/plugins" +) + +// The scenario runner, the dev server, and the demo host share one SQL +// implementation (host-runtime/sqlstore), so a plugin sees the same limits and +// the same errors whichever one it runs under. See that package for what it does +// and does not reproduce from Owncast. + +func (m *MockHost) sqlStore() *sqlstore.Store { + m.mu.Lock() + defer m.mu.Unlock() + if m.sql == nil { + m.sql = sqlstore.NewMemory() + } + return m.sql +} + +func (m *MockHost) sqlExec(ctx context.Context, pluginName string, req plugin.SQLRequest) plugin.SQLExecResult { + return m.sqlStore().Exec(ctx, pluginName, req) +} + +func (m *MockHost) sqlQuery(ctx context.Context, pluginName string, req plugin.SQLRequest) plugin.SQLQueryResult { + return m.sqlStore().Query(ctx, pluginName, req) +} + +// closeSQL drops every database the scenario opened. +func (m *MockHost) closeSQL() { + m.mu.Lock() + store := m.sql + m.sql = nil + m.mu.Unlock() + if store != nil { + store.Close() + } +} diff --git a/host-runtime/plugin/testing/sql_test.go b/host-runtime/plugin/testing/sql_test.go new file mode 100644 index 0000000..82f9f4c --- /dev/null +++ b/host-runtime/plugin/testing/sql_test.go @@ -0,0 +1,165 @@ +package testing + +import ( + "context" + "encoding/json" + "strings" + "testing" + + plugin "github.com/owncast/owncast/services/plugins" +) + +func TestMockHostSQLRoundTripAndIsolation(t *testing.T) { + mock := NewMockHost() + defer mock.closeSQL() + env := mock.HostEnv() + ctx := context.Background() + + if result := env.SQLExec(ctx, "demo", plugin.SQLRequest{SQL: "CREATE TABLE items (id INTEGER PRIMARY KEY, name TEXT)"}); result.Error != "" { + t.Fatal(result.Error) + } + if result := env.SQLExec(ctx, "demo", plugin.SQLRequest{SQL: "INSERT INTO items (name) VALUES (?)", Params: []any{"one"}}); result.Error != "" || result.LastInsertID != 1 { + t.Fatalf("insert result: %+v", result) + } + result := env.SQLQuery(ctx, "demo", plugin.SQLRequest{SQL: "SELECT id, name FROM items"}) + if result.Error != "" || len(result.Rows) != 1 || result.Rows[0][1] != "one" { + t.Fatalf("query result: %+v", result) + } + if result := env.SQLQuery(ctx, "demo", plugin.SQLRequest{SQL: "SELECT id FROM items WHERE id = ?", Params: []any{int64(99)}}); result.Error != "" || result.Rows == nil || len(result.Rows) != 0 { + t.Fatalf("empty query result: %+v", result) + } + if result := env.SQLQuery(ctx, "other", plugin.SQLRequest{SQL: "SELECT name FROM items"}); result.Error == "" || !strings.Contains(result.Error, "no such table") { + t.Fatalf("expected each plugin to get its own database: %+v", result) + } +} + +// The limits a plugin can hit are shared with Owncast rather than reimplemented +// here, so a plugin that passes its scenarios meets the same limits in +// production. This pins the behaviours that come from plugin.SQLRunner. +func TestMockHostSQLSharesTheHostLimits(t *testing.T) { + mock := NewMockHost() + defer mock.closeSQL() + env := mock.HostEnv() + ctx := context.Background() + + // exec is one transaction: a failing statement discards the whole batch. + if result := env.SQLExec(ctx, "demo", plugin.SQLRequest{SQL: "CREATE TABLE items (n INTEGER); INSERT INTO items VALUES (1); INSERT INTO missing VALUES (1)"}); result.Error == "" { + t.Fatal("expected the failing statement to roll back the batch") + } + if result := env.SQLQuery(ctx, "demo", plugin.SQLRequest{SQL: "SELECT count(*) FROM items"}); result.Error == "" { + t.Fatal("expected the rolled-back table to be gone") + } + + if result := env.SQLExec(ctx, "demo", plugin.SQLRequest{SQL: "CREATE TABLE items (n INTEGER)"}); result.Error != "" { + t.Fatal(result.Error) + } + seed := "WITH RECURSIVE c(i) AS (SELECT 0 UNION ALL SELECT i + 1 FROM c WHERE i < 10050) INSERT INTO items SELECT i FROM c" + if result := env.SQLExec(ctx, "demo", plugin.SQLRequest{SQL: seed}); result.Error != "" { + t.Fatal(result.Error) + } + + // An unbounded query that overruns the row cap is an error, not a silently + // short result. + unbounded := env.SQLQuery(ctx, "demo", plugin.SQLRequest{SQL: "SELECT n FROM items"}) + if unbounded.Error == "" || !strings.Contains(unbounded.Error, "add a LIMIT") { + t.Fatalf("expected the row cap to reject an unbounded read, got %d rows and error %q", len(unbounded.Rows), unbounded.Error) + } + + // A caller-supplied MaxRows is intent: the host stops there and says so, + // which is how queryRow reads one row out of a large table. + bounded := env.SQLQuery(ctx, "demo", plugin.SQLRequest{SQL: "SELECT n FROM items ORDER BY n", MaxRows: 1}) + if bounded.Error != "" || len(bounded.Rows) != 1 || !bounded.Truncated { + t.Fatalf("expected one row and a truncation flag, got %+v", bounded) + } + + // A single oversized value is refused rather than handed to the plugin. + oversized := env.SQLQuery(ctx, "demo", plugin.SQLRequest{SQL: "SELECT hex(zeroblob(1048576))"}) + if oversized.Error == "" || !strings.Contains(oversized.Error, "too big") { + t.Fatalf("expected the per-value limit to reject the blob, got %q", oversized.Error) + } + + // Request validation is the same parser the production host uses, so a + // scenario catches a malformed request before Owncast does. + if _, err := plugin.ParseSQLRequest(`{"sql":"SELECT ?","params":[[1,2]]}`); err == nil { + t.Fatal("expected a nested parameter to be rejected") + } + req, err := plugin.ParseSQLRequest(`{"sql":"INSERT INTO items VALUES (?)","params":[1152921504606846977]}`) + if err != nil { + t.Fatal(err) + } + if result := env.SQLExec(ctx, "demo", req); result.Error != "" { + t.Fatal(result.Error) + } + stored := env.SQLQuery(ctx, "demo", plugin.SQLRequest{SQL: "SELECT typeof(n), n FROM items WHERE n > 1000000000000000000", MaxRows: 1}) + if stored.Error != "" || len(stored.Rows) != 1 { + t.Fatalf("expected the large integer back, got %+v", stored) + } + if stored.Rows[0][0] != "integer" || stored.Rows[0][1] != int64(1152921504606846977) { + t.Fatalf("large integer round-tripped as %v (%v)", stored.Rows[0][1], stored.Rows[0][0]) + } +} + +// A plugin error surfaces with the same text Owncast would report, so a scenario +// that asserts on it keeps passing in production. The pure-Go driver decorates +// SQLite's message and the store strips that decoration back off. +func TestMockHostSQLErrorsReadLikeProduction(t *testing.T) { + mock := NewMockHost() + defer mock.closeSQL() + env := mock.HostEnv() + + result := env.SQLQuery(context.Background(), "demo", plugin.SQLRequest{SQL: "SELECT * FROM nope"}) + if result.Error != "no such table: nope" { + t.Fatalf("error text is %q, want the undecorated SQLite message", result.Error) + } + + ctx := context.Background() + schema := "CREATE TABLE parents (id INTEGER PRIMARY KEY); CREATE TABLE children (value TEXT NOT NULL UNIQUE, parent_id INTEGER REFERENCES parents(id))" + if result := env.SQLExec(ctx, "demo", plugin.SQLRequest{SQL: schema}); result.Error != "" { + t.Fatal(result.Error) + } + if result := env.SQLExec(ctx, "demo", plugin.SQLRequest{SQL: "INSERT INTO children VALUES ('taken', NULL)"}); result.Error != "" { + t.Fatal(result.Error) + } + for statement, want := range map[string]string{ + "INSERT INTO children VALUES ('taken', NULL)": "UNIQUE constraint failed: children.value", + "INSERT INTO children VALUES (NULL, NULL)": "NOT NULL constraint failed: children.value", + "INSERT INTO children VALUES ('orphan', 99)": "FOREIGN KEY constraint failed", + } { + if result := env.SQLExec(ctx, "demo", plugin.SQLRequest{SQL: statement}); result.Error != want { + t.Errorf("%q returned %q, want %q", statement, result.Error, want) + } + } +} + +// Every statement Owncast refuses has to be refused here too, or a plugin's +// scenario tests are worthless: an author would watch a test pass and then hit a +// failure on a real server. The refusal comes from the shared check in +// plugins.ParseSQLRequest, which runs at the host-function boundary in every +// host, because the pure-Go driver here cannot install Owncast's SQLite +// authorizer. pluginhost has the matching test against that authorizer. +func TestMockHostSQLRefusesEverythingOwncastRefuses(t *testing.T) { + mock := NewMockHost() + defer mock.closeSQL() + env := mock.HostEnv() + ctx := context.Background() + + for _, statement := range plugin.DeniedSQLStatementExamples { + if _, err := plugin.ParseSQLRequest(sqlRequestJSON(statement)); err == nil { + t.Errorf("the host accepted %q, which Owncast refuses", statement) + } + } + + // The store still runs ordinary SQL, so the check is not simply refusing + // everything. + if result := env.SQLExec(ctx, "demo", plugin.SQLRequest{SQL: "CREATE TABLE items (v TEXT)"}); result.Error != "" { + t.Fatal(result.Error) + } +} + +func sqlRequestJSON(statement string) string { + encoded, err := json.Marshal(map[string]string{"sql": statement}) + if err != nil { + panic(err) + } + return string(encoded) +} diff --git a/host-runtime/sqlstore/sqlstore.go b/host-runtime/sqlstore/sqlstore.go new file mode 100644 index 0000000..c19cea6 --- /dev/null +++ b/host-runtime/sqlstore/sqlstore.go @@ -0,0 +1,158 @@ +// Package sqlstore gives this repo's host binaries the per-plugin SQLite +// databases the storage.sql host functions need, so a plugin author can develop +// and test against owncast.sql without a running Owncast. The scenario test +// runner, the localhost dev server, and the demo binary all wire the same store. +// +// It is deliberately not part of Owncast. Owncast uses the cgo +// mattn/go-sqlite3 driver, the same one its own datastore uses, and configures +// each connection with a SQLite authorizer. This package uses +// modernc.org/sqlite instead, because owncast-plugin-test and +// owncast-plugin-serve are cross-compiled for every release target with +// CGO_ENABLED=0 and a cgo driver cannot go in them. +// +// A different driver would normally mean a plugin could pass its scenario tests +// and still fail on a real server, which would make those tests worthless. The +// pieces that decide whether a statement is allowed, and what it costs, +// therefore live above the driver in the runtime both hosts share: +// plugins.SQLRunner applies the same request validation, parameter typing, call +// timeout, atomic-exec semantics, and row, value, and result limits, and +// plugins.DeniedSQLReason refuses the same statements Owncast's authorizer +// refuses. plugins.DeniedSQLStatementExamples is the fixture both repositories +// test against. The DSN below pins the same page size and database size cap. +// +// What the driver still owns is the last line of defence rather than the +// contract. Owncast's authorizer sees compiled statements, so it stays +// authoritative there, and nothing in this package is a security boundary. +package sqlstore + +import ( + "context" + "database/sql" + "fmt" + "regexp" + "sync" + + plugins "github.com/owncast/owncast/services/plugins" + sqlite "modernc.org/sqlite" + sqlite3 "modernc.org/sqlite/lib" +) + +const ( + driverName = "sqlite" + pageSize = 4096 + // maxBytes mirrors pluginhost's per-plugin database cap, so a plugin that + // outgrows its storage in development outgrows it in production too. + maxBytes = 128 << 20 +) + +// memoryDSN configures a private in-memory database like a production one. +// In-memory keeps a dev host stateless the way its key-value store already is, +// and keeps scenario runs hermetic. +var memoryDSN = fmt.Sprintf( + ":memory:?_pragma=page_size(%d)&_pragma=max_page_count(%d)&_pragma=foreign_keys(on)&_pragma=trusted_schema(off)&_pragma=temp_store(memory)", + pageSize, + maxBytes/pageSize, +) + +// driverNoise matches the decoration modernc.org/sqlite adds to SQLite's own +// messages ("SQL logic error: no such table: t (1)" or "string or blob too big +// (18)"). Stripping it leaves the message Owncast's driver reports, so a +// scenario asserting on an error sees the production text. +var driverNoise = regexp.MustCompile(`^(?:(?:SQL logic error|constraint failed): )?(.*) \(\d+\)$`) + +// Store holds one in-memory database per plugin. Plugins cannot see each +// other's tables, as in production. +type Store struct { + mu sync.Mutex + dbs map[string]*sql.DB +} + +// NewMemory returns a store whose databases live only as long as it does. +func NewMemory() *Store { + return &Store{dbs: make(map[string]*sql.DB)} +} + +// Exec runs one statement batch for a plugin as a single transaction. +func (s *Store) Exec(ctx context.Context, pluginName string, req plugins.SQLRequest) plugins.SQLExecResult { + runner, err := s.runner(pluginName) + if err != nil { + return plugins.SQLExecResult{Error: err.Error()} + } + result := runner.Exec(ctx, req) + result.Error = normalizeError(result.Error) + return result +} + +// Query runs one query for a plugin and returns a bounded result set. +func (s *Store) Query(ctx context.Context, pluginName string, req plugins.SQLRequest) plugins.SQLQueryResult { + runner, err := s.runner(pluginName) + if err != nil { + return plugins.SQLQueryResult{Error: err.Error()} + } + result := runner.Query(ctx, req) + result.Error = normalizeError(result.Error) + return result +} + +// Close drops every database the store opened. +func (s *Store) Close() { + s.mu.Lock() + dbs := s.dbs + s.dbs = nil + s.mu.Unlock() + for _, db := range dbs { + _ = db.Close() + } +} + +func (s *Store) runner(pluginName string) (plugins.SQLRunner, error) { + db, err := s.database(pluginName) + if err != nil { + return plugins.SQLRunner{}, err + } + return plugins.SQLRunner{DB: db}, nil +} + +func (s *Store) database(pluginName string) (*sql.DB, error) { + s.mu.Lock() + defer s.mu.Unlock() + if s.dbs == nil { + s.dbs = make(map[string]*sql.DB) + } + if db := s.dbs[pluginName]; db != nil { + return db, nil + } + db, err := sql.Open(driverName, memoryDSN) + if err != nil { + return nil, err + } + // One connection, matching production: statements serialize instead of + // contending, and an in-memory database exists only while a connection to + // it does. + db.SetMaxOpenConns(1) + db.SetMaxIdleConns(1) + if err := db.Ping(); err != nil { + _ = db.Close() + return nil, err + } + conn, err := db.Conn(context.Background()) + if err != nil { + _ = db.Close() + return nil, err + } + if _, err := sqlite.Limit(conn, sqlite3.SQLITE_LIMIT_LENGTH, plugins.MaxSQLValueBytes); err != nil { + _ = conn.Close() + _ = db.Close() + return nil, err + } + _ = conn.Close() + s.dbs[pluginName] = db + return db, nil +} + +func normalizeError(message string) string { + if match := driverNoise.FindStringSubmatch(message); match != nil { + return match[1] + } + return message +} diff --git a/host-runtime/sqlstore/sqlstore_test.go b/host-runtime/sqlstore/sqlstore_test.go new file mode 100644 index 0000000..035ece4 --- /dev/null +++ b/host-runtime/sqlstore/sqlstore_test.go @@ -0,0 +1,23 @@ +package sqlstore + +import ( + "context" + "strings" + "testing" + + plugins "github.com/owncast/owncast/services/plugins" +) + +func TestStoreMatchesProductionValueLimit(t *testing.T) { + store := NewMemory() + t.Cleanup(store.Close) + ctx := context.Background() + + if result := store.Exec(ctx, "plugin", plugins.SQLRequest{SQL: "CREATE TABLE items (value BLOB)"}); result.Error != "" { + t.Fatal(result.Error) + } + result := store.Exec(ctx, "plugin", plugins.SQLRequest{SQL: "INSERT INTO items VALUES (zeroblob(?))", Params: []any{int64(plugins.MaxSQLValueBytes + 1)}}) + if result.Error == "" || !strings.Contains(result.Error, "too big") { + t.Fatalf("oversized value returned %q, want SQLite size-limit error", result.Error) + } +} diff --git a/sdks/js/create-owncast-plugin/template/.agents/skills/create-owncast-plugin-js/SKILL.md b/sdks/js/create-owncast-plugin/template/.agents/skills/create-owncast-plugin-js/SKILL.md index ef46655..717ea1c 100644 --- a/sdks/js/create-owncast-plugin/template/.agents/skills/create-owncast-plugin-js/SKILL.md +++ b/sdks/js/create-owncast-plugin/template/.agents/skills/create-owncast-plugin-js/SKILL.md @@ -188,6 +188,7 @@ combine for richer plugins. | Read live stream state (title, viewers, uptime) | (any) | `owncast.stream.current()` | `server.read` | | Read server info / socials / emotes / tags | (any) | `owncast.server.*()` | `server.read` | | Store per-user or persistent state | (any) | `owncast.kv.get/set` (+ `getJSON/setJSON`)| `storage.kv` | +| Rank or aggregate data (private SQL database) | (any) | `owncast.sql.exec/query/queryRow` | `storage.sql` | | Expose admin-configurable settings | (read at runtime) | `owncast.config.get(key, fallback?)` | — (declare under `config` in manifest) | | Call an external API / webhook | (any) | `owncast.http.fetch(url, opts?)` | `network.fetch` + `network.allowedHosts` | | Do delayed / periodic work | `onTick({now})` or `owncast.timer.*` | `owncast.timer.setTimeout/setInterval` | — (ambient) | @@ -259,6 +260,12 @@ Important shape/behavior notes: failures auto-disable the plugin for the session. - **`Date`/`Date.now()` work**, but there is no global `setTimeout`. Use `owncast.timer.*`. Timers don't survive a host restart. +- **`owncast.sql.*` is one private SQLite database per plugin.** Each `exec` is + a single atomic transaction, so a multi-statement schema setup either lands + whole or not at all. An unbounded `query` that overruns the host's row or + result budget is an error, not a truncated result, so write a `LIMIT`, or use + `queryRow` when you only need one row. Plugin databases are not included in + Owncast's backups. Worked example: `examples/js/chat-leaderboard`. - **Chat commands:** declare a `commands` table in `definePlugin`. Command tables support prefixes, aliases, parsed arguments, moderator gating, and per-user cooldowns. Unknown and gated invocations are silent. diff --git a/sdks/js/create-owncast-plugin/template/AGENTS.md b/sdks/js/create-owncast-plugin/template/AGENTS.md index 63b3d22..5ee6f97 100644 --- a/sdks/js/create-owncast-plugin/template/AGENTS.md +++ b/sdks/js/create-owncast-plugin/template/AGENTS.md @@ -65,6 +65,7 @@ the plugin. Admins judge trust by the declared list, so don't over-declare. | React to stream live / stop | `onStreamStarted` / `onStreamStopped` | — | — | | Read live stream / server state | (any) | `owncast.stream.current()` / `server.*()` | `server.read` | | Store state | (any) | `owncast.kv.get/set` (+ `getJSON/setJSON`)| `storage.kv` | +| Rank or aggregate data (private SQL database) | (any) | `owncast.sql.exec/query/queryRow` | `storage.sql` | | Admin-configurable settings | (read at runtime) | `owncast.config.get(key, fallback?)` | — (declare under `config`) | | Call an external API | (any) | `owncast.http.fetch(url, opts?)` | `network.fetch` + `network.allowedHosts` | | Delayed / periodic work | `onTick({now})` / `owncast.timer.*` | `owncast.timer.setTimeout/setInterval` | — (ambient, no global setTimeout) | @@ -89,6 +90,7 @@ the plugin. Admins judge trust by the declared list, so don't over-declare. - **Chat text is HTML-escaped on display.** `chat.send`/`sendAction` take plain text. Only `chat.system(body)` renders HTML, so escape untrusted content yourself. - **Filters are time-capped (50 ms) and fail open.** Keep `filterChatMessage` fast. An erroring filter is treated as `filter.pass()`. Five consecutive failures auto-disable the plugin for the session. - **No global `setTimeout`.** Use `owncast.timer.*` (timers don't survive a host restart). `Date`/`Date.now()` work normally. +- **`owncast.sql.*` is one private SQLite database per plugin.** Each `exec` is a single atomic transaction, so a multi-statement schema setup lands whole or not at all. An unbounded `query` that overruns the host's row or result budget is an error, not a truncated result: write a `LIMIT`, or use `queryRow` for a single row. Plugin databases are not included in Owncast's backups. The SDK's `examples/js/chat-leaderboard` is a worked example. - **`network.fetch` requires `network.allowedHosts`** in the manifest, e.g. `"network": { "allowedHosts": ["api.example.com"] }`. The bare `"*"` is allowed but must be explicit. - **Any UI field** (`actions`, `styles`, `scripts`, `extraPageContent`, `tabs`) **requires `ui.modify`**. - **Declare chat commands in `definePlugin({ commands: {...} })`.** Command tables support aliases, moderator gating, and per-user cooldowns. Unknown and gated invocations are silent. diff --git a/sdks/js/index.d.ts b/sdks/js/index.d.ts index a0fec52..982f714 100644 --- a/sdks/js/index.d.ts +++ b/sdks/js/index.d.ts @@ -173,6 +173,7 @@ export const Permissions: { readonly StorageKV: "storage.kv"; readonly StorageUpload: "storage.upload"; readonly StorageFS: "storage.fs"; + readonly StorageSQL: "storage.sql"; readonly EventsEmit: "events.emit"; readonly NetworkFetch: "network.fetch"; readonly HttpServe: "http.serve"; @@ -277,13 +278,38 @@ export interface UploadResult { url: string; } -/** Result of a mutating owncast.fs call (write/delete). `ok` is false and - * `error` is set when the host rejected the operation. */ +/** Result of a mutating owncast.fs call (write/delete). An empty object means + * success. `error` is set when the host rejected the operation. */ export interface FsResult { - ok: boolean; error?: string; } +/** A value a plugin can bind to a statement parameter, or read back out of a + * column. Blobs arrive base64-encoded as strings. */ +export type SQLValue = null | boolean | number | string; + +/** Result of `owncast.sql.exec`. Absence of `error` means success. Both + * counters are SQLite 64-bit integers, so they lose precision in JavaScript + * above `Number.MAX_SAFE_INTEGER`. */ +export interface SQLExecResult { + error?: string; + rowsAffected: number; + lastInsertId: number; +} + +/** One row as `owncast.sql.query` hands it back: column name to value. */ +export type SQLRow = Record; + +/** The host's raw query response, before the SDK keys rows by column name. + * Absence of `error` means success. `rows` holds values in `columns` order, + * and `truncated` is set when more rows matched than the caller's row limit. */ +export interface SQLQueryResult { + error?: string; + columns: string[]; + rows: SQLValue[][]; + truncated?: boolean; +} + export const filter: { pass(): FilterResult; modify(payload: any): FilterResult; @@ -595,7 +621,7 @@ export const owncast: { storage: { upload(name: string, data: Uint8Array | string): UploadResult | null; }; - /** Private, sandboxed filesystem under data/plugin-data//. The bytes + /** Private, sandboxed filesystem under data/plugin-storage//files/. The bytes * stay server-side (never served over HTTP) and the host confines every * path to this plugin's own directory. All methods require `storage.fs`. */ fs: { @@ -612,6 +638,29 @@ export const owncast: { /** Report whether a path exists inside the sandbox. */ exists(path: string): boolean; }; + /** Private SQLite database, one per plugin, stored in `db/` next to the + * `storage.fs` sandbox in `files/`, outside anything `owncast.fs.*` can name, + * and quota'd separately. Every call runs with a 2 second timeout. Absence + * of `error` means success. An error, missing response, or non-object + * response throws. JavaScript loses unsafe integers before `JSON.stringify` + * on writes and during `JSON.parse` on reads, so store values above + * `Number.MAX_SAFE_INTEGER` (2^53 - 1) as TEXT when they must remain exact. + * Requires `storage.sql`. */ + sql: { + /** Execute one statement batch as a single transaction: it commits whole + * or leaves the database untouched. A transaction cannot stay open + * across calls. */ + exec(sql: string, params?: SQLValue[]): SQLExecResult; + /** Query rows as objects keyed by column name. Alias duplicate columns. + * The result is never silently shortened: a query returning more than + * 10000 rows, or more than 1 MiB of encoded data, throws asking for a + * LIMIT. */ + query(sql: string, params?: SQLValue[]): SQLRow[]; + /** Return the first matching row, or null. Only that row is read back, so + * this stays under the result budget on a table `query` is too big + * for. */ + queryRow(sql: string, params?: SQLValue[]): SQLRow | null; + }; /** Post to the fediverse on the streamer's behalf. Requires `fediverse.post`, * which is high-trust (posts go out under the streamer's own handle), so * admins should grant it sparingly. */ @@ -720,7 +769,8 @@ export const owncast: { tags(): string[]; }; /** Read/change video/transcoding configuration. read() requires - * `videoconfig.read`, and write() requires `videoconfig.write`. */ + * `videoconfig.read`. write() requires `videoconfig.write` and throws when + * the host rejects the update or does not return an operation result. */ videoConfig: { read(): VideoConfig; write(config: VideoConfigUpdate): void; diff --git a/sdks/js/index.js b/sdks/js/index.js index c4beeaa..f9f89b6 100644 --- a/sdks/js/index.js +++ b/sdks/js/index.js @@ -65,6 +65,7 @@ const Permissions = Object.freeze({ StorageKV: "storage.kv", StorageUpload: "storage.upload", StorageFS: "storage.fs", + StorageSQL: "storage.sql", EventsEmit: "events.emit", NetworkFetch: "network.fetch", HttpServe: "http.serve", @@ -392,6 +393,53 @@ function hostFns(name, perm) { return fns; } +function operationResult(offset, failureMessage) { + if (offset == 0) return { error: failureMessage }; + try { + const result = JSON.parse(Memory.find(offset).readString()); + if (result === null || typeof result !== "object" || Array.isArray(result)) { + return { error: failureMessage }; + } + return result; + } catch { + return { error: failureMessage }; + } +} + +function requireOperationResult(offset, failureMessage) { + const result = operationResult(offset, failureMessage); + if (Object.prototype.hasOwnProperty.call(result, "error")) { + throw new Error(result.error || failureMessage); + } + return result; +} + +function sqlResult(offset) { + return requireOperationResult(offset, "SQL host call failed"); +} + +function sqlRows(result) { + if (!Array.isArray(result.columns) || !Array.isArray(result.rows)) { + throw new Error("SQL host returned an invalid result"); + } + return result.rows.map((values) => { + if (!Array.isArray(values)) { + throw new Error("SQL host returned an invalid result"); + } + return Object.fromEntries(result.columns.map((column, i) => [column, values[i]])); + }); +} + +// sqlQuery issues one query request. maxRows is deliberately not an author +// parameter: it exists so queryRow can ask the host for a single row. +function sqlQuery(sql, params, maxRows) { + const fns = hostFns("owncast_sql_query", Permissions.StorageSQL); + const payload = { sql: String(sql), params: Array.from(params || []) }; + if (maxRows) payload.maxRows = maxRows; + const request = Memory.fromString(JSON.stringify(payload)); + return sqlResult(fns.owncast_sql_query(request.offset)); +} + const owncast = { chat: { send(text) { @@ -537,7 +585,7 @@ const owncast = { return JSON.parse(Memory.find(offset).readString()); }, }, - // Private, sandboxed filesystem under data/plugin-data//. Unlike + // Private, sandboxed filesystem under data/plugin-storage//files/. Unlike // storage.upload (which publishes browser-accessible files), these bytes // stay server-side. The host confines every path to this plugin's own // directory. All methods require the 'storage.fs' permission. @@ -559,7 +607,7 @@ const owncast = { return Memory.find(offset).readString(); }, // Write bytes (Uint8Array) or a string to a file, creating parent - // directories as needed. Returns { ok, error? }. + // directories as needed. Returns { error? }. write(path, data) { const fns = hostFns("owncast_fs_write", Permissions.StorageFS); const dataMem = @@ -575,8 +623,7 @@ const owncast = { Memory.fromString(path).offset, dataMem.offset, ); - if (offset == 0) return { ok: false, error: "write failed" }; - return JSON.parse(Memory.find(offset).readString()); + return operationResult(offset, "write failed"); }, // List the entry names (files and subdirectories) directly inside dir. // A missing directory lists as empty. Returns string[]. @@ -586,12 +633,11 @@ const owncast = { if (offset == 0) return []; return JSON.parse(Memory.find(offset).readString()); }, - // Remove a single file or empty directory. Returns { ok, error? }. + // Remove a single file or empty directory. Returns { error? }. delete(path) { const fns = hostFns("owncast_fs_delete", Permissions.StorageFS); const offset = fns.owncast_fs_delete(Memory.fromString(path).offset); - if (offset == 0) return { ok: false, error: "delete failed" }; - return JSON.parse(Memory.find(offset).readString()); + return operationResult(offset, "delete failed"); }, // Report whether a path exists inside the sandbox. Returns boolean. exists(path) { @@ -599,6 +645,23 @@ const owncast = { return fns.owncast_fs_exists(Memory.fromString(path).offset) === 1; }, }, + sql: { + exec(sql, params = []) { + const fns = hostFns("owncast_sql_exec", Permissions.StorageSQL); + const request = Memory.fromString( + JSON.stringify({ sql: String(sql), params: Array.from(params || []) }), + ); + return sqlResult(fns.owncast_sql_exec(request.offset)); + }, + query(sql, params = []) { + return sqlRows(sqlQuery(sql, params)); + }, + queryRow(sql, params = []) { + // Asking the host for one row keeps a first-row read off the result + // budget, so this works against a table `query` would be too big for. + return sqlRows(sqlQuery(sql, params, 1))[0] || null; + }, + }, fediverse: { /** Publish a public text-only post to the fediverse on the streamer's * behalf. Returns { url } on success, null on failure (rate-limited, @@ -692,10 +755,7 @@ const owncast = { const offset = fns.owncast_video_config_write( Memory.fromString(JSON.stringify(config || {})).offset, ); - if (offset == 0) throw new Error("videoConfig.write failed"); - const result = JSON.parse(Memory.find(offset).readString()); - if (!result.ok) - throw new Error(result.error || "videoConfig.write failed"); + requireOperationResult(offset, "videoConfig.write failed"); }, }, kv: { diff --git a/sdks/python/owncast_plugin/__init__.py b/sdks/python/owncast_plugin/__init__.py index 8651e5e..f82eb45 100644 --- a/sdks/python/owncast_plugin/__init__.py +++ b/sdks/python/owncast_plugin/__init__.py @@ -448,6 +448,11 @@ def upload(self, name, data): return _call_json("owncast_storage_upload", str(name), str(data)) +def _operation_result(name, failure_message, *args): + result = _call_json(name, *args) + return result if isinstance(result, dict) else {"error": failure_message} + + class _FS: def read_text(self, path): return _host("owncast_fs_read")(str(path)) or None @@ -457,18 +462,72 @@ def read_text(self, path): def write(self, path, data): if isinstance(data, (bytes, bytearray)): data = data.decode("utf-8", "replace") - return _call_json("owncast_fs_write", str(path), str(data)) + return _operation_result( + "owncast_fs_write", "write failed", str(path), str(data) + ) def list(self, directory): return _call_json("owncast_fs_list", str(directory)) or [] def delete(self, path): - return _call_json("owncast_fs_delete", str(path)) + return _operation_result("owncast_fs_delete", "delete failed", str(path)) def exists(self, path): return bool(_host("owncast_fs_exists")(str(path))) +class _SQL: + """Private SQLite database, one per plugin (permission: storage.sql). It + lives in db/ next to the storage.fs sandbox in files/, outside anything + owncast.fs.* can name, and has its own quota. + A result without an error field is successful. An error, missing response, + or non-dict response raises RuntimeError. Integral parameters bind as SQLite + INTEGERs exactly, including values past 2**53.""" + + def _request(self, sql, params, max_rows=0): + request = {"sql": str(sql), "params": list(params or [])} + # max_rows is not an author parameter: query_row uses it to ask the + # host for a single row. + if max_rows: + request["maxRows"] = int(max_rows) + return json.dumps(request) + + def _result(self, name, sql, params, max_rows=0): + result = _operation_result( + name, "SQL host call failed", self._request(sql, params, max_rows) + ) + if "error" in result: + raise RuntimeError(result.get("error") or "SQL host call failed") + return result + + def _rows(self, result): + columns = result.get("columns") + rows = result.get("rows") + if not isinstance(columns, list) or not isinstance(rows, list): + raise RuntimeError("SQL host returned an invalid result") + if any(not isinstance(row, list) for row in rows): + raise RuntimeError("SQL host returned an invalid result") + return [dict(zip(columns, row)) for row in rows] + + def exec(self, sql, params=None): + """Run one statement batch as a single transaction, committed whole or + not at all, and return its result dict. Raises RuntimeError on error. + A call has 2 seconds to finish.""" + return self._result("owncast_sql_exec", sql, params) + + def query(self, sql, params=None): + """Return matching rows as dicts keyed by column name. The result is + never silently shortened: over 10000 rows, or over 1 MiB of encoded + data, raises RuntimeError asking for a LIMIT.""" + return self._rows(self._result("owncast_sql_query", sql, params)) + + def query_row(self, sql, params=None): + """Return the first matching row as a dict, or None. Only that row is + read back, so this works on a table query() is too big for.""" + rows = self._rows(self._result("owncast_sql_query", sql, params, 1)) + return rows[0] if rows else None + + class _Events: def emit(self, event_type, payload): _host("owncast_emit_event")(str(event_type), json.dumps(payload)) @@ -504,7 +563,15 @@ def read(self): return _wrap(_call_json("owncast_video_config_read")) def write(self, config): - return _call_json("owncast_video_config_write", json.dumps(config)) + """Apply a partial config update. Raise RuntimeError when the host + rejects the update or does not return an operation result.""" + result = _operation_result( + "owncast_video_config_write", + "video_config.write failed", + json.dumps(config), + ) + if "error" in result: + raise RuntimeError(result.get("error") or "video_config.write failed") class _Notifications: @@ -682,6 +749,7 @@ class _Owncast: timer = _Timer() config = _Config() assets = _Assets() + sql = _SQL() owncast = _Owncast() diff --git a/sdks/python/owncast_plugin/template/.agents/skills/create-owncast-plugin-py/SKILL.md b/sdks/python/owncast_plugin/template/.agents/skills/create-owncast-plugin-py/SKILL.md index b2674ec..e97f49c 100644 --- a/sdks/python/owncast_plugin/template/.agents/skills/create-owncast-plugin-py/SKILL.md +++ b/sdks/python/owncast_plugin/template/.agents/skills/create-owncast-plugin-py/SKILL.md @@ -194,6 +194,7 @@ Several rows combine for richer plugins. | Read live stream state (title, viewers, uptime) | (any) | `owncast.stream.current()` | `server.read` | | Read server info / socials / emotes / tags | (any) | `owncast.server.*()` | `server.read` | | Store per-user or persistent state | (any) | `owncast.kv.get/set` (+ `get_json/set_json`) | `storage.kv` | +| Rank or aggregate data (private SQL database) | (any) | `owncast.sql.exec/query/query_row` | `storage.sql` | | Expose admin-configurable settings | (read at runtime) | `owncast.config.get(key, fallback=None)` | — (declare under `config` in manifest) | | Call an external API / webhook | (any) | `owncast.http.fetch(url, opts=None)` | `network.fetch` + `network.allowedHosts` | | Do delayed / periodic work | `@plugin.on_tick` or `owncast.timer.*` | `owncast.timer.set_timeout/set_interval` | — (ambient) | @@ -265,6 +266,12 @@ Important shape/behavior notes: - **`filter.pass_()` has a trailing underscore** (`pass` is a Python keyword). - **No `time.sleep` / threads for delays.** Use `owncast.timer.*`. Timers don't survive a host restart. +- **`owncast.sql.*` is one private SQLite database per plugin.** Each `exec` is + a single atomic transaction, so a multi-statement schema setup either lands + whole or not at all. An unbounded `query` that overruns the host's row or + result budget is an error, not a truncated result, so write a `LIMIT`, or use + `query_row` when you only need one row. Plugin databases are not included in + Owncast's backups. Worked example: `examples/python/chat-leaderboard`. - **Chat commands:** declare commands with `plugin.commands({...})`. Command tables support prefixes, aliases, parsed arguments, moderator gating, and per-user cooldowns. Unknown and gated invocations are silent. diff --git a/sdks/python/owncast_plugin/template/AGENTS.md b/sdks/python/owncast_plugin/template/AGENTS.md index 38f20ce..5de5c57 100644 --- a/sdks/python/owncast_plugin/template/AGENTS.md +++ b/sdks/python/owncast_plugin/template/AGENTS.md @@ -66,6 +66,7 @@ the plugin. Admins judge trust by the declared list, so don't over-declare. | React to stream live / stop | `@plugin.on_stream_started` / `_stopped` | — | — | | Read live stream / server state | (any) | `owncast.stream.current()` / `server.*()` | `server.read` | | Store state | (any) | `owncast.kv.get/set` (+ `get_json/set_json`) | `storage.kv` | +| Rank or aggregate data (private SQL database) | (any) | `owncast.sql.exec/query/query_row` | `storage.sql` | | Admin-configurable settings | (read at runtime) | `owncast.config.get(key, fallback=None)` | — (declare under `config`) | | Call an external API | (any) | `owncast.http.fetch(url, opts=None)` | `network.fetch` + `network.allowedHosts` | | Delayed / periodic work | `@plugin.on_tick` / `owncast.timer.*` | `owncast.timer.set_timeout/set_interval` | — (ambient) | @@ -99,6 +100,12 @@ the plugin. Admins judge trust by the declared list, so don't over-declare. - **`filter.pass_()` has a trailing underscore** because `pass` is a Python keyword. - **No `time.sleep` / threads for delays.** Use `owncast.timer.*` (timers don't survive a host restart). +- **`owncast.sql.*` is one private SQLite database per plugin.** Each `exec` is a + single atomic transaction, so a multi-statement schema setup lands whole or not + at all. An unbounded `query` that overruns the host's row or result budget is + an error, not a truncated result: write a `LIMIT`, or use `query_row` for a + single row. Plugin databases are not included in Owncast's backups. The SDK's + `examples/python/chat-leaderboard` is a worked example. - **`network.fetch` requires `network.allowedHosts`** in the manifest, e.g. `"network": { "allowedHosts": ["api.example.com"] }`. The bare `"*"` is allowed but must be explicit.