Skip to content

[ML-896] docs: align docs with implementation #72

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kxzk wants to merge 1 commit into
base: main
Choose a base branch
from
+87 −10
Open

[ML-896] docs: align docs with implementation #72

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 80 additions & 3 deletions docs/API_REFERENCE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ Block receives a configuration object with these properties:
| `cache_max_size` | Integer | No | `1000` | Max cached prompts |
| `cache_backend` | Symbol | No | `:memory` | `:memory` or `:rails` |
| `cache_lock_timeout` | Integer | No | `10` | Lock timeout (seconds) |
| `cache_stale_while_revalidate` | Boolean | No | `false` | Enable SWR (requires stale TTL) |
| `cache_stale_ttl` | Integer | No | `0` | Stale TTL (seconds, >0 enables) |
| `cache_stale_while_revalidate` | Boolean | No | `false` | Advisory SWR flag (effective activation depends on `cache_stale_ttl`) |
| `cache_stale_ttl` | Integer, Symbol (`:indefinite`) | No | `0` | Stale TTL (seconds, `>0` enables SWR) |
| `cache_refresh_threads` | Integer | No | `5` | Background refresh threads |
| `batch_size` | Integer | No | `50` | Score + trace export batch size |
| `flush_interval` | Integer | No | `10` | Score + trace export interval (s) |
Expand Down Expand Up @@ -149,7 +149,7 @@ get_prompt(name, version: nil, label: nil, fallback: nil, type: nil)
| `name` | String | Yes | Prompt name |
| `version` | Integer | No | Specific version (mutually exclusive with `label`) |
| `label` | String | No | Version label (e.g., "production") |
| `fallback` | String | No | Fallback template if not found |
| `fallback` | String, Array<Hash> | No | Fallback prompt if not found (`String` for text, `Array<Hash>` for chat) |
| `type` | Symbol | Conditional | `:text` or `:chat` (required if `fallback` provided) |

**Returns:** `TextPromptClient` or `ChatPromptClient`
Expand Down Expand Up @@ -246,6 +246,83 @@ prompts.each do |p|
end
```

### `Client#create_prompt`

Create a new prompt or a new version of an existing prompt.

**Signature:**

```ruby
create_prompt(name:, prompt:, type:, config: {}, labels: [], tags: [], commit_message: nil)
```

**Parameters:**

| Parameter | Type | Required | Description |
| ---------------- | ------------------ | -------- | ----------- |
| `name` | String | Yes | Prompt name |
| `prompt` | String, Array<Hash> | Yes | Prompt body (`String` for text, `Array<Hash>` for chat) |
| `type` | Symbol | Yes | `:text` or `:chat` |
| `config` | Hash | No | Prompt config metadata (model params, etc.) |
| `labels` | Array<String> | No | Version labels |
| `tags` | Array<String> | No | Prompt tags |
| `commit_message` | String | No | Optional commit message |

**Returns:** `TextPromptClient` or `ChatPromptClient`

**Raises:**

- `ArgumentError` for invalid type/content
- `UnauthorizedError` if authentication fails
- `ApiError` for other API errors

**Example:**

```ruby
prompt = client.create_prompt(
name: "greeting",
prompt: "Hello {{name}}!",
type: :text,
labels: ["production"]
)
```

### `Client#update_prompt`

Update labels on an existing prompt version.

**Signature:**

```ruby
update_prompt(name:, version:, labels:)
```

**Parameters:**

| Parameter | Type | Required | Description |
| --------- | ------------- | -------- | ----------- |
| `name` | String | Yes | Prompt name |
| `version` | Integer | Yes | Version to update |
| `labels` | Array<String> | Yes | Replacement label set |

**Returns:** `TextPromptClient` or `ChatPromptClient`

**Raises:**

- `NotFoundError` if the prompt version does not exist
- `UnauthorizedError` if authentication fails
- `ApiError` for other API errors

Comment on lines +310 to +315
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Client#update_prompt can raise ArgumentError when labels is not an Array (enforced in ApiClient#update_prompt). Please document this in the Raises section so callers know what to expect for invalid inputs.

Copilot uses AI. Check for mistakes.
**Example:**

```ruby
client.update_prompt(
name: "greeting",
version: 3,
labels: ["production"]
)
```

### Cache Warming

Cache warming is handled by the `CacheWarmer` class, not directly on the Client.
Expand Down
14 changes: 7 additions & 7 deletions docs/CONFIGURATION.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,20 +121,20 @@ See [CACHING.md](CACHING.md#stampede-protection) for details.

- **Type:** Boolean
- **Default:** `false`
- **Description:** Enable stale-while-revalidate caching pattern
- **Description:** Advisory SWR intent flag (effective SWR behavior is controlled by `cache_stale_ttl`)

```ruby
config.cache_stale_while_revalidate = true # Enable SWR
config.cache_stale_while_revalidate = true # Optional intent flag
```

When enabled, serves stale cached data immediately while refreshing in the background. This dramatically reduces P99 latency by avoiding synchronous API waits after cache expiration.
This flag does not independently turn SWR on/off in runtime cache behavior. SWR is active when `cache_stale_ttl > 0`.

Comment on lines +124 to 131
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section clarifies SWR activation depends on cache_stale_ttl > 0, but later in this document the Production example still suggests cache_stale_while_revalidate = true “Enable SWR” without setting cache_stale_ttl. Please update the examples/comments below to match the new semantics (e.g., set cache_stale_ttl in the Production snippet or adjust the wording).

Copilot uses AI. Check for mistakes.
**Behavior:**

- `false` (default): Cache expires at TTL, next request waits for API (~100ms)
- `true`: After TTL, serves stale data instantly (~1ms) + refreshes in background
- `cache_stale_ttl <= 0` (default): Cache expires at TTL, next request waits for API (~100ms)
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs state cache_stale_ttl <= 0 disables SWR, but cache_stale_ttl is validated as non-negative in runtime config (negative values raise ConfigurationError). Consider documenting the disabled case as cache_stale_ttl == 0 (default) to avoid implying negative values are supported.

Suggested change
- `cache_stale_ttl <= 0` (default): Cache expires at TTL, next request waits for API (~100ms)
- `cache_stale_ttl == 0` (default): Cache expires at TTL, next request waits for API (~100ms)

Copilot uses AI. Check for mistakes.
- `cache_stale_ttl > 0`: After TTL, serves stale data instantly (~1ms) + refreshes in background

**Important:** SWR only activates when `cache_stale_ttl` is a positive value. Set it explicitly (typically equal to `cache_ttl`).
**Important:** Set `cache_stale_ttl` to a positive value (typically equal to `cache_ttl`) to activate SWR.

**Compatibility:**

Expand Down Expand Up @@ -435,7 +435,7 @@ Langfuse.configure do |config|
config.public_key = 'pk-lf-test'
config.secret_key = 'sk-lf-test'
config.cache_backend = :memory # Isolated per-process cache
config.cache_stale_while_revalidate = false # Disable SWR for predictable tests
config.cache_stale_ttl = 0 # Disable SWR for predictable tests
end
```

Expand Down
Loading