Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/examples/fastapi-app.md
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,7 @@ rebar3 shell
hornbeam:start("app:app", #{
worker_class => asgi,
lifespan => on,
streaming => true, %% Required for the /stream SSE endpoint
pythonpath => ["fastapi_demo"],
workers => 4
}).
Expand Down
93 changes: 93 additions & 0 deletions docs/guides/asgi.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,29 @@ hornbeam:start("app:app", #{

## Streaming Responses

By default, Hornbeam buffers every ASGI HTTP response fully before sending it to the client. This works well for typical request/response endpoints, but blocks real-time use cases like Server-Sent Events (SSE) and LLM token streaming.

To enable true HTTP streaming, set `streaming => true` in your configuration:

```erlang
hornbeam:start("app:application", #{
worker_class => asgi,
streaming => true
}).
```

With `streaming => true`, Hornbeam automatically detects per-request whether to stream or buffer. After the ASGI app sends its first body chunk, Hornbeam checks the `more_body` flag:

- **`more_body: False`** — single-chunk response, uses efficient buffered reply (same as non-streaming mode)
- **`more_body: True`** — multi-chunk response, switches to chunked transfer encoding and sends each chunk to the client as it arrives

Non-streaming endpoints still work correctly through this path — they produce the same buffered response. The streaming path uses a single `py:call` that pushes chunks directly to the Erlang handler via `erlang.send()`, so overhead per chunk is minimal. However, it bypasses the worker pool's optimized NIF path, so you should only enable it on apps or mounts that actually need streaming. In multi-app mode, you can enable `streaming` on just the mounts that serve SSE or chunked responses.

### Basic Streaming Example

```python
import asyncio

async def application(scope, receive, send):
if scope['type'] == 'http':
await send({
Expand All @@ -195,6 +217,8 @@ async def application(scope, receive, send):
})
```

> **Note:** Without `streaming => true`, the above app still works — but the client won't see any output until all chunks have been buffered and sent as one response. With streaming enabled, each chunk is flushed to the client as soon as it's produced.

## Erlang-Native Async Primitives

Hornbeam 1.4.0 introduces native Erlang timer support for async operations, providing significant performance improvements for async Python code.
Expand Down Expand Up @@ -247,6 +271,59 @@ The ASGI runner includes several optimizations:

## Server-Sent Events (SSE)

SSE requires `streaming => true` so that events are flushed to the client in real time.

### Raw ASGI SSE

```python
import asyncio

async def application(scope, receive, send):
if scope['type'] != 'http':
return

path = scope.get('path', '/')

if path == '/events':
await send({
'type': 'http.response.start',
'status': 200,
'headers': [
[b'content-type', b'text/event-stream'],
[b'cache-control', b'no-cache'],
],
})
for i in range(10):
chunk = f'data: Event {i}\n\n'.encode()
await send({
'type': 'http.response.body',
'body': chunk,
'more_body': i < 9,
})
if i < 9:
await asyncio.sleep(1)
else:
body = b'Hello!\n'
await send({
'type': 'http.response.start',
'status': 200,
'headers': [[b'content-type', b'text/plain']],
})
await send({
'type': 'http.response.body',
'body': body,
})
```

```erlang
hornbeam:start("app:application", #{
worker_class => asgi,
streaming => true %% Required for SSE
}).
```

### FastAPI SSE

```python
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
Expand All @@ -267,6 +344,21 @@ async def sse():
)
```

```erlang
hornbeam:start("app:app", #{
worker_class => asgi,
streaming => true, %% Required for SSE
lifespan => on
}).
```

Test with curl:

```bash
curl -N http://localhost:8000/events
# Events appear one per second as they're produced
```

## Request Body

Read request body asynchronously:
Expand Down Expand Up @@ -361,6 +453,7 @@ hornbeam:start("app:app", #{
%% Protocol
worker_class => asgi,
lifespan => auto,
streaming => true, %% Enable HTTP streaming (SSE, chunked responses)
root_path => "",

%% Workers
Expand Down
6 changes: 4 additions & 2 deletions docs/guides/multi-app.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ hornbeam:start(#{
| `worker_class` | atom | `wsgi` | Protocol: `wsgi` or `asgi` |
| `workers` | integer | `4` | Number of Python workers |
| `timeout` | integer | `30000` | Request timeout in ms |
| `streaming` | boolean | `false` | Enable HTTP streaming (ASGI only) |

## Global Options

Expand Down Expand Up @@ -182,10 +183,11 @@ A common pattern is mixing sync (WSGI) and async (ASGI) apps:
```erlang
hornbeam:start(#{
mounts => [
%% FastAPI for real-time API
%% FastAPI for real-time API (with SSE streaming)
{"/api/v2", "api_v2:app", #{
worker_class => asgi,
workers => 8
workers => 8,
streaming => true %% Enable SSE/streaming for this mount
}},

%% Legacy Flask API
Expand Down
19 changes: 18 additions & 1 deletion docs/reference/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ Each mount is a tuple: `{Prefix, AppSpec, Options}`
| `worker_class` | atom | `wsgi` | Protocol: `wsgi` or `asgi` |
| `workers` | integer | `4` | Number of Python workers for this mount |
| `timeout` | integer | `30000` | Request timeout in ms |
| `streaming` | boolean | `false` | Enable HTTP streaming for this mount (ASGI only) |

### Multi-App Example

Expand All @@ -61,7 +62,8 @@ hornbeam:start(#{
{"/api/v2", "api_v2:app", #{
worker_class => asgi,
workers => 8,
timeout => 60000
timeout => 60000,
streaming => true %% Enable SSE/streaming for this mount
}},
{"/api/v1", "api_v1:app", #{
worker_class => wsgi,
Expand Down Expand Up @@ -136,8 +138,23 @@ hornbeam:start("app:app", #{
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `lifespan` | atom | `auto` | Lifespan handling: `auto`, `on`, `off` |
| `streaming` | boolean | `false` | Enable HTTP streaming (SSE, chunked responses) |
| `root_path` | binary | `""` | ASGI root_path for mounted apps |

### Streaming

When `streaming` is `true`, Hornbeam uses a streaming-capable handler path for ASGI HTTP requests. Each request is auto-detected: single-chunk responses still produce a correct buffered reply, while multi-chunk responses (`more_body: True`) are streamed to the client in real time via chunked transfer encoding.

This is required for Server-Sent Events, LLM token streaming, and any endpoint that produces output incrementally. Enabling streaming is a non-breaking change — all endpoints continue to work correctly. The streaming path bypasses the worker pool's optimized NIF path, so only enable it on apps or mounts that need it. In multi-app mode, use per-mount `streaming` to scope this to specific mounts.

```erlang
hornbeam:start("app:app", #{
worker_class => asgi,
streaming => true,
timeout => 60000
}).
```

### Lifespan Values

- `auto` - Detect if app supports lifespan, use if available
Expand Down
Loading