Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions integrations/lingbot/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,60 @@ Then open:
- [http://localhost:8089/request_session](http://localhost:8089/request_session)
- [http://localhost:8089/healthz](http://localhost:8089/healthz) (`runtime_ready` indicates preload completion)

### Preset assets

Launch with a local preset directory to replace the initial frame, base prompt,
and text-event catalog without changing the model config:

```bash
uv run --package flashdreams-lingbot python -m lingbot.webrtc.server \
--host 0.0.0.0 --port 8089 \
--config_name lingbot-world-v2-14b-causal-fast-taehv-window15-sink3 \
--preset-assets-dir \
integrations/lingbot/lingbot/webrtc/presets/golden-hour-portrait
```

Four ready-to-run presets are bundled under `lingbot/webrtc/presets/`:

- `golden-hour-portrait`: sunlit portrait with subtle character actions.
- `moonlit-portal`: fantasy ruins with portal, light, and weather events.
- `cozy-reading-room`: rainy library with fire, book, and rain events.
- `misty-dinosaur-valley`: prehistoric landscape with wildlife and mist events.

The Initial Scene panel displays the bundled presets as clickable thumbnails.
Selecting one replaces the frame, prompt, and event catalog for the next WebRTC
session while keeping the model server loaded. Use **Disconnect Session** before
selecting another preset and reconnecting.

The directory must contain:

```text
my-preset/
├── first_frame.png
├── prompt.txt
└── event_texts.json
```

`event_texts.json` accepts either a top-level list or an object containing an
`events` or `text_events` list. Each event uses the same schema advertised by
the WebRTC initial-scene payload:

```json
[
{
"event_id": "hair-tuck",
"label": "Hair Tuck",
"prompt": "The woman gently tucks a strand of light brown hair behind her ear.",
"category": "portrait"
}
]
```

When `--preset-assets-dir` is set, `--example-idx` is ignored and no bundled
example assets are downloaded. Camera intrinsics and poses continue to use the
existing defaults. The launch helper accepts the same directory through the
`PRESET_ASSETS_DIR` environment variable.

### Runtime requirements

- CUDA-capable GPU.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[
{
"event_id": "fire-brightens",
"label": "Fire Brightens",
"prompt": "The fireplace flares gently, casting warmer moving light and soft shadows across the reading room.",
"category": "interior"
},
{
"event_id": "pages-turn",
"label": "Pages Turn",
"prompt": "A faint breeze turns several pages of the open book on the table.",
"category": "interior"
},
{
"event_id": "rain-intensifies",
"label": "Rain Intensifies",
"prompt": "Rain begins falling more heavily against the window while droplets trail down the glass.",
"category": "weather"
}
]
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
An empty, cozy old-library reading room glows at night. A deep green armchair sits between a stone fireplace and a rain-streaked window, with an open book on a small wooden table nearby. Warm lamps, dark bookcases, soft curtains, and leafy houseplants create an intimate atmosphere, while amber firelight contrasts with the cool blue rain outside.
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[
{
"event_id": "hair-tuck",
"label": "Hair Tuck",
"prompt": "The woman gently tucks a strand of light brown hair behind her ear.",
"category": "portrait"
},
{
"event_id": "subtle-smile",
"label": "Subtle Smile",
"prompt": "A soft, warm smile gradually forms on her face as she looks out the window.",
"category": "portrait"
},
{
"event_id": "head-turn",
"label": "Head Turn",
"prompt": "She slowly turns her head from the window to look directly at the camera.",
"category": "portrait"
}
]
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
A side profile portrait of a young woman with long, golden-brown hair, who is standing bathed in the warm, direct light of the golden hour streaming from a nearby window. Her gaze is directed thoughtfully and serenely out the window, her eyes reflecting the sunlight as she observes the world outside. The powerful, natural light dramatically highlights the strands of her hair and the contours of her face, leaving soft shadows that emphasize her profile. She is wearing a simple, light-colored button-down shirt, adding to the gentle and reflective atmosphere of the moment. The background is a soft-focus interior filled with various potted green plants, which blur into the shadows and create a cozy, organic contrast to the bright light.
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[
{
"event_id": "dinosaur-raises-head",
"label": "Dinosaur Raises Head",
"prompt": "The long-necked dinosaur slowly raises its head above the mist and looks across the valley.",
"category": "wildlife"
},
{
"event_id": "flock-crosses-sky",
"label": "Flock Crosses Sky",
"prompt": "A flock of prehistoric flying reptiles glides across the pale dawn sky above the cliffs.",
"category": "wildlife"
},
{
"event_id": "mist-rolls-in",
"label": "Mist Rolls In",
"prompt": "A thicker bank of cool morning mist rolls along the river and through the fern-covered valley.",
"category": "weather"
}
]
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
A peaceful prehistoric valley stretches beneath the soft golden light of dawn. A long-necked herbivorous dinosaur stands beside a winding river among low banks of cool mist. Giant ferns frame the foreground, while layered mountains, forested cliffs, a tall waterfall, and distant flying reptiles give the natural-history scene a grand cinematic depth.
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[
{
"event_id": "portal-awakens",
"label": "Portal Awakens",
"prompt": "The ancient portal awakens as its cyan runes brighten and a swirling field of light forms within the stone ring.",
"category": "fantasy"
},
{
"event_id": "fireflies-gather",
"label": "Fireflies Gather",
"prompt": "A cloud of luminous fireflies rises from the ferns and spirals slowly around the portal.",
"category": "fantasy"
},
{
"event_id": "storm-approaches",
"label": "Storm Approaches",
"prompt": "Dark clouds gather above the ruins as distant lightning briefly illuminates the mountains.",
"category": "weather"
}
]
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
An ancient circular stone portal stands among moss-covered ruins at blue hour. Dim cyan runes trace the weathered ring while a wet stone path winds through dense ferns toward it. Moonlit mist drifts between broken columns, waterfalls, and distant mountains, creating a mysterious cinematic fantasy landscape with deep layers and realistic natural detail.
Loading
Loading