A real-time social simulation where LLM-powered agents autonomously inhabit scenes, interact with objects and each other, and develop persistent personalities, memories, and relationships.
You act as the "director" — describe agents and scenes in natural language, then watch emergent behavior unfold. Rewind the timeline to replay moments and inspect agent decisions.
- Autonomous Agents — LLM-driven decision-making with persistent personalities and memories
- Emergent Relationships — Trust, affection, respect, and familiarity develop through interactions
- Natural Language Creation — Describe agents and scenes; the LLM generates the details
- Object Interactions — Agents sit, examine, play, and use objects based on affordances
- Multi-Turn Conversations — Agents engage in contextual dialogue with emotional awareness
- Timeline Playback — Scrub through simulation history, replay moments, inspect decisions
- Session Persistence — Auto-saves to IndexedDB; restore sessions on page reload
- Local or Cloud LLM — Use Ollama locally or OpenRouter for cloud inference
# Install dependencies
npm install
# Copy environment template
cp .env.example .env.local
# Start development server
npm run devOpen http://localhost:5173 in your browser.
Choose one or both providers:
# Install Ollama: https://ollama.ai
ollama pull llama3.2
ollama serveConfigure in .env.local:
VITE_OLLAMA_BASE_URL=http://localhost:11434
VITE_OLLAMA_MODEL=llama3.2
VITE_DEFAULT_LLM_PROVIDER=local
Get an API key from OpenRouter and add to .env.local:
VITE_OPENROUTER_API_KEY=your-api-key-here
VITE_DEFAULT_LLM_PROVIDER=cloud
Approximate token counts per request type:
| Request Type | System Prompt | User Prompt | Response | Total | Frequency |
|---|---|---|---|---|---|
| Ambient Decision | ~200 | 150-400 | 50-100 | 400-700 | Every 10s per agent |
| Dialogue Turn | ~300 | 200-500 | 50-100 | 550-900 | Per conversation |
| Agent Creation | ~500 | 20-50 | 300-400 | 800-1000 | Once per agent |
| Scene Creation | ~500 | 20-50 | 500-1000 | 1000-1500 | Once per scene |
Cost drivers: Ambient decisions run every ~10 seconds per agent. With 5 agents running for 1 hour:
- 5 agents × 6 decisions/min × 60 min = 1,800 decisions/hour
- At ~500 tokens each = ~900K tokens/hour
Estimated hourly cost (5 agents):
| Provider | Model | Cost/1M tokens | Hourly Cost |
|---|---|---|---|
| OpenRouter | Llama 3.1 8B | ~$0.06 | ~$0.05 |
| OpenRouter | Mistral 7B | ~$0.07 | ~$0.06 |
| Cloudflare | Workers AI (paid) | ~$0.01 | ~$0.01 |
Cloudflare Workers AI free tier (~10K tokens/day) is only suitable for demos or occasional agent creation.
| Command | Description |
|---|---|
/agent <description> |
Create an agent from a description |
/scene |
Create a default coffee shop scene |
/scene <description> |
Create a scene from a description |
/pause |
Pause the simulation |
/play |
Resume the simulation |
/clear |
Clear the current session |
/help |
Show available commands |
| Key | Action |
|---|---|
| Space | Play/pause simulation |
| Arrow Left | Skip back 10 seconds |
| Arrow Right | Skip forward 10 seconds |
[ |
Decrease playback speed |
] |
Increase playback speed |
| Layer | Technology |
|---|---|
| Language | TypeScript |
| Rendering | PixiJS 8 |
| Bundler | Vite |
| Testing | Vitest |
| State | Zustand |
| Persistence | IndexedDB (Dexie.js) |
| LLM (cloud) | OpenRouter |
| LLM (local) | Ollama |
npm run dev # Start development server
npm run build # Build for production
npm run preview # Preview production build
npm run test # Run tests in watch mode
npm run test:run # Run tests oncesrc/
├── engine/ # Simulation loop, events, state
├── agents/ # Agent model, behaviors, perception
├── rendering/ # PixiJS canvas, sprites, UI
├── scenes/ # Scene model, zones, objects
├── llm/ # LLM provider abstraction
├── services/ # Business logic services
├── commands/ # Command parsing and execution
├── navigation/ # A* pathfinding, steering behaviors
├── persistence/ # IndexedDB storage layer
├── store/ # Zustand global state
└── ui/ # UI components
- Command should be cleared once submitted
- Speech bubbles should always be on top
- approach movement does not follow normal move patterns / animations (they slide toward the other agent)
- Replay isn't showing conversation
- What are the yellow vs green in timeline view?
- Agents don't really "wrap up" conversation
- Agents aren't aware of their surrounding / location (indoor vs outdoor)
- Recent conversations aren't carrying over (related to relationships not storing)
- Don't include the original user prompt in future prompts to LLMs (it may not exactly align with the actual profile)
- Agents at top of screen can have head cut off (need some buffer space around the scene)
- Log should follow the latest when scroll to bottom, add a scroll to bottom button
- Show thought bubbles (configurable)
- Allow conversations / ambient / etc each to be toggled/configured independently
- Add an admin interface for each system (conversation, ambient thinking, relationships, etc)
- Show interactions in timeline view
- Agents should be less likely to start another conversation with the same person/group until some time has passed.
- Github Pages hosting with user-provided API keys
- Object Inspector - view/edit scene/agent/object details
- When talking, should show who they are talking to in agents list
- Interaction filter for timeline view (see certain events)
- Timeline log with filtering capabilities
- Agent "known as" names for relationships (nicknames, first names based on introduction context)
- Additional activity animations for agents (sit, sleep, eat, read, write, sing, dance)
- Scene sprites and visual assets
- Fix glasses/beard/accessories rendering issues
- Implement slim/heavy body model types
- Agent seeding system for pre-existing relationships (i.e. /agent Bobby's little sister)
- Direct agent interaction (talk, influence, control)
- LLM response validation and sanity/hallucination checks (i.e. "Bobby picks up the house and eats it")
- Custom LLM models based on specific actions
- Model Context Protocol (MCP) integration for enhanced context gathering
- Line-of-sight for perception (agents shouldn't be able to see through walls)
- Need a way to distinguish between talk & actions (i.e. "Watch this! Bobby throws the ball in the air and catches it. Wasn't that awesome?")
- Design Document — Full architecture and design decisions
- MVP Implementation Plan — Step-by-step implementation guide
Sprite assets from the Liberated Pixel Cup project and the Universal LPC Spritesheet Character Generator, dual-licensed under CC-BY-SA 3.0 and GPL 3.0.
Base sprite artists: Stephen Challener (Redshrike), Lanea Zimmerman (Sharm), Manuel Riecke (MrBeast), Charles Sanchez (CharlesGabriel). Additional contributions from Johannes Sjölund (wulax), Matthew Krohn (makrohn), bluecarrot16, ElizaWy, and many others.
Code: PolyForm Noncommercial 1.0.0
Sprite assets: CC-BY-SA 3.0 (see Credits above)