Skip to content

matthewmturner/pi-daemon

BranchesTags

Repository files navigation

pi-daemon

A persistent background daemon for pi agents. pi-daemon runs alongside your agent sessions and provides shared infrastructure that individual agents shouldn't have to manage themselves: collecting and exposing metrics, running scheduled jobs, and serving as a local coordination layer for the pi ecosystem.

What pi-daemon does

Today:

  • Accepts metric POST requests from agent extensions and exposes them as Prometheus metrics
  • Runs scheduled jobs — cron-driven prompts executed by a pi process independent of any active agent session

Coming soon:

  • RAG server — retrieval-augmented generation support for agent context and knowledge access

Quick Start

# Install dependencies
npm install

# Start the daemon in the foreground
npm run dev

# Or start in background
node --experimental-strip-types src/cli.ts start

# Check status
node --experimental-strip-types src/cli.ts status

# Stop
node --experimental-strip-types src/cli.ts stop

Configuration

Create ~/.pi/daemon/config.json or set PI_DAEMON_CONFIG environment variable.

{
  "server": {
    "listenAddress": "127.0.0.1",
    "port": 9876,
    "maxPayloadSize": 1048576
  },
  "metrics": {
    "enabled": true,
    "retentionMinutes": 60,
    "maxLabelCount": 10,
    "maxMetricNames": 1000
  },
  "plugins": {
    "metrics": true,
    "jobs": true
  },
  "jobs": {
    "enabled": true,
    "dbPath": "~/.pi/daemon/jobs.db",
    "runHistoryLimit": 100,
    "maxConcurrent": 1
  },
  "auth": {
    "enabled": false
  },
  "logging": {
    "level": "info"
  }
}

API Reference

pi-daemon exposes an HTTP API on 127.0.0.1:9876 by default.

Prometheus Exposition

GET /metrics

Returns metrics in Prometheus text format.

Response: 200 OK — text/plain; version=0.0.4; charset=utf-8

# HELP pi_compaction_tokens_before Counter metric: pi_compaction_tokens_before
# TYPE pi_compaction_tokens_before counter
pi_compaction_tokens_before{session="abc123",model="claude-sonnet-4"} 5000

Single Metric Ingestion

POST /api/metrics
Content-Type: application/json

{
  "name": "pi_compaction_tokens_before",
  "type": "counter",
  "labels": { "session": "abc123", "model": "claude-sonnet-4" },
  "increment": 5000,
  "timestamp": 1700000000000
}

Auth: Required when auth.enabled: true (Bearer token in Authorization header). Response: 204 No Content on success, 400 Bad Request on validation error.

Batch Ingestion

POST /api/metrics/batch
Content-Type: application/json

{
  "requestId": "uuid-here",
  "metrics": [
    { "name": "metric_a", "type": "counter", "increment": 1 },
    { "name": "metric_b", "type": "gauge", "set": 42 }
  ]
}

The optional requestId enables idempotency — if the same ID is sent within 5 minutes, the batch is rejected with 409 Conflict.

Response: 200 OK

{ "success": 2, "failed": [] }

Metric Types

Type Field Description
counter increment (number, >= 0) Increment by value. Defaults to 1.
gauge set (number) Set to this absolute value.
gauge delta (number) Add this amount (can be negative).
histogram observe (number) Record an observation.
summary observe (number) Record an observation.

Metric names must be valid Prometheus metric names. Label keys must match [a-zA-Z0-9_]+.

Health & Admin

Endpoint Description
GET /health Health check ({"status": "ok", "uptime": 12345.67, "version": "0.1.0"})
GET /config Current config (auth token redacted)
POST /admin/reload Reload configuration ({"status": "reloaded"})
GET /admin/stats Runtime statistics (uptime, memory, pid, node version)
GET /api/metrics/registry List registered metric names

Scheduled Jobs

The jobs plugin runs cron-scheduled prompts by spawning a pi process (overridable via the PI_BIN environment variable) and persists job definitions and run history to a SQLite database. Job names are unique; only one run of a given job is in flight at a time — overlapping triggers are skipped.

Job Config

Key Default Description
plugins.jobs true Master switch for the jobs plugin
jobs.enabled true Enables the scheduler and DB (can be toggled on reload)
jobs.dbPath ~/.pi/daemon/jobs.db SQLite database path. Requires a daemon restart to apply changes.
jobs.runHistoryLimit 100 Max runs retained per job (10–10000). Applied live on reload.
jobs.maxConcurrent 1 Max distinct jobs running at once (1–64). Excess triggers/cron fires are FIFO-queued. Applied live on reload (raising it drains the queue).

runHistoryLimit, maxConcurrent, and enabled are hot-reloadable; dbPath changes require a restart.

Job Endpoints

Endpoint Description
GET /api/jobs List all jobs (includes nextRun, isRunning, isQueued, queuePosition)
POST /api/jobs Create a job (201)
GET /api/jobs/:id Get a single job
PUT /api/jobs/:id Update a job (reschedules if enabled)
DELETE /api/jobs/:id Delete a job and its runs (204)
GET /api/jobs/:id/runs List run history (limit/offset query params, max 200)
GET /api/jobs/queue List runs waiting for a free slot (FIFO order)
POST /api/jobs/:id/trigger Manually trigger a run (202 queued if at maxConcurrent cap; 409 JOB_BUSY if already running/queued)
POST /api/jobs/:id/kill Abort an in-flight or queued run (409 if neither)

Creating a Job

curl -X POST http://127.0.0.1:9876/api/jobs \
  -H "Content-Type: application/json" \
  -d '{
    "name": "nightly-report",
    "description": "Summarize the day's activity",
    "cronExpression": "0 22 * * *",
    "prompt": "summarize my day",
    "enabled": true
  }'

cronExpression must be a valid 5- or 6-field cron expression (validated with croner). name, cronExpression, and prompt are required; description and enabled (default true) are optional.

Optional provider and model fields select which pi provider/model runs the job. When provided, they are validated against pi --list-models (the result is cached for 12 hours). If pi cannot be found, validation is soft and the values are accepted. When omitted, pi's defaults are used.

curl -X POST http://127.0.0.1:9876/api/jobs \
  -H "Content-Type: application/json" \
  -d '{
    "name": "nightly-report",
    "cronExpression": "0 22 * * *",
    "prompt": "summarize my day",
    "provider": "anthropic",
    "model": "claude-sonnet-4"
  }'

When the job runs, the daemon spawns pi --provider <provider> --model <model> <prompt> (omitting the flags for any field that is unset).

Job Definition

{
  "id": 1,
  "name": "nightly-report",
  "description": "Summarize the day's activity",
  "cronExpression": "0 22 * * *",
  "prompt": "summarize my day",
  "provider": "anthropic",
  "model": "claude-sonnet-4",
  "enabled": true,
  "createdAt": "2026-06-21T00:00:00.000Z",
  "updatedAt": "2026-06-21T00:00:00.000Z",
  "nextRun": "2026-06-21T22:00:00.000Z",
  "isRunning": false
}

provider and model are null when unset; in that case the job runs with pi's default provider and model.

Run History

Each execution is recorded as a JobRun with status (pending → running → success | failed | killed), exitCode, stdout, and stderr. Stale running runs from an unclean shutdown are recovered to failed on daemon startup. A killed run is one aborted via the kill endpoint; its stderr ends with Job killed by user (or Job cancelled from queue for a queued run killed before it started). Runs are pruned to jobs.runHistoryLimit per job after each execution.

When the global jobs.maxConcurrent cap is reached, additional triggers and cron fires are placed in a durable FIFO queue (job_queue table) and started as slots free up. A job that is already running or queued is not queued again (cron fires are skipped with a warn log; manual triggers get 409 JOB_BUSY). Queued runs are observable as isQueued: true with a 0-based queuePosition on job listings, and their job_runs row stays pending until dequeued. On daemon shutdown, queued runs are marked killed with stderr = "Daemon shutdown; run cancelled from queue"; on a crash, recoverStaleRuns leaves genuinely queued runs in place and marks orphaned pending rows failed.

curl http://127.0.0.1:9876/api/jobs/1/runs?limit=10

Error Responses

Code Status Description
NOT_INITIALIZED 503 Jobs controller not initialized (plugin disabled)
NOT_FOUND 404 Job not found
CONFLICT 409 Duplicate job name, job already running on trigger, or kill while not running

Error Responses

All errors return structured JSON:

{
  "error": "Description of the error",
  "code": "ERROR_CODE",
  "details": {}
}
Code Status Description
VALIDATION_ERROR 400 Request body failed validation
INGESTION_ERROR 400 Metric could not be ingested
AUTH_MISSING 401 Authorization header not provided
AUTH_INVALID 403 Invalid auth token
DUPLICATE_REQUEST 409 Batch with same requestId already processed
RATE_LIMITED 429 Rate limit exceeded
INTERNAL_ERROR 500 Unexpected server error

Daemon Self-Metrics

The daemon exposes its own operational metrics on /metrics:

Metric Type Description
pi_daemon_requests_total counter Total requests (labels: method, path, status)
pi_daemon_request_duration_seconds histogram Request latency
pi_daemon_uptime_seconds gauge Daemon uptime
pi_daemon_memory_usage_bytes gauge Memory usage (labels: type)

pi Extension

The pi-daemon extension is a pi agent extension that integrates your agent sessions with a running pi-daemon instance. It listens to runtime events and sends metrics to the daemon via fire-and-forget POST requests.

Installation

// .pi/settings.json
{
  "packages": ["pi-daemon"]
}

Extension Config File

Create ~/.pi/daemon-client/config.json to configure the extension. Set PI_DAEMON_CLIENT_CONFIG to override the default path.

{
  "daemonUrl": "http://127.0.0.1:9876",
  "auth": {
    "token": "your-token"
  },
  "maxBufferSize": 50,
  "flushIntervalMs": 5000,
  "requestTimeoutMs": 2000,
  "enabled": true,
  "metrics": {
    "compaction": true,
    "sessions": true
  }
}

Config Merging

Config values are resolved in the following priority order (highest to lowest):

  1. Environment variables — override everything
  2. Config file — ~/.pi/daemon-client/config.json (or PI_DAEMON_CLIENT_CONFIG)
  3. Defaults — built-in defaults

Each layer is deep-merged onto the previous one. Only fields present in a layer override the layer below it.

Config Options

Key Default Description
daemonUrl http://127.0.0.1:9876 Daemon URL
auth.token (none) Bearer token for authenticated daemon endpoints
maxBufferSize 50 Max metrics to buffer before flushing
flushIntervalMs 5000 Flush interval (ms)
requestTimeoutMs 2000 HTTP request timeout (ms)
enabled true Enable/disable the extension
metrics.compaction true Send compaction metrics
metrics.sessions true Send session metrics

Environment Variables

Variable Maps To
PI_DAEMON_CLIENT_CONFIG Config file path (overrides default location)
PI_DAEMON_URL daemonUrl
PI_DAEMON_TOKEN auth.token
PI_DAEMON_BUFFER_SIZE maxBufferSize
PI_DAEMON_FLUSH_INTERVAL flushIntervalMs
PI_DAEMON_TIMEOUT requestTimeoutMs
PI_DAEMON_ENABLED enabled (set to "false" to disable)

Built-in Metrics

The extension automatically emits these metrics (all include session and model labels when available):

Compaction Metrics

Metric Type Description
pi_compaction_tokens_before counter Tokens before compaction
pi_compaction_tokens_after counter Tokens after compaction
pi_compaction_tokens_saved counter Tokens saved (before - after)
pi_compaction_summary_length gauge Length of the summary text
pi_compaction_duration_seconds histogram Compaction duration
pi_compaction_total counter Total compactions performed

Session Metrics

Metric Type Description
pi_sessions_started counter Sessions started
pi_sessions_ended counter Sessions ended

Sending Metrics Manually

# Single metric
curl -X POST http://127.0.0.1:9876/api/metrics \
  -H "Content-Type: application/json" \
  -d '{"name":"my_counter","type":"counter","increment":1}'

# Batch
curl -X POST http://127.0.0.1:9876/api/metrics/batch \
  -H "Content-Type: application/json" \
  -d '{"metrics":[{"name":"a","type":"counter","increment":1},{"name":"b","type":"gauge","set":42}]}'

# With auth
curl -X POST http://127.0.0.1:9876/api/metrics \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-token-here" \
  -d '{"name":"c","type":"counter","increment":1}'

Best Practices

  1. Use batch ingestion for multiple metrics — it reduces HTTP overhead.
  2. Include meaningful labels — session, model, and source help with filtering.
  3. Keep label cardinality low — avoid unique values in label keys (e.g., timestamps, UUIDs).
  4. Use counters for cumulative counts — they are the most memory-efficient.
  5. Use histograms for distributions — latency, token counts, etc.
  6. Set a reasonable timeout — the default 2s prevents blocking the pi event loop.

Deployment

systemd (Linux)

Run pi-daemon as a managed system service with auto-restart and journald logging.

Prerequisites

  • Node.js 22.19.0+
  • systemd (Linux)

Install

1. Install the daemon:

# Global install (recommended)
npm install -g pi-daemon

# Or from source
git clone https://github.com/earendil-works/pi-daemon.git
cd pi-daemon
npm install
npm run build
npm install -g .

2. Create config directory and config file:

sudo mkdir -p /etc/pi-daemon
sudo mkdir -p /var/lib/pi-daemon
sudo mkdir -p /var/log/pi-daemon

sudo cp contrib/pi-daemon.json /etc/pi-daemon/config.json

3. Install the systemd service:

sudo cp contrib/pi-daemon.service /etc/systemd/system/
sudo systemctl daemon-reload

4. Configure the service user and pi binary:

Scheduled jobs spawn a pi process. systemd services run with a minimal environment, so a pi binary that works in your interactive shell may not be on the service PATH. If pi is installed in a user-local directory, also run the daemon as that user so pi uses the expected home directory, config, and session storage.

Find the absolute path to pi:

which pi

Then create a systemd drop-in, replacing <user> and <path-to-pi>:

sudo systemctl edit pi-daemon.service
[Service]
User=<user>
Group=<user>
Environment=NODE_ENV=production
Environment=PI_BIN=<path-to-pi>
Environment=HOME=/home/<user>
ReadWritePaths=/var/lib/pi-daemon /var/log/pi-daemon /home/<user>/.pi

Make sure the runtime directories are writable by the service user:

sudo chown -R <user>:<user> /var/lib/pi-daemon /var/log/pi-daemon
sudo systemctl daemon-reload

If you see Failed to spawn pi: spawn pi ENOENT, the service cannot find the pi binary; set PI_BIN to the absolute path from which pi. If spawned jobs try to write under /root/.pi, the service is still running as root or does not have HOME set for the intended user.

5. Enable and start:

sudo systemctl enable --now pi-daemon.service
sudo systemctl status pi-daemon.service

Manage

# Start / stop / restart
sudo systemctl start pi-daemon.service
sudo systemctl stop pi-daemon.service
sudo systemctl restart pi-daemon.service

# Reload configuration (SIGHUP)
sudo systemctl reload pi-daemon.service

# View logs
sudo journalctl -u pi-daemon.service -f

Config

Edit /etc/pi-daemon/config.json. Key settings:

Setting Default Description
server.listenAddress 127.0.0.1 Bind address (keep 127.0.0.1 unless behind a reverse proxy)
server.port 9876 HTTP port
auth.enabled false Enable Bearer token auth on POST endpoints
logging.level info Log level (debug, info, warn, error)

Uninstall

sudo systemctl stop pi-daemon.service
sudo systemctl disable pi-daemon.service
sudo rm /etc/systemd/system/pi-daemon.service
sudo systemctl daemon-reload
sudo rm -rf /etc/pi-daemon /var/lib/pi-daemon /var/log/pi-daemon
npm uninstall -g pi-daemon

Architecture

Current:

┌─────────────────┐     POST      ┌──────────────────────────────┐
│  pi Agent       │ ──────────►   │  pi-daemon  :9876            │
│  (Extension)    │               │                              │
│                 │               │  • Metrics plugin            │
│  • Compaction   │               │  • Scheduled jobs            │
│  • Session      │               │  • /api/jobs                 │
│  events         │               │  • /api/metrics (ingest)     │
└─────────────────┘               │  • /metrics     (expose)     │
                                  │  • /health, /admin/*         │
        ┌─────────────┐           └──────────────────────────────┘
        │  pi (spawn) │   spawn            ▲
        │  job runner │ ◄──────────         │ GET (scrape)
        └─────────────┘                ┌────┴─────┐
                                       │Prometheus│
                                       └──────────┘

Planned:

┌─────────────────┐     POST      ┌──────────────────────────────┐
│  pi Agent       │ ──────────►   │  pi-daemon  :9876            │
│  (Extension)    │               │                              │
│                 │               │  • Metrics plugin            │
│  • Compaction   │               │  • Scheduled jobs            │
│  • Session      │               │  • RAG server (planned)      │
│  events         │               │                              │
└─────────────────┘               └──────────────────────────────┘
                                          ▲
                                          │ GET (scrape)
                                    ┌─────┴──────┐
                                    │ Prometheus │
                                    └────────────┘

Security

  • Default listen address is 127.0.0.1 (localhost only)
  • Enable auth.enabled: true for Bearer token authentication on POST endpoints
  • When auth is enabled without a token, one is auto-generated
  • Config file contains auth token in plaintext — ensure chmod 600 on the config file

License

MIT

About

Background server for Pi Coding Agent to enable metrics, scheduled jobs, and alerts

Resources

Readme

License

MIT license
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages