Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 38 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Pre-alpha. We are working through the milestones defined in
| **M3** | Persistence (sqlite + jsonl replay) | ✅ complete — `SqliteConcernStore` ([PR-13 / #15](https://github.com/HyperdustLabs/OpenCOAT/pull/15)), `SqliteDCNStore` ([PR-14 / #16](https://github.com/HyperdustLabs/OpenCOAT/pull/16)), JSONL replay ([PR-15 / #18](https://github.com/HyperdustLabs/OpenCOAT/pull/18)), `examples/03_persistent_agent_demo` ([PR-16 / #20](https://github.com/HyperdustLabs/OpenCOAT/pull/20)) |
| **M4** | Daemon + CLI + HTTP/JSON-RPC | ✅ complete — `build_runtime` ([PR-17 / #21](https://github.com/HyperdustLabs/OpenCOAT/pull/21)), in-proc JSON-RPC ([PR-18 / #22](https://github.com/HyperdustLabs/OpenCOAT/pull/22)), stdlib HTTP JSON-RPC ([PR-19 / #23](https://github.com/HyperdustLabs/OpenCOAT/pull/23)), daemon lifecycle ([PR-20 / #24](https://github.com/HyperdustLabs/OpenCOAT/pull/24)), `opencoat runtime up\|down\|status` ([PR-21 / #25](https://github.com/HyperdustLabs/OpenCOAT/pull/25)), `opencoat concern \| dcn \| inspect` ([PR-22 / #26](https://github.com/HyperdustLabs/OpenCOAT/pull/26)), `examples/06_long_running_daemon` ([PR-23 / #27](https://github.com/HyperdustLabs/OpenCOAT/pull/27)) |
| **M5** | OpenClaw host plugin | ✅ complete — event map ([#28](https://github.com/HyperdustLabs/OpenCOAT/pull/28)), injection + spans ([#29](https://github.com/HyperdustLabs/OpenCOAT/pull/29)), tool guard ([#30](https://github.com/HyperdustLabs/OpenCOAT/pull/30)), memory bridge + hooks ([#31](https://github.com/HyperdustLabs/OpenCOAT/pull/31)), `examples/04_openclaw_with_runtime` ([#32](https://github.com/HyperdustLabs/OpenCOAT/pull/32)) |
| M6 | Heartbeat + Meta governance workers | pending |
| **M6** | Heartbeat + meta governance (decay, conflict scan, merge/archive, meta-review) | 🚧 in progress — PR1 ([#72](https://github.com/HyperdustLabs/OpenCOAT/pull/72)) merged; PR2–4 ([#73](https://github.com/HyperdustLabs/OpenCOAT/pull/73)) open |
| M7 | Second host (langgraph/hermes) | pending |
| M8 | Postgres + Helm/K8s | pending |

Expand Down Expand Up @@ -268,6 +268,43 @@ LLM keys are not embedded in the unit files — use `opencoat configure llm`
LLM keys from that env file at startup; add `EnvironmentFile=` only if you
need variables outside that allow-list or a non-default env file path.

### Heartbeat + DCN maintenance (M6)

With `runtime.loops.heartbeat_enabled: true` (bundled default), the daemon
starts a background scheduler (default **30s**) that calls
`OpenCOATRuntime.tick()`. Each tick runs:

| Worker | Role |
| --- | --- |
| `DecayWorker` | Bumps `activation_state.decay`; weakens / archives stale concerns |
| `MergeArchiverWorker` | Merges duplicate concerns into the DCN; archives cold `weakened` rows |
| `ConflictScannerWorker` | Writes `conflicts_with` edges for background analysis (weave-time drops stay in `ConflictResolver`) |
| `MetaReviewWorker` | Inventories `meta_concern` rows (governance capabilities) |

On startup you should see `heartbeat scheduler started` in the daemon log.
Tune overlap and cold-archive thresholds under `runtime.loops.maintenance` in
[`docs/config/daemon.yaml.example`](docs/config/daemon.yaml.example).

**Verify prerequisites** (joinpoint hot path + RPC smoke):

```bash
./scripts/verify-m6-prerequisites.sh # daemon on 127.0.0.1:7878
```

**Hermetic soak** (10 heartbeat ticks, no 24h wait):

```bash
uv run python -m pytest packages/opencoat-runtime/tests/soak/test_heartbeat_maintenance_soak.py -q
```

See [`examples/07_meta_governance_soak/README.md`](examples/07_meta_governance_soak/README.md)
and [`docs/07-mvp/m6-conflict-paths.md`](docs/07-mvp/m6-conflict-paths.md).

**OpenClaw:** weave on user chat happens at `before_prompt_build` → `before_response`,
not on `message_received` (`on_user_input`). Optional chat mining:
`extract_from_chat` on `joinpoint.submit` or bridge config `extractOnUserMessage`
(see [`integrations/openclaw-opencoat-bridge/README.md`](integrations/openclaw-opencoat-bridge/README.md)).

---

## Contributing
Expand Down
14 changes: 8 additions & 6 deletions docs/07-mvp/m6-prerequisites-status.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,15 @@ Re-run automation: `./scripts/verify-m6-prerequisites.sh` from repo root (daemon
| **P2b** | Live OpenClaw gateway + bridge | **PASS** | 2026-05-18 local smoke — bridge README §3; NVDA concerns weave on `before_response` |
| **P3** | Conflict paths documented | **PASS** | [m6-conflict-paths.md](./m6-conflict-paths.md); [ADR-0010](../adr/0010-concern-aop-syntax.md) |

## M6 implementation (`feat/m6-lifecycle-workers`)
## M6 implementation

| PR slice | Status | Notes |
| --- | --- | --- |
| **PR1** decay + `ConflictScannerWorker` + scheduler | **in progress** | `DecayWorker`, `ConflictScannerWorker`, `HeartbeatLoop` maintenance hook, `Scheduler.start` in daemon |
| **PR2** merge + archive | pending | `merge_archiver.py` stub |
| **PR3** meta-review | pending | ADR-0008 governance loop |
| **PR4** 24h soak + example | pending | `examples/07_meta_governance_soak` |
| **PR1** decay + `ConflictScannerWorker` + scheduler | **merged** | [#72](https://github.com/HyperdustLabs/OpenCOAT/pull/72) |
| **PR2** merge + archive | **open** ([#73](https://github.com/HyperdustLabs/OpenCOAT/pull/73)) | `DCNEvolver`, `MergeArchiverWorker`, `HeartbeatMaintenance` config |
| **PR3** meta-review | **open** (same branch) | `MetaReviewWorker` + `DefaultEvolutionControl` inventory |
| **PR4** soak + example | **open** (same branch) | `tests/soak/`, `examples/07_meta_governance_soak` |

**Next:** finish PR1 tests on CI, then merge/archive workers (PR2).
**Docs:** root [`README.md`](../../README.md) (M6 table + heartbeat section), [`examples/README.md`](../../examples/README.md) row 07, [`packages/opencoat-runtime/README.md`](../../packages/opencoat-runtime/README.md).

**Next:** merge [#73](https://github.com/HyperdustLabs/OpenCOAT/pull/73); optional 24h live soak on daemon.
2 changes: 1 addition & 1 deletion docs/07-mvp/milestones.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,6 @@ Source: [`design/v0.2-system-design.md`](../design/v0.2-system-design.md) §12.
| **M3 — Persistence** | sqlite backend + restart recovery + jsonl replay | DCN survives restart; `opencoat replay` reproduces a turn | ✅ — see [`README.md`](../../README.md) M3 row for per-PR breakdown |
| **M4 — Daemon + CLI** | Daemon HTTP/JSON-RPC + `opencoat` CLI + host-sdk HTTP transport | Host calls daemon over socket and completes a turn | ✅ — see [`README.md`](../../README.md) M4 row for per-PR breakdown |
| **M5 — OpenClaw plugin** | Full `host-plugins/openclaw` adapter | `04_openclaw_with_runtime` runs end-to-end | ✅ — see [`README.md`](../../README.md) M5 row for per-PR breakdown |
| **M6 — Heartbeat + Meta** | Decay / conflict / merge / archive / meta-review workers | 24h soak: DCN converges, token budget stable. Activation-time conflicts stay in `ConflictResolver` (already on `main`); M6 adds heartbeat workers + DCN evolution. | pending — [§5A prerequisites](./post-m5-roadmap.md#5a-m6-split-4-prs) then 4 PRs in [`post-m5-roadmap.md`](./post-m5-roadmap.md) |
| **M6 — Heartbeat + Meta** | Decay / conflict / merge / archive / meta-review workers | 24h soak: DCN converges, token budget stable. Activation-time conflicts stay in `ConflictResolver` (already on `main`); M6 adds heartbeat workers + DCN evolution. | 🚧 PR1 merged ([#72](https://github.com/HyperdustLabs/OpenCOAT/pull/72)); PR2–4 ([#73](https://github.com/HyperdustLabs/OpenCOAT/pull/73)) — see [`m6-prerequisites-status.md`](./m6-prerequisites-status.md) |
| **M7 — Second host** | LangGraph (or Hermes) adapter; multi-host shared DCN | Two hosts share one DCN without conflict | pending |
| **M8 — Postgres + K8s** | Postgres backend + helm chart | 7-day stability on a K8s cluster | pending |
12 changes: 9 additions & 3 deletions docs/config/daemon.yaml.example
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,16 @@
runtime:
schema_version: "0.2"
loops:
# How often the heartbeat worker runs DCN maintenance (lifecycle
# transitions, decay, archiving). 30s is plenty for human-scale
# agents; tighten to 5–10s for high-traffic deployments.
# How often the background scheduler calls OpenCOATRuntime.tick().
# 30s is plenty for human-scale agents; tighten to 5–10s for high traffic.
heartbeat_interval_seconds: 30
# Set false to disable the background heartbeat thread (tests / embedded).
heartbeat_enabled: true
# M6 workers: decay, merge/archive, conflict scan tuning.
maintenance:
merge_min_keyword_overlap: 3
archive_cold_decay_threshold: 0.85
archive_cold_max_score: 0.15
budgets:
# How many concerns can be active in one Concern Vector. Above
# this the coordinator drops the lowest-priority ones.
Expand Down
36 changes: 36 additions & 0 deletions examples/07_meta_governance_soak/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# 07 — Meta governance heartbeat soak (M6)

Stand-in for the M6 **24h soak** exit criterion: the daemon's background
scheduler calls `OpenCOATRuntime.tick()`, which runs decay, merge/archive,
conflict scan, and (optionally) meta-review workers.

## Layout

```text
examples/07_meta_governance_soak/
└── README.md ← you are here (no main.py — use daemon + pytest soak)
```

## Hermetic (CI)

From repo root — ten maintenance ticks in-process:

```bash
uv run python -m pytest packages/opencoat-runtime/tests/soak/test_heartbeat_maintenance_soak.py -q
```

## Live daemon

1. `opencoat runtime up` — log should include `heartbeat scheduler started`.
2. Optional: tune `runtime.loops.maintenance` in `~/.opencoat/daemon.yaml`
(see [`docs/config/daemon.yaml.example`](../../docs/config/daemon.yaml.example)).
3. Leave running; periodically check `opencoat runtime snapshot` and
`opencoat dcn activation-log`.
4. Re-run [`scripts/verify-m6-prerequisites.sh`](../../scripts/verify-m6-prerequisites.sh)
after long runs.

## Related docs

- [`docs/07-mvp/m6-conflict-paths.md`](../../docs/07-mvp/m6-conflict-paths.md) — activation-time vs background conflict paths
- [`docs/07-mvp/post-m5-roadmap.md`](../../docs/07-mvp/post-m5-roadmap.md) — M6 PR split
- Root [`README.md`](../../README.md) — heartbeat overview
1 change: 1 addition & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ End-to-end usage of the OpenCOAT Runtime.
| 04 | `04_openclaw_with_runtime/` | M5 ([#32](https://github.com/HyperdustLabs/OpenCOAT/pull/32)) — toy OpenClaw bus + `install_hooks` + memory bridge |
| 05 | `05_langgraph_with_runtime/` | M7 |
| 06 | `06_long_running_daemon/` | M4 (PR-23) — programmatic Daemon ↔ HTTP JSON-RPC end-to-end |
| 07 | `07_meta_governance_soak/` | M6 — heartbeat maintenance soak (decay / merge / conflict scan) |

Each example contains a `README.md` with the user story, a runnable
`main.py`, and a frozen transcript so we can diff future runs.
15 changes: 15 additions & 0 deletions integrations/openclaw-opencoat-bridge/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,21 @@ curl -sS http://127.0.0.1:7878/rpc -H 'Content-Type: application/json' \

Requires **JoinpointDiscovery** (`expand_prompt_surface` on by default). Older daemons ignore `messages[]` and only match lifecycle names.

## Weaving expectations

| Hook / joinpoint | Typical injections |
| --- | --- |
| `message_received` → `on_user_input` | Often **empty** if your concerns only list `before_response` / `user_message` |
| `before_prompt_build` → `before_response` | Main weave path when keywords match flattened prompt or discovered `user_message` rows |

For background DCN maintenance (decay, merge, conflict edges), run the daemon with
heartbeat enabled — see root [`README.md`](../../README.md) § Heartbeat + DCN maintenance (M6).

**Optional chat mining:** set `extractOnUserMessage: true` so the bridge passes
`extract_from_chat: true` on `joinpoint.submit` (requires a configured LLM on the
daemon). Extraction updates the concern store; it does not always add rows to
`injections` on that same submit.

## Limitations (v0.1 bridge)

- Prompt folding uses `prependSystemContext` only (not full dotted-path injector parity with Python `OpenClawInjector`).
Expand Down
27 changes: 27 additions & 0 deletions packages/opencoat-runtime/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,38 @@ pip install "opencoat-runtime[grpc]" # daemon gRPC transport
```bash
opencoat --version
opencoat concern import --demo
opencoat runtime up # sqlite + HTTP JSON-RPC + heartbeat scheduler (M6)
opencoat-daemon --help
```

See <https://github.com/HyperdustLabs/OpenCOAT> for the full runtime guide.

## Heartbeat maintenance (M6)

When the daemon runs with `runtime.loops.heartbeat_enabled: true`, a background
thread invokes `OpenCOATRuntime.tick()` every `heartbeat_interval_seconds`
(default 30). Each tick:

1. **DecayWorker** — increases per-concern `activation_state.decay`, then weakens or archives.
2. **MergeArchiverWorker** — runs `DCNEvolver` to merge duplicates and archive cold weakened concerns.
3. **ConflictScannerWorker** — syncs `conflicts_with` edges into the DCN (background only; hot-path weave still uses `ConflictResolver`).
4. **MetaReviewWorker** — counts active `meta_concern` rows for governance review ticks.

Configure thresholds under `runtime.loops.maintenance` in your daemon YAML
(see [`docs/config/daemon.yaml.example`](https://github.com/HyperdustLabs/OpenCOAT/blob/main/docs/config/daemon.yaml.example)).

**JSON-RPC:** `joinpoint.submit` accepts optional `extract_from_chat: true` to run
`concern.extract` on user `messages[]` before weaving (needs a real LLM).

**Tests / soak:**

```bash
uv run python -m pytest packages/opencoat-runtime/tests/daemon/test_m6_workers.py -q
uv run python -m pytest packages/opencoat-runtime/tests/soak/test_heartbeat_maintenance_soak.py -q
```

**Prerequisites gate:** from repo root, `./scripts/verify-m6-prerequisites.sh` (daemon on `:7878`).

## License

Apache-2.0.
11 changes: 11 additions & 0 deletions packages/opencoat-runtime/opencoat_runtime_core/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,23 @@ class RuntimeBudgets(BaseModel):
max_advice_per_concern: int = Field(default=2, ge=1)


class HeartbeatMaintenance(BaseModel):
"""M6 background worker tuning (decay / merge / conflict scan)."""

model_config = ConfigDict(extra="forbid")

merge_min_keyword_overlap: int = Field(default=3, ge=1)
archive_cold_decay_threshold: float = Field(default=0.85, ge=0.0, le=1.0)
archive_cold_max_score: float = Field(default=0.15, ge=0.0, le=1.0)


class RuntimeLoops(BaseModel):
model_config = ConfigDict(extra="forbid")

heartbeat_interval_seconds: float = Field(default=30.0, gt=0.0)
#: When false, the daemon does not start the background heartbeat scheduler.
heartbeat_enabled: bool = Field(default=True)
maintenance: HeartbeatMaintenance = Field(default_factory=HeartbeatMaintenance)


class JoinpointAutomation(BaseModel):
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
"""Catalog scan helpers for DCN evolution and daemon heartbeat workers."""

from __future__ import annotations

from opencoat_runtime_protocol import Concern


def joinpoint_names(concern: Concern) -> frozenset[str]:
names: set[str] = set()
if concern.pointcut is not None:
for jp in concern.pointcut.joinpoints:
if isinstance(jp, str) and jp:
names.add(jp)
for pc in concern.pointcuts:
for jp in pc.joinpoints:
if isinstance(jp, str) and jp:
names.add(jp)
return frozenset(names)


def activation_keywords(concern: Concern) -> frozenset[str]:
keywords: set[str] = set()
if concern.pointcut is not None and concern.pointcut.match is not None:
raw = concern.pointcut.match.any_keywords
if raw:
keywords.update(k for k in raw if isinstance(k, str) and k)
for pc in concern.pointcuts:
if pc.match is None or not pc.match.any_keywords:
continue
keywords.update(k for k in pc.match.any_keywords if isinstance(k, str) and k)
return frozenset(keywords)


__all__ = ["activation_keywords", "joinpoint_names"]
Loading
Loading