Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,7 @@ htmlcov/
*.sqlite
.DS_Store
**/.DS_Store

# Growth-loop per-iteration log: rewritten every run, would just be commit-hash
# churn if tracked (a row can't cite its own commit's hash until after it exists).
results.tsv
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Fixed
- **`examples/financial_incidents/run.py`** crashed on a fresh Windows install
(`UnicodeEncodeError` from the console's default `cp1252` codepage) when running
the exact command the README's Demos table advertises. Stdout/stderr now force
UTF-8 if the console isn't already.

### Added
- **`CHRONICLE_ENABLED`**: set to `0` / `false` / `off` / `no` to turn off LIVE
recording. `@boundary`, `wrap`, `wrap_llm`, `record()`, and `EnvelopeRecorder`
Expand Down
48 changes: 48 additions & 0 deletions docs/growth/fixes-log.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Growth-loop fixes log

One entry per loop iteration that changed something (skipped/discarded attempts are
in `results.tsv` and narrated in `train.md`, not repeated here). This file is the
flat, chronological "what actually shipped" list; `train.md` has the reasoning.

---

## Iteration 1 — Windows demo crash (2026-08-09)

- **Commit:** `9775deb`
- **File:** `examples/financial_incidents/run.py`
- **Broke:** the exact command the README's Demos table tells a new user to run,
`python examples/financial_incidents/run.py refund test`, crashed on a fresh
Windows install with `UnicodeEncodeError` — the console's default `cp1252`
codepage can't encode the box-drawing character (`─`, U+2500) the demo prints.
- **Fixed:** reconfigure `sys.stdout`/`sys.stderr` to UTF-8 (`errors="replace"`) at
the top of the script if the console isn't already UTF-8, before any output prints.
- **Score:** 0.0 -> 128.0 (`scripts/growth_eval.py`; TTFSR gate was the failure).
- **Status:** kept.

## Iteration 2 — untested example (2026-08-09)

- **Commit:** `dce6fcd`
- **File:** `tests/test_langgraph_demo.py` (new)
- **Broke:** nothing crashed, but `examples/langgraph_demo` was the one integration
example with zero test coverage, so a future change could silently break it.
- **Fixed:** added a deterministic `layer1` test (`pytest.importorskip("langgraph")`
guard) asserting both graph nodes record correctly as Envelopes.
- **Score:** 128.0 -> 133.0 (`example_coverage` 3/4 -> 4/4).
- **Status:** kept.

## Iteration 3 — CHANGELOG hygiene (2026-08-09, score-neutral)

- **Commit:** `a69627f`
- **File:** `CHANGELOG.md`
- **Fixed:** logged the iteration-1 Windows fix under `[Unreleased] / Fixed`. Not
scored by `prepare.md`; done because a real fix with no changelog trail is a
trust gap on its own.
- **Score:** unchanged (133.0), gates still pass.
- **Status:** kept.

## Run closed here

The scored surface saturated after iteration 2 — see `train.md` "Stopping the
score-driven part of this run here" for why the remaining gap
(`lines_before_first_code_block`) isn't worth chasing, and what that implies about
where the real growth bottleneck is (distribution, not on-page quality).
245 changes: 245 additions & 0 deletions docs/growth/outreach-templates.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,245 @@
# Outreach templates

Drafts only. Nothing here gets sent or posted until Tisha approves the exact text
per venue (see `program.md`: outreach is never autonomous). Every number below is
pulled straight from `docs/benchmark-results.json` or the README — no invented
stats. No em dashes or en dashes (repo style).

Common framing to stay consistent with the README's own positioning: Chronicle is
not a tracing dashboard or an eval framework, it is what makes a recorded agent run
replayable and testable. Scope is honestly narrow: control-flow and tool-safety
regressions caught deterministically from recorded incidents, not general model
quality.

---

## 1. Show HN

**Title (pick one, HN penalizes hype):**
- `Show HN: Chronicle – turn a production agent failure into a committed regression test`
- `Show HN: Record-and-replay for agent decision graphs, no live LLM calls on replay`

**Body:**

```
Chronicle records what an agent did at each decision point (LLM call, tool call,
routing choice) as an immutable "Envelope." When something goes wrong in
production, you commit the recording as a fixture under fixtures/traces/, then
write a cut-point test: stub everything upstream, run your fix live at the one
boundary that changed, and assert deterministically with no LLM calls on replay.

It's not a tracing dashboard or an eval framework, there are good ones already
(LangSmith, Langfuse, Phoenix). Chronicle is the piece that makes a recorded run
replayable and testable, and it sits alongside those tools rather than
replacing them.

Recording overhead in our benchmark harness is ~20-25us/crossing (~0.008% of a
300ms model call) and ~1.4KB/crossing on disk (examples/benchmark, numbers in
docs/benchmark-results.json). It has a LangGraph integration
(instrument_langgraph) and wraps OpenAI/Anthropic-style clients directly
(chronicle.wrap(client)) or plain functions (@boundary).

Honest scope: this catches control-flow and tool-safety regressions in
multi-agent systems, not model-quality drift. There's an optional LLM-as-judge
layer for meaning, but Layer 1 (deterministic replay) never calls a live model.

pip install agent-chronicle
GitHub: https://github.com/theagentplane/chronicle
Quick start: https://github.com/theagentplane/chronicle#quick-start

Happy to answer questions about the design, especially the cut-point mechanism
and what "boundary" does and doesn't capture (I/O only, not side effects).
```

---

## 2. r/LocalLLaMA

Framing: developer-tool, technical, skip the "why agents matter" preamble.

```
Built Chronicle: record-and-replay for agent decision graphs. If your agent's
prod incident is a plain text trace you can't re-run, this turns it into a
committed fixture and a deterministic regression test (no live LLM calls on
replay).

@boundary decorator, or chronicle.wrap(client) for zero-decorator recording,
or instrument_langgraph(nodes) if you're on LangGraph. Layer 1 replay is
structural/deterministic; there's an optional LLM-as-judge layer if you want
to check meaning instead of exact output.

MIT licensed, pip install agent-chronicle.
https://github.com/theagentplane/chronicle

Genuinely curious if this matches how people here are already debugging agent
regressions, or if most of you are rolling your own trace-replay scripts.
```

---

## 3. r/MachineLearning

Framing: slightly more academic/methods-oriented, this subreddit is stricter
about self-promotion, lead with the problem and be upfront it's your project.

```
[P] Chronicle: deterministic replay for regression-testing LLM agents

Sharing a project I've been building: a record-and-replay system for agent
decision graphs. The problem it targets is narrow and specific: a production
agent fails, you want to reproduce that exact failure as a committed test, fix
one component, and verify the fix without re-running the whole agent against a
live model (nondeterministic, slow, costs money).

Mechanism: an immutable "Envelope" captures I/O at each decision boundary
(LLM call, tool call, routing choice). Cut-point replay lets you stub every
boundary upstream of the one you changed, run that one live, and assert
deterministically. Two verification layers: structural replay (no LLM calls)
for control flow, and an optional LLM-as-judge layer for output meaning.

This is not a general observability or eval framework, it complements those.
Benchmark numbers (methodology + harness in examples/benchmark) are in the
repo: ~20-25us/crossing recording overhead, ~1.4KB/crossing storage.

MIT license, source + docs: https://github.com/theagentplane/chronicle

Feedback on the approach (especially where cut-point replay breaks down) is
what I'm actually here for, not just the link.
```

---

## 4. r/AI_Agents

Framing: more practitioner/building-in-public tone fits this sub.

```
Shipped a fix for something that kept biting me building multi-agent systems:
reproducing a specific prod failure without needing the whole agent + a live
model to re-run every time.

Chronicle records each decision boundary (LLM call, tool call, routing
decision) as an immutable Envelope, then lets you commit an incident as a
fixture and cut-point test just the one boundary you fixed, live, with
everything else stubbed from the recording. No LLM calls on replay.

Works with @boundary, or zero-decorator via chronicle.wrap(openai_client),
or instrument_langgraph(nodes) for LangGraph.

pip install agent-chronicle · https://github.com/theagentplane/chronicle

Scope is honest: this is for control-flow/tool-safety bugs, not for chasing
model-quality regressions, there's a separate LLM-as-judge layer for that if
you want it.
```

---

## 5. r/LangChain

Framing: lead with the LangGraph integration specifically.

```
If you're on LangGraph and have hit "the agent regressed and I can't cheaply
tell you why without re-running the whole graph against a live model" -
Chronicle has an instrument_langgraph(nodes) entry point that records every
node crossing as an Envelope, then lets you replay a past run deterministically
(stub upstream, run your fix live at one node) without calling the model again.

Two verification layers: structural replay for control flow (no LLM calls),
optional LLM-as-judge for output meaning.

https://github.com/theagentplane/chronicle - pip install agent-chronicle,
MIT licensed. Would love feedback from anyone running LangGraph in prod on
whether the node-level granularity is the right cut for your failures.
```

---

## 6. LangChain Community Slack intro

Post in an introductions/showcase channel, not a DM blast.

```
Hi all, I'm Tisha, building Chronicle, a record-and-replay system for agent
decision graphs (MIT, pip install agent-chronicle). It has an
instrument_langgraph(nodes) integration that records LangGraph node crossings
and lets you replay a past run deterministically to regression-test a fix,
no live model calls needed on replay.

https://github.com/theagentplane/chronicle

Happy to answer questions, and genuinely interested in how people here
currently reproduce a specific prod LangGraph failure as a test.
```

---

## 7. AIE WF / Latent Space follow-up (warm, not cold)

Only use this framing, referencing the actual talk, adjust to match what was
actually said in the AIE WF 2026 talk before sending:

```
Hi [name], I spoke at AIE WF 2026 about Chronicle, record-and-replay for agent
decision graphs. Wanted to share what's shipped since: cut-point replay,
LangGraph + OpenTelemetry integrations, pluggable storage (SQLite/remote), and
a benchmark harness with real incident scenarios (numbers in the repo). MIT
licensed, on PyPI as agent-chronicle now.

https://github.com/theagentplane/chronicle

Would love to share it with the Latent Space community if there's a good venue
for it, and happy to do a short follow-up writeup if useful.
```

---

## 8. Reply/comment to Hamel Husain (not cold email)

Draft for a reply on X or a comment on a specific relevant post of his, adjust
to actually respond to what he wrote rather than posting this verbatim:

```
This resonates with something we built Chronicle around: eval-driven
development is great for output quality, but a lot of agent failures are
control-flow/tool-safety bugs (wrong tool call, ungated destructive action,
retry logic), and those need a different kind of test, deterministic replay
of the exact failing trace, not another eval run. Chronicle commits the
production incident as a fixture and lets you cut-point test just the fixed
boundary. Curious whether that distinction (eval failures vs replay-testable
control-flow failures) matches what you're seeing in the course.
https://github.com/theagentplane/chronicle
```

---

## 9. X / Twitter launch post

```
Chronicle: record-and-replay for agent decision graphs.

Turn a production agent failure into a committed regression test. Fix one
boundary, replay the rest from the recording, no live LLM calls needed to
verify the fix.

pip install agent-chronicle
MIT, LangGraph + OTel integrations, cut-point replay.

https://github.com/theagentplane/chronicle
```

---

## 10. Awesome-list PR description (reuse for all five lists)

```
Add Chronicle (agent-chronicle): record-and-replay for agent decision graphs.
Records LLM/tool/routing decisions as immutable Envelopes; supports committing
production incidents as fixtures and cut-point replay testing (stub upstream,
run the fix live, no LLM calls on replay). MIT licensed, pip install
agent-chronicle. https://github.com/theagentplane/chronicle
```

Match each list's existing entry format/section before opening the PR, some
enforce alphabetical order or a strict one-line description length.
2 changes: 2 additions & 0 deletions docs/growth/stars.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
date stars forks watchers open_issues views_14d uniques_14d clones_14d clone_uniques_14d pypi_downloads_month pypi_downloads_week
2026-08-09 10 2 1 7 195 44 190 66 712 29
74 changes: 74 additions & 0 deletions docs/growth/target-list.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Outreach target list

Composition: community venues (where hundreds of interested people self-select) plus
a short list of specific, real, verifiable people worth a personalized note. I did not
force this to 100 individual names — see "Why not 100 individual contacts" at the
bottom. Every entry below is a real, currently-existing venue or person found via
search on 2026-08-09; confidence is marked because "currently exists" and "will
respond" are different things.

## A. Warm lead (start here — not cold outreach)

- **AI Engineer World's Fair (AIE WF) community / Latent Space Discord** — run by
Shawn Wang (swyx); described as the closest thing to a single home for applied AI
engineering in 2026, very active Discord + regular meetups. **Tisha already spoke
at AIE WF 2026** — this is a warm relationship, not a cold pitch. A short "here's
what shipped since the talk" post/message fits naturally.
Confidence: **verified** (community exists, active); the prior-talk relationship is
from our own project history, not a web search.

## B. Awesome-lists (PR submissions — concrete, low-risk, durable backlinks)

Adding Chronicle to a maintained curated list is not outreach in the cold-email
sense — it's a normal, expected OSS contribution (a short PR: one line + one-sentence
description), reviewed by that list's own maintainer on their own terms.

| List | URL | Fit |
|---|---|---|
| aloth/awesome-ai-agents | https://github.com/aloth/awesome-ai-agents | Tracing/evals/observability section |
| benchflow-ai/awesome-evals | https://github.com/benchflow-ai/awesome-evals | Agent-specific evaluation infra |
| goabiaryan/awesome-observability | https://github.com/goabiaryan/awesome-observability | LLM/agent observability tools |
| danielrosehill/Awesome-AI-Evaluations-Tools | https://github.com/danielrosehill/Awesome-AI-Evaluations-Tools | Agentic AI eval tooling |
| Picrew/awesome-agent-harness | https://github.com/Picrew/awesome-agent-harness | Agent harness / tracing / eval analytics |

Confidence: **verified** (all five exist and are actively maintained as of this
search). Each PR should be reviewed by Tisha before submitting — some list
maintainers have strict contribution formats.

## C. Community venues (posts, not cold DMs — people opt in by reading)

| Venue | Why it fits | Confidence |
|---|---|---|
| Show HN (news.ycombinator.com) | Highest-leverage single post for a dev tool at this stage; self-selecting technical audience | Verified (venue exists; outcome is inherently unpredictable) |
| r/LocalLLaMA | Widely regarded as the top subreddit for LLM developers/tooling | Verified |
| r/MachineLearning | Broader ML audience, good for the "record/replay as regression testing" framing | Verified |
| r/AI_Agents | Fastest-growing agent-specific subreddit in 2026 | Verified |
| r/LangChain | Direct audience overlap — Chronicle already has a LangGraph integration | Verified |
| LangChain Community Slack (join-community page) | 30k+ developer community; note it's **Slack now, not Discord** (corrects an earlier assumption) | Verified |

## D. Specific people (short, personal note — not a template blast)

| Person | Why | Contact approach | Confidence |
|---|---|---|---|
| **Hamel Husain** (hamel.dev, evals course w/ Shreya Shankar) | The most prominent current voice specifically on LLM/agent eval methodology; writes extensively on eval-driven development for agents in 2026 — a natural, substantive audience for a regression-testing tool, not a generic observability pitch | Reply/quote to a relevant recent post of his on X (`@HamelHusain`), or a thoughtful comment on a relevant hamel.dev post — **not** a cold email, no personal address found or assumed | Person/relevance verified via search; no personal email sourced (none should be guessed) |

**Why only one confidently-verified individual, not 15-20:** finding a real person's
correct, current, personal contact information requires per-person research I can't
responsibly shortcut — search results surface *articles about* the space, not
verified inboxes. Padding this list with plausible-sounding names and guessed
contact methods would fail the "no fabricated contacts" rule in `program.md`. If
Tisha has existing personal contacts (past conference connections, GitHub follows,
people who've starred/commented already) worth a note, those are higher-confidence
than anything I can source cold — worth adding here from her own network rather
than mine.

## Suggested order

1. AIE WF / Latent Space (warm, already-earned).
2. Awesome-list PRs (five small, low-effort, durable).
3. Show HN (single highest-leverage post — pick a quiet week, have the top comment
response ready).
4. Subreddit posts (can follow within the same week; each needs its own framing,
not a copy-paste of the HN post — see `outreach-templates.md`).
5. Hamel Husain reply/comment, timed to a specific relevant post of his rather than
sent cold with no context.
Loading
Loading