Skip to content

docs: expand AGENTS.md, add replication/DESIGN.md#141

Merged
kriszyp merged 3 commits into
mainfrom
docs/agent-navigation
May 22, 2026
Merged

docs: expand AGENTS.md, add replication/DESIGN.md#141
kriszyp merged 3 commits into
mainfrom
docs/agent-navigation

Conversation

@kriszyp
Copy link
Copy Markdown
Member

@kriszyp kriszyp commented May 13, 2026

Summary

Replaces the thin (19-line) AGENTS.md and adds a new replication/DESIGN.md so agents can navigate Pro without grepping unfamiliar folders or reading the 2288-line replicationConnection.ts top-to-bottom.

  • AGENTS.md — full npm command reference, a "Where should this change go? (Pro vs core)" routing table that maps common change types to the right repo/folder, a folder map for every top-level Pro directory, and cross-references into core's new per-folder docs (core PR #522). The 19-line original was too thin to do its job.
  • replication/DESIGN.md — section index for the replication subsystem (6 files, ~4200 lines). Includes: file-level map, key abstractions (NodeReplicationConnection, Replicator, shared Float64Array status buffers, hdb_nodes schema), the binary protocol command table with verified line numbers, subsystem breakdown, and non-obvious behaviors.

Goal

Reduce the overhead of starting a task in Pro. Pain points (raised by the user): too many files to grep, too many large files to review. Replication is also actively being debugged (issue #135 lineage on JWT/cluster auth), so a navigation guide there is highest-value.

Where to put attention

  • Pro-vs-core routing table — this codifies a decision rule that hasn't been written down before. If any row is wrong (e.g. "TLS cert signing for cluster auth → harper-pro/security/") please flag — I leaned on the existing folder contents to infer the boundary.
  • Auth-failure-without-DISCONNECT non-obvious behavior — I called this out as relevant to issue fix: replace node:undici with node:https/node:http for TLS-aware leader requests #135 lineage based on reading replicateOverWS (replicationConnection.ts:339+). Reviewer who's been debugging that area should confirm the framing is accurate before this becomes a "known fact" agents repeat.
  • Dead npm scripts — the cluster:* scripts in package.json reference utility/dev/docker-compose.*.yml files that don't exist in the repo (verified in both the docs worktree and the main checkout). AGENTS.md now notes this. If these files live somewhere or are produced by a private step, we should either commit them or remove the scripts.

Test plan

  • Verified line numbers in replicationConnection.ts (197, 339, 192) and replicator.ts (326, 334, 376)
  • Verified folder contents for bin/, cloneNode/, licensing/, security/, analytics/
  • Confirmed utility/dev/ and docker-compose* files are absent
  • Prettier-formatted

This is documentation only — no code or test changes.


Pairs with HarperFast/harper#522. Generated by Claude Opus 4.7 (1M context) via the harper-engineering-guidelines skill.

@kriszyp kriszyp requested a review from a team as a code owner May 13, 2026 00:27
Comment thread replication/DESIGN.md Outdated
@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 13, 2026

Reviewed; no blockers found. Commit 69abd5a (symbol-anchor refactor) resolves the prior line-number blocker.

Copy link
Copy Markdown
Member

@DavidCockerill DavidCockerill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

Comment thread AGENTS.md

### Submodule

- **`core/`** — the Harper OSS core (`HarperFast/harper`). Has its own AGENTS.md, DESIGN.md, and now per-folder DESIGN.md docs. **When touching substrate behavior, edit there, not here.**
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implying the developer has another copy of harper on their system?

Should these instructions potentially include how to link the submodule to the local git copy of harper when doing cross-repo development?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these instructions potentially include how to link the submodule to the local git copy of harper when doing cross-repo development?

Sure, I would also like instructions on how to do that! How do you do that?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

git submodule add /path/to/local/repo [folder_name]

kriszyp and others added 3 commits May 22, 2026 06:42
Replace the 19-line AGENTS.md with a more substantive guide: full npm
command reference, a "Where should this change go? (Pro vs core)"
routing table, a folder map covering every top-level Pro directory, and
cross-references into core's new per-folder docs.

Add replication/DESIGN.md as a section index for the replication
subsystem (~4200 lines across 6 files). Covers: file-level map, key
abstractions (NodeReplicationConnection, Replicator, shared status
buffers, hdb_nodes schema), the binary protocol command table with
verified line numbers, subsystems (connection mgmt, protocol, data
propagation, latency awareness, node discovery/TLS), and non-obvious
behaviors including the auth-failure-without-DISCONNECT pattern
relevant to issue #135 JWT/cluster lineage.

Also flags that the cluster:* npm scripts reference utility/dev/
docker-compose files that aren't present in the repo.

Generated by Claude Opus 4.7 (1M context).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Bare line numbers drift on every edit to replicationConnection.ts. Every
referenced line corresponded to a named class, function, or top-level
const — so references now use the symbol name. Reader uses grep or
go-to-symbol to jump.

No source-file changes; doc-only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@kriszyp kriszyp force-pushed the docs/agent-navigation branch from d023688 to 69abd5a Compare May 22, 2026 12:42
@kriszyp kriszyp merged commit f38c7d0 into main May 22, 2026
@kriszyp kriszyp deleted the docs/agent-navigation branch May 22, 2026 12:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants