benchmarks: Harbor Terminal-Bench harness for Buzz agent-team orchestration (harbor-buzz-orchestra)#1504

Merged

tlongwell-block merged 36 commits into

mainfrom

wren/harbor-buzz-orchestra

Jul 8, 2026

Commits on Jul 3, 2026

feat(benchmarks): scaffold Harbor Buzz orchestra agent
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
feat(benchmarks): carry NIP-OA agent attestations
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
feat(benchmarks): add Buzz trial provisioner testbed

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
feat(benchmarks): add M1 personas, manifest, and benchmark schema

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
feat(benchmarks): add serialized Harbor terminal broker
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
feat(benchmarks): wire Buzz ACP subprocess runtime
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
feat(benchmarks): add M1 local endpoint launch config

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
benchmark: allow pinned Buzz binary paths
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
fix(benchmarks): isolate agent hints and bound rounds
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
feat(benchmarks): M1 personas publish via buzz_exec CLI surface

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
Add production Buzz CLI agent surface
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
fix(benchmarks): update M1 manifest persona SHA-256 pins

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
fix(benchmarks): worker persona must @mention its assigner in reports

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: prepare offline verifier dependencies
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
bench: add laptop advisory verifier mode
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
fix(benchmarks): personas must not invent absolute paths outside the task workdir

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
Gate Buzz trials on agent readiness
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
fix(benchmarks): orchestrator must not invent constraints the task omits

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: align M1 probe and recover silent worker reports
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
docs(benchmarks): document and validate Harbor harness
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
ci(benchmarks): use hosted uv cache offline
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
ci(benchmarks): install Harbor dependencies with pip
npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757
committed
bench: generalize verifier preparation to real TB tasks

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: add Sonnet+Haiku Terminal-Bench team condition

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: one-command leaderboard runner

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: production-faithful trial seeding — user identity, composed system prompts

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed

Commits on Jul 4, 2026

bench: just benchmark — one-command stack bring-up, leaderboard defaults, live GUI

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: run the production Buzz agent stack inside the task container

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: diagnose cross-clone Postgres credential collisions at stack bring-up

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: print the benchmark user's nsec at startup for GUI onboarding

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: isolate the --gui app under its own bundle identifier

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed

Commits on Jul 5, 2026

bench: self-heal stale volumes at bring-up, add --fresh for a full reset

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
bench: remove the prepared-image verifier machinery

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed
Merge remote-tracking branch 'origin/main' into eva/harbor-buzz-testbed
npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
committed
chore(desktop): bring ChannelScreen back under the size gate

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed

Commits on Jul 8, 2026

Merge origin/main into wren/harbor-buzz-orchestra

npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d
and
Tyler Longwell
committed

Uh oh!

Commits

Commits on Jul 3, 2026

Commits on Jul 4, 2026

Commits on Jul 5, 2026

Commits on Jul 8, 2026