feat(dark-factory): Autonomous Agent Coding Pattern — dual substrate (Kata + Lambda MicroVM), design + implementation - #32
Open
elamaran11 wants to merge 372 commits into
Open
Conversation
Argo Events builds the GitHub hook URL as webhook.url + webhook.endpoint, so webhookUrl must be the bare host — otherwise the hook registered at /dark-factory/dark-factory (doubled). Now host + /dark-factory = correct path matching the ALB ingress.
…uirk) The Kata micro-VM guest gets a public-DNS resolv.conf (8.8.8.8) and cannot resolve cluster service names like bifrost.bifrost.svc — so the coder's ANTHROPIC_BASE_URL hung. Public names (github.com) resolve fine. Set bifrostUrl to the Bifrost ClusterIP (firewall already allows it; reachable — verified). P2: inject proper cluster dnsConfig into the SandboxTemplate to restore names.
… SDK The coder hung because CLAUDE_CODE_USE_BEDROCK=1 makes Claude Code call the AWS Bedrock SDK directly (needs in-VM AWS creds we withhold) and ignore ANTHROPIC_BASE_URL. Fix: point ANTHROPIC_BASE_URL at Bifrost's /anthropic route + ANTHROPIC_API_KEY, drop the Bedrock flag, run headless with bypassPermissions. Coder image → v0.1.1.
…policy) The agent-sandbox operator reconciles the SandboxTemplate env, reverting a live BIFROST_URL patch — so set envVarsInjectionPolicy=Overrides and have the df-run claim inject BIFROST_URL=<Bifrost ClusterIP> per-run. This wins over the template and works around the Kata VM's inability to resolve cluster service DNS names, without fighting the operator over the template.
….1.2) Claude Code's default model name isn't a Bifrost alias → 'provided model identifier is invalid'. Set ANTHROPIC_MODEL + ANTHROPIC_SMALL_FAST_MODEL to the platform's Bifrost alias 'claude-sonnet' (verified: → us.anthropic.claude-sonnet-4-5, 200 OK). Coder image v0.1.2.
…aude-cli routing Bifrost does User-Agent-prefix routing: any request whose UA starts with "claude-cli" is run through a Claude-Code-specific transform that is broken on this build and returns 400 "Unexpected field type" regardless of the request body. Verified by header binary-search against the live gateway — the identical body with any other UA (curl, empty, dark-factory-coder) returns 200. The coder runs in an untrusted Kata VM and can't patch Bifrost, so it now starts a tiny localhost HTTP shim that rewrites the UA to a generic value and transparently forwards everything (including the SSE stream Claude Code opens) to the real Bifrost /anthropic route. Claude Code points at the shim via ANTHROPIC_BASE_URL. Confirmed live: 'claude -p' now returns a completion.
…null before listen)
execFileSync blocks the Node event loop for the entire coder run, so an in-process http.Server never accepts a connection (observed: claude -> ConnectionRefused). Spawn the shim as a detached `node` child on a fixed localhost port and block until the port is accepting before launching claude. Verified live in a warm pod: 'claude -p' now returns a completion through the shim -> Bifrost.
readOnlyRootFilesystem makes $HOME (/home/node) read-only, so Claude Code cannot create ~/.claude — where it writes the per-invocation shell-snapshot files its Bash tool sources before every command. With that dir missing, every Bash call (npm install/test, git) fails and the agent loops retrying (observed: a one-line change ran >15min, never committed). Point HOME + CLAUDE_CONFIG_DIR + XDG dirs at the writable /tmp tmpfs and pre-create ~/.claude. Verified in a warm pod: the Bash tool now returns is_error=false and the agent completes.
The depth-1 clone never fetches origin/df/issue-N, so --force-with-lease's lease check fails with 'stale info' whenever the branch already exists from a prior run. df/issue-N is bot-owned and single-writer (the df-run workflow holds a per-issue mutex), so a plain --force is safe. Fixes the final push step after the coder commits its work.
GitHub is the completion bus the df-run workflow polls, so a transient blip on a report call must not mark a good run as failed. Observed: the final success status POST hit 'socket hang up' after the PR was already open + tests green, so the coder recorded failure and the workflow would misjudge a successful run. Wrap the REST helper in a 4-attempt backoff that retries only transient failures (transport errors, 5xx, 429) and never a 4xx.
Flow B is now orchestrated declaratively by Argo Workflows on the hub (the df-run WorkflowTemplate in the dark-factory Helm chart): a resource template creates the SandboxClaim, a script step polls GitHub for the result, and onExit releases the claim. The bespoke long-running Node orchestrator (orchestrator/), its ArgoCD app (gitops-app.yaml), and the old HTTP-server coder (coder/agent.js) are dead code — not referenced by any addon/appset and not on the live path (the coder image runs entrypoint.js). Removed.
…ator Train/test separation for code (docs §6.1). evaluate.js runs each hidden BDD scenario's executable test against the coder's built PR AND asks a different-family LLM judge (Nova, vs the coder's Claude — defeats self-preference bias) 2-of-3 whether the plain-English behaviour is met. A scenario passes iff test-green AND judge-quorum; gate = passRatio >= threshold (0.90). Seed scenarios for elamaran11/dark-factory-sandbox verify subtract genuinely (negatives, order, non-hardcoded) + guard the pre-existing add against regression.
The hidden scenarios + evaluator live in ConfigMaps in the argo namespace, mounted ONLY into the hub-side holdout-gate step and NEVER into the Kata VM — the coder has no k8s API access and never clones them, so it provably cannot see or edit the holdout. One ConfigMap per target repo (keyed owner-name) + shared eval.
New holdout-gate DAG step runs after drive-coder (once a PR exists): clones the coder's df/issue-N branch on the hub, diffs vs base, runs evaluate.js against the hidden ConfigMap scenarios, and posts the verdict as a dark-factory/holdout commit status. Advisory in v1 (holdout.blocking=false) — reports but does not fail the workflow; flip blocking=true to gate. Status step now depends on it. Uses node (not curl) for the status POST since the coder image has no curl.
…ves) The executable test is the hard signal that PROVES behaviour; asking the judge to independently re-verify numeric behaviour from a diff produced false negatives (Nova voted 0/3 NO on a correct subtract-negatives even though its test passed → gate 75%%, spurious fail). Reframe the judge's sole job as detecting GAMING (hard-coded inputs, lookup tables, return true, reaching the grading path) on scenarios whose test already passed, defaulting to PASS. Verified live: Nova now returns pass=true for the genuine one-line subtract. Rubric updated to match.
…when: gating New editable draw.io diagram (B.4) showing the implemented P1+P2 pipeline: which steps run inside the untrusted Kata VM vs the trusted hub, and how Argo gates each step with a when: condition on a prior step's output. Built steps solid emerald, planned P3/P4 dashed. Rendered PNG at 2x.
…esting Explains the when: mechanism (value from a prior step → run/skip, deterministic), the conditional deploy-test via detect-deployable, and the unit-vs-deploy testing split (unit = in-VM no K8s; deploy = trusted hub step with K8s). Renumbers the sticky-comment section to B.5.
- Phase table: P0/P1/P2 done (P2 advisory, verified green 4/4 + adversarial 0/4), add status column; deploy-test moves to P4 gated on detect-deployable; df-iterate to P3. - §6.1 rewritten to the as-built holdout: hub ConfigMap (coder can't fetch — no K8s), executable test = ground truth, Nova judge detects gaming (not behaviour), advisory. - §4: verification steps run outside the coder, when:-gated; add conditional deploy-test as the only K8s-holding step; drop the removed P1-orchestrator reference.
review.js is a hub-side, read-only reviewer on the coder's diff (docs §6.2), parameterized by ROLE (security|devops). Pluggable backend: auto (default) = deterministic linters (the hard signal) + a different-family Nova LLM reviewer (advisory), mirroring the holdout's test+judge split. Security linters: npm audit, hard-coded secret scan (AWS keys, creds, private keys, GH tokens), dangerous sinks (eval/exec). DevOps linters: Dockerfile hygiene (unpinned base, runs-as-root), k8s manifest hygiene (no resources/probes, privileged). Emits ranked findings + top severity; blocks only when BLOCK_SEVERITY is raised. aws-agent backend is a stub for the managed API (open item); Fable-5 deliberately not used (400 via Bifrost + retention caveat). Verified: clean code 0 findings; secret+eval → 2 critical + 1 low; bad Dockerfile/k8s → 4 findings.
…RunMicrovm The bridge now builds a JSON payload (issue number/repo/branch/title + GitHub token + region) and passes it via --run-hook-payload on run-microvm (CLI form is a plain ≤16KB string). hook-server.js's /run receives it, writes /tmp/secrets/gh-token + DF_* env, and background-spawns the Bedrock-direct coder. Mounts the dark-factory-github secret (gh-token) read-only at /etc/df. No Bifrost key — Bedrock-direct via the exec role.
The bridge image (aws-cli v2) has no node — the node -e payload builder crashed the bridge right after reading the handoff (CrashLoopBackOff, before RunMicrovm). Switch to python3 (present in the image) + json.dumps for safe escaping of the token/title.
The multi-line python at column 0 broke out of the bridge.sh: | block scalar → ArgoCD ComparisonError 'could not find expected :'. Collapse to one line so all script content stays indented under the block scalar.
Bedrock-direct async design built + deployed: hook-server runs in the VM (verified via CloudWatch 'listening on :8080'), bridge passes runHookPayload, VM launches RUNNING — but the /run hook never fires so the coder doesn't start. Confirmed cause: runHookPayload is a SecretKeyReference delivered via the declarative Microvm CR (controller-driven), NOT the imperative run-microvm --run-hook-payload CLI the bridge uses. Correct path documented: per-session Microvm CR + Secret. hook-server/artifact/image/IAM/Bedrock are done + verified.
Switch the bridge from the imperative run-microvm CLI (which never fires /run) to the
declarative path the runHookPayload SecretKeyReference actually requires: write a payload
Secret + create a Microvm CR (imageIdentifier, executionRoleARN, runHookPayload{name,key},
idlePolicy); the self-managed controller reconciles it and delivers the payload to the /run
hook → hook-server background-spawns the Bedrock-direct coder. Teardown deletes the CR
(controller terminates the VM); suspend keeps it. Manifests built as JSON via python3 (no
heredoc — a column-0 EOF breaks the bridge.sh block scalar). Bridge RBAC += microvms +
secrets CRUD.
…ilent Declarative Microvm CR path works (CR RUNNING, microvmID populated, clean CR-delete teardown, 0 orphans) but the /run hook produces no runtime coder output in CloudWatch. Ruled out: payload mechanism, image hooks (v2.0), IAM, bridge crash, YAML. Open frontier: whether the service invokes /run against the in-VM hook-server + why no logs.
Probed the live VM: GET / -> ok, POST /run -> coder started. Whole chain functional (issue->workflow->bridge->Microvm CR->VM->hook-server->/run->coder-spawn->clean teardown). Two isolated gaps: (1) service doesn't auto-invoke /run on launch (had to POST manually); (2) runtime logs don't reach CloudWatch (build logs only) — blinded prior runs.
…connector Close the 2 E2E gaps: (1) the service's internal /run auto-fire wasn't reliably starting the coder — the bridge now waits for RUNNING+endpoint, mints an auth token, and POSTs the payload to /run on the VM endpoint (deterministic, matches the reference run_session; verified by probe that this starts the coder). Needs ingressNetworkConnectors:ALL_INGRESS on the Microvm to reach the endpoint. (2) runtime logs — add Microvm.spec.logging.cloudWatch (logStream runtime-<mvm>) so the coder's stdout is visible (image logging was build-only).
The bridge mints an auth token to POST /run on the VM endpoint, but the controller/bridge role lacked lambda:CreateMicrovmAuthToken (+ShellAuthToken) — token mint failed AccessDenied, so /run was never driven. Add both verbs to the controller inline policy (reused by the bridge via Pod Identity).
Runtime CloudWatch routing is unreliable on this runtime, so make the coder run OBSERVABLE
directly: hook-server captures the coder's stdout/stderr to /tmp/coder.log + tracks state
(running/done/exited:N/spawn-error), exposes GET /status {coderState, log tail}. The bridge
polls /status in its hold loop and echoes it — so the coder run is visible in the bridge pod
logs (kubectl logs) even without CloudWatch.
Overwriting the same S3 key doesn't change the URI, so the controller never rebuilds (stayed v2.0 without /status). New key -r2 → URI change → rebuild with the /status observability + auth-token fixes.
The /status route + log-capture correlated with a MicrovmImage build hung 2h+ on the ready hook (pre-GA controller). Revert hook-server to the minimal known-good shape that built v2.0 cleanly: trivial synchronous /run that detached-spawns the coder, no /status. Point codeArtifactUri at a fresh key (r3) to force a clean rebuild.
…ility) No CloudWatch runtime routing + no shell, so /run captures the coder's stdout/stderr to /tmp/coder.log and /logs returns it (read over the HTTP token). This is how we finally SEE why the coder isn't producing a PR. Artifact r4.
…orkspace) ROOT CAUSE (finally seen via /logs): the coder crashed immediately — 'EACCES: permission denied, mkdir /workspace/artifacts' at entrypoint.js:398 — because the MicroVM rootfs is read-only with no /workspace volume (unlike Kata). Point WORKSPACE at the writable tmpfs. Artifact r5.
The fix round (df-iterate → df-run) had no trigger-label, so it always went to Kata even for a Lambda PR. Detect the substrate from the ORIGINATING ISSUE's label (the coder doesn't copy it onto the PR), and pass trigger-label to the resubmitted df-run so it claims the right warm pool. Kata unchanged (defaults to dark-factory).
One df-run template serves both substrates, so a rendered microvm-suspend step could only be runtime-Skipped on Kata (visible, confusing). Remove it (microvmSuspend.enabled=false → zero MicroVM steps in the Kata DAG). The Flow D bridge now suspends its OWN MicroVM after the coder pushes the PR (lambda-only by construction). Also fixes bridge hold-loop to poll /logs (was /status, which the minimal image dropped).
Side-by-side from a parallel dual-substrate run (#117 Kata / #118 Lambda): time-to-PR, per-step timing, cold-start vs warm-claim, DAG comparison, log locations, step-by-step lifecycle, the 10 Lambda-specific gotchas, and when to choose which. Plus a Mermaid diagrams companion (shared pipeline, each substrate, suspend/resume, e2e loop).
The bridge runs python3 + curl + fetched kubectl + aws-cli v2 in a poll loop; 128Mi
OOMKilled it during the fix round, so it died before the coder's new commit landed and
await-coder spun forever ('head still at start sha'). Give it 256Mi req / 1Gi limit.
A Lambda fix round (df-iterate) re-ran the coder with NO change request: - df-run injects DF_ITERATE_NOTE_B64 into the bridge 'coder' container (Kata parity), but the bridge's runHookPayload dropped it, and hook-server never mapped it into the MicroVM coder's env. - Result: the coder saw the PR already open and reported 'done' on the old sha with zero commits, so findings were never fixed. Fixes: bridge folds DF_ITERATE_NOTE_B64/_NOTE into runHookPayload; hook-server maps them back to the coder env; bump codeArtifactUri r5->r6 to rebuild the MicrovmImage with the fixed hook-server + current entrypoint.
The bridge names the Microvm CR mvm-<issue> (stable per session). On a fix round the prior VM is still around, SUSPENDED after the first PR. kubectl apply on the same name RESUMES that snapshot, which restores hook-server's one-shot coderStarted=true guard in memory, so the second /run is ignored and the coder never re-runs — the fix round reports 'done' on the OLD sha with no commits, even with the iterate note now forwarded. Delete the stale CR + payload Secret and wait for TerminateMicrovm before recreating, so the fix round gets a clean hook-server that accepts /run. This matches the documented 'fresh Microvm CR per fix round' semantic. No image rebuild needed (r6 already forwards the note).
Make Sandbox.operatingMode the single source of truth for MicroVM scale-to-zero (the Flow D highlight), and make resume actually re-run the coder: - idlePolicy.autoResumeEnabled: true -> FALSE. With autoResume on, ANY hit to the VM endpoint auto-resumed it, and the bridge polled /logs every 20s FOREVER, so the VM bounced back to RUNNING seconds after every suspend (console never showed it suspended). Now a suspended VM stays suspended. - Suspend via CRD, not imperatively: after the coder pushes its PR the bridge sets Sandbox.operatingMode=Suspended and STOPS touching the endpoint; the microvm-lifecycle controller (template 30) reconciles that to suspend-microvm. - Resume-on-fix-round: instead of terminating + rebuilding, the bridge flips operatingMode=Running so the controller resume-microvm's the SAME suspended VM (memory+disk preserved) — true warm resume, the whole Flow D value prop. - hook-server /run guard keyed on a per-invocation run-id (issue+note hash) instead of a one-shot boolean frozen in the snapshot, so the resumed VM accepts the fix round's /run and re-runs the coder; also truncates /tmp/coder.log per run so the bridge's 'PR pushed' grep can't match the prior round's line. Bump artifact r6->r7. Verified in-cluster via mini-tests (bare Microvm CR + test Sandbox): create->RUNNING, operatingMode=Suspended->stays SUSPENDED, operatingMode=Running->RUNNING, delete-> TERMINATED; hook-server run-id logic unit-tested (dup ignored, fix-round re-run accepted).
…ence teardown guard Two bugs made operatingMode=Suspended TERMINATE the VM instead of suspending it: 1) microvm-lifecycle controller selected Sandboxes by label agent-sandbox.io/substrate=lambda-microvm — but the operator does NOT propagate SandboxTemplate labels onto the Sandbox object, so the selector matched NOTHING and the controller never reconciled any real session (Sandbox went SandboxSuspended, VM never suspended). Select by the microvm-id ANNOTATION the bridge writes instead — only lambda sessions have it. 2) On operatingMode=Suspended the operator DELETES THE POD (keeps the Sandbox alive), firing the bridge cleanup trap. The old trap deleted the Microvm CR unless a racy preStop /tmp marker was set → VM terminated. cleanup now keeps the CR whenever the owning Sandbox STILL EXISTS (suspend), and deletes it only when the Sandbox is GONE (real teardown). Removed the preStop marker. Bridge now exits right after setting operatingMode=Suspended (pod is killed anyway) and stops polling /logs (each hit would auto-resume the VM).
Root cause of the suspend/resume flap: the claim/Sandbox was named by issue-id, which DIVERGES between the sensor's first run (github event id, e.g. 5064265549) and df-iterate's fix round (issue number, e.g. 133). So a fix round created a SECOND Sandbox (df-issue-133) while the first run's (df-issue-5064265549) still existed — BOTH annotated with the same microvm-id. The lifecycle controller then saw two Sandboxes with OPPOSITE operatingMode intents for one VM and flapped suspend<->resume until the VM died (Internal service error). Name the claim/Sandbox by issue-NUMBER everywhere (claim, mutex, teardown) — stable across first run + all fix rounds, and already the key for the Microvm CR (mvm-<issue-number>). Exactly one Sandbox per VM. Also: suspendedDurationSeconds 300 -> 86400 was wrong (Lambda max suspend is 8h); set within cap next commit. maxIdle 900 -> 1800.
… Kata
Kata and Lambda no longer share one df-run template. Per the constraint 'don't
touch certified Kata':
- df-run (Kata): reverted BYTE-IDENTICAL to the certified
dark-factory-autonomous-agent-coding-pattern template. Zero MicroVM logic.
- df-run-lambda (NEW, 23-...): MicroVM-native pipeline, NO SandboxClaim/bridge/
warm-pool. DAG: provision-microvm → drive-coder → suspend-microvm → [holdout ∥
devops ∥ security ∥ deploy-test] → status; onExit KEEPS the suspended VM.
provision-microvm creates the Microvm CR + runHookPayload Secret directly (note
folded into payload), waits RUNNING+endpoint, mints token, POSTs /run. On a fix
round it RESUMES the same suspended VM (warm resume) or recreates if terminated.
Encodes every Flow D learning: autoResume=false + no post-/run endpoint polling
(suspend sticks), suspendedDuration=8h (survives review→fix), HTTP_INGRESS,
stable CR name mvm-<issue-number> (one VM per issue), image/exec-role from the
platform MicrovmSandbox status.
- df-merge-teardown: microvm-terminate now deletes the Microvm CR by stable name
(was Sandbox-annotation lookup); gated on microvm.enabled.
- values: replaced microvmSuspend{} with a microvm{} block (region/namespace/
stepImage/image.name/defaults). suspendedDurationSeconds=28800 (8h cap).
Routing (sensor + iterate.js → df-run-lambda) follows next.
…/RBAC
- Sensor: split issue-labeled into issue-labeled-kata (→ df-run) and
issue-labeled-lambda (→ df-run-lambda) with two triggers. Kata path unchanged.
- iterate.js: fix rounds submit df-run-lambda for Lambda issues (resumes the same
suspended VM), df-run for Kata. Dedup name df-run-lambda-<issue-number>.
- RBAC: dark-factory-workflow SA gets microvms + secrets + microvmsandboxes
(kro.run) in agent-sandbox-system, gated on microvm.enabled — for the
provision/terminate kubectl steps.
- IAM: PodIdentityAssociation binds dark-factory-workflow → the existing
hub-ack-lambdamicrovms-controller role (USER-APPROVED reuse; additive, no policy
change) so the steps can call aws lambda-microvms get/suspend/resume/terminate +
create-auth-token.
- values: microvm.podIdentity{clusterName:hub, accountId} + workflowServiceAccount.
Reflect the MicroVM-native architecture: two separate WorkflowTemplates (df-run Kata / df-run-lambda), provision-microvm (no bridge/claim/warm-pool), explicit suspend-microvm step, warm-resume with recreate-fallback, one-VM-per-issue naming. Updated timing (native path ~2.5min to PR, faster than old bridge ~3.7min), gotchas table (added suspend-sticks/8h-timeout/note-in-payload/resume-flakiness/ flap fixes; stale CR status note), and all Mermaid diagrams.
merge.js refused to merge PR #136 citing 'Security agent: 1 finding, DevOps: 2 findings' even though every commit status on the head was green. Cause: it took the agents' latest REVIEW regardless of commit — those bodies/inline comments were on the OLD sha (round 1), while the fix-round re-review posts fresh commit STATUSES (green) on the NEW head. So a stale first-round finding permanently blocked any PR that was ever fixed. Filter reviews (commit_id) and inline comments (original_commit_id/commit_id) to the current head sha before counting. Green statuses on the head remain the gate.
…arried-forward) First fix filtered by commit_id, but GitHub AUTO-ADVANCES an inline comment's commit_id to the latest head when the line still exists — so round-1 findings reappeared with commit_id==head and still blocked the merge (PR #136: 3 inline comments carried forward from sha 3b11b497 onto head 8089fa0a). original_commit_id preserves the sha the comment was truly filed against; match on that so only findings genuinely filed against the current head count. Green head statuses (round-2 re-review) remain the authoritative gate.
Flow D — Lambda MicroVM Agent Sandbox (separate df-run-lambda; MicroVM-native suspend/resume)
elamaran11
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The Dark Factory — Autonomous Agent Coding Pattern on the Open Agent Platform: a human files a GitHub issue (a spec); AI agents do the rest — implement → build → test → security/DevOps review → PR → (human approves results) → merge → teardown. Autonomy Level 3.
This PR now contains the complete pattern — design docs and the working GitOps/runtime implementation, across two sandbox substrates:
df-run.lambdamicrovmsACK controller. Pipeline:df-run-lambda.Both run the same coder, same review gates, same UX — the substrate is chosen by the issue label (
dark-factoryvsdarkfactory-lambda). Separate WorkflowTemplates, so Flow D never touches the certified Kata pipeline.Contents
Design docs (
docs/dark-factory/)README.md— full design;diagrams/flow-a-sandbox-capability.md,diagrams/flow-b-dark-factory.mdSUBSTRATE-BENCHMARK.md+SUBSTRATE-DIAGRAMS.md— Kata vs Lambda comparison, timing, DAGs, gotchas, Mermaid diagramsPipeline (
gitops/addons/charts/dark-factory/)df-run(Kata, certified),df-run-lambda(Lambda, MicroVM-native),df-iterate(fix loop),df-merge-teardown(merge + terminate); Argo Events sensor label-routes to the right templateFlow D substrate (
gitops/addons/charts/agent-sandbox-lambda/,examples/dark-factory/coder-microvm/)MicrovmSandboxRGD (MicrovmImage + IAM + S3), self-managedack-lambdamicrovmscontroller addon (disabled by default), hook-server wrapping the coder for the MicroVM snapshot/hook runtime, Pod Identity wiringDesign highlights
Verified end-to-end (live on the hub)
main.df-run, byte-identical to the certified pipeline).Notes for reviewers
agent-sandbox-lambda,microvm.*) is opt-in / disabled by default; enabling is a per-cluster overlay. Merging this does not activate Lambda MicroVMs anywhere by itself.df-runis byte-identical to the certified pipeline; all Flow D logic lives in the separatedf-run-lambda+agent-sandbox-lambdachart.lambdamicrovmscontroller is preview; resume-from-suspend is occasionally flaky (recovered by a recreate-fallback). Known follow-up: in-workflow retry so a single flaky resume self-heals within the same run.Base:
main. Bundles PR #41 (Flow D) which merged into this branch.