fix: reconcile control-plane production routes

[samo-agent]

What changed

• coordinate production routing as a two-hop project-VM → control-plane transaction
• snapshot the project route before deploy, prepare and locally health-check the desired project topology first, then update the control-plane route, and stamp state only after project commit
• probe the complete control-plane TLS → configured upstream → project route locally before commit, without public DNS
• require HTTP 200 and exact production version/tag/SHA/environment identity for every static deploy (releaseTag ?? sha)
• keep the old project route live during host renames and roll back both hops on prepare, CP update, reachability, 502, or identity failure
• restore static active-route metadata and prove the snapshotted old host/scheme before removing a failed candidate
• run the two-hop transaction for every cp-http80 deploy, including unchanged Caddy configuration and same-SHA configuration drift
• make new-SHA/tag CI or deploy failures leave both routing hops and deploy/release state unchanged
• preserve staticRoot and its owning worktree across two-hop route transitions and cleanup
• enforce injection-safe app names/IDs at manifest, registration, and generated-script boundaries

Why

Production links require both Caddy hops and the deployed release to agree. Configuration validation alone cannot prove that the control plane can reach the project VM or that the routed static release has the intended identity. Deploy/tag bookkeeping now advances only after the exact path that will serve the client has passed its local end-to-end probe.

Transaction order

1. CI/preflight succeeds.
2. Snapshot the project-VM route, static active-route metadata, and the base Caddyfile for static release apps.
3. Deploy and health-check the candidate release when the SHA changed.
4. Prepare and health-check the desired project-VM topology while retaining the prior route during address changes.
5. Atomically reconcile and validate the control-plane route.
6. Probe https://<mainHost>/... through CP 127.0.0.1:443 with --resolve, preserving production TLS SNI and Host; use the documented internal-TLS insecure policy. Static probes require exact version, tag, SHA, and production environment.
7. Commit the project route, clean the prior static release, then persist the route fingerprint/deployed SHA/release state.

Prepare, deploy, CP update, reachability, non-200, or identity failure restores the CP snippet and project snapshot and leaves state unstamped. Static deploy rollback probes the snapshotted old route and removes the failed worktree only after successful restoration. CI failure never begins a route transaction. A project commit failure remains unstamped and resumable without withdrawing the already-functional transition route.

Safety

• never merges, tags, deploys, or edits production as part of this PR
• the end-to-end probe is CP-local and bypasses public DNS and proxy environment variables
• Caddy snippets are staged, validated, atomically renamed, and restored on failure
• per-app hashed transaction/transition paths keep apps on a shared VM independent
• unsafe app identity payloads fail closed before reaching Caddy, shell, systemd, or path generation
• hand-authored control-plane host collisions fail closed

Validation

• bunx tsc --noEmit
• bun test — 1,881 pass, 0 fail, 7,219 assertions across 85 files on head f4fac13e18171c4caf63b4ff3f1b9bbc371bcc91
• executed generated CP and project transaction scripts for unreachable upstream, HTTP 502, wrong branch/release static identity, exact-identity success, and both-hop rollback
• executed old-host → new-host and cp-http80 → tls static failure rollback, proving old-route re-probe and failed-candidate cleanup ordering
• generated control-plane and project-VM scripts pass bash -n; generated route passes the installed Caddy validator

[samo-agent]

samorev Code Review Report

• PR: fix: reconcile control-plane production routes #153 - fix: reconcile control-plane production routes
• Author: @samo-agent
• AI-Assisted: Unknown

Pipeline	Coverage
PASS	Not reported

---

No issues found. Reviewed for security, bugs, tests, guidelines, and documentation.

Result: PASSED

---

Summary

Area	Findings	Potential	Filtered
CI/Pipeline	0	0	0
Security	0	0	0
Bugs	0	0	0
Tests	0	0	0
Guidelines	0	0	0
Docs	0	0	0
Metadata	0	0	0

Note:

• Findings: High-confidence issues (8-10/10) - blocking or non-blocking per severity
• Potential: Medium-confidence issues (4-7/10) - review manually
• Filtered: Low-confidence issues (0-3/10) - excluded as likely false positives

Review metadata

provider=github
kind=pr
project=NikolayS/samohost
number=153
target=github:NikolayS/samohost#153
state=OPEN
draft=false
diff_lines=5122
diff_added=4001
diff_removed=154
diff_bytes=226449
comments_count=0
commits_count=16
ci_status=success
ci_summary=total=1 success=1 failure=0 pending=0 other=0
prompt=.claude/commands/review-mr.md
blocking=true
posted_by=gh
no_comment=false
live_posting=posted

---

samorev-assisted review (AI analysis by Tanya301/samorev)

[samo-agent]

BLOCKED / DO NOT MERGE this mixed head (5d30b46). Independent exact-head review found release blockers despite green CI and the earlier automated gate: unsafe root-helper reads/writes of app-controlled state (including symlink/TOCTOU), missing safe adoption of existing TEMP control-plane routes, stale bootstrap-frozen helper config after manifest drift, missing central appUser validation, same-SHA ownership mismatch, and over-broad generic :80 legacy migration. We are splitting the work into narrow PRs: firewall wiring; Linux app-user validation; control-plane/admin routing; then shared-VM execution boundary. This PR remains draft as evidence/source material only until the safe routing slice replaces it. No live rollout should use this head.