Port backend remainder: submissions, orchestrator, reports, host-local signals, settings admin (#1612) - #1613
Merged
Merged
Conversation
Services control (Unix-socket passthrough to hp-services-adapter), config revision history + rollback, an audit trail, per-subject preference sync (GET/PUT /api/v1/preferences, POST .../reset), reporter stats passthrough, and a user-retention-sweep worker loop. All new ES-backed stores work at the JSON Value level like the existing config.rs, not the full typed dashboardConfig/behavior/honeypot patch machinery. backend-service now mounts dashboard-state and services-adapter-socket directly (tier decision recorded in compose.yml). Deliberately out of scope for this pass: frontend wiring (BFF/UI still reads localStorage for prefs), full typed config PATCH with pinned-field source tracking and impact-classification preview, and ETag/If-Match optimistic concurrency on preferences (nothing else in this tier has that yet either). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Extends the alert-notifier worker loop with the seven signals that need a host mount or service URL rather than pure ES: OT command detection (T1692.001, actually ES-only via honeypot.canonical_attck_techniques), log-stream size (#1389 rotation guard), sandbox/ghidra/cape/ github-analysis spool + worker-status health (with ghidra/cape's live-spool recheck against a stale status.json), ghidra/github-analysis findings thresholds, and Filebeat ingestion stats via FILEBEAT_URL. backend-worker gains read-only bind mounts for /logs and the sandbox/windows-sandbox/ghosts-sandbox/ghidra/cape/github-analysis request+results spools (mirroring the dashboard service's existing mounts), plus the corresponding env vars. Every subsystem stays silent when its directory env var is unset, matching the Go tier's "no noise for an unopted-in subsystem" posture. Left out: an in-memory IngestState (the Go tier's ingest=/age= half of the "ingestion unhealthy" message) — this pass covers the Filebeat half only; write request-dir mounts (submission) are Phase 3's concern, these are read-only. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…heal (#1612 phase 3a) Payload classification (payload_kind.rs, hand-rolled PE/ELF header checks instead of a new parsing crate) and payload-dir path resolution (payload_paths.rs, PAYLOAD_DIRS/BINARIES_DIR/SCRIPT_PAYLOAD_DIR) as shared foundations, then three submission endpoints on top: POST /api/v1/sandbox/submit (classifies + routes to linux/windows/ghosts), /api/v1/ghidra/submit, /api/v1/github-analysis/submit (requires confirm=publish, every outcome audited) plus GET /api/v1/sandbox/golden-image-status. Adapted from Go's form-POST-redirect flow to plain JSON for the BFF. New backend-service-mounted compose service: same image/route table as backend-service, but with read-write access to the sandbox/ghidra/ github-analysis request spools — a distinct trust/mount boundary kept separate from backend-service's Phase-1 (state+socket) scope and from backend-worker (loops only, not network-reachable). Internal URL http://backend-service-mounted:8082; BFF wiring is a follow-up for the frontend track. Also wires payload_bytes.rs's on-demand mirror self-heal into payload_detail.rs: a hash not yet in dashboard-payload-bytes-v1 is now mirrored from disk on demand instead of coming back empty until payload-inventory-worker's next scan. Needed read-only payload-dir mounts added to backend-service itself (single-instance-safe, unlike the mounted service's write access). Deliberately simplified from the Go tier: no SHA-256-vs-Dionaea-MD5 fallback path resolution (payloadPathBySHA256), no byte-fingerprint staleness recheck on an already-mirrored payload. Phase 3b (Payload Workbench orchestrator) builds on payload_kind.rs/payload_paths.rs next. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Recipe-driven and one-off analysis runs across ghidra/linux-sandbox/
windows-sandbox/windows-ghosts/revdeck, ported from workbench_domain.go/
workbench_es.go/workbench_orchestrator.go onto backend-service-mounted
(phase 3a's spool-write service). A run's own document id is its
idempotency key, so creation is one atomic Elasticsearch op_type=create
(new es.rs primitives: index_doc_create/index_doc_cas/get_doc_meta) --
cross-instance dedup without a process-local mutex. Reconciliation reads
ghidra/sandbox/revdeck results straight off disk, never through their ES
mirrors, so a just-completed job's state change is never hidden behind an
import interval.
New: GET /api/v1/workbench/analyzers, POST/GET /api/v1/workbench/runs,
GET /api/v1/workbench/runs/{id}, POST .../children/{analyzer}/{action}
(cancel/retry), GET/POST /api/v1/workbench/recipes. Identity travels as
plain owner params (Phase 1's established BFF-supplied-identity
precedent), not a new header scheme.
"cape" stays selectable/validated but has no submission adapter -- mirrors
a real gap in the Go source today, not an omission here. "deterministic"
reports a clear not-yet-implemented failure rather than a fake result:
this tier has no YARA/entropy/IOC engine yet. No trueSHA256/
staticAnalysisFor (ghidra/revdeck submissions use the run's own SHA-256
directly -- same Dionaea MD5-vs-SHA256 gap phase 3a's payload_paths
already has) and no cross-process flock on the per-directory queue-depth
check (backend-service-mounted is already single-instance by phase 3a's
own tier decision).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Hand-rolled, dependency-free PDF 1.4 writer ported from report_pdf.go + report_pdf_brandmark.go + report_pdf_watermark.go: dark/light theme palettes, branding defaults, the full report-element set (cover, metric grid, assessment, findings/recommendations, top-N tables, operational alerts, event appendix, parameters), watermark + header-mark image masks, and the raw object/xref/trailer assembly — same fixed object numbering as the Go source (watermark=6, its ExtGState=7, header-mark=8, pages from 9). No new dependency: this is plain byte/string formatting on both sides of the port, same as the Go original. The two embedded 1-bit image masks (assets_pdf/*.maskdata, pre-FlateDecode compressed) are copied byte-for-byte from the Go tier and embedded via include_bytes!. Verified independently with poppler (pdfinfo/pdftotext, not just internal self-consistency): a 23-page multi-section sample parses cleanly, correct page count/size/version, text extracts correctly. Pure module, no ES/HTTP dependency of its own — render_report_pdf(data, theme, branding, elements, appendix_limit) is the entry point. Not yet wired to a route: reportDataFor's ES-backed data gathering, the definitions CRUD API, generate endpoint, and scheduler are phase 4b. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…er (#1612 phase 4b) Definitions CRUD (dashboard-reports-definitions-v1, singleton doc like config.rs/preferences.rs, not Go's generic esSettingsStore[T]), the 9-entry template + 14-entry element catalog, and the on-demand generate endpoint, built on phase 4a's pure PDF composer. reportDataFor's real translation: Go filters an in-memory event cache this tier doesn't have, so the summary/top-N/findings dataset is rebuilt as ES aggregations (reports_data.rs) using the field names events.rs/ aggregates.rs/dashboard.rs already established, instead of in-memory map-counting. Operational-alert matching and event-appendix rows are ported field-for-field from reportAlertMatches/eventAppendix. reports-scheduler joins backend-service's WORKER_LOOPS (ES-only, no host mount, same tier as user-retention-sweep): every 30s, render due definitions through the same pipeline as manual generate and advance next_run_at (daily/weekly/monthly, ported from nextScheduleRun) — success advances last_run_at too, failure only advances next_run_at so it never hot-loops. Scope decision: the sandbox/payload/ghidra templates render one referenced artifact through dedicated renderers (sandbox_pdf.go/ghidra_pdf.go/a payload equivalent) this pass does not port. They validate and save correctly; generating one returns 501 with a clear message instead of attempting to render. The 6 generic templates (executive/security/threat/ incident/sensors/custom) work end-to-end. Also: es.rs gains delete_doc (generated-report retention pruning). 7 new unit tests (validation + schedule-math) alongside phase 4a's PDF tests, all passing. Verified compose.yml resolves cleanly with the new WORKER_LOOPS value. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…phase 4b review) reportDataFor's risk_level helper diverged from payload_analysis.go's canonical riskLevel scale (this dashboard's one shared risk-word scale, also used by sandbox/payload scoring) — was missing the "critical" tier and used different score boundaries (>=70/>=40 instead of >=75/>=50/>=25). Also fixed the "payloads" aggregation reading honeypot.sha256, a field that doesn't exist in the indexed events — the correct field (confirmed against fusion.rs's existing usage and the Go classify.go source) is honeypot.shasum. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The phase 4a composer emitted raw PDF-1.4 bytes by hand (manual object numbering, xref table, content-stream strings) to mirror the Go source's own zero-dependency approach. Per explicit request, replaced it with the vendored printpdf crate (0.12.6, default-features off to skip its HTML/text-layout dependency chain — azul-layout/taffy/hyphenation — none of which this module needs): object/xref/trailer assembly, content-stream encoding, and PDF conformance now come from a maintained library instead of hand-rolled byte formatting. Public API (render_report_pdf, PdfTheme/PdfBranding, ReportData and friends, ELEMENT_* constants) is unchanged, so reports_data.rs/ reports_api.rs needed zero edits. Both embedded emblems (watermark + header-mark, 1-bit PDF image masks) are registered as XObject::External with their original FlateDecode-compressed bytes reused verbatim — printpdf's built-in image pipeline has no 1-bit ImageMask path, so this was the documented escape hatch rather than re-encoding them as 8-bit raster. Text uses printpdf's builtin Helvetica/Helvetica-Bold (one of the 14 standard PDF fonts, no embedding or font-license question needed). Verified beyond compilation: full test suite green, plus a real 23-page sample (forced pagination, both themes) checked with poppler (pdfinfo/pdftotext/pdfimages) — correct A4 page size, correct page count, clean text extraction, and both emblems confirmed as proper 1-bit stencil masks at their original byte-identical sizes, each a single shared XObject reused across every page. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Dependency ReviewThe following issues were found:
|
Both failures predate this branch's own commits: - port-tests/README.md and lib.sh leaked the real home-server LAN IP in an example SSH tunnel command, tripping the public-repository safety check; replaced with $HOMESERVER_HOST. - dashboard/frontend/e2e/responsive-screens.spec.ts was committed by accident (its own header says "Throwaway capture harness... NOT committed") with a hardcoded path into a prior session's Claude Code job-scratch directory, which doesn't exist in CI. Deleted — it's a one-off manual screenshot dump, not a real regression test, and nothing else references it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ite paths) Last remaining scope item of #1612. Ports honeyfs_implant_client.go (plain HTTP passthrough to the WireGuard-tunnel-only honeyfs-implant service) and credentials_manager.go/credentials_api.go (the ES-backed credentialRecord store keyed by a random cred_ id, CAS-retry writes via es.rs's index_doc_create/index_doc_cas primitives, and the list/create/rotate/link-token API surface). Also adds GET /api/v1/canarytokens, a small pre-existing gap found while wiring link-token's "does this token id exist" check — an earlier pass ported create/download/types but never a history-list endpoint; the response redacts auth_token the same way create()'s response already does. Confirmed the issue's other "misc write paths" bullet (canarytoken fired-event ack/management) doesn't exist in the Go source: fired tokens are plain honeypot events through the existing pipeline, acknowledged the same way any other alert is (already ported). No new backend surface needed there. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This was referenced Aug 18, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the majority of #1612 on top of
port-foundation, in the Rustbackend-service/backend-workertier:GET/PUT /api/v1/preferences), reporter-stats passthrough, user-retention-sweep worker loop.backend-service-mountedcompose service.printpdf, swapped from an initial hand-rolled writer for long-term maintainability), definitions CRUD, ES-driven report data gathering, on-demand generate, and a scheduler worker loop. The three artifact-referenced templates (sandbox/payload/ghidra one-click reports) validate/save butgeneratereturns 501 — their dedicated renderers weren't in scope for this pass.Not included (see note below): issue #1612's "Misc write paths" section (honeyfs-implant credential passthrough, canarytokens fired-event ack beyond mint+download) — flagged to the requester as out of scope for this PR, open as follow-up.
Frontend/BFF wiring for the new endpoints (settings UI reading
/api/v1/preferencesinstead of localStorage, Workbench UI, Reports studio write actions, the newbackend-service-mountedinternal URL) is left to the parallel visual-completion track per the issue's own coordination note.Test plan
cargo buildclean on every commit (no new warnings beyond one pre-existing unrelatedcanarytokens.rswarning)cargo test— 10/10 passing (1 ignored scratch/manual-inspection test)pdfinfo/pdftotext/pdfimages) — correct page count/dimensions, clean text extraction, both embedded emblems render as proper shared 1-bit stencil masksdocker compose --profile next configresolves all new/changed services cleanlyport-foundation(after the parallel visual-completion pass landed) with no conflicts; rebuilt and retested clean🤖 Generated with Claude Code