Full-stack ES store coverage audit — implement into the Rust/BFF tier

[Xore]

After the modernization port (#1608) and worker port (#1610), audit every ES index family against what the new tier actually surfaces, and implement the gaps directly in the Rust service (per Xore: after the port, so it's built once into the new tier, not twice).

Guiding principle (per Xore): full visibility. Every artifact the pipeline captures must be renderable in the dashboard, not just referenced. When an event is an email, the operator sees the message (headers, body); when it's a TTY recording, it replays; when it's an ICS request/response pair, both directions render; when a sensor captured a payload/body/query, the actual content is on the row or one click away. "N bytes saved to " is not visibility.

---

Status (2026-08-18, after PR #1617)

The backend half is done. All seven workstreams landed on port-foundation via #1617:

• A 427bb10 — per-sensor detail lines (event_detail.rs, shared by /events + SSE live); suricata flow/netflow/stats excluded from the default explorer view.
• B 5182c1f — mailoney-mail-v1 importer source (es_importer.rs + compose mount) and GET /api/v1/mail/{session_id} (mail.rs, mail-parser crate; text-only bodies, attachments as metadata).
• C 6b23739 — login/auth-kind completeness across aggregates/dashboard/reports.
• D 168f27e — full ATT&CK set promoted in ip-enrichment-worker; kill_chain.rs supplemental aggs removed.
• E 2a674c1 — new GET /api/v1/ml-health, GET /api/v1/gpu-queue, GET /api/v1/revdeck/{sha}; portbridge profile (p0f OS, first/last seen, ports touched) joined into /api/v1/investigate/ip/{ip}; E.3/E.4/E.7/E.9/E.10 resolved as documentation (/api/v1/reporter-stats already existed; auth-events passthrough already carries all fields; worker-state indices intentionally unsurfaced).
• F b9f13db — typed held_ms/user.password mappings; endlessh histogram now a real ES range agg.
• G d97f92f — NUL/control bytes stripped from cowrie creds at ingest.

What remains — Round 2, frontend consumption (verified: frontend-next references none of the new endpoints yet). The detail lines from Workstream A are already user-visible (the events table renders row.detail as-is), but everything endpoint-shaped is invisible until the UI consumes it:

1. Email view (/api/v1/mail/{session_id} → events record pane + session view): a "Message" tab on mail-body rows — parsed From/To/Subject/Date, plain-text body, attachment metadata list; fall back to honeypot.body_preview when the endpoint 404s (pre-importer history). This is also the flagship case of open issue dashboard: no per-sensor detail page/tabs showing sensor-specific structured data (e.g. SMTP mail body/sender/receiver) #1538 (per-sensor structured detail views) — implement it as the pattern that issue generalizes.
2. ML model health (/api/v1/ml-health → ml-anomalies.tsx): last-retrain card per model (accepted/reason, anomaly-rate old→new, samples) so a drifting/rejected model is visible.
3. GPU queue (/api/v1/gpu-queue → payloads/analysis pages): status strip (job_type, model, status, attempts, abort_requested, requested_at).
4. Revdeck run view (/api/v1/revdeck/{sha} → revdeck.tsx detail): render exit_status/error instead of a blank page.
5. Reporter health (/api/v1/reporter-stats → settings/source-health): attempted / suppressed_cooldown / dry_run / sent / failed + updated_at.
6. Investigate page (investigate.ip.$ip.tsx): render the new portbridge profile block (OS guess, first/last seen, ports touched).
7. Arkime pivot: clickable pivot from the record pane's network.community_id / time+ip into arkime.xore.rocks (E.4 put the key in the pane; the affordance is frontend).
8. Ghidra report link: prominent link to the stored HTML report from ghidra.$sha.tsx via the existing artifacts download path (download-only or sandboxed iframe).
9. auth-events columns (auth-events.tsx): add details.username and details.redirect_uri columns (time/type/ip/error/realm/client/user already exist).

After Round 2, per the ordering below: the responsive 4K/mobile + screenshot pass (#1576), then the remaining port-track issues (#1608 close-out, #1616 BFF scalability).

---

This issue is the research + spec. The audit below was run 2026-08-18 against the live cluster (per-sensor field inventories sampled from real documents; index census from _cat/indices). Everything an implementer needs — exact field names, current code locations, and the target behavior — is in the workstreams.

---

Ground truth (live audit, 2026-08-18)

Event indices the Rust tier queries by default (backend-service/src/es.rs: EVENT_INDICES = ["honeypot-v2-*", "suricata-v2-*"]). honeypot.* is a flattened mapping (honeypot-init/analysis/elasticsearch-setup.sh:259, ignore_above: 32000) — exact-term queries and terms-aggs on leaves work; wildcard/substring queries and stats/range aggs do not.

Per-sensor event vocabularies + key fields (from live sampling; event.sensor is the routing key, per-sensor payload rides in flattened honeypot.*):

sensor	event kinds (honeypot.eventid / honeypot.event)	fields that carry the actual content
cowrie (7.8M)	cowrie.telnet.option, login.failed/success, command.input/failed/success, session.connect/closed/params, log.closed, client.version/kex/fingerprint/var/size, session.file_download(.failed)/file_upload, telnet.error/exploit_attempt, client.malformed_packet, direct-tcpip.request/ja4/ja4h	username, password, input, shasum, destfile/url/filename, version, hassh, fingerprint, ttylog, duration_ms, message (human-readable fallback on every event)
dionaea (15.8M)	(no eventid; origin: dionaea.* incident shape or connection summary shape)	data.name + data.cve (exploit identity, e.g. "DoublePulsar connection attempt" / CVE-2017-0144..0148), data.connection.{protocol,transport,local_port,remote_ip}, data.uuid/opnum/transfersyntax (SMB DCERPC), SIP: data.method, data.user_agent, data.to/from/via, credentials, canonical_shasum
multipot (695k)	connect, handshake, login, auth_attempt, command, http_request, envelope, message, write, open, (skip: listening, multipot_started)	proto (vnc/postgres/redis/smtp/hl7/adb/elasticsearch/…), username/password, command, data (VNC auth hash, HL7 message, ADB shell input, SOCKS5 target…), client (banner), session
conpot × {base, kamstrup, guardian, s7-1200, s7-1500, iec104} (245k)	event_type: NEW_CONNECTION/CONNECTION_LOST/…	data_type (modbus/kamstrup_protocol/guardian_ast/IEC104), request + response (raw protocol bytes; guardian responses are full readable tank readouts), dst_port
endlessh (53k)	connect, disconnect, (skip listening)	held_ms, lines
http-honeypot / api-honeypot (66k)	(no event; category: landing/…, skip startup)	method, path, status, headers, user_agent, host, tarpitted + tarpit_bytes/tarpit_ms, body (POST)
dicompot (27k)	connect, associate, c_echo, c_find, c_move, c_get, c_store, (skip listening)	called_ae, calling_ae, data, bytes
sentrypeer (7.4k)	(no event)	sip_method, sip_user_agent, sip_message (full SIP message!), called_number, transport_type
rdp-honeypot (4.3k)	connect, (skip listening)	username (mstshash), requested_protocols (TLS+CredSSP), data (base64 connection request)
cisco-asa-honeypot (779)	get, post, ike_sa_init, ike_unexpected_exchange, ike_malformed, ike_no_further_reply, (skip *_listening), cve_2018_0101_payload	path, headers, user_agent, data (POST body / IKE exchange type)
hellpot (727)	(no event; message: NEW/FINISH)	path, user_agent, BYTES, DURATION (ms), error
beelzebub (474)	(no event kind; event is an object)	protocol, username/password, command, path, event.Description (e.g. "Wordpress 6.0"), event.Headers
dnp3 (343)	frame, malformed_frame, connect	function (link), app_function (e.g. direct_operate), dnp3_source/dnp3_destination, frame_hex
dns-honeypot (324)	query, malformed_query(_dropped), query_dropped, (skip listening)	query (domain), qtype, opcode, rd, req_bytes/resp_bytes
citrix-honeypot (272)	get, post, method_pri, cve_2019_19781_payload, (skip listening)	path (e.g. /../../../../etc/passwd), headers (incl. x-ja3/x-ja4), data (payload/SAMLRequest)
mailoney (24)	envelope, mail-body, login	command (MAIL FROM/RCPT TO line), body_preview (first 512 chars only), body_path (.eml on disk), size, truncated, session_id, server_name; login: username/password
canarytokens (17)	(no event)	memo, token_type, channel, manage_url, additional_data.{useragent,referer,location,src_ip}
tanner (10)	(no event; method/path shape)	method, path, headers, cookies, post_data, response_msg.response.message.detection.{name,payload.value} (emulator verdict + real execution result), uuid
galah (4)	(no event; msg: successfulResponse)	path, user_agent, httpRequest (full request obj), httpResponse (full LLM-generated response incl. body), responseMetadata.generationSource
elasticpot (3)	elasticpot.recon, elasticpot.attack	request (method), url, payload (ES query body), user_agent, content_type
wordpot (0 live, supported in legacy)	—	username/password, plugin, theme, path

Suricata (suricata-v2-*, in EVENT_INDICES): alert (signature/category/severity, payload_printable, http.http_body_printable, alert.metadata.mitre_*, full flow counters, capture_file), anomaly (anomaly.{type,event,app_proto,layer}), http (url/method/status/user_agent/redirect), tls (ja3/ja3s/ja4, sni, version, cert subject/issuer/serial/notbefore/notafter), ssh (client+server proto/software versions), smtp (helo, mail_from, rcpt_to), dns (queries/answers/rcode), fileinfo (per-file over HTTP), flow/netflow (volumetric), stats.

Other index families (census, docs): arkime_sessions3-* (~2M/day full session metadata: protocols, srcPayload8/dstPayload8 hex, sip.*, community_id, tcpflags), arkime_history_v1-*, portbridge-v2-* (per-connect via_port→target mapping + p0f os), dionaea-incidents-v1-* (~1.5M/day raw incident duplicates of the honeypot-v2 dionaea docs), cowrie-ttylog-v1 (full recordings, base64), attackers-v1 / attacker-clusters-v1 / campaigns-v1 / agent-intrusion-campaigns (correlator output incl. per-campaign matched_categories + per-event source refs), auth-failure-events (Keycloak login failures incl. error kind + redirect_uri), yara-analysis-v1, ml-anomalies (per-model scores + explanation), ml-worker-metrics (retrain accept/reject history, anomaly-rate drift — only backlog_count surfaced today), gpu-job-queue (job status incl. abort_requested, evidence payload), reporter-metrics-v1 (attempted/suppressed/sent/failed counters), sandbox-analysis-v1 (stdout/stderr, changed_files, sockets before/after, top_syscalls, network attempts, exiftool, risk score…), ghidra-report-artifacts-v1 (full HTML report, base64), sandbox-export-artifacts-v1 (pcaps etc., base64 chunks), revdeck-analysis-v1, dashboard-payload-{inventory,bytes}-v1 (hex preview + raw bytes), dashboard-intelligence-archive-v1, auth-events-worker-state (worker cursor — intentionally not a UI surface).

---

✅ Workstream A (backend done, #1617) — per-sensor event detail (events.rs + live.rs)

backend-service/src/events.rs:89 collapses detail to honeypot.event (fallback message). The legacy renderer dashboard/classify.go:178-1218 had per-sensor detail lines; port that logic into row_from_source() (shared by the list endpoint and the SSE live stream) as a per-sensor match on event.sensor, reading the flattened honeypot.* fields in the table above. Target formats — follow classify.go's exact lines (they encode many live-verified fixes; the annotations there explain each):

• cowrie: login.failed: user / pass (+ (key <shortHash>) for pubkey), cmd: <input>, payload <shortHash> -> <destfile>, download failed: <url> (<error>), client: <version>, SSH HASSH: <hassh>, terminal WxH, port-forward request -> ip:port, JA4 (tunneled TLS …): <ja4>, pubkey offered (<type>): <hash>, CVE <cve> attempt/succeeded, TTY session recorded: <ttylog> (+ replay link from shasum), closed after <duration_ms>ms; default: message before bare eventid suffix.
• multipot: <proto> login: user / pass, <proto>: <command>, else <proto> <kind> + append command, data, client: <banner> when present (see classify.go's Plan user settings and configuration system #41 note: http_request's command goes in detail only, never the commands aggregate).
• dionaea: prefer data.name (data.cve) over the module path; append download/shasum, SMB uuid=… opnum=… transfersyntax=…; connection shape: proto/transport type -> :port + credentials.
• conpot*: <data_type> <request|event_type> + -> <response> (the response is what the decoy disclosed — guardian tank readouts render well).
• suricata (new — the legacy tier skipped these dirs; the Rust events explorer queries suricata-v2-* wholesale, so today alert/flow/netflow rows all render with an empty detail): alert → <signature> [<category>] [sev] <mitre> payload: <payload_printable|http_body_printable>; anomaly → <event> [<app_proto>] (layer: …); http → <method> <url> <status>; tls → TLS <version> JA4 <ja4> SNI <sni>; ssh → client/server software versions; smtp → HELO <helo> MAIL FROM <mail_from>; dns → query <rrname> (<rrtype>); fileinfo → filename/size. Additionally add a default kind/category story so flow/netflow/stats don't swamp the explorer: either exclude them from the default view (legacy behavior, classify.go:1117) or make the existing filters cover event.category — decide in-implementation, but empty-detail rows must go.
• remaining sensors: per the table above (endlessh tarpit held line, hellpot BYTES/DURATION, dnp3 link/app function + addresses, dns-honeypot query+qtype+opcode+_dropped cap note, citrix/cisco CVE payload + body lines, beelzebub (note: honeypot.event is an object here — guard the current text() read), tanner post_data/cookies/detection result, galah LLM response, elasticpot payload, sentrypeer method→called_number, canarytokens memo/channel, mailoney (Workstream B), rdp mstshash+security, dicompot DIMSE labels + AE titles, wordpot).

Also populate EventRow.proto from honeypot.proto when network.protocol is absent, and port from honeypot.port/honeypot.dst_port — several sensors (multipot, conpot, dnp3…) only carry them there. Keep record as-is (full doc inspector already exists in frontend-next/src/routes/events.tsx).

Suggested shape: a detail.rs-style fn detail_for(sensor: &str, hp: &Value, src: &Value) -> String with unit tests per sensor using the sampled documents above as fixtures.

✅ Workstream B (backend done, #1617) — full message/content visibility (email first)

The user-visible motivation for this issue: when an event is an email, show the email.

• mailoney mail-body events store body_preview = first 512 chars (honeypot-mailoney/mailoney/json_log_patch.py:188), truncated, and body_path relative to the mail dir. The full .eml lands on the homeserver at /opt/stacks/apiary/logs/mailoney/mail/<body_path> (compose mount + MAILONEY_MAIL_DIR=/var/log/honeypot/mail). The Rust backend-service container mounts no log volumes (ES-only, honeypot-dashboard/compose.yml:384).
• Implement via the existing importer pattern (analysis/es-results-importer/importer.py already imports cowrie TTY logs from disk into cowrie-ttylog-v1 with ttylog_base64 precisely so the ES-only tier can serve them): add a MAILONEY_MAIL_DIR source → mailoney-mail-v1 (doc per .eml: session_id, body_path, eml_base64 or text, size_bytes, imported_at; id = sha of path so re-scans are idempotent). Wire the env + mount into es-results-importer's compose service.
• Rust: GET /api/v1/mail/{session_id} (join key: honeypot.session_id on the event == session_id in the new index) returning parsed headers (From/To/Subject/Date), decoded text body, and an attachments list (filename, content-type, size, sha256 — content downloadable, never auto-rendered; treat HTML bodies as text/escaped, this is attacker-controlled content).
• Frontend: the events explorer's record pane (and session view) shows a "Message" tab for mail-body rows: rendered headers + body, raw source toggle. Fall back to body_preview for docs that predate the importer.
• Same principle applied cheaply elsewhere while in there: suricata.eve.smtp.{helo,mail_from,rcpt_to} on the SMTP detail line; multipot envelope lines already carry the SMTP commands; galah's httpResponse.body (what the LLM served) and tanner's detection.payload.value (emulator execution result) render in the record pane rather than staying buried in flattened JSON.

✅ Workstream C (backend done, #1617) — login/auth-kind completeness (aggregates.rs, search.rs, dashboard.rs)

aggregates.rs:68 counts logins as honeypot.event ∈ {login, auth_attempt}. Audited against the live vocabularies, that misses most auth activity:

• cowrie: honeypot.eventid ∈ {cowrie.login.success, cowrie.login.failed} (1.26M docs — the largest auth source, entirely uncounted).
• rdp-honeypot: auth rides on connect with a non-empty honeypot.username (mstshash).
• mailoney: login (AUTH PLAIN) ✔ already counted; beelzebub/wordpot: username/password fields with no event kind; dionaea: credentials array on connection records / login-ish incidents; elasticpot has none.

Fix: a shared logins_filter() helper building bool.should: terms honeypot.event: [login, auth_attempt] + terms honeypot.eventid: [cowrie.login.success, cowrie.login.failed] + bool(term event.sensor: rdp-honeypot AND exists honeypot.username) + exists honeypot.canonical_user (worker-promoted, sensor-agnostic — covers beelzebub/wordpot/dionaea as promotion lands). Use it in aggregates.rs (sources page), and audit dashboard.rs's cred/login widgets and search.rs's "Credentials (usernames)" group (honeypot.username misses cowrie? verify — cowrie's enriched line does carry username at the honeypot.* level, but only on login events) for the same vocabulary.

✅ Workstream D (backend done, #1617) — promote the full ATT&CK technique set in ip-enrichment-worker

kill_chain.rs supplements T1190/T0886/T1692.001 with filter-aggs because ip-enrichment-worker/attck.go deliberately skipped them (see its header comment). Promote them in the worker so the supplements (and their flattened-field limitations) go away:

• T0886 / T1692.001 (ICS): the worker already tails every conpot persona and dnp3. Tag T0886 on any conpot/dnp3 protocol interaction (data_type/sensor check mirroring intelligence.go:73-78), plus T1692.001 when it's a write/command (dnp3 app_function present, conpot request non-empty on a control function, multipot/event ∈ {command, write} on ICS protos). This widens the worker's conpot handling from IP-resolution-only to field promotion — the very thing attck.go's comment deferred to this round.
• T1190: mirror intelligence.go:67-69 (path present AND (alert OR ?/% in path OR ../ OR wp- OR php)) at promotion time for the web sensors the worker already watches (citrix/cisco/hellpot/beelzebub/galah/wordpot + tanner/http via their tails); also tag the explicit exploit events (cve_2019_19781_payload, cve_2018_0101_payload, method_pri, elasticpot.attack, tanner detection.name != index).
• T1110 gap noted in attck.go: with per-sensor login knowledge from Workstream C, tag empty-credential login attempts too (cowrie.login.failed with empty pass, rdp connect+username), matching legacy IsLogin || HasCredential.
• Then delete the ics/ics_write/web_exploit supplemental aggs in kill_chain.rs:90-105 and the #33 comments; keep them until backfill/rollover makes the promoted tags dominant, or run both and take the max per technique during transition (documented flag).
• Suricata stays out of the worker (separate pipeline); its alerts already carry alert.metadata.mitre_* — surface those on the detail line (Workstream A) and consider a follow-up to include them in the coverage grid from the suricata indices directly.

✅ Workstream E (backend done, #1617) — under-surfaced index families

Concrete per-family targets (today's only consumers noted):

1. suricata-v2-{http,tls,ssh,smtp,dns,fileinfo,alert,anomaly}: become first-class event rows via Workstream A. tls/ssh already feed the two fingerprint charts; netflow feeds bytes/packets charts; anomaly the trend chart — unchanged.
2. portbridge-v2-* (today: OS-distribution chart only): add per-IP OS + first/last-seen + ports-touched to /api/v1/investigate/ip/{ip} (investigate.rs) — the p0f os, via_port→target and per-port connect counts are the only ground truth for "which ports did this IP actually knock on" across tunneled sensors.
3. dionaea-incidents-v1-*(today: CVE chart only): the same incident docs flow into honeypot-v2 (audited), so richer rendering comes free with Workstream A's dionaea branch; keep the CVE chart on the dedicated family. No new endpoint needed — document that in charts.rs.
4. arkime_sessions3-* / arkime_history_v1-*: don't re-implement Arkime; add a per-event/per-IP cross-link into the existing Arkime UI (arkime.xore.rocks) using time-window + src ip (community_id exists on both suricata and arkime docs — use it for exact session links where present on the event). Surface network.community_id in the record pane as a copyable pivot.
5. ml-worker-metrics (today: backlog chart only): a "model health" card/endpoint on the ML anomalies page — last retrain per model (kind: retrain, model, accepted, reason, anomaly_rate_new vs anomaly_rate_previous, train_samples) so a drifting/rejected model is visible.
6. gpu-job-queue: a queue-status strip on the analysis pages (job_type, model, status, attempts, abort_requested, requested_at) — 2 stuck queued jobs live right now, invisible.
7. reporter-metrics-v1: surface attempted/suppressed_cooldown/dry_run/sent/failed + updated_at on Settings→storage/health (health.rs) — live doc shows 851k attempted / 0 sent, which an operator should see at a glance.
8. Artifact indices (revdeck-analysis-v1, sandbox-export-artifacts-v1, ghidra-report-artifacts-v1 — today: raw listing via artifacts.rs/stores.rs): render, don't list. Ghidra: serve the stored HTML report (data_base64, content_type: text/html) via the existing /api/v1/artifacts download path and link it prominently from ghidra.$sha (sandboxed iframe or download-only; it's generated by our own worker but treat as untrusted). Sandbox exports: keep as downloads but label kind/size. Revdeck: show exit_status/error (the live doc is an unconfigured-worker error — that state should be visible, not a blank page).
9. auth-failure-events (today: /store/auth-events raw rows): keep, but add error, details.username, details.redirect_uri as first-class columns on auth-events.tsx — the interesting fields are nested today.
10. auth-events-worker-state, ml-worker-state, dashboard-*-state: worker cursors/state — explicitly out of scope for UI; document as such in stores.rs so the census question doesn't reopen.

✅ Workstream F (backend done, #1617) — promote hot flattened fields to typed mappings

honeypot is one flattened blob; documented costs today: no wildcard queries (kill_chain.rs:93-98 T1190 workaround — goes away with Workstream D), no stats/range aggs (charts.rs endlessh histogram buckets held_ms client-side from 2000 raw hits, charts.rs:341-357). Evaluate promoting in the index template (honeypot-init/analysis/elasticsearch-setup.sh) as real subfields alongside the flattened blob (new fields only apply on rollover; the datastream rolls daily, so coverage is ~immediate for new data):

• honeypot.held_ms → long (enables a real ES histogram/percentiles for the endlessh chart),
• honeypot.path → wildcard (path substring hunting in /search and T1190-adjacent queries),
• honeypot.username, honeypot.password → keyword (top-N + prefix/wildcard cred hunting; today only exact terms work),
• consider honeypot.command → wildcard (same rationale as process.command_line, already wildcard in the template).

Since the source JSON keys collide with the flattened root, the clean mechanism is copying at ingest (ip-enrichment-worker already rewrites every enriched line — emit typed top-level ECS fields, e.g. user.name/user.password-style or a hp.* typed namespace) OR a runtime→indexed field migration; pick during implementation, but the template + one chart (endlessh) + one query (search wildcard) must land as proof.

✅ Workstream G (backend done, #1617) — credential hygiene at ingest

Telnet NUL bytes ride into honeypot.username/honeypot.password and are currently stripped display-side in two places (backend-service/src/dashboard.rs:105, investigate.rs:129). Cowrie routes through ip-enrichment-worker (main.go:113 → enriched/cowrie.json): strip NUL/control bytes from username/password (and the promoted canonical_user/canonical_pass) in canonical.go's cowrie promotion before the line is written, so every downstream consumer (ES terms aggs, attackers-v1 credentials, search grouping) sees clean values and identical creds stop splitting into distinct buckets. Keep the display-side cleanup for historical documents; add a worker test with a real NUL-embedded sample.

---

Ordering (per Xore): after #1608 port + #1610 worker/BFF round, before the final 4K/mobile + screenshot pass. Suggested implementation order: A (biggest visible win) → C (small, correctness) → B (email visibility, needs the importer) → D → E → G → F.

Acceptance criteria

• Every sensor in the vocabulary table renders a content-bearing detail line in /events and the live stream (no bare connect-only rows where richer fields exist; no empty-detail suricata rows; flow/netflow/stats no longer swamp the default view).
• A mailoney mail-body event shows the full parsed email (headers + body + attachment list) in the dashboard; truncated preview only for pre-importer history. (backend endpoint + importer done — Round 2 item 1 is the remaining UI half)
• Sources/logins counts include cowrie login events (sanity check: an IP with only cowrie logins shows logins > 0).
• canonical_attck_techniques on new documents covers T1190/T0886/T1692.001 where applicable; kill_chain.rs supplemental aggs removed (or flag-gated during transition) with grid/sankey parity.
• Items E.2/E.5/E.6/E.7/E.8/E.9 each have a visible surface; E.10 documented as intentionally unsurfaced. (APIs + docs done; "visible" needs Round 2 items 2–6, 8, 9)
• endlessh histogram computed by ES aggregation on a typed held_ms; at least one wildcard-capable promoted field usable from /search.
• New cowrie credentials indexed NUL-free; display-side strips retained for old docs.
• Rust unit tests per sensor detail branch using the live-sampled fixture documents.

[Xore]

Backend/BFF side of this audit is implemented and merged into port-foundation via #1617.

Shipped

• Workstream A — per-sensor event detail (event_detail.rs, new): 21 sensors ported off dashboard/classify.go's renderer, including new Suricata alert/anomaly/http/tls/ssh/smtp/dns/fileinfo detail lines that never existed in the Rust tier. events.rs/live.rs both route through it; flow/netflow/stats excluded from the default view via suricata_noise_exclusion(). 21 unit tests against literal fixtures.
• Workstream B — GET /api/v1/mail/{session_id} (mail.rs, new): two-step ES join (event → body_path → mailoney-mail-v1), parsed with the mail-parser crate — real headers/body/attachment-metadata, no attachment bytes or raw HTML ever served. es_importer.rs gained the mailoney_mail source mirroring .eml files into ES the same way TTY logs already do. Frontend "Message" tab is not part of this PR — that's a frontend-next change and out of scope for the Rust/BFF tier work this issue asked for; the endpoint is ready for it.
• Workstream C — logins_filter() (es.rs), wired into aggregates.rs, dashboard.rs, and reports_data.rs: cowrie login.success/login.failed (1.26M previously-uncounted docs), rdp-honeypot connect+username, and canonical_user existence all now count as logins, not just the login/auth_attempt event kinds.
• Workstream D — attck.rs rewritten to take a persona param and promote T0886/T1692.001/T1190 at ingest from raw per-persona fields (not just canonical_* fields), plus a real T1110 fix (fires on login attempts, not just ones with both credentials present). Wired into all 7 sensors.rs enrich functions — 5 of them had no ATT&CK promotion call at all before this. kill_chain.rs's ics/ics_write/web_exploit supplemental filter-aggs removed now that the tags are real.
• Workstream E (all 10 sub-items):
  • E.1 → covered by A.
  • E.2 → investigate.rs gained a third parallel query against portbridge-v2-* (OS guess + ports touched) in the existing try_join!.
  • E.3 → documented in charts.rs: dionaea-incidents is a raw duplicate of honeypot-v2 dionaea docs, so A's dionaea branch already covers it; no new endpoint.
  • E.4 → documented in events.rs: the full ECS record field already carries network.community_id when suricata populates it; the Arkime pivot link itself is a frontend concern, not a backend gap.
  • E.5 → GET /api/v1/ml-health (ml_health.rs, new): per-model latest retrain outcome (accepted/reason/anomaly-rate drift) via terms-agg-with-top-hits.
  • E.6 → GET /api/v1/gpu-queue (gpu_queue.rs, new): read-only queue visibility (the audit's "2 stuck jobs, invisible" case is now visible).
  • E.7 → documented in reporter_stats.rs: the existing /api/v1/reporter-stats passthrough already carries whatever the reporter worker writes to metrics.json (attempted/suppressed_cooldown/dry_run/sent/failed/updated_at) — no separate field needed.
  • E.8 → detail.rs gained revdeck_run(): GET /api/v1/revdeck/{sha}, direct doc lookup matching the existing attacker_identity.rs convention.
  • E.9 → documented in stores.rs: the auth-events passthrough already carries error/details.username/details.redirect_uri; making them first-class columns is a frontend-next (auth-events.tsx) change.
  • E.10 → documented in stores.rs: *-worker-state indices are worker resume cursors, deliberately absent from every endpoint — on record so this doesn't reopen.
• Workstream F — held_ms and user.password promoted to typed fields alongside the existing user.name/process.command_line/url.path ingest-time copies (elasticsearch-setup.sh). The endlessh histogram (charts.rs) is now a genuine ES range aggregation instead of bucketing up to 10k raw hits client-side. /api/v1/search's "HTTP paths" group (search.rs) moved off flattened honeypot.path (prefix-only) onto url.path (wildcard) — real substring hunting, same trick already used for Suricata signatures.
• Workstream G — canonical.rs gained strip_nul_and_control(), applied to both raw and canonical username/password on the cowrie login-event branch. Strips real control chars and the literal \x00 text form. Display-side stripping (dashboard.rs, investigate.rs) kept as-is for historical documents.

Verified

Full crate build + test (162/162 passing). Every workstream's field names cross-checked against the pre-change code/legacy Go source where a concrete claim was made (e.g. held_ms's source path, logins_filter's clause set, the endlessh range-bucket contiguity). One review round caught and fixed real gaps along the way (risk-level scale mismatch and a wrong ES field name in reports_data.rs, a sort-on-nonexistent-field bug in stores.rs's agent-campaigns entry).

Deliberately deferred (frontend-next, separate track)

• Workstream B's "Message" tab UI in the events explorer / session view.
• Workstream E.4's Arkime cross-link/pivot UI.
• Workstream E.9's auth-events.tsx first-class columns.

All three are pure frontend work against endpoints/fields that already exist server-side — nothing here blocks on more backend work.

Closing as the Rust/BFF-tier audit this issue scoped is complete.