Skip to content

[VM Discovery P1] SSH inventory + signed content-pack runner #113

Description

@mayankpande88

Part of epic: nudgebee/nudgebee-enterprise#35404. No dependencies — runs in parallel with the server ticket (nudgebee/nudgebee-enterprise#35405).

What we're building, in one sentence

Teach the forager to SSH into a list of VMs, run a fixed set of read-only commands on each (like rpm -qa or dpkg-query -W), and send the raw output back — where the list of commands comes from a signed file downloaded from our server, not from the forager binary.

Why the command list is a downloaded file

If the commands were hardcoded, every tweak (support a new distro, add one more fact) would need a new forager release rolled out to every customer. Instead the binary stays frozen and only knows how to: verify a signature, run commands, return output. The command list ("content pack") is versioned, signed, and updated server-side.

The flow

  1. Relay sends the forager an ActionRequest (existing struct, pkg/proxy/registry.go:13) with action: "discovery_inventory" and params: list of target IPs, which SSH credential to use, which content-pack version to run.
  2. Forager fetches the content pack (or uses its cached copy), verifies its Ed25519 signature. Bad signature → refuse and report.
  3. For each target IP (bounded concurrency): open SSH session, run each command from the pack that matches the host (when: os_family == "debian" guards), capture stdout/stderr per command.
  4. Return one ActionResponse with, per host, per command: the raw output. No parsing on the forager — parsing lives server-side (P2).

What already exists — reuse, don't rebuild

Piece Where
Action plumbing: ActionRequest/ActionResponse, module registry pkg/proxy/registry.go
SSH client setup, auth, session exec (see ssh_command) pkg/proxy/ssh/proxy.go (getClient, ~line 397)
Credential storage/delivery pkg/secrets/ (local + cloud-push)
Ed25519 action signing (trust root to reuse for pack signatures) existing action-signing path
Module pattern to copy any of pkg/proxy/{db,http,redis}

What to build

1. pkg/proxy/discovery module implementing the Proxy interface (registry.go:35), registered like the other modules. First action: discovery_inventory.

2. Content-pack loader: fetch + cache + Ed25519 verify + parse. Example pack (illustrative — the real schema is this ticket's first deliverable, propose it for team review before hardening):

version: 3
kind: inventory        # future: probe, remediation — only inventory now
collectors:
  - id: os-release
    cmd: cat /etc/os-release          # runs on every host
  - id: pkgs-rpm
    when: os_family == "rhel"
    cmd: rpm -qa --qf '%{NAME}\t%{EPOCH}\t%{VERSION}\t%{RELEASE}\t%{ARCH}\n'
  - id: pkgs-dpkg
    when: os_family == "debian"
    cmd: dpkg-query -W -f '${Package}\t${Version}\t${Architecture}\n'
signature: <ed25519 over the rest>

The when evaluator should be tiny (equality checks on a few known facts like os_family, derived from the os-release output). Unknown variable or malformed expression → skip host, report error. Never guess.

3. The executor: takes targets + credential + verified pack, fans out over SSH with a concurrency limit, per-target timeout, and an output size cap per command. One target failing (unreachable, auth refused, timeout) must not fail the batch — it produces a per-target error entry in the response. Response shape (roughly):

{"targets": [
  {"ip": "10.0.1.15", "status": "ok",
   "collectors": {"os-release": "NAME=\"Ubuntu\"...", "pkgs-dpkg": "acl\t2.3.1-1\tamd64\n..."}},
  {"ip": "10.0.1.16", "status": "ssh-auth-failed", "error": "..."}
]}

Acceptance criteria (each is a test)

  • Pack with a tampered byte → signature verification fails → action returns an error, nothing executes.
  • Pack referencing an unknown when variable → that collector is skipped with an error recorded, not silently ignored.
  • Against one RHEL-like and one Debian-like VM: correct collectors run on each, raw output comes back verbatim (epoch/release/tab characters intact).
  • 50–100 concurrent targets (can be one sshd faked many times): no fd leak, memory bounded, batch completes. (Default target — real number tuned later from customer fleet size.)
  • One unreachable target in a batch of 10 → 9 ok results + 1 error entry, exit status of the batch is success.
  • SSH key/password never appears in logs or in the ActionResponse.

Explicitly out of scope here

Finding targets (sweep/LDAP/hypervisor — P3/P4), parsing the output (P2), scheduling (P5), the real production pack content (P6 — use a small hand-written pack for this ticket).

Design: docs/design/vm-discovery-phase0.md §6–§7, §9 (PR #117).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions