Skip to content

Audit third-party CLI: token login, kebab-case rename, compliance report commands#1483

Open
uvollmer wants to merge 6 commits into
getprobo:mainfrom
uvollmer:cli-third-party-audit
Open

Audit third-party CLI: token login, kebab-case rename, compliance report commands#1483
uvollmer wants to merge 6 commits into
getprobo:mainfrom
uvollmer:cli-third-party-audit

Conversation

@uvollmer

@uvollmer uvollmer commented Jul 13, 2026

Copy link
Copy Markdown

Summary

Audits and rounds out the prb third-party CLI surface. Three intended changes, plus three bugs caught while verifying end-to-end against a live org.

Intended changes

  • Token/API-key login — adds prb auth login --token, converging on the documented UX (--hostname … --token … --org …). Skips the browser device flow, validates the token, and persists a token-only host config. Org selection is shared with the device flow via a new selectOrganization helper.
  • Kebab-case rename (clean break)prb thirdPartyprb third-party; package pkg/cmd/thirdpartymgmtpkg/cmd/third-party; flags --thirdParty-ids--third-party-ids on asset/datum. No back-compat aliases. GraphQL field names and IAM action identifiers keep their canonical casing.
  • Compliance report commands — new prb third-party report upload | list | delete, wrapping the existing uploadThirdPartyComplianceReport / deleteThirdPartyComplianceReport mutations and the complianceReports connection.

Bugs found & fixed while verifying against a live backend

  1. Date-only flags rejected--report-date 2026-01-01 (as documented) failed server-side because the Datetime scalar only parses RFC3339. Added cmdutil.NormalizeDatetime (date-only → midnight UTC, RFC3339 passthrough) with unit tests.
  2. CLI uploads sent application/octet-streammultipart.CreateFormFile hardcodes that Content-Type, which the server's document validator rejects; the file-validation error then surfaced as an opaque HTTP 500. DoUpload now derives the real MIME from the file extension. This is in shared pkg/cli/api/client.go, so it fixes every CLI upload — evidence upload was broken the same way.
  3. evidence upload selected a non-existent field — its mutation queried evidence on UploadMeasureEvidencePayload (which only has evidenceEdge), failing GraphQL validation with an HTTP 422 before reaching the resolver. Fixed to evidenceEdge { node { … } }. Pre-existing; surfaced only because fix Bootstrap api #2 let the request reach query validation.

Also fixed a bogus --json example (real flag is -o json) in the report and third-party list commands. The same typo exists in ~21 other list commands and was left out of scope.

Testing

Verified end-to-end against a live org (all throwaway objects cleaned up):

  • third-party list works; old thirdParty command is gone (clean break confirmed).
  • report upload … --report-date 2026-01-01 --valid-until 2026-12-31 → succeeds; report list -o json confirms dates normalized to 2026-01-01T00:00:00Z / 2026-12-31T00:00:00Z.
  • Invalid date → clean client-side error; delete without --yes refused; delete --yes removes the report.
  • evidence upload ./x.pdf --measure <id> → succeeds (confirms both the content-type and mutation-selection fixes on that path).
  • go build, go vet, gofmt, and cmdutil unit tests all green.

Notes / follow-ups (out of scope)

  • The report commands exist only in the CLI; MCP (specification.yaml) and n8n have no compliance-report upload operation (api-surface.md sync gap).
  • --json example typo persists in ~21 other list commands.

🤖 Generated with Claude Code


Summary by cubic

Adds token login to prb auth login, renames the third-party CLI to kebab-case, and adds report upload/list/delete commands. Fixes broken uploads by sending real MIME types, normalizes date flags, and fixes evidence upload selection.

prb (CLI) +716 -99

  • Token/API-key login: prb auth login --hostname … --token … --org … validates the token and saves a host config; shared org picker for interactive mode.
  • Rename: prb thirdPartyprb third-party; flags --thirdParty-ids--third-party-ids; updated help, prompts, and outputs. Fixed GraphQL fields in list (thirdParties) and update (thirdParty).
  • Reports: new prb third-party report upload|list|delete. Upload supports --report-date, optional --report-name, --valid-until; list supports sorting and -o json; delete confirms or --yes.
  • Date handling: added cmdutil.NormalizeDatetime to accept YYYY-MM-DD or RFC3339; used by report flags.
  • Uploads: multipart now sets a real Content-Type from the file extension (applies to all CLI uploads, including evidence).
  • Evidence upload: selects evidenceEdge { node { … } } to match schema.
  • Minor docs/help fixes: corrected -o json examples; updated webhook help text.

Tests +68 -0

  • Unit tests for NormalizeDatetime: date-only anchoring, RFC3339 passthrough, and invalid input cases.

Other +3 -3

  • README: switched examples to prb third-party and added report upload usage.

Written for commit f110b9e. Summary will update on new commits.

Review in cubic

uvollmer added 6 commits July 10, 2026 17:20
The CLI documentation describes logging in with an authentication
token (prb auth login --hostname <host> --token <token> --org <org>),
but the command only implemented the OAuth2 device flow. Personal API
keys are already accepted by the backend as bearer tokens, so only the
login UX was missing.

When --token is provided, skip the device flow, validate the token
with a viewer query, and persist a token-only host config. The
interactive organization selection is shared with the device flow via
a new selectOrganization helper.

Signed-off-by: Ugo Vollmer <ugo.vollmer@gmail.com>
The third-party command group was registered as 'prb thirdParty'
(camelCase) and its help text mixed thirdParty, third_party, and
third-party spellings, while the CLI convention is kebab-case (cf.
trust-center, processing-activity, consent-record).

Rename the command group to 'prb third-party', move the package from
pkg/cmd/thirdpartymgmt to pkg/cmd/third-party, and align all command
descriptions, examples, prompts, and outputs. Rename the
--thirdParty-ids flag on asset and datum create/update to
--third-party-ids. This is a clean break: the old command and flag
spellings are no longer accepted.

Also fix two queries that were invalid against the GraphQL schema and
broke the commands at runtime: list queried third_parties instead of
thirdParties, and update selected third_party instead of thirdParty.

GraphQL field names and IAM action identifiers (core:thirdParty:*)
keep their canonical casing.

Signed-off-by: Ugo Vollmer <ugo.vollmer@gmail.com>
The backend has supported compliance report upload, listing, and
deletion since the uploadThirdPartyComplianceReport mutation was
introduced, but none of it was reachable from the CLI.

Add a 'prb third-party report' command group with three subcommands:

- upload: send a report file via the GraphQL multipart upload,
  with --third-party, --report-date, and optional --report-name
  (defaults to the file name) and --valid-until
- list: paginated table of a third party's reports, sortable by
  REPORT_DATE or CREATED_AT, with JSON output support
- delete: remove a report, with interactive confirmation or --yes

Signed-off-by: Ugo Vollmer <ugo.vollmer@gmail.com>
The report upload command documented --report-date/--valid-until as
YYYY-MM-DD (in help, examples, and the README), but the GraphQL
Datetime scalar only parses RFC3339, so the documented invocation was
rejected server-side before the upload ran.

Add cmdutil.NormalizeDatetime, which accepts either a date-only value
(anchored to midnight UTC) or a full RFC3339 timestamp and returns
RFC3339. Apply it to --report-date and --valid-until in the report
upload command, returning a clear client-side error on bad input.

Signed-off-by: Ugo Vollmer <ugo.vollmer@gmail.com>
multipart.CreateFormFile hardcodes each part's Content-Type to
application/octet-stream. The server's file validators only accept
specific document MIME types (e.g. application/pdf), so every CLI
upload was rejected -- and because the failure is a file-validation
error rather than a GraphQL validation error, the resolver returned an
opaque HTTP 500. This affected report upload and evidence upload alike.

Build the file part manually and derive its Content-Type from the file
extension via mime.TypeByExtension, falling back to octet-stream only
when the extension is unknown.

Also correct the report/third-party list examples to use '-o json',
the actual output flag (there is no --json flag).

Signed-off-by: Ugo Vollmer <ugo.vollmer@gmail.com>
The upload evidence command selected a non-existent 'evidence' field
on UploadMeasureEvidencePayload, whose only field is evidenceEdge.
Every evidence upload therefore failed GraphQL validation with an
opaque HTTP 422 before reaching the resolver.

Select evidenceEdge { node { ... } } and unmarshal accordingly,
matching the payload schema and the edge/node pattern used elsewhere.

Signed-off-by: Ugo Vollmer <ugo.vollmer@gmail.com>
@uvollmer uvollmer marked this pull request as ready for review July 13, 2026 08:49

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4 issues found across 24 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="pkg/cmd/third-party/report/list/list.go">

<violation number="1" location="pkg/cmd/third-party/report/list/list.go:104">
P2: The `--order-direction` flag accepts `ASC` or `DESC` but is not validated client-side. If a user passes an invalid value like `--order-direction INVALID`, the invalid string is sent directly to the GraphQL API, which would reject it with a generic server error. Consider adding a `cmdutil.ValidateEnum` call alongside the existing `flagOrderBy` validation to provide a clear, immediate error message.

```go
if flagOrderBy != "" {
    if err := cmdutil.ValidateEnum("order-by", flagOrderBy, []string{"REPORT_DATE", "CREATED_AT"}); err != nil {
        return err
    }

    if err := cmdutil.ValidateEnum("order-direction", flagOrderDir, []string{"ASC", "DESC"}); err != nil {
        return err
    }

    variables["orderBy"] = map[string]any{
        "field":     flagOrderBy,
        "direction": flagOrderDir,
    }
}
```</violation>

<violation number="2" location="pkg/cmd/third-party/report/list/list.go:115">
P3: `--limit=0` or a negative limit reports “No compliance reports found” without contacting the API, which can be mistaken for an empty third party. Validate the limit before pagination, as intended by `cmdutil.ValidateLimit`.</violation>
</file>

<file name="pkg/cmd/third-party/report/delete/delete.go">

<violation number="1" location="pkg/cmd/third-party/report/delete/delete.go:34">
P3: This adds another near-verbatim delete-command implementation, so confirmation or client-flow fixes will continue to require edits across every resource. A shared cmdutil delete-command helper with resource-specific mutation/input/output parameters would keep behavior consistent.</violation>
</file>

<file name="pkg/cmd/auth/login/login.go">

<violation number="1" location="pkg/cmd/auth/login/login.go:265">
P1: Token login can transmit a personal API key in cleartext when `--hostname http://…` is supplied. Reject non-HTTPS hosts before validating or persisting the token.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic


_, _ = fmt.Fprintf(f.IOStreams.ErrOut, "Validating token on %s...\n", host)

orgs, err := fetchOrganizations(baseURL, token)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: Token login can transmit a personal API key in cleartext when --hostname http://… is supplied. Reject non-HTTPS hosts before validating or persisting the token.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At pkg/cmd/auth/login/login.go, line 265:

<comment>Token login can transmit a personal API key in cleartext when `--hostname http://…` is supplied. Reject non-HTTPS hosts before validating or persisting the token.</comment>

<file context>
@@ -253,10 +238,92 @@ func NewCmdLogin(f *cmdutil.Factory) *cobra.Command {
+
+	_, _ = fmt.Fprintf(f.IOStreams.ErrOut, "Validating token on %s...\n", host)
+
+	orgs, err := fetchOrganizations(baseURL, token)
+	if err != nil {
+		return fmt.Errorf("cannot authenticate with token: %w", err)
</file context>
Suggested change
orgs, err := fetchOrganizations(baseURL, token)
if strings.HasPrefix(strings.ToLower(baseURL), "http://") {
return fmt.Errorf("token login requires an HTTPS hostname")
}
orgs, err := fetchOrganizations(baseURL, token)

"id": args[0],
}

if flagOrderBy != "" {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The --order-direction flag accepts ASC or DESC but is not validated client-side. If a user passes an invalid value like --order-direction INVALID, the invalid string is sent directly to the GraphQL API, which would reject it with a generic server error. Consider adding a cmdutil.ValidateEnum call alongside the existing flagOrderBy validation to provide a clear, immediate error message.

if flagOrderBy != "" {
    if err := cmdutil.ValidateEnum("order-by", flagOrderBy, []string{"REPORT_DATE", "CREATED_AT"}); err != nil {
        return err
    }

    if err := cmdutil.ValidateEnum("order-direction", flagOrderDir, []string{"ASC", "DESC"}); err != nil {
        return err
    }

    variables["orderBy"] = map[string]any{
        "field":     flagOrderBy,
        "direction": flagOrderDir,
    }
}
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At pkg/cmd/third-party/report/list/list.go, line 104:

<comment>The `--order-direction` flag accepts `ASC` or `DESC` but is not validated client-side. If a user passes an invalid value like `--order-direction INVALID`, the invalid string is sent directly to the GraphQL API, which would reject it with a generic server error. Consider adding a `cmdutil.ValidateEnum` call alongside the existing `flagOrderBy` validation to provide a clear, immediate error message.

```go
if flagOrderBy != "" {
    if err := cmdutil.ValidateEnum("order-by", flagOrderBy, []string{"REPORT_DATE", "CREATED_AT"}); err != nil {
        return err
    }

    if err := cmdutil.ValidateEnum("order-direction", flagOrderDir, []string{"ASC", "DESC"}); err != nil {
        return err
    }

    variables["orderBy"] = map[string]any{
        "field":     flagOrderBy,
        "direction": flagOrderDir,
    }
}
```</comment>

<file context>
@@ -0,0 +1,185 @@
+				"id": args[0],
+			}
+
+			if flagOrderBy != "" {
+				if err := cmdutil.ValidateEnum("order-by", flagOrderBy, []string{"REPORT_DATE", "CREATED_AT"}); err != nil {
+					return err
</file context>

}
`

func NewCmdDelete(f *cmdutil.Factory) *cobra.Command {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This adds another near-verbatim delete-command implementation, so confirmation or client-flow fixes will continue to require edits across every resource. A shared cmdutil delete-command helper with resource-specific mutation/input/output parameters would keep behavior consistent.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At pkg/cmd/third-party/report/delete/delete.go, line 34:

<comment>This adds another near-verbatim delete-command implementation, so confirmation or client-flow fixes will continue to require edits across every resource. A shared cmdutil delete-command helper with resource-specific mutation/input/output parameters would keep behavior consistent.</comment>

<file context>
@@ -0,0 +1,105 @@
+}
+`
+
+func NewCmdDelete(f *cmdutil.Factory) *cobra.Command {
+	var flagYes bool
+
</file context>

}
}

reports, _, err := api.Paginate(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: --limit=0 or a negative limit reports “No compliance reports found” without contacting the API, which can be mistaken for an empty third party. Validate the limit before pagination, as intended by cmdutil.ValidateLimit.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At pkg/cmd/third-party/report/list/list.go, line 115:

<comment>`--limit=0` or a negative limit reports “No compliance reports found” without contacting the API, which can be mistaken for an empty third party. Validate the limit before pagination, as intended by `cmdutil.ValidateLimit`.</comment>

<file context>
@@ -0,0 +1,185 @@
+				}
+			}
+
+			reports, _, err := api.Paginate(
+				client,
+				listQuery,
</file context>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant