Skip to content

feat: Add API client infrastructure and org/project commands#8

Open
devin-ai-integration[bot] wants to merge 2 commits intomainfrom
devin/1772923725-api-client-and-commands
Open

feat: Add API client infrastructure and org/project commands#8
devin-ai-integration[bot] wants to merge 2 commits intomainfrom
devin/1772923725-api-client-and-commands

Conversation

@devin-ai-integration
Copy link

@devin-ai-integration devin-ai-integration bot commented Mar 7, 2026

📝 Summary

Adds foundational infrastructure for interacting with the Judgment platform REST API, plus organization and project commands as the first reference implementation. New command groups: config, auth, orgs, projects. Existing self-host commands are preserved unchanged.

New files:

  • judgment/api_client.py — Shared HTTP client with auth resolution (env var → keyring → config) and error handling
  • judgment/config.py — TOML-based config management at ~/.judgment/config.toml
  • judgment/output.py — Output formatting (rich tables, JSON mode, styled messages)
  • judgment/commands/auth.pylogin, logout, whoami
  • judgment/commands/config_cmd.pyconfig set, config show, config path
  • judgment/commands/orgs.pyorgs list, orgs show, orgs members, orgs usage
  • judgment/commands/projects.pyprojects list, projects create, projects show, projects favorite, projects unfavorite

Modified files:

  • judgment/cli.py — Registers new command groups, adds global --json, --no-color, --org-id, --project-id options
  • pyproject.toml — Adds httpx, rich, keyring, pydantic, tomli/tomli-w dependencies

🎯 Purpose

Provides the scaffolding so that future CLI command groups (traces, evaluations, scorers, etc.) can be added by following the same pattern: create a Typer sub-app in judgment/commands/, use JudgmentAPIClient for HTTP, use output.py helpers for display.

Updates since initial revision

Fixed issues flagged by auto-review:

  • Python 3.8+ compatibility: Added from __future__ import annotations to all new modules so dict[str, Any] type hints work on Python 3.8/3.9
  • Keyring write on login: judgment login now stores the API key in both the config file and the system keyring
  • Fixed org_id context lookup: _resolve_org_id() in projects.py now correctly walks ctx.meta up the context chain instead of using a broken getattr(ctx, "org_id")
  • Removed unused imports: Cleaned up sys from api_client.py and Optional/os from config.py

🧪 Testing

Manual validation only:

  • pip install -e . succeeds
  • judgment --help shows all five command groups (self-host, config, auth, orgs, projects)
  • judgment config show works with no config file present
  • judgment config path prints ~/.judgment/config.toml
  • judgment auth --help, judgment orgs --help, judgment projects --help show expected subcommands

No unit tests were added.

✅ Checklist

  • Self-review
  • Video demo of changes
  • Unit Tests and CI/CD tests are passing
  • Reviewers assigned

Key items for human review

  1. No persistent HTTP client: Each API call creates a new httpx.get/post/... instead of using a persistent httpx.Client session. Acceptable for a CLI (one-shot commands), but worth noting for future batch operations.
  2. No unit tests: All validation was manual --help and config show checks. Consider adding tests before merging.
  3. Unused import in config_cmd.py: print_detail is imported but not used.
  4. ctx.meta fallback: _resolve_org_id() accesses current.meta which could theoretically be None on some Typer/Click context objects — verify this doesn't raise on edge cases.

📌 Linear Issue

N/A

✏️ Additional Notes

Session Link: https://app.devin.ai/sessions/81260eac71b34b638f85968cb63b0dd0
Requested by: @raoabinav

Future work:

  • Add unit tests for config/api_client/output modules
  • Add integration tests against a mock/staging API
  • Add traces, evaluations, scorers command groups

…roups

- Add new dependencies: httpx, rich, keyring, pydantic, tomli, tomli-w
- Add config management (judgment/config.py) with TOML config at ~/.judgment/config.toml
- Add shared HTTP client (judgment/api_client.py) with auth and error handling
- Add output formatting (judgment/output.py) with rich tables, JSON, styled messages
- Add config commands: judgment config set/show/path
- Add auth commands: judgment login/logout/whoami
- Add org commands: judgment orgs list/show/members/usage
- Add project commands: judgment projects list/create/show/favorite/unfavorite
- Add global options: --json, --no-color, --org-id, --project-id
- Register all new command groups in cli.py while keeping self-host intact

Co-Authored-By: abi <abinav_rao@berkeley.edu>
@devin-ai-integration
Copy link
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

…s, fix org_id context lookup, store API key in keyring on login

Co-Authored-By: abi <abinav_rao@berkeley.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant