Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
355 changes: 123 additions & 232 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,256 +4,147 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

## Project Overview

NUTS (Network Universal Testing Suite) is a Rust CLI tool for API testing, performance testing, and security scanning. It features an interactive shell with tab completion, AI-powered command suggestions, and OpenAPI flow management.
NUTS (Network Universal Testing Suite) is a Rust CLI tool for MCP server testing and API testing. It operates in two modes:

1. **Non-interactive CLI** -- `nuts call`, `nuts mcp discover`, `nuts mcp test`, etc. Designed for scripts and CI.
2. **Interactive REPL** -- `nuts shell` enters the original interactive shell with tab completion.

AI features (security scanning, test generation) use Anthropic's Claude API via a centralized `AiService`.

## Development Commands

### Build and Run
```bash
cargo build # Build the project
cargo run # Run the CLI tool
cargo install --path . # Install locally
```
cargo run # Run the CLI (shows help, not REPL)
cargo run -- shell # Enter the interactive REPL
cargo install --path . # Install locally as `nuts` binary

### Testing
```bash
cargo test # Run all tests
cargo test --lib # Run library tests only
cargo test --bin nuts # Run binary tests only
cargo test --lib # Library tests only
cargo test --bin nuts # Binary tests only
cargo test <test_name> # Run a single test by name
cargo test mcp # Run MCP-related tests

cargo fmt # Format code (CI enforces --check)
cargo clippy --all-targets --all-features -- -D warnings # Lint (CI treats warnings as errors)
cargo check # Quick compile check
```

### Code Quality
```bash
cargo fmt # Format code
cargo clippy # Run linter
cargo check # Check for compile errors
```
## CI Requirements

The GitHub Actions CI pipeline (`.github/workflows/ci.yml`) runs on PRs to `main` and `develop`:
- `cargo fmt -- --check` -- formatting must pass
- `cargo clippy --all-targets --all-features -- -D warnings` -- no clippy warnings allowed
- `cargo test --verbose`
- `cargo doc --no-deps --document-private-items`
- Cross-platform builds (Linux, Windows, macOS)
- `cargo audit` for security vulnerabilities

## Architecture

### Core Components
### Execution Flow (main.rs)

`main.rs` uses `clap` derive macros for CLI parsing. The `Cli` struct has a `Commands` enum with subcommands: `Call`, `Perf`, `Security`, `Ask`, `Mcp`, `Config`, `Shell`. No subcommand prints brief help.

Each `run_*` function creates a `tokio::runtime::Runtime` and calls `block_on` for async work. The `shell` subcommand delegates to `NutsShell::run()` (the original REPL).

Global flags: `--json`, `--quiet`, `--no-color`, `--verbose`, `--env`. TTY detection sets `NO_COLOR` when piped.

### MCP Commands (`nuts mcp <subcommand>`)

MCP subcommands use `McpTransportArgs` for transport selection (`--stdio`, `--sse`, `--http`). The `resolve_transport()` function converts these to a `TransportConfig` enum.

Working subcommands:
- `nuts mcp connect --stdio "cmd"` -- connect, print server info, disconnect
- `nuts mcp discover --stdio "cmd" [--json]` -- full capability listing
- `nuts mcp generate --stdio "cmd" [--json]` -- AI-generate test YAML from discovered schemas

Placeholder subcommands (not yet wired): `test`, `perf`, `security`, `snapshot`.

- **`src/main.rs`** - Entry point that initializes the shell
- **`src/shell.rs`** - Main shell implementation with command processing
- **`src/commands/`** - Command implementations (call, perf, security, config, monitor, etc.)
- **`src/flows/`** - OpenAPI flow management and collection system
- **`src/models/`** - Data structures for analysis and metrics
- **`src/config.rs`** - Configuration management with API key storage
- **`src/completer.rs`** - Tab completion for shell commands
- **`src/story/`** - AI-guided workflow system
### Error Handling (`src/error.rs`)

### Key Features
`NutsError` enum with `thiserror` + `miette`:
- Variants: `Http`, `Ai`, `Config`, `Mcp`, `Protocol`, `Flow`, `AuthRequired`, `Io`, `InvalidInput`
- `pub type Result<T> = std::result::Result<T, NutsError>;`
- Auto-conversions from `reqwest::Error`, `serde_json::Error`, `serde_yaml::Error`, `std::io::Error`

1. **Interactive Shell** - Uses `rustyline` for command line editing with tab completion
2. **API Testing** - HTTP client with support for all common methods
3. **Performance Testing** - Concurrent load testing with configurable parameters
4. **Security Scanning** - AI-powered security analysis using Anthropic's Claude
5. **OpenAPI Flows** - Create, manage, and execute API collections
6. **Mock Server** - Generate mock servers from OpenAPI specifications
7. **Story Mode** - AI-guided API workflow exploration
8. **Health Monitoring** - Real-time API health monitoring with AI insights
9. **Natural Language Interface** - AI-powered command generation from natural language
### MCP Module (`src/mcp/`)

### Configuration
- **`client.rs`** -- `McpClient` wraps the `rmcp` SDK. Methods: `connect_stdio`, `connect_sse`, `connect_http`, `connect` (from `TransportConfig`), `discover`, `list_tools`, `call_tool`, `list_resources`, `read_resource`, `list_prompts`, `get_prompt`, `disconnect`.
- **`types.rs`** -- Data types: `ServerCapabilities`, `Tool`, `Resource`, `ResourceTemplate`, `Prompt`, `PromptArgument`, `ToolResult`, `ContentItem` (Text/Image/Audio/Resource), `ResourceContent`, `PromptResult`, `PromptMessage`, `TransportConfig` (Stdio/Sse/Http).
- **`discovery.rs`** -- `discover()` convenience function, `format_discovery_human()`, `format_discovery_json()`.
- **`generate.rs`** -- `generate_tests(client, ai)` discovers tools and AI-generates YAML test cases. Handles markdown fence stripping.
- **`test_runner.rs`** -- YAML test file parser and assertion engine. `TestFile`/`ServerConfig`/`TestCase`/`TestStep` structs. `run_tests(path)` connects to server, executes tests, returns `TestSummary`. Supports captures (`$.field` JSONPath) and variable references (`${var}`). Human and JSON summary formatters.

- Configuration stored in `~/.nuts_config.json`
- Flow collections stored in `~/.nuts/flows/`
- API key required for AI features (security scanning, story mode, monitoring, natural language)
### AI Module (`src/ai/`)

- **`service.rs`** -- `AiService` holds a `Box<dyn AiProvider>`, tracks token usage, maintains conversation buffer. Methods: `complete()`, `complete_with_system()`, `chat()`, `converse()`, `generate_test_cases()`, `security_scan()`, `explain()`, `validate_output()`, `suggest_command()`. Default model: `claude-sonnet-4-5-20250929`.
- **`provider.rs`** -- `AiProvider` trait + `AnthropicProvider`. `MockProvider` for tests.
- **`prompts.rs`** -- All prompt templates as functions. MCP-specific: `mcp_test_generation()`, `mcp_security_scan()`, `mcp_output_validation()`. API: `api_security_analysis()`, `command_suggestion()`, `explain_response()`, `natural_language_command()`, etc.

### Output Module (`src/output/`)

- **`renderer.rs`** -- `render_status_line()`, `render_json_body()` (syntax highlighted), `render_headers()`, `render_table()` (comfy-table), `render_error()` (what/why/fix), `render_ai_insight()`, `render_test_result()`, `render_section()`, `spinner_style()`. `OutputMode` enum: Human/Json/Junit/Quiet.
- **`colors.rs`** -- Semantic color system using `console` crate. `init_colors()`, `colors_enabled()`. Styles: success, warning, error, info, muted, accent (+ bold). JSON: json_key (cyan), json_string (green), json_number (yellow), json_bool (magenta), json_null (dim red). Respects `NO_COLOR`.
- **`welcome.rs`** -- `welcome_message()` (3 lines), `first_run_message()`, `help_text()` (grouped by task: MCP TESTING, MAKING REQUESTS, etc.), `command_help(cmd)` per-command help.

### Legacy Modules (unchanged)

- **`src/shell.rs`** -- Interactive REPL with rustyline, tab completion, command dispatch via `process_command()` match block
- **`src/commands/`** -- Original command implementations (call, perf, security, ask, monitor, generate, etc.)
- **`src/flows/`** -- OpenAPI flow management and collection system
- **`src/story/`** -- AI-guided workflow exploration
- **`src/models/`** -- `ApiAnalysis`, `Metrics`, `MetricsSummary`
- **`src/config.rs`** -- Config stored at `~/.nuts/config.json`
- **`src/completer.rs`** -- Tab completion for shell mode

## Key Patterns

- **Error handling**: New modules use `NutsError` via `thiserror`/`miette`. Legacy code still uses `Box<dyn Error>`. `main.rs` run functions return `Box<dyn Error>` to bridge both.
- **AI integration**: New code uses `AiService` (one instance, shared). Legacy commands still create per-call `anthropic::client::Client`.
- **Async**: Each `run_*` function in main.rs creates its own `tokio::runtime::Runtime`. The shell creates one in `NutsShell::run()`.
- **MCP transport**: All MCP commands resolve `--stdio`/`--sse`/`--http` flags to `TransportConfig`, then call `McpClient::connect()`.
- **User data**: All persisted data lives under `~/.nuts/` (config, flows). No database.

## MCP Test YAML Format

Test files (`.test.yaml`) have `server:` (transport config) and `tests:` (array of test cases). See `docs/mcp-test-format.md` for the full specification. Key structures:

```yaml
server:
command: "node server.js" # or sse: / http:
timeout: 30

tests:
- name: "test name"
tool: "tool_name" # or resource: / prompt:
input: { key: value }
assert:
status: success
result.type: object
result.has_field: [id, name]
duration_ms: { max: 5000 }
capture:
var_name: "$.field.path"

- name: "multi-step"
steps:
- tool: "create"
input: { title: "test" }
capture: { id: "$.id" }
- tool: "get"
input: { id: "${id}" }
```

### Dependencies
## Dependencies

- **UI/UX**: `ratatui`, `crossterm`, `console`, `inquire`, `dialoguer`
Key crates:
- **CLI**: `clap` (derive macros)
- **MCP**: `rmcp` (client, transport-child-process, transport-streamable-http, transport-sse)
- **AI**: `anthropic` client crate
- **Error**: `thiserror`, `miette` (fancy diagnostics)
- **HTTP**: `reqwest`, `axum`, `hyper`, `tower`
- **AI**: `anthropic` client
- **CLI**: `clap` for argument parsing, `rustyline` for shell
- **Output**: `comfy-table`, `console`, `indicatif`, `crossterm`
- **Serialization**: `serde`, `serde_json`, `serde_yaml`
- **Async**: `tokio` runtime

## Complete Command Reference

### Core Commands

#### `call [OPTIONS] [METHOD] URL [BODY]`
Advanced HTTP client with CURL-like features
- **Options**: `-H` (headers), `-u` (basic auth), `--bearer` (token), `-v` (verbose), `-L` (follow redirects)
- **Examples**:
```bash
call GET https://api.example.com/users
call POST https://api.example.com/users '{"name": "John"}'
call -H "Content-Type: application/json" -v POST https://api.example.com/users '{"name": "John"}'
```

#### `ask "natural language request"`
AI-powered natural language to API call conversion
- **Examples**:
```bash
ask "Create a POST request with user data"
ask "Get all products from the API"
ask "Delete user with ID 123"
```

#### `perf [METHOD] URL [--users N] [--duration Ns] [BODY]`
Performance testing with concurrent load testing
- **Options**: `--users` (concurrent users), `--duration` (test duration)
- **Examples**:
```bash
perf GET https://api.example.com/users
perf GET https://api.example.com/users --users 100 --duration 30s
perf POST https://api.example.com/users --users 50 '{"name": "Test"}'
```

#### `security URL [--deep] [--auth TOKEN] [--save FILE]`
AI-powered security vulnerability scanning
- **Options**: `--deep` (thorough analysis), `--auth` (authentication token), `--save` (save results)
- **Examples**:
```bash
security https://api.example.com
security https://api.example.com --deep --auth "Bearer token123"
security https://api.example.com --save security_report.json
```

#### `monitor <URL> [--smart]`
Real-time API health monitoring with AI insights
- **Functionality**:
- Performs health checks every 30 seconds
- Monitors response times and status codes
- Detects issues (slow responses, errors, empty responses)
- With `--smart` flag: AI analysis every 3rd check providing trend analysis, predictions, and recommendations
- **Examples**:
```bash
monitor https://api.example.com
monitor https://api.example.com --smart
```

#### `discover <BASE_URL>`
Auto-discover API endpoints and generate OpenAPI specifications
- **Examples**:
```bash
discover https://api.example.com
```

#### `test "description" [base_url]`
AI-driven test case generation from natural language
- **Examples**:
```bash
test "Check if user registration works"
test "Verify pagination works correctly" https://api.example.com
```

#### `generate <data_type> [count]`
AI-powered realistic test data generation
- **Examples**:
```bash
generate users 10
generate products 5
generate orders 20
```

#### `predict <BASE_URL>`
AI-powered API health prediction and forecasting
- **Examples**:
```bash
predict https://api.example.com
```

#### `explain`
AI explains the last API response in human-friendly terms
- **Examples**:
```bash
explain
```

#### `fix <URL>`
AI-powered automatic API issue detection and fixing
- **Examples**:
```bash
fix https://api.example.com/broken-endpoint
```

#### `config [api-key|show]`
Configuration management
- **Examples**:
```bash
config api-key
config show
```

### Flow Management Commands

#### `flow new <name>`
Create a new OpenAPI flow collection
- **Examples**:
```bash
flow new myapi
flow new user-management
```

#### `flow add <name> <METHOD> <path>`
Add an endpoint to an existing flow
- **Examples**:
```bash
flow add myapi GET /users
flow add myapi POST /users
```

#### `flow run <name> <endpoint>`
Execute a specific endpoint from a flow
- **Examples**:
```bash
flow run myapi /users
flow run myapi /users/123
```

#### `flow list`
List all available flows
- **Examples**:
```bash
flow list
```

#### `flow docs <name>`
Generate documentation for a flow
- **Examples**:
```bash
flow docs myapi
```

#### `flow mock <name> [port]`
Start a mock server from OpenAPI specification
- **Examples**:
```bash
flow mock myapi
flow mock myapi 8080
```

#### `flow story <name>`
Start AI-guided interactive workflow exploration
- **Examples**:
```bash
flow story myapi
flow s myapi # shorthand
```

#### `flow configure_mock_data <name> <endpoint>`
Configure mock data for specific endpoints
- **Examples**:
```bash
flow configure_mock_data myapi /users
```

### Command Aliases
- `c` → `call`
- `p` → `perf`
- `s` → `flow story`
- `h` → `help`
- `q` → `quit`

## Development Notes

- Uses async/await throughout with tokio runtime
- Error handling with custom `ShellError` type
- Progress indicators with `indicatif` crate
- All user data stored in home directory under `.nuts/`
- AI features require Anthropic API key configuration
- Monitor command performs health checks every 30 seconds with optional AI analysis
- Natural language commands leverage Claude AI for intelligent command generation
- **Async**: `tokio`, `async-trait`
- **Shell**: `rustyline`
7 changes: 6 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,11 @@ hyper = { version = "1.0", features = ["full"] }
tower = "0.4"
axum-server = "0.6"
chrono = { version = "0.4", features = ["serde"] }
thiserror = "2"
miette = { version = "7", features = ["fancy"] }
comfy-table = "=7.1.3"
regex = "1"
rmcp = { version = "0.8", features = ["client", "transport-child-process", "transport-streamable-http-client-reqwest", "reqwest", "transport-sse-client-reqwest"] }
[[bin]]
name = "nuts"
path = "src/main.rs"
path = "src/main.rs"
Loading
Loading