Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
[alias]
t = "test --workspace --features cupcake-core/deterministic-tests"
test-deterministic = "test --workspace --features cupcake-core/deterministic-tests"
t = "test --workspace"
build-dist = "build --profile dist --bin cupcake"

# Distribution profile for optimized release builds
Expand Down
8 changes: 4 additions & 4 deletions .github/WINDOWS_ROUTING_TESTS_DEBUG.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ Fixed in `cupcake-core/tests/claude_code_routing_test.rs`:
```bash
cd C:/Users/Administrator/cupcake
export CLAUDE_CLI_PATH="/c/Users/Administrator/AppData/Roaming/npm/claude.cmd"
cargo test --features deterministic-tests --test claude_code_routing_test
cargo test --test claude_code_routing_test

running 10 tests
test test_multiple_events_routing ... ok
Expand Down Expand Up @@ -322,18 +322,18 @@ Check `C:\temp\cupcake_out.txt` for any error messages.

### Run all routing tests locally (if you have Claude CLI):
```bash
cargo test --test claude_code_routing_test --features deterministic-tests -- --nocapture
cargo test --test claude_code_routing_test -- --nocapture
```

### Run specific test:
```bash
cargo test --test claude_code_routing_test test_pretooluse_routing --features deterministic-tests -- --nocapture
cargo test --test claude_code_routing_test test_pretooluse_routing -- --nocapture
```

### Set environment for Claude CLI path:
```powershell
$env:CLAUDE_CLI_PATH = "C:\npm\prefix\claude.ps1"
cargo test --test claude_code_routing_test --features deterministic-tests -- --nocapture
cargo test --test claude_code_routing_test -- --nocapture
```

## Success Criteria
Expand Down
10 changes: 5 additions & 5 deletions .github/instructions/pr-review.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ cupcake-rewrite/
**Review test coverage:**
- [ ] Check that unit tests exist for new functions/modules
- [ ] Check that integration tests exist for new features
- [ ] Verify test code uses `--features deterministic-tests` pattern
- [ ] Verify tests don't rely on shared global state between parallel test cases
- [ ] Review test code to ensure tests are deterministic (no race conditions)
- [ ] Check that tests clean up resources (temp files, directories)

Expand Down Expand Up @@ -453,13 +453,13 @@ paths.targets_protected(file_path, ".cupcake/")

**Check that tests document the correct execution pattern:**
```bash
cargo test --features deterministic-tests
cargo test --workspace

# Or use the alias
cargo t
```

**Why this matters:** The trust system uses HMAC key derivation which is non-deterministic in production mode. Without this flag, tests will fail intermittently. When reviewing, look for:
**Why this matters:** Tests must be deterministic and isolated from shared global state, or they fail intermittently in CI. When reviewing, look for:
- Test documentation mentions this requirement (in comments or README)
- Test code structure suggests deterministic execution
- No randomness or timing dependencies in test logic
Expand Down Expand Up @@ -515,7 +515,7 @@ const SYSTEM_EVAL: &str = include_str!("fixtures/system_evaluate.rego");
```bash
# If policy changes don't appear in tests
cargo clean -p cupcake-core
cargo test --features deterministic-tests
cargo test --workspace
```

---
Expand Down Expand Up @@ -709,7 +709,7 @@ pub fn process(input: &str) -> Result<Output> {
### Before Submitting PR (Author Checklist)

**Authors should have:**
- [ ] Verified tests locally with `--features deterministic-tests`
- [ ] Verified tests locally with `cargo test --workspace`
- [ ] Formatted code with `cargo fmt`
- [ ] Addressed warnings from `cargo clippy`
- [ ] Updated documentation
Expand Down
13 changes: 6 additions & 7 deletions .github/instructions/security-review.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,14 +240,13 @@ commands.has_verb(cmd, "rm")

**Test Execution Requirements:**
```bash
# CRITICAL: Must use deterministic-tests feature
cargo test --features deterministic-tests
cargo test --workspace

# Or use alias
cargo t
```

**Why:** The trust system uses HMAC key derivation which is non-deterministic in production mode. The feature flag ensures deterministic keys for reliable test execution.
**Why:** Tests must run deterministically and isolated from shared global state to be reliable in CI.

**Test Structure Changes:**
- Moved `tests/test_helpers.rs` → `tests/common/mod.rs` (proper Rust test pattern)
Expand Down Expand Up @@ -621,7 +620,7 @@ pub async fn evaluate(&self, input: &Value) {
- ✅ Tests have clear comments explaining attack vectors
- ✅ Tests verify both blocking behavior AND error messages
- ✅ Tests clean up resources (temp directories, symlinks)
- ✅ Tests use `--features deterministic-tests` in documentation/CI config
- ✅ Tests are isolated and don't depend on shared global state across parallel cases

---

Expand All @@ -646,7 +645,7 @@ pub async fn evaluate(&self, input: &Value) {

### Test Review

- [ ] Review test structure to ensure they use `--features deterministic-tests`
- [ ] Review test structure to ensure tests are deterministic and isolated
- [ ] Check that tests cover positive and negative cases
- [ ] Verify tests use proper harness-specific helpers
- [ ] Check that tests clean up resources (TempDir, etc.)
Expand Down Expand Up @@ -719,8 +718,8 @@ If anything is unclear during review, consider asking:
- Are there linting tools to catch old patterns?

4. **Testing:**
- Why is `--features deterministic-tests` required?
- What would happen without this flag?
- Are tests deterministic and isolated from shared global state?
- Could parallel execution cause flakiness?
- Are there known flaky tests?

5. **Cross-Platform:**
Expand Down
6 changes: 4 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,13 @@ Cupcake uses OPA v1.71.0+ where Rego v1 is the default syntax.

## Testing Requirements

- **Feature Flag (MANDATORY):** All Rust tests **MUST** be run with the `--features deterministic-tests` flag for stable HMAC key generation.
- **Running tests:** Run the workspace test suite with:
```bash
cargo test --workspace --features cupcake-core/deterministic-tests
cargo test --workspace
# or: just test
```
(Note: the `cupcake-py` crate needs a Python dev environment to build; scope to
`-p cupcake-core -p cupcake-cli` if that toolchain is unavailable.)
- **Harness Testing (CRITICAL):** When testing a specific harness (e.g., Claude Code), use the dedicated helper function to avoid compilation errors:
```rust
// CORRECT
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Thank you for your interest in contributing to Cupcake! We welcome contributions
2. Fork the repository and create a branch from `main`
3. Make your changes, following the code style of the project
4. Add or update tests as appropriate
5. Ensure all tests pass: `just test` (or `cargo test --workspace --features cupcake-core/deterministic-tests`)
5. Ensure all tests pass: `just test` (or `cargo test --workspace`)
6. Submit a pull request

## Contributor License Agreement
Expand Down
2 changes: 1 addition & 1 deletion examples/fixtures/context_injection.rego
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,5 @@ add_context contains context_msg if {
change_keywords := {"modify", "update", "change", "fix", "add"}
some keyword in change_keywords
contains(lower(input.prompt), keyword)
context_msg := "Reminder: Run tests with 'cargo test --features deterministic-tests' after making changes."
context_msg := "Reminder: Run tests with 'cargo test' after making changes."
}