Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
074e449
refactor(env): remove config fields from environment and update usage
tusharmath Apr 1, 2026
dba8012
refactor(reasoning): clarify API instructions and update examples
tusharmath Apr 1, 2026
5a5ad31
refactor(reasoning): clarify API instructions and update examples
tusharmath Apr 1, 2026
3cf5822
refactor(reasoning): clarify API instructions and update examples
tusharmath Apr 1, 2026
ad3e867
feat(reasoning): add configurable reasoning settings to all agents
tusharmath Apr 1, 2026
46205dc
refactor(agents): merge reasoning config with agent settings prioriti…
tusharmath Apr 1, 2026
a50da3b
refactor(agents): improve reasoning config merging logic with agent-l…
tusharmath Apr 1, 2026
b240bb7
[autofix.ci] apply automated fixes
autofix-ci[bot] Apr 1, 2026
2a1e637
refactor(config): expand Effort enum and token budget mapping
tusharmath Apr 1, 2026
e1552fb
refactor(conversation_record): expand Effort enum and update conversions
tusharmath Apr 1, 2026
ca6f3dd
refactor(reasoning): expand Effort enum and update descriptions
tusharmath Apr 1, 2026
905e2f7
feat(reasoning): add comprehensive reasoning serialization tests and …
tusharmath Apr 2, 2026
58fd5e4
test(reasoning): add comprehensive serialization tests for reasoning …
tusharmath Apr 2, 2026
a592461
test(reasoning): add invalid effort tests and update reasoning script
tusharmath Apr 2, 2026
02aa6a5
test(reasoning): add tests for openrouter reasoning parameters and ef…
tusharmath Apr 2, 2026
50156bb
test(reasoning): add tests for reasoning scripts and parameters
tusharmath Apr 2, 2026
6454557
feat(reasoning): implement dynamic reasoning configuration serializat…
tusharmath Apr 2, 2026
7bfd2ba
[autofix.ci] apply automated fixes
autofix-ci[bot] Apr 2, 2026
f6512a0
refactor(script): improve JSON parsing with raw_decode for NDJSON han…
tusharmath Apr 2, 2026
813c397
Merge branch 'main' into reasoning-implementation
tusharmath Apr 2, 2026
795c69d
fix(reasoning): address PR review comments
tusharmath Apr 2, 2026
5fa3bfa
[autofix.ci] apply automated fixes
autofix-ci[bot] Apr 2, 2026
7c1ee66
Merge branch 'main' into reasoning-implementation
tusharmath Apr 2, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions .forge/skills/test-reasoning/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
name: test-reasoning
description: Validate that reasoning parameters are correctly serialized and sent to provider APIs. Use when the user asks to test reasoning serialization, run reasoning tests, verify reasoning config fields, or check that ReasoningConfig maps correctly to provider-specific JSON (OpenRouter, Anthropic, GitHub Copilot, Codex).
---

# Test Reasoning Serialization

Validates that `ReasoningConfig` fields are correctly serialized into provider-specific JSON
for OpenRouter, Anthropic, GitHub Copilot, and Codex.

## Quick Start

Run all tests with the bundled script:

```bash
./scripts/test-reasoning.sh
```

The script builds forge in debug mode, runs each provider/model combination, captures the
outgoing HTTP request body via `FORGE_DEBUG_REQUESTS`, and asserts the correct JSON fields.

## Running a Single Test Manually

```bash
FORGE_DEBUG_REQUESTS="forge.request.json" \
FORGE_SESSION__PROVIDER_ID=<provider_id> \
FORGE_SESSION__MODEL_ID=<model_id> \
FORGE_REASONING__EFFORT=<effort> \
target/debug/forge -p "Hello!"
```

Then inspect `.forge/forge.request.json` for the expected fields.

## Test Coverage

| Provider | Model | Config fields | Expected JSON field |
| ---------------- | ---------------------------- | ------------------------------------------------- | --------------------------------- |
| `open_router` | `openai/o4-mini` | `effort: none\|minimal\|low\|medium\|high\|xhigh` | `reasoning.effort` |
| `open_router` | `openai/o4-mini` | `max_tokens: 4000` | `reasoning.max_tokens` |
| `open_router` | `openai/o4-mini` | `effort: high` + `exclude: true` | `reasoning.effort` + `.exclude` |
| `open_router` | `openai/o4-mini` | `enabled: true` | `reasoning.enabled` |
| `open_router` | `anthropic/claude-opus-4-5` | `max_tokens: 4000` | `reasoning.max_tokens` |
| `open_router` | `moonshotai/kimi-k2` | `max_tokens: 4000` | `reasoning.max_tokens` |
| `open_router` | `moonshotai/kimi-k2` | `effort: high` | `reasoning.effort` |
| `open_router` | `minimax/minimax-m2` | `max_tokens: 4000` | `reasoning.max_tokens` |
| `open_router` | `minimax/minimax-m2` | `effort: high` | `reasoning.effort` |
| `anthropic` | `claude-opus-4-6` | `effort: low\|medium\|high\|max` | `output_config.effort` |
| `anthropic` | `claude-3-7-sonnet-20250219` | `enabled: true` + `max_tokens: 8000` | `thinking.type` + `budget_tokens` |
| `github_copilot` | `o4-mini` | `effort: none\|minimal\|low\|medium\|high\|xhigh` | `reasoning_effort` (top-level) |
| `codex` | `gpt-5.1-codex` | `effort: none\|minimal\|low\|medium\|high\|xhigh` | `reasoning.effort` + `.summary` |
| `codex` | `gpt-5.1-codex` | `effort: medium` + `exclude: true` | `reasoning.summary = "concise"` |
| all providers | one model each | `effort: invalid` | non-zero exit, no request written |

Tests for unconfigured providers are skipped automatically. Invalid-effort tests run regardless of credentials — the rejection happens at config parse time before any provider interaction.

## References

- [OpenAI Reasoning guide](https://developers.openai.com/api/docs/guides/reasoning)
- [OpenAI Chat Completions API reference](https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create)
- [Anthropic Extended Thinking](https://platform.claude.com/docs/en/build-with-claude/effort)
- [OpenRouter Reasoning Tokens](https://openrouter.ai/docs/guides/best-practices/reasoning-tokens)
Loading
Loading