Introduce visible-reasoning agent to resolve conflicting chain-of-thought guidance#183
Merged
Merged
Conversation
… for reason.prompt.md Agent-Logs-Url: https://github.com/Bryan-Roe/Aria/sessions/c4dc5aba-a88f-41ea-8264-102d07344b71 Co-authored-by: Bryan-Roe <74067792+Bryan-Roe@users.noreply.github.com>
Contributor
🔴 Coverage —
|
| Metric | Value |
|---|---|
| Total coverage | 59.7% |
→ vs main |
0.0% |
| Minimum threshold | 60% |
Updated on every push · 2026-05-09
github-actions
Bot
added
the
documentation
Copilot
AI
changed the title
[WIP] Fix conflicting agent description regarding chain-of-thought
Introduce Apr 30, 2026
visible-reasoning agent to resolve conflicting chain-of-thought guidance
Contributor
Reviewer's GuideIntroduces a new Sequence diagram for visible reasoning with the new visible-reasoning agentsequenceDiagram
actor Developer
participant CopilotExtension
participant ReasonPrompt as reason_prompt_md
participant VisibleReasoningAgent as visible-reasoning_agent_md
participant AGIProvider as AGIProvider_verbose
participant LLM
Developer->>CopilotExtension: Request structured visible reasoning
CopilotExtension->>ReasonPrompt: Load prompt configuration
ReasonPrompt-->>CopilotExtension: agent=visible-reasoning
CopilotExtension->>VisibleReasoningAgent: Invoke with problem description
VisibleReasoningAgent->>AGIProvider: create_agi_provider(enable_chain_of_thought=true,\nverbose=true, enable_self_reflection=true,\nenable_task_decomposition=true, reasoning_depth=3)
AGIProvider->>LLM: Send request with reasoning enabled
LLM-->>AGIProvider: Reasoning_steps + final_answer
AGIProvider-->>VisibleReasoningAgent: Structured reasoning_steps + outcome
VisibleReasoningAgent-->>CopilotExtension: Response with Analyze, Decompose, Execute, Reflect, Synthesize phases
CopilotExtension-->>Developer: Display full visible chain_of_thought response
Class diagram for AGI provider and the visible-reasoning agent behaviorclassDiagram
class VisibleReasoningAgent {
+string name
+string description
+list tools
+bool exposes_chain_of_thought
+bool is_temporary_specialist
+create_handoff_summary(string outcome, string decision, string assumptions, string risks, string next_action) string
}
class AGIProvider {
+bool enable_chain_of_thought
+bool enable_self_reflection
+bool enable_task_decomposition
+int reasoning_depth
+bool verbose
+AGIContext context
+generate_response(string input) string
}
class AGIContext {
+list conversation_history
+list reasoning_chains
+list goals
+list learned_patterns
+add_step(ReasoningStep step) void
}
class ReasoningStep {
+string step_type
+string content
+float confidence
+map metadata
}
VisibleReasoningAgent --> AGIProvider : configures
AGIProvider o-- AGIContext : uses
AGIContext o-- ReasoningStep : aggregates
class AGIProviderVisible {
+bool enable_chain_of_thought = true
+bool enable_self_reflection = true
+bool enable_task_decomposition = true
+int reasoning_depth = 3
+bool verbose = true
}
class AGIProviderInternal {
+bool enable_chain_of_thought = true
+bool enable_self_reflection = true
+bool enable_task_decomposition = true
+int reasoning_depth = 3
+bool verbose = false
}
AGIProviderVisible --|> AGIProvider
AGIProviderInternal --|> AGIProvider
class AGIReasoningAgent {
+bool exposes_chain_of_thought = false
}
AGIReasoningAgent --> AGIProviderInternal : uses
VisibleReasoningAgent --> AGIProviderVisible : uses
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- The
visible-reasoningspec hard-requires all five phases for every request; consider explicitly allowing a collapsed or abbreviated flow for trivial/simple queries to avoid overly verbose responses that may frustrate users. - The
Return-to-Agent Contractassumes anagentorchestrator will always be present; it may be worth clarifying how this behaves whenvisible-reasoningis invoked directly (e.g., fromreason.prompt.md) to avoid confusing or redundant handoff text in user-facing output.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The `visible-reasoning` spec hard-requires all five phases for every request; consider explicitly allowing a collapsed or abbreviated flow for trivial/simple queries to avoid overly verbose responses that may frustrate users.
- The `Return-to-Agent Contract` assumes an `agent` orchestrator will always be present; it may be worth clarifying how this behaves when `visible-reasoning` is invoked directly (e.g., from `reason.prompt.md`) to avoid confusing or redundant handoff text in user-facing output.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a dedicated visible-reasoning agent and routes reason.prompt.md to it, resolving contradictory “show reasoning” vs “hide reasoning” guidance when reason.prompt.md and agi.prompt.md previously shared the same agi-reasoning agent.
Changes:
- Adds
.github/agents/visible-reasoning.agent.mddefining a visible step-by-step reasoning contract. - Updates
.github/prompts/reason.prompt.mdto useagent: visible-reasoningand adjusts its description accordingly. - Updates
.github/copilot-instructions.mdto document both agents distinctly and clarifies thatreason.prompt.mdusesvisible-reasoning.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
.github/prompts/reason.prompt.md |
Switches the prompt’s agent to visible-reasoning and updates the prompt description to match visible reasoning. |
.github/copilot-instructions.md |
Documents the new agent and clarifies the intended hidden vs visible reasoning split across prompts/agents. |
.github/agents/visible-reasoning.agent.md |
Adds the new visible reasoning agent definition and its “Show Your Work” contract. |
Comment on lines
+2
to
+3
| name: visible-reasoning | ||
| description: "Visible step-by-step reasoning agent. Specializes in showing explicit chain-of-thought analysis, task decomposition, and structured reasoning to the user — every reasoning step is exposed in the response.\n\nTrigger phrases include:\n- 'show your reasoning'\n- 'walk me through your thinking'\n- 'explain step by step'\n- 'think through this out loud'\n- 'reason through this with me'\n- 'show all steps'\n- 'transparent reasoning'\n\nExamples:\n- User says 'walk me through your reasoning on this architecture decision' → invoke to expose each reasoning step\n- User asks 'explain step by step how you would debug this' → invoke for visible decomposition and analysis\n- User says 'think through this out loud' → invoke to surface all analysis and decision steps in the response\n\nDifference from agi-reasoning: agi-reasoning keeps chain-of-thought internal (only delivers the final answer). This agent surfaces reasoning explicitly so the user can follow, verify, and learn from each step." |
There was a problem hiding this comment.
The front-matter description claims agi-reasoning keeps chain-of-thought internal and "only delivers the final answer", but .github/agents/agi-reasoning.agent.md does not currently state that constraint. Consider either updating agi-reasoning.agent.md to explicitly require hidden reasoning (to match this statement) or softening this wording here so it doesn’t assert behavior the agent definition doesn’t guarantee.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Bryan <74067792+Bryan-Roe@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Bryan <74067792+Bryan-Roe@users.noreply.github.com>
Signed-off-by: Bryan <74067792+Bryan-Roe@users.noreply.github.com>
Contributor
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. Scanned FilesNone |
![bryan-roe-bot[bot]](https://avatars.githubusercontent.com/in/1073263?s=60&v=4){kind=small}
| ## Workspace Context | ||
|
|
||
| - **Provider chain**: Azure OpenAI → OpenAI → LMStudio → LoRA → Local | ||
| - **Provider chain**: explicit choice → LM Studio → Ollama → AGI → Quantum → Azure OpenAI → OpenAI → Local fallback → LoRA (source of truth: `detect_provider()` in `ai-projects/chat-cli/src/chat_providers.py`) |
Contributor
There was a problem hiding this comment.
Semgrep identified an issue in your code:
Possibly found usage of AI: OpenAI
To resolve this comment:
🔧 No guidance has been designated for this issue. Fix according to your organization's approved methods.
💬 Ignore this finding
Reply with Semgrep commands to ignore this finding.
/fp <comment>for false positive/ar <comment>for acceptable risk/other <comment>for all other reasons
Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by detect-generic-ai-oai.
You can view more details about this finding in the Semgrep AppSec Platform.
Contributor
|
Semgrep found 1 Possibly found usage of AI: OpenAI |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aae877fa9c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - Data: read-only `datasets/`, write-only `data_out/` | ||
| - Always `--dry-run` orchestrators first | ||
| - Test: `python scripts/test_runner.py --unit` | ||
| --- |
There was a problem hiding this comment.
This commit rewrites every line in this Markdown file using CRLF (\r\n) without changing content, which creates a no-op full-file diff and unnecessary churn; the same issue also appears in .github/copilot-instructions.md. Because the repo’s .gitattributes specifies *.md text eol=lf, committing CRLF here undermines that policy and makes future blame/merge review noisier for no functional gain.
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
agi-reasoningis used by bothagi.prompt.md(chain-of-thought hidden, final answer only) andreason.prompt.md(explicit "show your reasoning at each step") — directly contradictory instructions on the same agent.Changes
New agent —
.github/agents/visible-reasoning.agent.md: dedicated agent for exposed step-by-step reasoning. Core contract is "Show Your Work" — all five phases (Analyze → Decompose → Execute → Reflect → Synthesize) must appear in the response. Explicitly distinguishes itself fromagi-reasoning:reason.prompt.md: switchesagent: agi-reasoning→agent: visible-reasoning; description updated to reflect visible output contract.copilot-instructions.md: agents table now lists both agents with distinct purpose descriptions; prompts list clarifies thatreason.prompt.mdusesvisible-reasoningfor exposed chain-of-thought vs.agi.prompt.mdwhich usesagi-reasoningfor internal-only reasoning.Result
agi.prompt.mdagi-reasoningreason.prompt.mdvisible-reasoningSummary by Sourcery
Introduce a dedicated visible-reasoning agent and wire it into the structured reasoning prompt so visible chain-of-thought is explicitly separated from internal-only AGI reasoning.
New Features:
Enhancements: