Agent Loop Implementation + Smart Retrieval + Attachment Uploading

[sweettastebuds]

This pull request updates the documentation and configuration to reflect a major refactor and feature expansion of the forge-bot project, transitioning from a simple webhook handler with sandboxed code execution to a sophisticated AI bot platform supporting tool-calling, per-event workspace containers, YAML-driven API clients, and advanced verification systems. The .env.example and CLAUDE.md files are updated for clarity and accuracy, and a new Dockerfile.workspace is added for workspace container support. The changes also introduce improved test structure documentation, environment variable descriptions, and update legacy references to match the new architecture.

Documentation and Architecture Updates:

• Expanded CLAUDE.md to describe new features: tool-calling loop, per-event workspace containers, YAML-driven API client, tool system, status comment manager, verification system, and container lifecycle. Added details on new modules, critical implementation details, and command patterns. [1] [2]
• Added a "Test Structure" section to CLAUDE.md, listing key test files and their purposes, and clarified the style guide for code and tests.

Configuration and Environment Variables:

• Updated .env.example to include new variables for containerized workspace support (CONTAINER_*), clarified LLM and provider settings, and removed deprecated sandbox options.
• Improved documentation of optional environment variables in CLAUDE.md with a table of defaults and descriptions for new and existing settings.

Containerization Support:

• Added Dockerfile.workspace to define the workspace container image used for per-event isolation, supporting repo cloning and tool execution.
• Updated build instructions in CLAUDE.md to include workspace image creation.

Deprecation and Refactoring:

• Deprecated legacy sandbox subsystem in favor of new container manager, updating documentation and removing outdated references.

Minor Cleanup:

• Removed unnecessary copy of the prompts/ directory in the main Dockerfile (likely moved to a new location per refactor).

[Copilot]

Pull request overview

This PR upgrades forge-bot from a simple webhook + sandbox approach to a more capable platform: a YAML-driven Forge API client, per-event workspace containers, tool-calling agent loop, and a new “status + response” comment system, with a large accompanying test suite refactor/additions.

Changes:

• Replace the hardcoded Forge client with a YAML-defined GenericForgeClient and new API schema/loader.
• Introduce per-event persistent ContainerManager and status comment observability (StatusCommentManager + formatter).
• Add a multi-level “smart retrieval” subsystem (BM25 + parallel scanning + multi-hop) and extensive new tests; remove legacy sandbox implementation/tests.

Reviewed changes

Copilot reviewed 81 out of 83 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
tests/test_status.py	Adds unit tests for status comment formatting + manager behavior.
tests/test_server.py	Updates server lifecycle mocking to patch GenericForgeClient.call() instead of legacy client.
tests/test_sandbox/test_parser.py	Removes legacy sandbox parser tests.
tests/test_sandbox/test_orchestrator.py	Removes legacy sandbox orchestrator tests.
tests/test_sandbox/test_images.py	Removes legacy sandbox image registry tests.
tests/test_sandbox/conftest.py	Removes sandbox-specific shared fixtures.
tests/test_router.py	Minor cleanup in router tests file formatting.
tests/test_retrieval/test_tool.py	Adds tests for the new workspace-backed retrieval tool behavior.
tests/test_retrieval/test_token_budget.py	Adds tests for retrieval token budgeting helpers.
tests/test_retrieval/test_pipeline.py	Adds tests for the SmartRetriever orchestration logic.
tests/test_retrieval/test_level3.py	Adds tests for multi-hop Level 3 agent tool-calling flow.
tests/test_retrieval/test_level2.py	Adds tests for Level 2 parallel scanning + answering behavior.
tests/test_retrieval/test_level1.py	Adds tests for Level 1 BM25-based retrieval behavior.
tests/test_retrieval/test_chunking.py	Adds tests for retrieval chunking utilities.
tests/test_retrieval/test_bm25.py	Adds tests for the pure-Python BM25 implementation.
tests/test_retrieval/init.py	Adds retrieval test package marker.
tests/test_rag/test_pipeline.py	Updates RAG pipeline tests to use the generic API client mock.
tests/test_rag/test_ingester.py	Updates ingester tests to assert against GenericForgeClient.call() dispatch.
tests/test_rag/test_embedder.py	Small formatting update to embedder test expectations.
tests/test_rag/conftest.py	Replaces mock_forge fixture with mock_api_client using .call() dispatch.
tests/test_pull_request_handler.py	Rewrites PR handler tests for new agent loop + status + container lifecycle.
tests/test_forge_client.py	Removes legacy hardcoded Forge client tests.
tests/test_container_manager.py	Adds tests for the new per-event container manager.
tests/test_config.py	Updates config tests for container settings replacing sandbox settings.
tests/test_api_schema.py	Adds tests for Pydantic models backing YAML API definitions.
tests/test_api_loader.py	Adds tests for YAML loader caching + provider resolution.
tests/test_api_client.py	Adds tests for GenericForgeClient call/param handling + error propagation.
sandbox-images.json	Removes legacy sandbox image override file.
requirements.txt	Adds PyYAML dependency for API definition loading.
pyproject.toml	Bumps project version to 0.2.0.
forge_bot/utils/token_budget.py	Adds a (separate) token budgeting utility for LLM sessions.
forge_bot/utils/thinking.py	Adds helper to strip/log <think>...</think> blocks.
forge_bot/tools/base.py	Introduces base classes for tool-calling (schema + prompt text).
forge_bot/tools/init.py	Adds tools package marker.
forge_bot/status/manager.py	Implements status + response comment manager with attachment support.
forge_bot/status/formatter.py	Adds markdown formatting helpers for todos and tool call history.
forge_bot/status/init.py	Adds status package marker.
forge_bot/server.py	Switches server lifecycle to generic API client; adds error-comment attempt on failures; bumps API wiring.
forge_bot/sandbox/parser.py	Removes legacy sandbox command parsing.
forge_bot/sandbox/orchestrator.py	Removes legacy sandbox orchestrator.
forge_bot/sandbox/images.py	Removes legacy sandbox image registry.
forge_bot/sandbox/init.py	Removes sandbox package marker.
forge_bot/router.py	Updates dispatch plumbing to use GenericForgeClient; removes legacy /run and /index paths.
forge_bot/retrieval/token_budget.py	Adds retrieval-specific token budgeting and truncation helpers.
forge_bot/retrieval/pipeline.py	Adds SmartRetriever facade and diff review routing.
forge_bot/retrieval/level3.py	Adds Level 3 multi-hop agent that calls Level 2 as a tool.
forge_bot/retrieval/level2.py	Adds Level 2 parallel chunk relevance scanning + refinement + answer generation.
forge_bot/retrieval/level1.py	Adds Level 1 BM25 retrieval with LLM keyword expansion.
forge_bot/retrieval/chunking.py	Adds chunking utilities for text and per-file diffs.
forge_bot/retrieval/bm25.py	Adds pure-Python BM25 implementation.
forge_bot/retrieval/init.py	Adds retrieval package marker.
forge_bot/rag/store.py	Formatting-only changes in vector store operations.
forge_bot/rag/retriever.py	Adds optional max token truncation control for PR context retrieval.
forge_bot/rag/pipeline.py	Updates RAG pipeline to use GenericForgeClient.
forge_bot/rag/ingester.py	Refactors ingester to use .call() endpoints + adds concurrent file fetching.
forge_bot/rag/embedder.py	Makes local embedding execution async via asyncio.to_thread.
forge_bot/rag/chunker.py	Formatting updates / signature formatting for readability.
forge_bot/prompts/agent_system.j2	Adds new agent system prompt template for tool loop usage.
forge_bot/prompts/agent_pr_review.j2	Adds new PR review prompt template for the agent loop.
forge_bot/handlers/pull_request.py	Replaces old diff-fetch + single LLM call with container + agent loop + status updates + artifacts.
forge_bot/handlers/base.py	Switches handler base to generic API client and stable prompts directory resolution.
forge_bot/container/manager.py	Adds the per-event persistent container manager (create/exec/destroy).
forge_bot/container/init.py	Adds container package marker.
forge_bot/config.py	Replaces sandbox settings with container settings + adds FORGE_PROVIDER.
forge_bot/clients/llm.py	Adds chat_with_tools() for function-calling flows.
forge_bot/clients/forge.py	Removes legacy Forge client implementation.
forge_bot/api/schema.py	Adds Pydantic schema models for YAML endpoint definitions.
forge_bot/api/loader.py	Adds YAML loader with caching and provider resolution.
forge_bot/api/client.py	Adds GenericForgeClient implementation and multipart handling.
forge_bot/api/init.py	Adds API package marker.
docker-compose.yml	Updates Compose config for .env loading and DinD TLS wiring.
README.md	Updates docs to reflect new architecture (agent loop, containers, YAML API client).
Dockerfile.workspace	Adds workspace image Dockerfile for per-event containers.
Dockerfile	Removes legacy prompts/ copy step (prompts are now under forge_bot/prompts).
.env.example	Updates env var docs for new provider/container settings.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The error comment body includes the raw exception message (str(e)). This can leak internal details (and potentially sensitive info) into a public issue/PR thread. Consider logging the full exception server-side but posting a generic error message (or a redacted/hashed error ID) to the forge.

[Copilot]

BaseTool.parameters is initialized with dataclasses.field(...), but BaseTool is not a @DataClass. That means parameters becomes a dataclasses.Field object, and iterating over self.parameters in to_openai_schema()/to_prompt_text() will fail unless every subclass overrides it. Use a normal class attribute (e.g., [] or ClassVar[list[ToolParameter]]) or make BaseTool a dataclass if instance-level parameters are intended.

[Copilot]

_INIT_SCRIPT's construction of /usr/local/bin/forge-api via a single giant printf string has mismatched quoting (the final argument includes a stray "'"), which is likely to produce an invalid shell script and prevent the container from ever emitting FORGE_READY. Consider switching to a heredoc (cat <<'EOF' ... EOF) or writing the script as a small embedded multi-line string with clear quoting.

Suggested change

	"printf '%s\\n' '#!/bin/sh' "
	"'METHOD=\"${1:-GET}\"' "
	"'ENDPOINT=\"$2\"' "
	"'shift 2 2>/dev/null' "
	"'exec curl -sf "
	'-H "Authorization: token $FORGE_TOKEN" '
	'-H "Content-Type: application/json" '
	'-X "$METHOD" "$FORGE_URL/api/v1$ENDPOINT" "$@"\' '
	"> /usr/local/bin/forge-api && "
	"cat <<'EOF' > /usr/local/bin/forge-api\n"
	"#!/bin/sh\n"
	'METHOD="${1:-GET}"\n'
	'ENDPOINT="$2"\n'
	"shift 2 2>/dev/null\n"
	'exec curl -sf \\\n'
	' -H "Authorization: token $FORGE_TOKEN" \\\n'
	' -H "Content-Type: application/json" \\\n'
	' -X "$METHOD" "$FORGE_URL/api/v1$ENDPOINT" "$@"\n'
	"EOF\n"

[Copilot]

DOCKER_CERT_PATH is set to /certs/client/client, but the volume is mounted at /certs/client. With Docker-in-Docker TLS this path typically needs to point at the directory containing ca.pem/cert.pem/key.pem (i.e., /certs/client). As-is, the bot container may fail to connect to the dind daemon.

Suggested change

	- DOCKER_CERT_PATH=/certs/client/client
	- DOCKER_CERT_PATH=/certs/client

[Copilot]

In the error handler, the API endpoint name "create_issue_comment" isn't defined in the YAML API definitions (they use "post_issue_comment"). This will cause the attempt to notify users about failures to always error. Rename the endpoint to the correct defined name (and keep params aligned with the YAML schema).

[Copilot]

_extract_comment_target() requires repository.owner.login and repository.name, but the webhook payloads used elsewhere (and in tests) only provide repository.full_name. As a result, this function will return None and the bot won't post an error comment when it crashes. Consider falling back to parsing repository.full_name ("owner/repo") when owner/name fields are absent.