Multi-Agent Research Workbench

Ask a research question and get a footnoted, citation-checked Markdown report — produced by an open, inspectable team of agents you can watch work, not a black box.

Most "deep research" tools hand you a polished answer and hide how it was reached. This workbench does the opposite: every handoff between a Supervisor, a Planner, parallel Researchers, a Critic, and a Writer is a logged, queryable event, and the plan itself is scored against your question. You can see why the system believes what it wrote, and where it had to loop back because a claim wasn't sourced.

Who it's for

• Analysts & technical writers who need a first-draft literature scan with real citations, not confident hallucinations.
• Agent engineers who want a working reference for a genuine multi-agent topology (4 distinct roles + a re-research loop) on LangGraph — not a single ReAct loop in a costume.
• Anyone evaluating agentic systems who needs trajectory-level eval (did the plan match the question?) alongside output faithfulness.

Architecture

flowchart TD
    Q([Research question]) --> S[Supervisor]
    S --> P[Planner<br/>Tree-of-Thought<br/>→ 3–5 sub-queries]
    P --> R{Researchers<br/>parallel, ReWOO}
    R --> R1[Researcher · sub-q 1<br/>Tavily search]
    R --> R2[Researcher · sub-q 2<br/>Tavily search]
    R --> R3[Researcher · sub-q N<br/>Tavily search]
    R1 --> C[Critic<br/>rejects uncited claims]
    R2 --> C
    R3 --> C
    C -->|uncited claims found<br/>≤ 2 loops| R
    C -->|all claims cited<br/>or budget spent| W[Writer<br/>footnoted Markdown]
    W --> OUT([Report + References])

    S -.handoff events.-> EV[(Event log<br/>Postgres + structured trace)]
    P -.-> EV
    R -.-> EV
    C -.-> EV
    W -.-> EV

Loading

ReWOO (Reasoning WithOut Observation) means each Researcher plans its tool use, runs one search, then synthesises once — avoiding the observation explosion of interleaved ReAct loops. The Critic is a deterministic gate: a claim with no source, or one that never points to its sources, is rejected and sent back for re-research (bounded to 2 loops, then the Writer proceeds with what's verified and the report says so).

Run it

# 1. Offline demo — deterministic backends, no keys, no network
pip install -e ".[dev]"
python scripts/run_research.py --mock \
  --question "What are the trade-offs of ReWOO versus ReAct for tool-using agents?"

# 2. Live research — needs Gemini + Tavily keys
cp .env.example .env          # then fill in GOOGLE_API_KEY and TAVILY_API_KEY
make up                       # Postgres for the state log + scoreboard
python scripts/run_research.py \
  --question "How do hierarchical agent teams divide research work?"

# 3. Interactive UI
streamlit run app.py

# 4. Golden-set eval + CI regression gate
python scripts/run_eval.py --mock --out eval_artifact.json
python scripts/ci_gate.py --baseline tests/golden/baseline.json \
  --current eval_artifact.json --tolerance 0.05

GOOGLE_API_KEY + TAVILY_API_KEY enable live mode. With no keys the workbench runs deterministic offline backends so every path — tests, smoke, the UI — works with zero spend.

Why this stack

• LangGraph 0.3.x — the topology is a real state machine with a conditional re-research edge and a loop budget; LangGraph gives it checkpointing and a place to hang traces. The same node functions also run in a dependency-free sequential runtime, so behaviour is unit-testable.
• Gemini 2.5 Pro for Planner/Researcher/Writer reasoning; gpt-4o-mini only as the G-Eval faithfulness judge — a cheap, separate model judging the expensive one avoids self-grading bias.
• Tavily for live web search — research-grade snippets with URLs the Critic can actually check.
• Postgres for the handoff event log and eval scoreboard — the agent trail is queryable SQL, not scrollback.
• Non-LLM trajectory eval — plan-vs-question alignment is pure code, so it's deterministic, free, and runs in CI with no key.

Operational characteristics

• Latency (live): ~15–40 s per question — one Planner call, N parallel Researcher calls (1 search + 1 synthesis each), one Critic pass, one Writer pass; +1 Researcher round per re-research loop.
• Cost (live, ballpark): a few US cents per question on Gemini 2.5 Pro at 3–5 sub-queries; G-Eval adds a fraction of a cent per report on gpt-4o-mini.
• Eval at ship: non-LLM trajectory score is logged per question over a 25-question set; G-Eval faithfulness is logged when run with an OpenAI key. CI enforces a 5% regression tolerance against a frozen baseline.
• Parallelism: Researchers fan out via a thread pool (logical parallelism, LLMCompiler-style) — not a distributed worker pool.

Honest disclaimer

This is a portfolio-grade reference implementation. What is real versus illustrative, in plain language:

Component	Status	Notes
Supervisor / Planner / Researcher / Critic / Writer agents	Real	Full LangGraph topology with a bounded conditional re-research loop.
Tavily web search	Real	Live search when TAVILY_API_KEY is set.
Gemini 2.5 Pro reasoning	Real	Live when GOOGLE_API_KEY is set.
Critic citation gate	Real	Deterministic code: no source / no inline cite ⇒ rejected.
Trajectory eval (plan vs question)	Real	Deterministic, non-LLM, runs in CI.
G-Eval faithfulness judge	Real, but needs a key	gpt-4o-mini; skipped in offline/CI mode.
Handoff event log	Real	Stable schema; Postgres + structured logs.
Arize Phoenix trace UI	Illustrative / optional	An optional compose profile. The workbench emits the same structured events with or without it; it is not required for any feature.
RAG-over-arXiv source	Not included	The locked scope ships Tavily-only; vector-store retrieval is deferred.
Offline --mock backends	Illustrative	Canned, non-intelligent stand-ins for demos/tests/CI — not a research engine.
25-question golden set	Curated sample	A representative slice for regression detection, not a benchmark.

License

MIT © 2026 Pralay Ghosh