feat(agentic):Add Battleship agentic RL demo with web UI and autoplay comparison

[tal-ent]

Type of Change

• ✨ New Feature

Related Issue

#191

Overview

This experiment implements a multi-turn agentic RL example for the AReno framework — Battleship. The rules: on an 8×8 grid, the model makes sequential decisions via the fire tool call, targeting and sinking a hidden fleet [4, 3, 2, 2]; sinking all ships within 64 turns constitutes a win. Per the Proposed feature requirements, the framework remains unmodified; integration with areno occurs at three接缝 points only: data loading / trajectory collection / reward. Game rules are centralized in game.py as the single source of truth; a single game is modeled as a multi-turn tool conversation. Verification is two-tiered: pure CPU evaluation and baseline evaluation / web UI.

New Files

examples/agentic/battleship/
├── game.py              # Core game logic
├── reward.py            # RL reward function
├── run_agent.py         # areno agent entry point
├── dataset_generator.py # Training/eval JSONL generation
├── dataset_loader.py    # JSONL → areno prompt records
├── evaluate.py          # Offline baseline players (random/fake/heuristic)
├── play_llm.py          # LLM evaluation against any OpenAI-compatible endpoint
├── compare_modes.py     # Head-to-head comparison of multiple strategies on the same fleet sequence
├── web_ui.py            # Browser-based demo
└── README.md            # Rules, quick start, reward shaping, limitations

tests/
├── test_agentic_battleship_example_cpu.py
└── test_compare_modes_cpu.py

Issues Identified During Review

1. _tool_messages() raises TypeError: GameState not JSON serializable — The 4 error branches in _run_tool stuff the internally bookkept GameState into the "state" key, which is then fed directly to json.dumps.
2. LLM repeatedly fires at the same cell — The prompt includes a "do not repeat" rule, but the model cannot see which coordinates it has already fired.

Solutions

1. Strip the state key before serialization in _tool_messages() — Does not adopt the default= serializer approach. Reason: GameState.ships contains all ship positions; serializing it would leak the answer to the model.
2. Add explicit instructions to the system prompt in run_agent.py — Teach the LLM how to play Battleship, including the rules and strategy.

✅ Self-Check Checklist

• Follows project CONTRIBUTING.md / AGENTS.md conventions
• All local pytest tests/ -k cpu tests pass
• No existing public API broken
• Code formatted