diff --git a/.hermes/skills/predict-rlm/SKILL.md b/.hermes/skills/predict-rlm/SKILL.md new file mode 100644 index 00000000..ab23946f --- /dev/null +++ b/.hermes/skills/predict-rlm/SKILL.md @@ -0,0 +1,1118 @@ +--- +name: predict-rlm +description: Plan and build an RLM (Recursive Language Model) with predict-rlm. Guides through interactive planning (goals, inputs, outputs, skills, architecture) then implements the code. Use when creating autonomous document/code analysis agents that need multi-step reasoning. +version: "1.0" +author: Adapted from Trampoline-AI/predict-rlm for Hermes Agent +metadata: + hermes: + tags: [rlm, dspy, predict-rlm, agent-building, document-analysis, recursive-lm] + category: mlops + related_skills: [dspy, llama-cpp] + requires_packages: [predict-rlm, dspy] +--- + +# Build an RLM with predict-rlm + +An **RLM (Recursive Language Model)** is a callable, pre-configured agent that: +- Autonomously explores large inputs (documents, datasets, codebases) +- Writes and executes code in a sandboxed Pyodide/WASM REPL +- Calls tools and sub-LMs via `predict()` for perception/extraction +- Iterates until task completion with fully interpretable trajectories + +Unlike a chat agent, an RLM is a **function** — you define its inputs, outputs, and tools, then call it from your code. It returns structured data, not chat messages. + +> **🔧 Hermes Integration:** The generated RLM code automatically uses **your Hermes Agent's configured model** (`HERMES_MODEL`) for both the main LM and sub-LM by default. No manual model configuration needed — it inherits from your existing setup. + +**This skill has two phases:** +1. **Plan** — interactively define the RLM with the user, research feasibility +2. **Build** — implement the plan as code files with automatic Hermes model integration + +--- + +## Phase 1: Plan + +Use the `todo` tool to track progress through these steps. Work interactively — ask the user questions and confirm alignment before moving on. + +### Step 1: Goal Definition + +Understand what the user wants to build. + +**Ask:** +- What is the desired outcome? What does success look like? +- What is the input material? (documents, code, data, APIs, etc.) +- What does the output look like? (structured report, modified files, spreadsheet, etc.) + +**Validate RLM fit.** An RLM is the right tool when: +- ✅ Input is large and needs selective exploration (documents, datasets, codebases) +- ✅ Task is multi-step with tool use (extract → transform → validate) +- ✅ Actions modify state (redaction, form filling, generation) +- ✅ Parallel sub-LM calls needed across many items +- ✅ File-to-file transformations (PDFs → spreadsheets, documents → reports) + +**Not a fit:** Single LLM call, simple script, or straightforward API call — suggest alternative. + +### Step 2: Input Design + +Define every input to the RLM. + +For each input, determine: +- **Name** and **type**: `File`, `list[File]`, `str`, or Pydantic model +- **Description**: what it contains and how the RLM uses it +- **Source**: user-provided file, API response, config, generated data + +**Key principles:** +- Large content (PDFs, images, datasets) must be `File` references — RLM accesses content on-demand through skills, keeping its context small +- Metadata (file paths, page counts, config flags) can be strings or Pydantic models +- Use `list[File]` for variable-count file inputs + +Confirm input design with the user before proceeding. + +### Step 3: Output Design + +Define the structured output. + +For each output field, determine: +- **Name**, **type**, and **description** +- Whether it's a Pydantic model (structured data), `File` (generated file), or primitive + +**Push for specificity** — vague outputs lead to poor RLM performance. Sketch Pydantic models with `Field(description=...)` annotations. + +**Ask:** +- What fields matter most? What would they check first? +- Are there computed/derived fields (scores, summaries, counts)? +- Do they need output files (Excel, PDF, images)? + +Confirm output design before proceeding. + +### Step 4: Research (Autonomous) + +Tell the user you're researching, then proceed autonomously. + +Use `web_search` and `delegate_task` to: + +1. **Find Python packages** for the domain (e.g., `networkx` for graphs, `tree-sitter` for code parsing) + +2. **Check Pyodide compatibility.** The sandbox runs Pyodide (Python in WASM). Only **pure-Python wheels** or packages with **Emscripten builds** work. For each package: + - Search pypi.org for `py3-none-any` wheel (pure Python = works) + - Check Pyodide built-in list: https://pyodide.org/en/stable/usage/packages-in-pyodide.html + - Flag C extensions without Emscripten builds (won't work in sandbox) + +3. **Identify network needs.** Does the task require calling external APIs? Note domains for `allowed_domains`. + +4. **Identify host-side tool needs.** If functionality cannot run in WASM (native binaries, C extensions, heavy computation), it must be a **host-side tool**. + +5. **Check existing skills.** Built-in skills available: + - `pdf` — pymupdf for PDF rendering, text extraction, manipulation + - `spreadsheet` — openpyxl, pandas, formulas for Excel work + - `docx` — python-docx for Word documents + +Report findings to the user with a clear feasibility assessment. Flag any blockers. + +### Step 5: Skill Design + +Based on research, design the skill configuration. + +**Built-in skills:** +List which built-in skills to use and why. + +**Custom skills (if needed):** +For each custom skill: +- **name**: short identifier +- **instructions**: prose guidance injected into RLM's system prompt +- **packages**: PyPI packages installed in sandbox via micropip (Pyodide-compatible) +- **modules**: Python files mounted into sandbox as importable modules +- **tools**: host-side callable functions exposed to RLM + +**Host-side tool design:** +For each host-side tool: +- Function name and signature with type hints +- Docstring (RLM sees this to understand how to call it) +- What it does and why it must be host-side + +Confirm skill design before proceeding. + +### Step 6: Strategy and Architecture + +**Signature strategy:** +Write the step-by-step strategy for the signature's docstring. This is the RLM's playbook: +1. What to do first (survey/understand the input) +2. How to gather information (render pages, use predict() for extraction, call tools) +3. How to process and synthesize +4. What to produce and where to save output files + +**Single vs chained RLMs:** + +| Single RLM | Chained RLMs | +|------------|--------------| +| One coherent workflow | Distinct phases with different skill needs | +| Same context/state throughout | Artifacts passed between stages | +| Under 40 iterations | Combined task exceeds reasonable iteration counts | +| | Different phases benefit from different sub-LM models | + +**Configuration:** +- `max_iterations` estimate per RLM +- `allowed_domains` if network access needed +- `lm` / `sub_lm` configuration: + - By default, use Hermes's configured model (`HERMES_MODEL`) for both + - Consider using a cheaper/faster model for `sub_lm` if doing many `predict()` calls + - For complex chained RLMs, different stages may benefit from different model capabilities + +### Feasibility Checklist + +Before producing final plan, verify: + +- [ ] All packages are Pyodide-compatible (or have host-side fallbacks) +- [ ] Network access needs identified with specific domains +- [ ] Host-side tools defined for WASM-incompatible functionality +- [ ] Iteration count reasonable (under 50 per RLM) +- [ ] Input sizes manageable (or chunking strategy defined) +- [ ] Output schemas specific enough for reliable extraction +- [ ] Task is achievable — no unsupported capabilities assumed + +### Plan Output + +Write the plan to a markdown file in the workspace with these sections: + +1. **Overview** — one paragraph: what, why, and expected workflow +2. **File manifest** — every file to create with one-line description +3. **Input schemas** — complete Pydantic model code +4. **Output schemas** — complete Pydantic model code +5. **Signature** — complete `signature.py` code with strategy docstring +6. **Skills configuration** — built-in imports + custom `Skill()` definitions + tool signatures +7. **Service architecture** — single RLM wiring or chained DAG +8. **Feasibility notes** — constraints, risks, alternatives +9. **Estimated complexity** — iteration count, sub-LM calls, cost range, runtime +10. **LM Configuration** — By default uses `HERMES_MODEL` for both LM and sub-LM; note if different models recommended for specific stages + +Get user approval. Once approved, proceed to Phase 2. + +--- + +## Phase 2: Build + +Implement the approved plan. Create all files following the patterns below. + +> **Note on Model Configuration:** The generated `service.py` automatically reads `HERMES_MODEL` and `HERMES_PROVIDER` from the environment. The RLM will use **the same model you're currently using in Hermes** without any additional configuration. You can override this by passing explicit `lm` or `sub_lm` parameters. + +### File Structure + +``` +my_rlm/ +├── __init__.py # Public exports (service class, schema, signature) +├── schema.py # Pydantic models for inputs AND outputs +├── signature.py # DSPy Signature (inputs/outputs + strategy docstring) +├── service.py # DSPy Module wiring signature + PredictRLM + skills +└── skills.py # (optional) Custom skill definitions +``` + +**Always create:** `schema.py`, `signature.py`, `service.py`, `__init__.py` +**Create when needed:** `skills.py` (only if domain-specific instructions beyond built-in skills) + +### schema.py — Pydantic Models + +```python +from pydantic import BaseModel, Field + + +class KeyDate(BaseModel): + """A key date extracted from a document.""" + + name: str = Field(description="e.g. 'Submission Deadline', 'Effective Date'") + date: str = Field(description="ISO format date (YYYY-MM-DD)") + time: str | None = Field( + None, description="24-hour format (HH:MM), e.g. '14:00'" + ) + + +class DocumentAnalysis(BaseModel): + """Structured analysis of a document set.""" + + report: str = Field(description="Full analysis as markdown report") + key_dates: list[KeyDate] = Field( + default_factory=list, description="Important dates found" + ) +``` + +### signature.py — Strategy & I/O + +The docstring becomes the RLM's system instructions: + +```python +import dspy +from predict_rlm import File +from .schema import DocumentAnalysis + + +class AnalyzeDocuments(dspy.Signature): + """Analyze documents and produce a structured report. + + 1. **Read the report criteria** (appended below) to understand what + information to extract and in what format. + + 2. **Survey the documents** — understand file names, page counts, types. + + 3. **Gather information** systematically by rendering pages as images + and using predict() to extract content. + + 4. **Produce the report** following the format specified in the criteria. + """ + + documents: list[File] = dspy.InputField(desc="PDF documents to analyze") + analysis: DocumentAnalysis = dspy.OutputField( + desc="Structured analysis with markdown report and key dates" + ) +``` + +### service.py — Wiring It Together + +```python +import os +import yaml +import dspy +from pathlib import Path +from predict_rlm import File, PredictRLM +from predict_rlm.skills import pdf as pdf_skill +from .schema import DocumentAnalysis, KeyDate, Entity, FinancialItem +from .signature import AnalyzeDocuments + + +def get_hermes_config() -> dict: + """Read Hermes Agent configuration from ~/.hermes/config.yaml.""" + config_path = Path.home() / ".hermes" / "config.yaml" + if config_path.exists(): + with open(config_path) as f: + return yaml.safe_load(f) or {} + return {} + + +def get_hermes_lm() -> dspy.LM: + """Configure DSPy LM from Hermes config.yaml (reads active profile). + + Reads the active Hermes model and provider from ~/.hermes/config.yaml + and creates a configured dspy.LM instance with proper LiteLLM formatting. + Handles Fireworks, OpenRouter, Anthropic, and OpenAI providers. + + Returns: + Configured dspy.LM instance using Hermes's current model + + Example: + >>> lm = get_hermes_lm() + >>> predictor = PredictRLM(signature, lm=lm, sub_lm=lm) + """ + config = get_hermes_config() + + # Get model from config + model = config.get("model", {}).get("default", "openai/gpt-4o") + base_url = config.get("model", {}).get("base_url", "") + + # Build provider-specific config + lm_kwargs = {} + + # Find provider API config and determine provider prefix + providers = config.get("providers", {}) + provider_prefix = None + api_key = None + + # Match provider by URL and extract API key + if base_url: + # Find matching provider config before modifying base_url + for name, provider_config in providers.items(): + provider_url = str(provider_config.get("api", "")) + # Match if either URL contains the other + if base_url in provider_url or provider_url in base_url: + api_key = provider_config.get("api_key") + if api_key: + lm_kwargs["api_key"] = api_key + break + + # For openai/ prefix with Fireworks, pass the base URL as-is + # LiteLLM will append /chat/completions automatically + if "fireworks" in str(base_url).lower(): + # Use the v1 endpoint directly, LiteLLM handles the rest + lm_kwargs["base_url"] = base_url + + # Determine provider prefix from original URL + if "fireworks" in str(base_url).lower(): + provider_prefix = "fireworks" + elif "openrouter" in str(base_url).lower(): + provider_prefix = "openrouter" + elif "anthropic" in str(base_url).lower(): + provider_prefix = "anthropic" + elif "openai" in str(base_url).lower(): + provider_prefix = "openai" + + # Check env vars for API keys as fallback + if "api_key" not in lm_kwargs: + if "fireworks" in str(base_url).lower() or model.startswith("openai/"): + # For Fireworks or openai/ prefix with Fireworks endpoint + lm_kwargs["api_key"] = os.getenv("FIREWORKS_API_KEY") + elif "openrouter" in str(base_url).lower() or model.startswith("openrouter/"): + lm_kwargs["api_key"] = os.getenv("OPENROUTER_API_KEY") + elif "anthropic" in str(base_url).lower(): + lm_kwargs["api_key"] = os.getenv("ANTHROPIC_API_KEY") + + # Format model string for LiteLLM compatibility + # For Fireworks router models, use the openai/ prefix with the full model path + # openai/ + base_url works with any OpenAI-compatible endpoint including Fireworks + + has_provider_prefix = any( + model.startswith(p + "/") for p in ["fireworks", "openrouter", "anthropic", "openai", "deepseek", "custom"] + ) + + if not has_provider_prefix: + if "fireworks" in str(base_url).lower() or provider_prefix == "fireworks": + # Use openai/ prefix for Fireworks OpenAI-compatible endpoint + # This tells LiteLLM to use OpenAI client with our custom base_url + model = f"openai/{model}" + elif provider_prefix: + model = f"{provider_prefix}/{model}" + elif base_url: + model = f"custom/{model}" + else: + model = f"openrouter/{model}" + + return dspy.LM(model=model, **lm_kwargs) + + +class DocumentAnalyzer(dspy.Module): + def __init__( + self, + lm: dspy.LM | None = None, # Main LM (defaults to Hermes model) + sub_lm: dspy.LM | None = None, # Sub-LM for predict() calls + max_iterations: int = 30, + verbose: bool = False, + debug: bool = False, + ): + self.lm = lm or get_hermes_lm() + self.sub_lm = sub_lm or self.lm + self.max_iterations = max_iterations + self.verbose = verbose + self.debug = debug + + async def aforward( + self, + documents: list[File], + criteria: str | None = None + ) -> DocumentAnalysis: + signature = AnalyzeDocuments + if criteria: + signature = AnalyzeDocuments.with_instructions( + AnalyzeDocuments.instructions + "\n\n# Additional Criteria\n\n" + criteria.strip() + ) + + predictor = PredictRLM( + signature, + lm=self.lm, + sub_lm=self.sub_lm, + skills=[pdf_skill], + max_iterations=self.max_iterations, + verbose=self.verbose, + debug=self.debug, + ) + result = await predictor.acall(documents=documents) + return result.analysis +``` + +**Multiple skills or host-side tools:** + +```python +from predict_rlm.skills import pdf as pdf_skill +from predict_rlm.skills import spreadsheet as spreadsheet_skill + +async def aforward(self, documents: list[File]) -> MyOutput: + lm = self.lm or get_hermes_lm() + sub_lm = self.sub_lm or lm + + predictor = PredictRLM( + MySignature, + lm=lm, # Main orchestrator + sub_lm=sub_lm, # Perception/extraction calls + skills=[pdf_skill, spreadsheet_skill], + tools={"fetch_exchange_rate": fetch_exchange_rate}, + max_iterations=30, + ) + return await predictor.acall(documents=documents) +``` + +**Chaining pattern (multiple RLMs):** + +```python +async def aforward(self, documents: list[File]): + lm = self.lm or get_hermes_lm() + sub_lm = self.sub_lm or lm + + # Stage 1: Extract + extractor = PredictRLM( + ExtractSignature, + lm=lm, + sub_lm=sub_lm, + skills=[pdf_skill] + ) + extracted = await extractor.acall(documents=documents) + + # Stage 2: Analyze (can use same or different sub_lm) + analyzer = PredictRLM( + AnalyzeSignature, + lm=lm, + sub_lm=sub_lm, + skills=[analysis_skill] + ) + result = await analyzer.acall(data=extracted.data) + + return result +``` + +### skills.py — Custom Skills (Optional) + +```python +from predict_rlm import Skill +from predict_rlm.skills import pdf as pdf_skill + +redaction_skill = Skill( + name="redaction", + instructions="""How to redact content from PDFs using pymupdf. + +## Text redaction +Search for text, create redaction annotations, then apply: + page = doc[page_num] + hits = page.search_for("sensitive text") + for rect in hits: + page.add_redact_annot(rect, fill=(0, 0, 0)) + page.apply_redactions() +""", +) + +__all__ = ["pdf_skill", "redaction_skill"] +``` + +--- + +## Report Generation Pattern + +RLMs excel at extracting structured data, but often you need to present that data as a polished report. Add a `generate_markdown_report()` function to transform raw extraction output into formatted output matching the Trampoline-AI example style. + +**Add to schema.py:** + +```python +from datetime import datetime +from typing import Literal + + +def _format_date_for_sorting(date_str: str) -> str: + """Convert various date formats to YYYY-MM-DD for sorting.""" + if len(date_str) == 10 and date_str[4] == '-' and date_str[7] == '-': + return date_str + + formats = ["%Y-%m-%d", "%B %d, %Y", "%b %d, %Y", "%d %B %Y", "%m/%d/%Y"] + for fmt in formats: + try: + return datetime.strptime(date_str, fmt).strftime("%Y-%m-%d") + except ValueError: + continue + return date_str + + +def _classify_financial_item(item: FinancialItem) -> Literal["operating_rates", "contract_terms", "insurance", "other"]: + """Classify financial item by category for report grouping.""" + desc_lower = item.description.lower() + + if any(term in desc_lower for term in ["parking", "daily", "monthly", "rate", "airline commuter", "employee parking"]): + return "operating_rates" + if any(term in desc_lower for term in ["management fee", "security", "payment", "gross revenue", "interest"]): + return "contract_terms" + if any(term in desc_lower for term in ["insurance", "coverage"]): + return "insurance" + return "other" +``` + +**Add to service.py:** + +```python +def generate_markdown_report(analysis: DocumentAnalysis, title: str | None = None) -> str: + """Generate formatted markdown report from DocumentAnalysis. + + Creates structured report with: + - Executive Overview (procurement org, scope, contract term, key contact) + - Critical Deadlines table (Date/Time/Event/Notes with critical dates bolded) + - Financial Structure sections (Operating Rates, Contract Terms, Insurance) + - Key Entities grouped by type (Organizations, Contacts, Departments) + + Example: + >>> result = await analyzer.aforward([File(path="rfp.pdf")]) + >>> report = generate_markdown_report(result, title="YYJ RFP Analysis") + >>> with open("report.md", "w") as f: + ... f.write(report) + """ + lines = [] + report_title = title or f"Analysis Report: {analysis.document_type}" + + # Header + lines.append(f"# {report_title}") + lines.append("") + + # Executive Overview + lines.append("## Executive Overview") + lines.append("") + + orgs = [e for e in analysis.entities if "organization" in e.type.lower()][:3] + people = [e for e in analysis.entities if "person" in e.type.lower()][:2] + + if orgs: + lines.append(f"**Procurement:** {orgs[0].name} seeking contractor for relevant services") + if analysis.summary: + lines.append(f"**Scope:** {analysis.summary[:150]}...") + + if analysis.key_dates: + start_dates = [d for d in analysis.key_dates if any(t in d.name.lower() for t in ["start", "effective", "commencement"])] + end_dates = [d for d in analysis.key_dates if any(t in d.name.lower() for t in ["end", "expiration"])] + if start_dates and end_dates: + lines.append(f"**Term:** {start_dates[0].date} – {end_dates[0].date}") + + if people: + lines.append(f"**Key Contact:** {people[0].name}") + + lines.append("") + + # Critical Deadlines Table + if analysis.key_dates: + lines.append("## Critical Deadlines & Mandatory Actions") + lines.append("") + + sorted_dates = sorted(analysis.key_dates, key=lambda d: _format_date_for_sorting(d.date)) + + lines.append("| Date | Time | Event | Notes |") + lines.append("|---|---:|---|---|") + + for date in sorted_dates: + time_str = date.time or "—" + notes = date.context[:80] + "..." if date.context and len(date.context) > 80 else (date.context or "") + is_critical = any(term in date.name.lower() for term in ["deadline", "mandatory", "due"]) + date_display = f"**{date.date}**" if is_critical else date.date + event_display = f"**{date.name}**" if is_critical else date.name + lines.append(f"| {date_display} | {time_str} | {event_display} | {notes} |") + + lines.append("") + + # Financial Structure + if analysis.financial_info: + lines.append("## Financial Structure & Requirements") + lines.append("") + + # Group by category + operating_rates = [f for f in analysis.financial_info if _classify_financial_item(f) == "operating_rates"] + contract_terms = [f for f in analysis.financial_info if _classify_financial_item(f) == "contract_terms"] + insurance = [f for f in analysis.financial_info if _classify_financial_item(f) == "insurance"] + + if operating_rates: + lines.append("### Operating Rates") + lines.append("| Service | Rate | Notes |") + lines.append("|---|---|---|") + for item in operating_rates[:15]: + amount = item.amount or "N/A" + context = item.context[:60] + "..." if item.context and len(item.context) > 60 else (item.context or "") + lines.append(f"| {item.description} | {amount} | {context} |") + lines.append("") + + if contract_terms: + lines.append("### Contract Financial Terms") + for item in contract_terms: + amount = f" **{item.amount}** " if item.amount else "" + lines.append(f"> **{item.description}:**{amount}{item.context or ''}") + lines.append("") + + if insurance: + lines.append("### Insurance & Coverage Requirements") + for item in insurance: + amount = f" ({item.amount})" if item.amount else "" + lines.append(f"- **{item.description}**{amount}: {item.context or 'Required'}") + lines.append("") + + # Key Entities + if analysis.entities: + lines.append("## Key Entities & Contacts") + lines.append("") + + orgs = [e for e in analysis.entities if "organization" in e.type.lower()] + people = [e for e in analysis.entities if "person" in e.type.lower()] + + if orgs: + lines.append("### Organizations") + for e in orgs[:10]: + lines.append(f"- **{e.name}**") + lines.append("") + + if people: + lines.append("### Contacts") + for e in people[:10]: + lines.append(f"- **{e.name}** ({e.type})") + lines.append("") + + # Footer + lines.append("---") + lines.append("") + lines.append(f"*Document Type: {analysis.document_type} | Pages: {analysis.page_count} | Confidence: {analysis.confidence}*") + + return "\n".join(lines) +``` + +**Usage:** + +```python +from document_analyzer import DocumentAnalyzer, generate_markdown_report +from predict_rlm import File + +analyzer = DocumentAnalyzer(max_iterations=20) +result = await analyzer.aforward([File(path="rfp.pdf")]) + +# Generate polished report +report = generate_markdown_report(result, title="RFP Analysis Report") +with open("report.md", "w") as f: + f.write(report) +``` + +--- + +## Report Generation from Structured Output + +RLMs excel at extracting structured data into Pydantic models, but often you need to present that data as a polished markdown report. Add a `generate_markdown_report()` function to transform raw extraction output into formatted output. + +### Date Sorting and Classification Helpers + +```python +from datetime import datetime +from typing import Literal + + +def _format_date_for_sorting(date_str: str) -> str: + """Convert various date formats to YYYY-MM-DD for sorting.""" + if len(date_str) == 10 and date_str[4] == '-' and date_str[7] == '-': + return date_str + + formats = [ + "%Y-%m-%d", "%B %d, %Y", "%b %d, %Y", "%d %B %Y", "%d %b %Y", + "%m/%d/%Y", "%d/%m/%Y" + ] + for fmt in formats: + try: + return datetime.strptime(date_str, fmt).strftime("%Y-%m-%d") + except ValueError: + continue + return date_str + + +def _classify_financial_item(item: FinancialItem) -> Literal["operating_rates", "contract_terms", "insurance", "other"]: + """Classify financial item by category for report grouping.""" + desc_lower = item.description.lower() + + if any(term in desc_lower for term in ["parking", "daily", "monthly", "rate", "fee", "price"]): + return "operating_rates" + if any(term in desc_lower for term in ["management fee", "security", "payment", "revenue", "interest"]): + return "contract_terms" + if any(term in desc_lower for term in ["insurance", "coverage", "liability"]): + return "insurance" + return "other" + + +def _classify_entity(entity: Entity) -> Literal["organization", "person", "department", "role", "location", "other"]: + """Classify entity type, normalizing the type field.""" + type_lower = entity.type.lower() + name_lower = entity.name.lower() + + if "organization" in type_lower or any(t in name_lower for t in ["authority", "inc", "llc", "ltd", "company"]): + return "organization" + elif "person" in type_lower: + return "person" + elif "department" in type_lower: + return "department" + elif "role" in type_lower or any(t in name_lower for t in ["officer", "manager", "director", "ceo", "coordinator"]): + return "role" + elif "location" in type_lower or any(t in name_lower for t in ["airport", "terminal", "building"]): + return "location" + return "other" +``` + +### Report Generator Function + +```python +def generate_markdown_report(analysis: DocumentAnalysis, title: str | None = None) -> str: + """Generate formatted markdown report from DocumentAnalysis. + + Creates structured report with: + - Executive Overview (procurement org, scope, contract term, key contact) + - Critical Deadlines table (Date/Time/Event/Notes with critical dates bolded) + - Financial Structure sections (Operating Rates, Contract Terms, Insurance) + - Key Entities grouped by type (Organizations, Contacts, Departments) + + Args: + analysis: The DocumentAnalysis result from the RLM + title: Optional custom title for the report + + Returns: + Formatted markdown string ready to save or display + + Example: + >>> result = await analyzer.aforward([File(path="rfp.pdf")]) + >>> report = generate_markdown_report(result, title="YYJ RFP Analysis") + >>> with open("report.md", "w") as f: + ... f.write(report) + """ + lines = [] + report_title = title or f"Analysis Report: {analysis.document_type}" + + # Header + lines.append(f"# {report_title}") + lines.append("") + + # Executive Overview + lines.append("## Executive Overview") + lines.append("") + + # Build from entities + orgs = [e for e in analysis.entities if _classify_entity(e) == "organization"][:3] + people = [e for e in analysis.entities if _classify_entity(e) == "person"][:2] + + if orgs: + lines.append(f"**Procurement:** {orgs[0].name} seeking contractor for relevant services") + if analysis.summary: + lines.append(f"**Scope:** {analysis.summary[:150]}...") + + # Contract term from dates + start_dates = [d for d in analysis.key_dates if any(t in d.name.lower() for t in ["start", "effective", "commencement"])] + end_dates = [d for d in analysis.key_dates if any(t in d.name.lower() for t in ["end", "expiration"])] + if start_dates and end_dates: + lines.append(f"**Term:** {start_dates[0].date} – {end_dates[0].date}") + + if people: + contact = f"**Key Contact:** {people[0].name}" + if people[0].context: + contact += f" — {people[0].context[:100]}" + lines.append(contact) + + lines.append("") + + # Critical Deadlines Table + if analysis.key_dates: + lines.append("## Critical Deadlines & Mandatory Actions") + lines.append("") + + sorted_dates = sorted(analysis.key_dates, key=lambda d: _format_date_for_sorting(d.date)) + + lines.append("| Date | Time | Event | Notes |") + lines.append("|---|---:|---|---|") + + for date in sorted_dates: + time_str = date.time or "—" + notes = date.context[:80] + "..." if date.context and len(date.context) > 80 else (date.context or "") + is_critical = any(term in date.name.lower() for term in ["deadline", "mandatory", "due", "submission"]) + date_display = f"**{date.date}**" if is_critical else date.date + event_display = f"**{date.name}**" if is_critical else date.name + lines.append(f"| {date_display} | {time_str} | {event_display} | {notes} |") + + lines.append("") + + # Financial Structure + if analysis.financial_info: + lines.append("## Financial Structure & Requirements") + lines.append("") + + operating_rates = [f for f in analysis.financial_info if _classify_financial_item(f) == "operating_rates"] + contract_terms = [f for f in analysis.financial_info if _classify_financial_item(f) == "contract_terms"] + insurance = [f for f in analysis.financial_info if _classify_financial_item(f) == "insurance"] + + if operating_rates: + lines.append("### Operating Rates") + lines.append("| Service | Rate | Notes |") + lines.append("|---|---|---|") + for item in operating_rates[:15]: + amount = item.amount or "N/A" + context = item.context[:60] + "..." if item.context and len(item.context) > 60 else (item.context or "") + lines.append(f"| {item.description} | {amount} | {context} |") + lines.append("") + + if contract_terms: + lines.append("### Contract Financial Terms") + for item in contract_terms: + amount = f" **{item.amount}** " if item.amount else "" + lines.append(f"> **{item.description}:**{amount}{item.context or ''}") + lines.append("") + + if insurance: + lines.append("### Insurance & Coverage Requirements") + for item in insurance: + amount = f" ({item.amount})" if item.amount else "" + lines.append(f"- **{item.description}**{amount}: {item.context or 'Required'}") + lines.append("") + + # Key Entities + if analysis.entities: + lines.append("## Key Entities & Contacts") + lines.append("") + + orgs = [e for e in analysis.entities if _classify_entity(e) == "organization"] + people = [e for e in analysis.entities if _classify_entity(e) == "person"] + depts = [e for e in analysis.entities if _classify_entity(e) == "department"] + + if orgs: + lines.append("### Organizations") + for e in orgs[:10]: + ctx = f" — {e.context[:100]}" if e.context else "" + lines.append(f"- **{e.name}**{ctx}") + lines.append("") + + if people: + lines.append("### Contacts") + for e in people[:10]: + ctx = f" — {e.context[:100]}" if e.context else "" + lines.append(f"- **{e.name}** ({e.type}){ctx}") + lines.append("") + + if depts: + lines.append("### Departments") + for e in depts[:8]: + lines.append(f"- {e.name}") + lines.append("") + + # Footer + lines.append("---") + lines.append("") + lines.append(f"*Generated by DocumentAnalyzer RLM*") + lines.append(f"*Document Type: {analysis.document_type} | Pages: {analysis.page_count} | Confidence: {analysis.confidence}*") + + return "\n".join(lines) +``` + +### Usage Pattern + +```python +from document_analyzer import DocumentAnalyzer, generate_markdown_report +from predict_rlm import File + +# Initialize analyzer (uses Hermes config automatically) +analyzer = DocumentAnalyzer(max_iterations=30, verbose=True) + +# Run analysis +result = await analyzer.aforward([File(path="rfp.pdf")]) + +# Generate formatted report +report = generate_markdown_report(result, title="RFP Analysis Report") + +# Save to file +with open("report.md", "w") as f: + f.write(report) + +print(f"Extracted {len(result.key_dates)} dates, {len(result.entities)} entities") +print(f"Report saved: {len(report)} characters") +``` + +--- + +## Architecture Reference + +### How an RLM Works + +Two-level architecture: + +1. **Outer LLM** (the RLM itself) writes and executes Python in a sandboxed Pyodide/WASM REPL. It plans, orchestrates, and iterates. + +2. **Sub-LM** (via `predict()`) handles perception and extraction — analyzing images, understanding text, returning typed results. Each `predict()` call gets its own context window. + +The outer LLM's context stays small (code + tool results), while context-heavy work is offloaded to `predict()` calls. + +### File I/O with `File` + +```python +from predict_rlm import File + +# Input: mounts file from host into sandbox at /sandbox/input/{field_name}/ +doc = File(path="/absolute/path/to/file.pdf") + +# Output: declared as File output field, RLM writes to /sandbox/output/{field}/ +``` + +### PredictRLM Constructor + +```python +PredictRLM( + signature: type[Signature] | str, # DSPy signature class + lm: dspy.LM | str | None = None, # Main LM (code generation) + sub_lm: dspy.LM | str | None = None, # Sub-LM for predict() calls + max_iterations: int = 30, + max_llm_calls: int = 50, + verbose: bool = False, + tools: dict[str, Callable] | list[Callable] | None = None, + allowed_domains: list[str] | None = None, + skills: list[Skill] | None = None, + debug: bool = False, + output_dir: str | Path | None = None, +) +``` + +### Using Hermes's Default Model + +When building an RLM with this skill, **use Hermes's configured model by default** for seamless integration: + +```python +import os +import dspy +from predict_rlm import PredictRLM + +# Get Hermes's current model configuration +# Hermes sets these env vars when running: +LM_MODEL = os.getenv("HERMES_MODEL") # e.g., "accounts/fireworks/routers/kimi-k2p5-turbo" +SUB_LM_MODEL = os.getenv("HERMES_MODEL") # Same by default, or use cheaper variant + +# Configure DSPy to use the same provider as Hermes +if os.getenv("HERMES_PROVIDER") == "fireworks": + lm = dspy.LM( + model=LM_MODEL, + api_key=os.getenv("FIREWORKS_API_KEY"), + base_url="https://api.fireworks.ai/inference/v1", + ) +elif os.getenv("HERMES_PROVIDER") == "openrouter": + lm = dspy.LM( + model=LM_MODEL, + api_key=os.getenv("OPENROUTER_API_KEY"), + base_url="https://openrouter.ai/api/v1", + ) +elif os.getenv("HERMES_PROVIDER") == "anthropic": + lm = dspy.LM( + model=LM_MODEL, + api_key=os.getenv("ANTHROPIC_API_KEY"), + ) +else: + # Fallback to litellm-style model string + lm = dspy.LM(model=LM_MODEL) + +# Use same model for both LM and sub-LM, or configure sub_lm for cheaper/faster calls +sub_lm = lm # Or: dspy.LM(model="openai/gpt-4o-mini") for cheaper extraction + +# Initialize PredictRLM with Hermes's model +predictor = PredictRLM( + signature=MySignature, + lm=lm, # Main orchestrator LM (code generation) + sub_lm=sub_lm, # Sub-LM for predict() perception calls + skills=[pdf_skill], + max_iterations=30, +) +``` + +**Environment variables Hermes provides:** +- `HERMES_MODEL` — The active model identifier +- `HERMES_PROVIDER` — The provider (fireworks, openrouter, anthropic, etc.) +- Provider-specific API keys (already in your `.env`) + +**Recommended defaults for service.py:** +```python +class DocumentAnalyzer(dspy.Module): + def __init__( + self, + lm: dspy.LM | None = None, # Defaults to Hermes's model + sub_lm: dspy.LM | None = None, # Defaults to same as lm + max_iterations: int = 30, + verbose: bool = False, + debug: bool = False, + ): + # Auto-configure from Hermes environment if not provided + if lm is None: + lm = self._get_hermes_lm() + if sub_lm is None: + sub_lm = lm + + self.lm = lm + self.sub_lm = sub_lm + self.max_iterations = max_iterations + self.verbose = verbose + self.debug = debug + + def _get_hermes_lm(self) -> dspy.LM: + """Configure DSPy LM from Hermes environment.""" + import os + + model = os.getenv("HERMES_MODEL", "openai/gpt-4o") + provider = os.getenv("HERMES_PROVIDER", "openrouter") + + provider_configs = { + "fireworks": { + "api_key": os.getenv("FIREWORKS_API_KEY"), + "base_url": "https://api.fireworks.ai/inference/v1", + }, + "openrouter": { + "api_key": os.getenv("OPENROUTER_API_KEY"), + "base_url": "https://openrouter.ai/api/v1", + }, + "anthropic": { + "api_key": os.getenv("ANTHROPIC_API_KEY"), + }, + "openai": { + "api_key": os.getenv("OPENAI_API_KEY"), + }, + } + + config = provider_configs.get(provider, {}) + return dspy.LM(model=model, **config) +``` + +### Built-in Skills + +```python +from predict_rlm.skills import pdf as pdf_skill # pymupdf +from predict_rlm.skills import spreadsheet as ss_skill # openpyxl, pandas +from predict_rlm.skills import docx as docx_skill # python-docx +``` + +| Skill | Packages | What it teaches the RLM | +|-------|----------|------------------------| +| **pdf** | `pymupdf` | Read, render, modify, redact PDFs | +| **spreadsheet** | `openpyxl`, `pandas`, `formulas` | Build/modify Excel with formulas | +| **docx** | `python-docx` | Read/write Word documents | + +### Host-side Tools + +Use for operations that cannot run in WASM — host access, authenticated APIs, database queries, system resources. + +```python +async def fetch_exchange_rate(currency: str, date: str) -> str: + """Fetch the exchange rate for a currency on a given date. + + Args: + currency: ISO currency code (e.g. "EUR", "GBP") + date: Date in YYYY-MM-DD format + + Returns: + JSON string with the exchange rate data + """ + async with httpx.AsyncClient() as client: + resp = await client.get(f"https://api.example.com/rates/{currency}/{date}") + return resp.text +``` + +Pass to PredictRLM: `tools={"fetch_exchange_rate": fetch_fn}` or bundle in a Skill. + +### predict() Tool (Inside Sandbox) + +RLM calls `predict()` for sub-LM perception/extraction: + +```python +result = await predict( + "image: dspy.Image -> items: list[Item]", + instructions="Extract all line items from this invoice page", + image=page_image, +) +``` + +Each call gets its own context window. Supports `dspy.Image` for multimodal. + +### Key Imports + +```python +from predict_rlm import PredictRLM, Skill, File +from predict_rlm.skills import pdf, spreadsheet, docx +``` + +### WASM Sandbox Constraints + +- Only pure-Python wheels or Pyodide built-in packages work +- No subprocess, no native binaries, no C extensions (unless Emscripten-built) +- Network access requires `allowed_domains` whitelist +- File I/O within sandbox filesystem +- Host-side tools bridge the gap for WASM-incompatible operations