Skip to content

LLM extraction: model-specific prompt templates + Qwen3 support #59

@JordanCoin

Description

@JordanCoin

Problem

Qwen3 8B produces excellent entities when given a simple prompt, but fails with our full extraction prompt. The prompt includes:

  • 13 entity types (including obscure custom ones like SHELL_COMPANY_NAMES)
  • 9 relationship types with Unicode arrows
  • Detailed formatting instructions

Smaller models get confused by the complexity and return errors like "Please provide the full text."

What works

  • Simple prompt: "Extract entities, return JSON with entities array" → perfect results
  • Qwen3 8B correctly identifies Kirkland & Ellis, Clearview AI, Charlie Beck with high confidence
  • The model normalizes "KIRKLAND 8. ELLIS LLP" → "Kirkland & Ellis LLP" (corrects OCR errors!)

Fix

  1. Tiered prompts: Simple prompt for 3B-8B models, detailed prompt for 14B+
  2. Strip custom types from prompt unless they have real patterns (most are placeholder junk)
  3. Test with Qwen3 8B, llama3.1:8b, llama3.2:3b to ensure all work
  4. Qwen3.5 (thinking models): Skip format: json constraint, strip <think> tags from response

Current state

  • llama3.2:3b works (fast, lower quality)
  • llama3.1:8b works (slow, higher quality)
  • qwen3:8b works with simple prompts only
  • qwen3.5:9b doesn't work (thinking model + JSON constraint incompatible)
  • --model flag added to CLI
  • Parser accepts "text"/"name"/"raw_text" keys from any model

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions