Optimize search for large pattern collections

[james-livefront]

Problem

Current search implementation uses linear scan through all patterns, which will become slow with large pattern collections (1000+ patterns).

Current Implementation

• In-memory cache rebuilt on every search
• Linear string matching through all patterns
• Scoring: name(10pts) + content(5pts) + description(3pts) + tags(2pts)
• No persistent indexing

When to Optimize

• 1000+ patterns - linear search becomes noticeable
• Heavy search usage - frequent searches vs occasional lookups
• Large pattern files - multi-page documents vs current small files

Optimization Options

Phase 1: Simple Inverted Index

• Map terms → pattern names in memory
• Still rebuild on pattern changes
• Keeps current scoring system
• ~10x faster for exact term matching

Phase 2: Persistent Index

• Save/load index files to disk
• Only rebuild when patterns change
• Add file modification time tracking

Phase 3: Advanced Features (if needed)

• Vector search for semantic similarity
• Fuzzy matching capabilities
• Content recommendations

Implementation Notes

• Maintain backward compatibility
• Keep simple string search as fallback
• Add performance benchmarking
• Consider configurable indexing (off/simple/full)

Priority

Low - optimize when users actually hit performance problems rather than premature optimization.