Skip to content

Phase 7.3: LLMService facade (#644)#657

Merged
ShaerWare merged 1 commit intomainfrom
server/644-llm-service-facade
Mar 29, 2026
Merged

Phase 7.3: LLMService facade (#644)#657
ShaerWare merged 1 commit intomainfrom
server/644-llm-service-facade

Conversation

@ShaerWare
Copy link
Copy Markdown
Owner

Summary

  • New LLMServiceImpl in modules/llm/facade.py — unified facade implementing the LLMService Protocol from modules/llm/protocols.py
  • Wraps CloudLLMService / VLLMLLMService (generation) and CloudProviderService (provider CRUD) behind a single interface
  • Lazy access to container.llm_service — handles runtime backend switches by InternetMonitor
  • generate() — sync provider call wrapped via asyncio.to_thread
  • stream() — sync generator consumed in thread, yields normalized StreamChunk TypedDicts
  • _to_stream_chunk() normalizes both plain str and dict (tool_calls) from providers
  • _to_provider_info() converts provider dicts to ProviderInfo TypedDict
  • Registered in ServiceContainer.llm_service_facade during orchestrator startup
  • Module-level singleton llm_service_facade set during startup for direct imports

Scope note: This PR creates the facade layer only. Prompt assembly, RAG injection, agentic tool loops, and token accounting remain in callers (chat router, widget router). Migration of callers to use the facade is a separate step.

Closes #644

Files changed

File Change
modules/llm/facade.py NewLLMServiceImpl + converters
orchestrator.py Create facade during startup
app/dependencies.py Add llm_service_facade field to ServiceContainer
CLAUDE.md Document Phase 7.3 completion
tests/unit/test_llm_service.py New — 13 tests (protocol, signatures, converters)

Test plan

  • 13 new unit tests pass (protocol compliance, signatures, converters)
  • All 183 existing unit tests pass
  • Ruff lint + format clean
  • CI green

🤖 Generated with Claude Code

Phase 7.3: Create LLMServiceImpl that wraps CloudLLMService/VLLMLLMService
(generation) and CloudProviderService (provider CRUD) behind a single
Protocol-conformant facade.

- modules/llm/facade.py: LLMServiceImpl + dict→TypedDict converters
- Lazy llm_service access via ServiceContainer (handles backend switches)
- generate() wraps sync provider call via asyncio.to_thread
- stream() wraps sync generator, normalizes to StreamChunk TypedDict
- resolve_backend/list_providers delegate to CloudProviderService
- Registered in ServiceContainer.llm_service_facade during startup
- 13 unit tests (protocol compliance, signatures, converters)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ShaerWare ShaerWare merged commit 372f4ec into main Mar 29, 2026
3 checks passed
@ShaerWare ShaerWare deleted the server/644-llm-service-facade branch March 29, 2026 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Phase 7.3: LLMService facade — извлечь generate/stream из оркестратора

2 participants