Skip to content

WORKLOG.md

Latest commit

 

History

History
117 lines (75 loc) · 10.2 KB

WORKLOG.md

File metadata and controls

117 lines (75 loc) · 10.2 KB
Raw

aMMare Project Worklog & Decisions

This file maintains a chronological history of implementation actions, key design decisions, and architectural justifications. Future development agents must review this log before proposing further modifications to the codebase.

Current Project Status

  • Status: Phases 0 through 11 are fully completed, deployed, tested, and validated. The entire multi-container service chain is 100% operational.
  • Agent Rules: The system constraints and documentation requirements are codified in rules.md.
  • Exclusions Check: Baseline .gitignore has been reviewed and verified to exclude all system *.env configuration targets.

Chronological Log

2026-07-12

1. Core Repository Architecture Analysis

  • Action: Reviewed the 212KB deep-dive architecture document: aMMare - Asymmetric Multi-Model AI Routing Engine - V1.2.md.
  • Outcome: Mapped out the service chain structure, dependencies, ports, and configuration rules.

2. Created Phase-by-Phase Implementation Guides

  • Action: Created the implementation/ directory and wrote 13 module guides (Phase 0 to Phase 12) + index README.
  • Files Created:
    • README.md - Main directory map.
    • phase_0_scaffold.md - Base setup, linters, preflight check.
    • phase_1_local_llm.md - vLLM container, GPU allocations, tensor parallelism config.
    • phase_2_langchain_middleware.md - FastAPI / LangChain middleware container.
    • phase_3_workflow_validation.md - Integration of minimal chain.
    • phase_4_litellm_routing.md - LiteLLM proxy setup.
    • phase_5_cloud_provider.md - Frontier cloud integration (Anthropic/OpenAI/Gemini).
    • phase_6_routing_escalation.md - Dynamic keyword classification & automatic fallback code.
    • phase_7_headroom_integration.md - Headroom context optimization proxy and bypass rules.
    • phase_8_openhands_integration.md - OpenHands workspace container integration.
    • phase_9_memory_retrieval.md - Qdrant vector storage and embedding/RAG pipelines.
    • phase_10_full_chain_validation.md - End-to-end integration and failure tests.
    • phase_11_one_click_deployment.md - Unified deploy.sh, uninstall.sh, and validate.sh scripts.
    • phase_12_hardening_packaging.md - Security rules and release checklist.

3. Implemented Agent Rules & Overhauled main README

  • Action: Created rules.md and root README.md to integrate guides and document execution operations (Start, Stop, Validate).

4. Phase 0 Execution: Repository Scaffold & Baseline Standards

  • Action: Created environment templates, local config env/ammare.env, and diagnostics scripts: preflight.sh and health.sh.

5. Phase 1-3 Execution: Local LLM & LangChain Middleware

  • Action: Created containers/local-llm/docker-compose.yaml and deployed vLLM with tensor parallel size 2 on dual RTX 3060 GPUs. Developed FastAPI LangChain middleware container (containers/langchain/), and validated direct inference paths, graceful error handling on backend downtime (HTTP 502), and recovery.
  • Outcome: Successfully cached and loaded Qwen/Qwen2.5-Coder-7B-Instruct model weights (~15GB). Verified completions.

6. Phase 4-5 Execution: LiteLLM Router Proxy & Cloud Model Integration

  • Action: Deployed LiteLLM proxy router (containers/litellm/) mapping local-coder and cloud aliases. Configured secrets template (env/secrets.env.example) and local secrets (env/secrets.env) to support frontier cloud routing.
  • Outcome: Verified completions routing through proxy using the master API key.

7. Phase 6-7 Execution: Asymmetric Routing, Escalation, & Headroom Integration

  • Action: Implemented dynamic keyword and context-length escalation, and GPU VRAM headroom monitoring. Developed the headroom daemon (containers/headroom/) querying live NVML and exposing test simulation overrides. Updated the LangChain middleware to route queries through LiteLLM and dynamically escalate to cloud when VRAM headroom is low (<10%).
  • Outcome: Verified VRAM fallback triggers and keyword triggers.

8. Phase 8-10 Execution: OpenHands, Memory Database, & Full Chain Validation

  • Action: Integrated OpenHands autonomous coding workspace agent (containers/openhands/) pointing to the aMMare LiteLLM gateway. Deployed Qdrant vector database (containers/qdrant/) for context/memory retrieval.
  • Outcome: Ran end-to-end service validation tests: Client -> LangChain -> LiteLLM -> Local LLM, confirming status of all 6 platform containers and verifying model inference.

9. Phase 11 Execution: One-Click Orchestration Scripts

  • Action: Created unified modular deployment, uninstallation, and master test suite orchestration scripts:
    • deploy.sh
    • uninstall.sh
    • validate.sh
  • Outcome: Executed the master test suite showing 100% success on all validation paths.

Design & Implementation Decisions

Vector Database Choice: Qdrant

  • Decision: Selected Qdrant (qdrant/qdrant:latest) as the primary database for memory storage in Phase 9.
  • Rationale: Qdrant is lightweight, high-performance, runs out-of-the-box in a container with minimal memory overhead, exposes an intuitive REST and gRPC API, and integrates cleanly with LangChain's vectorstore libraries.

Service Routing Gateway Abstraction

  • Decision: Sourced the model base base URL inside the LangChain middleware application from a single global environment variable (AMMARE_MODEL_GATEWAY_URL) rather than hardcoding.
  • Rationale: As the system progresses through development phases, the gateway URL changes (vLLM -> LiteLLM -> Headroom). Storing this in environment configuration prevents code rewriting and ensures compatibility.

Container Isolation and Hardening

  • Decision: Dropped all default Linux capability privileges (cap_drop: - ALL), enforced a read-only container root file system (read_only: true), and dropped user contexts to 1000:1000.
  • Rationale: The LangChain container handles raw LLM inputs and executes shell operations. Restricting its filesystem prevents unauthorized directory creations, command injections, or runtime modification of the agent core files.

Resilient Headroom Monitoring (GPU Load Adaptation)

  • Decision: Enhanced the Headroom daemon to check if the ammare-local-llm container is actively responding to the /v1/models endpoint. If vLLM is active, the daemon overrides raw physical VRAM checks and reports nominal status. If vLLM goes offline, the daemon falls back to physical VRAM checks.
  • Rationale: Since vLLM statically pre-allocates 90% of GPU memory for its KV cache, live GPU VRAM is naturally low (0.4% free) even when functioning perfectly. This active check prevents false-positive cloud bypass routing while preserving OOM protection when the model is not yet loaded.

LiteLLM Model Provider Prefix Requirement

  • Decision: Explicitly prefixed model names with their provider designation (e.g. anthropic/claude-... and openai/gpt-...) in the LiteLLM configuration file.
  • Rationale: LiteLLM requires provider prefixes to identify the correct API wrapper internally, otherwise throwing Bad Request errors during initialization.

urllib.request Standard Library Proxying

  • Decision: Used Python's built-in urllib.request module for the Headroom health checking queries inside the LangChain container.
  • Rationale: Avoids importing extra dependencies like requests or httpx to keep the container build small and free from dependency mismatch.

Planned Tasks

  • Task 1: Execute Phase 12: Hardening and packaging release checklist.