Implement Assignment 1 Live Translate backend (AI service + gateway)#62
Open
iamSarang wants to merge 4 commits into
Open
Implement Assignment 1 Live Translate backend (AI service + gateway)#62iamSarang wants to merge 4 commits into
iamSarang wants to merge 4 commits into
Conversation
Fills in the assignment's TODOs so the translation widget works end to end: - lib/llm.py: Mexican-Spanish system prompt and Anthropic call, lazy-init the client so it picks up ANTHROPIC_API_KEY after load_dotenv() runs - lib/cache.py: SQLite schema init, two-tier get/set (memory + disk) with hit stats - app.py: wire cache -> LLM -> cache flow with server-side latency timing - gateway-node/server.js: request logging middleware and the AI-service proxy call Also bumps anthropic 0.39.0 -> 0.69.0 in requirements.txt; the old pin crashed on import due to an httpx compatibility break. Verified end-to-end in a browser: cache miss (13 chunks, ~22s, real LLM calls) -> cache hit (13/13 cache hits, instant) on a live demo page through the Node gateway. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Gateway generates a UUID (or reuses an inbound X-Request-Id), returns it on the response, forwards it as a header to the AI service, and logs it in a structured JSON line. AI service reads the header (falling back to its own UUID for direct calls with no header) and includes it in its translate/translate_batch log lines, so one request is greppable by a single ID across both services' logs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- ai-service-python: Dockerfile (binds uvicorn to :: for IPv6/6PN reachability), fly.toml with a persistent 1GB volume for translations.db (survives restarts) and an internal-only [[services]] block (no public ports) so the AI service and its API key never face the public internet - gateway-node: Dockerfile, fly.toml (public http_service on :8787). Bundles its own copy of translation-widget.js (same pattern the extension/ folder already uses) since Fly's build context is scoped to the app directory and can't reach ../../widget/ - Secrets (ANTHROPIC_API_KEY, AI_SERVICE_URL) set via `fly secrets`, not committed Live: https://sarang-livetranslate-gw.fly.dev — verified end-to-end (translate, cache hit, request-ID tracing all confirmed against the deployed gateway); AI service confirmed unreachable from the public internet, only via Fly's private network. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- eval/report.json + eval/REPORT.md: automated rubric run, 70/70 auto score, fresh-cache SLA gate pass, deploy health check against the live Fly URL - benchmark/_bench.json: corroborating fresh-cache benchmark run - PRODUCT_EVAL.md (+ PDF): live-website test against a real, uncontrolled homedepot.com product page (212 chunks) via the deployed gateway — prices, model codes and SKUs preserved, natural es-MX, 98x cache speedup on re-translate (199,564ms -> 2,035ms, 99% hit rate). Also documents a real finding: a 653-chunk search-listing page timed out on a single giant batch call, a genuine scaling gap worth fixing before production. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
lib/llm.py(Mexican-Spanish prompt + Anthropic call),lib/cache.py(SQLite schema + two-tier get/set),app.py(cache → LLM → cache flow with latency timing), and the Node gateway's request-logging middleware + AI-service proxy call.anthropicfrom0.39.0→0.69.0inrequirements.txt— the old pin crashed on import due to an httpx compatibility break, unrelated to the assignment TODOs.lib/llm.py: the Anthropic client was built at module-import time, beforeapp.py'sload_dotenv()ran, soANTHROPIC_API_KEYwas never picked up.Test plan
/health,/stats,/translate,/translate/batchexercised via curl with a live Anthropic key/health,/translate(200 and 502 on AI-service failure),/translate/batch,/widget.js, request-logging output all verified🤖 Generated with Claude Code