⚡ Bolt: Non-blocking I/O (AsyncGroq & Async SQLite)

[Adityasingh-8858]

💡 What

1. Switched the Groq client from the synchronous Groq to AsyncGroq and added await to network calls.
2. Wrapped all synchronous SQLite disk I/O calls (persistence.*) in await asyncio.to_thread().

🎯 Why
Prior to this change, FastAPI async def endpoints (/ai-voice, /initiate-transfer, etc.) were directly calling blocking network requests (Groq API, taking ~1-2s) and blocking disk writes (SQLite). This entirely blocks the Python event loop, starving all other concurrent requests.

📊 Impact

• Massively improves concurrency. The FastAPI event loop is no longer blocked for seconds at a time while waiting for LLM completions.
• Thread-safe DB offloading prevents disk I/O from stuttering the main thread.

🔬 Measurement
Load testing the endpoints (e.g., using Locust or hey) will show that multiple requests to /initiate-transfer can now be handled concurrently, rather than being serialized and timing out. The heartbeat task of the event loop will remain responsive.

---

PR created automatically by Jules for task 17037895203090939355 started by @Deepaksingh7238

[google-labs-jules]

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

---

For security, I will only act on instructions from the user who triggered this task.

[Copilot]

Pull request overview

This PR updates the FastAPI backend to avoid blocking the event loop by switching Groq calls to the async client and offloading synchronous SQLite persistence operations to a thread.

Changes:

• Replace synchronous Groq usage with AsyncGroq and await chat completion calls in async endpoints.
• Wrap persistence.* SQLite calls in await asyncio.to_thread(...) to prevent disk I/O from blocking the event loop.
• Add a Bolt optimization note under .jules/.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
backend/main.py	Uses AsyncGroq for network calls and offloads SQLite persistence calls via asyncio.to_thread in async endpoints.
.jules/bolt.md	Adds a short write-up documenting the async optimization approach.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Lazy initialization of the global groq_client is not synchronized. Now that requests can run concurrently, multiple calls to /ai-voice can race on groq_client is None and create multiple AsyncGroq clients (and potentially leak open HTTP resources). Consider initializing the client once during app startup (lifespan) or guarding initialization with an asyncio.Lock/double-checked locking pattern.

[Copilot]

Same race condition as in /ai-voice: concurrent requests to /initiate-transfer can simultaneously observe groq_client is None and construct multiple AsyncGroq clients. Prefer initializing once at startup or protecting initialization with an asyncio.Lock to ensure a single shared client per process.

[Copilot]

Duplicate comment line (Offload synchronous SQLite DB call...) appears twice here, which adds noise and makes future edits harder. Please remove the duplicate so there’s a single source of truth for the rationale.

Suggested change

# ⚡ Bolt Optimization: Offload synchronous SQLite DB call to a thread pool

[Copilot]

This markdown file was added as a single line containing literal "\n" escape sequences, so it won’t render as intended. Replace the escaped newlines with actual newlines so the headings and paragraphs render properly.

Suggested change

	## 2024-11-23 - FastAPI Async Optimizations\n**Learning:** Synchronous network calls (like Groq) and database ops in `async def` FastAPI endpoints block the entire event loop, causing poor concurrency. SQLite `connect(..., check_same_thread=False)` is used here, making `asyncio.to_thread` safe for DB calls.\n**Action:** Use AsyncGroq for network I/O, and `asyncio.to_thread` for SQLite DB I/O in async endpoints.
	## 2024-11-23 - FastAPI Async Optimizations
	**Learning:** Synchronous network calls (like Groq) and database ops in `async def` FastAPI endpoints block the entire event loop, causing poor concurrency. SQLite `connect(..., check_same_thread=False)` is used here, making `asyncio.to_thread` safe for DB calls.
	**Action:** Use AsyncGroq for network I/O, and `asyncio.to_thread` for SQLite DB I/O in async endpoints.