Skip to content

feat: implement resilient timeout handling for model switcher component#955

Open
prasiddhi-105 wants to merge 1 commit into
imDarshanGK:mainfrom
prasiddhi-105:feature/model-switcher-timeout
Open

feat: implement resilient timeout handling for model switcher component#955
prasiddhi-105 wants to merge 1 commit into
imDarshanGK:mainfrom
prasiddhi-105:feature/model-switcher-timeout

Conversation

@prasiddhi-105

@prasiddhi-105 prasiddhi-105 commented Jul 6, 2026

Copy link
Copy Markdown

Description

This PR resolves the issue regarding model switching delays by implementing comprehensive timeout handling mechanisms across both the client-side and server-side components. This prevents the application from hanging indefinitely when loading large models or waiting on a slow local Ollama instance.

Changes Implemented

🔹 Backend (backend/routes/models.py)

  • Added a dedicated POST /api/models/{model_name}/switch endpoint.
  • Wrapped the model loading logic inside an explicit asyncio.timeout block bounded to 15.0 seconds.
  • Gracefully handles asyncio.TimeoutError and converts it into a 504 Gateway Timeout HTTP response.

🔹 Frontend (frontend/src/services/api.js)

  • Created switchModelWithTimeout using a client-side AbortController as a backup boundary.
  • Dispatches an explicit client-side fallback cancellation if the endpoint request takes longer than 15 seconds.

🔹 Testing (backend/tests/test_model_cache.py)

  • Added a robust test_switch_model_timeout async unit test case using unittest.mock.patch to simulate deep runtime hangs and verify the correct 504 error codes are thrown.

Verification Results

All local backend unit tests ran successfully within the execution container environment:

Closes #814

Screenshot 2026-07-07 050544

@vercel

vercel Bot commented Jul 6, 2026

Copy link
Copy Markdown

@prasiddhi-105 is attempting to deploy a commit to the Darshan's projects Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add timeout handling to model switcher

1 participant