Skip to content

Fix async UDF event loop starvation in Jupyter#123

Open
kesmit13 wants to merge 12 commits into
mainfrom
fix/async-udf-event-loop-starvation
Open

Fix async UDF event loop starvation in Jupyter#123
kesmit13 wants to merge 12 commits into
mainfrom
fix/async-udf-event-loop-starvation

Conversation

@kesmit13
Copy link
Copy Markdown
Collaborator

@kesmit13 kesmit13 commented May 14, 2026

Summary

  • Async UDFs running under uvicorn in Jupyter notebooks become unresponsive under heavy load due to event loop starvation
  • Root cause: async UDFs ran directly in uvicorn's event loop via asyncio.create_task, competing with HTTP connection handling
  • Fix: dedicated event loop in a background thread for async UDF execution, submitted via run_coroutine_threadsafe() and awaited from the server loop
  • Sync UDFs are unchanged (already offloaded to threads)

Test plan

  • Existing test_ext_func.py tests pass
  • Manual: launch uvicorn from Jupyter, fire 50+ concurrent async UDF calls, confirm responsiveness
  • Verify cancellation: disconnect mid-execution, confirm UDF stops
  • Verify sync UDFs still work identically (regression)

🤖 Generated with Claude Code


Note

Medium Risk
Concurrency changes in the ASGI request path move async UDF execution onto a background event loop and add new cancellation/shutdown behavior, which could affect request handling and resource cleanup under load. CI/release workflow updates also change how tags/releases are created and assets uploaded.

Overview
Fixes async Python UDF unresponsiveness under load by running async endpoints on a dedicated asyncio event loop in a background thread (submitted via run_coroutine_threadsafe), instead of scheduling them on uvicorn’s main loop.

Adds explicit cleanup: Application.shutdown() stops/joins the UDF loop thread, run_udf_app now tracks and shuts down the current Application on restart and ensures the app is shut down on startup failures, and request cancellation/timeout/disconnect now also cancels the underlying async UDF future.

CI/release automation is adjusted: Fusion docs generation now triggers on published releases and uploads assets using the release tag name, and the publish workflow can create prerelease GitHub releases for v*-rc/test/alpha/beta tag pushes, uploads build artifacts to releases, and tightens PyPI publishing gating to release or explicitly opted-in workflow_dispatch runs.

Reviewed by Cursor Bugbot for commit 35a6658. Bugbot is set up for automated code reviews on this repo. Configure here.

Async UDFs were running directly in uvicorn's event loop via
asyncio.create_task, competing with connection handling under heavy
concurrent load. This caused unresponsiveness when running from Jupyter
notebooks where the event loop is shared.

The fix introduces a dedicated event loop in a background thread for
async UDF execution. Coroutines are submitted via
run_coroutine_threadsafe() and awaited from the server loop, isolating
UDF work from HTTP I/O while preserving cooperative async scheduling
between UDFs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Cancel the concurrent.futures.Future in the UDF loop on
disconnect/timeout so the coroutine is interrupted promptly,
not just at the next cancel_on_event row check.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comment thread singlestoredb/functions/ext/asgi.py
…ture

asyncio.create_task() requires a coroutine but asyncio.wrap_future()
returns a Future. Use asyncio.ensure_future() which accepts both.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Newer pandas versions use StringDtype ('str') instead of 'object' for
string columns. Detect the actual dtype at import time and use it in
test assertions. Binary columns remain 'object'.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

@mgiannakopoulos mgiannakopoulos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢 Ship IT!

Comment thread singlestoredb/functions/ext/asgi.py Outdated
Prevents UDF event loop thread leaks when run_udf_app() is called
repeatedly in Jupyter notebooks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comment thread singlestoredb/functions/ext/asgi.py
Tags matching v*-rc*, v*-test*, v*-alpha*, v*-beta* now trigger the
full wheel build pipeline and create a pre-release GitHub Release with
all wheels attached. Production releases also attach wheels to the
existing release before publishing to PyPI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comment thread singlestoredb/apps/_python_udfs.py
- Cancel udf_future when func_task is in pending set after asyncio.wait
- Cancel udf_future in finally block to ensure cleanup on any exit path
- Wrap post-construction code in try/except to call app.shutdown() if
  validation, config, or registration fails after Application is created

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gh release create doesn't need a git repo when not generating notes.
Use --notes "" for empty release body with just assets attached.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comment thread singlestoredb/functions/ext/asgi.py
Comment thread singlestoredb/functions/ext/asgi.py Outdated
kesmit13 and others added 2 commits May 14, 2026 12:00
Changed from push tag trigger (v*.*.*) to release event so it only
runs on published releases, not rc/test/alpha/beta tags.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move udf_future initialization before input_handler['load']() to
  prevent NameError in finally block if parsing raises
- Lazily create UDF event loop on first async UDF invocation instead
  of unconditionally in __init__, avoiding wasted resources for
  sync-only or metadata-only usage
- Guard shutdown() against None loop/thread

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default mode and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b84d9ea. Configure here.

Comment thread singlestoredb/functions/ext/asgi.py
gh release create requires a git repo to determine the repository
context even without --generate-notes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After stopping the event loop and joining the thread, set both
_udf_loop and _udf_thread back to None so that _get_udf_loop()
can safely recreate them if called after shutdown.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
kesmit13 added a commit that referenced this pull request May 15, 2026
Enable asset generation on test/rc/alpha/beta tag pushes without
publishing to PyPI. Also prevents fusion-docs from triggering on
pre-release tags.

Ref: PR #123

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants