Skip to content

fix(docker): exec gunicorn in run-server.sh so it receives SIGTERM#41173

Open
mjlshen wants to merge 1 commit into
apache:masterfrom
mjlshen:mjlshen/exec-run-server
Open

fix(docker): exec gunicorn in run-server.sh so it receives SIGTERM#41173
mjlshen wants to merge 1 commit into
apache:masterfrom
mjlshen:mjlshen/exec-run-server

Conversation

@mjlshen

@mjlshen mjlshen commented Jun 17, 2026

Copy link
Copy Markdown

SUMMARY

run-server.sh launches gunicorn without exec, so when the script is the container's entrypoint the shell stays PID 1 and gunicorn runs as its child. A container's PID 1 is only sent a signal if it has installed a handler for it; the shell installs none for SIGTERM, so the SIGTERM delivered on docker stop / Kubernetes pod termination never reaches gunicorn and gets SIGKILLed when the grace period expires, dropping in-flight requests on every restart and preventing graceful rollouts.

TESTING INSTRUCTIONS

You can make this change by overriding the default command in a Kubernetes cluster, or this minimally reproducible example that just demonstrates the issue with plain gunicorn where noexec.sh and exec.sh exist in the container with only exec as the difference:

mkdir exec-proof && cd exec-proof
cat > app.py <<'EOF'
def app(environ, start_response):
    start_response("200 OK", [("Content-Type", "text/plain")]); return [b"ok"]
EOF
printf '#!/usr/bin/env bash\ngunicorn --bind 0.0.0.0:8088 --workers 2 app:app\n'      > noexec.sh
printf '#!/usr/bin/env bash\nexec gunicorn --bind 0.0.0.0:8088 --workers 2 app:app\n' > exec.sh
cat > Dockerfile <<'EOF'
FROM python:3.12-slim
RUN pip install --no-cache-dir gunicorn
WORKDIR /work
COPY app.py noexec.sh exec.sh /work/
RUN chmod +x /work/*.sh
EOF
docker build -q -t exec-proof .

# Mirror the chart: sh -c that exec's the launch script, so the only
# difference between the two runs is `exec` inside the script.
for s in noexec exec; do
  docker run -d --name t-$s --entrypoint /bin/sh exec-proof -c "exec /work/$s.sh" >/dev/null
  sleep 4
  echo "[$s] PID1: $(docker exec t-$s cat /proc/1/cmdline | tr '\0' ' ')"
  t0=$(date +%s); docker stop -t 30 t-$s >/dev/null
  echo "[$s] stop $(( $(date +%s)-t0 ))s, exit $(docker inspect -f '{{.State.ExitCode}}' t-$s), graceful=$(docker logs t-$s 2>&1 | grep -c 'Handling signal: term')"
  docker rm -f t-$s >/dev/null
done

Results:

[noexec] PID1: bash /work/noexec.sh
[noexec] stop 30s, exit 137, graceful=0
[exec] PID1: /usr/local/bin/python3.12 /usr/local/bin/gunicorn --bind 0.0.0.0:8088 --workers 2 app:app
[exec] stop 0s, exit 0, graceful=1

ADDITIONAL INFORMATION

@dosubot dosubot Bot added infra:webserver Infra setup and configuration related to webserver install:docker Installation - docker container labels Jun 17, 2026
@bito-code-review

bito-code-review Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Code Review Agent Run #aef9e5

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: e0a731a..e0a731a
    • docker/entrypoints/run-server.sh
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@bito-code-review

bito-code-review Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Code Review Agent Run #99d29e

Actionable Suggestions - 0
Review Details
  • Files reviewed - 2 · Commit Range: d4b8160..d4b8160
    • docker/entrypoints/run-server.sh
    • helm/superset/values.yaml
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@mjlshen mjlshen force-pushed the mjlshen/exec-run-server branch from d4b8160 to 4edc157 Compare June 18, 2026 23:44
@rusackas

Copy link
Copy Markdown
Member

The exec fix is spot on — that's exactly the PID-1 signal-forwarding gotcha. Since you're touching helm/, don't we also need to bump version in helm/superset/Chart.yaml (and regen the README badge)? ct lint keys off a chart version increment, so I think it'll fail without it. @mjlshen

@mjlshen mjlshen force-pushed the mjlshen/exec-run-server branch from 4edc157 to 68239d5 Compare June 18, 2026 23:46
@netlify

netlify Bot commented Jun 18, 2026

Copy link
Copy Markdown

Deploy Preview for superset-docs-preview ready!

Name Link
🔨 Latest commit 4edc157
🔍 Latest deploy log https://app.netlify.com/projects/superset-docs-preview/deploys/6a3482f0d0431100083734ea
😎 Deploy Preview https://deploy-preview-41173--superset-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@netlify

netlify Bot commented Jun 18, 2026

Copy link
Copy Markdown

Deploy Preview for superset-docs-preview ready!

Name Link
🔨 Latest commit e5bc86b
🔍 Latest deploy log https://app.netlify.com/projects/superset-docs-preview/deploys/6a34be5ba2c22e0008b0aa67
😎 Deploy Preview https://deploy-preview-41173--superset-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@mjlshen

mjlshen commented Jun 18, 2026

Copy link
Copy Markdown
Author

The exec fix is spot on — that's exactly the PID-1 signal-forwarding gotcha. Since you're touching helm/, don't we also need to bump version in helm/superset/Chart.yaml (and regen the README badge)? ct lint keys off a chart version increment, so I think it'll fail without it. @mjlshen

Thanks for the review @rusackas ! Incremented the Chart version and regenerated the README :D

@bito-code-review

bito-code-review Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Code Review Agent Run #b151fd

Actionable Suggestions - 0
Additional Suggestions - 1
  • helm/superset/values.yaml - 1
    • Inconsistent exec pattern · Line 277-277
      This change adds `exec` to replace the shell with the gunicorn process, which is correct for proper signal propagation. However, the celery worker command (line 403) and celery beat command (line 491) in the same file do not use `exec`, creating an inconsistent pattern across related deployment commands.
Review Details
  • Files reviewed - 3 · Commit Range: 68239d5..68239d5
    • docker/entrypoints/run-server.sh
    • helm/superset/Chart.yaml
    • helm/superset/values.yaml
  • Files skipped - 1
    • helm/superset/README.md - Reason: Filter setting
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

run-server.sh launches gunicorn without exec, so when the script is the
container's entrypoint the shell stays PID 1 and gunicorn runs as its
child. A container's PID 1 is only sent a signal if it has installed a
handler for it; the shell installs none for SIGTERM, so the SIGTERM
delivered on `docker stop` / Kubernetes pod termination never reaches
gunicorn and gets SIGKILLed when the grace preiod expires, dropping
in-flight requests on every restart and preventing graceful rollouts.

Signed-off-by: Michael Shen <mishen@umich.edu>
@mjlshen mjlshen force-pushed the mjlshen/exec-run-server branch from 68239d5 to e5bc86b Compare June 19, 2026 03:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

infra:webserver Infra setup and configuration related to webserver install:docker Installation - docker container size/XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

run-server.sh launches gunicorn without exec, breaking SIGTERM-based graceful shutdown in containers

2 participants