Skip to content

Migrate from uWSGI to Gunicorn#3387

Open
ayushgupta704 wants to merge 4 commits into
intelowlproject:developfrom
ayushgupta704:migrate-uwsgi-to-gunicorn-clean
Open

Migrate from uWSGI to Gunicorn#3387
ayushgupta704 wants to merge 4 commits into
intelowlproject:developfrom
ayushgupta704:migrate-uwsgi-to-gunicorn-clean

Conversation

@ayushgupta704
Copy link
Copy Markdown
Contributor

Description

This PR replaces the legacy uWSGI server with Gunicorn to modernize the WSGI layer. Gunicorn is easier to maintain and
better supported in modern container environments, and it provides better stability for the high-concurrency
workloads IntelOwl handles.

  • Swapped uWSGI for gunicorn==23.0.0 in the requirements.
  • Created configuration/gunicorn.conf.py tuned for 12 workers and 2 threads.
  • Updated Nginx to use standard HTTP proxy_pass instead of the uwsgi protocol.
  • Refactored the Docker entrypoint (uwsgi.sh) to use exec gunicorn for better
  • signal handling and updated healthchecks to use curl.
  • Removed the old intel_owl.ini configuration.

Type of change

  • New feature (non-breaking change which adds functionality).

Checklist

  • I have read and understood the rules about [how to Contribute (https://intelowlproject.github.io/docs/IntelOwl/contribute/) to this project
  • The pull request is for the branch develop
  • I have inserted the copyright banner at the start of the file:
  • This file is a part of IntelOwl https://github.com/intelowlproject/IntelOwl #
  • See the file 'LICENSE' for copying permission.
  • Linters (Ruff) gave 0 errors. If you have correctly installed [pre-commit] (https://intelowlproject.github.io/docs/IntelOwl/contribute/#how-to-start-set
    up-project-and-development-instance), it does these checks and adjustments on
    your behalf.
  • I have added tests for the feature/bug I solved (see tests folder). All
    the tests (new and old ones) gave 0 errors.

This PR Closes #3375

- Replace uWSGI/uwsgitop with Gunicorn in project requirements.
- Add Gunicorn configuration with 12 workers and 2 threads for high concurrency.
- Update Nginx to use HTTP proxying instead of uWSGI protocol.
- Refactor entrypoint with exec for proper signal handling.
- Modernize healthchecks to use HTTP-based validation.
- Retain 'uwsgi' service naming for backward compatibility.
@ayushgupta704
Copy link
Copy Markdown
Contributor Author

Hi @mlodic
The changes are completed as per the GreedyBear repo. All tests are passing, and the PR is ready for review. Please let me know if any updates are needed.
Thankyou!!

@IshaanXCoder
Copy link
Copy Markdown
Contributor

hey!
I did a quick overview, i think some changes have to be made, PTAL :)

  • dockerfile/default.yml calls the service uwsgi and intelowl_uwsgi container(has to be renamed)
  • after making the above change, make sure to make changes in these scripts docker/scripts/manage.sh,tail-logs.sh and initdb.sh as all of them execute via docker exec -ti intelowl_uwsgi
  • IG all docker/*.override.yml should be changed
  • All integrations in integrations/*/compose.yml still have depends_on: uwsgi
  • changes have to be made in configuration/nginx

@ayushgupta704
Copy link
Copy Markdown
Contributor Author

Hi!
Thank you for the thorough review. The functional migration was just the first step, and I appreciate the push toward improving the structure, it was definitely needed.

@ayushgupta704 ayushgupta704 force-pushed the migrate-uwsgi-to-gunicorn-clean branch from 8409949 to 9037fc6 Compare February 28, 2026 12:00
Applied architectural refinements for Idea intelowlproject#9:
- Dynamic worker calculation based on CPU cores for better resource utilization.
- Switch to stdout/stderr logging for superior Docker observability.
- Configure forwarded_allow_ips for accurate remote IP logging through Nginx.
- Finalize L7 HTTP-based health checks.
@mlodic
Copy link
Copy Markdown
Member

mlodic commented Mar 8, 2026

you need to provide proof that the changes work, otherwise this PR won't be reviewed.

@ayushgupta704
Copy link
Copy Markdown
Contributor Author

ayushgupta704 commented Mar 9, 2026

Hi @mlodic,
I've just finished verifying the migration from uWSGI to Gunicorn on my end. I wanted to provide the proof you asked for to show that the new setup is working correctly and everything is tied together properly.

  • First, here are the logs from the backend container. You can see Gunicorn starting up and the workers booting without any issues:
InShot_20260309_162006711.mp4
  • I also double-checked that Nginx is proxying requests to Gunicorn correctly. If you look at the curl -I output, you can see that Gunicorn is the one handling the responses now:
InShot_20260309_162111003.mp4
  • Lastly, the new curl-based health checks are working exactly as intended. All the services are reporting as healthy in
    docker ps:
InShot_20260309_162159194.mp4

Everything seems to be running smoothly with the new architecture. Let me know if there’s anything else you’d like me to check!!
Thanks.

@mlodic
Copy link
Copy Markdown
Member

mlodic commented Mar 13, 2026

thanks for providing the info.
Considering this is a big impact, I reserve to review this PR with more available time and to plan to merge this in a major release where we'll also update other architecture stuff (see postgres and python update).
Right now we have a lot of small changes that will be released in April, I'll wait for that and then we'll consider add this too.

@mlodic mlodic added the keep-open To avoid workflow closing PRs label Mar 13, 2026
@ayushgupta704
Copy link
Copy Markdown
Contributor Author

ayushgupta704 commented Mar 15, 2026

Thank you for the update, @mlodic.
I completely understand waiting for the major release. I’ve done a final audit of the branch so it's fully polished and ready whenever you have more time to review it.
I’m also currently finalising my proposal! I'm really looking forward to the possibility of
contributing more deeply to IntelOwl’s core architecture.

@ayushgupta704
Copy link
Copy Markdown
Contributor Author

Hi @mlodic, just wanted to check if you'll be able to review this. Even if I'm not part of GSoC this year, I really want to keep contributing to the project and get this merged. Thanks!

@mlodic
Copy link
Copy Markdown
Member

mlodic commented Jun 4, 2026

we flagged this as "keep-open", but unfortunately it will consider only after the GSoC period because the are no other maintainers as of now and I can't merge such an impactful change without proper testing. I have no time for this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

keep-open To avoid workflow closing PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants