Replace Watchman with django-watchfiles for development auto-reloading. Closes #3376

[rootp1]

This PR migrates the file-watching mechanism used during development from Watchman (pywatchman) to django-watchfiles.

Changes:

• Requirements: Removed pywatchman and the django-server-requirements.txt file. Added django-watchfiles>=1.4.0 to test-requirements.txt (dev-only dependency).
• Dockerfile: Removed the WATCHMAN build arg, the watchman_install.sh script invocation, and the gcc/python3-dev build dependencies that were only needed to compile Watchman.
• Settings: Conditionally appends django_watchfiles to INSTALLED_APPS when DEBUG=True and the package is importable. This automatically hooks into Django's autoreload module, replacing WatchmanReloader with WatchfilesReloader.
• Scripts: Removed the watchman_install.sh script that downloaded and installed the Watchman binary.

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue).

Checklist

• I have read and understood the rules about how to Contribute to this project
• The pull request is for the branch develop
• A new plugin (analyzer, connector, visualizer, playbook, pivot or ingestor) was added or changed, in which case:
  • I strictly followed the documentation "How to create a Plugin"
  • Usage file was updated. A link to the PR to the docs repo has been added as a comment here.
  • Advanced-Usage was updated (in case the plugin provides additional optional configuration). A link to the PR to the docs repo has been added as a comment here.
  • I have dumped the configuration from Django Admin using the dumpplugin command and added it in the project as a data migration. ("How to share a plugin with the community")
  • If a File analyzer was added and it supports a mimetype which is not already supported, you added a sample of that type inside the archive test_files.zip and you added the default tests for that mimetype in test_classes.py.
  • If you created a new analyzer and it is free (does not require any API key), please add it in the FREE_TO_USE_ANALYZERS playbook by following this guide.
  • Check if it could make sense to add that analyzer/connector to other freely available playbooks.
  • I have provided the resulting raw JSON of a finished analysis and a screenshot of the results.
  • If the plugin interacts with an external service, I have created an attribute called precisely url that contains this information. This is required for Health Checks (HEAD HTTP requests).
  • If a new analyzer has beed added, I have created a unittest for it in the appropriate dir. I have also mocked all the external calls, so that no real calls are being made while testing.
  • I have added that raw JSON sample to the get_mocker_response() method of the unittest class. This serves us to provide a valid sample for testing.
  • I have created the corresponding DataModel for the new analyzer following the documentation
• I have inserted the copyright banner at the start of the file: # This file is a part of IntelOwl https://github.com/intelowlproject/IntelOwl # See the file 'LICENSE' for copying permission.
• Please avoid adding new libraries as requirements whenever it is possible. Use new libraries only if strictly needed to solve the issue you are working for. In case of doubt, ask a maintainer permission to use a specific library.
• If external libraries/packages with restrictive licenses were added, they were added in the Legal Notice section.
• Linters (Ruff) gave 0 errors. If you have correctly installed pre-commit, it does these checks and adjustments on your behalf.
• I have added tests for the feature/bug I solved (see tests folder). All the tests (new and old ones) gave 0 errors.
• If the GUI has been modified:
  • I have a provided a screenshot of the result in the PR.
  • I have created new frontend tests for the new component or updated existing ones.
• After you had submitted the PR, if DeepSource, Django Doctors or other third-party linters have triggered any alerts during the CI checks, I have solved those alerts.

Important Rules

• If you miss to compile the Checklist properly, your PR won't be reviewed by the maintainers.
• Everytime you make changes to the PR and you think the work is done, you should explicitly ask for a review by using GitHub's reviewing system detailed here.

[Copilot]

Pull request overview

Migrates the development auto-reload file-watching mechanism from Watchman/pywatchman to django-watchfiles, simplifying the Docker build and removing the custom Watchman installation flow.

Changes:

• Added django-watchfiles as a dependency and removed the standalone Watchman requirements file.
• Enabled django_watchfiles in INSTALLED_APPS when DEBUG=True.
• Removed Watchman-related Docker build args/env vars and the Watchman install script invocation.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
requirements/project-requirements.txt	Adds django-watchfiles dependency.
requirements/django-server-requirements.txt	Removes pywatchman requirements file.
intel_owl/settings/init.py	Appends django_watchfiles to INSTALLED_APPS in debug.
docker/test.override.yml	Removes Watchman-related build arg and env var.
docker/scripts/watchman_install.sh	Removes Watchman installer script.
docker/Dockerfile	Removes Watchman env var and installer script invocation.

Comments suppressed due to low confidence (1)

docker/scripts/watchman_install.sh:1

• The removed watchman_install.sh script downloaded and installed prebuilt watchman binaries directly from GitHub via wget and unzip without any checksum or signature verification. If the tag or download endpoint were ever compromised or intercepted, an attacker could supply a malicious watchman binary that would run inside the container with elevated privileges. This change removes that behavior; for any future bootstrap scripts that fetch binaries, ensure downloads are pinned to immutable identifiers (e.g., commit SHAs/releases) and verified with checksums or signatures.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

docker/scripts/watchman_install.sh:1

• The removed watchman_install.sh script previously downloaded and installed precompiled watchman binaries directly from GitHub using wget/unzip and cp into /usr/local/bin without any checksum or signature verification, creating a supply‑chain RCE risk if the download source or network were compromised. Because this PR deletes that script and its invocation from the Dockerfile, that specific risk is now eliminated. For any future installer scripts, ensure all remotely downloaded artifacts are verified (e.g., with pinned checksums or signatures) or installed via a trusted package manager instead of ad‑hoc wget/cp flows.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[rootp1]

@mlodic, i have done the watchman to watchfiles implementation, and now I have took a look at more changes, in the #890 of greedybear, and what scope I get to see is

1. Dockerfile restructuring for different images for dev stage
2. Entrypoint refactored to exec "$@" pattern (after the Migrate from uWSGI to Gunicorn #3387 )
3. Health checks moved from Dockerfile_nginx to compose

SO yeah, @mlodic please review the current implementation and please let me know, whether I can open a new issue for the next betterment, or just a new pr, or in this pr itself??

[rootp1]

@mlodic, i have done the watchman to watchfiles implementation, and now I have took a look at more changes, in the #890 of greedybear, and what scope I get to see is

1. Dockerfile restructuring for different images for dev stage
2. Entrypoint refactored to exec "$@" pattern (after the Migrate from uWSGI to Gunicorn #3387 )
3. Health checks moved from Dockerfile_nginx to compose

SO yeah, @mlodic please review the current implementation and please let me know, whether I can open a new issue for the next betterment, or just a new pr, or in this pr itself??

hey @mlodic, what is your opinion on the next ideas

[mlodic]

@rootp1 you have to exercise patience, we get notifications and we are doing this for free, so you need to respect our time

[mlodic]

I cannot accept this PR without proof that the change works. So please show that this works as it should with a video showing how the changes are properly reloaded with the new packages as they should and how this works only in testing environments

[mlodic]

unrelated watchman changes should be addressed in different issues/PRs

[github-actions]

This pull request has been marked as stale because it has had no activity for 10 days. If you are still working on this, please provide some updates or it will be closed in 5 days.

[mlodic]

updates on this?

[rootp1]

onto it, @mlodic
will upload a video till eod

[rootp1]

thankyou for considering

[rootp1]

@rootp1 you have to exercise patience, we get notifications and we are doing this for free, so you need to respect our time

Hi @mlodic , I’m really sorry for tagging you again on this. I realize now that it may have come across as pushy, which wasn’t my intention at all.

I think I was under the impression that the issue might go stale if there wasn’t any activity, and that led me to follow up once prematurely. That said, I completely understand and respect that you are contributing your time voluntarily, and I truly respecting the effort you put into reviewing and mentoring I ask apology.

I’ll make sure to be more patient going forward and won’t repeat this.

updates on this?

I’ve attached a video demonstrating that the changes work as expected. It took some time to put this video together, so I hope it helps clarify all the changes.

video.mp4