Skip to content

fix: changed opensearch startup to turn opensearch into main process#1326

Merged
lucaseduoli merged 4 commits intomainfrom
fix/opensearch_startup
Apr 2, 2026
Merged

fix: changed opensearch startup to turn opensearch into main process#1326
lucaseduoli merged 4 commits intomainfrom
fix/opensearch_startup

Conversation

@lucaseduoli
Copy link
Copy Markdown
Collaborator

@lucaseduoli lucaseduoli commented Apr 2, 2026

This pull request improves the OpenSearch container's startup and shutdown processes, enhances reliability, and simplifies configuration. The main changes introduce a custom entrypoint script to handle graceful shutdown and background security setup, update the Dockerfile and docker-compose setup to use this script, and add a healthcheck for better container monitoring.

Entrypoint and Graceful Shutdown Improvements:

  • Added a new script opensearch-entrypoint-wrapper.sh that starts OpenSearch in the background, runs the security setup after a delay, and handles graceful shutdown on termination signals.
  • Modified the Dockerfile to copy and use the new entrypoint wrapper, ensuring it has the correct permissions and ownership. The container now uses this script as its entrypoint instead of the default. [1] [2]

Docker Compose Configuration Enhancements:

  • Updated docker-compose.yml to remove the custom command logic for startup and security setup (now handled by the entrypoint wrapper), switched to using a named volume for persistent data (opensearch-data), and added a stop_grace_period for graceful shutdown. [1] [2]
  • Added a healthcheck to the OpenSearch service in docker-compose.yml to monitor cluster health, improving reliability and observability.

Related Pull Requests

lucaseduoli and others added 4 commits April 2, 2026 15:08
Issue

- #1170

Summary

- Improved OpenSearch graceful shutdown reliability by flushing pending writes, adding a force-kill fallback in the entrypoint wrapper, and preventing double-close of the client connection.
- Hardened the Docker Compose healthcheck to verify authenticated cluster health status rather than bare connectivity.

OpenSearch Shutdown Improvements

- Replaced `cluster.health()` call with `indices.flush(index="_all", wait_if_ongoing=True)` in `graceful_opensearch_shutdown` to ensure pending write operations are persisted before the client closes.
- Set `clients.opensearch = None` after graceful shutdown in `src/main.py` to prevent a redundant double-close during `clients.cleanup()`.
- Added a 90-second wait loop with `kill -0` polling in `opensearch-entrypoint-wrapper.sh` before issuing a force `SIGKILL`, ensuring the process has time to stop cleanly before being forcibly terminated.
- Removed stale "Made with Bob" comment from `opensearch-entrypoint-wrapper.sh`.

Docker Compose Healthcheck

- Updated the OpenSearch healthcheck command to authenticate with `admin:$OPENSEARCH_PASSWORD` and query `/_cluster/health`, asserting the cluster status is `green` or `yellow` rather than only checking for a successful TCP connection.
@lucaseduoli lucaseduoli self-assigned this Apr 2, 2026
@github-actions github-actions bot added docker bug 🔴 Something isn't working. labels Apr 2, 2026
@lucaseduoli lucaseduoli requested a review from mpawlow April 2, 2026 18:18
@github-actions github-actions bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Apr 2, 2026
Copy link
Copy Markdown
Collaborator

@mpawlow mpawlow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review 1

  • ✅ LGTM / Approved
  • Approval based on changes already reviewed & approved for release-0.4.1

@github-actions github-actions bot added the lgtm label Apr 2, 2026
@lucaseduoli lucaseduoli merged commit 87dfd2d into main Apr 2, 2026
13 checks passed
@github-actions github-actions bot deleted the fix/opensearch_startup branch April 2, 2026 19:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug 🔴 Something isn't working. docker lgtm

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants