feat(docker): upgrade litestream to v0.5.0#3165
Merged
peterschmidt85 merged 6 commits intodstackai:masterfrom Oct 8, 2025
Merged
feat(docker): upgrade litestream to v0.5.0#3165peterschmidt85 merged 6 commits intodstackai:masterfrom
peterschmidt85 merged 6 commits intodstackai:masterfrom
Conversation
sanbyk
approved these changes
Oct 8, 2025
Contributor
|
@DragonStuff @sanbyk Thank you very much for the PR. Of course I will review this PR shortly, but just want to confirm, do you plan to actually use dstack with Litestream? |
peterschmidt85
requested changes
Oct 8, 2025
Contributor
peterschmidt85
left a comment
peterschmidt85
left a comment
There was a problem hiding this comment.
Please see comments.
You can quickly validate if the configuration build is valid by running, for example:
docker build --build-arg VERSION=0.19.31 --load -f release/Dockerfile .
from docker/server (and -f stgn/Dockerfile for staging version)
Co-authored-by: Andrey Cheptsov <54148038+peterschmidt85@users.noreply.github.com>
Co-authored-by: Andrey Cheptsov <54148038+peterschmidt85@users.noreply.github.com>
Contributor
Author
|
@peterschmidt85 Incredible, you are so fast at reviewing, I apologize as I was just about to test and I think I inconvenienced you with my typo. Thank you very much for fixing my PR up! Looks like it builds successfully, and I'm going to run this image on our infra as well to test. Details
|
Contributor
|
Contributor
Author
|
@peterschmidt85 Good catch. After working with Litestream on another project, it has a few caveats and is definitely still pre-v1. Perhaps going down the route of using PostgreSQL is a good option, but I was thinking it would be really nice to not have another PostgreSQL database to manage. Don't worry, I totally understand the risks of basically having a single server for I've written a short bash script to implement the 
After deleting the container and restarting, it comes up correctly as well. 
Perhaps in a follow-up PR it would be good to add additional arguments to allow Litestream args to be set from environment variables. |
peterschmidt85
approved these changes
Oct 8, 2025
Contributor
peterschmidt85
left a comment
peterschmidt85
left a comment
There was a problem hiding this comment.
From my side, it looks good. @r4victor please have a look too just in case!
Contributor
|
@DragonStuff Merged. Please give it a thorough test, and let us know if all is good! |
Contributor
Author
@peterschmidt85, we did some testing overnight with a service running, with the dstack deployment on spot instances to make sure it gets interrupted a few times and everything seems to work as expected. We will keep this running for our staging environment and if we encounter any issues, I’ll make sure to make an issue and PR to fix it. Thanks again for your prompt assistance, dstack is awesome! 🥂 |
un-def
added a commit
that referenced
this pull request
Nov 14, 2025
SSHTunnel.open() starts the ssh process with -f option, which causes it to become a daemon (it calls daemon(3) after authentication). Then, SSHTunnel.close() uses `ssh -O exit` to ask that daemon process to exit (the processes communicate via the control socket, -S option). The daemon exits, and then PID 1 should reap it. Before #3165, PID 1 was bash, which handles SIGCHLD, properly wait(2)ing children, but in that pull request exec was added, making `dstack server` PID 1. dstack does nothing with SIGCHLD (neither handles nor explicitly ignores it), thus the default disposition (ignore the signal but do not discard children) leads to an evergrowing number of unreaped zombies. Fixes: #3291
un-def
added a commit
that referenced
this pull request
Nov 17, 2025
SSHTunnel.open() starts the ssh process with -f option, which causes it to become a daemon (it calls daemon(3) after authentication). Then, SSHTunnel.close() uses `ssh -O exit` to ask that daemon process to exit (the processes communicate via the control socket, -S option). The daemon exits, and then PID 1 should reap it. Before #3165, PID 1 was bash, which handles SIGCHLD, properly wait(2)ing children, but in that pull request exec was added, making `dstack server` PID 1. dstack does nothing with SIGCHLD (neither handles nor explicitly ignores it), thus the default disposition (ignore the signal but do not discard children) leads to an evergrowing number of unreaped zombies. Fixes: #3291
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The SRE team at TableCheck is attempting to deploy dstack for our GPU workloads, but we are currently blocked due to an issue we have identified with EKS Pod Identity (OIDC) support and Litestream.
This pull request in Litestream adds support benbjohnson/litestream#683, but the version of litestream used in dstack's Docker images is quite old (v0.3.9) and doesn't include this support.
As a result, the Docker image will start with the following error from Litestream:
This pull request updates the version of Litestream to the latest version available in https://github.com/benbjohnson/litestream/releases.