Skip to content

Health check support #18

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
crummy wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Health check support #18

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
aebc77b
Health check lets us know when sync is finished
crummy Dec 13, 2020
94ef408
Increase timeout; it can take a while to download files
crummy Mar 8, 2021
7da4cca
Wait max 4min for container to boot
crummy Mar 8, 2021
081b1df
10min timeout
crummy Mar 10, 2021
4003684
Add timestamps, and support for exclusions
crummy Mar 28, 2024
0516152
Make sure failures are properly caught
crummy Mar 28, 2024
8a172b0
No point in adding timestamps, just use docker logs --timestamps
crummy Mar 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
FROM alpine:3.10
label maintainer="Elementar Sistemas <contato@elementarsistemas.com.br>"
LABEL maintainer="Malcolm Crum <crummynz@gmail.com>"

RUN apk --no-cache add bash py3-pip && pip3 install --no-cache-dir awscli
RUN apk --no-cache add bash py3-pip && \
pip3 install --no-cache-dir awscli
ADD watch /watch

VOLUME /data

HEALTHCHECK --interval=2s --retries=300 \
CMD stat /var/healthy.txt || exit 1

ENV S3_SYNC_FLAGS "--delete"
ENTRYPOINT [ "./watch" ]
CMD ["/data"]
4 changes: 2 additions & 2 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
NAME = elementar/s3-volume
NAME = crummy/s3-volume

.PHONY: build release

build:
docker build -t $(NAME):latest .
docker build --platform linux/amd64 -t $(NAME):latest .

release:
docker push $(NAME):latest
17 changes: 13 additions & 4 deletions watch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/bin/bash
set -o pipefail

[[ "$TRACE" ]] && set -x

Expand Down Expand Up @@ -39,11 +40,16 @@ done
PROGNAME=$0
LOCAL=$1
REMOTE=$2
HEALTHCHECK_FILE=/var/healthy.txt

if [ "$ENDPOINT_URL" ]; then
AWS="aws --endpoint-url $ENDPOINT_URL"
else
AWS=aws
AWS="aws"
fi

if [ "$EXCLUDE_PATTERN" ]; then
S3_EXCLUDE_FLAGS="--exclude $EXCLUDE_PATTERN"
fi

function restore {
Expand All @@ -54,22 +60,25 @@ function restore {
fi

echo "restoring $REMOTE => $LOCAL"
if ! $AWS s3 sync "$REMOTE" "$LOCAL"; then
if ! $AWS s3 sync "$REMOTE" "$LOCAL" $EXCLUDE_PATTERN; then
error_exit "restore failed"
fi
touch $HEALTHCHECK_FILE
}

function backup {
echo "backup $LOCAL => $REMOTE"
if ! $AWS s3 sync "$LOCAL" "$REMOTE" $S3_SYNC_FLAGS; then
if ! $AWS s3 sync "$LOCAL" "$REMOTE" $S3_SYNC_FLAGS $EXCLUDE_PATTERN; then
echo "backup failed" 1>&2
rm $HEALTHCHECK_FILE
return 1
fi
touch $HEALTHCHECK_FILE
}

function final_backup {
echo "backup $LOCAL => $REMOTE"
while ! $AWS s3 sync "$LOCAL" "$REMOTE" $S3_SYNC_FLAGS; do
while ! $AWS s3 sync "$LOCAL" "$REMOTE" $S3_SYNC_FLAGS $EXCLUDE_PATTERN; do
echo "backup failed, will retry" 1>&2
sleep 1
done
Expand Down