Improve robustness, security, and CI-friendliness of run-tests-in-docker.sh

[AyushInKC]

Description

The bin/run-tests-in-docker.sh script currently works well for local testing, but it can be made more robust, secure, and CI-friendly with a few targeted improvements. These changes align the script more closely with production-grade container execution and common best practices for infrastructure scripts.

Motivation

Improve failure detection and error handling in CI environments

Make the script resilient to being executed from any working directory

Strengthen container sandboxing to better mirror Exercism’s production runner

Improve reproducibility and debuggability for contributors

Proposed Improvements

Stricter shell safety

Use #!/usr/bin/env bash with set -Eeuo pipefail instead of relying solely on -e

Path robustness

Resolve the project root dynamically instead of relying on $PWD

Ensures the script works correctly when executed from any directory or CI runner

More reproducible Docker builds

Add --pull and --no-cache to docker build to avoid stale base images

Stronger container isolation

Add resource limits and security flags:

--pids-limit

--memory

--cpus

--security-opt no-new-privileges

Better reflects the sandboxing used in production test runners

Read-only bind mounts

Mount test data and scripts as read-only to prevent accidental mutation

Explicit exit-code handling

Capture and propagate the container exit code to ensure CI fails correctly on test errors

Lightweight logging

Add clear, minimal log messages to improve traceability during CI failures

Benefits

More reliable CI behavior

Clearer failure modes and logs

Improved security posture of the Docker execution

Easier local and automated testing for contributors

Better alignment with Exercism’s production environment

Backwards Compatibility

These changes do not alter the external behavior of the script or its interface.
They only improve safety, clarity, and reliability.

[AyushInKC]

I'd like to take this on!

[kahgoh]

I think the suggestions here largely seem fine. The first item about adding set -Eeuo pipefail would be handy to spot certain errors in the script. My only concern is related to this item:

Add --pull and --no-cache to docker build to avoid stale base images

Would this be adding to the docker build command in run-tests-in-docker.sh and run-in-docker.sh? If so, then would it be pulling down the base image each time I run the script (even if the base image hasn't changed)? If so, I'd prefer not to because that would make testing and development cycle slower if the base image hasn't changed. Alternatively, could it be made an option passed to the script?

Could some one from @exercism/guardians also have a look over?

[IsaacG]

Use #!/usr/bin/env bash with set -Eeuo pipefail instead of relying solely on -e

This one is a bit of a hammer that people like blindly applying and isn't always a good change 😄 See
http://mywiki.wooledge.org/BashFAQ/105
It can be fine but isn't always an improvement. Running shellcheck and applying fixes it catches is usually much more valuable.

Add --pull and --no-cache to docker build to avoid stale base images

I'm not very familiar with docker so I don't know what this will do without testing it. It does sound like this will slow builds down. I'm not sure that there's much advantage for us to using this here. Stale base images isn't exactly a big concern here.