diff --git a/README.md b/README.md index bbd8dff..aeb80be 100644 --- a/README.md +++ b/README.md @@ -1,110 +1,131 @@ # fetch-runner -Run scripts when `git fetch` finds new commits to specified git branches +Run scripts when `git fetch` finds new commits to specified git branches. + +## Overview + +Two users matter: + +- **`[general].user`** — the user fetch-runner itself runs as (also + `User=` in the systemd unit). fetch-runner refuses to start if the + running uid doesn't match. +- **`[[jobs]].run_as`** — per-job; owns this job's repo and runs its git + ops and deploy script. Defaults to `[general].user`. When set + differently, fetch-runner dispatches everything via `sudo -n -u ` + and a sudoers rule must allow it (generate one with + `fetch-runner --print-sudoers `). + +Convention assumed throughout the docs is that each repo lives at +`/srv///`, owned `:` mode `0755`. +That gives every path under `/srv//` a single owner and lets +`[general].user` traverse with just search permission. ## Setup ### 1. Install fetch-runner -As the deploy user (we use `my-app-user` in this document), install fetch-runner with `uv`: +As `[general].user` (we use `fetch-runner` below): ```bash uv tool install git+https://github.com/BYU-ODH/fetch-runner ``` -Note the path of the installed executable (typically something like -`/home/my-app-user/.local/bin/fetch-runner`). You will need it in step 4. +Note the installed executable path (typically +`/home/fetch-runner/.local/bin/fetch-runner`). -### 2. Add a deploy script to your app +### 2. Add a deploy script to each app -Copy `examples/deploy.sh` from this repository into your app's directory: +For a job with `run_as = "app1"` deploying the `api` repo: ```bash -cp /path/to/fetch-runner/examples/deploy.sh /srv/myapp/deploy.sh -chmod +x /srv/myapp/deploy.sh +sudo -u app1 cp /path/to/fetch-runner/examples/deploy.sh /srv/app1/api/deploy.sh +sudo -u app1 chmod +x /srv/app1/api/deploy.sh ``` -Open the script and replace every occurrence of `deploy-user` with the -username that runs your app's deployments (e.g. `my-app-user`). The guard block -at the top of the script prevents accidental execution as the wrong user or -as root. +Replace every `deploy-user` in the script with the job's `run_as` user +(`app1` here). The guard block at the top refuses to run as any other +user. Regenerate it for a different user with: -Consider committing this script to your app's repository so the deploy -procedure is version-controlled alongside the code it deploys. +```bash +fetch-runner --print-guard app1 +``` -### 3. Create the jobs config +Commit the script to the app's repo so deploys are version-controlled. -Copy the example config to the deploy user's home directory and edit it: +### 3. Create the jobs config ```bash -cp /path/to/fetch-runner/examples/jobs.toml /home/my-app-user/jobs.toml +cp /path/to/fetch-runner/examples/jobs.toml /home/fetch-runner/jobs.toml ``` -Edit `/home/my-app-user/jobs.toml`: +Per `[[jobs]]`: +- `name` — label shown in logs +- `path` — absolute repo path, owned and writable by `run_as` +- `branch` — branch to watch +- `script` — absolute script path +- `run_as` — optional; defaults to `[general].user` +- `timeout_seconds` — optional script timeout -- Set `user` under `[general]` to the deploy user (e.g. `"my-app-user"`). - fetch-runner exits at startup if the running user does not match this value. -- Set `poll_interval_seconds` to how often fetch-runner should check for new - commits (default: `60`). -- For each `[[jobs]]` entry, set: - - `name` — a human-readable label shown in logs - - `path` — absolute path to the local git repository to poll - - `branch` — the branch to watch (e.g. `"main"` or `"production"`) - - `script` — absolute path to the script to run when new commits are found - (e.g. `/srv/myapp/deploy.sh`) - - `timeout_seconds` — how long to let the script run before killing it - (optional; omit to use the default) +Validate without starting: -### 4. Install and start the systemd service +```bash +fetch-runner --check /home/fetch-runner/jobs.toml +``` -Copy the example unit file to systemd's unit directory: +### 4. Install the systemd service ```bash sudo cp /path/to/fetch-runner/examples/fetch-runner.service \ /etc/systemd/system/fetch-runner.service ``` -Edit `/etc/systemd/system/fetch-runner.service` and update the lines in the -`CUSTOMIZE` block: - -- **`User` / `Group`** — set both to your deploy user (must match `user` in - `jobs.toml`). -- **`ExecStart`** — replace `/usr/local/bin/fetch-runner` with the full path - to the executable you noted in step 1, then replace - `/etc/fetch-runner/jobs.toml` with the path to the config file from step 3. - For example: - ``` - ExecStart=/home/my-app-user/.local/bin/fetch-runner /home/my-app-user/jobs.toml - ``` -- **`ReadWritePaths`** — list every directory your deploy scripts need to - write to (at minimum, the parent directories of your git repositories). - Space-separate multiple paths, e.g.: - ``` - ReadWritePaths=/srv/myapp /srv/anotherapp - ``` - -Reload systemd and enable the service: +In the `CUSTOMIZE` block, set: +- `User` / `Group` to `[general].user` +- `ExecStart` to the binary path from step 1 plus your config path +- `ReadWritePaths` to every directory any child process writes to — + including the repos themselves (sudo'd git is still inside the unit's + filesystem sandbox) + +The example unit omits `NoNewPrivileges=` and `RestrictSUIDSGID=` +because they block sudo's setuid. The sudoers fragment (step 5) is what +bounds the privilege. If every job uses `run_as = [general].user`, you +can re-enable both. + +### 5. Install the sudoers fragment (only if any job sets a different `run_as`) ```bash -sudo systemctl daemon-reload -sudo systemctl enable --now fetch-runner +fetch-runner --print-sudoers /home/fetch-runner/jobs.toml \ + | sudo tee /etc/sudoers.d/fetch-runner > /dev/null +sudo chmod 0440 /etc/sudoers.d/fetch-runner +sudo visudo -cf /etc/sudoers.d/fetch-runner # syntax check ``` -## Debugging +Re-run after any `jobs.toml` change. The git rule is intentionally not +arg-restricted: running git as `run_as` is no broader than what the +deploy-script rule already grants. -Check whether the service is running and see its recent log output: +### 6. Enable and start ```bash -systemctl status fetch-runner +sudo systemctl daemon-reload +sudo systemctl enable --now fetch-runner ``` -Stream the full journal for the service (most useful when a deployment fails): +## Migrating from single-user mode -```bash -journalctl -u fetch-runner -f -``` +Existing configs without `run_as` keep working unchanged — sudo is +skipped entirely. To split, add `run_as` per job, update each script's +guard for the new user, regenerate the sudoers fragment, reload. -To review all logs since the service last started: +## Debugging ```bash +systemctl status fetch-runner +journalctl -u fetch-runner -f journalctl -u fetch-runner -b ``` + +- `sudo: a password is required` → sudoers fragment is missing or stale; + re-run step 5. +- `fetch-runner-guard: refusing to run as ` → the script's guard + names a user that doesn't match the job's `run_as`; regenerate with + `fetch-runner --print-guard `. diff --git a/examples/fetch-runner.service b/examples/fetch-runner.service index 4393399..7d86b03 100644 --- a/examples/fetch-runner.service +++ b/examples/fetch-runner.service @@ -11,19 +11,20 @@ Type=simple # CUSTOMIZE: edit the lines in this block for each deployment. # =========================================================================== -# MUST match the `user` field in the jobs.toml this unit starts. -# fetch-runner exits at startup if they do not match. -User=deploy -Group=deploy +# Must match [general].user in jobs.toml — fetch-runner refuses to start +# otherwise. This user does not need to own the repos; each repo is +# owned by its job's run_as user, and git runs as that user via sudo. +User=fetch-runner +Group=fetch-runner # Path to the fetch-runner binary and to the config file it should load. # If you installed fetch-runner from a venv, point ExecStart at that venv's # bin/fetch-runner. The config path is positional. ExecStart=/usr/local/bin/fetch-runner /etc/fetch-runner/jobs.toml -# Every directory your deploy scripts need to write to must appear here -# (the git repos you poll, plus any app state they touch). Separate with -# spaces. The rest of the filesystem is read-only thanks to ProtectSystem. +# Every directory any child process writes to — git repos and any app +# state the deploy scripts touch. Sudo'd children inherit this sandbox. +# The rest of the filesystem is read-only thanks to ProtectSystem. ReadWritePaths=/srv # =========================================================================== @@ -41,9 +42,9 @@ TimeoutStopSec=30s StandardOutput=journal StandardError=journal -# Block privilege escalation; the deploy user never needs more than it has. -NoNewPrivileges=true -RestrictSUIDSGID=true +# NoNewPrivileges / RestrictSUIDSGID block sudo's setuid transition, so +# they are off here. The sudoers fragment is what bounds the privilege. +# If every job uses run_as = [general].user, you can re-enable both. CapabilityBoundingSet= AmbientCapabilities= diff --git a/examples/fetch-runner.sudoers b/examples/fetch-runner.sudoers new file mode 100644 index 0000000..c08a159 --- /dev/null +++ b/examples/fetch-runner.sudoers @@ -0,0 +1,27 @@ +# Example sudoers fragment for fetch-runner. Generate the live copy from +# your jobs.toml — do not edit this by hand on the host: +# +# fetch-runner --print-sudoers /etc/fetch-runner/jobs.toml \ +# | sudo tee /etc/sudoers.d/fetch-runner > /dev/null +# sudo chmod 0440 /etc/sudoers.d/fetch-runner +# sudo visudo -cf /etc/sudoers.d/fetch-runner # syntax check +# +# The shape produced for the example jobs.toml is shown below. + +# Preserve FETCH_RUNNER_* env vars only when running these scripts. +Defaults!/srv/app1/api/deploy.sh env_keep += "FETCH_RUNNER_JOB FETCH_RUNNER_BRANCH FETCH_RUNNER_COMMIT FETCH_RUNNER_REPO" +Defaults!/srv/app2/web/deploy.sh env_keep += "FETCH_RUNNER_JOB FETCH_RUNNER_BRANCH FETCH_RUNNER_COMMIT FETCH_RUNNER_REPO" + +# Run git as each run_as user (repos are owned by that user). +fetch-runner ALL=(app1) NOPASSWD: /usr/bin/git +fetch-runner ALL=(app2) NOPASSWD: /usr/bin/git + +# Run each deploy script as its run_as user. +fetch-runner ALL=(app1) NOPASSWD: /srv/app1/api/deploy.sh +fetch-runner ALL=(app2) NOPASSWD: /srv/app2/web/deploy.sh + +# Notes: +# * Pin script paths absolutely; pin run_as per line — never (ALL). +# * NOPASSWD is required (no tty); the narrow pinning is what keeps it safe. +# * The git rule is not arg-restricted — running git as run_as is no +# broader than what the deploy-script rule already allows. diff --git a/examples/jobs.toml b/examples/jobs.toml index bac6528..e5dfb3d 100644 --- a/examples/jobs.toml +++ b/examples/jobs.toml @@ -1,20 +1,28 @@ # Sample fetch-runner config. -# fetch-runner will refuse to load this unless the process is running as the -# user named below (and not as root). +# +# [general].user — the user fetch-runner itself runs as. Must match the +# running uid or fetch-runner refuses to start. +# [[jobs]].run_as (optional) — the user that owns this job's repo and +# runs its git ops and script. Defaults to [general].user. When set, +# sudo is used and a sudoers rule is required +# (see `fetch-runner --print-sudoers `). +# +# Convention: each repo lives at /srv///. [general] -user = "deploy" +user = "fetch-runner" poll_interval_seconds = 60 - [[jobs]] name = "my example api" -path = "/srv/api" +path = "/srv/app1/api" branch = "main" -script = "/srv/api/deploy.sh" +script = "/srv/app1/api/deploy.sh" +run_as = "app1" timeout_seconds = 600 [[jobs]] name = "my example web" -path = "/srv/web" +path = "/srv/app2/web" branch = "production" -script = "/srv/web/deploy.sh" +script = "/srv/app2/web/deploy.sh" +run_as = "app2" diff --git a/src/fetch_runner/cli.py b/src/fetch_runner/cli.py index 63ccf87..43036c8 100644 --- a/src/fetch_runner/cli.py +++ b/src/fetch_runner/cli.py @@ -8,7 +8,11 @@ from . import __version__ from .config import ConfigError +from .config import RunnerConfig from .config import load_config +from .git_ops import GitError +from .git_ops import _git_absolute_path +from .guard import PRESERVED_ENVIRONMENT_VARIABLE_NAMES from .guard import GuardError from .guard import render_canonical_script_guard from .runner import GitPollingRunner @@ -32,6 +36,11 @@ def main(argv: list[str] | None = None) -> int: metavar="USER", help="print the canonical guard block for USER and exit (for pasting into a new script)", ) + argument_parser.add_argument( + "--print-sudoers", + action="store_true", + help="print the sudoers fragment required by the config and exit", + ) argument_parser.add_argument( "-v", "--verbose", @@ -64,6 +73,10 @@ def main(argv: list[str] | None = None) -> int: print(f"config error: {e}", file=sys.stderr) return 2 + if cli_args.print_sudoers: + sys.stdout.write(render_sudoers_fragment(runner_config)) + return 0 + if cli_args.check: print( f"ok: user={runner_config.runtime_user} " @@ -83,6 +96,58 @@ def _handle_stop_signal(signum, _frame): return runner.run_forever() +def render_sudoers_fragment(runner_config: RunnerConfig) -> str: + """Return a sudoers fragment authorizing every job whose ``run_as`` + differs from ``[general].user``. Output is sorted/deduplicated so + regenerating after a config change produces a reviewable diff. + """ + cross_user_jobs = [ + configured_job + for configured_job in runner_config.jobs + if configured_job.run_as_user != runner_config.runtime_user + ] + rendered_lines: list[str] = [] + rendered_lines.append( + "# Generated by `fetch-runner --print-sudoers`. " + "Install with `visudo -f /etc/sudoers.d/fetch-runner`, root:root mode 0440.\n" + ) + if not cross_user_jobs: + rendered_lines.append( + f"# (no jobs use a run_as different from {runner_config.runtime_user!r}; " + "no sudoers rules needed.)\n" + ) + return "".join(rendered_lines) + + try: + git_path = _git_absolute_path() + except GitError as e: + raise SystemExit(f"fetch-runner --print-sudoers: cannot resolve git path: {e}") from e + + unique_runas_users = sorted({j.run_as_user for j in cross_user_jobs}) + unique_script_paths = sorted({str(j.script_path) for j in cross_user_jobs}) + unique_runas_and_script_pairs = sorted( + {(j.run_as_user, str(j.script_path)) for j in cross_user_jobs} + ) + env_keep_value = " ".join(PRESERVED_ENVIRONMENT_VARIABLE_NAMES) + + rendered_lines.append("\n# Preserve FETCH_RUNNER_* env vars when running deploy scripts.\n") + for script_path in unique_script_paths: + rendered_lines.append(f'Defaults!{script_path} env_keep += "{env_keep_value}"\n') + + rendered_lines.append("\n# Run git as each run_as user (repos are owned by that user).\n") + for run_as_user_name in unique_runas_users: + rendered_lines.append( + f"{runner_config.runtime_user} ALL=({run_as_user_name}) NOPASSWD: {git_path}\n" + ) + + rendered_lines.append("\n# Run each deploy script as its run_as user.\n") + for run_as_user_name, script_path in unique_runas_and_script_pairs: + rendered_lines.append( + f"{runner_config.runtime_user} ALL=({run_as_user_name}) NOPASSWD: {script_path}\n" + ) + return "".join(rendered_lines) + + def _configure_logging(verbose: bool) -> None: logging.basicConfig( level=logging.DEBUG if verbose else logging.INFO, diff --git a/src/fetch_runner/config.py b/src/fetch_runner/config.py index b9ee7d3..7e387c9 100644 --- a/src/fetch_runner/config.py +++ b/src/fetch_runner/config.py @@ -9,12 +9,14 @@ from __future__ import annotations import os +import pwd import stat import tomllib from dataclasses import dataclass from pathlib import Path from .guard import GuardError +from .guard import _require_safe_user_name from .guard import require_expected_runtime_user from .guard import validate_canonical_script_guard @@ -30,6 +32,9 @@ class ConfiguredJob: branch_name: str script_path: Path script_timeout_seconds: int | None + # The user this job's git ops and script run as. Defaults to + # ``RunnerConfig.runtime_user`` when ``run_as`` is omitted in TOML. + run_as_user: str @dataclass(frozen=True) @@ -41,7 +46,7 @@ class RunnerConfig: _ALLOWED_TOP_LEVEL_KEYS = {"general", "jobs"} _ALLOWED_GENERAL_KEYS = {"user", "poll_interval_seconds"} -_ALLOWED_JOB_KEYS = {"name", "path", "branch", "script", "timeout_seconds"} +_ALLOWED_JOB_KEYS = {"name", "path", "branch", "script", "timeout_seconds", "run_as"} _DISALLOWED_BRANCH_CHARACTERS = frozenset(" \t\n\r\x00'\";|&`$<>()[]{}\\*?") @@ -139,7 +144,25 @@ def load_config(config_path: Path) -> RunnerConfig: script_path = Path( _require_non_empty_string(raw_job_section, "script", section_label, config_path) ).resolve() - _validate_job_script_file(script_path, runtime_user, section_label, config_path) + + # Resolve via passwd at load time so a typo fails fast instead of + # surfacing as a confusing sudo error during the first poll. + run_as_user = raw_job_section.get("run_as", runtime_user) + if not isinstance(run_as_user, str) or not run_as_user: + raise ConfigError(f"{config_path}: {section_label}.run_as must be a non-empty string") + try: + _require_safe_user_name(run_as_user) + except GuardError as e: + raise ConfigError(f"{config_path}: {section_label}.run_as: {e}") from e + try: + pwd.getpwnam(run_as_user) + except KeyError as e: + raise ConfigError( + f"{config_path}: {section_label}.run_as user {run_as_user!r} " + f"does not exist on this system" + ) from e + + _validate_job_script_file(script_path, run_as_user, section_label, config_path) script_timeout_seconds = raw_job_section.get("timeout_seconds") if script_timeout_seconds is not None: @@ -159,6 +182,7 @@ def load_config(config_path: Path) -> RunnerConfig: branch_name=branch_name, script_path=script_path, script_timeout_seconds=script_timeout_seconds, + run_as_user=run_as_user, ) ) diff --git a/src/fetch_runner/git_ops.py b/src/fetch_runner/git_ops.py index b5c835a..abb8bc3 100644 --- a/src/fetch_runner/git_ops.py +++ b/src/fetch_runner/git_ops.py @@ -1,32 +1,65 @@ """Minimal ``git`` wrappers. -The runner intentionally uses a very small, audited subset of ``git``. Subprocess calls use an argv list (never ``shell=True``); repo paths are passed with ``-C`` and branch names have already been validated against a conservative allowlist at config load. + +Each operation runs as the job's ``run_as`` user — directly when that is +the current process user, otherwise wrapped in ``sudo -n -u ``. """ from __future__ import annotations +import os +import pwd +import shutil import subprocess from pathlib import Path +from .guard import _require_safe_user_name + class GitError(Exception): pass -def _run_git_command(repo_path: Path, *git_args: str, timeout: float = 120) -> str: +def _git_absolute_path() -> str: + """Resolve git lazily. Sudoers Commands rules need the absolute path, so + this is shared between the sudo argv and ``render_sudoers_fragment``. + """ + git_path = shutil.which("git") + if git_path is None: + raise GitError("git executable not found in PATH") + return git_path + + +def _build_git_argv(repo_path: Path, run_as_user_name: str, git_args: tuple[str, ...]) -> list[str]: + _require_safe_user_name(run_as_user_name) + git_path = _git_absolute_path() + direct_git_argv = [git_path, "-C", str(repo_path), *git_args] + current_process_user_name = pwd.getpwuid(os.getuid()).pw_name + if run_as_user_name == current_process_user_name: + return direct_git_argv + return ["sudo", "-n", "-u", run_as_user_name, "--", *direct_git_argv] + + +def _run_git_command( + repo_path: Path, + *git_args: str, + run_as_user_name: str, + timeout: float = 120, +) -> str: + argv = _build_git_argv(repo_path, run_as_user_name, git_args) try: result = subprocess.run( - ["git", "-C", str(repo_path), *git_args], + argv, capture_output=True, text=True, timeout=timeout, check=False, ) except FileNotFoundError as e: - raise GitError("git executable not found in PATH") from e + raise GitError(f"executable not found: {argv[0]!r}") from e except subprocess.TimeoutExpired as e: raise GitError(f"git {' '.join(git_args)} in {repo_path} timed out after {timeout}s") from e if result.returncode != 0: @@ -35,10 +68,21 @@ def _run_git_command(repo_path: Path, *git_args: str, timeout: float = 120) -> s return result.stdout.strip() -def git_get_local_branch_commit_sha(repo_path: Path, branch_name: str) -> str: +def git_get_local_branch_commit_sha( + repo_path: Path, + branch_name: str, + *, + run_as_user_name: str, +) -> str: """Return the commit the local ``branch_name`` ref points at, or ``""`` if missing.""" try: - return _run_git_command(repo_path, "rev-parse", "--verify", f"refs/heads/{branch_name}") + return _run_git_command( + repo_path, + "rev-parse", + "--verify", + f"refs/heads/{branch_name}", + run_as_user_name=run_as_user_name, + ) except GitError: return "" @@ -46,6 +90,8 @@ def git_get_local_branch_commit_sha(repo_path: Path, branch_name: str) -> str: def git_fetch_branch_from_origin( repo_path: Path, branch_name: str, + *, + run_as_user_name: str, timeout: float = 120, ) -> str: """Fetch ``origin/`` and return the resulting ``FETCH_HEAD`` SHA.""" @@ -56,15 +102,32 @@ def git_fetch_branch_from_origin( "--no-tags", "origin", branch_name, + run_as_user_name=run_as_user_name, timeout=timeout, ) - return _run_git_command(repo_path, "rev-parse", "FETCH_HEAD") + return _run_git_command( + repo_path, + "rev-parse", + "FETCH_HEAD", + run_as_user_name=run_as_user_name, + ) def git_force_checkout_branch_to_commit( repo_path: Path, branch_name: str, commit_sha: str, + *, + run_as_user_name: str, ) -> None: """Force the local ``branch_name`` ref and working tree to ``commit_sha``.""" - _run_git_command(repo_path, "checkout", "--quiet", "--force", "-B", branch_name, commit_sha) + _run_git_command( + repo_path, + "checkout", + "--quiet", + "--force", + "-B", + branch_name, + commit_sha, + run_as_user_name=run_as_user_name, + ) diff --git a/src/fetch_runner/guard.py b/src/fetch_runner/guard.py index 2c34185..ece2406 100644 --- a/src/fetch_runner/guard.py +++ b/src/fetch_runner/guard.py @@ -25,6 +25,15 @@ GUARD_BEGIN_MARKER_PREFIX = "# >>> fetch-runner-guard:BEGIN" GUARD_END_MARKER = "# <<< fetch-runner-guard:END" +# Env vars fetch-runner sets per script. Sudo's --preserve-env= and sudoers +# env_keep both render from this tuple to prevent drift. +PRESERVED_ENVIRONMENT_VARIABLE_NAMES: tuple[str, ...] = ( + "FETCH_RUNNER_JOB", + "FETCH_RUNNER_BRANCH", + "FETCH_RUNNER_COMMIT", + "FETCH_RUNNER_REPO", +) + # Keep the guard as a literal byte template. The validator compares a script's # bytes against the rendered output exactly, so "helpful" rewrites do not # silently weaken the check. @@ -55,6 +64,26 @@ def render_canonical_script_guard(user_name: str) -> str: return _GUARD_TEMPLATE.format(user=user_name) +def render_sudo_argv(run_as_user_name: str, script_path: Path) -> list[str]: + """Build the argv used to execute ``script_path`` as ``run_as_user_name``. + + ``-n`` makes sudo fail immediately if a password is required (no tty). + ``--`` terminates option parsing so a script path starting with ``-`` + cannot be misread as a sudo flag. + """ + _require_safe_user_name(run_as_user_name) + preserve_env_flag = "--preserve-env=" + ",".join(PRESERVED_ENVIRONMENT_VARIABLE_NAMES) + return [ + "sudo", + "-n", + "-u", + run_as_user_name, + preserve_env_flag, + "--", + str(script_path), + ] + + def get_current_real_uid_user_name() -> str: """Return the login name of the real UID. diff --git a/src/fetch_runner/runner.py b/src/fetch_runner/runner.py index 0d0923c..5d1722b 100644 --- a/src/fetch_runner/runner.py +++ b/src/fetch_runner/runner.py @@ -16,6 +16,7 @@ import logging import os +import stat import subprocess import threading @@ -25,6 +26,7 @@ from .git_ops import git_fetch_branch_from_origin from .git_ops import git_force_checkout_branch_to_commit from .git_ops import git_get_local_branch_commit_sha +from .guard import render_sudo_argv from .guard import validate_canonical_script_guard log = logging.getLogger("fetch_runner") @@ -67,6 +69,7 @@ def _initialize_last_processed_commits(self) -> None: initial_commit_sha = git_get_local_branch_commit_sha( configured_job.repo_path, configured_job.branch_name, + run_as_user_name=configured_job.run_as_user, ) except GitError as e: log.warning( @@ -88,6 +91,7 @@ def _poll_job_for_new_commit(self, configured_job: ConfiguredJob) -> None: fetched_commit_sha = git_fetch_branch_from_origin( configured_job.repo_path, configured_job.branch_name, + run_as_user_name=configured_job.run_as_user, ) except GitError as e: log.warning("job %s: fetch failed: %s", configured_job.name, e) @@ -114,23 +118,24 @@ def _poll_job_for_new_commit(self, configured_job: ConfiguredJob) -> None: configured_job.repo_path, configured_job.branch_name, fetched_commit_sha, + run_as_user_name=configured_job.run_as_user, ) except GitError as e: log.error("job %s: checkout failed: %s", configured_job.name, e) return # Re-validate after checkout because the fetched commit controls the - # script bytes on disk. A config-time pass only proves the *previous* - # checkout was safe. - guard_validation = validate_canonical_script_guard( + # script bytes (and mode) on disk. A config-time pass only proves the + # *previous* checkout was safe. + post_checkout_error = _post_checkout_script_problem( configured_job.script_path, - self.runner_config.runtime_user, + configured_job.run_as_user, ) - if not guard_validation.is_valid: + if post_checkout_error is not None: log.error( - "job %s: script at %s failed guard check after checkout: %s", + "job %s: script at %s failed post-checkout validation: %s", configured_job.name, fetched_commit_sha, - guard_validation.error_reason, + post_checkout_error, ) # Record the bad commit so the service does not hammer the same # broken revision forever. Recovery should be an intentional human @@ -154,10 +159,24 @@ def _run_job_script_for_commit( "FETCH_RUNNER_COMMIT": commit_sha, "FETCH_RUNNER_REPO": str(configured_job.repo_path), } - log.info("job %s: running %s", configured_job.name, configured_job.script_path) + # Skip sudo when run_as matches the service user — single-user setups + # then need no sudoers rule at all. + if configured_job.run_as_user == self.runner_config.runtime_user: + script_argv = [str(configured_job.script_path)] + else: + script_argv = render_sudo_argv( + configured_job.run_as_user, + configured_job.script_path, + ) + log.info( + "job %s: running %s as %s", + configured_job.name, + configured_job.script_path, + configured_job.run_as_user, + ) try: completed_process = subprocess.run( - [str(configured_job.script_path)], + script_argv, cwd=configured_job.repo_path, env=script_environment, timeout=configured_job.script_timeout_seconds, @@ -181,3 +200,19 @@ def _run_job_script_for_commit( def _short_commit_sha(commit_sha: str) -> str: return commit_sha[:12] + + +def _post_checkout_script_problem(script_path, run_as_user_name: str) -> str | None: + """Re-run the file-state and guard checks against the freshly checked-out + script. A new commit can change any of these between deploys. + """ + if not script_path.is_file(): + return f"{script_path} does not exist" + if not os.access(script_path, os.X_OK): + return f"{script_path} is not executable" + if script_path.stat().st_mode & stat.S_IWOTH: + return f"{script_path} is world-writable" + guard_validation = validate_canonical_script_guard(script_path, run_as_user_name) + if not guard_validation.is_valid: + return guard_validation.error_reason + return None diff --git a/tests/test_config.py b/tests/test_config.py index 9dde878..f773d87 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -1,6 +1,7 @@ from __future__ import annotations import os +import pwd import stat from pathlib import Path @@ -12,6 +13,24 @@ from fetch_runner.guard import render_canonical_script_guard +def _pick_other_real_user_or_skip() -> str: + """Return a real user name that is *not* the current user, for run_as tests. + + ``pwd.getpwnam`` must succeed for ``run_as`` validation, so we can't fake + one. ``nobody`` is conventional on macOS and Linux but not strictly + required; skip the test if nothing suitable exists. + """ + current_user_name = get_current_real_uid_user_name() + for candidate in ("nobody", "daemon"): + try: + pwd.getpwnam(candidate) + except KeyError: + continue + if candidate != current_user_name: + return candidate + pytest.skip("no second real user available for run_as test") + + @pytest.fixture(autouse=True) def _not_running_as_root(): if os.getuid() == 0: @@ -315,6 +334,111 @@ def test_load_rejects_leading_dash_branch(tmp_path: Path): load_config(config_path) +def test_load_run_as_defaults_to_general_user(tmp_path: Path): + user_name = get_current_real_uid_user_name() + config_path = _write_minimal_valid_jobs_toml(tmp_path, user_name=user_name) + runner_config = load_config(config_path) + assert runner_config.jobs[0].run_as_user == user_name + + +def test_load_accepts_per_job_run_as(tmp_path: Path): + runtime_user_name = get_current_real_uid_user_name() + run_as_user_name = _pick_other_real_user_or_skip() + repo_path = _create_repo_directory(tmp_path / "repo") + # The guard text must match the *run-as* user, not the runtime user. + script_path = _create_guarded_script(tmp_path / "deploy.sh", run_as_user_name) + config_path = _write_jobs_toml( + tmp_path / "jobs.toml", + f""" +[general] +user = "{runtime_user_name}" +poll_interval_seconds = 30 + +[[jobs]] +name = "j1" +path = "{repo_path}" +branch = "main" +script = "{script_path}" +run_as = "{run_as_user_name}" +""", + ) + runner_config = load_config(config_path) + assert runner_config.runtime_user == runtime_user_name + assert runner_config.jobs[0].run_as_user == run_as_user_name + + +def test_load_rejects_run_as_user_not_in_passwd(tmp_path: Path): + runtime_user_name = get_current_real_uid_user_name() + repo_path = _create_repo_directory(tmp_path / "repo") + script_path = _create_guarded_script(tmp_path / "deploy.sh", "ghost-user-xyz-9999") + config_path = _write_jobs_toml( + tmp_path / "jobs.toml", + f""" +[general] +user = "{runtime_user_name}" +poll_interval_seconds = 30 + +[[jobs]] +name = "j1" +path = "{repo_path}" +branch = "main" +script = "{script_path}" +run_as = "ghost-user-xyz-9999" +""", + ) + with pytest.raises(ConfigError, match="does not exist on this system"): + load_config(config_path) + + +def test_load_rejects_unsafe_run_as_value(tmp_path: Path): + runtime_user_name = get_current_real_uid_user_name() + repo_path = _create_repo_directory(tmp_path / "repo") + script_path = _create_guarded_script(tmp_path / "deploy.sh", runtime_user_name) + config_path = _write_jobs_toml( + tmp_path / "jobs.toml", + f""" +[general] +user = "{runtime_user_name}" +poll_interval_seconds = 30 + +[[jobs]] +name = "j1" +path = "{repo_path}" +branch = "main" +script = "{script_path}" +run_as = "evil; rm -rf /" +""", + ) + with pytest.raises(ConfigError, match="run_as"): + load_config(config_path) + + +def test_load_rejects_guard_naming_runtime_user_when_run_as_differs(tmp_path: Path): + # The script's guard must match the run-as user. A guard that still names + # the runtime user should be refused once run_as diverges. + runtime_user_name = get_current_real_uid_user_name() + run_as_user_name = _pick_other_real_user_or_skip() + repo_path = _create_repo_directory(tmp_path / "repo") + script_path = _create_guarded_script(tmp_path / "deploy.sh", runtime_user_name) + config_path = _write_jobs_toml( + tmp_path / "jobs.toml", + f""" +[general] +user = "{runtime_user_name}" +poll_interval_seconds = 30 + +[[jobs]] +name = "j1" +path = "{repo_path}" +branch = "main" +script = "{script_path}" +run_as = "{run_as_user_name}" +""", + ) + with pytest.raises(ConfigError, match="guard"): + load_config(config_path) + + def test_load_rejects_zero_poll_interval(tmp_path: Path): user_name = get_current_real_uid_user_name() repo_path = _create_repo_directory(tmp_path / "repo") diff --git a/tests/test_git_ops.py b/tests/test_git_ops.py new file mode 100644 index 0000000..228fd30 --- /dev/null +++ b/tests/test_git_ops.py @@ -0,0 +1,70 @@ +from __future__ import annotations + +import os +import shutil +from pathlib import Path +from unittest import mock + +import pytest + +from fetch_runner.git_ops import _build_git_argv +from fetch_runner.git_ops import git_fetch_branch_from_origin +from fetch_runner.guard import get_current_real_uid_user_name + + +@pytest.fixture(autouse=True) +def _not_running_as_root(): + if os.getuid() == 0: + pytest.skip("git_ops tests require a non-root test user") + + +def test_build_git_argv_direct_when_run_as_matches_process_user(tmp_path: Path): + current_user_name = get_current_real_uid_user_name() + argv = _build_git_argv(tmp_path, current_user_name, ("rev-parse", "HEAD")) + # No sudo wrapper at all — fetch-runner runs git directly when the + # process user already matches run_as. + assert argv[0] != "sudo" + assert argv[0].endswith("git") + assert argv[1:] == ["-C", str(tmp_path), "rev-parse", "HEAD"] + + +def test_build_git_argv_sudo_wraps_when_run_as_differs(tmp_path: Path): + argv = _build_git_argv(tmp_path, "someone-else", ("fetch", "origin", "main")) + assert argv[0] == "sudo" + assert argv[1] == "-n" + assert argv[2:4] == ["-u", "someone-else"] + assert argv[4] == "--" + assert argv[5].endswith("git") + assert argv[6:] == ["-C", str(tmp_path), "fetch", "origin", "main"] + + +def test_build_git_argv_uses_absolute_git_path(tmp_path: Path): + # Sudoers Commands rules require absolute paths, so the argv must use the + # resolved git binary path, not the bare name. Skip if git isn't on PATH + # (e.g. minimal CI image) — the test has nothing to verify there. + git_absolute_path = shutil.which("git") + if git_absolute_path is None: + pytest.skip("git not on PATH") + argv_same_user = _build_git_argv(tmp_path, get_current_real_uid_user_name(), ("status",)) + assert argv_same_user[0] == git_absolute_path + argv_cross_user = _build_git_argv(tmp_path, "someone-else", ("status",)) + assert argv_cross_user[5] == git_absolute_path + + +def test_git_fetch_passes_through_to_subprocess_with_sudo_when_run_as_differs(tmp_path: Path): + # Drive a real git_ops function to make sure run_as_user_name is plumbed + # all the way through, not just the helper. + with mock.patch("fetch_runner.git_ops.subprocess.run") as mocked_subprocess_run: + mocked_subprocess_run.return_value = mock.Mock(returncode=0, stdout="cafef00d\n", stderr="") + git_fetch_branch_from_origin( + tmp_path, + "main", + run_as_user_name="someone-else", + ) + # Two calls: `git fetch ...` then `git rev-parse FETCH_HEAD`. Both must + # be sudo-wrapped to the same target user. + assert mocked_subprocess_run.call_count == 2 + for invocation_call in mocked_subprocess_run.call_args_list: + invocation_argv = invocation_call.args[0] + assert invocation_argv[0] == "sudo" + assert invocation_argv[2:4] == ["-u", "someone-else"] diff --git a/tests/test_guard.py b/tests/test_guard.py index 1eb0bd6..c1b737d 100644 --- a/tests/test_guard.py +++ b/tests/test_guard.py @@ -6,9 +6,11 @@ import pytest +from fetch_runner.guard import PRESERVED_ENVIRONMENT_VARIABLE_NAMES from fetch_runner.guard import GuardError from fetch_runner.guard import get_current_real_uid_user_name from fetch_runner.guard import render_canonical_script_guard +from fetch_runner.guard import render_sudo_argv from fetch_runner.guard import require_expected_runtime_user from fetch_runner.guard import validate_canonical_script_guard @@ -140,3 +142,27 @@ def test_require_runtime_user_accepts_match(): if os.getuid() == 0: pytest.skip("test suite is running as root") require_expected_runtime_user(get_current_real_uid_user_name()) + + +def test_render_sudo_argv_shape(tmp_path: Path): + script_path = tmp_path / "deploy.sh" + argv = render_sudo_argv("app1", script_path) + # Order matters: -n must come before sudo can prompt, -u must precede the + # target user, --preserve-env must appear before --, and -- must precede + # the script path so a path starting with `-` cannot be misread as a flag. + assert argv[0] == "sudo" + assert argv[1] == "-n" + assert argv[2:4] == ["-u", "app1"] + preserve_env_flag = argv[4] + assert preserve_env_flag.startswith("--preserve-env=") + preserved_names = preserve_env_flag[len("--preserve-env=") :].split(",") + assert tuple(preserved_names) == PRESERVED_ENVIRONMENT_VARIABLE_NAMES + assert argv[5] == "--" + assert argv[6] == str(script_path) + assert len(argv) == 7 + + +@pytest.mark.parametrize("bad", ["", "-evil", "a;b", "user$x"]) +def test_render_sudo_argv_rejects_unsafe_user(bad, tmp_path: Path): + with pytest.raises(GuardError): + render_sudo_argv(bad, tmp_path / "deploy.sh") diff --git a/tests/test_runner.py b/tests/test_runner.py new file mode 100644 index 0000000..f4df295 --- /dev/null +++ b/tests/test_runner.py @@ -0,0 +1,83 @@ +from __future__ import annotations + +import os +from pathlib import Path +from unittest import mock + +import pytest + +from fetch_runner.config import ConfiguredJob +from fetch_runner.config import RunnerConfig +from fetch_runner.guard import get_current_real_uid_user_name +from fetch_runner.runner import GitPollingRunner + + +@pytest.fixture(autouse=True) +def _not_running_as_root(): + if os.getuid() == 0: + pytest.skip("runner tests require a non-root test user") + + +def _make_runner(run_as_user: str, script_path: Path, runtime_user: str | None = None): + runtime_user_name = runtime_user or get_current_real_uid_user_name() + runner_config = RunnerConfig( + runtime_user=runtime_user_name, + poll_interval_seconds=60, + jobs=( + ConfiguredJob( + name="j", + repo_path=script_path.parent, + branch_name="main", + script_path=script_path, + script_timeout_seconds=None, + run_as_user=run_as_user, + ), + ), + ) + return GitPollingRunner(runner_config), runner_config.jobs[0] + + +def test_runner_invokes_script_directly_when_run_as_matches_runtime_user(tmp_path: Path): + # Single-user mode: no sudo, script is exec'd directly so no sudoers + # configuration is required for an unchanged jobs.toml. + script_path = tmp_path / "deploy.sh" + script_path.write_text("#!/bin/sh\n") + script_path.chmod(0o755) + current_user_name = get_current_real_uid_user_name() + runner, job = _make_runner(run_as_user=current_user_name, script_path=script_path) + with mock.patch("fetch_runner.runner.subprocess.run") as mocked_subprocess_run: + mocked_subprocess_run.return_value = mock.Mock(returncode=0) + runner._run_job_script_for_commit(job, "deadbeef" * 5) + invocation_argv = mocked_subprocess_run.call_args.args[0] + assert invocation_argv == [str(script_path)] + + +def test_runner_invokes_script_via_sudo_when_run_as_differs(tmp_path: Path): + script_path = tmp_path / "deploy.sh" + script_path.write_text("#!/bin/sh\n") + script_path.chmod(0o755) + runner, job = _make_runner(run_as_user="someone-else", script_path=script_path) + with mock.patch("fetch_runner.runner.subprocess.run") as mocked_subprocess_run: + mocked_subprocess_run.return_value = mock.Mock(returncode=0) + runner._run_job_script_for_commit(job, "cafef00d" * 5) + invocation_argv = mocked_subprocess_run.call_args.args[0] + assert invocation_argv[0] == "sudo" + assert invocation_argv[1] == "-n" + assert invocation_argv[2:4] == ["-u", "someone-else"] + assert invocation_argv[-2] == "--" + assert invocation_argv[-1] == str(script_path) + + +def test_runner_passes_fetch_runner_env_vars_through_subprocess(tmp_path: Path): + script_path = tmp_path / "deploy.sh" + script_path.write_text("#!/bin/sh\n") + script_path.chmod(0o755) + runner, job = _make_runner(run_as_user="someone-else", script_path=script_path) + with mock.patch("fetch_runner.runner.subprocess.run") as mocked_subprocess_run: + mocked_subprocess_run.return_value = mock.Mock(returncode=0) + runner._run_job_script_for_commit(job, "1234567890ab") + passed_env = mocked_subprocess_run.call_args.kwargs["env"] + assert passed_env["FETCH_RUNNER_JOB"] == "j" + assert passed_env["FETCH_RUNNER_BRANCH"] == "main" + assert passed_env["FETCH_RUNNER_COMMIT"] == "1234567890ab" + assert passed_env["FETCH_RUNNER_REPO"] == str(script_path.parent) diff --git a/tests/test_sudoers.py b/tests/test_sudoers.py new file mode 100644 index 0000000..20c8d24 --- /dev/null +++ b/tests/test_sudoers.py @@ -0,0 +1,95 @@ +from __future__ import annotations + +import shutil +from pathlib import Path + +import pytest + +from fetch_runner.cli import render_sudoers_fragment +from fetch_runner.config import ConfiguredJob +from fetch_runner.config import RunnerConfig + + +def _make_runner_config(*jobs: ConfiguredJob) -> RunnerConfig: + return RunnerConfig( + runtime_user="fetch-runner", + poll_interval_seconds=60, + jobs=tuple(jobs), + ) + + +def _make_job( + name: str, + run_as_user: str, + script_path: str = "/srv/app1/repo/deploy.sh", +) -> ConfiguredJob: + return ConfiguredJob( + name=name, + repo_path=Path(script_path).parent, + branch_name="main", + script_path=Path(script_path), + script_timeout_seconds=None, + run_as_user=run_as_user, + ) + + +def test_render_sudoers_fragment_empty_when_all_jobs_match_runtime_user(): + # No cross-user jobs means no sudoers rules are required at all. + runner_config = _make_runner_config(_make_job("j", run_as_user="fetch-runner")) + rendered_fragment = render_sudoers_fragment(runner_config) + assert "Defaults!" not in rendered_fragment + assert "NOPASSWD" not in rendered_fragment + assert "no sudoers rules needed" in rendered_fragment + + +def test_render_sudoers_fragment_emits_lines_for_cross_user_jobs(): + if shutil.which("git") is None: + pytest.skip("git not on PATH; cannot render git sudoers lines") + git_absolute_path = shutil.which("git") + runner_config = _make_runner_config( + _make_job("api", run_as_user="app1", script_path="/srv/app1/api/deploy.sh"), + _make_job("web", run_as_user="app2", script_path="/srv/app2/web/deploy.sh"), + ) + rendered_fragment = render_sudoers_fragment(runner_config) + assert ( + 'Defaults!/srv/app1/api/deploy.sh env_keep += "FETCH_RUNNER_JOB FETCH_RUNNER_BRANCH ' + 'FETCH_RUNNER_COMMIT FETCH_RUNNER_REPO"' + ) in rendered_fragment + # One git rule per unique run_as user; pinned absolutely to the resolved + # git binary so the rule cannot be sidestepped by a different git on PATH. + assert f"fetch-runner ALL=(app1) NOPASSWD: {git_absolute_path}" in rendered_fragment + assert f"fetch-runner ALL=(app2) NOPASSWD: {git_absolute_path}" in rendered_fragment + assert "fetch-runner ALL=(app1) NOPASSWD: /srv/app1/api/deploy.sh" in rendered_fragment + assert "fetch-runner ALL=(app2) NOPASSWD: /srv/app2/web/deploy.sh" in rendered_fragment + + +def test_render_sudoers_fragment_deduplicates_shared_script_and_runas(): + # Two jobs that share the same (run_as, script) should not produce two + # identical NOPASSWD lines — sudoers would still parse but a diff against + # the running file would be noisy. + if shutil.which("git") is None: + pytest.skip("git not on PATH") + git_absolute_path = shutil.which("git") + runner_config = _make_runner_config( + _make_job("a", run_as_user="app1", script_path="/srv/app1/shared/deploy.sh"), + _make_job("b", run_as_user="app1", script_path="/srv/app1/shared/deploy.sh"), + ) + rendered_fragment = render_sudoers_fragment(runner_config) + assert rendered_fragment.count("Defaults!/srv/app1/shared/deploy.sh") == 1 + assert rendered_fragment.count("NOPASSWD: /srv/app1/shared/deploy.sh") == 1 + # Two jobs share the same run_as, so only one git rule should be emitted. + assert rendered_fragment.count(f"NOPASSWD: {git_absolute_path}") == 1 + + +def test_render_sudoers_fragment_skips_matching_runas_but_keeps_diverging(tmp_path: Path): + runner_config = _make_runner_config( + _make_job( + "same", + run_as_user="fetch-runner", + script_path="/srv/fetch-runner/x/deploy.sh", + ), + _make_job("diff", run_as_user="app1", script_path="/srv/app1/y/deploy.sh"), + ) + rendered_fragment = render_sudoers_fragment(runner_config) + assert "/srv/fetch-runner/x/deploy.sh" not in rendered_fragment + assert "fetch-runner ALL=(app1) NOPASSWD: /srv/app1/y/deploy.sh" in rendered_fragment