Skip to content

build-in-container.md

Latest commit

 

History

History
273 lines (210 loc) · 12.9 KB

build-in-container.md

File metadata and controls

273 lines (210 loc) · 12.9 KB
Raw

build-in-container

Build CFEngine packages inside Docker containers using build scripts. Requires only Docker and Python 3 on the host.

Quick start

# Build a community agent .deb for Ubuntu 22
./build-in-container.py --platform ubuntu-22 --project community --role agent --build-type DEBUG

# Build a nova hub release package for Debian 12
./build-in-container.py --platform debian-12 --project nova --role hub --build-type RELEASE

In the examples above, we run the script from inside buildscripts/ (with buildscripts as our current working directory). This is not required — if not specified, defaults will:

  • Look for sources relative to the script (parent directory of build-in-container.py).
  • Place cache files in the user's home directory (~/.cache/cfengine/buildscripts).
  • Use the current working directory for output packages (./output/).

Usage

./build-in-container.py --platform PLATFORM --project PROJECT --role ROLE --build-type TYPE [OPTIONS]

Required arguments

Option Description
--platform Target platform (e.g. ubuntu-22, debian-12)
--project community or nova (not required for --push-image)
--role agent or hub (not required for --push-image)
--build-type DEBUG or RELEASE (not required for --push-image)

None of the above arguments are required for --update.

Optional arguments

Option Default Description
--output-dir ./output Where to write output packages
--cache-dir ~/.cache/cfengine/buildscripts Dependency cache directory
--build-number 1 Build number for package versioning
--version auto Override version string
--rebuild-image Force rebuild of Docker image (bypasses Docker layer cache)
--push-image Build image and push to registry, then exit
--update Fetch latest image versions from registry and update platforms.json
--update-sha Fetch latest base image manifest digests from Docker Hub and update platforms.json
--shell Drop into a bash shell inside the container for debugging
--list-platforms List available platforms and exit
--source-dir parent of buildscripts/ Root directory containing repos
--arch host architecture Override the container architecture (see Architecture)

Supported platforms

Name Base image
ubuntu-20 ubuntu:20.04
ubuntu-22 ubuntu:22.04
ubuntu-24 ubuntu:24.04
debian-11 debian:11
debian-12 debian:12
debian-13 debian:13
rhel-8 rockylinux/rockylinux:8
rhel-9 rockylinux/rockylinux:9
rhel-10 rockylinux/rockylinux:10

RHEL packages are built on Rocky Linux base images. The build scripts detect OS=rhel from /etc/redhat-release (which reports Rocky Linux release ...), so the produced .rpms are ordinary Red Hat / rpm packages. AlmaLinux is not recognized by build-scripts/detect-environment, which is why Rocky is used.

Adding a new Debian/Ubuntu platform requires a new entry in platforms.json and adding the platform name to the matrix in .github/workflows/build-base-images.yml so the weekly job builds and pushes its image to ghcr.io. Without the matrix entry, no image is ever pushed and the update-base-images.yml workflow will fail with a 403 from ghcr.io when it queries tags for the missing repository.

The new entry in platforms.json needs:

  • image_version: set to "latest" as a placeholder. The update-base-images.yml workflow (or ./build-in-container.py --update run locally) will replace it with the real ghcr.io tag after the first image is pushed.
  • base_image_sha: the Docker Hub manifest digest for the base_image. Don't copy this by hand — run ./build-in-container.py --update-sha --platform <new-platform> and it will fetch the current digest from Docker Hub and write it into platforms.json.
  • architectures (optional): the list of docker platforms to publish, e.g. ["linux/amd64", "linux/arm64"]. Omit it to get the multi-arch default; set it only to restrict a platform to specific architectures (see Architecture).

Adding another RHEL-family platform (a new Rocky/RHEL major version) works the same way: add a platforms.json entry with "dockerfile": "Dockerfile.rhel" and a matrix entry, then set any per-version extra_build_args — CRB_REPO (powertools on 8, crb on 9+), PHP_MODULE_STREAM (remi-8.3 where the distro's default PHP is older than 8.3; RHEL 10 already ships 8.3), and EXTRA_PKGS for version-specific packages. Note that --update-sha also works for the namespaced rockylinux/rockylinux base images, not just official Docker Hub library images.

Adding an entirely different, non-RHEL/non-Debian platform family (e.g. SUSE) would require a new container/Dockerfile.<family> plus platform entries.

Architecture

By default the build runs on the host machine's architecture, and Docker picks the matching image variant automatically. Use --arch to override this and build for another architecture - the value is passed straight to Docker's --platform flag:

# Build an arm64 community agent .deb for Ubuntu 24 on an amd64 host
./build-in-container.py --platform ubuntu-24 --project community --role agent \
    --build-type DEBUG --arch linux/arm64

The registry images are published as multi-arch manifests (linux/amd64 and linux/arm64), so --arch normally just pulls the matching variant. If the registry does not provide the requested architecture (for example an older, single-arch image that predates multi-arch support), the script falls back to building the image locally for that architecture.

Building a non-host architecture - whether locally or in CI - relies on QEMU/binfmt emulation being registered on the build host. If it isn't set up, register it once with:

docker run --privileged --rm tonistiigi/binfmt --install all

Emulated builds are considerably slower than native ones.

The set of architectures published for each platform defaults to linux/amd64 and linux/arm64. A platform can override this with an "architectures" list in platforms.json. The ubuntu-24-mingw platform, for instance, cross-compiles to Windows x64 regardless of the container's architecture, so it is pinned to ["linux/amd64"].

How it works

The system has three components:

  1. build-in-container.py (Python) -- the orchestrator that runs on the host. Parses arguments, builds the Docker image, and launches the container with the correct mounts and environment variables.

  2. build-in-container-inner.sh (Bash) -- runs inside the container. Copies source repos from the read-only mount, then calls the existing build scripts in order.

  3. container/Dockerfile.debian and container/Dockerfile.rhel -- parameterized Dockerfiles shared across platforms of the same family via a BASE_IMAGE build arg (plus per-platform extra_build_args in platforms.json, e.g. the CRB repo name and PHP module stream for RHEL).

Container mounts

Host path Container path Mode Purpose
Source repos (parent of buildscripts/) /srv/source read-only Protects host repos from modification
~/.cache/cfengine/buildscripts/ /home/builder/.cache/buildscripts_cache read-write Dependency cache shared across builds
./output/ /output read-write Output packages copied here

Build steps

The inner script runs these steps in order:

  1. autogen -- runs autogen.sh in each repo
  2. install-dependencies -- builds and installs bundled dependencies
  3. mission-portal-deps -- (hub only) installs PHP/npm/LESS assets
  4. configure -- runs ./configure with platform-appropriate flags
  5. compile -- compiles and installs to the dist tree
  6. package -- creates .deb or .rpm packages

Docker image management

By default, the script pulls a pre-built image from the container registry (ghcr.io/cfengine). If the pull fails (e.g. no network, image not yet published), it falls back to building the image locally.

Use --rebuild-image to skip the registry and force a local rebuild — useful when iterating on the Dockerfile. The local build tracks the Dockerfile content hash and skips rebuilding when nothing has changed.

Container registry

Images are hosted at ghcr.io/cfengine and versioned per-platform via image_version in platforms.json. To push a new image:

# Build and push a single platform
./build-in-container.py --platform ubuntu-22 --push-image

--push-image uses docker buildx build --push to build every architecture the platform targets (linux/amd64 and linux/arm64 by default; see Architecture) and publish them under a single multi-arch manifest. It always builds fresh to pick up the latest upstream packages.

GitHub Actions workflow

The build-base-images.yml workflow builds and pushes images for every supported platform. It runs weekly (Sunday at midnight UTC) and can also be triggered manually via workflow_dispatch.

After the workflow pushes new images, update platforms.json to use them:

# Update all platforms to the latest registry version
./build-in-container.py --update

# Update a single platform
./build-in-container.py --update --platform ubuntu-22

The update-base-images.yml workflow automates this step. It runs weekly (Monday at midnight UTC) and can also be triggered manually. It calls ./build-in-container.py --update and opens a pull request with any platforms.json changes. This workflow requires contents: write and pull-requests: write permissions.

The base_image_sha digests in platforms.json pin each platform to a specific Docker Hub manifest. To refresh them to the current digests:

# Update all platforms
./build-in-container.py --update-sha

# Update a single platform
./build-in-container.py --update-sha --platform ubuntu-22

The update-base-image-shas.yml workflow automates this. It runs weekly (Monday at 01:00 UTC) and opens a pull request with any digest changes.

The workflow authenticates to ghcr.io using the automatic GITHUB_TOKEN provided by GitHub Actions. For this to work:

  • The repository must grant GITHUB_TOKEN write access to packages. In the GitHub repository settings, go to Actions → General → Workflow permissions and select Read and write permissions.
  • After the first push, each package defaults to private. To allow anonymous pulls, go to the package on GitHub (your org → Packages), open Package settings, and change the visibility to Public. This is a one-time step per package — new tags (e.g. from bumping image_version) inherit the existing visibility.

Updating the toolchain

  1. Edit container/Dockerfile.debian as needed
  2. Test locally with --rebuild-image
  3. Commit and merge the Dockerfile change
  4. Push new images by triggering the build-base-images.yml workflow
  5. Trigger the update-base-images.yml workflow to open a PR updating platforms.json

Debugging

# Drop into a shell inside the container
./build-in-container.py --platform ubuntu-22 --project community --role agent --build-type DEBUG --shell

The shell session has the same mounts and environment as a build run. The container is ephemeral (--rm), so any changes are lost on exit.