Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

88 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

dk distribute GitHub Action

This GitHub Action distributes your dk values and their dependencies.

Follow the Preparation and add the GitHub Workflow file .github/workflows/distribute-0.1.yml (adjust the filename and contents if major/minor version is different):

# file: .github/workflows/distribute-0.1.yml
on:
  push:
    tags:
      - '0.1.*' # secrets are tied to major/minor version
  workflow_dispatch: # allow manual triggering from GitHub page

jobs:
  # Job will run distribution scripts on one or more platforms
  distribute:
    # The secrets created by `./dk0 prepare-version --ci github MAJOR.MINOR`
    # are tied to this environment only.
    environment: dk-distribution

    strategy:
      fail-fast: true
      matrix:
        # the list of platforms to build on.
        include:
          # IS YOUR PACKAGE CROSS-PLATFORM?
          # Yes ...
          - runs-on: ubuntu-latest
            distscript: dist-any.u

          # No .... each of your supported platforms should have its own distribution script
          - runs-on: windows-latest
            distscript: dist-win32.u
          - runs-on: ubuntu-latest
            distscript: dist-linux-glibc.u
            image: quay.io/pypa/manylinux_2_28_x86_64:2026.04.08-5 # glibc 2.28 + gcc 14.2.1: highly compatible glibc; system gcc deprecated once GCC bootstrapped in dk0.
          - runs-on: macos-latest
            distbase: dist-macos.u

    runs-on: ${{ matrix.runs-on }}
    container:
      image: ${{ matrix.image }}
    steps:
      - name: Harden Runner # Optional but recommended
        uses: step-security/harden-runner@f808768d1510423e83855289c910610ca9b43176 # v2.17.0
        with: { egress-policy: audit }
      - name: Checkout repository
        uses: actions/checkout@v6

      - name: Distribute Modules
        uses: diskuv/dk-distribute@v2
        with:
            pubkey: ${{ secrets.distribute_1_0_pubkey }} # change based on MAJOR.MINOR
            seckey: ${{ secrets.distribute_1_0_seckey }} # change based on MAJOR.MINOR
            short-build-dir: ${{ runner.os == 'Windows' && 'C:\b' || '' }}
            use-cache: true
            distscript: ${{ matrix.distscript }}

  # Job will combine artifacts from multiple platforms and release as a single distribution
  combine:
    needs: distribute
    permissions:
      contents: write # for action-gh-release
      id-token: write # for actions/attest-build-provenance
      attestations: write # for actions/attest-build-provenance
      artifact-metadata: write # for actions/attest-build-provenance
    runs-on: ubuntu-latest
    steps:
      - name: Harden Runner # Optional but recommended
        uses: step-security/harden-runner@f808768d1510423e83855289c910610ca9b43176 # v2.17.0
        with: { egress-policy: audit }
      - name: Checkout repository
        uses: actions/checkout@v6

      - name: Combine Modules
        uses: diskuv/dk-distribute/combine@v2
        with:
            pubkey: ${{ secrets.distribute_1_0_pubkey }} # change based on MAJOR.MINOR
            seckey: ${{ secrets.distribute_1_0_seckey }} # change based on MAJOR.MINOR
            experimental-dk-owner: ${{ github.repository_owner == 'dkpkg' && 'diskuv' || github.repository_owner }}
            experimental-mlfront-ref: ${{ github.ref_type != 'tag' && 'HEAD' }}

      - name: Attest and Release
        uses: diskuv/dk-distribute/attest-release@v3

Be sure to review the following places carefully:

  • the filename should match the MAJOR.MINOR version you prepared
  • the jobs / 'distribute' / strategy / matrix / include
  • the jobs / 'distribute' / steps / 'Distribute Modules' / pubkey
  • the jobs / 'distribute' / steps / 'Distribute Modules' / seckey
  • the jobs / 'combine' / steps / 'Combine Modules' / pubkey and seckey (they let combine sign the distribution build payload)
  • if your workspace imports packages other than CommonsBase_Std (the built-in dk trust root) and your own package, list them in the jobs / 'distribute' / steps / 'Distribute Modules' / trust-packages input (ex. trust-packages: 'CommonsBase_GNU CommonsBase_Win32'); dk0 denies unknown producer keys by default and CI has no terminal to accept them

Attesting and releasing

attest-release performs the three steps that have to run in this order:

  1. attest every file of the distribution with actions/attest-build-provenance,
  2. publish the sigstore bundle for values.json as a release asset named values.json.sigstore.jsonl, and
  3. create the GitHub release.

Step 2 is why these are one action rather than three workflow steps: the bundle can only be downloaded once step 1 has produced it, so it cannot live in combine or the root action, both of which finish before anything is attested.

Publishing the bundle is what lets a consumer import the release without a GitHub API token:

$ dk0 add github-l2 dkpkg/YourPackage@1.0.0   # no GH_TOKEN needed

A consumer fetches values.json.sigstore.jsonl beside values.json with a plain download and verifies it exactly as it verifies a bundle from the API, so moving the transport relaxes no check. A release published before this action existed carries no such asset, and consumers fall back to the GitHub API for it.

The job needs the same permissions the individual steps needed:

    permissions:
      contents: write         # create the release
      id-token: write         # attest
      attestations: write     # attest, and read back the bundle
      artifact-metadata: write

Use attest: false to create a release without attesting, and prerelease: to override the default, which publishes a partial (dk0 restore-seedable) distribution as a prerelease.

Now, when you push a git tag, the GitHub Actions will create the following directories in your project directory:

Directory Contents
dksrc/ The dk build system
t/ Cache, data and key directories

and build the dk values from your distribution script (distscript).

Preparation

The distribution keys and files will be prepared for you when you run the command prepare-version --ci github MAJOR.MINOR.

For example, if this is your first major and minor version, open PowerShell or a UNIX shell and type:

./dk0 -- prepare-version --ci github 0.1

Cache value and trace stores

Caching reduces build and download time. Caching is disabled by default since pristine builds are recommended when distributing builds to the public.

You can opt in to caching by passing 'true' to the use-cache input:

- name: Distribute Modules
  uses: diskuv/dk-distribute@v2
  with:
    use-cache: 'true'

Windows short build directory

Large Windows distributions can hit the default 260-character MAX_PATH limit under the standard GitHub Actions checkout path. Set short-build-dir to an absolute path like C:\b to have the action delete and recreate that directory, create a workspace junction at C:\b\w that points at the original checkout, change the current directory to C:\b\w, and run dk with --keys-dir C:\b\k --data-dir C:\b\d --cache-dir C:\b\c.

The junction is needed because shortening only the dk data/cache/key directories does not shorten workspace-relative paths like etc\dk\i, dist-any.u, and dk-dist. Running from C:\b\w shortens both the workspace path and the dk store paths.

The action removes short-build-dir before recreating it, so a leftover C:\b\w junction from an earlier run on the same machine is deleted first. On GitHub-hosted runners each job normally gets a fresh machine, so another GitHub job should not leave that behind. On self-hosted or otherwise shared runners, use a job-unique short directory if concurrent jobs could collide.

- name: Distribute Modules
  uses: diskuv/dk-distribute@v2
  with:
    short-build-dir: ${{ runner.os == 'Windows' && 'C:\b' || '' }}
    distscript: ${{ matrix.distscript }}

Publish flat object assets

A release normally ships only the content-addressed value store (imported with dk0 add github-l2 <owner>/<repo>). If you also want to publish a built object as a plain, whole-file asset -- for example the runnable bin/dk0.exe so it can be downloaded directly with curl -- list it in the publish-objects input.

Each line is <object@version>|<zip-member>|<asset-basename>. After distribute, the action runs dk0 get-object for each line, selecting the slot from this job's distribution script (Release.<distscript-basename>), and stages the extracted member into dk-dist/ as <asset-basename>-<abi>[.exe] (<abi> is the lowercased distscript basename; .exe is appended only for Windows_* slots). Because it lands in dk-dist/, it is shared, combined, attested (attest-build-provenance), and released (action-gh-release) exactly like every other distribution asset -- no extra artifact wiring in your workflow.

- name: Distribute Modules
  uses: diskuv/dk-distribute@v3
  with:
    distscript: ${{ matrix.distscript }}   # ex. dist/Windows_x86_64.u -> slot Release.Windows_x86_64
    publish-objects: |
      CommonsBase_Dk.Dk0@2.4.2|./bin/dk0.exe|dk0
      CommonsBase_Dk.Dk1@2.4.2|./bin/dk1.exe|dk1

For the matrix above this produces, per slot, dk-dist/dk0-<abi>[.exe] and dk-dist/dk1-<abi>[.exe] (ex. dk-dist/dk0-windows_x86_64.exe, dk-dist/dk0-linux_x86_64). The input is a no-op when empty or when standalone is 'true'.

Experimental Features

Reference Build System

Normally the Action uses the latest release of the dksrc/dk0 reference build system.

However, the following will build the reference build system with the specific git reference:

- name: dk distribute
  uses: diskuv/dk-distribute@v2
  with:
    experimental-mlfront-ref: HEAD

Building the reference system takes time. And the reference system builds slower than dk.

Do not use for production. This is meant only for troubleshooting issues or in high-compliance situations.

Do not mix experimental-mlfront-ref with non-experimental-mlfront-ref in the same workflow. The version checks in the combine job will fail.

About

A GitHub Action to distribute your modules.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Contributors