End-to-end recipes that show pipekit replacing real pipeline bash. Each example is copy-pasteable into the relevant CI system.
For per-command flag reference see COMMANDS.md.
Direct bash → pipekit swaps. Each one removes a pile of jq / sed / awk / curl boilerplate.
JSON config → env vars
# BEFORE
for key in $(jq -r 'keys[]' config.json); do
value=$(jq -r ".[\"$key\"]" config.json)
echo "${key^^}=$value" >> "$GITHUB_ENV"
done
# AFTER
pipekit env from-json config.json --uppercase-keys --to-githubWait for service to be ready
# BEFORE
for i in {1..30}; do
curl -sf http://localhost:8080/healthz && break
sleep 5
done
# AFTER
pipekit wait url http://localhost:8080/healthz --timeout 150sRetry a flaky command
# BEFORE
n=0; until [ $n -ge 5 ]; do
helm upgrade --install myapp ./chart && break
n=$((n+1)); sleep $((5 * 2 ** n))
done
# AFTER
pipekit retry run --attempts 5 --delay 5s --backoff -- helm upgrade --install myapp ./chartMap branch → environment
# BEFORE
case "$GITHUB_REF" in
refs/heads/main) echo "TARGET_ENV=production" >> "$GITHUB_ENV" ;;
refs/heads/develop) echo "TARGET_ENV=dev" >> "$GITHUB_ENV" ;;
refs/heads/release/*) echo "TARGET_ENV=staging" >> "$GITHUB_ENV" ;;
*) echo "TARGET_ENV=preview" >> "$GITHUB_ENV" ;;
esac
# AFTER
pipekit config branch-env --to-github \
--mapping '{"main":"production","develop":"dev","release/*":"staging"}'Generate a deterministic cache key
# BEFORE
KEY="go-mod-$(uname -s)-$(uname -m)-$(sha256sum go.sum | cut -d' ' -f1)"
echo "cache_key=$KEY" >> "$GITHUB_OUTPUT"
# AFTER
pipekit cache-key composite "$(uname -s)" "$(uname -m)" \
"$(pipekit transform hash --file go.sum)" \
--prefix "go-mod-" --to-github-output cache_keyMask a value in logs
# BEFORE
echo "::add-mask::$SECRET_VALUE"
# AFTER
pipekit mask github "$SECRET_VALUE"
# or, mask many at once
pipekit mask env --env-match "*_TOKEN,*_SECRET,*_KEY" --githubReads an env-specific config (resolves prod → production automatically), masks the resulting secrets, asserts the deploy token is present, then deploys.
name: Deploy
on:
workflow_dispatch:
inputs:
env:
type: choice
options: [dev, staging, prod]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install pipekit
run: |
curl -L https://github.com/AxeForging/pipekit/releases/latest/download/pipekit-linux-amd64.tar.gz | tar xz
sudo mv pipekit-linux-amd64 /usr/local/bin/pipekit
- name: Resolve env config
run: pipekit config resolve envs.json --env "${{ inputs.env }}" --uppercase-keys --to-github
- name: Mask sensitive values in logs
run: pipekit mask env --env-match "*_TOKEN,*_SECRET,*_KEY" --github
- name: Guard required vars
run: pipekit assert env-exists DEPLOY_TOKEN CLUSTER_NAME IMAGE_TAG
- name: Deploy
run: pipekit retry run --attempts 3 --delay 30s --backoff -- ./scripts/deploy.sh
- name: Notify Slack
if: always()
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
run: |
pipekit notify slack --status "${{ job.status }}" \
--title "Deploy to ${{ inputs.env }}" \
--field "image=$IMAGE_TAG" \
--field "actor=${{ github.actor }}"Build only the services whose paths changed in this PR.
name: CI
on: [pull_request]
jobs:
detect:
runs-on: ubuntu-latest
outputs:
services: ${{ steps.diff.outputs.services }}
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- run: |
curl -L https://github.com/AxeForging/pipekit/releases/latest/download/pipekit-linux-amd64.tar.gz | tar xz
sudo mv pipekit-linux-amd64 /usr/local/bin/pipekit
- id: diff
run: |
pipekit diff affected --config .pipekit-diff.yaml \
--base origin/${{ github.base_ref }} \
--output json --to-github-output services
build:
needs: detect
if: needs.detect.outputs.services != '[]'
runs-on: ubuntu-latest
strategy:
matrix:
service: ${{ fromJSON(needs.detect.outputs.services) }}
steps:
- uses: actions/checkout@v4
- run: ./build.sh ${{ matrix.service }}.pipekit-diff.yaml:
services:
api: [api/, shared/]
web: [web/, shared/]
worker: [worker/]Generate the matrix from the contents of services/:
jobs:
generate:
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.gen.outputs.matrix }}
steps:
- uses: actions/checkout@v4
- id: gen
run: pipekit matrix from-dirs services/ --key service --to-github-output matrix
build:
needs: generate
runs-on: ubuntu-latest
strategy:
matrix: ${{ fromJSON(needs.generate.outputs.matrix) }}
steps:
- run: echo "Building ${{ matrix.service }}"A /deploy comment on an issue includes a YAML block. The workflow parses it, exposes the values as env vars, and acts.
on:
issue_comment:
types: [created]
jobs:
deploy:
if: contains(github.event.comment.body, '/deploy')
runs-on: ubuntu-latest
steps:
- run: |
echo "${{ github.event.comment.body }}" \
| pipekit parse extract-yaml --to-github -u
- run: pipekit assert env-exists ENV REPLICAS IMAGE
- run: ./deploy.shA comment like:
/deploy
```yaml
env: production
replicas: 3
image: ghcr.io/me/app:v1.2.3
```
…sets ENV=production, REPLICAS=3, IMAGE=ghcr.io/me/app:v1.2.3.
Render a readable PR comment with a hidden anchor, then let gh create or update the matching comment.
- name: Build preview body
run: |
{
echo "## Preview deployment"
echo
echo "URL: https://preview.example.com"
echo
pipekit comment fence --language yaml preview.yaml
} > preview-body.md
pipekit comment render --anchor preview-deploy --body-file preview-body.md > preview-comment.md
- name: Upsert PR comment
env:
GH_TOKEN: ${{ github.token }}
run: |
gh api repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments \
> comments.json
if pipekit comment select comments.json --anchor preview-deploy --format id > comment-id.txt; then
pipekit comment payload preview-comment.md > payload.json
gh api \
--method PATCH \
repos/${{ github.repository }}/issues/comments/$(cat comment-id.txt) \
--input payload.json
else
gh pr comment ${{ github.event.pull_request.number }} --body-file preview-comment.md
fiGenerate a clean, k8s-friendly slug for the preview environment per branch.
- id: slug
run: |
pipekit transform slug \
--prefix "preview-" --max-length 40 \
"${{ github.head_ref }}" >> slug.txt
echo "slug=$(cat slug.txt)" >> "$GITHUB_OUTPUT"
- run: kubectl apply -f overlays/${{ steps.slug.outputs.slug }}/Make the run-summary actually useful.
pipekit summary badge --label "Build" --status success --to-github-summary
pipekit summary table --title "Deployed services" deployed.json --to-github-summary
cat build.log | pipekit summary section --title "Build logs" --to-github-summaryvariables:
PIPEKIT_VERSION: latest
before_script:
- curl -L https://github.com/AxeForging/pipekit/releases/$PIPEKIT_VERSION/download/pipekit-linux-amd64.tar.gz | tar xz
- mv pipekit-linux-amd64 /usr/local/bin/pipekit
deploy:
script:
# Resolve config and source it into the current shell
- pipekit config resolve envs.yaml --env "$CI_ENVIRONMENT_NAME" --format yaml --to-gitlab > env.sh
- source env.sh
- pipekit assert env-exists DEPLOY_TOKEN PROJECT_ID
- pipekit wait url "$HEALTH_URL" --timeout 60s
- pipekit retry run --attempts 3 --delay 10s -- ./deploy.shpipeline {
agent any
stages {
stage('Setup') {
steps {
sh '''
curl -L https://github.com/AxeForging/pipekit/releases/latest/download/pipekit-linux-amd64.tar.gz | tar xz
sudo mv pipekit-linux-amd64 /usr/local/bin/pipekit
'''
}
}
stage('Deploy') {
steps {
sh 'pipekit assert env-exists DEPLOY_TOKEN'
sh 'pipekit wait url http://localhost:8080/healthz --timeout 120s'
sh 'pipekit retry run --attempts 3 --delay 15s -- ./deploy.sh'
}
}
stage('Notify') {
steps {
sh '''
pipekit notify slack \
--status "${currentBuild.currentResult}" \
--title "Build #${BUILD_NUMBER} on ${BRANCH_NAME}"
'''
}
}
}
}Recipes built on the new commands.
Helm-style values overlay → render → kubectl apply
# Stack values: base + per-env + inline overrides
pipekit render manifests/deployment.yaml.tpl \
--values values/base.yaml \
--values values/prod.yaml \
--set image.tag=$(pipekit time now --format tag) \
--output /tmp/deployment.yaml
kubectl apply -f /tmp/deployment.yamlPull config from helm chart, mutate, save
# Bump only the .image.tag in values.yaml without touching anything else
# (--preserve keeps comments, key order, and quoting exactly as-is)
pipekit yaml set chart/values.yaml --path '.image.tag' --value 'v1.2.3' --in-place --preserve
# Or: deep-merge a per-env overlay
pipekit yaml merge chart/values.yaml chart/values.prod.yaml --output /tmp/merged.yamlRender a step-summary table from a JSON manifest
pipekit json table services.json --columns name,version,status \
| pipekit summary section --title "Deployed services" --to-github-summaryWrap a flaky deploy in retry + mask + tee in one verb
pipekit exec \
--attempts 5 --backoff --jitter \
--timeout 5m --max-elapsed 20m \
--mask-preset github,aws \
--tee deploy.log \
--retry-on-stderr "(rate limit|connection reset|EOF)" \
-- ./scripts/deploy.shDatabase URL → connection env vars
pipekit url parse "$DATABASE_URL" --prefix DB_ --to-github
# Now DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, DB_PATH (etc) are set
pipekit assert env-exists DB_HOST DB_USER DB_PASSWORDTest sharding with matrix shard
# .github/workflows/test.yml
jobs:
test:
strategy:
matrix:
shard: [0, 1, 2, 3]
steps:
- run: |
go test -list . ./... | tail -n +2 > tests.txt
pipekit matrix shard --total 4 --index ${{ matrix.shard }} \
--from-stdin-lines < tests.txt > my-shard.txt
xargs -a my-shard.txt go test -runThe real value comes from chaining a few commands together. A few patterns:
Resolve config → mask secrets → assert → deploy → notify
pipekit config resolve envs.json --env prod --uppercase-keys --to-github
pipekit mask env --env-match "*_TOKEN,*_SECRET" --github
pipekit assert env-exists DEPLOY_TOKEN CLUSTER_NAME
pipekit retry run --attempts 3 --delay 30s -- ./deploy.sh
pipekit notify slack --status success --title "Deploy succeeded"Compute version → tag → build cache key → publish
NEXT=$(pipekit version next --source package.json)
pipekit assert semver "$NEXT"
KEY=$(pipekit cache-key composite "$(uname -s)" "$NEXT" "$(pipekit transform hash --file package-lock.json)")
echo "Building $NEXT with cache key $KEY"Build a matrix from changed dirs only
pipekit diff dirs --base origin/main \
| jq -R -s 'split("\n") | map(select(length > 0))' \
| pipekit matrix from-json --key service \
| tee matrix.jsonRelease artifacts → checksums → manifest → notes
make build-all
pipekit artifact assert "dist/pipekit-*"
pipekit checksum files dist/pipekit-* --output dist/checksums.txt
pipekit checksum verify dist/checksums.txt
pipekit artifact manifest "dist/pipekit-*" "dist/checksums.txt" \
--pretty --output dist/artifacts.json
pipekit git sha --short --to-github-output git_sha
pipekit git ref --slug --to-github-output ref_slug
pipekit changelog since-tag --conventional --output RELEASE_NOTES.mdSee also: Commands · Requirements · Install · ← README