-
Notifications
You must be signed in to change notification settings - Fork 0
130 lines (115 loc) · 5.22 KB
/
Copy pathexport.yml
File metadata and controls
130 lines (115 loc) · 5.22 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
name: export
# Manual only — this pulls in PyTorch and ONNX Runtime and runs for tens of minutes, which is the
# whole reason summo-models is a separate repo from summo-app (see README.md). Nothing about it
# should run on every push.
on:
workflow_dispatch:
inputs:
model:
description: >
Model id — must match a directory under models/ that has an export.py, and a manifest at
models/<model>.json in summo-registry.
required: true
type: string
version:
description: "Release version for this export, e.g. v2. Never reuse one."
required: true
type: string
keep_fp32:
description: Also upload the unquantised intermediate (export.py --keep-fp32)
required: false
type: boolean
default: false
permissions:
contents: write # to create the release and upload assets
jobs:
export:
runs-on: ubuntu-latest
steps:
- name: Checkout summo-models
uses: actions/checkout@v4
- name: Checkout summo-registry (to read this model's manifest)
uses: actions/checkout@v4
with:
repository: Techainer/summo-registry
path: registry
# This is the check that matters in this workflow. redistributable: false means the model's
# licence does not permit us to mirror it into our own Releases — sense-voice-small and both
# MiLMMT sizes, today. Publishing them here anyway would make Summo the distributor of
# something it has no right to distribute, so this is a hard stop, not a warning.
- name: Refuse to run for a non-redistributable model
run: |
python3 - <<'PY'
import json
import sys
from pathlib import Path
manifest_path = Path("registry/models/${{ inputs.model }}.json")
if not manifest_path.exists():
sys.exit(
f"no manifest at {manifest_path} — is '${{ inputs.model }}' a real model id "
"in summo-registry?"
)
manifest = json.loads(manifest_path.read_text())
if manifest.get("redistributable") is False:
sys.exit(
f"{manifest['id']} is marked redistributable: false in summo-registry "
f"(licence: {manifest.get('license', 'unknown')}). Its terms do not permit "
"mirroring the weights into this repo's Releases. Refusing to export."
)
print(f"{manifest['id']} is redistributable — continuing.")
PY
- name: Check that this model has an export script
run: |
if [ ! -f "models/${{ inputs.model }}/export.py" ]; then
echo "models/${{ inputs.model }}/export.py does not exist."
echo "Only models Summo exports itself have one — see README.md's redistributable/not table."
echo "A model that is only mirrored is downloaded and re-uploaded as-is, which this workflow does not do."
exit 1
fi
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install export dependencies
working-directory: models/${{ inputs.model }}
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Run export.py
working-directory: models/${{ inputs.model }}
run: |
python export.py --out build/ ${{ inputs.keep_fp32 && '--keep-fp32' || '' }}
# `<id>-vN`, one release per model, not one branch per model.
#
# A branch is for a change in flight; a release is for an artefact. Branch-per-model would
# mean twenty branches that never merge and never die, each drifting from the export scripts
# on `master`, and a bug fixed in one export never reaching the other nineteen. Tags cost
# nothing and are what a release is addressed by: `small100-v2` is a name a manifest URL can
# contain forever.
#
# The version is bumped by hand — `v1`, `v2` — because a re-export is a decision somebody
# made, not a date that passed. Two exports on the same day are two versions; a date-based tag
# would collide and quietly overwrite an artefact a published manifest already points at.
- name: Check the release tag is free
id: tag
run: |
tag="${{ inputs.model }}-${{ inputs.version }}"
if gh release view "$tag" >/dev/null 2>&1; then
echo "::error::release $tag already exists — bump the version rather than replacing an" \
"artefact that a published manifest may already point at"
exit 1
fi
echo "tag=$tag" >> "$GITHUB_OUTPUT"
env:
GH_TOKEN: ${{ github.token }}
- name: Upload artefacts to a release
uses: softprops/action-gh-release@v2
with:
tag_name: ${{ steps.tag.outputs.tag }}
name: ${{ inputs.model }} (${{ steps.tag.outputs.tag }})
draft: true
files: models/${{ inputs.model }}/build/*
- name: Print the manifest snippet
run: |
python scripts/manifest.py "models/${{ inputs.model }}/build" \
--id "${{ inputs.model }}" \
--release-tag "${{ steps.tag.outputs.tag }}"