ci/nightly: manifest aggregator + 90-build retention sweep (#92)
PR-Bld-B in the mirror of OpenIPC/firmware's nightly redesign. Adds
the two workflows that turn dated nightly-* releases (from PR-Bld-A)
into a queryable index served via GitHub Pages.
`manifest.yml` — triggers on successful `Build` workflow_run completion
(or manual dispatch). Runs `.github/scripts/enrich_manifest.py` to
enumerate dated releases, keep the 90 newest, parse the body
(sha/short/built_at from PR-Bld-A) and asset list, then emit:
- `manifest.json` — rich JSON for hosts/agents/CI.
- `manifest.flat` — whitespace-delimited columns
`build_id platform flash size url` plus `@channel` records,
parseable by pure busybox `awk` (no jq, no jsonfilter) for
on-device sysupgrade in Phase 2.
Both committed to the `gh-pages` branch (already bootstrapped).
`cleanup.yml` — Mondays 06:00 UTC (offset by +1h from firmware's
05:00 UTC cleanup). Deletes dated nightly releases beyond the 90
newest via `gh release delete --cleanup-tag`, then re-triggers
manifest.yml.
`enrich_manifest.py` differs from firmware's only in the asset parser:
builder produces TWO filename forms (per `master.yml`'s COMMON awk
trick that counts underscores in the matrix entry name):
- **Compound** `<soc>_<variant>_<vendor>-<model>-nor.tgz` →
platform key = the full string. This is the common case
(per-device builds).
- **Simple** `openipc.<soc>-<flash>-<variant>.tgz` (firmware-style,
when matrix name has one underscore) → platform key =
`<soc>_<variant>`.
The platform key gets carried into manifest.flat as a single column.
sysupgrade's existing `awk '$2==p'` lookup works unchanged; Phase 3
(model-aware BUILD_PLATFORM field in os-release) will let cameras
filter against the full per-model key.
URLs after merge:
https://openipc.github.io/builder/manifest.json
https://openipc.github.io/builder/manifest.flat
(`gh-pages` branch + Pages config already set up; mirrors firmware-
side setup.)
Concurrency-grouped `gh-pages-manifest` so manifest.yml and
cleanup.yml can't race the gh-pages push.
Includes the retry hardening from firmware OpenIPC/firmware#2129
(4-attempt backoff for transient GitHub API 401/5xx, fast-fail on
permanent 404).
First-run behaviour validated locally: empty index emits explicit
placeholder. Parser unit-tested across 7 synthetic filenames covering
compound, simple, and edge cases.
See ~/.claude/plans/mirror-nightly-redesign-to-builder.md.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
6bf71492fd
commit
63994266e2
@@ -0,0 +1,44 @@
|
||||
name: cleanup
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 6 * * 1' # Mondays 06:00 UTC (offset by +1h from firmware's cleanup)
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
concurrency:
|
||||
group: gh-pages-manifest
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
prune:
|
||||
name: Prune old nightly releases
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Delete releases beyond the 90 newest
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
to_delete=$(gh release list --limit 200 --json tagName,createdAt \
|
||||
| jq -r '[ .[] | select(.tagName | test("^nightly-[0-9]{8}-[0-9a-f]{7}$")) ]
|
||||
| sort_by(.createdAt) | reverse | .[90:] | .[].tagName')
|
||||
|
||||
if [ -z "$to_delete" ]; then
|
||||
echo "Nothing to delete; <=90 dated nightlies present."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "$to_delete" | while read -r tag; do
|
||||
[ -z "$tag" ] && continue
|
||||
echo "Deleting $tag"
|
||||
gh release delete "$tag" --cleanup-tag --yes
|
||||
done
|
||||
|
||||
- name: Refresh manifest
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: gh workflow run manifest.yml
|
||||
@@ -0,0 +1,56 @@
|
||||
name: manifest
|
||||
on:
|
||||
workflow_run:
|
||||
workflows: [Build]
|
||||
types: [completed]
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
concurrency:
|
||||
group: gh-pages-manifest
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
generate:
|
||||
name: Generate manifest
|
||||
# Publish on success AND on partial-failure: one flaky board (e.g. a
|
||||
# transient toolchain 502) should not deny manifest updates for the
|
||||
# platforms that did upload artifacts. enrich_manifest.py reads
|
||||
# whatever assets actually exist on the release.
|
||||
if: >-
|
||||
github.event_name == 'workflow_dispatch' ||
|
||||
contains(fromJSON('["success", "failure"]'), github.event.workflow_run.conclusion)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout master (for the script)
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
path: master
|
||||
|
||||
- name: Checkout gh-pages (for output)
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: gh-pages
|
||||
path: pages
|
||||
|
||||
- name: Generate manifest
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
run: python3 master/.github/scripts/enrich_manifest.py pages
|
||||
|
||||
- name: Publish to gh-pages
|
||||
working-directory: pages
|
||||
run: |
|
||||
git config user.email "actions@github.com"
|
||||
git config user.name "github-actions[bot]"
|
||||
git add manifest.json manifest.flat
|
||||
if git diff --cached --quiet; then
|
||||
echo "No manifest changes; nothing to commit."
|
||||
else
|
||||
newest=$(jq -r '.channels.nightly // "empty"' manifest.json)
|
||||
git commit -m "manifest: $(date -u +%FT%TZ) — ${newest}"
|
||||
git push
|
||||
fi
|
||||
Reference in New Issue
Block a user