ci/nightly: manifest aggregator + 90-build retention sweep (#92)

PR-Bld-B in the mirror of OpenIPC/firmware's nightly redesign. Adds
the two workflows that turn dated nightly-* releases (from PR-Bld-A)
into a queryable index served via GitHub Pages.

`manifest.yml` — triggers on successful `Build` workflow_run completion
(or manual dispatch). Runs `.github/scripts/enrich_manifest.py` to
enumerate dated releases, keep the 90 newest, parse the body
(sha/short/built_at from PR-Bld-A) and asset list, then emit:
  - `manifest.json` — rich JSON for hosts/agents/CI.
  - `manifest.flat` — whitespace-delimited columns
    `build_id platform flash size url` plus `@channel` records,
    parseable by pure busybox `awk` (no jq, no jsonfilter) for
    on-device sysupgrade in Phase 2.
Both committed to the `gh-pages` branch (already bootstrapped).

`cleanup.yml` — Mondays 06:00 UTC (offset by +1h from firmware's
05:00 UTC cleanup). Deletes dated nightly releases beyond the 90
newest via `gh release delete --cleanup-tag`, then re-triggers
manifest.yml.

`enrich_manifest.py` differs from firmware's only in the asset parser:
builder produces TWO filename forms (per `master.yml`'s COMMON awk
trick that counts underscores in the matrix entry name):
  - **Compound** `<soc>_<variant>_<vendor>-<model>-nor.tgz` →
    platform key = the full string. This is the common case
    (per-device builds).
  - **Simple** `openipc.<soc>-<flash>-<variant>.tgz` (firmware-style,
    when matrix name has one underscore) → platform key =
    `<soc>_<variant>`.

The platform key gets carried into manifest.flat as a single column.
sysupgrade's existing `awk '$2==p'` lookup works unchanged; Phase 3
(model-aware BUILD_PLATFORM field in os-release) will let cameras
filter against the full per-model key.

URLs after merge:
  https://openipc.github.io/builder/manifest.json
  https://openipc.github.io/builder/manifest.flat

(`gh-pages` branch + Pages config already set up; mirrors firmware-
side setup.)

Concurrency-grouped `gh-pages-manifest` so manifest.yml and
cleanup.yml can't race the gh-pages push.

Includes the retry hardening from firmware OpenIPC/firmware#2129
(4-attempt backoff for transient GitHub API 401/5xx, fast-fail on
permanent 404).

First-run behaviour validated locally: empty index emits explicit
placeholder. Parser unit-tested across 7 synthetic filenames covering
compound, simple, and edge cases.

See ~/.claude/plans/mirror-nightly-redesign-to-builder.md.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dmitry Ilyin
2026-05-23 23:05:36 +03:00
committed by GitHub
co-authored by Claude Opus 4.7
parent 6bf71492fd
commit 63994266e2
3 changed files with 279 additions and 0 deletions
+44
View File
@@ -0,0 +1,44 @@
name: cleanup
on:
schedule:
- cron: '0 6 * * 1' # Mondays 06:00 UTC (offset by +1h from firmware's cleanup)
workflow_dispatch:
permissions:
contents: write
concurrency:
group: gh-pages-manifest
cancel-in-progress: false
jobs:
prune:
name: Prune old nightly releases
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Delete releases beyond the 90 newest
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
to_delete=$(gh release list --limit 200 --json tagName,createdAt \
| jq -r '[ .[] | select(.tagName | test("^nightly-[0-9]{8}-[0-9a-f]{7}$")) ]
| sort_by(.createdAt) | reverse | .[90:] | .[].tagName')
if [ -z "$to_delete" ]; then
echo "Nothing to delete; <=90 dated nightlies present."
exit 0
fi
echo "$to_delete" | while read -r tag; do
[ -z "$tag" ] && continue
echo "Deleting $tag"
gh release delete "$tag" --cleanup-tag --yes
done
- name: Refresh manifest
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: gh workflow run manifest.yml
+56
View File
@@ -0,0 +1,56 @@
name: manifest
on:
workflow_run:
workflows: [Build]
types: [completed]
workflow_dispatch:
permissions:
contents: write
concurrency:
group: gh-pages-manifest
cancel-in-progress: false
jobs:
generate:
name: Generate manifest
# Publish on success AND on partial-failure: one flaky board (e.g. a
# transient toolchain 502) should not deny manifest updates for the
# platforms that did upload artifacts. enrich_manifest.py reads
# whatever assets actually exist on the release.
if: >-
github.event_name == 'workflow_dispatch' ||
contains(fromJSON('["success", "failure"]'), github.event.workflow_run.conclusion)
runs-on: ubuntu-latest
steps:
- name: Checkout master (for the script)
uses: actions/checkout@v4
with:
path: master
- name: Checkout gh-pages (for output)
uses: actions/checkout@v4
with:
ref: gh-pages
path: pages
- name: Generate manifest
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}
run: python3 master/.github/scripts/enrich_manifest.py pages
- name: Publish to gh-pages
working-directory: pages
run: |
git config user.email "actions@github.com"
git config user.name "github-actions[bot]"
git add manifest.json manifest.flat
if git diff --cached --quiet; then
echo "No manifest changes; nothing to commit."
else
newest=$(jq -r '.channels.nightly // "empty"' manifest.json)
git commit -m "manifest: $(date -u +%FT%TZ) — ${newest}"
git push
fi