piotr/openipc-builder
1
0
Fork 0
Code Issues 2 Pull Requests 1 Actions Packages Projects Releases 2 Wiki Activity

ci: mirror firmware's size-report sidecar (OpenIPC/firmware#2166) (#102)

Browse Source 
Firmware PR #2166 added a `size-report` Makefile target + the
`general/scripts/size_report.py` analyser that emits a per-package /
per-kernel-module `sizes.<soc>-<variant>.json` next to the .tgz. Builder
clones firmware fresh on every run and shells to `make BOARD=...`, so the
script + target are already available downstream — this PR adds the CI wiring
and manifest extension so the JSON ships as a release asset and surfaces
through https://openipc.github.io/builder/manifest.json.

Three changes:

* `builder.sh`: best-effort `make size-report` after the main build, and
  copy `sizes.*.json` into the timestamped archive next to `.tgz` /
  `rootfs.squashfs.*`. Failure is swallowed so a legacy `OPENIPC_FW_REV` pin
  to a pre-#2166 firmware ref can't sink the build.

* `.github/workflows/master.yml`: new `Build size report` step after the
  existing `Build firmware`. Mirrors the bimodal rename trick the .tgz path
  already does — compound matrix entries (`<soc>_<variant>_<vendor>-<model>`)
  share `<soc>_<variant>` and would otherwise collide on the firmware-side
  `sizes.<soc>-<variant>.json` filename when uploaded to the same release
  tag. Compound entries get the JSON renamed to `sizes.<NAME>.json`; simple
  entries keep `sizes.<soc>-<variant>.json`. `${{env.SIZES}}` is added to
  all three upload `files:` blocks (dated / nightly / latest).

* `.github/scripts/enrich_manifest.py`: `SIZES_COMPOUND_RE` +
  `SIZES_SIMPLE_RE`, a `parse_sizes()` helper that tries compound first
  (because `[^._-]+` in the simple soc class would otherwise accept the
  compound form too), the sidecar is recorded at the platform level under
  a `"sizes"` key in `manifest.json`, and a new `@sizes <build_id>
  <platform> <size> <url>` line type is emitted in `manifest.flat`
  alongside the existing `@channel` directive.

Schema is unchanged for existing consumers — on-device sysupgrade reads
positional 4-column `manifest.flat` lines and stays inert on unknown
`@`-prefixed directives.

Co-authored-by: John <zekccsripfwrocp-15689@yn.furvionx.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dmitry Ilyin
2026-06-03 15:05:05 +03:00
committed by GitHub
co-authored by John Claude Opus 4.7
parent 83faaad635
commit da13271f94
3 changed files with 75 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/scripts/enrich_manifest.py
+40 -6
View File
@@ -41,6 +41,15 @@ TAG_RE = re.compile(r"^nightly-(\d{8})-([0-9a-f]{7})$")
COMPOUND_RE = re.compile(r"^(?P<platform>[a-z0-9]+_[a-z0-9]+_[a-z0-9_.-]+)-(?P<flash>nor|nand)\.tgz$")
SIMPLE_RE = re.compile(r"^openipc\.(?P<soc>[^.]+)-(?P<flash>nor|nand)-(?P<variant>\w+)\.tgz$")
# Per-platform size report sidecar (PR #97, mirrors firmware PR #2166).
# Compound form is renamed in master.yml to `sizes.<NAME>.json` so two
# compound devices sharing `<soc>_<variant>` don't collide on upload;
# simple form keeps firmware's `sizes.<soc>-<variant>.json` shape.
# Compound is tried first because `[^._-]+` in simple would otherwise
# accept the underscore-laden compound form too.
SIZES_COMPOUND_RE = re.compile(r"^sizes\.(?P<platform>[a-z0-9]+_[a-z0-9]+_[a-z0-9_.-]+)\.json$")
SIZES_SIMPLE_RE = re.compile(r"^sizes\.(?P<soc>[^._-]+)-(?P<variant>\w+)\.json$")
# Retry budget for transient GitHub API failures (mirrors firmware/#2129).
GH_RETRY_DELAYS = (0, 5, 15, 40)
@@ -107,6 +116,17 @@ def parse_asset(name: str) -> tuple[str, str] | None:
return None
def parse_sizes(name: str) -> str | None:
"""Returns the platform_key for a sizes.*.json sidecar, or None."""
m = SIZES_COMPOUND_RE.match(name)
if m:
return m.group("platform")
m = SIZES_SIMPLE_RE.match(name)
if m:
return f"{m.group('soc')}_{m.group('variant')}"
return None
def main() -> None:
out_dir = Path(sys.argv[1] if len(sys.argv) > 1 else ".")
out_dir.mkdir(parents=True, exist_ok=True)
@@ -133,13 +153,19 @@ def main() -> None:
platforms: dict[str, dict[str, dict]] = {}
for a in info.get("assets") or []:
parsed = parse_asset(a["name"])
if not parsed:
if parsed:
platform, flash = parsed
platforms.setdefault(platform, {})[flash] = {
"url": a["url"],
"size": a["size"],
}
continue
platform, flash = parsed
platforms.setdefault(platform, {})[flash] = {
"url": a["url"],
"size": a["size"],
}
platform = parse_sizes(a["name"])
if platform:
platforms.setdefault(platform, {})["sizes"] = {
"url": a["url"],
"size": a["size"],
}
builds.append({
"id": info["tagName"],
"sha": sha,
@@ -162,11 +188,19 @@ def main() -> None:
"# OpenIPC builder build index",
f"# generated_at={now}",
"# columns: build_id platform flash size url",
"# also: @sizes <build_id> <platform> <size> <url> (per-platform size report sidecar)",
]
for b in builds:
for platform, flashes in sorted(b["platforms"].items()):
for flash, info in sorted(flashes.items()):
if flash == "sizes":
continue
lines.append(f"{b['id']} {platform} {flash} {info['size']} {info['url']}")
sizes = flashes.get("sizes")
if sizes:
lines.append(
f"@sizes {b['id']} {platform} {sizes['size']} {sizes['url']}"
)
lines.append("# channels")
for ch, target in manifest["channels"].items():
lines.append(f"@channel {ch} {target}")
.github/workflows/master.yml
+27
View File
@@ -279,6 +279,30 @@ jobs:
exit 1
fi
- name: Build size report
if: env.NORFW || env.NANDFW
run: |
cd openipc
NAME=${{matrix.platform}}
make BOARD=${NAME} size-report || \
{ echo "::warning::size-report failed for ${NAME}"; exit 0; }
SIZES=$(find output/images -name 'sizes.*.json' | head -1)
if [ -z "${SIZES}" ]; then
exit 0
fi
# Mirror the .tgz rename trick: compound matrix entries
# (`<soc>_<variant>_<vendor>-<model>`) share `<soc>_<variant>` and
# would collide on the firmware-side `sizes.<soc>-<variant>.json`
# filename when uploaded to the same release tag. Rename to
# `sizes.<NAME>.json` so every compound device gets its own asset.
COMMON=$(echo ${NAME} | awk -F_ '{print NF-1}')
if [ ${COMMON} -eq 1 ]; then
echo "SIZES=openipc/${SIZES}" >> ${GITHUB_ENV}
else
mv ${SIZES} ../sizes.${NAME}.json
echo "SIZES=sizes.${NAME}.json" >> ${GITHUB_ENV}
fi
- name: Upload firmware (dated)
if: env.NORFW || env.NANDFW
uses: softprops/action-gh-release@v2
@@ -292,6 +316,7 @@ jobs:
files: |
${{env.NORFW}}
${{env.NANDFW}}
${{env.SIZES}}
- name: Upload firmware (rolling nightly)
if: env.NORFW || env.NANDFW
@@ -305,6 +330,7 @@ jobs:
files: |
${{env.NORFW}}
${{env.NANDFW}}
${{env.SIZES}}
- name: Upload firmware (latest — legacy alias)
if: env.NORFW || env.NANDFW
@@ -314,6 +340,7 @@ jobs:
files: |
${{env.NORFW}}
${{env.NANDFW}}
${{env.SIZES}}
- name: Send binary
if: env.NORFW
builder.sh
+8
View File
@@ -57,6 +57,9 @@ copy_to_archive() {
${FIRMWARE_DIR}/output/images/openipc.*.tgz \
${BUILDER_DIR}/archive/${DEVICE}/${TIMESTAMP}
cp -a ${FIRMWARE_DIR}/output/images/sizes.*.json \
${BUILDER_DIR}/archive/${DEVICE}/${TIMESTAMP} 2>/dev/null || true
if [ -f "${FIRMWARE_DIR}/output/images/autoupdate-kernel.img" ]; then
cp -a ${FIRMWARE_DIR}/output/images/autoupdate* ${BUILDER_DIR}/archive/${DEVICE}/${TIMESTAMP}
fi
@@ -153,6 +156,11 @@ cp -afv ${BUILDER_DIR}/${ITEM}/* ${FIRMWARE_DIR}
echo_c 33 "\nBuilding the device"
make BOARD=${DEVICE}
# Best-effort: emit per-package/per-kernel-module size JSON next to the .tgz.
# Target lives in firmware's Makefile (PR #2166); ignore failure so legacy
# pinned firmware refs without the target don't sink the build.
make BOARD=${DEVICE} size-report || true
copy_to_archive
echo_c 35 "\nDone"
cd "$BUILDER_DIR"