mirror of
https://github.com/Klipper4FlashForge/firmware.git
synced 2026-10-19 03:49:50 -05:00
Oleksandr ShyshatskyiandClaude Opus 5
9973019ec8
Phase 3 of docs/notes/85-packaging.md was three things: signing, a URL the
printer asks, and a tree at that URL. The migration landed the first. This is
the second, and the tooling for the third.
THE POINTER HAS TO SHIP BEFORE THE HOST EXISTS, which is why this lands with
nothing serving yet. A feed cannot tell a printer where the feed is: a machine
installed without etc/apk/repositories needs a USB stick before it can ever
fetch anything, which is the thing packages were adopted to stop. So the file
travels in the .tgz, naming a host that does not answer yet, and every printer
installed from today starts fetching the day one does.
The cost of that is bounded, and was read out of apk's source rather than
assumed: an unreachable repository fails `apk upgrade` and nothing else --
apk_db_repository_check is called from app_upgrade.c and not from app_add.c,
so `apk add ./file.apk` does not consult a repository at all.
http://reforge.8941973.xyz/apk/mipsel_xburst2/anvil.adb
bin/common.sh's FEED_URL is the base, config.env can override it for a fork,
and pkgs/anvil-core generates the file from it beside the public key it
already shipped. Three things about that URL:
* IT NAMES THE INDEX FILE, not a directory. src/repoparser.c reads a line
ending .adb as a v3 index in place, with the packages as siblings named by
the pkgname-spec mkndx wrote into it; anything else sends apk looking for
<url>/<arch>/APKINDEX.tar.gz. That is the layout this repo already builds
and the stick already uses, so the network feed is the same shape --
exercised by every payload build rather than only in production. The v3
directory form would work too, but only with
`mkndx --pkgname-spec '${arch}/${name}-${version}.apk'`.
* THE ARCH IS A DIRECTORY. Package file names carry no architecture, so two
architectures served from one directory would be two files with one name.
* PLAIN HTTP. libfetch defaults to SSL_VERIFY_PEER and this printer has no
certificate bundle -- the same gap tools/python/README.md records for
CPython. Every package is signed, and the signature rather than the
transport is what decides whether a printer installs one.
bin/publish-feed.sh writes the matching tree and optionally rsyncs it.
Packages go first and the index last, because the index names every package by
sha256 and a mirror carrying a new index with old packages hands printers a
checksum for a file that 404s. Nothing is deleted at the far end. It refuses
an unsigned feed unless ALLOW_UNSIGNED=1: a printer verifies against the key
that shipped in its own anvil-core, so an unsigned feed is one no printer
installs from. The release workflow attaches the same tree as
reforge-apk-feed.tar.gz, so a host can be filled from any release -- including
ones tagged before the host existed -- without rebuilding.
THE REPLICA STOPPED WRITING THE FILE IT USED TO DELETE.
case-build-payload.sh wrote its own etc/apk/repositories naming /mnt and
removed it before tarring; it now passes --repositories-file with a path on
the stick, which apk reads INSTEAD of the default (database.c consults the
default file and repositories.d only when no file is named). So what ships is
whatever anvil-core installed and the build never reads it. The `apk info`
line takes the same flag: without it that one command reads the shipped file
and reaches for the network, which a build's output must not depend on.
Measured: 47 packages built from a cold cache, the payload assembled in the
replica twice, qa/static at 319 passed. Two of those are new -- the default
URL in common.sh (read from the file, not resolved, so a fork setting
FEED_URL in config.env still passes) and the built payload carrying a
package-owned entry that ends /mipsel_xburst2/anvil.adb.
What is left is operational: a host answering at $FEED_URL, and who runs
publish-feed.sh -- a release step with a deploy secret, or a person with the
tarball.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017rgrDeu9YAUuMM2G7gKTpL
303 lines
15 KiB
YAML
303 lines
15 KiB
YAML
name: release
|
|
|
|
# A release is blocked on the printer replica passing.
|
|
#
|
|
# The replica is the printer's real rootfs.squashfs, chrooted under
|
|
# qemu-mipsel, with /usr/prog installed by FlashForge's own updater. The
|
|
# installer under test runs on the printer's busybox, tar, md5sum and unTar --
|
|
# not on Debian stand-ins. If that fails, nothing ships.
|
|
#
|
|
# The stock FlashForge packages and the factory image are large but not
|
|
# secret: they are published at ghzserg/FF and this workflow fetches what is
|
|
# already public, exactly as ci.yml does. Every one is pinned by sha256 and
|
|
# the checksum is enforced, not advisory -- an unpinned download would let the
|
|
# archive change under us while the gates kept reporting green against a
|
|
# different firmware than they were written for.
|
|
#
|
|
# TAGS. `v<YYYYMMDD>-<city>`, e.g. v20260824-nova-kakhovka: the release date,
|
|
# and the name of a Ukrainian city under occupation. Only the date reaches the
|
|
# package filename -- the installer's version field is a date and nothing else
|
|
# -- so the city is carried by the tag and the release title.
|
|
|
|
on:
|
|
push:
|
|
tags: [ 'v*' ]
|
|
workflow_dispatch:
|
|
|
|
permissions:
|
|
contents: write
|
|
|
|
jobs:
|
|
release:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Fetch the stock firmware
|
|
# Same artefacts, same checksums as ci.yml. Enforced: a mismatch here
|
|
# means the thing we test against is not the thing we tested against
|
|
# last time, and that has to stop the release rather than colour it.
|
|
env:
|
|
STOCK_URL_CREATOR5PRO: https://github.com/ghzserg/FF/releases/download/R/Creator5Pro-1.9.7-1.2.9-20260810.tgz
|
|
STOCK_SHA_CREATOR5PRO: 1f3252feaae6d8fc72d7b0fd1ba832cc63d369659f8b2d2896efcf7adab2976b
|
|
STOCK_URL_CREATOR5: https://github.com/ghzserg/FF/releases/download/R/Creator5-1.9.7-1.2.9-20260810.tgz
|
|
STOCK_SHA_CREATOR5: 72009be183e74a4562050a927787ba686c6b558a075f00958892197ac52659cc
|
|
run: |
|
|
set -euo pipefail
|
|
mkdir -p stock
|
|
curl -fsSL "$STOCK_URL_CREATOR5PRO" -o stock/Creator5Pro-stock.tgz
|
|
curl -fsSL "$STOCK_URL_CREATOR5" -o stock/Creator5-stock.tgz
|
|
# Exactly two spaces before the path: sha256sum -c reads everything
|
|
# after them as the filename, so a column-aligned line looks for a
|
|
# file whose name begins with spaces and fails as "not found".
|
|
echo "$STOCK_SHA_CREATOR5PRO stock/Creator5Pro-stock.tgz" | sha256sum -c -
|
|
echo "$STOCK_SHA_CREATOR5 stock/Creator5-stock.tgz" | sha256sum -c -
|
|
ls -lh stock
|
|
|
|
- name: Write test.env (the replica)
|
|
env:
|
|
# THE replica: the printer's rootfs, its real /usr/prog and
|
|
# /usr/data, and the stock package already installed on top. The
|
|
# end-to-end test reinstalls the baseline itself when the package
|
|
# under test is for the other model, so this only ever changes how
|
|
# long a run takes, never what it proves. vars.PRINTER_IMAGE
|
|
# overrides it for a fork building its own.
|
|
#
|
|
# A VERSION TAG, NOT :latest. A release is blocked on this image, so
|
|
# it is as much a pinned input as the stock packages above -- and it
|
|
# is the only one whose contents nothing here checksums. A floating
|
|
# tag would let the substrate a release was proved on change without
|
|
# a commit, which is exactly what the sha256 lines exist to stop.
|
|
PRINTER_IMAGE: ${{ vars.PRINTER_IMAGE || 'monstrofil/creator5-printer:1.9.7-1.2.9-20260810' }}
|
|
run: |
|
|
set -euo pipefail
|
|
if [ -z "${PRINTER_IMAGE:-}" ]; then
|
|
echo "::error title=No replica::PRINTER_IMAGE is empty and there is no other way to get a replica."
|
|
exit 1
|
|
fi
|
|
echo "PRINTER_IMAGE=\"$PRINTER_IMAGE\"" > test.env
|
|
cat test.env
|
|
|
|
# BEFORE config.env, which has to name the file this writes.
|
|
#
|
|
# OUTSIDE THE CHECKOUT, in the runner's scratch directory: the private
|
|
# half must not be somewhere a later step can archive into a release,
|
|
# and $RUNNER_TEMP is wiped with the runner. Mode 0600 because openssl
|
|
# will read a world-readable key without complaint and that is the sort
|
|
# of thing that stays wrong for a year.
|
|
#
|
|
# NOT REQUIRED. A fork building its own firmware has no such secret, and
|
|
# an unsigned feed still installs -- the printer is handed the packages
|
|
# on a stick it already trusts, and bin/build-payload.sh passes
|
|
# --allow-untrusted when nothing signed them. It is a warning rather
|
|
# than a failure for that reason, and a loud one because a release that
|
|
# quietly stopped being signed is the thing worth noticing.
|
|
- name: Unlock the feed signing key
|
|
env:
|
|
APK_SIGN_KEY_PEM: ${{ secrets.APK_SIGN_KEY }}
|
|
run: |
|
|
set -euo pipefail
|
|
if [ -z "${APK_SIGN_KEY_PEM:-}" ]; then
|
|
echo "::warning title=Unsigned feed::secrets.APK_SIGN_KEY is not set, so this release's packages carry no signature. See docs/notes/85-packaging.md."
|
|
exit 0
|
|
fi
|
|
install -m 0600 /dev/null "$RUNNER_TEMP/anvil.rsa"
|
|
printf '%s\n' "$APK_SIGN_KEY_PEM" > "$RUNNER_TEMP/anvil.rsa"
|
|
# The public half is DERIVED, never stored as a second secret: two
|
|
# secrets can disagree, and then the printer ships a key that does
|
|
# not verify the packages beside it.
|
|
openssl pkey -in "$RUNNER_TEMP/anvil.rsa" -pubout \
|
|
-out "$RUNNER_TEMP/anvil.rsa.pub"
|
|
echo "APK_SIGN_KEY=$RUNNER_TEMP/anvil.rsa" >> "$GITHUB_ENV"
|
|
echo "the feed will be signed with $(openssl pkey -in "$RUNNER_TEMP/anvil.rsa" -pubout -outform DER | sha256sum | cut -c1-16)"
|
|
|
|
- name: Write config.env
|
|
run: |
|
|
cat > config.env <<CFG
|
|
MOD_NAME=anvil
|
|
# The tag is v<YYYYMMDD>-<city>; the package version is the date
|
|
# alone. FlashForge's installer reads this as the software version
|
|
# and it is compared as a number, so the codename stays in the tag.
|
|
# A manual run has no tag -- GITHUB_REF_NAME would be the branch --
|
|
# so leave it empty and let the build stamp today's UTC date.
|
|
# The optional trailing letter is the same-day re-release
|
|
# convention common.sh documents (20260824b): nothing on the printer
|
|
# reads it, so it only keeps the outer filenames distinct.
|
|
MOD_VER=$(case "$GITHUB_REF" in
|
|
refs/tags/*) echo "${GITHUB_REF_NAME#v}" | sed -n 's/^\([0-9]\{8\}[a-z]\{0,1\}\).*/\1/p' ;;
|
|
*) echo "" ;;
|
|
esac)
|
|
TARGET_MACHINE=Creator5Pro
|
|
STOCK_TGZ_CREATOR5PRO="$PWD/stock/Creator5Pro-stock.tgz"
|
|
STOCK_TGZ_CREATOR5="$PWD/stock/Creator5-stock.tgz"
|
|
STOCK_TGZ=""
|
|
# No Klipper source knob any more: pkgs/klipper builds from the
|
|
# commit pinned in versions.env and cross-compiles c_helper.so from
|
|
# those same sources. KLIPPER_FORK used to be settable here, and
|
|
# setting it to "" is what made v20260824 ship the stock 0.12-era
|
|
# klippy; a recipe names one source, so that cannot recur.
|
|
# The secret is INTENTIONALLY unset: a hash baked in here would be
|
|
# the same root password in every public release. Empty means each
|
|
# printer picks a random one on its first install and writes it to
|
|
# anvil-password.txt on the stick; updates keep it. The plumbing
|
|
# stays so a fork can bake one in deliberately.
|
|
ROOT_PW_HASH='${{ secrets.ROOT_PW_HASH }}'
|
|
FF_KEY='FFP0331&*%root'
|
|
# Empty unless the step above unlocked a key, and empty means an
|
|
# unsigned feed rather than a failure -- see there.
|
|
APK_SIGN_KEY='${APK_SIGN_KEY:-}'
|
|
CFG
|
|
|
|
# Before the suite, not after: the replica gates run ./bin/payload.sh
|
|
# directly and it expects vendor/ already populated. `make release`
|
|
# further down would have fetched them, but that runs after the gates.
|
|
#
|
|
# Cached because the set now includes the Klipper fork and the ~203MB
|
|
# MIPS toolchain that compiles its chelper; the key is the pin file, so
|
|
# a version bump refetches and anything else reuses.
|
|
- name: Cache the pinned vendor assets
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: vendor
|
|
key: vendor-${{ hashFiles('versions.env') }}
|
|
|
|
- name: Fetch the pinned Mainsail, HelixScreen, Moonraker and Klipper assets
|
|
run: make vendor ASSET_ROOT=$PWD
|
|
|
|
# THE FEED FIRST. bin/payload.sh builds the payload by installing it, so
|
|
# it refuses to start without one, and the python recipes resolve their
|
|
# build dependencies out of it. This used to run unpack/patch/pack with
|
|
# no feed at all and could not have passed.
|
|
- name: Build the package feed
|
|
run: make packages ASSET_ROOT=$PWD
|
|
|
|
- name: Build the package the suite installs
|
|
run: make build ASSET_ROOT=$PWD
|
|
|
|
- name: Brick-safety suite, replica required
|
|
# No REQUIRE_PRINTER_SIM and no ALLOW_SKIP: qa has no skip mode, so a
|
|
# missing docker, image or package fails at the point that needs it.
|
|
#
|
|
# Through `make`, not around it. These steps used to hand-roll their
|
|
# own `docker run`, which meant two things: the release path exercised
|
|
# entry points no human uses -- so a bug in one of the Makefile's own
|
|
# runners could not be caught here -- and the container ran as root,
|
|
# leaving a work/ that the `make release` below, which runs as the
|
|
# runner user, could not write to.
|
|
run: make qa ASSET_ROOT=$PWD
|
|
|
|
- name: Build both models
|
|
run: make release ASSET_ROOT=$PWD
|
|
|
|
# THE SAME PACKAGES, SERVED RATHER THAN INSTALLED. `make release` puts
|
|
# two .tgz on the release page, each carrying the whole payload; this
|
|
# attaches the feed those payloads were built from, laid out exactly as
|
|
# $FEED_URL serves it. Unpacking this tarball into a web root IS the
|
|
# publish step -- so a host can be filled from any release, including
|
|
# one tagged before the host existed, without rebuilding anything.
|
|
#
|
|
# ALLOW_UNSIGNED because an unsigned feed must not be the thing that
|
|
# fails a release: the run already warns about the missing secret above,
|
|
# and a release whose .tgz packages are unsigned should hand over an
|
|
# asset that says the same rather than no asset at all.
|
|
- name: Stage the package feed as a release asset
|
|
run: |
|
|
set -euo pipefail
|
|
ALLOW_UNSIGNED=1 ./bin/publish-feed.sh
|
|
tar -C work/feed-site -czf dist/reforge-apk-feed.tar.gz .
|
|
ls -lh dist/reforge-apk-feed.tar.gz
|
|
|
|
# BEFORE the end-to-end test, so a build that produced packages hands
|
|
# them over even when the lane below fails. This is not a weakening of
|
|
# the release gate: Publish runs after that step and the job stops if it
|
|
# fails, so a red run still cannot reach the release page.
|
|
- uses: actions/upload-artifact@v4
|
|
with:
|
|
name: firmware-packages
|
|
path: dist/*.tgz
|
|
retention-days: 30
|
|
|
|
- name: End-to-end update test on the package that will actually ship
|
|
# The suite above tested a package built from the same tree; this
|
|
# tests the exact file that gets attached to the release.
|
|
#
|
|
# ONE MODEL, NOT BOTH. Two models are built and both are attached, but
|
|
# there is one replica and it is a Creator5Pro -- its own app_startup.sh
|
|
# globs /mnt/Creator5Pro-*.tgz and refuses anything else, which is the
|
|
# model gate doing its job. Feeding it the Creator5 package produced 123
|
|
# errors that all said "could not install", and the package it was
|
|
# complaining about was correct.
|
|
#
|
|
# This loop used to run both because sim-install.py picked its own
|
|
# machine per package; the replica lane takes PRINTER_IMAGE from
|
|
# test.env and there is one of those. A Creator5 end-to-end needs a
|
|
# Creator5 replica image, which nothing publishes yet.
|
|
#
|
|
# REAL_PKG points the replica lane at one exact package instead of the
|
|
# newest thing in work/out -- 22 named results where sim-install.py
|
|
# gave one exit code. `make qa-replica` passes it through (see RUNSIM).
|
|
run: |
|
|
set -euo pipefail
|
|
p=$(ls dist/${TARGET_MACHINE}-*.tgz)
|
|
echo "=== $p ==="
|
|
REAL_PKG="$p" make qa-replica ASSET_ROOT=$PWD
|
|
env:
|
|
TARGET_MACHINE: Creator5Pro
|
|
|
|
- name: Assemble the release notes
|
|
if: startsWith(github.ref, 'refs/tags/')
|
|
# The fixed half is the same every time; the half that names the city
|
|
# is not. docs/releases/<tag>.md carries it, and is appended when it
|
|
# exists -- so the story of the city the release is named for lives in
|
|
# the repo at the tagged commit, not only in a hand-edited release
|
|
# page that nothing reproduces.
|
|
run: |
|
|
set -euo pipefail
|
|
cat > RELEASE_NOTES.md <<'NOTES'
|
|
Copy the file matching YOUR printer to the root of a FAT32 USB
|
|
stick and power the machine on. A `Creator5Pro-` package will not
|
|
install on a Creator 5 and vice versa.
|
|
|
|
Every package here was installed into a replica of the printer --
|
|
the real rootfs running under qemu-mipsel, with the package on a
|
|
real FAT filesystem that the machine's own app_startup.sh found
|
|
and installed -- and the machine was booted again afterwards and
|
|
checked to come up.
|
|
|
|
**If the screen stays dark**, the printer is still up: ssh and
|
|
Mainsail run from their own services, before and independently of
|
|
the UI, so reach it at `http://<printer-ip>/` or ssh in and read
|
|
`/usr/data/logs/helixscreen.log`. There is no automatic fallback --
|
|
`apk del anvil-helixscreen` to boot headless on
|
|
purpose. **To go back to stock**, put FlashForge's own package on a
|
|
USB stick and power on -- that path is tested on every release too.
|
|
|
|
---
|
|
|
|
NOTES
|
|
CITY="docs/releases/${GITHUB_REF_NAME}.md"
|
|
if [ -f "$CITY" ]; then
|
|
cat "$CITY" >> RELEASE_NOTES.md
|
|
else
|
|
echo "::warning::no $CITY -- publishing the generic line instead"
|
|
echo "Releases are named for a Ukrainian city under occupation." \
|
|
>> RELEASE_NOTES.md
|
|
fi
|
|
cat RELEASE_NOTES.md
|
|
|
|
- name: Publish
|
|
if: startsWith(github.ref, 'refs/tags/')
|
|
uses: softprops/action-gh-release@v2
|
|
with:
|
|
# Both models' installers, and the feed they were built from.
|
|
files: |
|
|
dist/*.tgz
|
|
dist/reforge-apk-feed.tar.gz
|
|
# Every build so far is a pre-release: this replaces the stock UI on
|
|
# a machine whose only recovery path is a USB stick with the stock
|
|
# package on it. That is worth saying with the flag, not just in
|
|
# prose people scroll past.
|
|
prerelease: true
|
|
name: ${{ github.ref_name }}
|
|
body_path: RELEASE_NOTES.md
|