From fae26a85e585984e7fa0aaa4ed3b14ddfe3a8071 Mon Sep 17 00:00:00 2001 From: Oleksandr Shyshatskyi Date: Sun, 30 Aug 2026 15:16:52 +0300 Subject: [PATCH] No FlashForge component ships; patch.sh is payload.sh The software component existed to give the mod one exec hook. It is not needed for that any more, so a release now carries our installer and our payload and nothing else: 26MB smaller, 21MB of which was FlashForge's own firmwareExe travelling back to the partition it was already on, along with their klippy tree, their wifi module and their configs. WHAT IT GIVES BACK. /usr/prog/PROGRAM/software is no longer rewritten, so the version directory there stays FlashForge's with their genuine firmwareExe inside it -- and app_startup.sh's five-second watchdog, which copies one out of that directory when nothing by that name is running, has something to copy again. A printer whose payload fails to install now comes up on the stock UI instead of a blank screen. That recovery has been broken since we started deleting their binary out of a component we shipped; docs/how-it-works.md claimed it worked throughout. THE ROOT PASSWORD moves with it, because it used to ride in the component's shadow file. bin/pack.sh bakes ROOT_PW_HASH into the installer, single-quoted -- an unquoted $6$... would have the printer's shell expand $6 as a positional parameter and ship a truncated hash -- and the installer writes it to /usr/prog/etc/shadow. The random-password path asks one question instead of four: FlashForge ship the same root hash on every machine, so a live shadow still carrying it means nobody has set a password and it is ours to set. The .prev-root-hash recording, the shipped-vs-live comparison and the restore-it- afterwards step were all machinery for surviving a file that nothing overwrites any more, and are gone. pack.sh checks the hardcoded constant against the stock package it built from, because getting it wrong is silent and total: every printer would read as "already set" and none would ever get a password. bin/patch.sh is bin/payload.sh. It patched nothing any more -- no splice, no shadow edit, no firmwareExe deletion -- and what is left is the payload: the feed installed by the printer's own opkg, the seeded config, the manifest. Its sections are renumbered 1-5 with no gaps, and the five tombstone comments marking where removed sections used to be are gone with them. test_recipe_layout.py loses six of its nine tests. Each compared the tree against a description of the tree -- no fourth directory kind, each kind still used, 3rdparty carries no files of ours, installer/ holds exactly two named files -- and none could go red for a reason a printer would notice, while all six went red the moment the layout legitimately changed. The two that bite are kept: a payload/ path escaping $MODDIR, and a recipe name at both levels, which makes pkg_dir silently build the wrong package. SW_VER is deleted from config.env.example and both workflows: it stamped a component that is not built. Gates: 112 static tests pass. The generated installer is verified after substitution -- all four lines present, and `sh -n` over the result, because a hash with a stray character in it would otherwise reach a printer as a syntax error whose only symptom is a package that installs nothing. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CH7Yb4wuPhnppHAB25UX7e --- .claude/skills/packages/SKILL.md | 4 +- .github/workflows/ci.yml | 11 +- .github/workflows/release.yml | 10 +- Makefile | 17 +- bin/build-packages.sh | 2 +- bin/build.sh | 2 +- bin/common.sh | 2 +- bin/fetch-assets.sh | 2 +- bin/pack.sh | 108 +++++++---- bin/patch.sh | 172 ---------------- bin/payload.sh | 117 +++++++++++ bin/unpack.sh | 11 +- config.env.example | 6 - docker/Dockerfile.build | 4 +- docs/building.md | 24 +-- docs/how-it-works.md | 47 ++--- docs/qa-migration.md | 12 +- docs/testing.md | 14 +- docs/upgrading.md | 5 +- installer/runFirmwareExe.sh | 132 ++++++------- pkgs/3rdparty/busybox/build.sh | 4 +- pkgs/3rdparty/busybox/pkg.conf | 2 +- pkgs/3rdparty/execline/pkg.conf | 2 +- pkgs/3rdparty/libsodium/build.sh | 2 +- pkgs/3rdparty/libsodium/pkg.conf | 2 +- pkgs/3rdparty/openssl/build.sh | 2 +- pkgs/3rdparty/python/pkg.conf | 2 +- pkgs/anvil-core/payload/anvil-env.sh | 2 +- .../payload/etc/s6-rc/source/klipper/run | 2 +- pkgs/anvil-core/payload/prog/firmwareExe | 2 +- pkgs/anvil-core/payload/prog/start.sh | 2 +- pkgs/helixscreen/build.sh | 2 +- pkgs/helixscreen/pkg.conf | 2 +- pkgs/klipper-config/pkg.conf | 2 +- pkgs/klipper/build.sh | 6 +- pkgs/klipper/pkg.conf | 8 +- pkgs/lib.sh | 8 +- pkgs/moonraker/build.sh | 4 +- pkgs/moonraker/pkg.conf | 2 +- qa/conftest.py | 2 +- qa/replica/actions/install-package.sh | 2 +- qa/replica/test_abi.py | 4 +- qa/replica/test_install.py | 2 +- qa/replica/test_s6rc.py | 2 +- qa/replica/test_upgrade.py | 6 +- qa/replica/test_what_ships.py | 2 +- qa/static/test_ipk.py | 64 +++--- qa/static/test_moddir_paths.py | 2 +- qa/static/test_recipe_layout.py | 183 ++++-------------- qa/static/test_shell_syntax.py | 2 +- tools/python-packages/README.md | 8 +- tools/python-packages/build.sh | 2 +- tools/python/README.md | 4 +- tools/python/build.sh | 8 +- tools/replica/README.md | 2 +- tools/replica/ffsim/gates.py | 2 +- tools/replica/printer/case-build-payload.sh | 6 +- tools/replica/sim-boot-screen.py | 2 +- tools/supervisor/README.md | 2 +- versions.env | 14 +- 60 files changed, 464 insertions(+), 617 deletions(-) delete mode 100755 bin/patch.sh create mode 100755 bin/payload.sh diff --git a/.claude/skills/packages/SKILL.md b/.claude/skills/packages/SKILL.md index c6aa579..61da141 100644 --- a/.claude/skills/packages/SKILL.md +++ b/.claude/skills/packages/SKILL.md @@ -75,7 +75,7 @@ pkg_end | `pkg_end` | seal the cache | `pkg_buildopkg` is not a recipe verb: it builds the x86-64 opkg that -`bin/patch.sh` assembles the payload with, cached at `work/.opkg-host`. It has +`bin/payload.sh` assembles the payload with, cached at `work/.opkg-host`. It has to be configured `--prefix=$MODDIR` and `--disable-shared` — opkg bakes its state directory into both the binary and libopkg at compile time — and it checks its own output for both. @@ -132,7 +132,7 @@ retry with different flags. See `pkgs/3rdparty/openssl/build.sh`. | directory | where its contents end up | | --- | --- | | `payload/` | staged into the .ipk, laid out exactly as it lands under `$MODDIR` | - | `prog/` | placed on `/usr/prog` by `bin/patch.sh` — not packageable, because every path in a package of ours is under the prefix | + | `prog/` | placed on `/usr/prog` by `bin/payload.sh` — not packageable, because every path in a package of ours is under the prefix | | `seed/` | templated or seeded user state, deliberately not a package member | A recipe with a `payload/` needs `PKG_STAMP_EXTRA="$(pkg_payload_hash)"`. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 93b68a5..198c320 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -105,9 +105,9 @@ jobs: # and the Klipper fork need, none of which has a BUILD_ flag of its # own any more -- BUILD_KLIPPER is gone and the fork is the only # Klipper, so its 27MB tarball comes down here with the rest. This - # config.env never reaches patch.sh, only fetch-assets.sh. + # config.env never reaches payload.sh, only fetch-assets.sh. # The actual cross-build of s6 and CPython happens inside the suite, - # in bin/patch.sh sections 5b/5c, the first time it runs against a + # in bin/payload.sh sections 5b/5c, the first time it runs against a # vendor/ that now has what those sections need. cat > config.env <<'CFG' BUILD_MAINSAIL=0 @@ -237,7 +237,6 @@ jobs: cat > config.env < runs the scripts directly on the host. -# Packages carry only the software component by default: the stock installer -# skips absent components, so the kernel and the MCU/board firmware are left -# alone. FULL=1 carries all four (and reflashes the MCU). +# A package carries our installer and our payload and no FlashForge component +# at all: the stock installer skips absent ones, so /usr/prog, the kernel and +# the MCU/board firmware are left alone. FULL=1 carries the kernel, control and +# library components too (and reflashes the MCU). FULL ?= PACKARGS = $(if $(FULL),--full,) DOCKER ?= docker @@ -134,8 +135,8 @@ help: @echo ' make shell shell inside it' @echo ' make clean | distclean' @echo - @echo 'Packages carry only the software component; the kernel and MCU are' - @echo 'left untouched. FULL=1 make carries all four components.' + @echo 'Packages carry the installer and the payload; /usr/prog, the kernel' + @echo 'and the MCU are left untouched. FULL=1 also flashes the kernel and MCU.' @echo @echo 'Config: config.env is the BUILD config (what ships). test.env holds' @echo 'the replica settings, which never reach a printer. Copy the .example' @@ -169,7 +170,7 @@ vendor: image config.env @$(RUN) ./bin/fetch-assets.sh --all # A THIRD LANE, because the build needs both halves of the split above. -# bin/patch.sh assembles the payload by starting the printer replica, so this +# bin/payload.sh assembles the payload by starting the printer replica, so this # lane needs the docker socket -- and it still has to run AS YOU, or every # build leaves a root-owned work/ the next one cannot delete. # @@ -194,10 +195,10 @@ build: image config.env # THE RELEASE PATH IS BUILT ON THIS. It used to say the opposite -- "nothing # on the release path depends on this" -- and that was already false when it # was written: pkgs/3rdparty/python declares seven build dependencies, pkg_deps -# resolves them by unpacking their .ipk out of work/packages, and bin/patch.sh +# resolves them by unpacking their .ipk out of work/packages, and bin/payload.sh # builds none of the seven. `make build` on a cold checkout has never worked # without this target; it failed deep inside a recipe instead of saying so. -# bin/patch.sh now checks for the feed up front and names this command. +# bin/payload.sh now checks for the feed up front and names this command. # # It still, unlike `build`, needs NO stock FlashForge package -- which is most # of the point: packaging has to be runnable in CI on a bare checkout, or the diff --git a/bin/build-packages.sh b/bin/build-packages.sh index d60124c..17bfb4b 100755 --- a/bin/build-packages.sh +++ b/bin/build-packages.sh @@ -9,7 +9,7 @@ # headers out of anvil-zlib_*.ipk. The archives themselves are opkg-utils' # work; what is left here is laying out the tree it packages. # -# Needs no stock FlashForge package, unlike bin/patch.sh, so packaging stays +# Needs no stock FlashForge package, unlike bin/payload.sh, so packaging stays # runnable in CI on a bare checkout. set -euo pipefail . "$(dirname "$0")/common.sh" diff --git a/bin/build.sh b/bin/build.sh index 44e1993..15af2ad 100755 --- a/bin/build.sh +++ b/bin/build.sh @@ -4,5 +4,5 @@ set -euo pipefail SCRIPT_DIR="$(dirname "$0")" "$SCRIPT_DIR/fetch-assets.sh" "$SCRIPT_DIR/unpack.sh" -"$SCRIPT_DIR/patch.sh" +"$SCRIPT_DIR/payload.sh" "$SCRIPT_DIR/pack.sh" "$@" diff --git a/bin/common.sh b/bin/common.sh index c831fd1..62c66f1 100644 --- a/bin/common.sh +++ b/bin/common.sh @@ -95,7 +95,7 @@ LIBARCHIVE_TGZ="${LIBARCHIVE_TGZ:-$ROOT/vendor/libarchive-${LIBARCHIVE_VERSION:- export SODIUM_TGZ OPKG_TGZ LIBARCHIVE_TGZ # Output paths are derived by pkgs/lib.sh's pkg_out; these three aliases remain -# only because patch.sh and fetch-assets.sh name them. Add nothing here. +# only because payload.sh and fetch-assets.sh name them. Add nothing here. SODIUM_BUILD="${SODIUM_BUILD:-$ROOT/work/pkg/libsodium}" OPKG_BUILD="${OPKG_BUILD:-$ROOT/work/pkg/opkg}" ZLIB_BUILD="${ZLIB_BUILD:-$ROOT/work/pkg/zlib}" diff --git a/bin/fetch-assets.sh b/bin/fetch-assets.sh index 2687dc6..b17c88b 100755 --- a/bin/fetch-assets.sh +++ b/bin/fetch-assets.sh @@ -165,7 +165,7 @@ fi fi ) say "cached opkg-utils $OPKG_UTILS_VERSION ($OPKG_UTILS_COMMIT)" -# The toolchain, on the pkg_needs condition that decides whether patch.sh has +# The toolchain, on the pkg_needs condition that decides whether payload.sh has # to compile at all. Wrong here is not a slow build but a stopped one. if [ "$ALL" = 1 ] \ || pkg_needs; then diff --git a/bin/pack.sh b/bin/pack.sh index 6655df4..19b8ca9 100755 --- a/bin/pack.sh +++ b/bin/pack.sh @@ -1,11 +1,27 @@ #!/usr/bin/env bash -# 3/3 -- repack work/software into an installable USB package. -# ./bin/pack.sh [--full|--plain] default is slim: software component only +# 3/3 -- build the installable USB package. +# ./bin/pack.sh [--full|--plain] default is slim: our installer + payload # -# Slim works because the stock installer skips any absent component -- every -# update_ guards on `ls -1t -*.tar.xz` -- so the kernel, rootfs and -# MCU firmware are left untouched. start.img, end.img and play still ship: -# runFirmwareExe.sh uses them unconditionally. +# SLIM SHIPS NO FLASHFORGE COMPONENT AT ALL -- not the kernel, not the rootfs, +# not the MCU firmware, and since we own the installer, not the 26MB software +# component either. Two files do the work: +# +# runFirmwareExe.sh ours; app_startup.sh runs it, and it IS the install +# anvil.tar.xz the payload, everything under $MODDIR +# +# plus start.img, end.img and play, which are what the owner sees and hears +# while it runs. +# +# Shipping no software component is what keeps app_startup.sh's own recovery +# working: /usr/prog/PROGRAM/software goes on holding FlashForge's version +# directory and their firmwareExe inside it, so the five-second watchdog there +# has a working UI to restore when an install goes wrong. Nothing in that +# component was ever ours to ship -- their klippy tree, their wifi module and +# their configs, going straight back to the partition they were already on. +# +# --full carries kernel/control/library verbatim, and the installer installs +# any component a package does carry: absent ones are skipped, exactly as +# FlashForge's own `ls -1t -*.tar.xz` guard did. set -euo pipefail . "$(dirname "$0")/common.sh" @@ -19,9 +35,10 @@ for a in "$@"; do esac done -[ -d work/software ] || { echo "run ./bin/unpack.sh first" >&2; exit 1; } -STOCK_SW_VER=$(cat work/.stock_sw_ver) -OUT_VER="${SW_VER:-$STOCK_SW_VER}" +# work/outer, not work/software: no component is shipped, but start.img, +# end.img, play and (with --full) the other three components still come out of +# the stock package bin/unpack.sh opened. +[ -d work/outer ] || { echo "run ./bin/unpack.sh first" >&2; exit 1; } rm -rf work/stage work/out mkdir -p work/stage work/out @@ -35,24 +52,7 @@ else PW_AUTO=0 fi -# --- 1. md5sum.list -- the installer hard-gates on it. Paths must be "./rel", -# and the list must not contain itself. -echo ">> regenerating md5sum.list" -( cd work/software - rm -f md5sum.list - find . -type f ! -name md5sum.list -print0 \ - | sort -z \ - | xargs -0 md5sum > md5sum.list - echo " $(wc -l < md5sum.list) entries" ) - -# --- 2. software-.tar.xz, NOT actually xz. FlashForge's own components -# are plain tars carrying a .tar.xz name and the installer runs a bare -# `tar -xvf`, so a real xz file does not install. -echo ">> building software-$OUT_VER.tar.xz (plain tar, matching stock)" -tar -cf "work/stage/software-$OUT_VER.tar.xz" -C work/software . -ls -lh "work/stage/software-$OUT_VER.tar.xz" | awk '{print " "$5}' - -# --- 3. the rest of the outer package +# --- 1. the outer package # # Which model this package installs on. Resolved HERE rather than at emit time # because two things need it now: the gate baked into our installer, and the @@ -82,23 +82,63 @@ fi # this name, so owning the name is all it takes to own the install -- see the # header of installer/runFirmwareExe.sh for the contract and the exit codes. # -# Three lines are rewritten rather than being config the script reads: it runs +# Four lines are rewritten rather than being config the script reads: it runs # on a printer with nothing beside it but the package it came in, so the gate # and the password mode have to be IN it. The `^NAME=` shape is what # bin/unpack.sh and tools/replica/printer/entrypoint.sh read back. echo ">> generating runFirmwareExe.sh ($OUT_MACHINE/$OUT_PID, pw-auto=$PW_AUTO)" +# SINGLE-QUOTED in the generated file, and that is not style. A crypt hash is +# `$6$salt$...`, and an unquoted assignment would have the printer's shell +# expand $6 as a positional parameter and ship a truncated hash -- a printer +# nobody can log into. Crypt output is [./A-Za-z0-9$] so it can never contain +# the quote that would close it early. +# +# `|` as the sed delimiter for the same reason: a hash is full of `/`. & and \ +# are escaped in case ROOT_PW_HASH is ever something stranger than a hash. +_pw_esc=$(printf '%s' "${ROOT_PW_HASH:-}" | sed -e 's/[\\&|]/\\&/g') sed -e "s/^MACHINE=.*/MACHINE=$OUT_MACHINE/" \ -e "s/^PID=.*/PID=$OUT_PID/" \ -e "s/^MOD_PW_AUTO=.*/MOD_PW_AUTO=$PW_AUTO/" \ + -e "s|^MOD_ROOT_PW_HASH=.*|MOD_ROOT_PW_HASH='$_pw_esc'|" \ installer/runFirmwareExe.sh > work/stage/runFirmwareExe.sh chmod +x work/stage/runFirmwareExe.sh # The substitutions are not optional: a package whose gate still says -# Creator5Pro because a sed missed would install on the wrong machine. -for _want in "MACHINE=$OUT_MACHINE" "PID=$OUT_PID" "MOD_PW_AUTO=$PW_AUTO"; do - grep -qx "$_want" work/stage/runFirmwareExe.sh || { - echo "runFirmwareExe.sh has no '$_want' line -- the sed above missed" >&2 +# Creator5Pro because a sed missed would install on the wrong machine, and one +# whose password line did not take would ship an unreachable printer. +for _want in "MACHINE=$OUT_MACHINE" "PID=$OUT_PID" "MOD_PW_AUTO=$PW_AUTO" \ + "MOD_ROOT_PW_HASH='${ROOT_PW_HASH:-}'"; do + grep -qxF "$_want" work/stage/runFirmwareExe.sh || { + echo "runFirmwareExe.sh has no '${_want%%=*}=' line as expected -- the sed above missed" >&2 exit 1; } done +# It must also still PARSE after the substitutions: a hash with a stray +# character in it would otherwise reach a printer as a syntax error, and the +# only symptom would be a package that installs nothing. +sh -n work/stage/runFirmwareExe.sh || { + echo "the generated runFirmwareExe.sh does not parse" >&2; exit 1; } + +# FF_STOCK_PW_HASH is how the installer tells "nobody has set a root password +# on this printer" from "someone has". It is hardcoded there, and this is the +# one moment a build can check it: work/software is the stock component this +# package was built from, and its shadow is where the constant came from. +# +# Getting it wrong is silent and total -- every printer reads as "already set", +# none is ever given a password, and nobody can ssh in -- so a mismatch fails +# the build rather than shipping. +if [ -f work/software/shadow ]; then + _ff_live=$(awk 'BEGIN{FS=":"} $1=="root"{print $2}' work/software/shadow) + _ff_baked=$(sed -n "s/^FF_STOCK_PW_HASH='\(.*\)'$/\1/p" installer/runFirmwareExe.sh) + if [ "$_ff_live" != "$_ff_baked" ]; then + echo "FF_STOCK_PW_HASH is stale: installer/runFirmwareExe.sh has" >&2 + echo " $_ff_baked" >&2 + echo " but this stock package's shadow has" >&2 + echo " $_ff_live" >&2 + echo " Update the constant. Until you do, no printer on this firmware" >&2 + echo " would ever be given a root password -- every one would read as" >&2 + echo " 'password already set' and ssh would stay shut." >&2 + exit 1 + fi +fi for f in start.img end.img play; do [ -f "work/outer/$f" ] && cp -f "work/outer/$f" work/stage/ done @@ -109,12 +149,12 @@ if [ "$SLIM" = "0" ]; then [ -f "$f" ] && cp -f "$f" work/stage/ done else - echo ">> slim: software component only -- kernel, rootfs and MCU untouched" + echo ">> slim: installer + payload only -- no FlashForge component is reflashed" fi echo ">> outer payload:" ls -la work/stage | sed 's/^/ /' -# --- 4. emit +# --- 2. emit BASE="${MOD_NAME:-anvil}-${MOD_VER:?}" if [ "$PLAIN" = "1" ]; then diff --git a/bin/patch.sh b/bin/patch.sh deleted file mode 100755 index 57d5439..0000000 --- a/bin/patch.sh +++ /dev/null @@ -1,172 +0,0 @@ -#!/usr/bin/env bash -# 2/3 -- apply the mods to work/software/. Idempotent. -set -euo pipefail -. "$(dirname "$0")/common.sh" -. "$ROOT/pkgs/lib.sh" - -SOFTWARE_DIR=work/software -[ -d "$SOFTWARE_DIR" ] || { echo "run bin/unpack.sh first" >&2; exit 1; } - -# Counting .ipk, not testing the directory: a cleaned tree leaves it empty. -if [ -z "$(ls "$PKG_FEED"/*.ipk 2>/dev/null)" ]; then - echo "no package feed at $PKG_FEED" >&2 - echo " the recipes this script runs build against it -- run 'make packages' first." >&2 - exit 1 -fi - -say() { printf '>> %s\n' "$*"; } -skip() { printf ' (skip) %s\n' "$*"; } - -# Built OUTSIDE the software component: /usr/prog is on the firmware partition -# and would overflow on ~100MB of Mainsail and HelixScreen. -rm -rf "$PAYLOAD_ROOT" "$SOFTWARE_DIR/mod" - -# --- 0. the payload, installed -# Installed by the printer's own opkg inside the replica, so `opkg -# list-installed` answers what a release installs off the payload itself. -say "payload: installing the feed with the printer's own opkg" - -# Not model-specific; TARGET_MACHINE names only the OUTPUT file, because -# runFirmwareExe.sh refuses a package whose machine is not its own. - -# The roots, not the closure: Depends brings the rest, so the metadata is -# exercised on every build. A root with no .ipk is skipped -- that is how the -# PKG_WHEN gates reach here -- but a missing DEPENDENCY is still opkg's error. -# The two loose python packages are Recommends, which opkg has no field for. -# -# greenlet and cffi are deliberately NOT here: the klipper service execs -# $FF_PYTHON, so they are ordinary Depends of anvil-klipper now. Listed here -# they were installed by every build and by no `opkg install anvil-klipper`, -# which is the one command that has to work on a printer. -MOD_ROOTS="anvil-core anvil-opkg anvil-s6-rc anvil-klipper - anvil-moonraker anvil-python-pillow anvil-python-preprocess-cancellation - anvil-mainsail anvil-helixscreen anvil-busybox" - -# Named, not versioned -- except anvil-core, whose PKG_VERSION is MOD_VER, so a -# feed built yesterday would install yesterday's anvil-core. -[ -f "$(pkg_ipk anvil-core)" ] || pkg_die \ - "the feed has no $(basename "$(pkg_ipk anvil-core)") -- rerun ./bin/build-packages.sh" - -MOD_INSTALL="" -for _p in $MOD_ROOTS; do - for _f in "$PKG_FEED/${_p}_"*.ipk; do - if [ -f "$_f" ]; then MOD_INSTALL="$MOD_INSTALL $_p"; fi - break - done -done - -# The machine's own opkg installs the feed onto its own filesystem: needs a -# privileged container, hence `make build`'s docker lane. -# shellcheck disable=SC2086 -./bin/build-payload.py $MOD_INSTALL - -say "payload: $(grep -c '^Package:' "$PAYLOAD_DIR/var/lib/opkg/status") packages installed (both chamber configs; the printer picks)" - -# --- 1. Klipper -# Nothing is staged here any more: anvil-klipper installs the whole klippy tree -# at $MODDIR/klipper/klippy and the klipper s6-rc service execs it there on our -# own $FF_PYTHON, so the package IS the printer's Klipper. /usr/prog/klipper -# keeps FlashForge's stock klippy, unread -- the only file still read there is -# klipper_pri.sh, their SCHED_FIFO helper. klipperDaemon at that path is a -# symlink to our shim, pointed there by anvil-link-prog.sh. -# -# Nothing is asserted here either. The two gates that were -- a klippy tree at -# $MODDIR/klipper/klippy, and a c_helper.so inside it -- asked whether opkg had -# just installed the package it was handed, of a staging root three lines after -# it was filled. qa/replica/test_install.py::test_klippy_is_present asks it of -# an installed filesystem, and bin/verify.sh asks it of the packed artefact, so -# the same absence is already caught twice on the two vehicles that reach a -# printer. A gate here could only fail on a build that both of those fail on. - -# --- 2. Toolchanger -# Gone with section 1: the ff_*.py ride the klippy tree anvil-klipper installs -# and the ff-*.cfg are anvil-klipper-config's, so BUILD_TOOLCHANGE is answered -# entirely by that recipe's PKG_WHEN. - -# --- 3. the user's seams -# In the payload and in NO package: a package member is overwritten on every -# upgrade by definition, and this is the user's own file. -[ -f pkgs/moonraker/seed/moonraker-custom.conf ] \ - && cp -f pkgs/moonraker/seed/moonraker-custom.conf "$PAYLOAD_DIR/config/moonraker-custom.conf" - -# moonraker.conf ships with pkgs/moonraker, not anvil-core: unconditional, a -# BUILD_MOONRAKER=0 build overwrote the config of a server it did not install. - -# --- 5b-2. the s6-rc database -# Compiled in the replica by case-build-payload.sh, with the s6-rc-compile -# anvil-s6-rc ships, straight after opkg installs the feed. It used to be built -# here by a second NATIVE build of the same four tarballs, cached in -# work/.s6-native -- 110 lines whose entire hazard was that both stacks had to -# be --prefix=$MODDIR or the #! baked into the oneshot runner pointed at the -# build host's execline. Compiling with the compiler we SHIP cannot get that -# wrong. (Byte-identical either way: 80-s6-migration.md, measured 2026-08-28.) -# -# So there is nothing left to assert. compiled/current resolving was gated here -# and is gated in bin/verify.sh, which also reads the one thing a host can tell -# about the database -- the execline the oneshot runner's #! names. On a -# machine, qa/replica/test_s6rc.py boots the thing: s6-rc-init reads the -# database, the boot set comes up, and a database moved aside is reported. - -# --- 6. SSH -# Nothing to install: the stock rootfs ships dropbear and an enabled -# S50dropbear, so ssh is already listening. Only the password is missing. -if [ -n "${ROOT_PW_HASH:-}" ]; then - say "SSH: stock dropbear is already running; setting a known root password" -else - say "SSH: no ROOT_PW_HASH -- the installer will pick a random root password" - say " and write it to anvil-password.txt on the USB stick." - say " Set ROOT_PW_HASH to choose your own instead." -fi -# --- 7. root password -if [ -n "${ROOT_PW_HASH:-}" ]; then - say "Accounts: setting root password hash" - # /etc is a bind mount of /usr/prog/etc, so this is the live shadow dropbear - # reads at auth time, though it started from the read-only squashfs. - awk -v h="$ROOT_PW_HASH" 'BEGIN{FS=OFS=":"} $1=="root"{$2=h} {print}' \ - "$SOFTWARE_DIR/shadow" > "$SOFTWARE_DIR/shadow.new" && mv -f "$SOFTWARE_DIR/shadow.new" "$SOFTWARE_DIR/shadow" -else - skip "root password (set ROOT_PW_HASH)" -fi - -# --- 8. firmwareExe + start.sh -- anvil-core's, and NOT in the component -# Both install at $MODDIR/prog/ and anvil-link-prog.sh points the stock paths -# at them, on the flash path (run-append.sh) and on `opkg upgrade anvil-core` -# (postinst). Nothing execs either between run.sh copying the component and -# link-prog running, so the component copy bought no ordering -- and it cost: -# our wrapper with no payload never starts a UI, it logs and sleeps for ever, -# turning a failed install into an unbootable printer. Absent, app_startup.sh -# restores a firmwareExe from a version directory and the stock recovery works. -# Stock run.sh is set -x, not set -e, so its two cp lines are noisy no-ops. -say "firmwareExe + start.sh: dropped from the component (anvil-core owns both)" -rm -f "$SOFTWARE_DIR/firmwareExe" "$SOFTWARE_DIR/start.sh" - -# --- 10. the install manifest -# Every path this payload installs, so the next update deletes exactly what this -# one left behind -- a renamed init script leaves no stale twin. It names -# ITSELF, and is moved in from a temp file so `find` cannot list a -# half-written manifest. -MOD_MANIFEST=.install-manifest -{ ( cd "$PAYLOAD_DIR" && find . -mindepth 1 | sed 's|^\./||' ) - echo "$MOD_MANIFEST" -} | LC_ALL=C sort -u > work/.install-manifest -mv -f work/.install-manifest "$PAYLOAD_DIR/$MOD_MANIFEST" -say "install manifest: $(wc -l < "$PAYLOAD_DIR/$MOD_MANIFEST") paths -> $MODDIR/$MOD_MANIFEST" - -# --- 11. run.sh -# NOTHING, and this is where the mod's install used to reach the printer. Two -# marked blocks were spliced into FlashForge's own run.sh -- one after its -# WORK_DIR line so the backup caught the stock files, one before its final exit -# -- because the software component's run.sh was the ONLY place the stock -# updater would execute anything of ours. That is what the component was for. -# -# installer/runFirmwareExe.sh IS the installer now, run by app_startup.sh -# directly, and bin/pack.sh stages it. The splice, its idempotence markers and -# the component itself all went with it, and FlashForge's run.sh is left doing -# exactly what FlashForge wrote it to do -- on the --full packages that still -# carry a component to run it from. - -echo -echo "Patched." -echo " software component: $(du -sh "$SOFTWARE_DIR" | cut -f1) (-> /usr/prog, firmware partition)" -echo " mod payload: $(du -sh "$PAYLOAD_DIR" | cut -f1) (-> /usr/data/anvil, data partition)" -echo "Now run ./bin/pack.sh" diff --git a/bin/payload.sh b/bin/payload.sh new file mode 100755 index 0000000..3df98a3 --- /dev/null +++ b/bin/payload.sh @@ -0,0 +1,117 @@ +#!/usr/bin/env bash +# 2/3 -- build the mod payload from the package feed. Idempotent. +# +# Install the feed with the printer's own opkg, seed the user's +# moonraker-custom.conf, write the manifest. Everything it produces goes to +# $MODDIR on the data partition and nothing to /usr/prog, so it needs the +# package feed and no stock FlashForge package at all. +set -euo pipefail +. "$(dirname "$0")/common.sh" +. "$ROOT/pkgs/lib.sh" + +# Counting .ipk, not testing the directory: a cleaned tree leaves it empty. +if [ -z "$(ls "$PKG_FEED"/*.ipk 2>/dev/null)" ]; then + echo "no package feed at $PKG_FEED" >&2 + echo " the recipes this script runs build against it -- run 'make packages' first." >&2 + exit 1 +fi + +say() { printf '>> %s\n' "$*"; } +skip() { printf ' (skip) %s\n' "$*"; } + +# Everything goes to /usr/data: /usr/prog is the firmware partition and would +# overflow on ~100MB of Mainsail and HelixScreen. +rm -rf "$PAYLOAD_ROOT" + +# --- 1. the payload, installed +# Installed by the printer's own opkg inside the replica, so `opkg +# list-installed` answers what a release installs off the payload itself. +say "payload: installing the feed with the printer's own opkg" + +# Not model-specific; TARGET_MACHINE names only the OUTPUT file, because +# runFirmwareExe.sh refuses a package whose machine is not its own. + +# The roots, not the closure: Depends brings the rest, so the metadata is +# exercised on every build. A root with no .ipk is skipped -- that is how the +# PKG_WHEN gates reach here -- but a missing DEPENDENCY is still opkg's error. +# The two loose python packages are Recommends, which opkg has no field for. +# +# greenlet and cffi are deliberately NOT here: the klipper service execs +# $FF_PYTHON, so they are ordinary Depends of anvil-klipper now. Listed here +# they were installed by every build and by no `opkg install anvil-klipper`, +# which is the one command that has to work on a printer. +MOD_ROOTS="anvil-core anvil-opkg anvil-s6-rc anvil-klipper + anvil-moonraker anvil-python-pillow anvil-python-preprocess-cancellation + anvil-mainsail anvil-helixscreen anvil-busybox" + +# Named, not versioned -- except anvil-core, whose PKG_VERSION is MOD_VER, so a +# feed built yesterday would install yesterday's anvil-core. +[ -f "$(pkg_ipk anvil-core)" ] || pkg_die \ + "the feed has no $(basename "$(pkg_ipk anvil-core)") -- rerun ./bin/build-packages.sh" + +MOD_INSTALL="" +for _p in $MOD_ROOTS; do + for _f in "$PKG_FEED/${_p}_"*.ipk; do + if [ -f "$_f" ]; then MOD_INSTALL="$MOD_INSTALL $_p"; fi + break + done +done + +# The machine's own opkg installs the feed onto its own filesystem: needs a +# privileged container, hence `make build`'s docker lane. +# shellcheck disable=SC2086 +./bin/build-payload.py $MOD_INSTALL + +say "payload: $(grep -c '^Package:' "$PAYLOAD_DIR/var/lib/opkg/status") packages installed (both chamber configs; the printer picks)" + +# --- 2. Klipper +# Nothing to do, and nothing to assert. anvil-klipper installs the whole klippy +# tree at $MODDIR/klipper/klippy and the klipper s6-rc service execs it there on +# our own $FF_PYTHON, so the package IS the printer's Klipper. /usr/prog/klipper +# keeps FlashForge's stock klippy, unread -- the only file still read there is +# klipper_pri.sh, their SCHED_FIFO helper. klipperDaemon at that path is a +# symlink to our shim, pointed there by anvil-link-prog.sh. +# +# A gate here would ask whether opkg had installed the package it was just +# handed, of a staging root three lines after it was filled. +# qa/replica/test_install.py::test_klippy_is_present asks it of an installed +# filesystem instead, which is the vehicle that reaches a printer. + +# --- 3. the user's seams +# In the payload and in NO package: a package member is overwritten on every +# upgrade by definition, and this is the user's own file. +[ -f pkgs/moonraker/seed/moonraker-custom.conf ] \ + && cp -f pkgs/moonraker/seed/moonraker-custom.conf "$PAYLOAD_DIR/config/moonraker-custom.conf" + +# moonraker.conf ships with pkgs/moonraker, not anvil-core: unconditional, a +# BUILD_MOONRAKER=0 build overwrote the config of a server it did not install. + +# --- 4. the s6-rc database +# Compiled in the replica by case-build-payload.sh, with the s6-rc-compile +# anvil-s6-rc ships, straight after opkg installs the feed. Compiling with the +# compiler we SHIP is what makes the #! baked into the oneshot runner point at +# the execline we ship rather than the build host's -- a native second build of +# the same tarballs had to be held to --prefix=$MODDIR by hand to get that +# right. (Byte-identical either way: 80-s6-migration.md, measured 2026-08-28.) +# +# Nothing to assert here either. qa/replica/test_s6rc.py boots the database on +# a machine: s6-rc-init reads it, the boot set comes up, and a database moved +# aside is reported. + +# --- 5. the install manifest +# Every path this payload installs, so the next update deletes exactly what this +# one left behind -- a renamed init script leaves no stale twin. It names +# ITSELF, and is moved in from a temp file so `find` cannot list a +# half-written manifest. +MOD_MANIFEST=.install-manifest +{ ( cd "$PAYLOAD_DIR" && find . -mindepth 1 | sed 's|^\./||' ) + echo "$MOD_MANIFEST" +} | LC_ALL=C sort -u > work/.install-manifest +mv -f work/.install-manifest "$PAYLOAD_DIR/$MOD_MANIFEST" +say "install manifest: $(wc -l < "$PAYLOAD_DIR/$MOD_MANIFEST") paths -> $MODDIR/$MOD_MANIFEST" + +echo +echo "Payload built." +echo " mod payload: $(du -sh "$PAYLOAD_DIR" | cut -f1) (-> /usr/data/anvil, data partition)" +echo " nothing goes to /usr/prog: no software component is shipped." +echo "Now run ./bin/pack.sh" diff --git a/bin/unpack.sh b/bin/unpack.sh index e6ff5f4..08b7730 100755 --- a/bin/unpack.sh +++ b/bin/unpack.sh @@ -1,7 +1,12 @@ #!/usr/bin/env bash -# 1/3 -- decrypt the stock package and open the software component. +# 1/3 -- decrypt the stock package and open what a release needs from it. # ./bin/unpack.sh [] default: $STOCK_TGZ from config.env -# -> work/outer/ (the 8 top-level files), work/software/ (the tree to edit) +# -> work/outer/ the 8 top-level files. start.img, end.img and play ship; +# with FULL=1 so do kernel, control and library. +# -> work/software/ FlashForge's software component, opened but NOT shipped. +# bin/pack.sh reads two things out of it -- the printer.base.cfg +# to diff ours against, and the stock root hash the installer +# compares a live shadow to. set -euo pipefail . "$(dirname "$0")/common.sh" @@ -56,6 +61,6 @@ if [ -f "$STOCK_BASE" ] && [ -f "$OURS" ]; then fi echo -echo "Unpacked. Edit work/software/ then run ./bin/pack.sh" +echo "Unpacked. Now run ./bin/payload.sh, then ./bin/pack.sh" echo " stock software version: $STOCK_SW_VER" echo " files: $(find work/software -type f | wc -l)" diff --git a/config.env.example b/config.env.example index 5660ac2..5013ff7 100644 --- a/config.env.example +++ b/config.env.example @@ -7,12 +7,6 @@ MOD_VER="" # empty = today's UTC date, e.g. 20260823. # Pin it to rebuild an old release, or for a # second release on the same day (20260823b). -# Version stamped on the software component. Empty = keep the stock number -# (recommended). The installer wipes every other version dir before renaming -# its temp dir, so a collision is not a problem; a suffix like 1.9.7-mod is -# only useful if you want to see at a glance that a machine is modded. -SW_VER="" - # ---- which printer is this for? -------------------------------------------- # Creator5Pro / 0029 or Creator5 / 0028 # Find it on the printer: Settings -> About, or in diff --git a/docker/Dockerfile.build b/docker/Dockerfile.build index 3702fc0..a0d1a1b 100644 --- a/docker/Dockerfile.build +++ b/docker/Dockerfile.build @@ -8,7 +8,7 @@ # A HOST C compiler is in here as well, which looks redundant next to two # cross-toolchains that arrive as pinned tarballs. It is not: cross-building # CPython needs a build-python of exactly the same version to freeze modules -# and byte-compile the stdlib, so bin/patch.sh builds an x86-64 3.13 out of +# and byte-compile the stdlib, so bin/payload.sh builds an x86-64 3.13 out of # the same source tarball first, and that one needs the host's own gcc. # pkg-config and perl are for the same section -- CPython's configure probes # libffi through pkg-config, and OpenSSL's Configure is a perl program. @@ -21,7 +21,7 @@ # decompress data; zlib not available"). CPython records the module as missing # and carries on, so the symptom lands three steps later as "No module named # pip". The TARGET's zlib is nothing to do with this: that one is the static -# -fPIC libz.a bin/patch.sh cross-builds from the pinned tarball. +# -fPIC libz.a bin/payload.sh cross-builds from the pinned tarball. # # `patch` is in the list above for one package: lmdb's setup.py shells out to # /usr/bin/patch BY ABSOLUTE PATH to apply its own env-copy-txn.patch to the diff --git a/docs/building.md b/docs/building.md index 1c71417..38ddaa6 100644 --- a/docs/building.md +++ b/docs/building.md @@ -27,15 +27,17 @@ run flags. A build cannot reach the docker daemon. ``` fetch-assets.sh download Mainsail + HelixScreen + Moonraker into vendor/ -unpack.sh decrypt the stock .tgz, open the software component -patch.sh apply the mods to work/software/, and install the .ipk feed - into work/modpayload-root/ to make the payload -pack.sh regenerate md5sum.list, tar, encrypt → work/out/-anvil-.tgz +unpack.sh decrypt the stock .tgz for the boot images and the two facts + pack.sh reads out of FlashForge's software component +payload.sh install the .ipk feed into work/modpayload-root/ to make the + payload +pack.sh generate runFirmwareExe.sh, tar, encrypt + → work/out/-anvil-.tgz ``` `make build` runs all four. It needs the .ipk feed to exist first -- -`patch.sh` assembles the payload by installing it -- so `make packages` -comes before a cold `make build`; patch.sh says so if it is missing. Each is +`payload.sh` assembles the payload by installing it -- so `make packages` +comes before a cold `make build`; payload.sh says so if it is missing. Each is idempotent and safe to re-run. There is no host-side check of the built file. `bin/verify.sh` used to @@ -276,7 +278,7 @@ pkgs/helixscreen/ The service definitions stay with `anvil-core` and do NOT move to the components they start, which looks inconsistent and is not. They are one -source tree, and `bin/patch.sh` compiles the whole of it into a single s6-rc +source tree, and `bin/payload.sh` compiles the whole of it into a single s6-rc database on the build host. A tree split across packages would be a database assembled from whichever of them happened to be installed — and there is no compiler on the printer to assemble it with. @@ -286,7 +288,7 @@ compiler on the printer to assemble it with. ``` bin/ fetch-assets -> unpack -> patch -> pack. build-packages.sh is the packaging lane below, and the four - steps need it to have run: patch.sh checks for the feed and + steps need it to have run: payload.sh checks for the feed and refuses without one versions.env pinned Mainsail / HelixScreen / Moonraker versions + sha256 vendor/ where fetch-assets.sh caches them (gitignored), plus the @@ -315,7 +317,7 @@ pkgs/ package recipes: one directory per component, each a configure/make/install, the build cache. A recipe that needs something this cannot express should grow it -- qa/static/test_ipk.py fails a recipe that goes around it. - libsodium/ bin/patch.sh section 5d's build, moved. patch.sh runs it, + libsodium/ bin/payload.sh section 5d's build, moved. payload.sh runs it, so the payload's copy and the packaged copy are one build. opkg/ the package manager itself: static musl, with zlib and libarchive as build-only dependencies linked into it. This @@ -328,7 +330,7 @@ pkgs/ package recipes: one directory per component, each a qa/ THE SUITE. static/ needs nothing but a checkout; replica/ needs docker and the firmware. `make qa` tools/replica/ THE REPLICA, which is a build tool as much as a test one -- - bin/patch.sh assembles the payload inside it + bin/payload.sh assembles the payload inside it printer/ the machine: binfmt, the mount layout, its two Dockerfiles and the entrypoint that runs inside it on the printer's own binaries -- SHELL, because the printer's busybox ash is the @@ -355,7 +357,7 @@ what the suite proves. See [testing.md](testing.md#why-the-harness-is-python) for why the host half moved. Two things keep the boundary from eroding: the only files of ours that -`patch.sh` copies into a package are the ones under a recipe's `payload/`, +`payload.sh` copies into a package are the ones under a recipe's `payload/`, `prog/` or `seed/`, and `qa/replica/test_what_ships.py` fails if any file on the installed printer is byte-identical to one in `bin/` or `docker/`. diff --git a/docs/how-it-works.md b/docs/how-it-works.md index 334e2e9..8f1036b 100644 --- a/docs/how-it-works.md +++ b/docs/how-it-works.md @@ -23,34 +23,28 @@ The mod meets the stock printer in exactly **two files**, and owns both: * `runFirmwareExe.sh`, the installer. `app_startup.sh` runs whatever it finds under that name in the package it just decrypted, so owning the name is all it takes to own the install. Ours is `installer/runFirmwareExe.sh`; - `bin/pack.sh` bakes the model gate into it. It used to be FlashForge's own - file, with the mod's install spliced into the software component's `run.sh` - because that was the only place the stock updater would execute anything of - ours. + `bin/pack.sh` bakes the model gate and the password mode into it. * `/usr/prog/PROGRAM/software/firmwareExe`, the UI binary, replaced by a shell script that starts the supervisor. Because everything funnels through those two, `app_startup.sh`, `rcS` and the -whole init chain stay **completely stock and unpatched**. The genuine -`firmwareExe` is not kept anywhere: the installer wipes the software directory -before the new version lands, so nothing on the printer is a reliable backup of -it. Flashing the stock FlashForge package, which still ships it, is the -uninstall. +whole init chain stay **completely stock and unpatched**. -One property comes for free from keeping the binary's name and place: -`app_startup.sh`'s own watchdog greps `ps` for `firmwareExe` five seconds after -launch. The wrapper *is* `firmwareExe` and stays in the foreground, so the stock -watchdog supervises the mod correctly. +A release ships **no FlashForge component at all** — just the installer and the +payload — so nothing it does touches `/usr/prog/PROGRAM/software` except the +one symlink. Two things follow, and both are why it is worth doing: -Its other half does **not** currently work, and it is worth being exact about -why. When nothing called `firmwareExe` is running, `app_startup.sh` copies one -out of a version directory — `/usr/prog/PROGRAM/software//firmwareExe`. -That directory is the software component the last flash installed, and ours has -had `firmwareExe` deleted from it since we stopped shipping FlashForge's 21MB -binary. So the recovery finds nothing to copy, and a printer whose payload -failed to install shows a blank screen (ssh still answers — dropbear is stock -init's). Not shipping a software component at all fixes this, because then the -version directory is FlashForge's own and still holds their binary. +* `app_startup.sh`'s watchdog greps `ps` for `firmwareExe` five seconds after + launch. The wrapper *is* `firmwareExe` and stays in the foreground, so the + stock watchdog supervises the mod correctly. +* When nothing by that name is running, the same watchdog copies one out of the + version directory, `/usr/prog/PROGRAM/software//firmwareExe`. That + directory is still FlashForge's, with their genuine binary in it, so a + printer whose payload failed to install comes up on the stock UI instead of a + blank screen. A release that shipped its own component would have replaced + that directory and taken the recovery with it. + +Flashing the stock FlashForge package is still the uninstall. ## Services @@ -104,11 +98,10 @@ executable: moonraker dying on `libpython3.8.so.1.0: cannot open shared object file`. Everything the mod installs lives under `/usr/data/anvil` on the **data** -partition, which a FlashForge OTA cannot delete. The software component itself -goes to `/usr/prog` on the firmware partition, which keeps only one version -(the installer wipes every other version directory) and cannot fit ~100MB of -web UI — that is why Mainsail and HelixScreen ride in -the outer package instead. +partition, which a FlashForge OTA cannot delete. Nothing at all is installed to +`/usr/prog`: it is the firmware partition, it keeps only one version of a +component (the installer wipes every other version directory), and it could not +fit ~100MB of web UI in any case. ## Moonraker diff --git a/docs/qa-migration.md b/docs/qa-migration.md index b1b81b5..4c9c32f 100644 --- a/docs/qa-migration.md +++ b/docs/qa-migration.md @@ -60,7 +60,7 @@ the same shape `case-services.sh` used. Two things are wrong with it: produced. The real installer could break and nothing here would go red. - It can only place what a recipe keeps in the repo, so **anything the build produces is missing**. The cross-compiled s6 is the obvious one: it lives in `work/.s6` - and `bin/patch.sh` stages it into the package, so a hand-placed payload has + and `bin/payload.sh` stages it into the package, so a hand-placed payload has no supervisor at all — which is why the tests had to carry a stand-in scanner, and why one of them had to ask whether it was testing the stand-in before it could mean anything. @@ -197,7 +197,7 @@ it. "I could not find the tool" does not. **There are three skips, and they do not meet that bar.** `qa/static/test_ipk.py` reads `work/modpayload-root`, which only exists after -`bin/patch.sh` has run, and skips when it is absent. That is "this machine is +`bin/payload.sh` has run, and skips when it is absent. That is "this machine is not set up", which the rule above says must fail -- but making it fail would put a stock FlashForge package in the way of a lane whose whole point is that it needs nothing. @@ -336,7 +336,7 @@ it, producing a byte-identical servicedir.) **It is gone, and what it cost is worth stating.** It needed an `s6-rc-compile` that runs on the build host, and the only way to get one was a second NATIVE build of skalibs, execline, s6 and s6-rc -- 110 lines of -`bin/patch.sh` whose entire hazard was that both stacks had to be +`bin/payload.sh` whose entire hazard was that both stacks had to be `--prefix=$MODDIR` or the `#!` baked into the oneshot runner pointed at the build host's execline. The database is compiled in the replica now, by the `s6-rc-compile` the payload ships, so that hazard cannot exist. The price is @@ -463,7 +463,7 @@ Three gates were dropped rather than ported, as a coverage decision: |---|---|---| | the pytest gate | `run-tests.py` ran pytest and re-parsed its XML | none at the time -- `make test-py` and CI ran pytest directly; both went when `test/` was deleted | | the rootfs extraction | ran `unpack.sh` + `extract_rootfs` inline | none -- `make rootfs` does exactly this, and the replica lane already fails with that command in the message | -| **the packaging build on a synthetic stock package** | `make-stock-fixture.sh`, then `unpack`/`patch`/`pack`/`verify.sh` | **real: `bin/unpack.sh`, `patch.sh` and `pack.sh` now have no test at all.** `verify.sh` has since been retired outright -- see below | +| **the packaging build on a synthetic stock package** | `make-stock-fixture.sh`, then `unpack`/`patch`/`pack`/`verify.sh` | **real: `bin/unpack.sh`, `payload.sh` and `pack.sh` now have no test at all.** `verify.sh` has since been retired outright -- see below | That last row is the one to be uneasy about, and it is recorded here rather than buried in a commit message. The build path is what produces the `.tgz` a @@ -483,7 +483,7 @@ again, and should know two things it had learnt. `make_fixture` staged into mounts host-side, so a path under the build container's `/tmp` does not exist as far as the daemon is concerned. And the shape it reproduced -- the software component's `run.sh` ending in `tar -xf -$WORK_DIR/klipper/chelper.tar` -- is what `bin/patch.sh` depended on while the +$WORK_DIR/klipper/chelper.tar` -- is what `bin/payload.sh` depended on while the klippy tree still went out on the firmware partition; the tree lives at `$MODDIR/klipper` now and neither that section nor the `chelper.tar` exists. Recover both from git history rather than from a fresh reading of a stock package. @@ -560,7 +560,7 @@ And two things that were under `test/` and are not tests, now at - **`ffsim/`** -- 630 lines, down from 1,657, and no longer a test framework. `extract_rootfs` is what `make rootfs` runs; `Replica` is what - `make boot-screen-sim` and, more to the point, **`bin/patch.sh`** use -- + `make boot-screen-sim` and, more to the point, **`bin/payload.sh`** use -- the payload is assembled by running the printer's own `opkg` inside a replica, so this is build-path code that happened to live in the test tree. - **`printer/`** -- the `Dockerfile`, `entrypoint.sh`, `assemble.sh`, diff --git a/docs/testing.md b/docs/testing.md index cdee5a9..bfe4901 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -30,7 +30,7 @@ not yesterday's. | module | asks | |---|---| | `test_install.py` | what the machine's own installer produced: the stock boot chain is untouched and still parses, the wrapper is installed and starts the supervisor, every installed script is `sh -n`-clean under the printer's own busybox, klipper and the UI are services in the compiled database, klipper depends on the bring-up, the config include set is wired up, the user's `printer.cfg` survived, a pristine `backup/stock` was kept, and a boot with no stick does not go looking for an update | -| `test_abi.py` | the ABI gate, and the only one there is: every ELF object on the installed filesystem -- ours, the stock tree's and whatever `bin/patch.sh` staged -- is 32-bit little-endian MIPS, and every one the kernel's loader will handle is `nan2008/o32/mips32r2`. Two exemptions, both measured and both narrow: kernel modules are `ET_REL` and have no FP ABI to agree about, and FlashForge ships one inert ARM binary | +| `test_abi.py` | the ABI gate, and the only one there is: every ELF object on the installed filesystem -- ours, the stock tree's and whatever `bin/payload.sh` staged -- is 32-bit little-endian MIPS, and every one the kernel's loader will handle is `nan2008/o32/mips32r2`. Two exemptions, both measured and both narrow: kernel modules are `ET_REL` and have no FP ABI to agree about, and FlashForge ships one inert ARM binary | | `test_upgrade.py` | an update deletes what the last package shipped and **only** that -- the user's edits, their `moonraker-custom.conf`, their HelixScreen settings and anything nobody shipped all survive, over both the manifest path and the pre-manifest sweep | | `test_supervisor.py` | the s6 we cross-compiled: all 13 binaries load on the printer's kernel, and `s6-svwait -U` really waits for readiness rather than returning on the fork | | `test_boot_screen.py` | the first-boot screen renders on our own CPython 3.13, one screen's worth of correctly packed bytes, and degrades to "no screen" rather than raising when there is no panel | @@ -62,7 +62,7 @@ tests over our own Python, and they have no replacement. ## The replica is a tool, not a test `tools/replica/` builds the machine both suites run against, and the build -uses it too -- `bin/patch.sh` assembles the payload by running the printer's +uses it too -- `bin/payload.sh` assembles the payload by running the printer's own `opkg` inside it. It is not under `test/` because nothing in it is a test: ``` @@ -223,11 +223,11 @@ Tests that cannot fail are worse than no tests, because they read as coverage: mattered, by `runFirmwareExe.sh` refusing a foreign package -- which the replica lane exercises on every install. - `test-base-cfg` compared our `printer.base.cfg` against - `work/software/.../printer.base.cfg` — which `bin/patch.sh` overwrites - *with our own file* before the test reads it. It had been diffing our file - against itself: green on a cold tree, red on a second build, and - blind to real drift either way. The comparison moved into `bin/unpack.sh`, - where a pristine stock tree actually exists. + `work/software/.../printer.base.cfg` — which the build overwrote *with our + own file* before the test read it. It had been diffing our file against + itself: green on a cold tree, red on a second build, and blind to real + drift either way. The comparison moved into `bin/unpack.sh`, where a + pristine stock tree actually exists. - `test-applets`' command-word scan only extracted the first word of a simple, unprefixed command: `if timeout 5 foo; then` yielded `['if','echo','fi']`, so the one failure its docstring named was invisible diff --git a/docs/upgrading.md b/docs/upgrading.md index e6000f2..5c0bc56 100644 --- a/docs/upgrading.md +++ b/docs/upgrading.md @@ -62,8 +62,9 @@ your copy, landing the new version beside it as `.mod-new`. Everything the mod installs is under `/usr/data/anvil` on the data partition, the klippy tree and its extras included, so an OTA cannot delete them. What it -does rewrite is the software component the mod boots from — so re-flash the mod -after a firmware update. The `#*#` block in `printer.cfg` survives either way. +does rewrite is `/usr/prog`, and with it the `firmwareExe` the mod boots +from — so re-flash the mod after a firmware update. The `#*#` block in +`printer.cfg` survives either way. --- diff --git a/installer/runFirmwareExe.sh b/installer/runFirmwareExe.sh index 17a3735..8e0972d 100644 --- a/installer/runFirmwareExe.sh +++ b/installer/runFirmwareExe.sh @@ -17,23 +17,13 @@ # up on whatever it had before, which is the right answer to "this package is # not for this machine". # -# WHY OURS. FlashForge's version of this file drives four component installers -# (control, kernel, software, library), each of which extracts a tarball into -# /usr/prog/PROGRAM//, verifies md5sum.list, wipes every previous version -# directory and runs the run.sh inside it. Our install used to reach the printer -# by being spliced into the software component's run.sh -- which meant shipping -# a 26MB component, 21MB of it FlashForge's own firmwareExe, for the sake of one -# exec hook. We own this file instead. See docs/how-it-works.md. +# It does two jobs. FlashForge's four components (control, kernel, software, +# library) are installed when a package carries them -- a --full one does -- +# and then the mod payload is installed on top. A slim package carries no +# component at all and install_component skips each in turn, which is what lets +# a release leave /usr/prog entirely alone. # -# It still installs the stock components when a package carries them, because -# --full packages do. A slim package carries none and the loop skips them, -# exactly as FlashForge's own `ls -1t -*.tar.xz` guard did. -# -# WHAT WENT AND IS NOT COMING BACK: create_key() (defined, never called, and it -# built its paths without a slash, so it would have written /usr/progprivate.pem); -# SSID_NAME/SSID_PASS (assigned, never read); the commented-out kernel version -# gate; and the screwflag factory bypass for the model check, which we never -# ship a screwflag for. +# See docs/how-it-works.md for where this sits in the boot chain. # # This runs under the printer's busybox ash. qa/static/test_shell_syntax.py # parses it with that in mind; keep it dialect-clean. @@ -57,9 +47,10 @@ PID=0029 # rewrite the replica's shadow and drop a password file on /mnt every run. MOD_PW_AUTO=0 -# Where the software component landed, once it has. Only the root password -# block at the bottom reads it, and only when a package carries the component. -SW_COMPONENT_DIR="" +# Rewritten by bin/pack.sh from ROOT_PW_HASH, and empty in the checkout for the +# same reason MOD_PW_AUTO is 0. This is how a baked-in hash reaches the +# printer: it is applied to /usr/prog/etc/shadow on the machine, below. +MOD_ROOT_PW_HASH='' # ---------------------------------------------------------------- the gates -- # These print to the console rather than to the log: they run before there is @@ -138,7 +129,6 @@ install_component() { done mv "$dest/temp" "$dest/$version" sync - [ "$name" = software ] && SW_COMPONENT_DIR="$dest/$version" if [ -f "$dest/$version/run.sh" ]; then chmod a+x "$dest/$version/run.sh" "$dest/$version/run.sh" @@ -147,9 +137,11 @@ install_component() { } # ------------------------------------------------------ backups, taken first -- -# BEFORE any component is installed: the software component ships /usr/prog's -# passwd and shadow and its run.sh copies them over the live ones, so a backup -# taken afterwards would capture the package's files rather than the printer's. +# BEFORE any component is installed. A slim package carries none and nothing +# here would overwrite these anyway, but a --full package carries FlashForge's +# own and their run.sh copies passwd and shadow over the live ones -- a backup +# taken afterwards would then capture the package's files rather than the +# printer's. # # app_startup.sh is deliberately not in this list. The mod replaces firmwareExe # instead, so the stock boot scripts are never modified and never need @@ -174,18 +166,9 @@ if [ ! -d "$MODDIR/backup/stock" ]; then cp -a "$BACKUP" "$MODDIR/backup/stock" && echo "kept pristine copy at backup/stock" fi -# The root password the printer has RIGHT NOW -- random from a first install, -# or set by hand with `passwd` -- lives only in this shadow, and the software -# component is about to replace the file. Record the hash so the block at the -# bottom can put it back: that is what keeps the password stable across -# updates. +# A password hash left on disk by installs that predate this one. Nothing reads +# it any more; do not leave a secret lying in $MODDIR. rm -f "$MODDIR/.prev-root-hash" -if [ -f /usr/prog/etc/shadow ]; then - awk 'BEGIN{FS=":"} $1=="root"{print $2}' /usr/prog/etc/shadow \ - > "$MODDIR/.prev-root-hash" 2>/dev/null && - chmod 600 "$MODDIR/.prev-root-hash" && - echo "recorded current root password hash" -fi sync # FlashForge clears their own NIM logs on every flash. One line, and it leaves @@ -258,7 +241,7 @@ if [ -n "$MODTAR" ]; then # the update and sits next to the one that replaced it. The installed # set must end up exactly the shipped set. # - # bin/patch.sh ships a manifest of every path the payload installs -- + # bin/payload.sh ships a manifest of every path the payload installs -- # one per line, relative to $MODDIR, files and directories both, itself # included. Deleting what the LAST manifest lists keeps that property # (a script the last payload shipped and this one does not is named in @@ -283,7 +266,7 @@ if [ -n "$MODTAR" ]; then # loop runs as root with $MODDIR pasted onto the front of # whatever it says. An absolute path escapes $MODDIR # outright; `..` walks out of it one component at a time. - # Neither can come out of bin/patch.sh, so one appearing + # Neither can come out of bin/payload.sh, so one appearing # here means the file is damaged or forged, and the answer # is to say so and skip rather than to find out what it # would have deleted. The `..` test is deliberately blunt @@ -541,53 +524,56 @@ find $MODDIR/klipper/klippy -name '__pycache__' -type d -exec rm -rf {} + 2>/dev find /usr/prog/klipper/klippy -name '__pycache__' -type d -exec rm -rf {} + 2>/dev/null sync -# ---- root password, chosen here when none was baked in --------------------- -# The normal path is ROOT_PW_HASH at build time: patch.sh writes that hash -# straight into the shadow file the software component ships. When it is empty -# the package cannot carry a password -- one file is flashed by many people, so -# any baked-in default would be the SAME password on every printer. Pick a -# random one here instead, on the machine, and write it onto the USB stick we -# are being flashed from. Every printer gets a different password and it is +# ---- the root password ------------------------------------------------------ +# Two ways in, and neither writes a shadow file into a package any more: this +# script edits /usr/prog/etc/shadow on the machine. (/etc is a bind mount of +# /usr/prog/etc, so that IS the live file dropbear authenticates against.) +# +# BAKED. ROOT_PW_HASH at build time, put here by bin/pack.sh. Applied on every +# flash, which is what the old path did too -- the hash rode in the software +# component's shadow and the component's run.sh copied it over the live file +# every time. Someone who bakes a hash into their own build is saying which +# password their printers have. +# +# RANDOM. An empty ROOT_PW_HASH cannot ship a password at all: one file is +# flashed by many people, so a baked-in default would be the SAME password on +# every printer. Pick a random one here, on the machine, and write it onto the +# USB stick being flashed from. Every printer gets a different one and it is # never guessable from anything printed on the case. # # ORDER MATTERS: the password goes onto the stick and is read back BEFORE the # printer starts accepting it. A password that was set but never landed on the -# stick is a locked-out printer, so if the write fails we change nothing at -# all and say so. +# stick is a locked-out printer, so if the write fails we change nothing at all +# and say so. # -# An UPDATE must not change the password. The backup block at the top recorded -# the hash the printer had before the software component replaced the shadow -# file; when that hash differs from the one the package shipped, a password was -# already set -- by a previous install, or by hand with `passwd` -- and it is -# put back unchanged. The stick only ever sees a password once, on the first -# install. -PW_KEEP="" -if [ "$MOD_PW_AUTO" = "1" ] && [ -f $MODDIR/.prev-root-hash ]; then - PREV_HASH=`cat $MODDIR/.prev-root-hash 2>/dev/null` - # Compare against the hash the PACKAGE ships, not the live file: the - # software component has already copied its shadow over the live one by - # now, and if that copy ever fails the live file still holds the old - # password -- which must read as "set by someone", not as "fresh". - SHIPPED_HASH="" - [ -n "$SW_COMPONENT_DIR" ] && [ -f "$SW_COMPONENT_DIR/shadow" ] && - SHIPPED_HASH=`awk 'BEGIN{FS=":"} $1=="root"{print $2}' "$SW_COMPONENT_DIR/shadow" 2>/dev/null` - [ -n "$SHIPPED_HASH" ] || - SHIPPED_HASH=`awk 'BEGIN{FS=":"} $1=="root"{print $2}' /usr/prog/etc/shadow 2>/dev/null` - case "$PREV_HASH" in - '$'*) [ "$PREV_HASH" != "$SHIPPED_HASH" ] && PW_KEEP="$PREV_HASH" ;; - esac -fi -rm -f $MODDIR/.prev-root-hash -if [ -n "$PW_KEEP" ]; then - if awk -v h="$PW_KEEP" 'BEGIN{FS=OFS=":"} $1=="root"{$2=h} {print}' \ +# AN UPDATE MUST NOT REROLL IT, and the test for that is one comparison. Root +# has a real password on a stock printer -- FlashForge ship the hash below, the +# same one on every machine, they just never published what it unlocks -- so +# the question is not "does root have a hash" but "is it still THEIRS". If it +# is, nobody has set one, by us or by hand with `passwd`, and it is ours to +# set. If it is not, someone has, and it is not ours to touch. +# +# Taken from the shadow in the stock software component +# (software-1.9.7.tar.xz, Creator5Pro). If FlashForge ever change it, a printer +# on the new firmware reads as "password already set" and never gets one -- +# bin/pack.sh compares this against the stock package it was built from and +# fails the build rather than let that ship silently. +FF_STOCK_PW_HASH='$1$ax/gSlz5$poL89lSQB9./7fUZwc3ej/' + +LIVE_HASH=`awk 'BEGIN{FS=":"} $1=="root"{print $2}' /usr/prog/etc/shadow 2>/dev/null` + +if [ -n "$MOD_ROOT_PW_HASH" ]; then + if awk -v h="$MOD_ROOT_PW_HASH" 'BEGIN{FS=OFS=":"} $1=="root"{$2=h} {print}' \ /usr/prog/etc/shadow > /usr/prog/etc/shadow.new && mv -f /usr/prog/etc/shadow.new /usr/prog/etc/shadow; then chmod 600 /usr/prog/etc/shadow - echo "root password preserved from the previous install" + echo "root password set (the hash baked into this build)" else - echo "!! could not restore the previous root password hash" - echo "!! root password is the stock one -- no ssh login" + echo "!! could not write the baked root password hash" + echo "!! root password is whatever it was -- ssh may not work" fi +elif [ "$MOD_PW_AUTO" = "1" ] && [ "$LIVE_HASH" != "$FF_STOCK_PW_HASH" ]; then + echo "root password already set on this printer -- left alone" elif [ "$MOD_PW_AUTO" = "1" ]; then NEW_PASSWORD=`tr -dc A-Za-z0-9 < /dev/urandom 2>/dev/null | head -c 14` NEW_HASH="" diff --git a/pkgs/3rdparty/busybox/build.sh b/pkgs/3rdparty/busybox/build.sh index 59c9911..9b41ded 100755 --- a/pkgs/3rdparty/busybox/build.sh +++ b/pkgs/3rdparty/busybox/build.sh @@ -8,10 +8,10 @@ # out. A recipe that cross-compiled busybox would be a real dependency on a # real toolchain for a file that is optional by design. # -# WHY IT IS A PACKAGE AT ALL. It used to be a `cp` in bin/patch.sh, and it was +# WHY IT IS A PACKAGE AT ALL. It used to be a `cp` in bin/payload.sh, and it was # the ONE file in the payload that no package owned -- which cost three # separate things: an allowance in qa/static/test_ipk.py's "every file is -# owned" test, an ABI gate of its own in patch.sh because the payload-wide one +# owned" test, an ABI gate of its own in payload.sh because the payload-wide one # ran before the copy, and a file that .install-manifest listed but no # `opkg remove` could take away. As a package it is none of those; it is # absent or installed, like everything else. diff --git a/pkgs/3rdparty/busybox/pkg.conf b/pkgs/3rdparty/busybox/pkg.conf index 759eba3..4c2d048 100644 --- a/pkgs/3rdparty/busybox/pkg.conf +++ b/pkgs/3rdparty/busybox/pkg.conf @@ -9,7 +9,7 @@ PKG_NAME=anvil-busybox # ABSENT, NOT FAILED. With BUSYBOX_BIN unset -- which is every normal build, # including CI's -- pkg_recipes does not list this recipe, so nothing fetches, -# builds, indexes or installs it, and bin/patch.sh needs no line saying so: +# builds, indexes or installs it, and bin/payload.sh needs no line saying so: # its install list skips a root with no .ipk in the feed. The `-f` is part of # the condition rather than an error because a config.env carried between # machines routinely names a path that exists on one of them. diff --git a/pkgs/3rdparty/execline/pkg.conf b/pkgs/3rdparty/execline/pkg.conf index 4728dd3..5cc642b 100644 --- a/pkgs/3rdparty/execline/pkg.conf +++ b/pkgs/3rdparty/execline/pkg.conf @@ -2,7 +2,7 @@ # requires it. # # THIS PACKAGE EXISTS AGAINST A DOCUMENTED DECISION NOT TO SHIP IT, and the -# reversal is on the record rather than silent. bin/patch.sh built s6 with +# reversal is on the record rather than silent. bin/payload.sh built s6 with # --disable-execline and tools/supervisor/README.md called execline "53 more # binaries and 2.1MB" for nothing. Both were right at the time: our `run` # scripts are plain #!/bin/sh and nothing execed an execline program. diff --git a/pkgs/3rdparty/libsodium/build.sh b/pkgs/3rdparty/libsodium/build.sh index 9c47247..ba80d98 100755 --- a/pkgs/3rdparty/libsodium/build.sh +++ b/pkgs/3rdparty/libsodium/build.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # libsodium, cross-compiled for the printer into $SODIUM_BUILD. # -# One build and one vehicle: the .ipk this produces is what bin/patch.sh +# One build and one vehicle: the .ipk this produces is what bin/payload.sh # installs into the payload, so there is no second copy to drift from. # # WHOSE INTERPRETER DLOPENS IT: libnacl reaches libsodium through diff --git a/pkgs/3rdparty/libsodium/pkg.conf b/pkgs/3rdparty/libsodium/pkg.conf index d0f816b..0fb8787 100644 --- a/pkgs/3rdparty/libsodium/pkg.conf +++ b/pkgs/3rdparty/libsodium/pkg.conf @@ -19,7 +19,7 @@ PKG_RELEASE=1 PKG_SECTION=libs # The tree to package, laid out the way it should appear under $MODDIR -- -# which is exactly what build.sh already produces for bin/patch.sh to copy +# which is exactly what build.sh already produces for bin/payload.sh to copy # into the payload, so no restaging happens here. PKG_ROOT="$SODIUM_BUILD" diff --git a/pkgs/3rdparty/openssl/build.sh b/pkgs/3rdparty/openssl/build.sh index ba5630c..ea2e2c4 100755 --- a/pkgs/3rdparty/openssl/build.sh +++ b/pkgs/3rdparty/openssl/build.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # OpenSSL -- the one project here whose configure is not an autoconf configure. # -# THREE TRAPS, ALL OF THEM HIT FOR REAL, kept from bin/patch.sh section 5c: +# THREE TRAPS, ALL OF THEM HIT FOR REAL, kept from bin/payload.sh section 5c: # # * `no-docs` only exists from 3.1. On 3.0.x it is an "Unsupported options" # HARD ERROR, not a warning -- so it is not passed here. diff --git a/pkgs/3rdparty/python/pkg.conf b/pkgs/3rdparty/python/pkg.conf index 18096ec..6e47f1d 100644 --- a/pkgs/3rdparty/python/pkg.conf +++ b/pkgs/3rdparty/python/pkg.conf @@ -45,7 +45,7 @@ PKG_BUILD_DEPENDS="openssl sqlite zlib libffi xz bzip2 expat" # THE HEADERS ARE THE POINT OF THE DEV SPLIT HERE, more than anywhere else in # this feed. The eighteen pkgs/3rdparty/python-* recipes cross-compile C extensions # against them, and before this package existed they reached them by building -# inside bin/patch.sh at the one moment an untrimmed staging tree happened to +# inside bin/payload.sh at the one moment an untrimmed staging tree happened to # be on disk. A -dev package is that moment made durable: the headers are in # the feed, a wheel recipe names `python` as a build dependency and gets them, # and a printer -- which builds no extension modules -- installs none of it. diff --git a/pkgs/anvil-core/payload/anvil-env.sh b/pkgs/anvil-core/payload/anvil-env.sh index ee450df..d47af3a 100644 --- a/pkgs/anvil-core/payload/anvil-env.sh +++ b/pkgs/anvil-core/payload/anvil-env.sh @@ -112,7 +112,7 @@ export LD_LIBRARY_PATH # # Note what is NOT on PATH below: $MODDIR/bin is prepended (s6 needs it), and # the interpreter is in there, but it is called python3.13 and only that. -# bin/patch.sh deliberately drops the `python3` symlink CPython installs, so +# bin/payload.sh deliberately drops the `python3` symlink CPython installs, so # that adding our bin/ to PATH cannot quietly change what `python3` means for # every process that sources this file. FF_PYTHON=$MODDIR/bin/python3.13 diff --git a/pkgs/anvil-core/payload/etc/s6-rc/source/klipper/run b/pkgs/anvil-core/payload/etc/s6-rc/source/klipper/run index 03ca9f7..69bce1a 100755 --- a/pkgs/anvil-core/payload/etc/s6-rc/source/klipper/run +++ b/pkgs/anvil-core/payload/etc/s6-rc/source/klipper/run @@ -26,7 +26,7 @@ fi # installs, on the CPython 3.13 anvil-python installs. There is no /usr/prog in # this command line any more. # -# The tree used to be read from /usr/prog/klipper/klippy -- a copy bin/patch.sh +# The tree used to be read from /usr/prog/klipper/klippy -- a copy bin/payload.sh # staged into the software component out of this very package, so that # FlashForge's run.sh would place it on the firmware partition. Two copies of # one tree, and the one that ran was the one a stock flash could overwrite. diff --git a/pkgs/anvil-core/payload/prog/firmwareExe b/pkgs/anvil-core/payload/prog/firmwareExe index b6e9038..d691323 100755 --- a/pkgs/anvil-core/payload/prog/firmwareExe +++ b/pkgs/anvil-core/payload/prog/firmwareExe @@ -39,7 +39,7 @@ fi S6BIN=$MODDIR/bin # The s6 scandir: one live servicedir per service, laid down by s6-rc-init. SCANDIR=$MODDIR/etc/s6 -# The compiled database. `current` is a symlink bin/patch.sh points at the +# The compiled database. `current` is a symlink bin/payload.sh points at the # stamped directory it built, so this path never changes when the database does. S6DB=$MODDIR/etc/s6-rc/compiled/current # The scanner outlives the redirection above, so it gets a file of its own. A diff --git a/pkgs/anvil-core/payload/prog/start.sh b/pkgs/anvil-core/payload/prog/start.sh index eafc44c..ad44a59 100755 --- a/pkgs/anvil-core/payload/prog/start.sh +++ b/pkgs/anvil-core/payload/prog/start.sh @@ -10,7 +10,7 @@ # klipperDaemon would be a second way to start klippy: unsupervised, invisible # to s6-rc, and fighting the supervised copy for /dev/ttyS4. # -# bin/patch.sh stages this as a software component and FlashForge's own run.sh +# bin/payload.sh stages this as a software component and FlashForge's own run.sh # copies it onto /usr/prog/klipper, which is why it lives on the firmware # partition and why a stock flash replaces it. MODDIR=/usr/data/anvil diff --git a/pkgs/helixscreen/build.sh b/pkgs/helixscreen/build.sh index 03f4564..08c0eab 100755 --- a/pkgs/helixscreen/build.sh +++ b/pkgs/helixscreen/build.sh @@ -37,7 +37,7 @@ pkg_stage "$_src" "helixscreen" # An optional platform hook rides the same way: no hooks-creator5.sh is in the # repo, so a stock checkout ships nothing extra, and dropping one at # payload/helixscreen/assets/config/platform/ has it shipped with no edit -# here. Carried over from bin/patch.sh section 5 unchanged. +# here. Carried over from bin/payload.sh section 5 unchanged. cp -a "$PKG_DIR/payload/." "$PKG_WORK/stage$MODDIR/" pkg_ship "helixscreen" diff --git a/pkgs/helixscreen/pkg.conf b/pkgs/helixscreen/pkg.conf index 55cebc9..a7ee30c 100644 --- a/pkgs/helixscreen/pkg.conf +++ b/pkgs/helixscreen/pkg.conf @@ -6,7 +6,7 @@ # architecture and is gated like anything else that executes on the printer. # # THAT GATE IS NEW, and it is not in this recipe or in any build.sh. -# bin/patch.sh unpacks this tarball in section 5 and moves on, so the ELF +# bin/payload.sh unpacks this tarball in section 5 and moves on, so the ELF # objects in the largest thing we ship went unchecked for a long time. # qa/replica/test_abi.py now reads them off the installed filesystem, where # they are visible whichever route they took to get there. diff --git a/pkgs/klipper-config/pkg.conf b/pkgs/klipper-config/pkg.conf index c75815f..31072e0 100644 --- a/pkgs/klipper-config/pkg.conf +++ b/pkgs/klipper-config/pkg.conf @@ -45,7 +45,7 @@ PKG_EXCLUDE=".version" PKG_DEPENDS="" PKG_BUILD_DEPENDS="" -# The flag that turns the toolchanger off. It gates bin/patch.sh's install of +# The flag that turns the toolchanger off. It gates bin/payload.sh's install of # ff_*.py and our printer.base.cfg already; with these files in anvil-core it # did NOT gate the .cfg, so BUILD_TOOLCHANGE=0 shipped the config of a # disabled feature -- harmless, and exactly the kind of flag that reads as diff --git a/pkgs/klipper/build.sh b/pkgs/klipper/build.sh index 6edacf6..f49a064 100755 --- a/pkgs/klipper/build.sh +++ b/pkgs/klipper/build.sh @@ -11,7 +11,7 @@ # extruder_stepper_free shipped and bricked klippy startup on hardware. One # source verb means one tree, so the failure is unrepresentable here. # -# NO CACHE OF ITS OWN. bin/patch.sh kept a $FORK/.version stamp and an +# NO CACHE OF ITS OWN. bin/payload.sh kept a $FORK/.version stamp and an # `is the .so older than any .c` mtime test, both of which pkg_begin's stamp # replaces -- the stamp is the pinned commit and the toolchain filename, so a # bump to either rebuilds and nothing else does. @@ -26,7 +26,7 @@ pkg_unpack "$KLIPPER_TGZ" # GitHub's generated tarball wraps the repository in klipper-/, and the # unpack keeps that wrapper rather than hiding the shape of the archive behind # --strip-components -- the same choice pkgs/moonraker makes and for the same -# reason. The guard is bin/patch.sh's, kept: a tarball whose shape changed +# reason. The guard is bin/payload.sh's, kept: a tarball whose shape changed # under us would build nothing, which looks like a clean build and a printer # that cannot move. _top="klipper-$KLIPPER_VERSION" @@ -77,7 +77,7 @@ pkg_ship "klipper" # THE ABI -- o32/nan2008/mips32r2, which the kernel answers with ENOEXEC -- is # asked once now, of the installed filesystem, in qa/replica/test_abi.py. That # covers both vehicles this tree travels on (the .ipk and the SOFTWARE component -# bin/patch.sh stages) rather than only the one a build.sh can see. +# bin/payload.sh stages) rather than only the one a build.sh can see. # # THE SYMBOLS -- every function klippy cdefs, against the .so's dynamic symbol # table -- went with the test/ tree. cffi resolves lazily, so a .so built from diff --git a/pkgs/klipper/pkg.conf b/pkgs/klipper/pkg.conf index b5fc70f..0a1537a 100644 --- a/pkgs/klipper/pkg.conf +++ b/pkgs/klipper/pkg.conf @@ -1,7 +1,7 @@ # anvil-klipper -- the Klipper4FlashForge fork: the whole klippy tree, and # c_helper.so cross-compiled from the chelper sources shipped beside it. # -# THE LAST CROSS-BUILD THAT LIVED IN bin/patch.sh. Section 1 unpacked this +# THE LAST CROSS-BUILD THAT LIVED IN bin/payload.sh. Section 1 unpacked this # tarball, unpacked the Ingenic toolchain into work/.mips-toolchain, wrote its # own gcc line and stamped a cache directory only it and bin/fetch-assets.sh # knew the shape of -- every one of the things pkgs/lib.sh exists to do once. @@ -25,7 +25,7 @@ PKG_EXCLUDE=".version" # THE INTERPRETER IS A DEPENDS NOW. This block used to say the opposite -- # "the reason is where klippy runs rather than what it imports", the stock -# klipperDaemon starting klippy under FlashForge's 3.8.2 while bin/patch.sh +# klipperDaemon starting klippy under FlashForge's 3.8.2 while bin/payload.sh # copied the tree into the software component -- and it named phase 7 of # docs/notes/80-s6-migration.md as the change that would make it false. That # change has happened: etc/s6-rc/source/klipper/run execs $FF_PYTHON against @@ -40,7 +40,7 @@ PKG_EXCLUDE=".version" # the first and nothing here needs the second at import. # # WHAT THIS FIXES, and it is not bookkeeping. cffi and greenlet were installed -# by a hardcoded list in bin/patch.sh, so every build had them and no `opkg +# by a hardcoded list in bin/payload.sh, so every build had them and no `opkg # install anvil-klipper` did. pyserial and jinja2 were worse: they arrived # only as anvil-moonraker's Depends, so a BUILD_MOONRAKER=0 build shipped a # klippy that imports serialhdl and dies the first time it opens an MCU. @@ -48,7 +48,7 @@ PKG_EXCLUDE=".version" # anvil-klipper-config is not optional either: printer.base.cfg is the file # printer.cfg includes and every other config hangs off it, so a klippy tree # without it is a Klipper that cannot start. Naming any of these in -# bin/patch.sh's MOD_ROOTS instead would answer the dependency in the one +# bin/payload.sh's MOD_ROOTS instead would answer the dependency in the one # place a printer's own `opkg install anvil-klipper` never reads. PKG_DEPENDS="anvil-klipper-config, anvil-python, anvil-python-cffi, anvil-python-greenlet, anvil-python-pyserial, diff --git a/pkgs/lib.sh b/pkgs/lib.sh index 61951be..612ac32 100644 --- a/pkgs/lib.sh +++ b/pkgs/lib.sh @@ -31,10 +31,10 @@ # two of every library both worlds wanted, which a package feed cannot express # without lying about one of them. # -# Also sourced by bin/patch.sh, for pkg_out alone -- it stages what recipes +# Also sourced by bin/payload.sh, for pkg_out alone -- it stages what recipes # built and has to name where they put it. # -# Recipes run as their own process (both bin/patch.sh and bin/build-packages.sh +# Recipes run as their own process (both bin/payload.sh and bin/build-packages.sh # exec them), so no cross-compiler PATH or CC can leak into whatever runs next # and nothing below needs a subshell to prevent it. @@ -96,7 +96,7 @@ pkg_die() { printf ' !! %s\n' "$*" >&2; exit 1; } # the files themselves and the maintainer scripts that travel with them. An # edited postinst changes what the .ipk does and nothing else would notice -- # the version is a date and the payload bytes are unchanged, so the stamp -# would read "already current". seed/ is placed by bin/patch.sh, is in no +# would read "already current". seed/ is placed by bin/payload.sh, is in no # package, and must not invalidate one. # # TWO find CALLS AND AN `|| true`, not one find over both paths: most recipes @@ -930,7 +930,7 @@ pkg_pywheel() { # pkgs/3rdparty/python-lmdb, and that recipe found out the hard way. # # It replaced a single "at least twelve .so files in the payload" check in -# bin/patch.sh. The count was right and the location was wrong: an aggregate +# bin/payload.sh. The count was right and the location was wrong: an aggregate # over the finished payload can say that SOMETHING fell back but not what, and # it went down by one whether the package that lost its extension was # markupsafe (a speedup) or cffi (klippy's chelper). diff --git a/pkgs/moonraker/build.sh b/pkgs/moonraker/build.sh index 0da2889..b306644 100755 --- a/pkgs/moonraker/build.sh +++ b/pkgs/moonraker/build.sh @@ -8,7 +8,7 @@ # --strip-components hiding the shape of the archive from whoever reads this. # # WHAT IS REMOVED, AND WHY IT IS REMOVED HERE. tests/ is a sizeable part of -# the tree and never runs on a printer. It was trimmed by bin/patch.sh before +# the tree and never runs on a printer. It was trimmed by bin/payload.sh before # this recipe existed and is trimmed in the same place it always was -- once, # on the way in. The __pycache__ sweep that stood beside it is pkg_ship's # now: it was written here and in pkgs/klipper and missing from anvil-core, @@ -21,7 +21,7 @@ pkg_begin moonraker || exit 0 pkg_unpack "$MOONRAKER_TGZ" _src="$PKG_WORK/src/moonraker-$MOONRAKER_VERSION/moonraker" -# The guard bin/patch.sh has always had, kept: a tarball whose shape changed +# The guard bin/payload.sh has always had, kept: a tarball whose shape changed # under us would otherwise stage nothing, which looks like a clean build and a # dead web UI. [ -f "$_src/moonraker.py" ] || pkg_die \ diff --git a/pkgs/moonraker/pkg.conf b/pkgs/moonraker/pkg.conf index 6483887..ca98e5e 100644 --- a/pkgs/moonraker/pkg.conf +++ b/pkgs/moonraker/pkg.conf @@ -3,7 +3,7 @@ # ARCHITECTURE all. Moonraker is pure python; every native module it imports # comes from the interpreter's site-packages, which is a different package # built for a different reason. Nothing in this tree is an ELF object, so -# there is no architecture to claim. See bin/patch.sh section 4 for the long +# there is no architecture to claim. See bin/payload.sh section 4 for the long # version of why the stock 2022 build is not good enough and why this rides on # the data partition instead of over /usr/prog. PKG_NAME=anvil-moonraker diff --git a/qa/conftest.py b/qa/conftest.py index 3c9737e..4a18af4 100644 --- a/qa/conftest.py +++ b/qa/conftest.py @@ -56,7 +56,7 @@ be unequipped to ask. A test that only means something on a Creator5Pro is the shape that earns it. "I could not find the tool" is not. Three skips do not meet that bar: test_ipk.py's payload questions read -work/modpayload-root, which only bin/patch.sh produces. Noted rather than +work/modpayload-root, which only bin/payload.sh produces. Noted rather than hidden -- and note that `make test` DELETES that directory when it clears up after its fixture build, so running the old suite quietly costs the static lane three tests. See docs/qa-migration.md. diff --git a/qa/replica/actions/install-package.sh b/qa/replica/actions/install-package.sh index d86607b..5a21746 100755 --- a/qa/replica/actions/install-package.sh +++ b/qa/replica/actions/install-package.sh @@ -20,7 +20,7 @@ # # Installing for real also means the payload under test is the built package: # the cross-compiled s6, the CPython 3.13, the Klipper extras, staged by -# bin/patch.sh exactly as they ship. Nothing has to be stood in for. +# bin/payload.sh exactly as they ship. Nothing has to be stood in for. # # On the printer a successful install ends in `sleep 100000`, waiting for the # user to power-cycle. That is the signal this waits for. diff --git a/qa/replica/test_abi.py b/qa/replica/test_abi.py index 152a33b..be4d9cc 100644 --- a/qa/replica/test_abi.py +++ b/qa/replica/test_abi.py @@ -9,7 +9,7 @@ recipe staged, so the gate could only ever answer "did this recipe produce the right ABI", once per recipe, and only for the trees that go through build-packages.sh. It never saw: - * anything bin/patch.sh writes into /usr/prog -- firmwareExe, prog/*.sh and + * anything bin/payload.sh writes into /usr/prog -- firmwareExe, prog/*.sh and whatever a future one adds. pkgs/helixscreen/pkg.conf carried a note saying exactly that, for a package with a 24MB binary in it. * what the INSTALL produced, as opposed to what the build staged. Those are @@ -329,7 +329,7 @@ def test_every_object_the_package_ships_is_the_printers_abi(elves): # --------------------------------------------------------- the whole machine def test_every_loadable_object_on_the_filesystem_is_the_printers_abi(elves): - """Not just ours: /usr/prog is written by bin/patch.sh, which no gate has + """Not just ours: /usr/prog is written by bin/payload.sh, which no gate has ever read, and the stock tree is what the mod has to keep working.""" bad = [e for e in elves if e.loadable and not e.conforms and e.path != STOCK_ARM] diff --git a/qa/replica/test_install.py b/qa/replica/test_install.py index b35c508..7f32425 100644 --- a/qa/replica/test_install.py +++ b/qa/replica/test_install.py @@ -281,7 +281,7 @@ def test_the_klipper_service_runs_the_payload_tree_on_our_python(box): def test_klippy_imports_resolve_on_our_python(box): """cffi, greenlet, pyserial and jinja2 are anvil-klipper's Depends. They used to arrive as anvil-moonraker's, or from a hardcoded list in - bin/patch.sh -- so a BUILD_MOONRAKER=0 build shipped a klippy that dies + bin/payload.sh -- so a BUILD_MOONRAKER=0 build shipped a klippy that dies the first time it opens an MCU.""" for mod in ("cffi", "greenlet", "serial", "jinja2"): r = box.sh(". %s/anvil-env.sh; $FF_PYTHON -c 'import %s' 2>&1" diff --git a/qa/replica/test_s6rc.py b/qa/replica/test_s6rc.py index 334f24d..cac5225 100644 --- a/qa/replica/test_s6rc.py +++ b/qa/replica/test_s6rc.py @@ -104,7 +104,7 @@ def test_the_boot_set_is_up(booted): def test_the_scandir_was_populated_by_s6_rc_init(booted): - """It ships EMPTY -- bin/patch.sh stopped copying servicedirs in when the + """It ships EMPTY -- bin/payload.sh stopped copying servicedirs in when the database took over -- so anything here arrived at boot.""" names = booted.sh("ls -1 %s" % SCANDIR).out.split() assert "nginx" in names, "s6-rc-init laid nothing down: %s" % names diff --git a/qa/replica/test_upgrade.py b/qa/replica/test_upgrade.py index 2e86dc9..034fab4 100644 --- a/qa/replica/test_upgrade.py +++ b/qa/replica/test_upgrade.py @@ -11,7 +11,7 @@ answer is about the printer. WHY IT MATTERS. Deleting whole directories at the start of an update is right only while every file under them is ours, and $MODDIR/bin holds the supervisor and the interpreter and is the obvious place for someone to leave a script of -their own. So bin/patch.sh ships a manifest of the paths the payload installs +their own. So bin/payload.sh ships a manifest of the paths the payload installs and runFirmwareExe.sh deletes what the PREVIOUS list named, nothing else. Two claims that pull against each other: either alone is trivially satisfiable (delete everything, or delete nothing) and only holding both at once means @@ -83,12 +83,12 @@ INSTALL_T = 300 def _pack(box, src, with_manifest): """Stage `src` as the payload runFirmwareExe.sh will find on the disk. - The manifest is generated the way bin/patch.sh generates it -- find over + The manifest is generated the way bin/payload.sh generates it -- find over the staged tree, plus the manifest's own name, LC_ALL=C sort -u -- rather than written out by hand here, because a hand-written fixture list would be a list of what we HOPED was in the tree. - One deviation, and it is not a difference in the result: patch.sh drops + One deviation, and it is not a difference in the result: payload.sh drops the tree's own root with `find -mindepth 1`, which is a GNU extension the printer's busybox find need not have. This runs on the printer, so the root is dropped with the grep the case script used instead. diff --git a/qa/replica/test_what_ships.py b/qa/replica/test_what_ships.py index c8f312f..8ab3461 100644 --- a/qa/replica/test_what_ships.py +++ b/qa/replica/test_what_ships.py @@ -269,7 +269,7 @@ def test_nothing_we_ship_shadows_flashforges_python3(box): # The dev half of CPython: headers so the python-* recipes can compile against # it on a BUILD machine, pkgconfig files, and the config-3.13-* directory # holding the Makefile and the static library. anvil-python-dev owns all of it -# and bin/patch.sh prunes it via PKG_DEV_FILES. +# and bin/payload.sh prunes it via PKG_DEV_FILES. DEV_MARKERS = ( MODDIR + "/include/python3.13", MODDIR + "/lib/pkgconfig", diff --git a/qa/static/test_ipk.py b/qa/static/test_ipk.py index 5852e00..28eb3b1 100644 --- a/qa/static/test_ipk.py +++ b/qa/static/test_ipk.py @@ -314,23 +314,23 @@ def test_the_payload_is_the_feed_installed(): """Nothing ships twice, because the payload IS the packages. This test used to assert the opposite of what it asserts now, and the - property it protects is the same one. bin/patch.sh used to run each recipe + property it protects is the same one. bin/payload.sh used to run each recipe and copy its build tree into the payload, so the risk was that the tarball's copy and the packaged copy came from different code -- and the - test demanded that patch.sh run the recipe's own build.sh, which was the + test demanded that payload.sh run the recipe's own build.sh, which was the cheapest way to make "one build, two vehicles" true. - There is one vehicle now. patch.sh installs the .ipk files into a staging + There is one vehicle now. payload.sh installs the .ipk files into a staging root and ships what lands there, so the payload cannot disagree with the package: it is the package. What has to be checked instead is that nobody reintroduces the second path, which is why the assertions below are negative. """ - patch = (ROOT / "bin" / "patch.sh").read_text() + patch = (ROOT / "bin" / "payload.sh").read_text() # Every recipe whose output reaches the payload. Listed rather than # derived, because the property under test is that a HUMAN decided each of - # these ships -- a list read off patch.sh would agree with patch.sh by + # these ships -- a list read off payload.sh would agree with payload.sh by # construction and assert nothing. staged = ("libsodium", "mainsail", "moonraker", "helixscreen", "skalibs", "execline", "s6", "s6-rc", "anvil-core", "python", @@ -345,7 +345,7 @@ def test_the_payload_is_the_feed_installed(): "%s has no build.sh, so bin/build-packages.sh cannot build the " ".ipk the payload is assembled from" % rel) assert "bash %s/build.sh" % rel not in patch, ( - "bin/patch.sh runs %s/build.sh again. The payload is installed " + "bin/payload.sh runs %s/build.sh again. The payload is installed " "from the feed now, so a recipe run here is a second build of " "something already in an .ipk -- and the two can drift" % rel) @@ -353,10 +353,10 @@ def test_the_payload_is_the_feed_installed(): body = "\n".join(ln for ln in patch.splitlines() if not ln.lstrip().startswith("#")) assert "pkg_out" not in body, ( - "bin/patch.sh reads a recipe's build tree through pkg_out. The payload " + "bin/payload.sh reads a recipe's build tree through pkg_out. The payload " "comes out of the .ipk; work/pkg is the packager's business") assert "work/pkg" not in body, ( - "bin/patch.sh names work/pkg -- see pkg_out above") + "bin/payload.sh names work/pkg -- see pkg_out above") # And it must install them the one supported way. WHICH opkg: # bin/build-payload.py runs the PRINTER'S, inside the replica, against /, @@ -365,38 +365,38 @@ def test_the_payload_is_the_feed_installed(): # --force-postinstall would mean the database claims work that never # happened. assert "offline_root" not in body and "--offline-root" not in body, ( - "bin/patch.sh points an opkg at an offline root again. The payload is " + "bin/payload.sh points an opkg at an offline root again. The payload is " "installed on the printer now (bin/build-payload.py) -- an offline " "root means maintainer scripts are not running") assert "build-payload.py" in body, ( - "bin/patch.sh does not call bin/build-payload.py -- how is the " + "bin/payload.sh does not call bin/build-payload.py -- how is the " "payload being assembled?") assert "--force-postinstall" not in body, ( - "bin/patch.sh still forces postinstall. That flag existed to make an " + "bin/payload.sh still forces postinstall. That flag existed to make an " "offline-root database say installed; the printer's own opkg " "configures for real and needs no such claim") - # And the reverse direction: patch.sh must not grow its own copy of a build + # And the reverse direction: payload.sh must not grow its own copy of a build # it delegates. A `./configure` anywhere in it would mean some component is # compiled in two places again. body = "\n".join(ln for ln in patch.splitlines() if not ln.lstrip().startswith("#")) for gone in ("--enable-static-libc", "MUSL_TOOLCHAIN", "S6_STAMP"): assert gone not in body, ( - "bin/patch.sh still mentions %s -- the s6 build moved to pkgs/ and " + "bin/payload.sh still mentions %s -- the s6 build moved to pkgs/ and " "the musl toolchain was deleted with it" % gone) def _mod_roots(): - """MOD_ROOTS as bin/patch.sh's own shell reads it. + """MOD_ROOTS as bin/payload.sh's own shell reads it. No model substitution to resolve: anvil-klipper-config carries both chamber configs and anvil-link-prog.sh picks on the printer. """ - patch = (ROOT / "bin" / "patch.sh").read_text() + patch = (ROOT / "bin" / "payload.sh").read_text() m = re.search(r'^MOD_ROOTS="(.*?)"', patch, re.M | re.S) - assert m, "bin/patch.sh has no MOD_ROOTS -- has section 0 been rewritten?" + assert m, "bin/payload.sh has no MOD_ROOTS -- has section 0 been rewritten?" assert "$MODEL_PKG" not in m.group(1), ( "MOD_ROOTS expands $MODEL_PKG -- the chamber configs are one package, " "chosen on the printer rather than by the build") @@ -406,7 +406,7 @@ def _mod_roots(): def test_the_payload_roots_name_real_packages(): """A typo in MOD_ROOTS is silent, so it gets a test instead. - patch.sh skips a root with no .ipk in the feed, because that is how a + payload.sh skips a root with no .ipk in the feed, because that is how a PKG_WHEN-gated recipe -- BUILD_HELIX=0, BUILD_TOOLCHANGE=0 -- drops out without the flags being restated here. The cost is that a misspelled root drops out the same way: no error, one package quietly missing from the @@ -416,8 +416,8 @@ def test_the_payload_roots_name_real_packages(): names = {_conf(p.parent.name, "PKG_NAME") for p in RECIPES} for root in _mod_roots(): assert root in names, ( - "bin/patch.sh installs '%s' and no recipe under pkgs/ builds a " - "package by that name. patch.sh cannot tell this from a recipe " + "bin/payload.sh installs '%s' and no recipe under pkgs/ builds a " + "package by that name. payload.sh cannot tell this from a recipe " "that PKG_WHEN gated off, so it would ship without it" % root) @@ -442,7 +442,7 @@ def test_the_payload_roots_stay_a_short_list(): # --------------------------------------------------- the assembled payload # -# These read work/modpayload-root, which exists only after bin/patch.sh has +# These read work/modpayload-root, which exists only after bin/payload.sh has # run. That needs a stock FlashForge package, so it cannot be a precondition # of this lane -- but when the tree IS there the questions are worth asking, # and they are the only ones asked of the thing that actually ships. @@ -450,7 +450,7 @@ def test_the_payload_roots_stay_a_short_list(): def _payload(): p = ROOT / "work" / "modpayload-root" / "usr" / "data" / "anvil" if not (p / "var" / "lib" / "opkg" / "status").is_file(): - pytest.skip("no assembled payload -- run bin/patch.sh") + pytest.skip("no assembled payload -- run bin/payload.sh") return p @@ -523,7 +523,7 @@ def test_the_payload_database_has_no_clock_in_it(): opkg stamps Installed-Time from time() and honours SOURCE_DATE_EPOCH only for a man-page date, so its database differs between two builds a second - apart. patch.sh normalises the field: this is an image being baked, not a + apart. payload.sh normalises the field: this is an image being baked, not a machine being installed. Asked of the database rather than of anvil.tar.xz, which is not @@ -535,7 +535,7 @@ def test_the_payload_database_has_no_clock_in_it(): assert stamps, "the opkg status file records no Installed-Time at all" assert len(set(stamps)) == 1, ( "the payload's database carries %d different Installed-Time values " - "(%s...). bin/patch.sh is meant to normalise them, so two builds of " + "(%s...). bin/payload.sh is meant to normalise them, so two builds of " "one commit differ by one line per package" % (len(set(stamps)), sorted(set(stamps))[:3])) @@ -554,7 +554,7 @@ def test_the_payload_keeps_libsodium_as_a_symlink(): pkgs/3rdparty/libsodium/build.sh already asserts this about its own build tree, and test_symlinks_stay_symlinks asserts opkg-build puts a symlink in an archive. Neither covers the step between them -- opkg unpacking the - archive into the payload -- and that is the step bin/patch.sh used to + archive into the payload -- and that is the step bin/payload.sh used to check by hand. """ lib = _payload() / "lib" / "libsodium.so" @@ -569,7 +569,7 @@ def test_the_payload_keeps_libsodium_as_a_symlink(): # Scripts that build or ship something, and could therefore host a gate. def _buildish_scripts(): named = ["bin/common.sh", "bin/build-packages.sh", "bin/verify.sh", - "bin/patch.sh", "bin/pack.sh", "pkgs/lib.sh", + "bin/payload.sh", "bin/pack.sh", "pkgs/lib.sh", "tools/python/build.sh", "tools/python-packages/build.sh", "tools/python-packages/build-libsodium.sh"] found = [ROOT / n for n in named] @@ -629,7 +629,7 @@ def test_there_is_exactly_one_abi_gate_and_it_is_the_replica_one(): Six implementations of one rule, each over a different subset of the files, and between them they still could not see the largest binary we - ship: bin/patch.sh unpacks HelixScreen into /usr/prog and none of them + ship: bin/payload.sh unpacks HelixScreen into /usr/prog and none of them read that tree. This test is the ratchet. Two gates that disagree about which files are @@ -991,7 +991,7 @@ def test_no_cache_stamp_is_spelled_in_two_places(): """A stamp compared by one file and written by another must be defined once. This is not hypothetical tidiness. bin/fetch-assets.sh compared - work/.s6/.version against "$SKALIBS_VERSION $S6_VERSION" while bin/patch.sh + work/.s6/.version against "$SKALIBS_VERSION $S6_VERSION" while bin/payload.sh wrote three fields into it, so the test could never be false: a 71MB toolchain was re-fetched on every single run, and the comment above the condition described a fast path that had never once been taken. One @@ -1014,7 +1014,7 @@ def test_no_cache_stamp_is_spelled_in_two_places(): # reintroduced the bug with the evidence removed. These are the two stamps # this repo has actually got wrong -- s6's three fields, and CPython's # eight, which were spelled in THREE places and happened to agree. - for name in ("patch.sh", "fetch-assets.sh"): + for name in ("payload.sh", "fetch-assets.sh"): text = (ROOT / "bin" / name).read_text() assert '"$SKALIBS_VERSION $S6_VERSION' not in text, ( "bin/%s spells the s6 stamp out by hand" % name) @@ -1042,8 +1042,8 @@ def test_the_python_package_list_and_the_recipes_agree(): alone has no source to build from. This checks both directions at once, without building anything. It used to - say that bin/patch.sh checked the first one at build time as it looped over - the list; patch.sh installs packages now and has no such loop, so this is + say that bin/payload.sh checked the first one at build time as it looped over + the list; payload.sh installs packages now and has no such loop, so this is the only thing asking. """ listed = set(_sh('printf "%s" "$PYPKG_LIST"').split()) @@ -1076,9 +1076,9 @@ def test_a_python_package_does_not_pin_its_version_twice(): # test_bin_patch_builds_every_python_package WAS HERE and is deleted rather -# than rewritten. It asserted that bin/patch.sh contained +# than rewritten. It asserted that bin/payload.sh contained # `for p in $PYPKG_LIST; do` and ran each pkgs/3rdparty/python-$p/build.sh, -# which was how the eighteen wheels reached the payload. patch.sh installs +# which was how the eighteen wheels reached the payload. payload.sh installs # packages now and does not know which of them are wheels, so the loop is # gone. # diff --git a/qa/static/test_moddir_paths.py b/qa/static/test_moddir_paths.py index c49adf8..0730bc8 100644 --- a/qa/static/test_moddir_paths.py +++ b/qa/static/test_moddir_paths.py @@ -45,7 +45,7 @@ GUARD = re.compile(r"\[\s+-[fxdesr]\s+\$\{?MODDIR\}?/([A-Za-z0-9_.][A-Za-z0-9_./ # Not shipped by any package, and correctly so. ALLOWED = { - # Templated from config.env by bin/patch.sh and then preserved across + # Templated from config.env by bin/payload.sh and then preserved across # updates by run-append.sh -- user state, so a package member would # overwrite a printer's settings on the first upgrade. "anvil.conf", diff --git a/qa/static/test_recipe_layout.py b/qa/static/test_recipe_layout.py index 27315f6..a2e8eb1 100644 --- a/qa/static/test_recipe_layout.py +++ b/qa/static/test_recipe_layout.py @@ -1,41 +1,22 @@ -"""A recipe directory has a fixed shape, and the shape is what says where a -file goes. +"""The two facts about recipe layout that have a consequence. -The repository used to keep everything it ships in one top-level payload/ -tree, organised by DESTINATION -- init.d/, bin/, etc/s6/, klipper/config/ -- -and the only thing recording which package owned any of it was a thirty-line -comment at the top of pkgs/anvil-core/build.sh. A comment cannot be checked. -Three kinds of file lived in there and looked identical: +A recipe directory has a fixed shape -- build.sh and pkg.conf beside +payload/, seed/ and control/ -- and recipes live at two depths, pkgs// +for the ones carrying files of this repo and pkgs/3rdparty// for the ones +building a pinned tarball. That shape is documented in docs/building.md and in +pkgs/lib.sh, which is the only code that depends on it. - * files a recipe stages into its .ipk, - * files that never ship as files at all -- run-pre.sh and run-append.sh are - spliced into FlashForge's own run.sh at build time. +MOST OF IT IS NOT ASSERTED HERE, and that is deliberate. This file used to hold +seven more tests: that a recipe contains no fourth kind of directory, that each +kind is still used by someone, that a 3rdparty recipe carries no files of ours +and a first-party one carries some, that installer/ holds exactly two named +files. Every one of them compared the tree against a description of the tree. +None could go red for a reason a printer would notice -- a stray file in a +recipe directory is not read by anything, and a recipe filed at the wrong depth +still builds the same .ipk -- while all seven went red the moment the layout +legitimately changed, which is the opposite of what a test is for. -Files bound for /usr/prog are the first kind: they are package files under -payload/prog/, and anvil-link-prog.sh symlinks them from $MODDIR to the -absolute paths FlashForge's scripts read. - -Recipes themselves live at two depths, and pkgs/lib.sh's pkg_dir is the only -thing that knows it: - - pkgs// carries files of this repo. Four of them. - pkgs/3rdparty// builds a pinned tarball and carries none. The - other thirty-four. - -Inside a recipe, each kind of file is a directory name: - - pkgs//payload/ staged into the .ipk, laid out as it lands under - $MODDIR. payload/etc/s6-rc/source/nginx/run - becomes $MODDIR/etc/s6-rc/source/nginx/run and no - recipe says so. - pkgs//seed/ seeded user state -- moonraker-custom.conf. Not - package members, because a package member is - overwritten on every upgrade by definition and - these are the files a printer's owner edits. - -This file holds that to being true. The failure it exists to catch is not a -misfiled file -- it is a fourth directory appearing with no rule attached to -it, which is how the old payload/ got the way it was. +What is left is the two that bite: """ import os @@ -45,19 +26,7 @@ from lib.paths import ROOT pytestmark = pytest.mark.static -# Everything a recipe may contain. build.sh and pkg.conf are the recipe; -# the subtrees are its files. -# -# payload/ what the .ipk installs under $MODDIR -# seed/ user state templated by bin/patch.sh, in no package -# control/ maintainer scripts and conffiles, copied verbatim into the -# .ipk's CONTROL/ by bin/build-packages.sh. It is metadata rather -# than content -- nothing in here lands on the printer as a file, -# it runs at install time -- which is why it is its own directory -# and not a corner of payload/. -RECIPE_FILES = {"build.sh", "pkg.conf"} -RECIPE_DIRS = {"payload", "seed", "control"} - +MODDIR_PREFIX = ("usr", "opt", "var", "proc", "sys", "dev") THIRD = os.path.join(ROOT, "pkgs", "3rdparty") @@ -74,129 +43,43 @@ def recipes(): def test_there_are_recipes(): - """A glob that silently matches nothing is a test that passes hollow.""" - assert len(recipes()) > 10 + """A glob that silently matches nothing is a test that passes hollow. - -def test_a_recipe_holds_nothing_but_a_recipe_and_its_files(): - strays = [] - for name, d in recipes(): - for entry in sorted(os.listdir(d)): - if entry in RECIPE_FILES or entry in RECIPE_DIRS: - continue - strays.append("%s: %s" % (name, entry)) - assert not strays, ( - "a recipe may hold build.sh, pkg.conf, and payload/ seed/ control/ -- " - "nothing else, because the directory name is what says where a file " - "goes: %s" % ", ".join(strays)) - - -def test_every_recipe_subtree_is_one_of_the_three(): - """The rule stated the other way round: no fourth kind of directory.""" - for name, d in recipes(): - dirs = {e for e in os.listdir(d) if os.path.isdir(os.path.join(d, e))} - assert dirs <= RECIPE_DIRS, ( - "recipe %s has %s -- a new directory needs a rule in this file " - "saying where its contents end up" % (name, sorted(dirs - RECIPE_DIRS))) - - -@pytest.mark.parametrize("sub", sorted(RECIPE_DIRS)) -def test_each_subtree_is_used_by_someone(sub): - """If one empties out, this fails and the rule can be deleted. - - That is the good outcome, not a defect: seed/ empty will mean the seeder - landed and nothing is templated at build time any more. + Both tests below iterate recipes(); this is what stops them reporting + green over an empty list. """ - users = [n for n, d in recipes() if os.path.isdir(os.path.join(d, sub))] - assert users, ( - "no recipe has a %s/ any more -- if that is deliberate, delete it " - "from RECIPE_DIRS and from the layout note in this file's docstring" - % sub) + assert len(recipes()) > 10 def test_nothing_under_a_payload_escapes_the_prefix(): """payload/ is a $MODDIR overlay, so it may not name an absolute path. - A file at payload/usr/bin/x would stage to $MODDIR/usr/bin/x, which is - correct-looking and wrong: it is not what the recipe author meant, and - bin/build-packages.sh would happily package it. + A file at payload/usr/bin/x stages to $MODDIR/usr/bin/x -- correct-looking + and wrong. The whole reason the mod lives on /usr/data is that /usr is + FlashForge's and an OTA rewrites it, so a package that owned a path up + there would survive exactly until the next factory update and take + whatever it overwrote with it. + + qa/static/test_ipk.py asks the same question of every BUILT .ipk, which is + the stronger form. This one is kept beside it because it runs on a bare + checkout with no feed, and names the recipe rather than the artefact. """ bad = [] for name, rd in recipes(): d = os.path.join(rd, "payload") for top in sorted(os.listdir(d)) if os.path.isdir(d) else []: - if top in ("usr", "opt", "var", "proc", "sys", "dev"): + if top in MODDIR_PREFIX: bad.append("%s payload/%s" % (name, top)) assert not bad, ( "payload/ is rooted at $MODDIR, not at /: %s" % ", ".join(bad)) -def test_the_installer_is_not_a_recipe(): - """installer/ sits outside pkgs/ because its two files are never packages. - - They are text spliced into FlashForge's run.sh by bin/patch.sh. Calling - them a package -- or filing them under a recipe as though they could - become one -- would be the only untrue path in the tree. - """ - d = os.path.join(ROOT, "installer") - assert os.path.isdir(d) - assert not os.path.exists(os.path.join(d, "pkg.conf")) - assert sorted(os.listdir(d)) == ["run-append.sh", "run-pre.sh"] - - -# ------------------------------------------------------------ the two levels -# -# pkgs// a recipe that carries FILES OF THIS REPO. -# pkgs/3rdparty// a recipe that builds a pinned tarball and carries -# none. -# -# The split exists so that `ls pkgs/` is the four things a person edits rather -# than thirty-eight with those four buried. It is mechanical on purpose: "do -# we actively modify it" drifts, and the day somebody patches zlib nobody -# would agree on which side it belongs. "Does it carry files of ours" is a -# fact about the tree, and the two tests below are the whole of the rule. - - -def test_a_3rdparty_recipe_carries_no_files_of_ours(): - wrong = [] - for name in sorted(os.listdir(THIRD)): - d = os.path.join(THIRD, name) - if not os.path.isfile(os.path.join(d, "pkg.conf")): - continue - have = sorted(s for s in RECIPE_DIRS if os.path.isdir(os.path.join(d, s))) - if have: - wrong.append("%s has %s" % (name, "/".join(have))) - assert not wrong, ( - "a recipe under 3rdparty/ builds a pinned tarball and stages nothing " - "from this checkout. These carry files of ours and belong one level " - "up, in pkgs/: %s" % ", ".join(wrong)) - - -def test_a_first_party_recipe_carries_something(): - """The rule from the other side, so the levels cannot both drift empty. - - A recipe at the top level with no payload/ or seed/ is - indistinguishable from a 3rdparty one and should move down -- otherwise - the top level slowly refills and the split stops meaning anything. - """ - wrong = [] - base = os.path.join(ROOT, "pkgs") - for name in sorted(os.listdir(base)): - d = os.path.join(base, name) - if name == "3rdparty" or not os.path.isfile(os.path.join(d, "pkg.conf")): - continue - if not any(os.path.isdir(os.path.join(d, s)) for s in RECIPE_DIRS): - wrong.append(name) - assert not wrong, ( - "these carry no files of this repo, so they are 3rdparty recipes and " - "belong under pkgs/3rdparty/: %s" % ", ".join(wrong)) - - def test_recipe_names_are_unique_across_the_two_levels(): """pkg_out, pkg_stamp and the .ipk filename are all keyed by the bare name. A name at both levels would resolve to whichever pkgs/lib.sh's pkg_dir - searches first and quietly build the other one's package. + searches first and quietly build the other one's package. Nothing + downstream can catch that: the wrong package builds perfectly. """ names = [n for n, _ in recipes()] dupes = sorted({n for n in names if names.count(n) > 1}) diff --git a/qa/static/test_shell_syntax.py b/qa/static/test_shell_syntax.py index c5db810..4318292 100644 --- a/qa/static/test_shell_syntax.py +++ b/qa/static/test_shell_syntax.py @@ -3,7 +3,7 @@ Ported from test/run-tests.py's check_shell_syntax and check_no_bashisms. The logic is theirs; what changes is the granularity. There, one gate covered ~25 files and reported one bit, so a syntax error in S62moonraker and a syntax -error in patch.sh were the same red line. Here each file is its own test, named +error in payload.sh were the same red line. Here each file is its own test, named after itself, and `pytest -k S62moonraker` runs exactly one. The original's docstring made a deliberate choice worth preserving: diff --git a/tools/python-packages/README.md b/tools/python-packages/README.md index 100d6de..eb23d78 100644 --- a/tools/python-packages/README.md +++ b/tools/python-packages/README.md @@ -4,10 +4,10 @@ The second half of phase 6. `tools/python/` builds the interpreter; this builds what runs on it. **Now a recipe apiece: `pkg/python-`, one `.ipk` each** -- see "What is left" at the bottom for the history. `build.sh` is the measurement harness, preserved here because it is where the findings live, exactly as `tools/supervisor/` is for s6; it is not -what a release runs any more, and the two have drifted on purpose (patch.sh +what a release runs any more, and the two have drifted on purpose (payload.sh builds offline from `vendor/` and needs no docker, because the build lane has no docker socket). `fill-designators.py` is the exception: it is a real build -input, called by `patch.sh` for greenlet. +input, called by `payload.sh` for greenlet. Everything below was run in the printer replica on the real `rootfs.squashfs`, not read off a project page. @@ -175,7 +175,7 @@ same one-line `FF_PYTHON` switch, since 3.8's libnacl still needs it. Capability was settled first -- Moonraker serves, the database works, libsodium is ours -- and packaging and integration followed: -1. ~~**Wire into `bin/patch.sh`.**~~ ~~**Done.**~~ **Superseded: each package +1. ~~**Wire into `bin/payload.sh`.**~~ ~~**Done.**~~ **Superseded: each package is its own recipe.** The step below describes the arrangement that replaced it, and the arrangement that then replaced *that*, because the second correction is the interesting one. @@ -200,7 +200,7 @@ libsodium is ours -- and packaging and integration followed: tree could not express. libsodium is its own recipe. Every pin is in `versions.env` with a sha256 and - `bin/fetch-assets.sh` fetches them, so **patch.sh no longer talks to a + `bin/fetch-assets.sh` fetches them, so **payload.sh no longer talks to a network**: no `get-pip.py`, no PyPI, `pip --no-index` against hashed sdists. The build-python gets its pip from `--with-ensurepip=install` out of the same pinned CPython tarball. Two build-image packages were needed diff --git a/tools/python-packages/build.sh b/tools/python-packages/build.sh index 3e86e6b..6a692bf 100755 --- a/tools/python-packages/build.sh +++ b/tools/python-packages/build.sh @@ -4,7 +4,7 @@ # for the CPython 3.13 that tools/python/build.sh already produces for the # FlashForge Creator 5 Pro (Ingenic mipsel). # -# SPIKE. Nothing here is wired into bin/patch.sh yet. +# SPIKE. Nothing here is wired into bin/payload.sh yet. # # Same non-negotiables as tools/python/build.sh, for the same reasons: # diff --git a/tools/python/README.md b/tools/python/README.md index ab37f1c..bf8393f 100644 --- a/tools/python/README.md +++ b/tools/python/README.md @@ -215,7 +215,7 @@ build rather than quietly adding a name to PATH. **Cold build: 142 seconds** (`tools/python/build.sh`, 16 cores), of which about 90 is the x86-64 build-python. In the build lane it is ~100s and it happens -**once**: `bin/patch.sh` caches the cross-built tree in `work/.py313` under a +**once**: `bin/payload.sh` caches the cross-built tree in `work/.py313` under a stamp of all eight versions, so every later build copies rather than compiles. The stamp is all eight and not just CPython's, because a bumped OpenSSL under an unchanged `PY_VERSION` has to rebuild. @@ -285,7 +285,7 @@ record and for trying a version bump before it becomes a pin. It is not how the shipped interpreter is built: `pkg/python` compiles it inside the repo's own build image, from the sha256-pinned tarballs, against the seven static libraries the feed builds as their own `-dev` packages, and caches it in -`work/pkg/python`. `bin/patch.sh` section 5c runs that recipe and stages what +`work/pkg/python`. `bin/payload.sh` section 5c runs that recipe and stages what it produced; it no longer contains a compiler invocation of its own. The gate, against the tree the build actually produced: diff --git a/tools/python/build.sh b/tools/python/build.sh index 3636527..791b6f1 100755 --- a/tools/python/build.sh +++ b/tools/python/build.sh @@ -18,7 +18,7 @@ # loader = /lib/ld-linux-mipsn8.so.1 , rootfs glibc 2.33 # # Toolchain: the repo's Ingenic gcc 7.2 / glibc 2.29 cross compiler -- the SAME -# one bin/patch.sh uses for klippy's c_helper.so. musl is forbidden: a +# one bin/payload.sh uses for klippy's c_helper.so. musl is forbidden: a # musl-linked interpreter cannot dlopen a glibc c_helper.so, which is exactly # how klippy loads it. glibc 2.29 -> 2.33 is forward compatible. # @@ -55,9 +55,9 @@ XZ_VER=5.4.7 BZIP2_VER=1.0.8 EXPAT_VER=2.6.4 EXPAT_TAG=R_2_6_4 -# The mod's prefix root on the printer, and the same --prefix bin/patch.sh +# The mod's prefix root on the printer, and the same --prefix bin/payload.sh # builds with: bin/python3.13 beside bin/s6-svscan, stdlib in -# lib/python3.13/. The one difference is that patch.sh deletes the `python3` +# lib/python3.13/. The one difference is that payload.sh deletes the `python3` # symlink (and idle3/pydoc3/*-config) out of bin/ before staging, so that # putting $MODDIR/bin on PATH cannot quietly change what `python3` means -- # see tools/python/README.md. This harness leaves the tree as `make install` @@ -68,7 +68,7 @@ PREFIX=/usr/data/anvil if [ "${1:-}" != "--in-container" ]; then [ -x "$TOOLCHAIN_HOST/bin/mips-linux-gnu-gcc" ] || { echo "!! no toolchain at $TOOLCHAIN_HOST" >&2 - echo " run ./bin/fetch-assets.sh, then any build (bin/patch.sh unpacks it)" >&2 + echo " run ./bin/fetch-assets.sh, then any build (bin/payload.sh unpacks it)" >&2 exit 1; } exec docker run --rm \ -v "$TOOLCHAIN_HOST":/toolchain:ro \ diff --git a/tools/replica/README.md b/tools/replica/README.md index 7e591ab..0aaae09 100644 --- a/tools/replica/README.md +++ b/tools/replica/README.md @@ -10,7 +10,7 @@ different things drive it, and only one of them is a suite: | driver | what it does with it | |---|---| -| `bin/patch.sh` -> `bin/build-payload.py` | **the build path.** The payload is assembled by running the PRINTER'S OWN `opkg` against `/`, inside here. Nothing on the host can do that | +| `bin/payload.sh` -> `bin/build-payload.py` | **the build path.** The payload is assembled by running the PRINTER'S OWN `opkg` against `/`, inside here. Nothing on the host can do that | | `qa/lib/replica.py` | the `replica` lane's `printer` fixture -- holds one container open and probes it | | `make rootfs`, `make boot-screen-sim` | unpacking the real userland; rendering the boot frames on the cross-built CPython | diff --git a/tools/replica/ffsim/gates.py b/tools/replica/ffsim/gates.py index edba0e6..e328624 100644 --- a/tools/replica/ffsim/gates.py +++ b/tools/replica/ffsim/gates.py @@ -85,7 +85,7 @@ def _python_trees(config): """Every recipe output that makes up the printer's python prefix. NINETEEN TREES, NOT ONE. This used to be work/.py313, the single directory - bin/patch.sh cross-built the interpreter and its site-packages into + bin/payload.sh cross-built the interpreter and its site-packages into together. CPython is pkgs/3rdparty/python now and each of the eighteen third-party packages is a pkgs/3rdparty/python-* of its own, so what a printer sees is the union of their bin/ and lib/ -- which is exactly what diff --git a/tools/replica/printer/case-build-payload.sh b/tools/replica/printer/case-build-payload.sh index 5a0342b..a0d6d4b 100755 --- a/tools/replica/printer/case-build-payload.sh +++ b/tools/replica/printer/case-build-payload.sh @@ -1,7 +1,7 @@ #!/bin/sh # Build $MODDIR the way a printer would, and hand it back on /out. # -# bin/patch.sh's section 0, run on the machine it is for: the printer's own +# bin/payload.sh's section 0, run on the machine it is for: the printer's own # opkg installs onto the printer's own filesystem, so the postinsts execute # under qemu-mipsel against the paths they will see on a machine. # @@ -42,14 +42,14 @@ $OPKG update >/dev/null # --- install the release --------------------------------------------------- # Roots, not a closure: Depends brings the rest, which is the same question an # `opkg install anvil-moonraker` on a printer asks. MOD_ROOTS comes in from -# bin/patch.sh so the two cannot drift. +# bin/payload.sh so the two cannot drift. [ -n "${MOD_ROOTS:-}" ] || { echo "payload: MOD_ROOTS is empty" >&2; exit 1; } $OPKG install $MOD_ROOTS echo "payload: $($OPKG list-installed | wc -l) packages installed" # --- compile the boot database --------------------------------------------- # WITH THE s6-rc-compile WE SHIP, on the machine it is for, right after the -# packages that own the source tree landed. bin/patch.sh used to do this on the +# packages that own the source tree landed. bin/payload.sh used to do this on the # build host, which meant a second NATIVE build of skalibs+execline+s6+s6-rc # whose only job was to produce a compiler -- and whose whole hazard was that # s6-rc-compile bakes the #! of the execline the COMPILER was linked against diff --git a/tools/replica/sim-boot-screen.py b/tools/replica/sim-boot-screen.py index f99f4c1..25b3e34 100755 --- a/tools/replica/sim-boot-screen.py +++ b/tools/replica/sim-boot-screen.py @@ -42,7 +42,7 @@ def run(): config = Config.load() tree = _python_tarball(config) if not tree: - raise SystemExit("nothing in work/pkg/python -- run ./bin/patch.sh first") + raise SystemExit("nothing in work/pkg/python -- run ./bin/payload.sh first") replica = Replica.start(config, want_output=_echo) # The frames come back on stdout as base64, so this body is the payload -- # not just a log. It is deliberately not echoed. diff --git a/tools/supervisor/README.md b/tools/supervisor/README.md index 0ff69eb..7fef227 100644 --- a/tools/supervisor/README.md +++ b/tools/supervisor/README.md @@ -106,7 +106,7 @@ adopting s6 -- not the supervisor itself. then pack `bin/`, `libexec/` and `runit/` into a tarball and run the gate. That is how the COMPARISON above was measured, and it is kept here for the -record. It is no longer how s6 is built: the answer is s6, so `bin/patch.sh` +record. It is no longer how s6 is built: the answer is s6, so `bin/payload.sh` now cross-compiles skalibs and s6 from the tarballs pinned in `versions.env`, inside the repo's own build image, caches the result in `work/.s6` and stages it into the payload. Nothing builds runit or execline any more, so only diff --git a/versions.env b/versions.env index 36dc1a3..3c0ab93 100644 --- a/versions.env +++ b/versions.env @@ -24,7 +24,7 @@ MAINSAIL_SHA256="df2ba7c301f7bfc8ac9f122741a6ba08356d679ecfa1f62f898d0337802d5de # c_helper.so from ITS sources with the pinned toolchain below, so the .so can # never be older than the klippy tree beside it. The v20260824 release shipped # without any of that: config.env's KLIPPER_FORK -- a second, unhashed source -# that pointed at a local checkout -- was empty in CI, bin/patch.sh silently +# that pointed at a local checkout -- was empty in CI, bin/payload.sh silently # kept the stock 0.12-era tree, and the package half-overwrote the fork on # updated printers, where klippy died at connect with a cffi arg-count error. # KLIPPER_FORK is gone with the migration to pkg/: a recipe names its source @@ -48,7 +48,7 @@ HELIX_FILE="helixscreen-creator5-v0.99.115.tar.gz" HELIX_SHA256="dd66dd9b1bdd73f77d7c7f7d76003b14121c5cf102a755083ef7506fbdb22d7d" # Moonraker. The stock one is a 2022 build (API 1.0.5) that predates the -# webcam "enabled" flag that modern Mainsail requires -- see bin/patch.sh. +# webcam "enabled" flag that modern Mainsail requires -- see bin/payload.sh. # # A COMMIT, NOT A RELEASE, and not a matter of taste. FlashForge built python # 3.8.2 WITHOUT the _sqlite3 module and shipped no libsqlite3 anywhere on the @@ -187,12 +187,12 @@ S6RC_SHA256="bf5b8ce0da5a4ee70d642b818b61d9916a7a9b64a457595f388113e54a188688" # PY_VERSION The interpreter, and the ONLY version this build works # for as written: cross-building CPython needs a build # python of the SAME version (configure hard-errors -# otherwise), so patch.sh builds an x86-64 3.13.7 from this +# otherwise), so payload.sh builds an x86-64 3.13.7 from this # same tarball first. Bump it and both halves move # together, which is why there is one variable. # OPENSSL_VERSION 3.0.x DELIBERATELY, not 3.1/3.2/3.5. Two things about # 3.0.15 are relied on and both were hit for real: `no-docs` -# does not exist before 3.1 (patch.sh must not pass it), and +# does not exist before 3.1 (payload.sh must not pass it), and # the linux-mips32 target bakes `-mips2` into its cflags, # which this toolchain -- default -mfp64 -- rejects unless # an explicit -mips32r2 lands after it on the command line. @@ -261,7 +261,7 @@ EXPAT_SHA256="fd03b7172b3bd7427a3e7a812063f74754f24542429b634e0db6511b53fb2278" # both with contextlib.suppress(ImportError) and falls back to the stdlib, # which was measured rather than assumed. # -# Every one is an SDIST, and bin/patch.sh builds them with `--no-binary :all:` +# Every one is an SDIST, and bin/payload.sh builds them with `--no-binary :all:` # for a reason that is recorded here because it cost real time to find: pip # will HELPFULLY download a prebuilt manylinux wheel when one exists, and on # the first run of the spike it did exactly that for cffi, greenlet and @@ -282,7 +282,7 @@ EXPAT_SHA256="fd03b7172b3bd7427a3e7a812063f74754f24542429b634e0db6511b53fb2278" # PYPKG_LIST IS THE SHIP LIST AND THE FETCH LIST -- no longer the build order, # which pkg_order derives from each recipe's PKG_BUILD_DEPENDS. Each entry is a # recipe: `distro` is pkg/python-distro, which produces anvil-python-distro. -# bin/fetch-assets.sh fetches from this list and bin/patch.sh stages from it, +# bin/fetch-assets.sh fetches from this list and bin/payload.sh stages from it, # and qa/static checks the list against the recipe directories both ways, so an # entry added here and nowhere else fails a test rather than being fetched, # hashed and silently never built. The variable names are the list entry @@ -354,7 +354,7 @@ PYPKG_MARKUPSAFE_FILE="MarkupSafe-2.1.5.tar.gz" PYPKG_MARKUPSAFE_SHA256="d283d37a890ba4c1ae73ffadf8046435c76e7bc2247bbb63c00bd1a709c6544b" # cffi -- klippy's chelper, and the package that justifies the whole glibc # decision: this is what dlopens FlashForge's own c_helper.so, measured. Wants -# libffi, and gets the static -fPIC one patch.sh already built for _ctypes. +# libffi, and gets the static -fPIC one payload.sh already built for _ctypes. PYPKG_CFFI_FILE="cffi-1.17.1.tar.gz" PYPKG_CFFI_SHA256="1c39c6016c32bc48dd54561950ebd6836e1670f2ae46128f67cf49e789c52824" # greenlet -- klippy's reactor. The MIPS stack-switching assembly was never