starred/firmware
1
0 Fork 0
mirror of https://github.com/Klipper4FlashForge/firmware.git synced 2026-10-21 12:52:16 -05:00
Code Issues Packages Projects Releases 8 Wiki Activity

bin/*.sh: 1,110 comment lines were half the code; 327 remain

Browse Source 
The scripts under bin/ carried more prose than shell. Most of it was
rationale for decisions already made, archaeology of flags since deleted
(BUILD_KLIPPER=stock, the five MOD_* switches, KLIPPER_FORK), or a
paragraph restating the line beneath it.

What survives is what a reader cannot get from the code: the s6-rc
--prefix trap that turns every oneshot into an ENOENT, chelper.tar being
the stock installer's vehicle rather than a second build, PKG_EXCLUDE
matching Mainsail's own .version at any depth, Perl's || making
SOURCE_DATE_EPOCH=0 the one value that falls through, the herestrings
that avoid a pipefail 141, unTar's -md md5. Each is now one or two lines
instead of a paragraph, and the section dividers lost their rules.

No code changed. qa/static is 122 passed / 124 errors before and after,
the errors being the same missing vendor/opkg-utils and uncompiled
s6-rc database in both runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0111WSZKjXNsvfWS2PR9wK7o
This commit is contained in:
Oleksandr ShyshatskyiandClaude Opus 5 committed 2026-08-30 00:06:20 +03:00
1 parent 53292b4d78
commit b25a19683d
8 files changed
+324 -1070
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

bin/build-packages.sh
+61 -226
View File
@@ -1,37 +1,16 @@
#!/usr/bin/env bash
# Build every recipe under pkgs/ into the feed, in dependency order, and index
# it.
# Build every recipe under pkgs/ into the feed, in dependency order, and index it.
#
# ./bin/build-packages.sh all recipes
# ./bin/build-packages.sh opkg that one and everything it needs
#
# WHAT THIS IS. The feed bin/patch.sh installs to make the payload, so this
# runs before a build rather than beside it. It still needs no stock FlashForge
# package, which is why it is a separate script -- see the end of this header.
# Each recipe is packaged before the next is built, because a recipe gets its
# PKG_BUILD_DEPENDS out of this feed: libarchive's configure reads zlib's
# headers out of anvil-zlib_*.ipk. The archives themselves are opkg-utils'
# work; what is left here is laying out the tree it packages.
#
# ONE RECIPE PRODUCES ONE PACKAGE, and a recipe that needs a library names it
# in PKG_BUILD_DEPENDS and gets it out of this feed. That is why the loop below
# packages each recipe before it moves to the next one rather than building
# everything and packaging afterwards: pkgs/3rdparty/libarchive's configure reads zlib's
# headers out of anvil-zlib_1.3.1-1_mipsel_xburst2.ipk, so that file has to
# exist by the time libarchive is built. The feed is not an output of this
# script so much as the medium it works in.
#
# THE ARCHIVES ARE BUILT BY UPSTREAM'S OWN TOOLS, not by this file. opkg-build
# and opkg-make-index -- and opkg-unbuild, which pkgs/lib.sh uses to take them
# apart again -- come from opkg-utils, pinned by commit in versions.env. An
# earlier revision of this work carried a hand-written ar-and-two-tarballs
# script instead, which was 120 lines of this repo re-deriving a format
# somebody else already maintains, including the parts that are easy to get
# subtly wrong (member ORDER, the CONTROL field validation, the tar flags that
# make a build reproducible) and whose failure mode is a package that inspects
# fine and installs nowhere. What is left here is the part that is genuinely
# ours: laying out the tree that opkg-build packages.
#
# IT NEEDS NO STOCK FIRMWARE PACKAGE, which bin/patch.sh does. That is
# deliberate and it is most of why this is a separate script: packaging has to
# be runnable in CI, on a checkout, without a FlashForge .tgz -- otherwise the
# gate only runs where the secrets are and stops being a gate.
# Needs no stock FlashForge package, unlike bin/patch.sh, so packaging stays
# runnable in CI on a bare checkout.
set -euo pipefail
. "$(dirname "$0")/common.sh"
. "$ROOT/pkgs/lib.sh"
@@ -42,31 +21,14 @@ for t in "$OPKG_BUILD_BIN" "$OPKG_INDEX_BIN" "$OPKG_UNBUILD_BIN"; do
[ -x "$t" ] || { echo "!! $t is missing -- run ./bin/fetch-assets.sh" >&2; exit 1; }
done
# Reproducible by default. opkg-build reads SOURCE_DATE_EPOCH and, when it is
# set, adds --clamp-mtime to its tar calls on top of the --sort=name it always
# passes; without it every package carries the second it was built and two
# builds of an unchanged tree cannot be compared. Defaulting it rather than
# leaving it to `date` is the difference between reproducible-by-default and
# reproducible-if-you-remember.
#
# THE DEFAULT IS 1 AND NOT 0, WHICH IS NOT A TYPO. It was 0, and OpenSSL was
# the one package in the feed that would not reproduce: two builds an hour
# apart differed in exactly one object, libcrypto-lib-cversion.o, which
# carries a "built on:" string. OpenSSL does support SOURCE_DATE_EPOCH --
# util/mkbuildinf.pl reads it -- but the line is
#
# my $date = gmtime($ENV{'SOURCE_DATE_EPOCH'} || time()) . " UTC";
#
# and that is Perl's ||, not //. Zero is FALSE in Perl, so the one value we
# were passing is the single value that silently falls through to the current
# time. One second past the epoch is just as fixed and is true.
# Reproducible by default: opkg-build adds --clamp-mtime when SOURCE_DATE_EPOCH
# is set. THE DEFAULT IS 1 AND NOT 0, WHICH IS NOT A TYPO -- OpenSSL reads it as
# `$ENV{'SOURCE_DATE_EPOCH'} || time()`, and Perl's || makes 0 the one value
# that silently falls through to the current time.
export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-1}"
# The recipes to build, expanded to include everything they build against and
# sorted so a dependency always precedes its dependent. Iterating the recipe
# directories and
# taking the order the shell gives is wrong the moment there are two recipes:
# alphabetically, libarchive comes before the zlib it needs.
# Sorted so a dependency precedes its dependent -- alphabetically, libarchive
# comes before the zlib it needs.
if [ $# -gt 0 ]; then
REQUESTED=("$@")
else
@@ -83,13 +45,11 @@ for r in "${RECIPES[@]}"; do
|| { echo "!! no recipe named '$r'" >&2; exit 1; }
[ -f "$_d/build.sh" ] || { echo "!! $_d has no build.sh" >&2; exit 1; }
# Every recipe caches on its own stamp -- pkgs/lib.sh's pkg_begin is that
# check -- so this is a no-op on a warm tree and costs a process spawn.
# Caches on the recipe's own stamp (pkg_begin), so a no-op on a warm tree.
bash "$_d/build.sh"
# Sourced in a subshell, one recipe at a time. PKG_DEPENDS from one recipe
# leaking into the next is precisely the bug that makes a package declare a
# dependency nobody wrote down, and it would install fine and stay wrong.
# A subshell per recipe: PKG_DEPENDS leaking into the next one is a package
# declaring a dependency nobody wrote.
(
pkg_conf "$r"
@@ -100,47 +60,26 @@ for r in "${RECIPES[@]}"; do
echo "!! $r: PKG_ROOT '$PKG_ROOT' does not exist -- did build.sh run?" >&2
exit 1; }
# THE ABI GATE, AT THE PACKAGE BOUNDARY, and now the only one on the
# release path. bin/patch.sh used to walk the assembled payload as
# well; it stopped once every file in the payload came out of an .ipk,
# because then the only thing that walk could see was three thousand
# files this loop had already answered for.
#
# It is HERE and not in a build.sh deliberately: a recipe's own gates
# do not run on a cache hit, and this one has to. It is gated over
# PKG_ROOT rather than over the finished archive because readelf cannot
# look inside a tarball.
# THE ABI GATE, at the package boundary. Here and not in a build.sh
# because a recipe's own gates do not run on a cache hit and this one
# must; over PKG_ROOT because readelf cannot look inside a tarball.
n=$(mips_abi_gate "$PKG_ROOT") || exit 1
say "$PKG_NAME: $n ELF object(s) pass nan2008/o32/mips32r2"
# ---------------------------------------------------- the layout
#
# opkg-build packages a DIRECTORY: the file tree exactly as it will
# appear on the target, plus a CONTROL/ subdirectory it lifts the
# metadata out of and does not ship. So the recipe's tree is laid down
# under $MODDIR here -- work/pkg/libsodium/lib/libsodium.so.26.2.0
# becomes ./usr/data/anvil/lib/libsodium.so.26.2.0 -- which is why
# `opkg install` needs no prefix of its own later, and why a package
# built for this mod can never land on the rootfs by accident: every
# path it owns is under /usr/data.
#
# cp -a and not cp: libsodium ships libsodium.so -> .so.26 ->
# .so.26.2.0, and the first of those is the name libnacl's dlopen
# fallback constructs. A copy that dereferenced them would put three
# identical 400KB files in the package and still work, until somebody
# wondered why it had tripled in size.
# --- the layout
# opkg-build packages a DIRECTORY: the tree as it appears on the target,
# plus a CONTROL/ it does not ship. Laid down under $MODDIR, so `opkg
# install` needs no prefix and nothing of ours can land on the rootfs.
# cp -a, not cp: libsodium's .so -> .so.26 -> .so.26.2.0 chain would
# otherwise become three identical 400KB files.
LAYOUT="work/.ipk-$PKG_NAME"
rm -rf "$LAYOUT"
mkdir -p "$LAYOUT$MODDIR" "$LAYOUT/CONTROL"
cp -a "$PKG_ROOT/." "$LAYOUT$MODDIR/"
# -maxdepth 1 is not a micro-optimisation. PKG_EXCLUDE is always
# ".version", the stamp pkg_end writes at the ROOT of a recipe's
# output, but `find -name` matches a basename at any depth -- and
# Mainsail ships a .version of its own at www/mainsail/.version. This
# deleted it, and anvil-mainsail shipped without it for as long as the
# payload was copied from work/pkg rather than from the package.
#
# PKG_EXCLUDE is a space-separated list of patterns, meant to split.
# -maxdepth 1: PKG_EXCLUDE is always ".version", the stamp pkg_end writes
# at a recipe's ROOT, but `find -name` matches a basename at any depth --
# and Mainsail ships its own www/mainsail/.version, which this deleted.
# PKG_EXCLUDE is a space-separated pattern list, meant to split.
# shellcheck disable=SC2086
for p in $PKG_EXCLUDE; do
find "$LAYOUT$MODDIR" -maxdepth 1 -name "$p" -exec rm -rf {} + 2>/dev/null || true
@@ -148,25 +87,9 @@ for r in "${RECIPES[@]}"; do
[ -n "$(find "$LAYOUT$MODDIR" \( -type f -o -type l \) -print -quit)" ] \
|| { echo "!! $r staged nothing -- empty package refused" >&2; exit 1; }
# ------------------------------------------- the runtime/dev split
#
# A library that is BOTH linked against and run produces two kinds of
# file from one build: binaries a printer executes, and headers and a
# static archive only a build machine will ever open. PKG_DEV_FILES
# names the second kind, and everything it matches is MOVED out of the
# layout into a second one that becomes <name>-dev.
#
# WHY MOVE RATHER THAN COPY. A file in both packages is a file two
# packages own, and opkg resolves that by letting whichever installed
# last win -- silently. The dev half is a partition of the build, not a
# view onto it, so every path lands in exactly one archive and
# removing either package can never delete a file the other still
# needs.
#
# WHAT IT BUYS: the headers and .a files stop being installed on a
# printer that has no compiler, and the feed still contains everything
# the next recipe needs to build against -- in a package that says so
# in its name and its section.
# --- the runtime/dev split
# PKG_DEV_FILES is MOVED, not copied: a file two packages own is resolved
# by letting whichever installed last win, silently.
DEVLAYOUT="work/.ipk-$PKG_NAME-dev"
rm -rf "$DEVLAYOUT"
if [ -n "$PKG_DEV_FILES" ]; then
@@ -182,32 +105,15 @@ for r in "${RECIPES[@]}"; do
done
[ -n "$(find "$DEVLAYOUT$MODDIR" \( -type f -o -type l \) -print -quit)" ] \
|| { echo "!! $r sets PKG_DEV_FILES and none of it matched" >&2; exit 1; }
# Directories the move emptied. Left behind they would be shipped
# as an empty include/ on every printer -- harmless, and exactly
# the sort of harmless that accumulates.
# Or every printer ships an empty include/.
find "$LAYOUT$MODDIR" -type d -empty -delete 2>/dev/null || true
fi
# ------------------------------------------- deterministic modes
#
# A PACKAGE MUST NOT DEPEND ON WHO BUILT IT. opkg-build's `-o 0 -g 0`
# settles ownership and SOURCE_DATE_EPOCH settles timestamps, and
# neither touches PERMISSIONS -- which arrive from whatever the source
# archive happened to carry, filtered through the umask of whoever
# extracted it.
#
# Measured: the Moonraker and HelixScreen packages built in the pinned
# container and on a developer's host differed, and nothing else did.
# GNU tar restores a directory's exact mode when it runs as root and
# applies the umask when it does not, so Moonraker's own drwxrwxr-x
# tree came out 0775 in the container and 0755 outside it. Two
# byte-different packages containing the same files, and no way to tell
# from the outside which one a printer got.
#
# 0755 for directories, 0755 for anything executable, 0644 for
# everything else -- the same normalisation every distro's packaging
# applies, for the same reason. Symlinks are left alone: their mode is
# not meaningful and chmod would follow them to the target.
# --- deterministic modes
# -o 0 -g 0 and SOURCE_DATE_EPOCH settle owner and mtime but not MODE:
# GNU tar restores a directory's exact mode as root and applies the umask
# otherwise, so Moonraker came out 0775 in the container, 0755 outside.
# Symlinks are left alone -- chmod would follow them.
for lay in "$LAYOUT" "$DEVLAYOUT"; do
[ -d "$lay$MODDIR" ] || continue
find "$lay$MODDIR" -type d -exec chmod 0755 {} +
@@ -216,20 +122,12 @@ for r in "${RECIPES[@]}"; do
done
# emit <layout> <name> <section> <depends> <description>
#
# The five fields opkg-build's own required_field() insists on, plus
# the ones opkg reads. Description LAST because it is the only one that
# may continue onto further lines, and a continuation line is defined
# as one starting with whitespace -- so a field after it would have to
# be unindented, and an unindented line is where the next stanza begins.
# Description LAST: it is the only field that may continue onto further
# lines, so any field after it would begin the next stanza.
emit() {
_lay=$1; _nm=$2; _sect=$3; _dep=$4; _desc=$5
# ONE LINE, WHATEVER THE RECIPE WROTE. A Depends list of thirteen
# packages does not fit on one line of a pkg.conf anybody wants to
# read, so anvil-moonraker's is wrapped -- and a raw newline inside
# a control field is a new stanza to some parsers and a folded
# continuation to others. Unquoted, so the shell splits on every
# run of whitespace and rejoins with single spaces.
# ONE LINE: a raw newline in a control field is a new stanza to some
# parsers and a continuation to others. Unquoted, so it re-splits.
# shellcheck disable=SC2086
_dep=$(echo $_dep)
{
@@ -240,13 +138,8 @@ for r in "${RECIPES[@]}"; do
printf 'Section: %s\n' "$_sect"
printf 'Priority: optional\n'
[ -n "$_dep" ] && printf 'Depends: %s\n' "$_dep"
# PROVIDES AND CONFLICTS, TOGETHER, BECAUSE THEY ONLY WORK
# TOGETHER. A virtual name that several packages Provide lets
# a dependency be satisfied by any one of them -- which is
# what "install either chamber config" means -- and on its own
# it also lets a printer install BOTH, whereupon two packages
# own one path and opkg resolves that by letting whichever
# unpacked last win, silently. Conflicts on the same virtual
# Provides alone lets a printer install BOTH chamber configs, and
# the last to unpack wins silently; Conflicts on the same virtual
# name is what makes "either" mean "exactly one".
# shellcheck disable=SC2086
[ -n "$PKG_PROVIDES" ] \
@@ -257,19 +150,10 @@ for r in "${RECIPES[@]}"; do
printf 'Description: %s\n' "$_desc"
} > "$_lay/CONTROL/control"
# Maintainer scripts, for the one recipe that has any. A file in
# $PKG_DIR/control/ is copied in verbatim -- opkg-build already
# knows what to do with postinst, prerm and conffiles.
#
# RUNTIME PACKAGE ONLY. The -dev half is headers and a static
# library; a postinst there would run on a machine that installed
# them by accident and act on a payload that is not its own.
#
# A postinst runs wherever the package is installed, and that
# includes the build: opkg runs maintainer scripts with
# IPKG_INSTROOT empty, so absolute paths are taken as written
# rather than rebased. One put here guards itself on something
# only a printer has -- see pkgs/anvil-core/control/postinst.
# Runtime package only: a postinst on the -dev half would act on a
# payload that is not its own. It also runs on the BUILD, with
# IPKG_INSTROOT empty, so one here guards on something only a printer
# has.
if [ "$_nm" = "$PKG_NAME" ] && [ -d "$PKG_DIR/control" ]; then
for _cs in "$PKG_DIR"/control/*; do
[ -f "$_cs" ] || continue
@@ -281,19 +165,9 @@ for r in "${RECIPES[@]}"; do
done
fi
# -o 0 -g 0: every file in the archive is owned by root. Without
# them opkg-build hands tar whatever uid the build ran as, which is
# a developer's account on one machine and a CI runner's on another
# -- two packages that differ in nothing that matters and compare
# unequal.
#
# BOTH PATHS ABSOLUTE, and that is a requirement rather than a
# style: opkg-build builds its scratch directory as
# "$dest_dir/IPKG_BUILD.$$" and then reads it from inside
# `( cd $pkg_dir/CONTROL && ... )`, so a relative destination
# resolves against the wrong directory and the build dies on a
# missing control_list -- an error that names a temporary file and
# not the argument that caused it.
# BOTH PATHS ABSOLUTE is a requirement: opkg-build reads its scratch
# dir from inside a `cd $pkg_dir/CONTROL`, so a relative destination
# dies on a missing control_list.
"$OPKG_BUILD_BIN" -o 0 -g 0 "$PWD/$_lay" "$PKG_FEED" > /dev/null
rm -rf "$_lay"
@@ -302,20 +176,14 @@ for r in "${RECIPES[@]}"; do
say "$_nm: $(basename "$_ipk") ($(du -h "$_ipk" | cut -f1))"
}
# The runtime half, unless the split took everything -- which is the
# normal case for a library that is only ever linked against, like
# zlib. An empty runtime package would be a name in the feed that
# installs nothing.
# Unless the split took everything -- normal for a link-only library.
if [ -n "$(find "$LAYOUT$MODDIR" \( -type f -o -type l \) -print -quit)" ]; then
emit "$LAYOUT" "$PKG_NAME" "$PKG_SECTION" "$PKG_DEPENDS" "$PKG_DESCRIPTION"
else
rm -rf "$LAYOUT"
fi
# The dev half depends on the runtime half by name when there is one:
# its headers describe that build, and installing them against a
# different version of the library is the mistake the dependency
# prevents.
# By name: the dev half's headers describe that build of the library.
if [ -d "$DEVLAYOUT/CONTROL" ]; then
if [ -f "$PKG_FEED/${PKG_NAME}_${PKG_VERSION}-${PKG_RELEASE}_${PKG_ARCH}.ipk" ]; then
_devdep="$PKG_NAME"
@@ -328,22 +196,10 @@ for r in "${RECIPES[@]}"; do
)
done
# ------------------------------------------------------------------ the prune
#
# A FULL BUILD OWNS THE FEED. Every .ipk here should be one some recipe under
# pkgs/ produces right now; anything else is an orphan, and an orphan in a feed
# is worse than a stray file because opkg-make-index puts it in the index and
# a printer can then install it.
#
# They appear the moment a package is RENAMED, which is not hypothetical: the
# runtime/dev split renamed anvil-zlib to anvil-zlib-dev, and the old archive
# sat in the feed advertising a package no recipe had built since. It would
# have installed perfectly and been impossible to rebuild.
#
# Only on a full build. `./bin/build-packages.sh s6` asks for one recipe and
# its dependencies and must not delete everything it did not ask about --
# which is also why the expected set below is computed from ALL recipes rather
# than from the ones this run happened to build.
# --- the prune
# An orphan .ipk goes into the index and a printer can install a package no
# recipe builds any more; renames leave them behind. Full builds only, and the
# expected set comes from ALL recipes, not the ones this run built.
if [ $# -eq 0 ]; then
expected=""
for r in $(pkg_recipes); do
@@ -361,31 +217,10 @@ if [ $# -eq 0 ]; then
done
fi
# ------------------------------------------------------------------ the index
#
# `Packages` is what turns a directory of .ipk files into something opkg can be
# pointed at: one stanza per package -- the package's own control file, plus
# the fields only the feed can know (where the file is, how big it is, what it
# hashes to). opkg-make-index is upstream's tool for exactly this and is used
# for exactly the same reason opkg-build is.
#
# THE HASH IS THE POINT, not the convenience. It is the same argument
# versions.env makes about vendored tarballs: once the index is trusted, the
# packages under it do not have to be, because opkg refuses one whose sha256
# does not match its stanza. Signing the INDEX (usign, phase 3) therefore signs
# the whole feed, and nothing has to sign each package.
#
# Both plain and gzipped, because opkg asks for Packages.gz first and falls
# back to Packages, and a feed served off a USB stick or a laptop's `python3 -m
# http.server` is easier to eyeball when the plain one is there too.
#
# --checksum sha256 IS NOT THE DEFAULT and has to be asked for: opkg-make-index
# writes MD5Sum alone unless told otherwise, and an index whose only integrity
# claim is MD5 cannot carry the argument above -- a collision against a stanza
# is a solved problem, and the whole point of trusting the index instead of the
# packages is that it is the thing worth attacking. md5 is kept beside it
# because opkg still reports it and some tooling looks for it; the sha256 is
# what actually gates an install.
# --- the index
# opkg refuses a package whose sha256 does not match its stanza, so signing the
# INDEX signs the whole feed. --checksum sha256 is not the default; opkg asks
# for Packages.gz first and falls back to Packages, so both are written.
say "index: writing $PKG_FEED/Packages"
( cd "$PKG_FEED" && python3 "$OPKG_INDEX_BIN" --checksum md5 --checksum sha256 . > Packages )
gzip -n -9 -c "$PKG_FEED/Packages" > "$PKG_FEED/Packages.gz"
bin/build.sh
+1 -5
View File
@@ -1,9 +1,5 @@
#!/usr/bin/env bash
# One-shot: fetch -> unpack -> patch -> pack.
#
# ./bin/build.sh build the package
# ./bin/build.sh --full options are passed through to pack.sh
# (--slim is accepted but is the default)
# One-shot: fetch -> unpack -> patch -> pack. Options pass through to pack.sh.
set -euo pipefail
SCRIPT_DIR="$(dirname "$0")"
"$SCRIPT_DIR/fetch-assets.sh"
bin/common.sh
+40 -225
View File
@@ -1,10 +1,6 @@
# Sourced by every script in bin/. Resolves the repo root, loads config.env
# and exports the feature flags.
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")/.." && pwd)"
cd "$ROOT"
# CONFIG_ENV=<path> points at a different one. The test suite uses it to run
# against a throwaway config instead of writing over the one you edited.
CONFIG_ENV="${CONFIG_ENV:-$ROOT/config.env}"
if [ ! -f "$CONFIG_ENV" ]; then
if [ -f config.env.example ]; then
@@ -16,108 +12,39 @@ fi
# shellcheck disable=SC1090
. "$CONFIG_ENV"
# WHAT GOES INTO THE PACKAGE. There is exactly one build -- the firmware:
# forked Klipper with toolchanger support, Mainsail/Moonraker, ssh, and
# HelixScreen driving the touchscreen in place of FlashForge's UI.
#
# KLIPPER IS NOT ON THIS LIST any more. BUILD_KLIPPER=stock kept FlashForge's
# v0.12 tree, and every other thing we ship is now written against the fork:
# the ff_*.py extras are anvil-klipper's own files, the ff-*.cfg declare
# sections only they implement, and c_helper.so is built from the tree beside
# it. A stock build was a printer with the mod's configuration and none of the
# code behind it.
#
# Plain defaults, overridable from config.env, which is sourced above.
#
# THE MOD_* SWITCHES ARE GONE -- MOD_WEB, MOD_CAM, MOD_UI, MOD_SSH, MOD_WIFI.
# Every one of them defaulted to 1, and each bought a second state that every
# init script, gate and test then had to reason about: nginx that might not
# run, a camera that might not stream, a screen that might be off on purpose.
# Nobody wanted those states -- what people want when they set one is a stock
# printer, which is one FlashForge package away. What remains configurable is
# what has a real range: the BUILD_* flags below decide what goes IN a build,
# and the MOD_CAM_* and NICE_* values in anvil.conf tune what is running.
#
# FlashForge's firmwareExe is REPLACED, not kept: HelixScreen is the only UI.
# If it will not start, `opkg remove anvil-helixscreen` boots headless.
# ssh and Mainsail are your recovery path if the screen is dark, and
# a USB stick with the STOCK FlashForge package on it is the uninstall for
# everything that came out of a package (`make test-recovery`). Moonraker is
# the exception: it lives only on the factory image, so a reflash cannot put
# FlashForge's back -- see BUILD_MOONRAKER and docs/how-it-works.md.
BUILD_TOOLCHANGE="${BUILD_TOOLCHANGE:-1}"
BUILD_MAINSAIL="${BUILD_MAINSAIL:-1}"
BUILD_MOONRAKER="${BUILD_MOONRAKER:-1}"
BUILD_HELIX="${BUILD_HELIX:-1}"
# Everything we add to the printer lives under this one directory on the DATA
# partition, so a FlashForge OTA cannot delete it. It is a --prefix root and
# not a junk drawer: bin/, lib/, libexec/, etc/ mean what they mean anywhere
# else, and s6 and CPython are both CONFIGURED with this path, so it is baked
# into shipped binaries and moving it means rebuilding them. Defined here
# because the recipes under pkgs/ need the same answer and must not be free to
# give a different one.
# A --prefix root on the DATA partition, so a FlashForge OTA cannot delete it.
# s6 and CPython are configured with this path, so moving it rebuilds them.
MODDIR="${MODDIR:-/usr/data/anvil}"
# Where the payload is assembled, before bin/pack.sh tars it into
# anvil.tar.xz. TWO NAMES because opkg unpacks a package's paths -- which are
# ./usr/data/anvil/... -- relative to the root it is given, so the payload is
# $MODDIR deep inside that root. patch.sh installs into the root and finishes
# the payload; pack.sh only tars the payload.
#
# Here rather than local to patch.sh because pack.sh, the Makefile's clean
# target and test/run-tests.py's teardown all name this tree, and the three
# that only delete it would have failed silently.
# Two names because opkg unpacks a package's ./usr/data/anvil/... paths relative
# to the root it is given: the payload sits $MODDIR deep inside PAYLOAD_ROOT.
PAYLOAD_ROOT="${PAYLOAD_ROOT:-$ROOT/work/modpayload-root}"
PAYLOAD_DIR="${PAYLOAD_DIR:-$PAYLOAD_ROOT$MODDIR}"
export PAYLOAD_ROOT PAYLOAD_DIR
# The Ingenic GLIBC cross-toolchain and its tool prefix: gcc 7.2.0 / glibc
# 2.29 for the X2000, the one that produces this printer's ABI. Used by every
# recipe that compiles, through pkg_toolchain. MIPS_TOOLCHAIN_TGZ below is the
# tarball it is unpacked from; this is where it lands.
PY_HOST="${PY_HOST:-mips-linux-gnu}"
PY_TOOLCHAIN_DIR="${PY_TOOLCHAIN_DIR:-work/.mips-toolchain/mips-gcc720-glibc229}"
# The architecture string stamped into every .ipk this repo builds, and into
# the feed index beside them. See docs/notes/85-packaging.md.
#
# DELIBERATELY NOT AN OpenWrt NAME. OpenWrt's nearest label for this silicon
# is `mipsel_24kc` -- same ISA, same o32 ABI, but musl. Those packages would
# satisfy an opkg dependency here, install without complaint and fail to load
# against the printer's glibc 2.29. A name no public feed uses makes an
# OpenWrt .ipk get refused on architecture, before it is unpacked.
#
# xburst2 is Ingenic's name for the X2000's core. What the string promises is
# the ABI mips_abi_gate checks -- o32, nan2008, mips32r2 -- linked against the
# Ingenic glibc.
# Deliberately not an OpenWrt name: mipsel_24kc is musl, so such an .ipk would
# satisfy a dependency here, install cleanly and fail against glibc 2.29. An
# unknown name gets it refused on architecture instead.
IPK_ARCH="${IPK_ARCH:-mipsel_xburst2}"
export MODDIR PY_HOST PY_TOOLCHAIN_DIR IPK_ARCH
# Third-party payload pieces (Mainsail, HelixScreen, Moonraker). They are
# downloaded on demand rather than vendored, so the repo carries no binaries
# and no binary history. versions.env pins the version and the sha256;
# bin/fetch-assets.sh puts the file in vendor/. Point MAINSAIL_ZIP /
# HELIX_TGZ / MOONRAKER_TGZ at your own build in config.env to override --
# but an explicit path is checksummed like any other, and fetch-assets.sh
# overwrites it with the pinned release when the hash does not match. Put
# your file's own sha256 in versions.env to keep it.
# Downloaded on demand, pinned in versions.env. An explicit override path is
# checksummed too and replaced when the hash differs -- put its own sha256 there.
# shellcheck disable=SC1091
[ -f "$ROOT/versions.env" ] && . "$ROOT/versions.env"
MAINSAIL_ZIP="${MAINSAIL_ZIP:-$ROOT/vendor/mainsail-${MAINSAIL_VERSION:-unpinned}.zip}"
HELIX_TGZ="${HELIX_TGZ:-$ROOT/vendor/${HELIX_FILE:-helixscreen.tar.gz}}"
MOONRAKER_TGZ="${MOONRAKER_TGZ:-$ROOT/vendor/moonraker-${MOONRAKER_VERSION:-unpinned}.tar.gz}"
# The Klipper fork tarball and the MIPS toolchain that compiles its chelper.
# Both are consumed by pkgs/klipper; see versions.env for why the pin exists.
KLIPPER_TGZ="${KLIPPER_TGZ:-$ROOT/vendor/klipper-${KLIPPER_VERSION:-unpinned}.tar.gz}"
MIPS_TOOLCHAIN_TGZ="${MIPS_TOOLCHAIN_TGZ:-$ROOT/vendor/${MIPS_TOOLCHAIN_FILE:-mips-toolchain.tar.gz}}"
# The supervision stack: skalibs (s6's own C library), execline (which s6-rc
# links unconditionally), s6 itself and s6-rc. Every package ships these, so
# unlike the Klipper pieces there is no flag that switches them off -- see
# versions.env for why the build is the way it is.
#
# Four tarballs and no cache variable: all four are recipes under pkgs/, so the
# output path is pkg_out's business and staleness is pkg_stale's.
SKALIBS_TGZ="${SKALIBS_TGZ:-$ROOT/vendor/skalibs-${SKALIBS_VERSION:-unpinned}.tar.gz}"
S6_TGZ="${S6_TGZ:-$ROOT/vendor/s6-${S6_VERSION:-unpinned}.tar.gz}"
EXECLINE_TGZ="${EXECLINE_TGZ:-$ROOT/vendor/execline-${EXECLINE_VERSION:-unpinned}.tar.gz}"
@@ -126,15 +53,8 @@ S6RC_TGZ="${S6RC_TGZ:-$ROOT/vendor/s6-rc-${S6RC_VERSION:-unpinned}.tar.gz}"
export MAINSAIL_ZIP HELIX_TGZ MOONRAKER_TGZ KLIPPER_TGZ MIPS_TOOLCHAIN_TGZ
export SKALIBS_TGZ S6_TGZ EXECLINE_TGZ S6RC_TGZ
# CPython and the seven C libraries it links against, each pinned here and
# built by its own recipe under pkgs/. Eight tarballs and not one, because
# sqlite, openssl, libffi, xz, bzip2, zlib and expat are separate projects the
# interpreter merely knows how to use, and none of them can be borrowed from
# this printer's rootfs -- which is why the stock 3.8.2 has no sqlite3 module.
#
# The compiler is MIPS_TOOLCHAIN_TGZ above, the Ingenic GLIBC one: a
# musl-linked interpreter cannot dlopen a glibc c_helper.so, and dlopen is
# exactly how klippy loads it. See tools/python/README.md.
# Built with the GLIBC toolchain above: a musl-linked interpreter cannot dlopen
# a glibc c_helper.so, and dlopen is how klippy loads it.
PY_TGZ="${PY_TGZ:-$ROOT/vendor/Python-${PY_VERSION:-unpinned}.tgz}"
OPENSSL_TGZ="${OPENSSL_TGZ:-$ROOT/vendor/openssl-${OPENSSL_VERSION:-unpinned}.tar.gz}"
SQLITE_TGZ="${SQLITE_TGZ:-$ROOT/vendor/sqlite-autoconf-${SQLITE_VERSION:-unpinned}.tar.gz}"
@@ -144,196 +64,93 @@ XZ_TGZ="${XZ_TGZ:-$ROOT/vendor/xz-${XZ_VERSION:-unpinned}.tar.gz}"
BZIP2_TGZ="${BZIP2_TGZ:-$ROOT/vendor/bzip2-${BZIP2_VERSION:-unpinned}.tar.gz}"
EXPAT_TGZ="${EXPAT_TGZ:-$ROOT/vendor/expat-${EXPAT_VERSION:-unpinned}.tar.gz}"
# The ABI series CPython names its own directories and binaries after --
# bin/python3.13, lib/python3.13/, config-3.13-mipsel-linux-gnu. Spelled out
# rather than sed'd out of PY_VERSION so that a 3.14 bump is a deliberate
# edit.
# Spelled out rather than derived from PY_VERSION so a 3.14 bump is deliberate.
PY_MM="${PY_MM:-3.13}"
export PY_MM
export PY_TGZ OPENSSL_TGZ SQLITE_TGZ ZLIB_TGZ LIBFFI_TGZ XZ_TGZ BZIP2_TGZ
export EXPAT_TGZ
# The third-party python packages that become the interpreter's
# site-packages, and libsodium, which libnacl dlopens out of $MODDIR/lib.
#
# The list moves, so these are not one variable per tarball the way everything
# above is: versions.env carries PYPKG_LIST plus a FILE/SHA256 pair per entry,
# and the three helpers below are how both bin/fetch-assets.sh and bin/patch.sh
# read that table. Written once, because a package the fetcher and the builder
# disagree about is a build that downloads one file and compiles another.
#
# pypkg_var takes the list entry (`streaming-form-data`) and a suffix and
# returns the variable versions.env spells for it
# (PYPKG_STREAMING_FORM_DATA_FILE). Bash indirection, which is why this file
# is only ever sourced by bin/*.sh -- payload/* is busybox ash and has none.
# pypkg_var maps a PYPKG_LIST entry plus a suffix to the variable versions.env
# spells for it (streaming-form-data -> PYPKG_STREAMING_FORM_DATA_FILE), by bash
# indirection -- which is why this file is never sourced by payload/*'s ash.
pypkg_var() {
local _n _v
_n=$(printf '%s' "$1" | tr 'a-z' 'A-Z' | tr '-' '_')
_v="PYPKG_${_n}_$2"
printf '%s' "${!_v-}"
}
# The sdist as bin/fetch-assets.sh leaves it in vendor/.
pypkg_tgz() { printf '%s/vendor/%s' "$ROOT" "$(pypkg_var "$1" FILE)"; }
# The version, READ OUT OF THE PINNED FILE NAME rather than pinned a second
# time beside it -- a package whose .ipk claims a version it does not contain
# is what a second pin buys. Every sdist on PyPI is <name>-<version>.tar.gz and
# the version is the part after the LAST dash, which is what makes this safe
# for the entries whose file name is not their list name: inotify_simple-1.3.5,
# streaming_form_data-1.19.1, pyserial-asyncio-0.6.
# Version taken from the pinned file name, after the LAST dash -- which is what
# makes inotify_simple-1.3.5 and pyserial-asyncio-0.6 both come out right.
pypkg_version() {
local _f
_f="$(pypkg_var "$1" FILE)"
_f=${_f%.tar.gz}; _f=${_f%.tgz}; _f=${_f%.zip}
printf '%s' "${_f##*-}"
}
# The sources every recipe under pkgs/ builds from. ZLIB_TGZ is NOT repeated
# here: it is pinned thirty lines up for CPython, and pkgs/3rdparty/zlib builds that same
# tarball once for everybody.
# ZLIB_TGZ is pinned above for CPython; pkgs/3rdparty/zlib builds it once for all.
SODIUM_TGZ="${SODIUM_TGZ:-$ROOT/vendor/libsodium-${SODIUM_VERSION:-unpinned}.tar.gz}"
OPKG_TGZ="${OPKG_TGZ:-$ROOT/vendor/opkg-${OPKG_VERSION:-unpinned}.tar.gz}"
LIBARCHIVE_TGZ="${LIBARCHIVE_TGZ:-$ROOT/vendor/libarchive-${LIBARCHIVE_VERSION:-unpinned}.tar.gz}"
export SODIUM_TGZ OPKG_TGZ LIBARCHIVE_TGZ
# WHERE A RECIPE'S OUTPUT LIVES IS DERIVED, NOT NAMED: pkgs/lib.sh's pkg_out
# spells work/pkg/<recipe>, so adding a package is one directory under pkgs/ and
# no edit to this file. These three aliases remain only because bin/patch.sh
# and bin/fetch-assets.sh refer to them. Nothing new should be added below.
# Output paths are derived by pkgs/lib.sh's pkg_out; these three aliases remain
# only because patch.sh and fetch-assets.sh name them. Add nothing here.
SODIUM_BUILD="${SODIUM_BUILD:-$ROOT/work/pkg/libsodium}"
OPKG_BUILD="${OPKG_BUILD:-$ROOT/work/pkg/opkg}"
ZLIB_BUILD="${ZLIB_BUILD:-$ROOT/work/pkg/zlib}"
export SODIUM_BUILD OPKG_BUILD ZLIB_BUILD
# The feed: where bin/build-packages.sh writes .ipk files and the index, and
# where pkgs/lib.sh's pkg_deps reads a recipe's build dependencies back out of.
# It is a local opkg repository, and it is the interface between recipes --
# which is why the path is defined once here rather than in the script that
# happens to write it.
PKG_FEED="${PKG_FEED:-$ROOT/work/packages}"
# The upstream tools, all three from one pinned checkout: opkg-build makes a
# package, opkg-make-index makes the feed, opkg-unbuild takes a package apart
# again to fill a build sysroot.
#
# OPKG_UTILS_DIR is a git CHECKOUT and not a tarball, because opkg-utils
# publishes no release archive anywhere -- see versions.env. It is the one
# vendor/ entry whose integrity is a commit sha rather than a sha256.
# A git checkout, not a tarball: opkg-utils publishes no release archive, so its
# integrity is a commit sha rather than a sha256.
OPKG_UTILS_DIR="${OPKG_UTILS_DIR:-$ROOT/vendor/opkg-utils}"
OPKG_BUILD_BIN="${OPKG_BUILD_BIN:-$OPKG_UTILS_DIR/opkg-build}"
OPKG_INDEX_BIN="${OPKG_INDEX_BIN:-$OPKG_UTILS_DIR/opkg-make-index}"
OPKG_UNBUILD_BIN="${OPKG_UNBUILD_BIN:-$OPKG_UTILS_DIR/opkg-unbuild}"
export PKG_FEED OPKG_UTILS_DIR OPKG_BUILD_BIN OPKG_INDEX_BIN OPKG_UNBUILD_BIN
# Replica-only settings: the factory image and the partition sizes. They exist
# for the tests and never reach a printer, so they live in their own file --
# see test.env.example. Values left in config.env keep working.
TEST_ENV="${TEST_ENV:-$ROOT/test.env}"
# shellcheck disable=SC1090
[ -f "$TEST_ENV" ] && . "$TEST_ENV"
# The version is the release date, today's, as YYYYMMDD. It only appears in the
# outer filename -- the stock installer reads the software component's own
# version, never this one -- so a date says something true about the build,
# where a semver would just be a number nobody remembers to bump.
#
# Set MOD_VER explicitly to pin it: for a reproducible rebuild of an old
# release, or for a second release on the same day (e.g. 20260824b).
# The release date, and only ever in the outer filename. Set MOD_VER to pin it.
MOD_VER="${MOD_VER:-$(date -u +%Y%m%d)}"
export MOD_VER
# Which model are we building for? Packages are model-specific: the two
# stock packages carry different firmwareExe binaries and each refuses to
# install on the other model, so one build can never serve both.
# Each stock package carries its own firmwareExe and refuses the other model.
TARGET_MACHINE="${MODEL:-${TARGET_MACHINE:-Creator5Pro}}"
case "$TARGET_MACHINE" in
Creator5Pro) TARGET_PID=0029; _stock="${STOCK_TGZ_CREATOR5PRO:-}" ;;
Creator5) TARGET_PID=0028; _stock="${STOCK_TGZ_CREATOR5:-}" ;;
*) echo "TARGET_MACHINE must be Creator5 or Creator5Pro (got '$TARGET_MACHINE')" >&2; exit 1 ;;
esac
# An explicit STOCK_TGZ always wins; otherwise use the per-model one.
if [ -z "${STOCK_TGZ:-}" ] && [ -n "$_stock" ]; then
STOCK_TGZ="$_stock"
fi
# PROG_DUMP is a real /usr/prog for the printer replica. One factory image
# serves both models: only the Pro's was ever published, and the replica
# installs the model's own stock package on top of it, which replaces every
# model-specific file it contains.
# One factory image serves both models; the replica installs the model's own
# stock package over it.
export TARGET_MACHINE TARGET_PID STOCK_TGZ PROG_DUMP
# ---------------------------------------------------------------- the ABI gate
# Moved here from bin/patch.sh, which was its only caller until the packaging
# lane arrived. bin/build-packages.sh can produce an .ipk without patch.sh ever
# running -- that is the point of `make packages` -- so the gate has to be
# reachable from both or the packaged copy of a library ships unchecked while
# the tarball's copy does not. Same rule, same two words, one definition.
#
# ABI, over every ELF this build cross-compiles: s6 in 5b below, the
# interpreter and its extensions in 5c, libsodium in 5d. The printer's kernel
# wants nan2008/o32/mips32r2 and says ENOEXEC to anything else, and a
# cross-build that quietly emitted one host object -- or one legacy-NaN
# object, because a flag did not reach one link line -- looks like a clean
# build here. Defined once, up here, because 5b needs it before 5c exists to
# borrow it from.
#
# TWO expected words, not one. 0x70001405 is the measured value for an
# EXECUTABLE; a shared object additionally carries EF_MIPS_PIC (0x2) and so
# reads 0x70001407. That is correct and unavoidable for a DYN -- klippy's own
# c_helper.so has it too -- so a gate that pinned one word would fail on every
# extension module in the tree. s6's own binaries are static EXECs, so they
# want 0x70001405 like the interpreter does.
#
# A FUNCTION, AND POINTED AT THE PAYLOAD rather than at any one build tree: a
# .so staged into $PAYLOAD_DIR by a path the gate did not know about ships
# ungated, and the first machine to notice is a printer. The payload is by
# definition everything that ships.
#
# NOTHING IS EXEMPT, s6 included. A legacy-NaN binary (e_flags 0x1007, from a
# mips32r1 build) looks harmless if you assume the NaN encoding only matters to
# floating point -- but it matters at exec(): a MIPS kernel built nan2008-only
# can refuse to run it outright, or silently misconfigure its FPU mode. Neither
# shows up under qemu-mipsel-static, because user-mode emulation does not
# enforce the ABI check a real kernel's binfmt loader does, which is exactly
# how such a binary once shipped with every replica gate still passing.
# --- the ABI gate
# The printer's kernel wants nan2008/o32/mips32r2 and says ENOEXEC to anything
# else. Pointed at the PAYLOAD, not a build tree, so nothing staged by an
# unknown path ships ungated. Nothing is exempt: qemu-mipsel-static does not
# enforce the check a real kernel's binfmt loader does, which is how a
# legacy-NaN binary once shipped with every gate green.
mips_abi_gate() {
local n=0 f hdr counts total good
while IFS= read -r f; do
# readelf itself is the ELF test, rather than comparing the first four
# bytes to \177ELF: that comparison was a command substitution over
# arbitrary binary content, and every data file in site-packages that
# happens to start with a NUL made bash print "warning: command
# substitution: ignored null byte in input" -- eight lines of noise
# across a clean build, from the gate that is supposed to be the quiet
# one. readelf exits non-zero on anything that is not an ELF, which is
# the same question asked of the tool that has to answer it anyway.
# readelf is the ELF test: a magic-byte compare made bash warn about
# ignored null bytes on every data file that starts with a NUL.
hdr=$(readelf -h "$f" 2>/dev/null) || continue
# ONE HEADER PER MEMBER, NOT ONE PER FILE. readelf -h on a static
# archive prints a full ELF header for every object inside it --
# measured: 11 for an 11-member .a, 298 for a host libgcc.a. Reading a
# single `Flags:` line would hand a multi-line string to a `case`
# expecting one word, and every .a would fail the gate with an error
# naming a value nobody could parse. pkgs/3rdparty/zlib and pkgs/3rdparty/skalibs are the
# recipes that ship an archive.
#
# So every header is checked and every header has to conform. Both
# conditions live on the same awk line because readelf prints them
# there: `Flags: 0x70001407, noreorder, pic, cpic, nan2008, o32,
# mips32r2` is one line carrying the word and the hex together, and
# checking them per-line is what keeps one member's flags from being
# matched against another member's ISA.
#
# 0x7000140[0-7] -- the ABI is the HIGH bits and the low three are not
# part of it. 0x70001400 is EF_MIPS_ARCH_32R2 | EF_MIPS_ABI_O32 |
# EF_MIPS_NAN2008, which is the whole of what this gate is for. The
# bottom three bits are EF_MIPS_NOREORDER (0x1), EF_MIPS_PIC (0x2) and
# EF_MIPS_CPIC (0x4), none of which says anything about whether the
# printer's kernel will exec the file.
#
# A MASK, NOT THE PAIR 0x70001405/0x70001407. Those two are what
# linked BINARIES read, because crt startup objects happen to set
# NOREORDER; object files need not. libarchive's xxhash.o comes out
# 0x70001406 -- identical ABI, no NOREORDER -- and an exact whitelist
# calls it a wrong-ABI object.
# One header per MEMBER: readelf -h on a static archive prints one per
# object, so every header must conform, and hex and words are matched on
# the same awk line so one member's flags are not paired with another's
# ISA. A mask, not the exact 0x70001405/07 pair: the low bits are
# NOREORDER/PIC/CPIC (libarchive's xxhash.o reads 0x70001406, same ABI).
counts=$(awk '
/Flags:/ {
total++
@@ -343,9 +160,7 @@ mips_abi_gate() {
}
END { print (total + 0), (good + 0) }' <<<"$hdr")
total=${counts% *}; good=${counts#* }
# total==0 is a failure and not a pass: it means readelf recognised
# the file and then found nothing to check, which is the shape of a
# gate that looks green because it examined nothing.
# total==0 is a failure: readelf saw the file and found nothing to check.
if [ "$total" -eq 0 ] || [ "$total" != "$good" ]; then
echo " !! $f is not nan2008/o32/mips32r2 with e_flags 0x7000140[0-7]" >&2
echo " ($good of $total ELF header(s) conform)" >&2
bin/fetch-assets.sh
+37 -130
View File
@@ -1,19 +1,12 @@
#!/usr/bin/env bash
# Download the pinned third-party payload pieces into vendor/.
#
# ./bin/fetch-assets.sh whatever the BUILD_* flags ask for
# (BUILD_MAINSAIL / BUILD_HELIX / BUILD_MOONRAKER)
# ./bin/fetch-assets.sh --all all three, regardless of those flags
#
# Nothing here is committed: the repo stays free of binaries and the versions
# live in versions.env. A cached file with the right sha256 is never
# re-downloaded, so this is a no-op on every build after the first.
# ./bin/fetch-assets.sh [--all] --all ignores the BUILD_* flags
# Pins live in versions.env; a cached file with the right sha256 is not refetched.
set -euo pipefail
# shellcheck disable=SC1091
. "$(dirname "$0")/common.sh"
# Sourced for pkg_needs alone, which answers "is any recipe under pkgs/ going to
# have to compile something?" -- the question that decides whether the ~203MB
# toolchain is worth downloading. It defines functions and nothing else.
# Sourced for pkg_needs alone: "is any recipe going to have to compile?" -- the
# question that decides whether the ~203MB toolchain is worth downloading.
# shellcheck disable=SC1091
. "$ROOT/pkgs/lib.sh"
say() { printf '>> %s\n' "$*"; }
@@ -35,19 +28,9 @@ get() {
say "stale $(basename "$dest") -- re-downloading"
fi
say "fetch $url"
# --retry: this fetches from several hosts and a plain -f turns any one
# of them having a bad minute into a hard build failure. Written after
# musl.cc -- the previous source for the s6 toolchain, replaced below
# over exactly this -- died mid-connect on CI runners often enough to be
# the norm rather than the exception; kept general because the next flaky
# host will not announce itself in advance.
#
# --connect-timeout: without it, a host that is not answering at all --
# not refusing, just silent -- makes curl wait on its own default (well
# over a minute) before it even counts as a failed attempt, so 3 retries
# can take the better part of ten minutes to give up. 20s is generous for
# a TCP handshake to a reachable host and turns the same three retries
# into about a minute and a half of real waiting.
# --retry: several hosts, and a plain -f turns one bad minute into a failed
# build. --connect-timeout: without it a silent host burns curl's own
# default (over a minute) before an attempt even counts as failed.
curl -fL --progress-bar --connect-timeout 20 \
--retry 5 --retry-connrefused --retry-all-errors --retry-delay 5 \
-o "$dest.part" "$url"
@@ -73,52 +56,33 @@ if [ "$ALL" = 1 ] || [ "${BUILD_MAINSAIL:-0}" = "1" ]; then
fi
if [ "$ALL" = 1 ] || [ "${BUILD_HELIX:-0}" = "1" ]; then
# Klipper4FlashForge is the org's current name; the old Klipper4Creator5
# URL only worked through GitHub's rename redirect, which dies the day
# someone claims the old handle.
# Klipper4FlashForge is the org's current name; the old URL only worked
# through GitHub's rename redirect.
get "https://github.com/Klipper4FlashForge/helixscreen/releases/download/$HELIX_VERSION/$HELIX_FILE" \
"$HELIX_TGZ" "$HELIX_SHA256"
fi
# The Klipper fork sources. There is one source now: KLIPPER_FORK -- the
# config.env knob that pointed the build at a local checkout instead -- is
# gone, because pkgs/klipper is a recipe and a recipe names its source exactly
# once. A checkout is no longer a second way in; re-pin KLIPPER_VERSION and
# KLIPPER_SHA256 at your own tarball to build something else.
# No BUILD_ flag: the fork is the only Klipper, so this comes down on every
# build like s6 and CPython do.
# One source, named once, because pkgs/klipper is a recipe. Re-pin
# KLIPPER_VERSION and KLIPPER_SHA256 to build something else.
get "https://github.com/Klipper4FlashForge/klipper/archive/$KLIPPER_VERSION.tar.gz" \
"$KLIPPER_TGZ" "$KLIPPER_SHA256"
# The Ingenic glibc toolchain -- ~203MB, and shared by every recipe that
# compiles: the interpreter and its seven libraries, the supervision stack,
# libsodium, opkg, and chelper.
#
# ONE CONDITION, WHERE THERE USED TO BE FOUR. The hand-written stamp
# comparisons that sat here were the same question in a second spelling, and
# the second spelling is the one that drifts. pkg_needs answers for every
# recipe at once, so the fetcher computes the cache key with the code that
# writes it -- which is exactly what went wrong with the s6 stamp below. The
# separate Klipper clause went with them: chelper is pkgs/klipper now, so
# pkg_needs already covers it.
# The Ingenic glibc toolchain -- ~203MB, shared by every recipe that compiles.
# The condition comes from pkg_needs, the code that writes the recipe stamps,
# rather than a second spelling that drifts.
if [ "$ALL" = 1 ] || pkg_needs; then
get "https://github.com/ballaswag/k1-discovery/releases/download/$MIPS_TOOLCHAIN_VERSION/$MIPS_TOOLCHAIN_FILE" \
"$MIPS_TOOLCHAIN_TGZ" "$MIPS_TOOLCHAIN_SHA256"
fi
if [ "$ALL" = 1 ] || [ "${BUILD_MOONRAKER:-0}" = "1" ]; then
# Moonraker ships no release asset, so this is GitHub's generated source
# tarball. /archive/<ref> takes a tag OR a commit sha, which matters
# because the pin is currently a commit -- see versions.env for why there
# is no release we can use. The sha256 is what makes either safe.
# Moonraker ships no release asset; /archive/<ref> takes a tag or a commit
# sha, and the pin is a commit. The sha256 is what makes either safe.
get "https://github.com/Arksine/moonraker/archive/$MOONRAKER_VERSION.tar.gz" \
"$MOONRAKER_TGZ" "$MOONRAKER_SHA256"
fi
# The supervision stack: skalibs, execline, s6, s6-rc. Every package ships
# them, so there is no BUILD_ flag here -- these come down on every build, not
# on request. Four tarballs, well under a megabyte between them.
#
# The supervision stack. No BUILD_ flag: every package ships it.
get "https://skarnet.org/software/skalibs/skalibs-$SKALIBS_VERSION.tar.gz" \
"$SKALIBS_TGZ" "$SKALIBS_SHA256"
get "https://skarnet.org/software/execline/execline-$EXECLINE_VERSION.tar.gz" \
@@ -128,34 +92,18 @@ get "https://skarnet.org/software/s6/s6-$S6_VERSION.tar.gz" \
get "https://skarnet.org/software/s6-rc/s6-rc-$S6RC_VERSION.tar.gz" \
"$S6RC_TGZ" "$S6RC_SHA256"
# CPython and the seven C libraries it links against. Like s6 there is no
# BUILD_ flag: every package ships the interpreter, so these come down on
# every build. They are ~40MB in total and cached by sha256 like everything
# else, so that is a first-build cost and nothing after it.
#
# The Ingenic toolchain that compiles them is fetched further up, on
# pkg_needs -- so it is asked for again here, on the condition that decides
# whether patch.sh has to compile at all. Without this a build could download
# 40MB of source and then stop because there is no compiler for it, halfway
# through patch.sh rather than here where the fix is one command.
#
# The condition is the same version stamp patch.sh writes: work/.py313 holds
# the cross-built tree and names the versions it came from, so a checkout that
# already has a current one pulls neither the ~203MB compiler nor a second
# copy of anything.
# The compiler is fetched above on the same pkg_needs condition, asked again
# here so a build cannot pull 40MB of source and then stop for want of it.
get "https://www.python.org/ftp/python/$PY_VERSION/Python-$PY_VERSION.tgz" \
"$PY_TGZ" "$PY_SHA256"
get "https://github.com/openssl/openssl/releases/download/openssl-$OPENSSL_VERSION/openssl-$OPENSSL_VERSION.tar.gz" \
"$OPENSSL_TGZ" "$OPENSSL_SHA256"
# sqlite.org files the amalgamation under the YEAR it was released, which is
# nowhere in the version number -- hence SQLITE_YEAR in versions.env. Get it
# wrong and this 404s rather than fetching the wrong thing, which is the
# failure mode to prefer.
# sqlite.org files the amalgamation under the YEAR of release, which is nowhere
# in the version number -- hence SQLITE_YEAR.
get "https://www.sqlite.org/$SQLITE_YEAR/sqlite-autoconf-$SQLITE_VERSION.tar.gz" \
"$SQLITE_TGZ" "$SQLITE_SHA256"
# /fossils/ and not the front page: zlib.net moves a release there the moment
# it is superseded, and the front-page URL then 404s for a pin that was
# perfectly good yesterday. The fossils path is stable for both.
# /fossils/, not the front page: zlib.net moves a release there the moment it
# is superseded, and the front-page URL then 404s for a good pin.
get "https://zlib.net/fossils/zlib-$ZLIB_VERSION.tar.gz" \
"$ZLIB_TGZ" "$ZLIB_SHA256"
get "https://github.com/libffi/libffi/releases/download/v$LIBFFI_VERSION/libffi-$LIBFFI_VERSION.tar.gz" \
@@ -164,73 +112,44 @@ get "https://github.com/tukaani-project/xz/releases/download/v$XZ_VERSION/xz-$XZ
"$XZ_TGZ" "$XZ_SHA256"
get "https://sourceware.org/pub/bzip2/bzip2-$BZIP2_VERSION.tar.gz" \
"$BZIP2_TGZ" "$BZIP2_SHA256"
# EXPAT_TAG, not EXPAT_VERSION: libexpat tags releases R_2_6_4 and names the
# file expat-2.6.4, so both spellings appear in the one URL.
# EXPAT_TAG, not EXPAT_VERSION: libexpat tags R_2_6_4 but names the file
# expat-2.6.4, so both spellings appear in the one URL.
get "https://github.com/libexpat/libexpat/releases/download/$EXPAT_TAG/expat-$EXPAT_VERSION.tar.gz" \
"$EXPAT_TGZ" "$EXPAT_SHA256"
# The third-party packages that become the interpreter's site-packages, and
# libsodium. Same rule as the eight tarballs above and for the same reason:
# every package ships them, so there is no BUILD_ flag and no condition --
# they come down once and the sha256 cache means never again. 53MB, of which
# 46MB is pillow's sdist alone (it carries its own test images; there is no
# smaller sdist to have).
#
# pypi <project> <file> <sha256>
#
# files.pythonhosted.org publishes every sdist at TWO urls: the hashed
# /packages/<a>/<b>/<64 hex>/<file> one that a browser copies, and this
# /packages/source/<initial>/<project>/<file> one. Only the second can be
# composed from a pin, because the first embeds a digest of the file itself --
# so bumping a version there means pasting a URL nobody can check by reading.
# The project name is the one thing not in versions.env, because it is a
# property of the URL and not of the version: `smart-open` publishes
# smart_open-6.4.0.tar.gz, `markupsafe` publishes MarkupSafe-2.1.5.tar.gz.
# Same division of labour as EXPAT_TAG above.
# Only /packages/source/<initial>/<project>/<file> can be composed from a pin;
# the other URL embeds a digest of the file itself. The project name is a
# property of the URL, not the version: `markupsafe` -> MarkupSafe-2.1.5.
pypi() {
proj="$1"; file="$2"; sha="$3"
get "https://files.pythonhosted.org/packages/source/$(printf '%.1s' "$proj")/$proj/$file" \
"$ROOT/vendor/$file" "$sha"
}
# PYPKG_HOST_LIST too: the PEP 517 backends never reach a printer, but they
# are what RUNS on the build machine to produce the objects that do, so they
# are fetched and checked exactly like the rest.
# produce the objects that do.
for p in $PYPKG_LIST $PYPKG_HOST_LIST; do
pypi "$p" "$(pypkg_var "$p" FILE)" "$(pypkg_var "$p" SHA256)"
done
get "https://github.com/jedisct1/libsodium/releases/download/$SODIUM_VERSION-RELEASE/libsodium-$SODIUM_VERSION.tar.gz" \
"$SODIUM_TGZ" "$SODIUM_SHA256"
# --------------------------------------------------------------------- opkg
# The package manager and its two build-only dependencies. See
# docs/notes/85-packaging.md; versions.env says why 0.7.0 and not 0.6.3.
#
# zlib is NOT fetched here: it is the same pinned tarball the CPython section
# above already pulls (ZLIB_TGZ), and asking for it twice is how two pins that
# are supposed to be one drift apart.
# --- opkg
# zlib is NOT fetched here: the CPython section already pulls that same pin.
get "https://downloads.yoctoproject.org/releases/opkgs/3rdparty/opkg-$OPKG_VERSION.tar.gz" \
"$OPKG_TGZ" "$OPKG_SHA256"
get "https://github.com/libarchive/libarchive/releases/download/v$LIBARCHIVE_VERSION/libarchive-$LIBARCHIVE_VERSION.tar.gz" \
"$LIBARCHIVE_TGZ" "$LIBARCHIVE_SHA256"
# opkg-utils: opkg-build and opkg-make-index, which bin/build-packages.sh
# drives instead of assembling .ipk archives itself.
#
# A GIT CLONE AND NOT A `get`, uniquely in this file, and not by choice:
# opkg-utils publishes no release tarball anywhere. Upstream's cgit has
# snapshots disabled and the GitHub mirror has been dead since 2012 --
# versions.env has the full account. So the integrity check is git's own: a
# commit sha is a hash of the entire tree, which is the same guarantee every
# sha256 above provides, and it is verified below rather than assumed from the
# tag. A tag can be moved; the sha it has to resolve to cannot.
# A GIT CLONE, uniquely here, because opkg-utils publishes no release tarball.
# Verified against the commit sha below: a tag can be moved, the sha cannot.
if [ ! -d "$OPKG_UTILS_DIR/.git" ]; then
say "clone opkg-utils $OPKG_UTILS_VERSION"
rm -rf "$OPKG_UTILS_DIR"
git clone -q https://git.yoctoproject.org/opkg-utils "$OPKG_UTILS_DIR"
fi
( cd "$OPKG_UTILS_DIR"
# Fetch only if the pinned commit is not already here, so the common case is
# offline and instant.
# Only if the pinned commit is missing, so the common case is offline.
git cat-file -e "$OPKG_UTILS_COMMIT^{commit}" 2>/dev/null || git fetch -q origin
git checkout -q "$OPKG_UTILS_COMMIT"
have=$(git rev-parse HEAD)
@@ -238,8 +157,7 @@ fi
echo " !! opkg-utils is at $have, not the pinned $OPKG_UTILS_COMMIT" >&2
exit 1
fi
# A dirty checkout is a build whose tooling nobody can name. The sha covers
# what git tracks and says nothing about what someone edited in place.
# The sha covers what git tracks and nothing about what was edited in place.
if [ -n "$(git status --porcelain)" ]; then
echo " !! $OPKG_UTILS_DIR has local modifications -- the pinned commit" >&2
echo " no longer describes what is in it. Delete it and re-run." >&2
@@ -247,19 +165,8 @@ fi
fi )
say "cached opkg-utils $OPKG_UTILS_VERSION ($OPKG_UTILS_COMMIT)"
# The Ingenic toolchain, on the condition that decides whether patch.sh has to
# compile at all. ONE question, asked of pkg_needs, because everything built
# with this compiler is a recipe under pkgs/ now -- the interpreter, its
# eighteen site-packages, libsodium, the s6 family and the rest.
#
# A hand-written comparison here would be a second spelling of a recipe's cache
# key -- which has to include its toolchain and its dependencies -- and the
# second spelling is the one that drifts. Asking the code that WRITES the stamp
# is the only spelling that cannot.
#
# Getting this wrong is not a slow build, it is a stopped one: patch.sh would
# get 53MB of sdists and then halt for want of a compiler, halfway through,
# rather than here where the fix is one command.
# The toolchain, on the pkg_needs condition that decides whether patch.sh has
# to compile at all. Wrong here is not a slow build but a stopped one.
if [ "$ALL" = 1 ] \
|| pkg_needs; then
get "https://github.com/ballaswag/k1-discovery/releases/download/$MIPS_TOOLCHAIN_VERSION/$MIPS_TOOLCHAIN_FILE" \
bin/pack.sh
+21 -51
View File
@@ -1,20 +1,11 @@
#!/usr/bin/env bash
# 3/3 -- repack work/software back into an installable USB package.
# 3/3 -- repack work/software into an installable USB package.
# ./bin/pack.sh [--full|--plain] default is slim: software component only
#
# ./bin/pack.sh software component + the mod payload (DEFAULT, ~63MB)
# ./bin/pack.sh --full also carry kernel / control / library
# ./bin/pack.sh --plain no encryption: emits runFirmwareExe.sh + payload
# for the /mnt/runFirmwareExe.sh dev path
#
# Slim is the default because the stock installer skips any component that is
# absent: every update_<name> guards on `ls -1t <name>-*.tar.xz` and returns
# early when it fails. So shipping only the component we actually modified
# leaves the kernel, the rootfs image and the MCU/board firmware completely
# untouched -- MCU flashing is the riskiest thing in the package and there is
# no reason to run it to install a userspace mod.
#
# (start.img, end.img and play are still shipped: runFirmwareExe.sh uses those
# unconditionally.)
# Slim works because the stock installer skips any absent component -- every
# update_<name> guards on `ls -1t <name>-*.tar.xz` -- so the kernel, rootfs and
# MCU firmware are left untouched. start.img, end.img and play still ship:
# runFirmwareExe.sh uses them unconditionally.
set -euo pipefail
. "$(dirname "$0")/common.sh"
@@ -35,10 +26,8 @@ OUT_VER="${SW_VER:-$STOCK_SW_VER}"
rm -rf work/stage work/out
mkdir -p work/stage work/out
# ---------------------------------------------------------------------------
# 1. md5sum.list -- the installer hard-gates on this. Paths must be "./rel",
# and the list must not contain itself.
# ---------------------------------------------------------------------------
# --- 1. md5sum.list -- the installer hard-gates on it. Paths must be "./rel",
# and the list must not contain itself.
echo ">> regenerating md5sum.list"
( cd work/software
rm -f md5sum.list
@@ -47,34 +36,23 @@ echo ">> regenerating md5sum.list"
| xargs -0 md5sum > md5sum.list
echo " $(wc -l < md5sum.list) entries" )
# ---------------------------------------------------------------------------
# 2. software-<ver>.tar.xz
#
# !! NOT actually xz-compressed. FlashForge's own components are PLAIN
# tar archives that merely carry a .tar.xz name -- verify with
# `file work/outer/software-*.tar.xz`. The stock installer extracts them
# with a bare `tar -xvf`, so a genuinely xz-compressed file here does not
# install. Keep this a plain tar.
# ---------------------------------------------------------------------------
# --- 2. software-<ver>.tar.xz, NOT actually xz. FlashForge's own components
# are plain tars carrying a .tar.xz name and the installer runs a bare
# `tar -xvf`, so a real xz file does not install.
echo ">> building software-$OUT_VER.tar.xz (plain tar, matching stock)"
tar -cf "work/stage/software-$OUT_VER.tar.xz" -C work/software .
ls -lh "work/stage/software-$OUT_VER.tar.xz" | awk '{print " "$5}'
# ---------------------------------------------------------------------------
# 3. the rest of the outer package
# ---------------------------------------------------------------------------
# The mod payload: rides in the outer package so it lands on /usr/data,
# not on the firmware partition.
# --- 3. the rest of the outer package
# The payload rides here so it lands on /usr/data, not the firmware partition.
if [ -d "$PAYLOAD_DIR" ]; then
# This one IS really xz: we extract it ourselves with `xz -dc`, and
# FlashForge's factory installer proves xz exists on the printer.
# This one IS really xz: we extract it ourselves with `xz -dc`.
echo ">> compressing anvil.tar.xz (Mainsail / HelixScreen / Moonraker / bin)"
tar -cf - -C "$PAYLOAD_DIR" . | xz -T0 -6 > work/stage/anvil.tar.xz
ls -lh work/stage/anvil.tar.xz | awk '{print " "$5}'
fi
# FlashForge's own installer, reused verbatim -- it already does the whole
# USB auto-install dance. We only change what it installs.
# FlashForge's own installer, reused verbatim; only what it installs changes.
cp -f work/outer/runFirmwareExe.sh work/stage/
chmod +x work/stage/runFirmwareExe.sh
for f in start.img end.img play; do
@@ -92,25 +70,18 @@ fi
echo ">> outer payload:"
ls -la work/stage | sed 's/^/ /'
# ---------------------------------------------------------------------------
# 4. emit
# ---------------------------------------------------------------------------
# MOD_VER is the release date; common.sh defaults it to today (UTC).
# --- 4. emit
BASE="${MOD_NAME:-anvil}-${MOD_VER:?}"
if [ "$PLAIN" = "1" ]; then
# app_startup.sh also honours a bare /mnt/runFirmwareExe.sh -- no crypto,
# no .tgz. Copy the whole work/out/plain/ tree to the USB root.
# app_startup.sh also honours a bare /mnt/runFirmwareExe.sh: copy this
# whole tree to the USB root.
mkdir -p work/out/plain
cp -a work/stage/. work/out/plain/
echo
echo "PLAIN package: work/out/plain/ -> copy its CONTENTS to the USB root"
else
# Name the output after the model the package is actually for. The two
# models ship DIFFERENT firmwareExe binaries, so emitting both filenames
# from one build (as an earlier version did) would hand the wrong
# firmware to one of them. Each model must be built from its own stock
# package.
# The two models ship DIFFERENT firmwareExe binaries.
PKG_MACHINE=$(cat work/.pkg_machine 2>/dev/null || echo "")
if [ "$PKG_MACHINE" = unknown ]; then PKG_MACHINE=""; fi
if [ -n "$PKG_MACHINE" ] && [ "$PKG_MACHINE" != "${TARGET_MACHINE:-$PKG_MACHINE}" ]; then
@@ -122,9 +93,8 @@ else
echo ">> tarring + encrypting"
# Outer tar is NOT gzipped despite the .tgz name -- unTar pipes the
# decrypted stream straight into `tar xvf -`.
# The Pro's app_startup.sh globs /mnt/Creator5Pro-*.tgz and the non-Pro
# globs /mnt/Creator5-*.tgz, so the filename prefix must match the model.
# decrypted stream into `tar xvf -`. The prefix must match the model:
# app_startup.sh globs /mnt/Creator5Pro-*.tgz.
OUTFILE="work/out/${OUT_MACHINE}-${BASE}.tgz"
tar -cf - -C work/stage . \
| openssl des3 -salt -md md5 -k "$FF_KEY" > "$OUTFILE"
bin/patch.sh
+89 -255
View File
@@ -1,19 +1,13 @@
#!/usr/bin/env bash
# 2/3 -- apply the mods to work/software/.
# Idempotent: safe to re-run after editing config.env or assets.
# 2/3 -- apply the mods to work/software/. Idempotent.
set -euo pipefail
. "$(dirname "$0")/common.sh"
# lib.sh for pkg_recipes, pkg_ipk and pkg_die. Not pkg_out:
# nothing here reads work/pkg, and qa/static/test_ipk.py asserts it.
. "$ROOT/pkgs/lib.sh"
SOFTWARE_DIR=work/software
[ -d "$SOFTWARE_DIR" ] || { echo "run bin/unpack.sh first" >&2; exit 1; }
# The payload is installed from the feed, so the feed has to exist. Counting
# .ipk rather than testing for the directory: a cleaned tree leaves $PKG_FEED
# present and empty, which is the case that used to fail obscurely inside a
# recipe two hundred lines down.
# Counting .ipk, not testing the directory: a cleaned tree leaves it empty.
if [ -z "$(ls "$PKG_FEED"/*.ipk 2>/dev/null)" ]; then
echo "no package feed at $PKG_FEED" >&2
echo " the recipes this script runs build against it -- run 'make packages' first." >&2
@@ -23,59 +17,33 @@ fi
say() { printf '>> %s\n' "$*"; }
skip() { printf ' (skip) %s\n' "$*"; }
# The payload is built OUTSIDE the software component: that component goes to
# /usr/prog on the firmware partition, which keeps one version and would
# overflow on ~100MB of Mainsail and HelixScreen. The payload rides in the
# outer package instead and lands on /usr/data.
#
# bin/build-payload.py replaces $PAYLOAD_DIR itself; this clears anything
# beside it and a stale directory from an older layout.
# Built OUTSIDE the software component: /usr/prog is on the firmware partition
# and would overflow on ~100MB of Mainsail and HelixScreen.
rm -rf "$PAYLOAD_ROOT" "$SOFTWARE_DIR/mod"
# ------------------------------------------------- 0. the payload, installed
# Every file bound for $MODDIR comes from a package, installed by the
# PRINTER'S OWN opkg out of the feed bin/build-packages.sh indexed, inside the
# replica. `opkg list-installed` answers "what does this release install?" off
# the payload itself.
# --- 0. the payload, installed
# Installed by the printer's own opkg inside the replica, so `opkg
# list-installed` answers what a release installs off the payload itself.
say "payload: installing the feed with the printer's own opkg"
# The payload is not model-specific: anvil-klipper-config ships both chamber
# configs and anvil-link-prog.sh symlinks the one the printer names.
# TARGET_MACHINE still names the OUTPUT, because runFirmwareExe.sh refuses a
# package whose machine is not its own and app_startup.sh globs for the model
# prefix, so bin/pack.sh stamps it on the filename.
# Not model-specific; TARGET_MACHINE names only the OUTPUT file, because
# runFirmwareExe.sh refuses a package whose machine is not its own.
# WHAT THE RELEASE IS, not the closure of it. Depends brings the rest, and
# whether it does is the same question an `opkg install anvil-moonraker` on a
# printer asks -- so the metadata is exercised on every build instead of only
# when somebody tries it. Naming the closure by hand would have hidden that.
# The roots, not the closure: Depends brings the rest, so the metadata is
# exercised on every build. A root with no .ipk is skipped -- that is how the
# PKG_WHEN gates reach here -- but a missing DEPENDENCY is still opkg's error.
# The two loose python packages are Recommends, which opkg has no field for.
#
# A root with no .ipk in the feed is skipped, which is how the PKG_WHEN gates
# reach here: BUILD_HELIX=0 builds no anvil-helixscreen, and an unset
# BUSYBOX_BIN builds no anvil-busybox, so there is nothing to install and no
# flag restated in this file. A missing DEPENDENCY is still an error, and opkg
# raises it.
#
# The two loose python packages are Recommends, which opkg has no field for:
# pillow and preprocess-cancellation are Moonraker's thumbnail path, and
# pkgs/moonraker argues them out of Depends because a Moonraker without them
# serves every other endpoint.
#
# GREENLET AND CFFI WERE HERE TOO, for klippy, with a comment saying
# pkgs/klipper could not declare them "while klippy still runs under
# FlashForge's interpreter". It does not: the klipper service execs
# $FF_PYTHON, so they are ordinary Depends of anvil-klipper now and this list
# does not name them. That is the difference that matters -- listed here they
# were installed by every build and by no `opkg install anvil-klipper`, which
# is the one command that has to work on a printer.
# greenlet and cffi are deliberately NOT here: the klipper service execs
# $FF_PYTHON, so they are ordinary Depends of anvil-klipper now. Listed here
# they were installed by every build and by no `opkg install anvil-klipper`,
# which is the one command that has to work on a printer.
MOD_ROOTS="anvil-core anvil-opkg anvil-s6-rc anvil-klipper
anvil-moonraker anvil-python-pillow anvil-python-preprocess-cancellation
anvil-mainsail anvil-helixscreen anvil-busybox"
# Named, not versioned: opkg reads the index and picks. The one version that
# cannot be left to it is anvil-core's -- PKG_VERSION is MOD_VER, which
# defaults to today's date, so a feed built yesterday installs yesterday's
# anvil-core without complaint.
# Named, not versioned -- except anvil-core, whose PKG_VERSION is MOD_VER, so a
# feed built yesterday would install yesterday's anvil-core.
[ -f "$(pkg_ipk anvil-core)" ] || pkg_die \
"the feed has no $(basename "$(pkg_ipk anvil-core)") -- rerun ./bin/build-packages.sh"
@@ -87,103 +55,54 @@ for _p in $MOD_ROOTS; do
done
done
# Built on the printer: bin/build-payload.py starts the replica, lets the
# machine's own opkg install the feed onto the machine's own filesystem, and
# tars the result back. That needs a privileged container and the printer
# image, which is why `make build` has its own docker lane.
# The machine's own opkg installs the feed onto its own filesystem: needs a
# privileged container, hence `make build`'s docker lane.
# shellcheck disable=SC2086
./bin/build-payload.py $MOD_INSTALL
say "payload: $(grep -c '^Package:' "$PAYLOAD_DIR/var/lib/opkg/status") packages installed (both chamber configs; the printer picks)"
# ---------------------------------------------------------------- 1. Klipper
# NOTHING IS STAGED HERE ANY MORE. anvil-klipper installs the whole klippy
# tree at $MODDIR/klipper/klippy and the klipper s6-rc service execs it there,
# on our own $FF_PYTHON -- so the package is not merely the source of the
# printer's Klipper, it IS the printer's Klipper.
# --- 1. Klipper
# Nothing is staged here any more: anvil-klipper installs the whole klippy tree
# at $MODDIR/klipper/klippy and the klipper s6-rc service execs it there on our
# own $FF_PYTHON, so the package IS the printer's Klipper. /usr/prog/klipper
# keeps FlashForge's stock klippy, unread -- except klipper_pri.sh (their
# SCHED_FIFO helper) and klipperDaemon, our shim, which the service greps
# KLIPPER_NICENESS out of.
#
# WHAT THIS SECTION USED TO BE, because the shape is worth not rebuilding: a
# `cp -a` of the payload's klippy tree into the software component, a
# chelper.tar wrapped around a c_helper.so the package already contained, an
# `rm -rf` of the stock extras/ and kinematics/ the fork replaces, and a copy
# of the ff_*.py toolchanger extras beside them. All four existed for one
# reason -- klippy was started from /usr/prog/klipper/klippy by FlashForge's
# klipperDaemon, so the tree had to be on the firmware partition, which meant
# shipping a second copy of it in the software component and hoping the two
# agreed. They are one copy now. See etc/s6-rc/source/klipper/run.
#
# WHAT IS LEFT AT /usr/prog/klipper: FlashForge's own stock klippy, untouched
# and unread. A stock flash restores it, which also replaces firmwareExe and
# takes the mod down whatever else survived -- so it was never the fallback it
# looked like. klipper_pri.sh and klipperDaemon are still read there: the
# first is FlashForge's SCHED_FIFO helper, the second is our shim, and the
# service greps KLIPPER_NICENESS out of it.
#
# THE ASSERTION STAYS. It is the whole of what this section still does, and it
# is cheap: an anvil-klipper that did not install is a printer with no Klipper
# at all now, with no component copy to mask it.
# The assertion is all that is left, and it is cheap: an anvil-klipper that did
# not install is now a printer with no Klipper at all, with no component copy
# to mask it.
[ -d "$PAYLOAD_DIR/klipper/klippy" ] || pkg_die \
"the payload has no $MODDIR/klipper/klippy -- anvil-klipper did not install"
[ -f "$PAYLOAD_DIR/klipper/klippy/chelper/c_helper.so" ] || pkg_die \
"the payload has no $MODDIR/klipper/klippy/chelper/c_helper.so -- klippy cannot connect to an MCU without it"
say "Klipper: fork tree from pinned commit ${KLIPPER_VERSION:0:8} (payload only)"
# ----------------------------------------------------------- 2. Toolchanger
# GONE, with section 1. The ff_*.py extras ride the klippy tree anvil-klipper
# installs, and the ff-*.cfg are anvil-klipper-config's -- anvil-link-prog.sh
# symlinks them into /usr/data/config, which is where printer.cfg looks. There
# is no longer any part of the toolchanger that cannot be a package, so
# BUILD_TOOLCHANGE is answered entirely by pkgs/klipper-config's PKG_WHEN and
# this script does not restate it.
# --- 2. Toolchanger
# Gone with section 1: the ff_*.py ride the klippy tree anvil-klipper installs
# and the ff-*.cfg are anvil-klipper-config's, so BUILD_TOOLCHANGE is answered
# entirely by that recipe's PKG_WHEN.
# ------------------------------------------------------- 3. the user's seams
# What is in the payload and in NO package, because being a package member
# would be wrong. moonraker.conf includes moonraker-custom.conf by name and
# run-append.sh creates it only when missing; a package member is overwritten
# on every upgrade by definition, which would destroy a printer's own
# Moonraker settings.
# --- 3. the user's seams
# In the payload and in NO package: a package member is overwritten on every
# upgrade by definition, and this is the user's own file.
[ -f pkgs/moonraker/seed/moonraker-custom.conf ] \
&& cp -f pkgs/moonraker/seed/moonraker-custom.conf "$PAYLOAD_DIR/config/moonraker-custom.conf"
# BUILD_MOONRAKER=0 LEAVES THE STOCK SERVER ALONE, INCLUDING ITS CONFIG. The
# stock Moonraker is a 2022 build (API 1.0.5) that ships on the factory image
# only, so a reflash cannot put it back once replaced. moonraker.conf is
# shipped by pkgs/moonraker with the server it configures, not by anvil-core:
# shipped unconditionally, a BUILD_MOONRAKER=0 build still overwrote the
# printer's config with one written for a server it was not installing.
# moonraker.conf ships with pkgs/moonraker, not anvil-core: unconditional, a
# BUILD_MOONRAKER=0 build overwrote the config of a server it did not install.
# --------------------------------------------- 5b-2. the s6-rc database
# The boot order, compiled. payload/etc/s6-rc/source/ holds the definition
# directories (one per service, plus the ok-all bundle) and this turns them
# into the binary database s6-rc-init reads at boot.
#
# COMPILED HERE, ON THE BUILD HOST, NOT ON THE PRINTER. That needs a
# s6-rc-compile that runs on x86, which means a second, native build of the
# same four tarballs -- which is why this step exists at all and is not two
# lines. The alternative was shipping the target s6-rc-compile (131KB) and
# compiling at install time, and it was rejected for the reason phase 6 of
# docs/notes/80-s6-migration.md states as "do the gate first, not the
# installer": a database compiled on the printer can fail on a printer in a
# way CI never sees, and the ABI gate above cannot look at a database.
#
# THAT THIS WORKS AT ALL WAS MEASURED, not reasoned from the manual. On
# 2026-08-28, in the replica: the same source tree compiled by the native
# s6-rc-compile and by the target one under qemu produced byte-identical
# db, n and resolve.cdb, and `diff -r` found no difference in the generated
# servicedirs. The database files are host-neutral by construction (every
# integer goes through uint32_pack_big) and the servicedirs are neutral
# BECAUSE both builds are configured --prefix=$MODDIR -- the #! line
# s6-rc-compile writes into the oneshot runner comes from the execline the
# COMPILER was linked against, not from the one we ship. Configure the native
# stack with a different prefix and every oneshot on the printer dies with
# ENOENT while every longrun keeps working.
# The source tree comes out of the INSTALLED payload, not out of the recipe
# directory -- anvil-core ships it, so compiling what shipped is the only way
# the database and the package cannot disagree.
# --- 5b-2. the s6-rc database
# Compiled on the BUILD HOST, hence the second native build of the same four
# tarballs -- and byte-identical to a qemu target compile only BECAUSE both
# stacks are --prefix=$MODDIR: s6-rc-compile bakes the #! of the execline the
# COMPILER was linked against into the oneshot runner, so a different native
# prefix means ENOENT on every oneshot. The source comes out of the installed
# payload, so the database and the package cannot disagree.
S6RC_SRC="$PAYLOAD_DIR/etc/s6-rc/source"
S6RC_NATIVE="$PWD/work/.s6-native"
# The four versions and nothing else: this compiler is built by the build
# image's own gcc, so there is no toolchain file to key on.
# The four versions and nothing else: this compiler is the build image's gcc.
S6RC_NATIVE_STAMP="$SKALIBS_VERSION $EXECLINE_VERSION $S6_VERSION $S6RC_VERSION"
[ -d "$S6RC_SRC" ] || { echo " !! no s6-rc source tree at $S6RC_SRC" >&2; exit 1; }
@@ -196,9 +115,7 @@ if [ ! -x "$S6RC_NATIVE/bin/s6-rc-compile" ] \
tar -xzf "$t" -C work/.s6-native-src
done
(
# A subshell for the same reason the cross build has one: nothing
# about this compiler's environment may leak into the rest of the
# build. Note the ABSENCE of --host: this one runs here.
# A subshell so nothing leaks into the rest of the build. Note no --host.
set -e
NSRC="$PWD/work/.s6-native-src"
ND="$PWD/work/.s6-native-stage"
@@ -209,8 +126,7 @@ if [ ! -x "$S6RC_NATIVE/bin/s6-rc-compile" ] \
"s6-$S6_VERSION" "s6-rc-$S6RC_VERSION"; do
cd "$NSRC/$pkg"
if [ "$pkg" = "skalibs-$SKALIBS_VERSION" ]; then
# No --with-sysdep answers here: configure can compile and RUN
# its probes, because the target is this machine.
# configure can compile and RUN its probes: the target is here.
./configure --prefix="$MODDIR" \
--disable-shared --enable-static >/dev/null
else
@@ -225,9 +141,8 @@ if [ ! -x "$S6RC_NATIVE/bin/s6-rc-compile" ] \
done
)
mkdir -p "$S6RC_NATIVE/bin"
# s6-rc-db comes too: qa/static/test_s6rc_source.py reads the boot graph
# back out of the compiled database rather than out of the source tree,
# and this is the only copy of it that runs on the build host.
# s6-rc-db too: qa/static/test_s6rc_source.py reads the boot graph out of
# the compiled database, and this is the only copy that runs on the host.
for b in s6-rc-compile s6-rc-db; do
cp -f "work/.s6-native-stage$MODDIR/bin/$b" "$S6RC_NATIVE/bin/"
done
@@ -237,10 +152,8 @@ else
skip "s6-rc: native compiler already built for $S6RC_NATIVE_STAMP"
fi
# The database goes to etc/s6-rc/compiled/<stamp>, with `current` a symlink to
# it -- skarnet's own advice, so the boot command never changes when the
# database does. Not /etc/s6-rc/, which is s6-rc-init's default and is inside
# the read-only squashfs.
# compiled/<stamp> with `current` a symlink, so the boot command never changes.
# Not /etc/s6-rc/, s6-rc-init's default, which is inside the read-only squashfs.
S6RC_DB_NAME="db-$MOD_VER"
rm -rf "$PAYLOAD_DIR/etc/s6-rc/compiled"
mkdir -p "$PAYLOAD_DIR/etc/s6-rc/compiled"
@@ -249,10 +162,7 @@ mkdir -p "$PAYLOAD_DIR/etc/s6-rc/compiled"
echo " !! s6-rc-compile refused $S6RC_SRC" >&2; exit 1; }
ln -sfn "$S6RC_DB_NAME" "$PAYLOAD_DIR/etc/s6-rc/compiled/current"
# The shebang the compiler baked in is the one assertion that catches a native
# stack built with the wrong prefix, and it is worth making here rather than
# discovering on a printer: this file is generated, so it is the only place
# the build can see what the database will ask the printer to exec.
# The one assertion that catches a native stack built with the wrong prefix.
S6RC_RUNNER="$PAYLOAD_DIR/etc/s6-rc/compiled/$S6RC_DB_NAME/servicedirs/s6rc-oneshot-runner/run"
[ -f "$S6RC_RUNNER" ] || {
echo " !! s6-rc-compile produced no oneshot runner -- has the source tree no oneshots?" >&2
@@ -266,53 +176,21 @@ case "$S6RC_SHEBANG" in
esac
say "s6-rc: database $S6RC_DB_NAME compiled -- $(ls "$S6RC_SRC" | wc -l) definitions"
# -------------------------------------------------- 5c. CPython 3.13 (shipped)
# A second Python for the printer, installed by section 0 as anvil-python plus
# the eighteen anvil-python-* packages. How they are built is in
# pkgs/3rdparty/python*/ and in pkg_buildpython / pkg_pytarget / pkg_pywheel.
#
# FF_PYTHON POINTS HERE: anvil-env.sh names this interpreter for Moonraker,
# ff-startup.py, ffscreen.py and ff_mcu_bringup.py. klippy is NOT one of its
# callers -- FlashForge's own /usr/prog/klipper/start.sh runs it on 3.8.2
# (init.d/S70klipper) -- so klippy's numpy gap is a separate, smaller item.
#
# WHY IT IS WORTH SHIPPING: FlashForge built 3.8.2 without _sqlite3, and that
# omission is what pins MOONRAKER_VERSION to a 2023 commit -- every Moonraker
# from v0.9.0 on keeps its database in sqlite. This one has a working sqlite3,
# measured on the replica (case-python.sh).
#
# The dev half is not installed, so verify.sh's "no headers in the payload"
# check passes because they were never put there.
# --- 5c. CPython 3.13 (shipped)
# FF_PYTHON points here -- but not for klippy, which runs on the stock 3.8.2.
# That 3.8.2 has no _sqlite3, which is what pins MOONRAKER_VERSION to 2023.
# The dev half is never installed, so verify.sh's "no headers" check passes.
# THE GATES ARE THE RECIPES'. pkgs/3rdparty/python/build.sh asks for _sqlite3
# by name, and the seven python-* recipes that ship a .so each assert their
# own extension -- an aggregate count could say that A native package had
# fallen back to a pure-python wheel but never WHICH.
#
# WHAT THAT COSTS: a gate inside build.sh does not run on a cache hit, and
# none of the python recipes sets PKG_STAMP_EXTRA, so editing one does not
# rebuild it. That is a property of pkg_stamp rather than of where a gate
# lives; the answer is to make the stamp see the recipe, not to keep a second
# copy of every check here.
# Each python-* recipe asserts its own extension: an aggregate count cannot say
# WHICH one fell back to a pure-python wheel. A gate in build.sh does not run
# on a cache hit.
du -sh "$PAYLOAD_DIR/lib/python$PY_MM" | awk '{print " "$1"\tlib/python'"$PY_MM"'/"}'
du -sh "$PAYLOAD_DIR/lib/python$PY_MM/site-packages" \
| awk '{print " "$1"\tlib/python'"$PY_MM"'/site-packages/"}'
# ------------------------------------------------------------------- 6. SSH
# Nothing to install. The stock rootfs (kernel-*.tar.xz -> rootfs.squashfs)
# already ships /usr/sbin/dropbear, /usr/bin/dropbearkey AND an enabled
# /etc/init.d/S50dropbear that busybox init runs at every boot. SSH is
# therefore ALREADY LISTENING on port 22 of a stock printer.
#
# The only thing missing is a root password anyone knows: stock /etc/shadow
# carries an unpublished hash. Setting ROOT_PW_HASH below is the entire
# "enable ssh" feature -- no cross-compiled binaries, no init script.
#
# MOD_SSH USED TO GATE THIS and is gone. It never turned ssh off -- dropbear is
# the rootfs's and listens either way -- so all it did was decline to set a
# password on a machine whose whole recovery story is "ssh in". A build that
# genuinely wants the stock hash left alone is a build that wants a stock
# package.
# --- 6. SSH
# Nothing to install: the stock rootfs ships dropbear and an enabled
# S50dropbear, so ssh is already listening. Only the password is missing.
if [ -n "${ROOT_PW_HASH:-}" ]; then
say "SSH: stock dropbear is already running; setting a known root password"
else
@@ -320,88 +198,48 @@ else
say " and write it to anvil-password.txt on the USB stick."
say " Set ROOT_PW_HASH to choose your own instead."
fi
# --------------------------------------------------------- 7. root password
# --- 7. root password
if [ -n "${ROOT_PW_HASH:-}" ]; then
say "Accounts: setting root password hash"
# /etc is a bind mount of /usr/prog/etc (app_startup.sh), and this file is
# what dropbear reads at authentication time -- so this is the live shadow
# even though dropbear started earlier from the read-only squashfs.
# /etc is a bind mount of /usr/prog/etc, so this is the live shadow dropbear
# reads at auth time, though it started from the read-only squashfs.
awk -v h="$ROOT_PW_HASH" 'BEGIN{FS=OFS=":"} $1=="root"{$2=h} {print}' \
"$SOFTWARE_DIR/shadow" > "$SOFTWARE_DIR/shadow.new" && mv -f "$SOFTWARE_DIR/shadow.new" "$SOFTWARE_DIR/shadow"
else
skip "root password (set ROOT_PW_HASH)"
fi
# ------------------------------------------------ 8. start.sh (web stack on)
# FROM THE INSTALLED PAYLOAD, not from a directory beside the recipe -- the
# same move section 1 already makes for the klippy fork. anvil-core ships this
# file at $MODDIR/prog/start.sh, so the package is the one source of truth and
# the component is a copy of it rather than a second original.
#
# It still travels in the component, because that is what FlashForge's run.sh
# copies to /usr/prog/klipper/start.sh and what keeps a printer bootable when
# the payload is not there. anvil-link-prog.sh replaces that copy with a
# symlink afterwards; see its header.
# --- 8. start.sh (web stack on)
# From the installed payload, but still carried in the component: that is what
# FlashForge's run.sh copies to /usr/prog, and what keeps a printer bootable
# when the payload is not there. anvil-link-prog.sh then swaps it for a link.
say "start.sh: enabling nginx + moonraker"
cp -f "$PAYLOAD_DIR/prog/start.sh" "$SOFTWARE_DIR/start.sh"
chmod +x "$SOFTWARE_DIR/start.sh"
# ------------------------------------- 9. firmwareExe -> our wrapper script
# --- 9. firmwareExe -> our wrapper script
# The stock chain is rcS -> S99factory_test_shell -> app_startup.sh ->
# firmwareExe, and firmwareExe is also what starts Klipper. Replacing this
# one file is therefore enough to own the whole userspace boot, which means
# app_startup.sh, rcS and the init chain are left COMPLETELY STOCK.
#
# The genuine binary is replaced, not kept aside: HelixScreen is the only UI,
# and the installer wipes the software dir before run.sh anyway, so nothing
# here could ever be a reliable backup. Flashing the stock FlashForge package
# -- which still ships the binary -- is the uninstall.
#
# Also from the installed payload, for the reason section 8 gives. The copy
# that ends up on /usr/prog is what app_startup.sh launches until
# anvil-link-prog.sh turns it into a symlink -- after which an `opkg upgrade
# anvil-core` is enough to change what the printer runs, with no .tgz.
# firmwareExe, which also starts Klipper: this one file owns the whole
# userspace boot and leaves the init chain stock. The binary is not kept
# aside -- flashing the stock package is the uninstall. From the payload, as
# in section 8.
say "firmwareExe: installing wrapper (replaces the stock binary)"
cp -f "$PAYLOAD_DIR/prog/firmwareExe" "$SOFTWARE_DIR/firmwareExe"
chmod +x "$SOFTWARE_DIR/firmwareExe"
# ---------------------------------------------------------- 10. anvil.conf
# THE MOD'S OWN FILES ARE anvil-core, INSTALLED BY SECTION 0: the shared
# environment and service libraries every init script sources, the init
# scripts themselves, the helper programs, the nginx config and the s6
# scandir.
#
# ANVIL.CONF IS NOT IN THAT PACKAGE, and this is the whole of what is left
# here. It is templated from config.env and then preserved across updates by
# run-append.sh, which makes it user state rather than a package member: a
# package would overwrite a printer's settings on the first upgrade. See the
# header of pkgs/anvil-core/build.sh.
#
# FIVE OF THE SEVEN SUBSTITUTIONS ARE GONE with the switches they wrote:
# MOD_WEB, MOD_CAM, MOD_UI, MOD_SSH and MOD_WIFI all defaulted to 1 and are now
# simply always true. What is left is the two that have a RANGE -- a nice level
# is a number, not a state -- which is the line between a tunable and a switch
# and the reason these two stayed.
# --- 10. anvil.conf
# anvil.conf is not in anvil-core and is the whole of what is left here:
# preserved across updates by run-append.sh, which makes it user state rather
# than a package member. Only the two values with a RANGE are substituted.
sed -e "s/^NICE_MOONRAKER=.*/NICE_MOONRAKER=${NICE_MOONRAKER:-5}/" \
-e "s/^NICE_CAM=.*/NICE_CAM=${NICE_CAM:-10}/" \
pkgs/anvil-core/seed/anvil.conf.in > "$PAYLOAD_DIR/anvil.conf"
# ------------------------------------------------ 10b. the install manifest
# Every path this payload installs, shipped inside it so the NEXT update
# deletes exactly what this one left behind -- and only that. run-append.sh
# used to `rm -rf` seven whole directories, which destroyed anything a user
# had put in them.
#
# The property the rm -rf had and this must keep: the installed set ends up
# exactly the shipped set, or a RENAMED init script leaves a stale twin and
# firmwareExe runs both. A file the last payload shipped and this one does not
# is still named in the last payload's list, so it still goes.
#
# Read off the staged tree at the last moment -- everything above this line
# has finished staging. Directories are listed too, so emptied ones can be
# rmdir'd. It names ITSELF, because the next update must be free to replace
# it. Written to a temp file and moved in, or `find` would list the
# half-written manifest as one more payload path.
# --- 10b. the install manifest
# Every path this payload installs, so the next update deletes exactly what this
# one left behind -- a renamed init script leaves no stale twin. It names
# ITSELF, and is moved in from a temp file so `find` cannot list a
# half-written manifest.
MOD_MANIFEST=.install-manifest
{ ( cd "$PAYLOAD_DIR" && find . -mindepth 1 | sed 's|^\./||' )
echo "$MOD_MANIFEST"
@@ -409,13 +247,10 @@ MOD_MANIFEST=.install-manifest
mv -f work/.install-manifest "$PAYLOAD_DIR/$MOD_MANIFEST"
say "install manifest: $(wc -l < "$PAYLOAD_DIR/$MOD_MANIFEST") paths -> $MODDIR/$MOD_MANIFEST"
# --------------------------------------------------- 11. run.sh install step
# --- 11. run.sh install step
say "run.sh: injecting mod install blocks (pre + post)"
POST=work/.run-post.sh
# 1 when nothing was baked in: a package is one file that many people flash, so
# a baked-in default would be the same password on every printer. The installer
# picks a random per-machine one instead and writes it onto the USB stick it
# was flashed from.
# A baked-in default would be the same password on every printer.
if [ -z "${ROOT_PW_HASH:-}" ]; then
PW_AUTO=1
else
@@ -436,8 +271,7 @@ for b, e in ((B1, E1), (B2, E2)):
pre = B1 + "\n" + open(pre_f, encoding='utf-8').read() + E1 + "\n"
post = B2 + "\n" + open(post_f, encoding='utf-8').read() + E2 + "\n\n"
# The pre-block must land AFTER WORK_DIR is defined (it uses $WORK_DIR's
# siblings) but BEFORE the first cp into /usr/prog.
# After WORK_DIR is defined, before the first cp into /usr/prog.
m = re.search(r"^WORK_DIR=.*$", src, flags=re.M)
if not m:
raise SystemExit("run.sh has no WORK_DIR assignment -- cannot place the pre-block")
bin/unpack.sh
+8 -27
View File
@@ -1,12 +1,7 @@
#!/usr/bin/env bash
# 1/3 -- decrypt the stock package and open the software component.
#
# ./bin/unpack.sh uses $STOCK_TGZ from config.env
# ./bin/unpack.sh <file> unpack some other package
#
# Result:
# work/outer/ the 8 top-level files (runFirmwareExe.sh, *.tar.xz, imgs)
# work/software/ the extracted software-<ver> tree -- EDIT THIS
# ./bin/unpack.sh [<file>] default: $STOCK_TGZ from config.env
# -> work/outer/ (the 8 top-level files), work/software/ (the tree to edit)
set -euo pipefail
. "$(dirname "$0")/common.sh"
@@ -32,10 +27,7 @@ STOCK_SW_VER=$(basename "$SW_TARBALL" | sed 's/^software-//; s/\.tar\.xz$//')
echo ">> extracting software component $STOCK_SW_VER"
tar -xf "$SW_TARBALL" -C work/software
# Every package carries a model gate. runFirmwareExe.sh compares the MACHINE
# and PID it was built for against the ones app_startup.sh passes in (which
# come from the firmware already on the printer) and REFUSES to install on a
# mismatch. Record it so verify.sh can check it against your printer.
# runFirmwareExe.sh refuses a MACHINE/PID mismatch; recorded for verify.sh.
PKG_MACHINE=$(sed -n 's/^MACHINE=//p' work/outer/runFirmwareExe.sh | head -n1)
PKG_PID=$(sed -n 's/^PID=//p' work/outer/runFirmwareExe.sh | head -n1)
echo "${PKG_MACHINE:-unknown}" > work/.pkg_machine
@@ -45,25 +37,14 @@ echo ">> package installs on: ${PKG_MACHINE:-unknown} (PID ${PKG_PID:-unknown})"
echo "$STOCK_SW_VER" > work/.stock_sw_ver
echo "$SRC" > work/.source_pkg
# ---- has FlashForge changed printer.base.cfg under us? ---------------------
# Our payload/klipper/config/printer.base.cfg is FlashForge's file with the
# chamber block replaced by an include, so a change on their side means a pin
# map, stepper current or endstop we are shipping stale.
#
# This lives HERE, and not in a test, because here is the only moment a
# pristine stock tree exists: bin/patch.sh copies our file straight over
# work/software/klipper/config/printer.base.cfg a few steps later. test-base-cfg.py
# read it afterwards and so spent its life diffing our file against itself --
# green on a cold tree, red on a second run, and blind to the drift either way.
#
# A warning, not a gate: their file changing is news, not a broken build, and
# it can only happen when you point the build at a new stock package.
# Ours is FlashForge's file with the chamber block replaced by an include, so
# drift means a stale pin map or stepper current. Checked here because this is
# the only moment a pristine stock tree exists. A warning, not a gate.
STOCK_BASE=work/software/klipper/config/printer.base.cfg
OURS=pkgs/klipper-config/prog/config/printer.base.cfg
if [ -f "$STOCK_BASE" ] && [ -f "$OURS" ]; then
# Compare only the section/option lines: comments and blank lines differ by
# design, our chamber include replaces their heater block, and the ff-*.cfg
# includes at the end of the file are ours (see printer.base.cfg's tail).
# Section/option lines only: comments differ by design, and the chamber
# and ff-*.cfg includes are ours.
strip() { grep -vE '^\s*(#|$)' "$1" | grep -vE '^\[(heater_generic|verify_heater) chamber_heater\]|^\[include printer\.chamber\.cfg\]|^\[include ff-[a-z-]+\.cfg\]'; }
if strip "$STOCK_BASE" | diff -q - <(strip "$OURS") >/dev/null 2>&1; then
echo ">> printer.base.cfg matches the stock file"
bin/verify.sh
+67 -151
View File
@@ -1,12 +1,10 @@
#!/usr/bin/env bash
# Simulate every check the printer performs, against a built package.
# ./bin/verify.sh verify work/out/Creator5Pro-*.tgz
# ./bin/verify.sh path/to/pkg.tgz
# ./bin/verify.sh [path/to/pkg.tgz] default: work/out/<model>-*.tgz
set -uo pipefail
. "$(dirname "$0")/common.sh"
# Default to the package for the model currently selected, not a hardcoded
# one -- otherwise a two-model release verifies the same file twice.
# The selected model's package, or a two-model release verifies one file twice.
PKG="${1:-$(ls -1 "work/out/${TARGET_MACHINE:-Creator5Pro}"-*.tgz 2>/dev/null | head -n1)}"
[ -f "${PKG:-}" ] || { echo "no package; run ./bin/pack.sh" >&2; exit 1; }
echo "verifying $PKG"; echo
@@ -18,7 +16,7 @@ warn() { printf ' \033[33mWARN\033[0m %s\n' "$*"; }
WORK=$(mktemp -d); trap 'rm -rf "$WORK"' EXIT
# 1 ---------------------------------------------------------------- decrypt
# --- 1. decrypt
if openssl des3 -d -k "$FF_KEY" -salt -md md5 -in "$PKG" 2>/dev/null \
| tar -xf - -C "$WORK" 2>/dev/null; then
ok "decrypts with the firmware key and is a valid tar"
@@ -26,14 +24,14 @@ else
bad "decrypt/untar failed -- the printer's unTar will reject this"; exit 1
fi
# 2 ------------------------------------------------------- installer present
# --- 2. installer present
if [ -f "$WORK/runFirmwareExe.sh" ]; then
ok "runFirmwareExe.sh present"
else
bad "no runFirmwareExe.sh -- app_startup.sh falls back to plain tar and gives up"
fi
# 3 ----------------------------------------- components are PLAIN tar, not xz
# --- 3. components are PLAIN tar, not xz
for f in "$WORK"/software-*.tar.xz "$WORK"/kernel-*.tar.xz "$WORK"/control-*.tar.xz "$WORK"/library-*.tar.xz; do
[ -f "$f" ] || continue
b=$(basename "$f")
@@ -44,13 +42,12 @@ for f in "$WORK"/software-*.tar.xz "$WORK"/kernel-*.tar.xz "$WORK"/control-*.tar
fi
done
# 4 ------------------------------------------------------------- md5sum gate
# --- 4. md5sum gate
SOFTWARE_TAR=$(ls -1 "$WORK"/software-*.tar.xz 2>/dev/null | head -n1)
if [ -n "$SOFTWARE_TAR" ]; then
mkdir -p "$WORK/sw" && tar -xf "$SOFTWARE_TAR" -C "$WORK/sw"
if [ -f "$WORK/sw/md5sum.list" ]; then
# NB: the printer's busybox md5sum takes -s for quiet; GNU coreutils
# spells it --status. Use whichever this host has.
# busybox md5sum spells quiet -s; GNU coreutils spells it --status.
if ( cd "$WORK/sw" && md5sum --status -c md5sum.list 2>/dev/null ) \
|| ( cd "$WORK/sw" && md5sum -s -c md5sum.list 2>/dev/null ); then
ok "md5sum.list verifies ($(wc -l < "$WORK/sw/md5sum.list") entries)"
@@ -64,14 +61,14 @@ if [ -n "$SOFTWARE_TAR" ]; then
fi
fi
# 5 -------------------------------------------------- shell syntax of scripts
# --- 5. shell syntax of scripts
for s in "$WORK/runFirmwareExe.sh" "$WORK/sw/run.sh" "$WORK/sw/start.sh" "$WORK/sw/app_startup.sh"; do
[ -f "$s" ] || continue
if sh -n "$s" 2>/dev/null; then ok "syntax OK: $(basename "$s")"
else bad "SYNTAX ERROR in $(basename "$s")"; sh -n "$s" 2>&1 | head -3 | sed 's/^/ /'; fi
done
# 6 ---------------------------------------------- boot chain actually hooks up
# --- 6. boot chain actually hooks up
if [ -f "$WORK/sw/app_startup.sh" ]; then
grep -q 'gt9xx_touch.ko' "$WORK/sw/app_startup.sh" \
&& ok "touchscreen driver still loaded at boot" \
@@ -81,23 +78,14 @@ if [ -f "$WORK/sw/app_startup.sh" ]; then
|| bad "nothing launches a UI -- printer boots to a blank screen"
fi
# 7 ------------------------------------------------- MIPS ABI of any binaries
# THE KLIPPY TREE IS IN THE PAYLOAD, NOT IN THE SOFTWARE COMPONENT. It used to
# be checked at $WORK/sw/klipper -- a copy bin/patch.sh staged for FlashForge's
# run.sh to place on the firmware partition, back when klipperDaemon started
# klippy from there. The klipper s6-rc service execs $MODDIR/klipper/klippy on
# $FF_PYTHON now, so the component copy and the chelper.tar that carried its
# .so are both gone and this is where the one remaining tree lives.
#
# A package that ships no fork is a broken package, not an optional check: the
# v20260824 release passed here precisely because everything below was
# conditional on files that the broken build simply did not contain. It is
# unconditional now for a second reason as well -- BUILD_KLIPPER is gone, so
# there is no configuration in which a package legitimately has no klippy.
# ./klipper alone, not the whole payload: the rest is Mainsail, HelixScreen
# and a 30MB interpreter, and section 8 answers everything it needs from a
# listing. The paths carry a leading ./ because bin/pack.sh tars `.` from
# inside $PAYLOAD_DIR.
# --- 7. MIPS ABI of any binaries
# The klippy tree is in the PAYLOAD now, not the software component: the
# klipper service execs $MODDIR/klipper/klippy, so the component copy and the
# chelper.tar that carried its .so are both gone. Unconditional, because
# v20260824 passed here by having every check below conditional on files the
# broken build did not contain. ./klipper alone, not the whole payload --
# section 8 answers the rest from a listing. Paths carry a leading ./ because
# bin/pack.sh tars `.` from inside $PAYLOAD_DIR.
PAYLOAD_TAR=$(ls -1 "$WORK"/anvil.tar.xz 2>/dev/null | head -n1)
if [ -n "$PAYLOAD_TAR" ]; then
mkdir -p "$WORK/pl"
@@ -122,9 +110,8 @@ if [ -f "$CHELPER" ]; then
else
bad "c_helper.so is NOT nan2008 -- kernel returns ENOEXEC, klippy dies"
fi
# The failure mode that reached a printer: a .so from older sources
# than the klippy tree beside it. cffi resolves lazily, so only a
# symbol-table check catches it before the machine does.
# The failure that reached a printer: a .so older than the klippy tree
# beside it. cffi resolves lazily; only a symbol check catches it.
if python3 "$ROOT/test/test-chelper.py" "$WORK/pl/klipper" >/dev/null 2>&1; then
ok "c_helper.so exports everything the shipped klippy declares"
else
@@ -132,46 +119,30 @@ if [ -f "$CHELPER" ]; then
fi
fi
fi
# THE COMPONENT MUST NOT CARRY ONE. Two klippy trees under one version number
# is the arrangement this release removed, and a stray one would be the copy a
# stock flash leaves running.
# The component must not carry one: two klippy trees under one version number
# is what this release removed, and a stray one is the copy a stock flash
# leaves running.
[ -e "$WORK/sw/klipper/klippy" ] \
&& bad "the software component still carries a klippy tree -- bin/patch.sh section 1 is staging one again" \
|| ok "no klippy tree in the software component (the payload is the only one)"
# 8 ------------------------------------------------------------ mod payload
# --- 8. mod payload
if [ -f "$WORK/anvil.tar.xz" ]; then
if xz -t "$WORK/anvil.tar.xz" 2>/dev/null; then ok "anvil.tar.xz is valid xz"
elif tar -tf "$WORK/anvil.tar.xz" >/dev/null 2>&1; then ok "anvil.tar.xz is a plain tar"
else bad "anvil.tar.xz is neither valid xz nor tar"; fi
LIST=$(xz -dc "$WORK/anvil.tar.xz" 2>/dev/null | tar -tf - 2>/dev/null || tar -tf "$WORK/anvil.tar.xz" 2>/dev/null)
# Herestrings, NOT `echo "$LIST" | grep -q`: under pipefail, grep -q
# exiting at an early match SIGPIPEs the echo and the pipeline returns
# 141 -- a race that stayed invisible until the fork klippy tree tripled
# the listing, then intermittently reported Mainsail/Moonraker missing
# from packages that carried both.
# The compiled s6-rc database IS the service set: a payload without it
# boots to a scanner supervising nothing.
# Herestrings, NOT `echo | grep -q`: under pipefail, grep -q exiting at an
# early match SIGPIPEs the echo and the pipeline returns 141.
# The compiled s6-rc database IS the service set.
grep -q 'etc/s6-rc/compiled/' <<<"$LIST" && ok "mod payload has the compiled s6-rc database" \
|| bad "mod payload has no s6-rc database -- no service would ever start"
grep -q 'servicedirs/klipper/run' <<<"$LIST" && ok "mod payload owns Klipper startup" \
|| bad "mod payload has no klipper service -- Klipper would never start"
# THE KLIPPER CONFIG. printer.base.cfg has a bare
# [include printer.chamber.cfg] and Klipper treats a missing include as
# fatal, so a payload without a chamber config is a printer that comes up
# with no motion and no heaters.
#
# Both the configs and the script that resolves which one are checked:
# either alone installs a printer that cannot start klippy, and neither
# says so until it does.
#
# The ff-*.cfg are the toolchanger itself: without them klippy starts fine
# and the machine is a single-tool printer that reports no error anywhere.
#
# Only when the toolchanger is in this build. BUILD_TOOLCHANGE=0 ships
# neither these nor our printer.base.cfg, which is a coherent package and
# not a broken one.
# printer.base.cfg has a bare [include printer.chamber.cfg] and Klipper
# treats a missing include as fatal. Without the ff-*.cfg, klippy starts
# fine and the machine is silently single-tool.
if [ "${BUILD_TOOLCHANGE:-1}" = "1" ]; then
for _m in Creator5 Creator5Pro; do
grep -q "config/chamber/$_m.cfg" <<<"$LIST" \
@@ -188,94 +159,55 @@ if [ -f "$WORK/anvil.tar.xz" ]; then
&& ok "mod payload carries the toolchanger config" \
|| bad "mod payload has no config/ff-*.cfg -- klippy starts as a single-tool printer, silently"
fi
# The manifest is what the NEXT update deletes before it extracts. A
# payload shipping without one still installs fine -- run-append.sh falls
# back to the old seven-directory rm -rf -- but every printer that takes
# it keeps falling back on every update after that, and the fallback is
# the thing that eats whatever else lives in $MODDIR/bin. So this is a
# build bug, not a warning.
# Without it run-append.sh falls back to the old seven-directory rm -rf,
# forever, and that fallback eats whatever else lives in $MODDIR/bin.
grep -q '\.install-manifest' <<<"$LIST" && ok "install manifest present (next update deletes by list, not by rm -rf)" \
|| bad "mod payload has no .install-manifest -- updates fall back to wiping whole directories"
# s6. Two checks and not one, because the second is the one that catches
# the failure nobody sees coming: s6-ftrigrd is not on anyone's PATH and
# nothing calls it by name -- s6-svlisten spawns it out of the libexecdir
# compiled into the binaries. A payload with bin/ and no libexec/ supervises
# perfectly well and then every WAITING verb (s6-svc -w, s6-svwait) fails,
# which is exactly the half of s6 we are adopting it for. Build bugs both:
# a package that ships neither cannot be fixed on the printer.
# s6-ftrigrd is on no PATH -- s6-svlisten spawns it from the compiled-in
# libexecdir -- so bin/ without libexec/ supervises fine until every
# waiting verb (s6-svc -w, s6-svwait) fails.
grep -q 'bin/s6-svscan' <<<"$LIST" && ok "s6 supervision binaries present" \
|| bad "no s6 in the payload -- bin/patch.sh did not stage the cross-build"
grep -q 'libexec/s6-ftrigrd' <<<"$LIST" && ok "s6-ftrigrd present in libexec (the waiting verbs can spawn it)" \
|| bad "no libexec/s6-ftrigrd -- s6-svwait and s6-svc -w will fail on the printer"
# CPython 3.13. anvil-env.sh points FF_PYTHON here -- Moonraker, klippy,
# ff-startup.py, ffscreen.py and ff_mcu_bringup.py all run on it -- so a
# package missing it breaks the boot, not just a future release. klippy IS
# among FF_PYTHON's callers: the klipper s6-rc service execs $FF_PYTHON
# against $MODDIR/klipper/klippy, and anvil-klipper declares it.
#
# Two checks, not one, and the second is the one worth having. The
# interpreter can be present and perfectly runnable while _sqlite3 is
# absent, because a dropped -lm makes configure's link probe fail and
# CPython records the module as "missing" rather than stopping -- see the
# LIBS comment in pkgs/3rdparty/python/build.sh. sqlite3 is the entire reason this interpreter
# is built (it is what eventually unpins MOONRAKER_VERSION), so a tree
# without it is a 30MB payload that bought nothing. Build bugs both: a
# package that ships neither cannot be fixed on the printer.
# CPython 3.13, which FF_PYTHON points at -- klippy included now, since the
# klipper service execs $FF_PYTHON against $MODDIR/klipper/klippy.
# The second check is the one worth having: the interpreter runs perfectly
# while _sqlite3 is absent, because a dropped -lm makes configure's link
# probe fail and CPython records the module missing rather than stopping.
grep -q 'bin/python3\.13$' <<<"$LIST" \
&& ok "CPython 3.13 present (FF_PYTHON -- Moonraker and the ff-startup scripts run on it)" \
|| bad "no bin/python3.13 in the payload -- bin/patch.sh did not stage the cross-build"
grep -q 'lib-dynload/_sqlite3' <<<"$LIST" \
&& ok "python3.13 carries _sqlite3 (the module FlashForge's 3.8.2 has not got)" \
|| bad "python3.13 ships WITHOUT _sqlite3 -- the one module it exists for; check LIBS/LIBSQLITE3_LIBS in pkgs/3rdparty/python/build.sh"
# And the other half of the same decision: the interpreter goes into the
# prefix root's bin/ like everything else, but WITHOUT the `python3`
# symlink CPython installs beside it. $MODDIR/bin is prepended to PATH by
# anvil-env.sh (s6 needs that), so shipping one would silently put our
# interpreter ahead of FlashForge's for every process that says `python3`
# -- an accidental switch, and the mod's own callers do not need it: every
# one of them names $FF_PYTHON. Reads the way round it does for the same
# reason the dropbear check below does: its presence is the bug.
# No `python3` symlink: anvil-env.sh prepends $MODDIR/bin to PATH, so one
# would silently put our interpreter ahead of FlashForge's for every process
# that says `python3`. Its presence is the bug.
grep -qE '(^|/)bin/python3$' <<<"$LIST" \
&& bad "payload ships bin/python3 -- it would shadow FlashForge's interpreter on PATH" \
|| ok "no bin/python3 symlink (nothing shadows FlashForge's python3 on PATH)"
# The half of the 3.13 story that is not the interpreter: what runs ON it.
# Same shape of check and the same reason -- Moonraker is live on this
# interpreter now, so a missing extension here is a Moonraker that will
# not start on the very next boot, not a future release's problem.
#
# Two of the twelve extension modules by name rather than a count, and
# these two: lmdb is Moonraker's DATABASE at the pinned commit (no lmdb,
# no Moonraker at all, not a degraded one) and _cffi_backend is what
# klippy dlopens c_helper.so through, which is the entire reason this
# interpreter had to be glibc rather than musl. They are also the two that
# a "cross build" quietly resolving to a manylinux wheel gets WRONG rather
# than missing -- and the .so suffix is what says which: a mipsel build
# spells itself cpython-313-mipsel-linux-gnu.so, an x86-64 wheel spells
# itself cpython-313-x86_64-linux-gnu.so, so the name is the architecture.
# lmdb is Moonraker's database at the pinned commit, _cffi_backend is what
# klippy dlopens c_helper.so through -- the reason this interpreter had to
# be glibc, not musl. Both are what a cross build resolving to a manylinux
# wheel gets WRONG rather than missing, and the .so suffix is the
# architecture: cpython-313-mipsel-linux-gnu.so, not x86_64.
grep -q 'site-packages/lmdb/cpython.cpython-313-mipsel-linux-gnu\.so' <<<"$LIST" \
&& ok "lmdb's mipsel CPython extension present (Moonraker's database at this pin)" \
|| bad "no mipsel lmdb extension in site-packages -- Moonraker cannot open its database; check work/.pkg-python-lmdb/wheel-lmdb.log"
grep -q 'site-packages/_cffi_backend.cpython-313-mipsel-linux-gnu\.so' <<<"$LIST" \
&& ok "cffi's mipsel extension present (klippy's route to c_helper.so)" \
|| bad "no mipsel _cffi_backend in site-packages -- klippy could not load chelper on 3.13"
# THE DEV HALF MUST NOT BE HERE. anvil-python and anvil-python-dev are one
# build split in two, and only the first is something a printer installs:
# the headers, lib/pkgconfig and config-3.13-* exist so that the eighteen
# pkgs/3rdparty/python-* recipes can compile extension modules on a BUILD machine.
# bin/patch.sh prunes them from the payload by reading PKG_DEV_FILES out of
# pkgs/3rdparty/python/pkg.conf, and this is the check that the pruning happened --
# it is three megabytes of files nothing on a printer can open, and it
# would ship silently, because a payload with extra files in it works.
# The dev half must not ship: headers, pkgconfig and config-3.13-* exist so
# the python-* recipes can compile on a BUILD machine. patch.sh prunes them
# via PKG_DEV_FILES, and extra files in a payload ship silently.
if grep -qE '(^|/)(include/python3\.13/|lib/pkgconfig/|config-3\.13-)' <<<"$LIST"; then
bad "payload carries CPython's dev files (headers/pkgconfig/config-3.13-*) -- anvil-python-dev's half was not pruned"
else
ok "no CPython dev files in the payload (headers and build config stay in anvil-python-dev)"
fi
# libsodium is the one library of ours that ships as a .so, because libnacl
# dlopens it -- and the bare `libsodium.so` name is not decoration, it is
# the exact name libnacl's path fallback constructs. A payload with the
# versioned file and no symlink would pass a naive "is libsodium there"
# check and fail at Moonraker's authorization component.
# The bare `libsodium.so` name is what libnacl's path fallback constructs; a
# payload with only the versioned file fails at Moonraker's auth component.
grep -qE 'lib/libsodium\.so\.[0-9]' <<<"$LIST" \
&& ok "libsodium present in lib/ (libnacl's ed25519, no /usr/prog needed)" \
|| bad "no lib/libsodium.so.* in the payload -- bin/patch.sh section 5d did not stage the cross-build"
@@ -284,29 +216,22 @@ if [ -f "$WORK/anvil.tar.xz" ]; then
|| bad "lib/libsodium.so link missing -- libnacl looks for that exact name and would fall through to /usr/prog"
grep -q 'mainsail/index.html' <<<"$LIST" && ok "Mainsail present" || warn "no Mainsail in payload"
grep -q 'helixscreen/bin/helix-screen' <<<"$LIST" && ok "HelixScreen present" || warn "no HelixScreen in payload"
# We deliberately ship no dropbear: the stock rootfs already has one, with
# an enabled S50dropbear, so ssh is up before the mod does anything. A
# dropbear in the payload would mean something unexpected, not something
# missing -- hence the check reads the way round it does.
# We ship no dropbear -- the stock rootfs has one. Its PRESENCE is the bug,
# hence the check reads the way round it does.
grep -q 'bin/dropbear' <<<"$LIST" \
&& warn "dropbear in the payload -- we ship none; stock S50dropbear provides ssh" \
|| ok "no dropbear in the payload (stock S50dropbear provides ssh)"
# This reads the PACKAGE, not a printer: it asks whether the tarball we
# just built carries moonraker/moonraker.py. That is now the whole
# installation -- the payload extracts to /usr/data/anvil and the init
# script starts $MODDIR/moonraker/moonraker.py from where it landed, so a
# payload carrying the directory but not that file ships a Moonraker that
# cannot start, which looks identical to a dead printer from the outside.
# Reads the PACKAGE: the init script starts $MODDIR/moonraker/moonraker.py
# from where the payload landed, so the directory without that file ships
# a Moonraker that cannot start.
grep -q 'moonraker/moonraker.py' <<<"$LIST" && ok "Moonraker present" \
|| warn "no Moonraker in payload -- the stock 2022 build stays, and Mainsail will hide the webcam"
else
warn "no anvil.tar.xz -- scripts only"
fi
# 8b ------------------------------------------------------- MODEL GATE
# The most easily missed failure: a package built from the wrong model's
# firmware refuses to install, and a package built from the RIGHT model but
# flashed to the wrong one would install the wrong binaries.
# --- 8b. MODEL GATE
# A package built from the wrong model's firmware refuses to install.
PKG_MACHINE=$(sed -n 's/^MACHINE=//p' "$WORK/runFirmwareExe.sh" 2>/dev/null | head -n1)
PKG_PID=$(sed -n 's/^PID=//p' "$WORK/runFirmwareExe.sh" 2>/dev/null | head -n1)
if [ -n "$PKG_MACHINE" ]; then
@@ -326,11 +251,9 @@ else
warn "no MACHINE= gate found in runFirmwareExe.sh"
fi
# 9 --------------------------------------------------------------- USB names
# The filename prefix must match the model, because app_startup.sh globs for
# it: the Pro looks for /mnt/Creator5Pro-*.tgz, the non-Pro for
# /mnt/Creator5-*.tgz. It must ALSO match the gate inside, or the printer
# picks the file up and then refuses it.
# --- 9. USB names
# The prefix must match the model (app_startup.sh globs /mnt/Creator5Pro-*.tgz)
# AND the gate inside, or the printer picks the file up and then refuses it.
PKG_NAME=$(basename "$PKG")
case "$PKG_NAME" in
Creator5Pro-*.tgz) FN_MACHINE=Creator5Pro ;;
@@ -346,13 +269,9 @@ if [ -n "$FN_MACHINE" ]; then
fi
fi
# 10 --------------------------------------------------- the ship boundary
# Only payload/ and assets/ may end up on a printer; bin/, test/ and docker/
# are host-side. Nothing enforces that except this check.
#
# It compares by CONTENT, not by name: a name blacklist would trip over any
# file Mainsail or HelixScreen happens to call common.sh, while a byte-for-byte
# match against a host-lane file is exactly the thing we are trying to catch.
# --- 10. the ship boundary
# Only payload/ and assets/ may reach a printer. Compared by CONTENT, not name:
# a blacklist would trip over any file Mainsail happens to call common.sh.
mkdir -p "$WORK/ship"
tar -xf "$WORK"/software-*.tar.xz -C "$WORK/ship" 2>/dev/null || true
if [ -f "$WORK/anvil.tar.xz" ]; then
@@ -365,11 +284,8 @@ find bin test docker -type f 2>/dev/null | xargs -r md5sum 2>/dev/null \
find "$WORK/ship" -type f 2>/dev/null | xargs -r md5sum 2>/dev/null > "$WORK/ship.md5"
LEAK=$(awk 'NR==FNR{h[$1];next} $1 in h {print}' "$WORK/host.md5" "$WORK/ship.md5" \
| sed "s|$WORK/ship/||")
# Those finds are RELATIVE, so from any cwd but the repo root host.md5 comes
# out empty, the join matches nothing and this reports ok having compared
# nothing -- retiring the only check that keeps config.env and the host tooling
# off a USB stick. common.sh cds to the root, so this is a guard against a
# future caller that does not.
# Those finds are RELATIVE: from any other cwd host.md5 comes out empty and
# this reports ok having compared nothing. common.sh cds to the root.
if [ ! -s "$WORK/host.md5" ]; then
bad "ship-boundary check compared nothing (no files found under bin/, test/, docker/ -- wrong cwd?)"
elif [ -z "$LEAK" ]; then