mirror of
https://github.com/Klipper4FlashForge/firmware.git
synced 2026-10-22 05:12:16 -05:00
Oleksandr ShyshatskyiandClaude Opus 5
f8b925e48f
docs/notes/85-packaging.md chose opkg in 2026-08 on one argument: every package
is cross-built here anyway, so the winner is whichever format is cheapest to
PRODUCE -- and .ipk had an upstream tool where .apk meant abuild, a keypair and
an Alpine host. Two lines of that comparison were wrong.
apk mkpkg is an applet of apk itself: -I key:value metadata, -F directory, no
keypair, no abuild, no Alpine host. The "making a package" row described
apk-tools 2. And the on-device client was counted a wash when it is better than
one -- openssl and zlib were already cross-built for CPython, and libarchive
existed in this tree only for opkg, so adopting apk REMOVES a cross-build.
What made it possible at all is that apk resolves its database relative to
--root and uses the same root as the file destination, while / here is a
read-only squashfs -- measured: neither /lib/apk nor /etc/apk can be created.
pkgs/3rdparty/apk-tools/prefix.patch splits the two. Upstream already carries
root_fd and dest_fd separately and merely aliases them in context.c, so the
patch defaults the database root to $MODDIR and leaves files extracting at /.
That is opkg's compile-time VARDIR plus `dest root /`, in apk's spelling.
THE FEED IS SIGNED NOW, which it never was: opkg signed the index and
bin/build-packages.sh signed nothing, so nothing was verified on a printer.
apk verifies with raw PEM public keys, not X.509 -- measured on the machine,
where every certificate directory is empty. ./bin/apk-keygen.sh makes an RSA
pair, APK_SIGN_KEY in config.env points at the private half, and the public
half both seeds the USB stick (so the first install can verify) and ships in
anvil-core (so the printer keeps trusting). An empty APK_SIGN_KEY still builds
an unsigned feed, deliberately: `make packages` has to run on a bare checkout
or the packaging lane stops being a CI gate.
WHAT IT COST is a carried patch, against an internal with one consumer, on a
branch mid v2->v3 transition, on an architecture with no upstream apk port to
share the rebasing. That is the standing risk and it does not go away.
Four bugs this shook out, all silent, all found by a gate rather than by
reading:
* The prune deleted seventeen packages it had just built. The all -> noarch
mapping went into the emitter and not into pkg_pkgfile, so the prune looked
for a name nothing wrote. `make packages` exited 0.
* Whitespace-separated Depends dropped nineteen packages. opkg's pkg_parse.c
splits that field on ',' and nothing else, so the whole list became one
dependency whose name contained spaces and resolved to nothing -- and opkg
neither warned nor failed. `make build` shipped a payload with no Tornado
in it. test_the_payload_is_dependency_closed is what catches that now.
* Ownership was recorded as `nobody`. apk resolves an owner to a NAME against
<--root>/etc/passwd and the build lane runs --user with no passwd entry, so
every file in every package would have installed as uid 65534, with no
error at build or install time. build-packages.sh writes a two-line
synthetic passwd naming the build user root and points --root at it.
* Four recipes carried versions apk cannot parse -- two spelled a commit
+git<sha> where the grammar's hash slot is ~, x264 had a snapshot name
whose -2245 reads as a malformed release, and helixscreen had
0.99.115.creator5.3 where apk allows one trailing letter and then only a
suffix from a closed set of nine words. All four were valid opkg versions,
which is why nothing had noticed. pkg_version_ok gates every recipe now.
A v3 .apk begins with ADBd magic and is not a tar stream, so there is no apk
equivalent of the `ar -p | gunzip | tar` that hand-unpacked anvil-opkg: you
cannot open a package without a package manager. bin/build-payload.py puts the
cross-built binary on the stick at /mnt/apk instead. It is the same build
anvil-apk-tools contains, that package installs normally, and the bootstrap
copy never lands.
Two gates retired rather than moved. apk's installed database is plain text
(F: sets a directory, R: names a file in it), so the payload-ownership check
reads one file where it used to glob info/*.list. And it carries no clock at
all -- apk records no install time and mkpkg sets no build-time -- so the
Installed-Time normalisation case-build-payload.sh did on every build is gone,
and the test asserts the ABSENCE of a t: line instead. Clock-free by
construction rather than by repair.
Also removed: PKG_SECTION, which 45 recipes declared and nothing read once
apk's schema had no field for it, and the `make apk-host` target -- the
packager is built on demand by bin/build-packages.sh and stamped on the pin,
the way pkg_toolchain and pkg_buildpython already work.
Verified from a cold work/host and work/packages: 47 packages built and signed,
`make build` installs 35 and verifies them against the shipped public key,
`make qa-static` 195 passed, `make qa-replica` 135 passed -- the printer
installing off a simulated USB stick and booting. The installed set was checked
against the declared closure of the eleven roots: 35 expected, 35 installed,
nothing extra and nothing missing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rbKEyLXrzSE6wm3iNApVp
59 lines
3.1 KiB
INI
59 lines
3.1 KiB
INI
# The only thing that runs on your machine is Docker itself.
|
|
#
|
|
# Every make target executes inside this image: the build scripts, the tests
|
|
# and the install simulations. The docker CLI is included so the simulation
|
|
# targets can start sibling containers through the mounted socket, and
|
|
# qemu-user-static so the printer's real MIPS userland can be executed.
|
|
#
|
|
# A HOST C compiler is here as well, which looks redundant beside two
|
|
# cross-toolchains: cross-building CPython needs a build-python of exactly the
|
|
# same version to freeze modules and byte-compile the stdlib, and that one
|
|
# needs the host's own gcc. pkg-config and perl are for the same section --
|
|
# CPython's configure probes libffi through pkg-config, and OpenSSL's
|
|
# Configure is a perl program.
|
|
#
|
|
# zlib1g-dev is for THE HOST HALF of that build: the build-python needs a
|
|
# working zlib module, or ensurepip cannot unpack pip's bundled .whl. CPython
|
|
# records the module as missing and carries on, so the symptom lands three
|
|
# steps later as "No module named pip". The TARGET's zlib is unrelated.
|
|
#
|
|
# `patch` is for one package: lmdb's setup.py shells out to /usr/bin/patch BY
|
|
# ABSOLUTE PATH to patch the bundled liblmdb, and raises "Applying patch
|
|
# failed" when it is absent -- with nothing to say that a small utility is
|
|
# what is missing. lmdb is Moonraker's database at the pinned commit.
|
|
#
|
|
# libssl-dev IS DEBIAN'S, DELIBERATELY, AND IS THE ONE EXCEPTION TO
|
|
# versions.env. tools/apk-host/build.sh builds a NATIVE apk to write the feed,
|
|
# and apk's crypto backend is OpenSSL. That binary is a BUILD TOOL that never
|
|
# leaves this container; the pin-and-hash rule governs what reaches the
|
|
# printer, and the printer's apk is cross-compiled against the openssl OUR
|
|
# feed built. Same call already made for gcc.
|
|
#
|
|
# ninja-build IS DEBIAN'S ON THE SAME TERMS, and MESON IS NOT -- the split is
|
|
# the point. numpy builds with meson-python, so both are needed; but meson is
|
|
# a Python module our build-python imports, and bookworm's 1.0.1 is below the
|
|
# >=1.2.99 numpy's meson.build demands, so it is pinned in versions.env with
|
|
# the rest of PYPKG_HOST_LIST. ninja is a program meson merely execs, no pin of
|
|
# ours would change what it emits, and nothing it produces reaches a printer
|
|
# except through the cross compiler that is pinned.
|
|
#
|
|
# IT IS NEEDED EARLIER THAN NUMPY, which is the part that surprises: meson-python
|
|
# is self-hosting (backend-path = ['.']), so pkg_buildpython runs meson and
|
|
# ninja just to install the backend itself.
|
|
FROM debian:bookworm-slim
|
|
|
|
ENV DEBIAN_FRONTEND=noninteractive
|
|
RUN echo 'Acquire::Retries "3";' > /etc/apt/apt.conf.d/80-retries \
|
|
&& apt-get update && apt-get install -y --no-install-recommends \
|
|
bash coreutils tar xz-utils gzip bzip2 openssl unzip curl ca-certificates \
|
|
python3 python3-jinja2 python3-pytest python3-pyflakes \
|
|
python3-pytest-xdist \
|
|
binutils file findutils grep sed gawk make git diffutils patch procps \
|
|
squashfs-tools qemu-user-static docker.io shellcheck \
|
|
gcc g++ libc6-dev zlib1g-dev pkg-config perl \
|
|
libssl-dev ninja-build \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
WORKDIR /src
|
|
CMD ["bash"]
|