mirror of
https://github.com/Klipper4FlashForge/firmware.git
synced 2026-10-21 12:52:16 -05:00
master
35
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 MonstrofilandClaude Opus 5
|
0747bc500b |
Keep the config in a directory of our own (#16)
* Keep the config in a directory of our own Flashing a stock FlashForge package back over the mod left a printer that would not start. /usr/data is the DATA partition: a stock flash rewrites /usr/prog and does not touch it, so the symlinked printer.base.cfg and the ff-*.cfg beside it outlived the mod, pointing into a /usr/data/anvil that was gone -- and Klipper treats an [include] it cannot resolve as fatal. Nothing of ours runs during a stock flash, so there is no undo to write from inside the mod. Not writing there is the only version of this that works. The live config is /usr/data/anvil-data/config now, seeded once by anvil-link-prog.sh with a copy of everything in /usr/data/config: printer.cfg with the SAVE_CONFIG block of calibrated values at the end of it, and the stock includes the mod neither ships nor parses. The seed test is printer.cfg, which is copied last and through a temp name, so a machine that never finished seeds again on the next boot rather than staying half-configured, and no update ever re-seeds over a tuned file. It is outside $MODDIR, so the installer's wipe-and-extract cannot reach it and nothing has to be stashed on /tmp. Moonraker moved with it, and the lever is the data path rather than -c: moonraker registers its config root as <data path>/config and nothing configures that separately, so -c alone would move the file it READS while file_manager kept registering FlashForge's directory. With -d /usr/data/anvil-data its config directory is klippy's, which is also what Mainsail's config editor lists. gcodes and logs stay where the machine keeps them -- /usr/data/gcodes is klippy's [virtual_sdcard] path -- as symlinks inside the data path, which works because register_directory() realpaths a root that is a link. The database is carried across once, by the service, below the stray-killer: the one point where moonraker is provably not running. What an older release left in /usr/data/config STAYS there. A sweep could delete the links but could not put stock's own printer.base.cfg back, because the release that replaced it kept no copy -- and stock's run.sh force-copies its own onto every flash, which repairs that directory whether or not anything was swept. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fn1k6kaHWhUrsmBfpxjUcG * Ask the config-directory test about this installer, not the bake test_flashforges_config_directory_is_never_written asserted that the mod's filenames are absent from /usr/data/config. On a replica baked from a release that still WROTE them, that is a question about the bake rather than about the installer under test, and it fails for a reason the change cannot fix. Remove them, run the installer over the top, then ask. The property is "an install writes nothing there", and that is now what is measured. Verified on the replica: qa/replica/test_upgrade.py is 18/18. The five test_install.py assertions about the seeded directory still need a machine baked from a package built with this branch -- work/out holds one from 2026-08-30, which symlinks into /usr/data/config as the old code did. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fn1k6kaHWhUrsmBfpxjUcG * Ask the config directory of a machine that has one The five assertions about /usr/data/anvil-data/config could only pass on a replica baked from a package built with this branch, and the bake installs work/out/*.tgz -- whatever was built last. So they failed for a reason the change cannot fix, and the gate could only go green after a full build, which is exactly when nobody runs it. Two different fixes, because the questions are two different kinds. WHAT THE INSTALLED PACKAGE PRODUCED is still asked of the bake, but the directory is now read off the machine instead of written down: PRINTER_CFG in the installed etc/s6-rc/source/klipper/run is the command line klippy is exec'd with, and therefore the only thing that decides which directory is live. test_install.py and test_klippy_extras_import.py follow it, so the include set, the graph walk and the user's printer.cfg are asked about whichever side of the move the installed package was built on. WHAT THIS CODE DOES is a new module, test_config_directory.py, which stages anvil-link-prog.sh from the checkout and runs it -- the way test_upgrade.py stages the installer. Only the timing is mocked: it seeds from a real printer.cfg with FlashForge's real includes beside it, links the ff-*.cfg the installed package really shipped, and reads MACHINE= out of the real app_startup.sh. It asserts the seeding carries the machine's own marker, the links point into $MODDIR, the chamber config matches the model, every include resolves, a second run does not re-seed over an owner's edit, and the run adds NOTHING to /usr/data/config. That last one is a DELTA, not an absolute. A machine that has been through a release which symlinked into FlashForge's directory still holds those links and nothing can restore what they replaced. What this code owes is that it adds nothing. Verified on the replica: 135 passed, and the 10 failures are the same ones master has on this bake -- ffmpeg, numpy, the timelapse component and the hardware services, all missing from the 2026-08-30 package in work/out. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fn1k6kaHWhUrsmBfpxjUcG * Link every .cfg the packages ship, not a list of names test_every_include_resolves_in_the_new_directory failed on CI: printer.base.cfg has [include timelapse.cfg] and no timelapse.cfg was in /usr/data/anvil-data/config, which Klipper treats as fatal. anvil-link-prog.sh linked printer.base.cfg and the ff-*.cfg by name. timelapse.cfg is anvil-timelapse's and matched neither, so it reached the live directory only through the installer's copy of $MODDIR/config -- and never at all on the `apk upgrade` path, where the link script is the only thing that runs. A named list is a file that resolves on a fresh install and goes stale the moment a package adds one. So the rule is the glob: every .cfg in $MODDIR/config is linked, and the installer skips every .cfg rather than the four names it knew. The .conf beside them stay the installer's -- moonraker-custom.conf is the owner's file to edit, so it is copied once and never linked to a package's copy. test_the_mods_configs_are_links_into_moddir asks the same way now, so a package whose config nothing includes is covered too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YULWHVHyxKm2HE9kiPfEjP --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
f8b925e48f |
Replace opkg with apk, and sign the feed
docs/notes/85-packaging.md chose opkg in 2026-08 on one argument: every package
is cross-built here anyway, so the winner is whichever format is cheapest to
PRODUCE -- and .ipk had an upstream tool where .apk meant abuild, a keypair and
an Alpine host. Two lines of that comparison were wrong.
apk mkpkg is an applet of apk itself: -I key:value metadata, -F directory, no
keypair, no abuild, no Alpine host. The "making a package" row described
apk-tools 2. And the on-device client was counted a wash when it is better than
one -- openssl and zlib were already cross-built for CPython, and libarchive
existed in this tree only for opkg, so adopting apk REMOVES a cross-build.
What made it possible at all is that apk resolves its database relative to
--root and uses the same root as the file destination, while / here is a
read-only squashfs -- measured: neither /lib/apk nor /etc/apk can be created.
pkgs/3rdparty/apk-tools/prefix.patch splits the two. Upstream already carries
root_fd and dest_fd separately and merely aliases them in context.c, so the
patch defaults the database root to $MODDIR and leaves files extracting at /.
That is opkg's compile-time VARDIR plus `dest root /`, in apk's spelling.
THE FEED IS SIGNED NOW, which it never was: opkg signed the index and
bin/build-packages.sh signed nothing, so nothing was verified on a printer.
apk verifies with raw PEM public keys, not X.509 -- measured on the machine,
where every certificate directory is empty. ./bin/apk-keygen.sh makes an RSA
pair, APK_SIGN_KEY in config.env points at the private half, and the public
half both seeds the USB stick (so the first install can verify) and ships in
anvil-core (so the printer keeps trusting). An empty APK_SIGN_KEY still builds
an unsigned feed, deliberately: `make packages` has to run on a bare checkout
or the packaging lane stops being a CI gate.
WHAT IT COST is a carried patch, against an internal with one consumer, on a
branch mid v2->v3 transition, on an architecture with no upstream apk port to
share the rebasing. That is the standing risk and it does not go away.
Four bugs this shook out, all silent, all found by a gate rather than by
reading:
* The prune deleted seventeen packages it had just built. The all -> noarch
mapping went into the emitter and not into pkg_pkgfile, so the prune looked
for a name nothing wrote. `make packages` exited 0.
* Whitespace-separated Depends dropped nineteen packages. opkg's pkg_parse.c
splits that field on ',' and nothing else, so the whole list became one
dependency whose name contained spaces and resolved to nothing -- and opkg
neither warned nor failed. `make build` shipped a payload with no Tornado
in it. test_the_payload_is_dependency_closed is what catches that now.
* Ownership was recorded as `nobody`. apk resolves an owner to a NAME against
<--root>/etc/passwd and the build lane runs --user with no passwd entry, so
every file in every package would have installed as uid 65534, with no
error at build or install time. build-packages.sh writes a two-line
synthetic passwd naming the build user root and points --root at it.
* Four recipes carried versions apk cannot parse -- two spelled a commit
+git<sha> where the grammar's hash slot is ~, x264 had a snapshot name
whose -2245 reads as a malformed release, and helixscreen had
0.99.115.creator5.3 where apk allows one trailing letter and then only a
suffix from a closed set of nine words. All four were valid opkg versions,
which is why nothing had noticed. pkg_version_ok gates every recipe now.
A v3 .apk begins with ADBd magic and is not a tar stream, so there is no apk
equivalent of the `ar -p | gunzip | tar` that hand-unpacked anvil-opkg: you
cannot open a package without a package manager. bin/build-payload.py puts the
cross-built binary on the stick at /mnt/apk instead. It is the same build
anvil-apk-tools contains, that package installs normally, and the bootstrap
copy never lands.
Two gates retired rather than moved. apk's installed database is plain text
(F: sets a directory, R: names a file in it), so the payload-ownership check
reads one file where it used to glob info/*.list. And it carries no clock at
all -- apk records no install time and mkpkg sets no build-time -- so the
Installed-Time normalisation case-build-payload.sh did on every build is gone,
and the test asserts the ABSENCE of a t: line instead. Clock-free by
construction rather than by repair.
Also removed: PKG_SECTION, which 45 recipes declared and nothing read once
apk's schema had no field for it, and the `make apk-host` target -- the
packager is built on demand by bin/build-packages.sh and stamped on the pin,
the way pkg_toolchain and pkg_buildpython already work.
Verified from a cold work/host and work/packages: 47 packages built and signed,
`make build` installs 35 and verifies them against the shipped public key,
`make qa-static` 195 passed, `make qa-replica` 135 passed -- the printer
installing off a simulated USB stick and booting. The installed set was checked
against the declared closure of the eleven roots: 35 expected, 35 installed,
nothing extra and nothing missing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rbKEyLXrzSE6wm3iNApVp
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
5d7754285e |
Wipe $MODDIR and extract, instead of pruning by a manifest
The installer deletes /usr/data/anvil outright and extracts into the empty directory. The installed set is the shipped set by construction, so a renamed file cannot survive beside its replacement and no list has to be shipped, read or trusted for that to hold. What that costs is everything else under $MODDIR, and the audit of what was actually there is docs/notes/86-wipe-and-extract.md. backup/stock had no restore path in any script and omitted the one file a restore would need. .prev-root-hash was already dead state. config-installed was a twelve-file snapshot serving one md5, for a compare whose stated rationale -- "there is no include-and-override seam" -- stopped being true when moonraker-custom.conf became the last include, and which inverted its own purpose by keeping the webcam block and the API lockdown off any printer that tripped it. So moonraker.conf is overwritten like every ff-*.cfg. HelixScreen's /tmp stash is untouched and is now the only preservation the installer does, so the gate asserts it with a value that could only have come from the printer. Gone with it: both delete passes, the forged-path guard, the rmdir pass, the pre-manifest branch, the unconditional init.d/anvil-service/ anvil.conf removal, the etc/s6 scandir sweep and the $MODDIR __pycache__ sweep -- the last two existed only because a manifest cannot see paths made at runtime. firmwareExe already makes the scandir itself. test_upgrade.py rewritten: the negative control is inverted rather than deleted, and the legacy fixture goes -- there is one path now. qa/static green (112). The replica lane needs a package in work/out, which needs config.env with FF_KEY and STOCK_TGZ; not run here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KXKk3zz8LAUPUDZkVEZ9fR |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
b98552386b |
No FlashForge component ships; patch.sh is payload.sh
The software component existed to give the mod one exec hook. It is not needed for that any more, so a release now carries our installer and our payload and nothing else: 26MB smaller, 21MB of which was FlashForge's own firmwareExe travelling back to the partition it was already on, along with their klippy tree, their wifi module and their configs. WHAT IT GIVES BACK. /usr/prog/PROGRAM/software is no longer rewritten, so the version directory there stays FlashForge's with their genuine firmwareExe inside it -- and app_startup.sh's five-second watchdog, which copies one out of that directory when nothing by that name is running, has something to copy again. A printer whose payload fails to install now comes up on the stock UI instead of a blank screen. That recovery has been broken since we started deleting their binary out of a component we shipped; docs/how-it-works.md claimed it worked throughout. THE ROOT PASSWORD moves with it, because it used to ride in the component's shadow file. bin/pack.sh bakes ROOT_PW_HASH into the installer, single-quoted -- an unquoted $6$... would have the printer's shell expand $6 as a positional parameter and ship a truncated hash -- and the installer writes it to /usr/prog/etc/shadow. The random-password path asks one question instead of four: FlashForge ship the same root hash on every machine, so a live shadow still carrying it means nobody has set a password and it is ours to set. The .prev-root-hash recording, the shipped-vs-live comparison and the restore-it- afterwards step were all machinery for surviving a file that nothing overwrites any more, and are gone. pack.sh checks the hardcoded constant against the stock package it built from, because getting it wrong is silent and total: every printer would read as "already set" and none would ever get a password. bin/patch.sh is bin/payload.sh. It patched nothing any more -- no splice, no shadow edit, no firmwareExe deletion -- and what is left is the payload: the feed installed by the printer's own opkg, the seeded config, the manifest. Its sections are renumbered 1-5 with no gaps, and the five tombstone comments marking where removed sections used to be are gone with them. test_recipe_layout.py loses six of its nine tests. Each compared the tree against a description of the tree -- no fourth directory kind, each kind still used, 3rdparty carries no files of ours, installer/ holds exactly two named files -- and none could go red for a reason a printer would notice, while all six went red the moment the layout legitimately changed. The two that bite are kept: a payload/ path escaping $MODDIR, and a recipe name at both levels, which makes pkg_dir silently build the wrong package. SW_VER is deleted from config.env.example and both workflows: it stamped a component that is not built. Gates: 112 static tests pass. The generated installer is verified after substitution -- all four lines present, and `sh -n` over the result, because a hash with a stray character in it would otherwise reach a printer as a syntax error whose only symptom is a package that installs nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CH7Yb4wuPhnppHAB25UX7e |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
56b5b6c334 |
The printer image is the only replica; make rootfs was a second way nothing took
`make rootfs` unsquashed rootfs.squashfs out of the stock package into
work/rootfs, and a thin container bind-mounted it with PROG_DUMP supplying
/usr/prog from the factory image. Every caller set PRINTER_IMAGE anyway --
both workflows wrote it into test.env BEFORE running `make rootfs`, so CI
unpacked a 93MB package, unsquashed a rootfs and downloaded a 182MB factory
image on every push and read none of it. build-printer-image.sh never used
work/rootfs either: Dockerfile.full does its own extraction from the same
public firmware and needs no stock package at all.
So the fallback went, and PROG_DUMP with it -- it was only ever read when the
replica was built locally, and both implementations already passed it empty
otherwise. Gone: the target, extract-rootfs.py, gates.extract_rootfs and its
two tar helpers, the thin printer/Dockerfile, the `prebuilt` flag threaded
through both Replica implementations, and Replica.start's want_output, whose
only message was that the local build was about to be slow.
PRINTER_IMAGE is pinned to 1.9.7-1.2.9-20260810 rather than :latest. It is
now the entire substrate the brick-safety gates run on and the only input
nothing checksums, so a floating tag would let it change under a green suite
-- the drift the sha256 pins in both workflows exist to prevent.
THREE THINGS THAT COULD NOT HAVE PASSED, found while doing it:
* Neither workflow built the feed. bin/patch.sh has refused to start
without one since
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
fbb3182e68 |
bin/verify.sh is retired; four of its forty checks move to qa/replica
verify.sh described itself as "simulate every check the printer performs,
against a built package". The word carrying that sentence is SIMULATE, and it
is why thirty-six of the forty are deleted rather than moved.
THE REPLICA LANE DOES NOT SIMULATE THE INSTALL. install-package.sh puts the
package on a genuine FAT filesystem at /dev/sda1 and runs the machine's own
/usr/prog/app_startup.sh over it, verbatim, under qemu. So these are
PRECONDITIONS of the lane now rather than assertions in it -- break any and
installed_image() raises with the install log and the whole lane goes red:
* the decrypt with the firmware key (the printer's unTar does it)
* runFirmwareExe.sh present, components plain tar not xz (bare `tar -xvf`)
* md5sum.list verifying (the installer aborts and deletes the payload)
* the MACHINE= gate (runFirmwareExe.sh refuses a foreign package)
* the /mnt/<Model>-*.tgz glob (app_startup.sh finds the file by it)
A host-side re-reading of those rules is a second implementation, and a second
implementation can agree with itself while disagreeing with the printer. That
is the defect qa/replica/conftest.py records this repo learning twice about
hand-placed payloads.
A second block was asking what the suite already answers, of an installed
filesystem rather than of a tar listing: the klippy tree and c_helper.so
(test_install, test_abi), the s6-rc database and the oneshot runner's execline
(test_s6rc), s6 and s6-ftrigrd (test_supervisor), the .install-manifest
(test_upgrade), libsodium's bare .so symlink and the chamber configs
(static/test_ipk), and every shipped script's syntax
(static/test_shell_syntax).
THE FOUR THAT MOVED are the ones nothing else asks, and each is stronger for
the move -- a `tar -t` listing cannot tell you that a shared object loads, or
what a name resolves to on a PATH:
* the interpreter IMPORTS sqlite3, lmdb and _cffi_backend, rather than a
grep finding three .so files. This is what catches a cross build that
resolved to an x86_64 manylinux wheel: present on disk, raises on import.
* nothing in $MODDIR/bin shadows FlashForge's python3 -- asked with
`command -v` after sourcing anvil-env.sh, which is the file that PREPENDS
$MODDIR/bin to PATH and so creates the hazard in the first place.
* CPython's dev half (headers, pkgconfig, config-3.13-*) did not ship.
* the ship boundary: no file on the installed printer is byte-identical to
one under bin/ or docker/. Asked of the machine rather than of the .tgz,
so it also sees whatever a maintainer script wrote as root at install
time. The sweep filters on exact size before hashing, which is what keeps
a walk of /usr/prog through qemu cheap.
WHAT GENUINELY LEFT, recorded in docs/qa-migration.md rather than only here:
there is no longer a check you can run on the build host, in seconds and
without docker, against a package you are about to flash. Nothing warns you
before you copy the file to a stick that it was built from the wrong model's
stock package. The printer still refuses it and `make qa-replica` still
catches it, but the fast host-side warning the hardware checklist used is
gone. `make verify` is removed and `make release` no longer calls it.
VERIFIED. All four ran against a real replica (the cached installed image,
PRINTER_IMAGE=monstrofil/creator5-printer:latest): 4 passed in 17s. Then each
was DRIVEN RED on purpose with a throwaway module, which is this repo's own
standard from
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
57c776ee63 |
test/ is deleted; qa/ is the only suite
The eight files were already gone from the working tree. What this commit does
is everything that had to follow them, because nothing else had been updated
and three of the four things pointing at test/ were on a build or release path.
WHAT WAS STILL WIRED TO IT
pkgs/klipper/build.sh ran test/test-chelper.py under `|| pkg_die`. With the
file gone `make packages` would have died on every
klipper build.
bin/verify.sh ran the same script and read a non-zero exit as
"c_helper.so does not match the shipped klippy tree
(stale build)" -- a false red on every release verify
rather than an error about a missing file. Its
ship-boundary check also globbed `find bin test
docker`; that is `find bin docker` now.
pytest.ini testpaths = test, so a bare `pytest` collected
nothing at all. It is `qa` now, which is what the
comment above it was already describing.
qa/static/ PY_GLOBS listed test/*.py and test/integration/*.py,
test_shell_syntax.py and _files() makes an empty glob a collection error
on purpose -- so the whole static lane failed to
collect, taking 93 unrelated tests with it.
Also dropped: the `pytest -- the host-side tree` step in ci.yml, the `test-py`
target with its .PHONY entry and two help lines, and qa/conftest.py's opening
paragraph explaining why qa/ sat beside test/ rather than inside it.
THE CHELPER SYMBOL CHECK IS GONE, NOT MOVED
It compared the .so's dynamic symbols against what klippy cdefs, and it was the
only check for a c_helper.so built from older sources than the klippy tree
beside it -- the failure that shipped once and bricked klippy startup on
hardware, because cffi resolves lazily and a stale .so imports cleanly.
qa/replica/test_install.py covers the ELF arch and nan2008 and nothing else.
What stands in its place is structural rather than a gate: pkgs/klipper
compiles the .so from the chelper sources of the very tree it ships, so the
mismatch is not something the recipe can produce. The comments in
pkgs/klipper/build.sh and docs/building.md now say that is the whole defence.
DOCS
docs/testing.md's "what is left in test/" section is now "test/ is gone", and
it names what went uncovered rather than leaving it to be found: the tool-frame
arithmetic in ff_toolchange, the chamber macros' per-model branching, the stamp
discipline in ff-startup.py, and the verification G-code against the shipped
macros. docs/hardware-testing.md said `make test-py` keeps the two
gcode/creator5-*.gcode files honest; it now says nothing does, and to read them
by hand when a macro is renamed. Paths corrected in docs/building.md,
docs/printer-replica.md and docs/qa-migration.md's intro.
Verified: `pytest` collects 366; qa/static is 136 passed, up from 43 because
the lane could not collect before this. Its 110 errors are this checkout's
missing vendor/opkg-utils, the same cause before and after. bin/verify.sh,
pkgs/klipper/build.sh and the Makefile parse; ci.yml is valid YAML.
Not touched, and stale before this commit: tools/python/README.md and
tools/supervisor/README.md still document test/integration/printer/case-*.sh
and a printer-exec.py that earlier commits removed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016bLGYpVAeVzAX6NeAxcU42
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
7dda2fee0d |
ABI gate: one check, on the filesystem the kernel loads from
mips_abi_gate answered "did this recipe stage the right ABI", once per recipe, over a PKG_ROOT. Four implementations of that one rule had grown up around it -- the function in bin/common.sh, the call at the packaging boundary, two recipes that re-ran it on their own output, and a readelf|grep nan2008 in bin/verify.sh reading a single .so -- and between them they still could not see the largest binary we ship: bin/patch.sh unpacks HelixScreen into /usr/prog and no gate ever read that tree. pkgs/helixscreen/pkg.conf had carried a note saying exactly that. qa/replica/test_abi.py replaces all of it. It walks the installed filesystem on the printer's own CPython -- one qemu process, ~24000 files, 8s -- and reads the ELF header of everything it finds. That is the only place where what the recipes staged, what opkg unpacked, what patch.sh copied and what FlashForge shipped are all visible at once, and it is the thing the question is actually about: the kernel does not read a PKG_ROOT. MEASURED, not assumed: 603 ELF objects on a real install. 569 the loader handles and 568 of those conform. The exemptions are the other two, both narrow. ET_REL objects (33 stock .ko files, plus a stray python.o in FlashForge's CPython 3.8 tree) are not asked about the FP ABI, because insmod and the linker read them rather than the ELF loader and kernel code has no FP ABI to agree about -- by e_type, so the python.o needs no case of its own. And /usr/prog/PROGRAM/ffstartup-arm is an ARM binary FlashForge ships that nothing in the stock tree names; exempt by path, so an ARM binary anywhere else still fails. The architecture is settled before anything is classified, and that ordering came from planting a counterexample rather than from reasoning: a big-endian object decodes its own e_type big-endian, so ET_DYN reads back as 768, lands in neither bucket, and an earlier draft looked straight at it and passed. Every gate here was driven red on purpose before it was kept -- an x86-64 .so in $MODDIR, a real mipsel object with EF_MIPS_NAN2008 cleared, an ARM binary outside the exempt path, a big-endian object, an x86-64 object among the .ko files, and a truncated sweep. test_install.py's two chelper header tests go with it: they read one file out of the several hundred on the same fixture, a strict subset of what now runs. Whether the package ships a chelper at all stays test_install.py's question and is not asked twice. test_ipk.py's assertion that the gate exists becomes an assertion that a second one has not grown back beside it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NpH87TJJEjDyRhQPu18pgp |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
965cf67d93 |
Compile the s6-rc database where the packages are installed
The replica already installs the feed with the printer's own opkg, and anvil-s6-rc already ships s6-rc-compile -- pkgs/3rdparty/s6-rc/build.sh keeps it deliberately, "the only way to recover a printer whose database is wrong without rebuilding a package on a laptop". So the compiler was sitting in the container the whole time, three lines after the install that puts the source tree there. bin/patch.sh section 5b-2 built a second, NATIVE skalibs+execline+s6+s6-rc instead -- four ./configure && make passes cached in work/.s6-native -- purely to get an s6-rc-compile that runs on x86. 110 lines, and its entire hazard was self-inflicted: s6-rc-compile bakes the #! of the execline the COMPILER was linked against into the oneshot runner, so a native stack configured with a different --prefix ships a database whose every oneshot ENOENTs on the printer. Compiling with the compiler we ship cannot get that wrong. The output is the same object. docs/notes/80-s6-migration.md records the measurement from 2026-08-28: one source tree, the host compiler and the target one under qemu, byte-identical db, n and resolve.cdb and identical servicedirs. That measurement was made to justify compiling on the host; it works just as well in the other direction. The rejected alternative in the old comment -- ship s6-rc-compile and compile at INSTALL time on the printer -- is a different thing and stays rejected. qa/static/test_s6rc_source.py is dropped. It was the only consumer of the native compiler, and it hard-failed on any checkout that had not run `make build`: 14 of the static suite's errors were it saying so. What goes with it is the timeout and edge assertions, which are now made nowhere; qa/replica/ test_s6rc.py still boots the real thing. bin/verify.sh keeps the half a host can still check, moved out of patch.sh: the oneshot runner's shebang, and that compiled/current resolves -- read as text straight out of the shipped payload, so it needs no mipsel binary. bin/patch.sh: 278 -> 207 lines. Static suite: 122 passed, errors 124 -> 110. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01La5MaDKqTW675k5fsXsP6S |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
53292b4d78 |
klippy runs from $MODDIR, so patch.sh stops staging a second copy
The klipper s6-rc service already execs klippy under $FF_PYTHON -- our own CPython 3.13 -- but it read the tree from /usr/prog/klipper/klippy: a copy bin/patch.sh staged into the software component out of the very anvil-klipper package that also installed it under $MODDIR. Two copies of one tree, and the one that ran was the one a stock flash could overwrite. One line moves it: KLIPPY=$MODDIR/klipper/klippy/klippy.py. That deletes sections 1 and 2 of bin/patch.sh -- the cp -a of the klippy tree, the chelper.tar wrapped around a .so the package already contained, the rm -rf of the stock extras/ and kinematics/, and the ff_*.py copy. The software component now carries firmwareExe, start.sh and run.sh and nothing else of ours, which is the phase 7 end state in docs/notes/80-s6-migration.md. It also lets pkgs/klipper declare the dependencies it has had since the interpreter moved. cffi and greenlet reached the printer only through a hardcoded list in patch.sh, so every build had them and no `opkg install anvil-klipper` did; pyserial and jinja2 came only as anvil-moonraker's Depends, so a BUILD_MOONRAKER=0 build shipped a klippy that dies the first time it opens an MCU. Both are real Depends now and MOD_ROOTS drops to ten. Also repointed: run-append.sh's __pycache__ sweep, which was clearing the tree nothing imports while klippy writes bytecode into the one it does -- and those runtime-created directories are in no .install-manifest, so the manifest delete walks past them. verify.sh reads the klippy tree and c_helper.so out of the payload (extracting ./klipper alone), and gains a check that the component does NOT carry a klippy tree. New replica tests assert the service execs the payload tree under $FF_PYTHON and that cffi/greenlet/serial/jinja2 import on it. Comments in anvil-env.sh, verify.sh and pkgs/klipper/pkg.conf that still said klippy runs on FlashForge's 3.8.2 are corrected; that stopped being true when FF_PYTHON moved. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01La5MaDKqTW675k5fsXsP6S |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
c185ef3285 |
The replica is a build tool; test/ is 28 unit tests
Two changes that turned out to be one.
MOVED, NOT DELETED. `test/ffsim/` and `test/integration/printer/` are
`tools/replica/`. They were never tests: bin/patch.sh calls
bin/build-payload.py, which starts a replica and assembles the payload by
running the PRINTER'S OWN opkg against / inside it -- so the "test harness"
was on the build path, and deleting it as asked would have broken `make
build`. qa/lib/replica.py drives the same directory unmodified. No content
changed; what changed is qa/lib/replica.py's build directory, three launcher
sys.path lines, the Makefile targets and the CI shellcheck glob.
tools/replica/README.md says who drives it and why it is not under test/.
CUT TO 28. The five surviving modules were 136 tests; they are 28, chosen on
one question: would this failure be SILENT? What is kept is the mistakes that
run perfectly and are wrong --
test_startup 8 no stamp unless the values are verifiably saved (a
stamp written early is a printer that has lost its
factory calibration for good), and the handover order
test_tool_transform 7 every one is a move to the wrong place: a sign, a
frame left applied while probing, a stale gcode_move
cache, a config a calibrated printer cannot parse
test_ffscreen 5 the four ways it can do harm rather than look wrong
test_chamber 4 the chamber mistakes that cost a print before anyone
sees them, plus the parse gate they rest on
test_gcode 4 a renamed macro, and the safe file being cold
What went: the panel narration, the geometry probe and parse_geometry, the
z_adjust and per-job-Z bookkeeping, the config layout checks, the G-code
bounds checks. Each is either a wrong number shown to a person, or a failure
the printer reports loudly the first time -- and qa/replica/test_boot_screen.py
already renders the real frames on the real machine.
The rewrite is not a delete: assertions were folded so a kept test asks its
question in every direction it can be got wrong (both models, both rotations,
all four ways the migration can fail), and each file's header now says what it
dropped and why, where the next person will open it.
Also gone: conftest's `rootfs` fixture, orphaned two commits ago. Nothing under
test/ needs the firmware, docker, or a network any more.
28 host-side tests pass in 0.15s. qa/static unchanged at 218. `make rootfs`
re-verified end to end from its new path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzLebCrvc8ssWR5JuEsha4
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
6aa622d2ff |
6,280 lines of case scripts reported 13 bits. They are 118 tests now.
test/run-tests.py is gone, and with it the bespoke harness that wrapped
pytest, re-parsed its JUnit XML, and drove thirteen replica case scripts to
get one boolean back from each. Every gate it ran is either a module under
qa/replica/ or deliberately retired.
PORTED. case-install.sh (579 lines, 1 bit) is test_install.py, 22 tests, asked
of the machine the printer's own app_startup.sh built rather than one the
harness assembled. case-upgrade.sh is 21, case-supervisor.sh 31,
case-boot-screen.sh 17, case-mcu-bringup.sh 8, and case-nginx.sh plus
case-moonraker313-s6.sh are test_web.py's 7.
RETIRED, subject deleted by the s6-rc switch: case-services.sh, case-nginx.sh,
case-moonraker.sh, case-moonraker313-s6.sh, case-camera.sh and
case-priority.sh drove $MODDIR/init.d/S* or sourced anvil-service.sh, neither
of which exists. Decided against real output -- `make test-services` failed
with "the payload ships no anvil-service.sh". case-pyext.sh and
case-moonraker313.sh went too: no gate, no target, no CI step, never run once.
RETIRED as a coverage decision: case-python.sh (test_web.py runs real Moonraker
on that interpreter, which is harder), case-libpath.sh, and case-recovery.sh.
The last is the brick-safety rollback and has no replacement; it is recorded in
docs/qa-migration.md rather than left to be discovered.
THREE ASSERTIONS HAD STOPPED MEANING ANYTHING, and each had been green for
months:
* case-mcu-bringup.sh's negative control had INVERTED. It asserted the
bring-up fails without LD_LIBRARY_PATH; CPython is --disable-shared now and
needs no library path, so the case's own skip branch was the one firing.
* case-upgrade.sh's headline assertion was VACUOUS. run-append.sh:150 does
`rm -rf $MODDIR/init.d` unconditionally, outside the manifest branch, so
"the update removed the stale S60web" passed regardless of the manifest
logic it was written to test.
* case-boot-screen.sh CONFIRMED ITS OWN ARITHMETIC, computing expected pixel
packing by calling ffscreen._pixel inside the replica -- a swapped R/B
matched itself.
case-install.sh's boot-log grep is not reproduced for the same reason: the log
line names every service in the boot set including the ones that then fail, so
grepping it cannot go red. The compiled database is asked instead.
TWO REAL BUGS FOUND ON THE WAY.
assemble.sh attached /stick.img with `losetup` and wrote the device to
/stick.loop, which NOTHING HAS EVER READ. Loop devices are the host's, not the
container's, so every USB_STICK=1 run leaked one permanently and a WSL2 kernel
has thirteen. Once they were gone every replica run failed with "failed to
setup loop device", which looks like a docker fault and is not. Detached now in
entrypoint.sh on the way out, and in qa's stop() for the held-open containers
that never take that path.
test_startup.py was 43 tests red: it built its argparse.Namespace by hand and
had never been taught --only-bringup / --no-bringup, and its FakeKlipper still
modelled start.sh plus a separate stop when klipper is restarted with one
`s6-svc -wr -t`.
CI AND RELEASE. The replica job was getting its package as a side effect of
run-tests.py running unpack/patch/pack; that is an explicit step now, as the
comment there predicted it would have to be. release.yml tested the exact files
it ships through sim-install.py, whose exit status the loop was discarding --
it is `REAL_PKG=<pkg> pytest ./qa/replica` per model now, 22 named results each.
WHAT IS NOT TESTED ANY MORE: bin/unpack.sh, patch.sh, pack.sh and verify.sh.
The packaging build on a synthetic stock package went with the runner and has
no counterpart in qa/. That is the path producing the .tgz a user flashes.
make-stock-fixture.sh is kept as the input a replacement needs.
Measured: qa/static 246 passed, qa/replica 110 passed and 8 blocked on hardware
the replica has not got (no /dev/ttyS*, no /dev/video*, so ok-all is
unreachable), test/ pytest 185 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BTAXgjtF8GdhqSxvdXCFW
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
1191f87411 |
The graph is asserted by compiling it, not by reading it.
Phase 8 made the boot a compiled s6-rc database. This is the gate for it, plus
the honest accounting of what the change did to the suites that tested the
boot it replaced.
qa/static/test_s6rc_source.py -- 14 tests, no docker, ~1s. It COMPILES
payload/etc/s6-rc/source/ with the native s6-rc-compile that bin/patch.sh
section 5b-2 builds, then reads every answer back out with s6-rc-db. Never out
of the source tree: the compiler is free to reject, rewrite or silently drop
what it was given, and `down` is the one that proved it -- accepted and
discarded, producing a byte-identical servicedir. A test that reads the source
would have called that a pass.
It asserts the bundle is the boot set; klipper waits for mcu-bringup,
ff-startup for moonraker and klipper, ui for ff-startup; mcu-bringup is
reachable but NOT in the bundle, which is the only way s6-rc can express "start
it, but not because the boot asked"; nginx does not gate moonraker, because as
an edge that preference would make `s6-rc -d change nginx` stop the API with
the UI; and the timeouts survive, without which a camera that never appears
holds the transition open for ever and the UI never starts.
Two of them earn their place beyond the graph:
* The generated oneshot-runner shebang must be $MODDIR/bin/execlineb. It
comes from the execline the COMPILER was linked against, so a native stack
configured with the wrong --prefix ships a database where every oneshot
dies with ENOENT and every longrun keeps working. Nothing else in the build
can see that.
* The oneshots' up/down are execline command lines carrying shell inside a
/bin/sh -c argument. bash -n and shellcheck parse the wrapper and pass, so
this extracts the body s6-rc-db actually stored and runs sh -n over it.
The NUL split is also the proof the shell arrived as ONE argument --
execline lexing it into several would be a script that runs its own first
line.
Both were checked by breaking them: dropping ui/dependencies.d/ff-startup
fails one named test with 13 still reporting, and a dangling `then` inside
wifi's up fails the parse test. A first attempt at that second control did not
fail, correctly -- `[ 1 = 1 ; then` is a missing `]`, a runtime error, not a
syntax one.
qa/replica/test_s6rc.py -- written, NEVER RUN, and it says so at the top. There
is no stock FlashForge package on this machine, so `make build` is impossible
and the replica image holds a pre-s6-rc payload. Its `box` fixture checks for
s6-rc-init once and fails with a single message naming `make build`, so the
lane reports twelve errors with one cause rather than forty assertion failures
sharing it. It covers what neither suite ever has: a kill -9'd daemon coming
back, which is the entire reason for shipping 14 binaries.
RETIRED: qa/replica/test_services.py, and the Service class it was the only
caller of. It was the port of case-services.sh and its subject was the wrapper
contract -- seven S* scripts loading a shared library, answering status, naming
themselves, rejecting an unknown verb. None of that exists. Three assertions
survived into test_s6rc.py: the scanner is the cross-built ELF we shipped, its
log is empty, every run in the scandir is executable.
THE OLD SUITE IS RED AND I HAVE NOT TOUCHED IT. case-services.sh,
case-nginx.sh and case-moonraker.sh assert the wrapper contract and the
MOD_S6=0 fallback; case-priority.sh calls svc_start_daemon; test_startup.py
asserts FF_SKIP_MCU_BRINGUP. All deleted. They also cannot be RUN here, and
editing shell I cannot execute to make it assert something new is the thing
this repo's own rule forbids -- never decide whether a script works by reading
it. docs/qa-migration.md now carries the table: per case, what it asserted and
where that lives now, including the two entries that honestly say "nowhere".
Retiring them is a coverage decision and it belongs to whoever has a stock
package and a real run.
bin/patch.sh also ships s6-rc-db beside the native compiler now; the static
lane needs it and it was the only piece missing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
16d00be525 |
case-services.sh is 360 lines and one exit code. That contract is 44 tests now.
test/ has two test frameworks. run-tests.py + ffsim is 1,657 lines reimplementing what pytest already does -- skip accounting, gating, reporting, then parsing pytest's own JUnit XML back out -- and it calls pytest as gate #5 of about twenty. Underneath it, 6,280 lines of case-*.sh execute inside the qemu chroot and report 13 booleans between them; case-moonraker.sh is 1,059 lines that collapse to one pass/fail. Nothing runs in parallel and nothing can be selected: each replica gate is its own `docker run --privileged`, re-registering binfmt and re-assembling the mount layout, thirteen of them in sequence. All three follow from one decision -- THE REPLICA CONTAINER IS TREATED AS A TEST RUNNER INSTEAD OF AS A FIXTURE. That is what pushes assertions into ash, which yields one bit per container, which then needs a bespoke framework above pytest to aggregate the bits. qa/ is the replacement, built beside test/ rather than inside it, with two lanes and one framework. Both suites run in CI; test/ is still the release gate; nothing has been deleted. docs/qa-migration.md has the order the rest moves in, and the rule: a case script is deleted only once its replacement here has been green, so there is never a window where coverage drops. THE MACHINE IS UNCHANGED. qa/ reuses test/integration/printer/'s Dockerfile, entrypoint.sh, assemble.sh, binfmt.sh and seed-prog.sh as they are, and its `docker run` argv is deliberately ffsim's, minus --rm and plus -d. While both suites run they have to be testing the same machine or neither proves anything about the other. The only new trick is how the container is held open: entrypoint.sh ends by running the case script it was given, and the one it is given here touches a marker and sleeps, so setup completes exactly as it always has and the container simply does not exit. The marker is also the readiness signal, because setup ranges from under a second to over a minute and a constant would either waste the fast case or fail the slow one. TESTS RUN AGAINST A MACHINE THE REAL INSTALLER PRODUCED. The `printer` fixture is a replica with the package installed by the printer's own app_startup.sh, off a genuine FAT filesystem on /dev/sda1. The first version of this tree did what case-services.sh does -- copy payload/*.sh and payload/init.d/S* into $MODDIR by hand -- and two things were wrong with it. It is a second implementation of the install, so everything downstream asserted against a layout the harness built rather than one the installer produced, and the real installer could have broken without a single test noticing. And it can only place what is in payload/, so everything the BUILD produces was missing: s6 is a build output that bin/patch.sh stages into the package, so a hand-placed payload has no supervisor at all, which is why the tests had to carry a stand-in scanner and gate one assertion on whether it was in use. Installing for real fixes both and deletes the stand-in. Hand-placed, /usr/data/anvil held anvil-env.sh, anvil-service.sh, anvil.conf, init.d and etc/s6. Really installed it holds VERSION, backup, bin, config, config-installed, helixscreen, lib, libexec, moonraker, nginx and www as well -- bin/lib/libexec being the cross-built s6 and CPython 3.13. The file is SHORTER than the case script it replaces while reporting 44 results instead of one. case-install.sh's header records this same lesson from the first time somebody learned it: "An earlier version of this file replayed app_startup.sh by hand -- which meant a bug in our reading of it could never be caught." The install is the machine's own shell under qemu, so it is baked into an image once per package and cached under a tag derived from the package's md5 -- build-printer-image.sh's trick for the stock baseline. Rebuild the package and the tag changes, so the next run bakes again instead of testing yesterday's build; deliberately not a hand-written staleness check, because those are the ones that eventually get it wrong. A MISSING TOOL OR RESOURCE FAILS, and there is no flag. This tree briefly had --strict-skip=<lane|all>, which was ALLOW_SKIP inverted -- and ALLOW_SKIP's defect was never the direction it named things in, it was that the decision lived in the CI invocation. An accept-list has to be edited whenever a gate moves; a strict-list has to be remembered whenever a job is added; both degrade silently when the command line is wrong, which is the same disease as the printer-sim job that was gated on an unset secret and never ran once in its existence. So the verdict lives with the knowledge: no shellcheck, no pyflakes, no docker, no daemon, no replica image, no built package -- each fails where it is needed, with the fix in the message. There are no skips in this suite. MEASURED, on monstrofil/creator5-printer:latest, both suites agreeing on the same machine: 131 tests in 40s, all green; qa/static alone 1.3s with no docker; one assertion selected by node id 2.9s, against the whole 360-line case being the smallest unit test/ can run. Granularity checked by injecting the regression case-services.sh exists to catch -- deleting SVC_EXTRA_VERBS="|force-start" from S70klipper -- which failed one named test, printed the usage line that came back, left the other 41 reporting, and took 4s to re-run alone. Two bugs the port found in its own harness, both only visible by running it. bake.sh shipped without its executable bit, so the lane died at container init with 43 errors all saying "could not install the package" -- a file mode wearing an installer failure's clothes; test_action_scripts_are_executable now guards it, the same check the payload's own s6 run scripts already have. And an ELF magic assertion used `od -An`, which the printer's busybox od does not have -- the replica catching host-vs-printer drift in the test code itself, which is the entire reason it exists. Also here: qa/static/test_probes.py, which has no counterpart in test/. Printer.ps() and Printer.listening() are parsers, and a parser is where a harness goes quietly wrong -- a listening() that never matches makes every readiness assertion pass by the absence of contradiction. Neither failure looks like a failure; they look like a green suite. So they are tested against captured output, in the static lane, with no docker involved. Two things found while surveying, neither fixed here. case-pyext.sh is 567 lines with no gate in gates.py, no Makefile target and no CI step: it has never run. And case-mcu-bringup.sh:4 says the handshake is "tested elsewhere against a pty" -- there is no pty anywhere in the repo, and every documented past bug in ff_mcu_bringup.py lived in the code that comment excuses from testing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
c870d8417d |
FF_PYTHON moves to our own 3.13, and CI stops rehearsing around it
The gate this switch was waiting on now exists and passes:
case-moonraker313-s6.sh drives Moonraker through the real boot path --
S40s6's scandir, S62moonraker, readiness gated on :7125 actually
listening, a kill -9 respawn, a stop that stays stopped -- on the
interpreter this commit ships. anvil-env.sh now points FF_PYTHON at
$MODDIR/bin/python3.13 and drops /usr/prog/libsodium/lib from
ANVIL_LIBS, since nothing left running under FF_PYTHON imports libnacl
from there any more (Moonraker carries its own libsodium; klippy is not
on this list at all -- it stays on FlashForge's 3.8.2, started
independently by /usr/prog/klipper/start.sh).
Three real bugs surfaced chasing this through the replica, none of them
about the switch itself:
* The Phase-1 manifest-based install can only delete files it once
tracked, so an init.d/S60web surviving from a release that predates
the manifest system would never be cleaned -- caught by
case-install.sh, which had never run against real stock packages
before today. run-append.sh now clears $MODDIR/init.d unconditionally
on every update: those scripts are 100% mod-owned and get executed,
so a diff against tracked history is not a strong enough guarantee.
* case-moonraker313-s6.sh's own readiness-ordering check flaked once:
its bind-time sampler polled at 1-second resolution while s6-svwait
is event-driven and returns within a fraction of a second, so
`date +%s` truncation could show readiness before the bind though the
real events were in order. Tightened to 0.2s.
* case-mcu-bringup.sh and case-boot-screen.sh source anvil-env.sh but
named their own directory variable $MOD, not $MODDIR -- harmless
while FF_PYTHON was a literal path, and silently wrong the moment it
started reading $MODDIR. Aliased.
Every other test/integration file touched here is the same story:
FF_PYTHON's callers were hardcoded to FlashForge's 3.8.2 throughout the
suite, and each one either now stages our interpreter as a tarball
(gates.py's mcu_bringup/boot_screen/moonraker, matching the pattern
already used for python()/moonraker313_s6) or asserts against the new
target. case-libpath.sh's premise changes with it: the three ANVIL_LIBS
entries that remain are read as what klippy's process tree still needs,
not what every mod process needs -- Moonraker maps zero /usr/prog
libraries now, measured.
payload/anvil-service.sh's SVC_S6_SCANDIR line tripped test_paths.py's
absolute-path regex, which reads a closing brace immediately followed by
a suffix as a stock rootfs path in its own right; split onto its own
line so a word character precedes it instead, which the regex already
excludes.
bin/fetch-assets.sh had a real latent bug of its own, independent of
this switch: the Ingenic toolchain fetch was gated on BUILD_KLIPPER=fork
alone, a condition that predates CPython 3.13's unconditional need for
the same compiler. A BUILD_KLIPPER=stock build from a clean vendor/
would reach the Python cross-build with sources and no compiler. Also
added curl --retry -- musl.cc flaked mid-download in CI while this was
being tracked down.
.github/workflows/ci.yml's fixture-only `test` job now actually
cross-builds s6 and CPython 3.13 instead of never exercising that path:
it fetches just those two build inputs (Mainsail/HelixScreen/Klipper
stay stubbed, unchanged) and caches vendor/, work/.s6, work/.py313 and
work/.sodium on versions.env.
Full suite, replica required, against real stock packages fetched for
this run: 23 passed, 0 failed, 0 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
8994b461b0 |
Every service starts the same way, and Moonraker stays on /usr/data
Moonraker is no longer copied to /usr/prog. The payload already extracts
that exact tree to /usr/data/anvil/moonraker, so the installer was making a
second identical copy onto the small firmware partition -- the one install
step that could fail on space, whose failure was "this printer has no
working web UI". Worse, /usr/prog is what a stock flash overwrites while
/usr/data survives, so a firmware update silently reverted Moonraker to
FlashForge's 2022 build and left the rest of the mod in place: exactly the
"which Moonraker is this printer running?" ambiguity the previous commit set
out to end. Neither of the two things thought to pin it there holds -- the
moonraker-env virtualenv is not on sys.path (imports resolve from
Python-3.8.2/lib/python3.8/site-packages, checked on the real image) and
moonrakerDaemon is never invoked. FlashForge's tree is left alone, unused.
S60web became S60nginx and S62moonraker. nginx and moonraker fail for
different reasons and are now stopped, started and diagnosed separately;
MOD_WEB still governs both.
payload/anvil-service.sh is the shape all six services now share. They had
been written one at a time and had arrived at four different answers to "is
it running" and two to "restart", none of them decisions. Two real bugs were
living in that drift, and both are fixed here:
* busybox start-stop-daemon -K returns before the process is dead and
never removes the pidfile, so restart raced its own start -- stop
returned, start found the pidfile still held, and the service ended up
stopped rather than restarted. The `sleep 2` in front of restart was
standing in for that wait; svc_stop_daemon does it properly and the
sleep is gone.
* klippy does not unlink /tmp/uds on exit. supervise() knew that; stop()
did not, so status reported a dead klippy as running and restart
declined to do anything at all.
run-append.sh now clears $MODDIR/init.d before extracting. Without it the
rename above would leave a stale S60web beside the two new scripts on any
printer that already has the mod, and firmwareExe runs every executable
S* in filename order -- starting nginx and moonraker twice, from scripts
disagreeing about where moonraker lives.
bin/patch.sh stages anvil-service.sh. It is sourced, so no chmod +x.
test/integration/printer/case-services.sh is a new replica gate: it runs
every service on the printer's own busybox and checks the contract they
share rather than grepping them for how they are spelled. case-moonraker.sh
now asserts moonraker runs from /usr/data, that nothing under /usr/prog is
executed, that restart yields a genuinely new pid, and that stopping
moonraker leaves nginx up -- the point of the split.
The .modold rollback recovery in docs/how-it-works.md described a code path
removed in the previous commit and is rewritten for what happens now.
Verified in the replica (real rootfs.squashfs, qemu-mipsel): case-moonraker,
case-services, case-mcu-bringup and case-boot-screen all pass; pytest 181
passed, 5 skipped. case-install.sh and case-recovery.sh are syntax-checked
only -- they need a stock FlashForge package, which is not in this checkout.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
bcdd059008 |
S60web sets moonraker's whole library path instead of inheriting most of it
A printer came up with no web UI at all: no Mainsail, no API, just the screen. S60web named three /usr/prog/*/lib directories and inherited the other nine from app_startup.sh, which exports twelve before it runs the wrapper. That inheritance holds on a normal boot and nowhere else -- re-run the script over ssh, or perturb the boot chain, and the environment is bare. The one that matters is libsodium. moonraker's `authorization` component is linked against it, so its absence is not a missing-module error on something obscure: moonraker exits during component load, which from the outside is indistinguishable from "moonraker did not start". moonraker-preflight.py exists because that same library caught us once already, and run-append.sh sets the full list explicitly for exactly this reason -- S60web was the last place still trusting somebody else's exports. The replica proves both halves on the printer's own interpreter: with the list S60web really sets, read out of the shipped script rather than retyped, `authorization` imports; with libsodium removed from that same list it dies on "Could not locate nacl lib". A path entry that is not load-bearing would fail that second check, so the list cannot quietly become cargo. Not taken from the reported workaround: running `python3.8 moonraker.py &` directly. moonrakerDaemon owns the pidfile that `stop` needs, and it sets TMPDIR=/usr/data/tmp -- without which moonraker writes to /tmp, a ramdisk here. Those are two regressions for no gain once the library path is right. Also: a background check now says in the boot log when moonraker is not running, with the tail of its own log. It exits quietly, minutes before anyone opens a browser, and until now the only symptom was a user reporting a blank page days later. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
c7a821bb9d |
ff_mcu_bringup.py is importable by name
The filename was the only reason for the importlib shim: a module called ff-mcu-bringup cannot be written as an import, so ff-startup.py loaded it from a spec and hand-rolled the failure handling. Renaming it removes all of that and leaves a plain guarded import beside the one ffscreen already had. start.sh still runs it as a script, by path, on the fallback route -- a rename costs that nothing. What the shim was hiding is worth keeping visible, though: these imports resolve only because python puts a SCRIPT's own directory on sys.path, which is true when the wrapper runs the file by path and false the moment anything loads it another way. So there is now a --selftest that reports what it can actually reach, and the replica gate calls that instead of loading the module itself -- which it was doing before, and which would have passed while the real path was broken. It is worth having on a printer over ssh too. The unit tests were making the same mistake in the other direction: loading ff-startup.py through importlib with payload/bin absent from sys.path, so its siblings were missing and the selftest correctly said so. The loader now puts that directory on the path, which is what the runtime looks like, and a separate test runs the script from another directory entirely to prove the premise rather than assume it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
ac558f6686 |
Document the Moonraker swap honestly, and finish its doc coverage
The reversibility claim was the strongest promise in the README and
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
b5982b09bb |
Fix eleven defects the first pass introduced
An adversarial re-read of master..HEAD, checking the new text rather than
trusting it. The substantive work held up -- all eight dead-code removals prove
out, SAFE-MODE->MOD_UI is complete, A3's reordering is correct -- but eleven
claims were wrong, and one made a document worse than it found it:
* docs/notes/10-hardware.md said `temperature_sensor motor_value` on
eboard:PA1 is "not in anything this repo ships". It is:
printer.motor.cfg:37 declares it and that file is in the package, which is
also why ff-toolchange.cfg can name MOTOR_STOP. The real nuance is that
printer.base.cfg does not include printer.motor.cfg. A correct claim in
25-app-vs-klipper-ownership.md was "corrected" into a false denial.
* docs/toolchange.md pointed at a worked example that reaches 0.125 mm. The
only example is NOZZLE=220 BED=80, which gives 0.045; the 0.08 bed term
needs a bed at 100 or above. Both numbers now stated, no forward reference.
* docs/testing.md said 48 tests; pytest collects 53.
* test_config_ownership.py cited bin/patch.sh:80/:88/:128 -- correct on
master, shifted by two when this branch deleted MARK_BEGIN/MARK_END. The
line numbers are dropped rather than re-pinned, so they cannot rot again.
* test_includes.py invented a reason for ff-legacy being last ("so its import
sees every [ff_tool] already declared"). ff_legacy resolves tools at
klippy:ready; order does not matter.
* fetch-assets.sh's replacement usage text omitted BUILD_MOONRAKER and was
less accurate than the vague line it replaced.
* 49-runout's sensor table still claimed `extruder: extruderN` after the A1
fix made that true only of our override, not of the stock file it describes.
* firmwareExe kept "newest version directory" after the same claim was
corrected in how-it-works.md.
* 25-app inverted the preflight call order and omitted that an M23 print gets
neither call.
* creator5-safe-moves.gcode called the prepare's G1 Z10 "the one move" below
Z50; the G28 and offset move before it also go below.
* DOC-DRIFT-AUDIT.md asserted both "Every finding has been applied" and
"Nothing in this file has been applied", and cited five commits after a
hash the rebase had rewritten.
53 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
d05565284d |
Sections E, F, G: wrong values, stale claims, overclaims
E -- values and counts. The START_PRINT prologue has no M400 and an idle timeout of 864000, not 1800000. The nozzle-clean temperature comes from the material table FIRST, so NOZZLE=240 with no per-tool temp cleans at 220 -- documented backwards before. TOOLS is the paired 0:220,2:240 form. The wipe Z is the raw eddy frame, not the homed one. The runout M117 strings are "T<n> clog" / "T<n> out of filament", not E-codes. Grab-verify raises E0051+tool, not E0143. Docks are at X~297; 250/280 are staging waypoints. The grab sequence is documented in the order _grab actually emits it. ff_print exposes seven status fields, not nine, and never emits TOOLS=. Package sizes reconciled against the real artifacts (62MB built, ~93MB stock, ~370MB of CI assets, ~60s not 90s). testing.md's sample output block matched no header the code prints, so it is replaced with the real ones; its pytest row listed two of six suites. F -- stale claims. _FF_PREFLIGHT is not "missing". SDCARD_SET_CHANNEL is not still issued ( |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
b2c651b11f |
Fix the runout insert path and TOOL=ALL staging; drop SAFE-MODE for MOD_UI
A2: _FF_INSERT's body was entirely inside {% if kind == 'switch' %}, yet
insert_gcode is wired on all four fm_ex* with KIND=motion. A motion insert --
which is what clearing a clog looks like, filament moving again -- produced no
output at all, on exactly the path where the operator most needs the next
step. Adds the motion branch the comment already described.
A3: TOOL_OFFSET_CALIBRATE staged each tool with set_nozzle() inside the loop,
so a gap-check failure on a later tool left earlier tools pending for
SAVE_CONFIG -- while raising an error that says "nothing saved". True for
TOOL=<n>, false for TOOL=ALL, and told to the operator at the moment they
decide whether to save. Staging now happens after every tool has passed.
A8: SAFE-MODE is removed rather than repaired. It never worked: the settle
check cleared the counter 90s after every boot on a condition
(`[ -s $CHOICE ]`) that cannot be false, since $CHOICE is written two lines
earlier and "none" is not empty. Nothing checked whether helix-screen was
alive, so the latch only fired on three reboots inside 90 seconds -- not the
crash-loop it was written for.
In its place, MOD_UI in anvil.conf, alongside MOD_WEB/MOD_CAM/MOD_SSH/MOD_WIFI
and templated by patch.sh the same way. It stops the UI being started; it does
not pretend to repair one. The docs said `touch SAFE-MODE` was how you recover
a blank screen. Nothing recovers a blank screen on the machine: HelixScreen is
the only UI, and getting one back means flashing a package. Said plainly now,
in the README, the recovery section and the troubleshooting table -- along with
what is genuinely unaffected, which is ssh, Mainsail, Klipper and printing.
48 passed, 5 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
e5bd171cf8 |
Correct the docs the code had drifted away from
Applies the audit's sections B, C and D, plus two decisions. Behaviour (A1): fm_ex1..3 carried stock's `extruder: extruder`, so a clog on T1..T3 was measured against T0's filament movement. Each motion sensor now watches its own extruder. Gate count (A6): "four gates" was the number of make test-* targets, not of gates run -- run-tests.py emits twelve. The number is dropped rather than restated, in docs/testing.md and the Makefile help. B, install instructions that failed as written: STOCK_TGZ_CREATOR5 missing its .tgz; a pointer to a ./build.sh that does not exist; klippy-extras/ paths that moved to payload/klipper/extras/ in 0ef9d6b; a calibration recipe both commands refuse without PLATE_REMOVED=1; a storage table that put dock_x/dock_y in an include, which SAVE_CONFIG refuses; four files telling the user to add includes printer.base.cfg already ships; ./pack.sh style paths that do not resolve from where the scripts leave you. The MAINSAIL_ZIP / HELIX_TGZ override was documented in three places as "used as-is, never checksummed". It is checksummed, and fetch-assets.sh overwrites the user's file with the pinned release -- SKIP re-downloads too. Corrected, with the way that actually works (match the hash in versions.env). C, files that contradicted themselves: the -5/-3 ESTOP target, the t<n>_offset_* storage names, the chamber heater's model split, the LED premise, the include order comment in test_includes.py, gear_stepper as a "feed hub" when it is the tool lock, an object list ff-toolchange.cfg says does not exist, the eddy probe vs the under-bed cylinder, "installs them without clobbering" against a gate that enforces the clobber, "keeps two versions" when the installer keeps one, "two changes from stock" when there are three, and ff_legacy's "nothing here runs during normal operation" when auto_import runs on every ready. D, dangling references: OKF/* and firmwareExe-decompiled/* repointed at docs/notes/*, plus case-boot.sh, sim-image.sh, printer-exec.sh, the test/printer build context, wait_for_banner, ff_toolchanger_compat and a nonexistent orca snippet. Not touched, pending a decision: sections E-H, and the printer.base.cfg "verbatim" claim (A7), which depends on how the unpack drift check is fixed. 48 passed, 5 skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
f70369155b |
Run the brick-safety gates in CI, and stop skipping quietly
The replica gates -- the only ones that decide whether a package bricks a
printer -- had never run in CI. They were gated on a STOCK_URL_CREATOR5PRO
secret that was never set, so printer-sim rendered as a grey dash beside
two green ticks and read as "CI passed".
The packages it needs are already public. ghzserg/FF publishes both stock
packages and the factory image, and this repo already points at that
release in docs/printer-replica.md. All three are now pinned in ci.yml by
sha256 and verified byte-for-byte against the copies used locally:
Creator5Pro-1.9.7-1.2.9-20260810.tgz 1f3252fe... 93MB
Creator5-1.9.7-1.2.9-20260810.tgz 72009be1... 93MB
Creator5Pro-factory.tar.xz 45657924... 190MB
Nothing proprietary is redistributed from here; the workflow fetches what
is published elsewhere, cached on their checksums so an unchanged pin
costs that archive nothing. Secrets still override for a fork with its own
firmware -- but a URL override without a matching checksum now refuses to
run rather than downloading something nobody verified. have-firmware is
gone with the `if:` it fed.
Rehearsing that job locally found the reason it would have failed anyway.
The replica gates call ./bin/patch.sh directly, and patch.sh expects
vendor/ already populated -- bin/build.sh is what fetches Mainsail and
HelixScreen, and the gates do not go through build.sh. No local run ever
noticed, because vendor/ was always already there. Both workflows now
fetch assets before the suite; in release.yml that step has to precede the
gates, since `make release` would otherwise fetch them one step too late.
Second half: a skip must be legible.
ALLOW_SKIP=1 silenced the explanation as well as the exit code -- the
summary printed "2 skipped" and then nothing about what or why. That is
the disease this harness exists to treat, reintroduced by the flag meant
to manage it. Now:
- Every skip is listed again in the summary with its reason and whether
it was accepted, whatever ALLOW_SKIP says, and becomes a GitHub
annotation. Nobody reads a green job's log.
- ALLOW_SKIP takes names: ALLOW_SKIP="pytest,the printer replica". `1`
still accepts everything, but it is a standing promise never to notice
a skip again, so CI uses the list and a third skip fails there.
- The pytest gate names the tests that skipped instead of counting them.
That immediately showed the exec-bit test skipping wherever .git is
absent -- a bare "6 skipped" would not have.
VERIFIED:
- printer-sim rehearsed end to end on the actual published bytes, real
git checkout, REQUIRE_PRINTER_SIM=1 and no ALLOW_SKIP: 12 passed,
0 failed, 0 skipped, exit 0, in about four minutes. Before the vendor
step it failed at patch.sh, which is what CI would have done.
- The four ALLOW_SKIP cases, in the build image with no firmware: none
set -> exit 1 naming both gaps; "pytest" only -> exit 1, replica still
NOT ACCEPTED; both named -> exit 0, both still printed; and under
GITHUB_ACTIONS the ::warning:: annotations appear.
- pytest 30 passed. Both workflows parse.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
e0dad3513c |
Rewrite the host half of the harness in Python, so a broken gate fails
The shell suite decided "this gate did not run" by grepping the child's
stdout:
if "$@" >"$out" 2>&1; then
if grep -qE '^[[:space:]]*SKIP:|[0-9]+ skipped' "$out"
then skip "$name"; else pass "$name"; fi
Text on stdout cannot tell you what happened inside a process. A launcher that
had ALREADY failed reached that same line by another road: the file that
sources config.env failed to load, so STOCK_TGZ_* was never set, so the
launcher concluded there was nothing to test, printed those five characters
and exited 0 -- on a machine with docker and the firmware sitting right there.
That is how five broken launchers shipped and stayed broken.
So the host half is Python and a skip is an exception. Output cannot imitate
one, and a gate that fails cannot claim it was skipped.
test/ffsim/ config loading, docker plumbing, gate reporting
test/run-tests.py the orchestrator
test/integration/ sim-install.py, sim-roundtrip.py, printer-exec.py,
extract-rootfs.py
test/integration/printer/*.sh and the whole payload stay shell. They are
executed by the printer's own busybox ash under qemu, and surviving that is a
large part of what the suite proves; rewriting them would test a Python the
printer does not have. build-printer-image.sh and make-stock-fixture.sh stay
too: both are `set -euo pipefail`, neither is a gate, and one is a release
tool run by hand.
Four things that used to report as skips are now failures -- a config.env that
does not parse, a missing case script, a missing package, a missing baseline.
A genuinely absent precondition is still a skip, because that is still true.
Three more bug classes are gone by construction rather than by being tested
for. The repo root is found by searching upward for bin/common.sh instead of
counting `..` from $0, so a launcher can move to any depth. Sourcing a config
with a syntax error is checked -- `.` returns non-zero and does NOT stop the
shell, so the old code carried on with half a file applied, which is exactly
the broken-looks-absent confusion again. And pytest counts come from the JUnit
XML rather than from grepping "N skipped".
test_script_roots.py becomes test_harness.py: the `..` check now has two
remaining shell targets rather than seven, and it gains a byte-compile of the
whole harness. That last one matters -- sim-install.py and friends are only
executed by `make test-install`, which needs docker and the firmware, so
without it a typo would wait quietly until release day. It also pins the
property that replaced the `..` counting, from four different depths.
VERIFIED, and the verification is the point, because this machine has no
docker daemon and these are exactly the scripts that cannot run here:
- Stubbed the docker binary and diffed the recorded `docker run` argv,
shell against Python, across five paths: install with a prebuilt image,
install with a local build, local build plus PROG_DUMP, roundtrip, and
printer-exec. IDENTICAL in all five.
- Ran the old orchestrator and the new one in the same tree: both give
6 passed, 1 failed, 2 skipped, exit 1, with the same gates in the same
order. (The failure is shellcheck, which lives in the build image.)
- pytest: 24 passed / 5 skipped bare, 29 passed against a real extracted
rootfs.
- Compared failure modes directly. The four cases above go from "exit 0,
reported skipped" to "exit 1, named". Case 2 is the sharpest: with a
missing case script the shell launcher exited 0 in silence, having cp'd
nothing and mounted a directory over /case.sh.
Still unproven here: the gates that need a live daemon. `make test-install`,
`test-mcu` and `test-recovery` want one run on a docker-capable machine. The
argv diff says they will be handed the same command as before; it cannot say
what docker does with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
43baf4054e |
Drop the unit lane; one suite, in test/integration
The split existed so a plain pull request had something to run. This repo has one maintainer who always has the firmware to hand, so it was a boundary kept in sync for a contributor who never arrives. One directory now. The tests moved rather than went away. test_chamber.py is the only check in the suite that can catch a printer heating something it has no element for, and the replica cannot cover it: case-install.sh touches .cfg files at exactly two lines, both a grep for a sentinel proving the user config survived, and it asserts klippy.py exists and c_helper.so has the right ABI without ever starting klippy. Declaring [heater_generic chamber_heater] on the plain Creator 5 fails four of these tests, including M191 waiting on a target the machine can never reach; that same package installs, boots, and passes every replica gate. make-stock-fixture.sh came too, since the packaging pipeline is built on it. Ordering changed, and that is the part to look at. With two lanes, pytest ran twice -- once per lane, each with its own -m expression or path. With one it runs once, so it has to run at the point where it can see the most: the stock package is now resolved and the rootfs extracted BEFORE pytest rather than after it. On a configured machine that is 28 tests in one pass; on a bare tree the five that read the rootfs skip. config.env is read in a subshell while doing that. It is the BUILD config, and it is now consulted upstream of the fixture build, where the entire point is that the fixture config wins -- sourcing it into this shell would put MOD_NAME and TARGET_MACHINE where that build could see them. Verified by pointing a config.env at MOD_NAME=LEAKED and confirming the fixture still builds Creator5Pro-anvil-ci.tgz. One thing gets worse, and it is inherent to having a single lane: on a tree with no rootfs, pytest reports 5 skips, so sub() marks the whole gate SKIP rather than ok, and the summary reads 6 passed instead of 7. Those 23 passes are real and the line above the SKIP says so. This is the existing policy working as designed -- a gate that did not fully run must not read as success -- but the summary is coarser than it was when a lane existed that never skipped. Verified: 28 passed with a real extracted rootfs, 23 passed / 5 skipped without; the full suite gives 7 passed / 1 failed / 1 skipped with a rootfs present and 6 / 1 / 2 without, the failure being shellcheck, which lives in the build image; no test/unit reference is left outside the passages that explain why it is gone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
2ae9736e97 |
Name the two lanes unit and integration, and fix what the move broke
test/repo -> test/unit, test/replica -> test/integration. The old names said
where the input comes from; these say what the tests are, which is the thing
you already have a word for.
The rename is also how the bug below surfaced, because "integration" is one
character-count away from nothing and running the launchers by hand is not.
Every launcher that moved out of test/ into a subdirectory opened with
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
and one `..` was correct at test/ and wrong a directory deeper. $ROOT became
test/, so the first thing each of them did -- source $ROOT/test/integration/
sim-image.sh -- went looking for test/test/integration/. Five scripts:
printer-exec.sh, sim-install.sh, sim-roundtrip.sh, build-printer-image.sh and
extract-rootfs.sh, which reached bin/common.sh the same way.
The previous commit shipped this. The path-existence sweep that was supposed
to catch it only checked paths written as literals; these are assembled at
runtime from $0. Nothing else caught it either, and that is the part worth
naming: the integration half cannot run without docker and the stock package,
so on any machine missing either -- this one, and a fork's CI without the
secrets -- five broken scripts still reported a clean SKIP.
So test/unit/test_script_roots.py now checks it statically: for every .sh
under test/, count the `..` after `dirname "$0"` and compare it against how
deep the file actually sits. It needs no docker, no firmware and no shell, so
it runs in the lane that always runs, which is the whole point -- the failure
it pins only appears in the lane that often does not.
Verified: the guard fails with "climbs 1, is 2 deep" against a reintroduced
bug and passes once it is reverted; all four launchers now reach their own
preconditions (docker daemon, config.env) instead of dying on a missing file;
21 in test/unit and 7 in test/integration against a real extracted rootfs;
run-tests.sh reports 7 passed / 1 failed / 1 skipped, the same as before the
rename (shellcheck lives in the build image, and no stock package is
configured in this worktree); no test/repo or test/replica reference is left
anywhere in the tree.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
8337359a68 |
Split test/ into the two lanes it already had
The suite has run in two halves for a long time, and the rest of the repo is
built around that: RUN vs RUNSIM in the Makefile, `test` vs `printer-sim` in
CI, two halves in run-tests.sh. Only test/ itself was flat, so which half a
file belonged to was something you learned by reading its header.
test/repo/ needs nothing but this checkout
test/replica/ needs the printer's real rootfs
test/ run-tests.sh, and the fixtures both lanes share
The axis is the precondition, not the runner. test/replica is everything that
cannot run without the proprietary package -- the qemu'd sim launchers and the
replica image, but also test_paths.py, which only reads the extracted rootfs
as files and needs no docker at all. Grouping by "does it use docker" would
have put that one in the wrong place; grouping by what has to exist first puts
every file where the question "can this run on a public checkout?" is answered
by the directory name.
The `rootfs` pytest marker goes with it. It existed to let run-tests.sh pass
`-m 'not rootfs'` in the PR lane and `-m rootfs` in the replica lane; the
directories say the same thing without anyone having to remember to tag a new
test, and a file in the wrong lane now shows up in `git status`. The two
lanes are `pytest ./test/repo` and `pytest ./test/replica`.
pytest.ini is new and is load-bearing: without a pinned rootdir, naming a lane
on the command line makes pytest take that lane as the root and never load
test/conftest.py, so the fixtures both lanes share would vanish depending on
which one you asked for.
Renames only, plus the path references: test/fixtures/make-stock-fixture.sh ->
test/repo/, the sim launchers and test/printer/ -> test/replica/, and
test_applets.py -> test/replica/test_paths.py, which is what it has actually
checked since the command-word half was dropped.
Verified: 19 in test/repo and 7 in test/replica against a real extracted
rootfs, the same 26 as before the split; run-tests.sh reports 7 passed /
1 failed / 1 skipped, identical to the pre-split run on this host (shellcheck
is absent here and lives in the build image, and no stock package is
configured in this worktree); every referenced path in the repo resolves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
91604c2dfb |
Cut the suite to four gates, and rewrite the Python half as pytest
Twelve gates, most of them the weaker copy of a gate that survives. What is
left: pytest for the Klipper config, and three replica gates -- the end-to-end
install, MCU bring-up, and stock recovery.
Fix first, because the cut depends on it: a skip counted as a pass. Every
launcher exits 0 when it decides it cannot run, and sub() reported that as
`ok`. That is how test-abi.sh sat in the suite printing green while checking
nothing -- run-tests.sh deleted work/modpayload immediately before it ran and
CI set KLIPPER_FORK="", so it had no targets on any run, ever. Skips are now
counted apart, the summary says so, and the suite exits non-zero unless
ALLOW_SKIP=1 says the gap is deliberate. With four gates instead of twelve, a
gate that silently does not run is the failure mode that matters most.
Dropped, each because something else already does the job properly:
test-abi.sh bin/patch.sh refuses to build a non-nan2008
c_helper.so, bin/verify.sh re-checks the packaged
one, and case-install.sh reads the ELF header of
the file that actually landed on the machine.
test-ash-conformance.sh case-install.sh runs `sh -n` over every installed
script with the same qemu'd busybox and then greps
the boot log for "Syntax error".
test-model-gate.sh verify.sh SS8b/SS9 checks the same invariant and
pack.sh exits on MODEL MISMATCH before a package
exists. Its header claimed it proved the two models
ship different files; no such check was in it.
test-macros.py It used jinja2.Environment() -- default '{{ }}'.
The configs have no '{{' at all, only single-brace
expressions, so it checked '{% %}' block structure
and never looked inside an expression. Folded into
the chamber tests, which had the delimiters right.
test-base-cfg.py It read work/software/klipper/config/printer.base.cfg
as its stock reference -- the file bin/patch.sh
overwrites with ours. It was diffing our file
against itself: green on a cold tree, red on a
second run, blind to real drift either way. The
comparison moves to bin/unpack.sh, where a pristine
tree exists, as a warning rather than a gate.
lint-danger.sh Its structural greps were satisfiable by comments:
firmwareExe:44 and S80ui:16 *mention* helix and
SAFE-MODE, so they passed with the logic deleted.
case-ui.sh Goes with the helix-only decision.
applets.allow Ten entries, none of which the extractor had asked
about in a long time.
The command-word half of test-applets went too. It extracted only the first
word of a simple unprefixed command, so `if timeout 5 foo; then` yielded
['if','echo','fi'] -- the one failure its docstring named was invisible to it.
The absolute-path half survives: S50wifi's binaries sit behind a
[ -d /sys/class/net/wlan0 ] guard no simulation ever reaches, so nothing else
checks they exist.
The Python half is pytest now. Real assertions instead of hand-rolled
PASS/FAIL counters, underscored filenames so the modules import normally
(ffcfg keeps only the parser; the dash in test-macros.py was why one test used
to reach the other's parser through importlib), and a `rootfs` marker that
splits the tests needing the proprietary package from those that do not --
deselected in the PR lane, run in the replica half, so neither lane reports
skips it did not expect. test_default_delimiters_are_blind pins the jinja2 bug
so it cannot come back.
The config gate also moves out of the replica half into the half that runs on
a plain pull request. It needs python3, jinja2 and files in the repo, and it
is the only safety-relevant check in the suite -- a chamber heater declared on
a Creator 5 that has no element for it -- so it had no business being gated
behind the proprietary firmware.
Verified: pytest 21 passed / 5 rootfs-marked; the fixture pipeline
(unpack, patch, pack, verify) passes end to end; injecting a broken
single-brace expression fails test_every_macro_parses with the exact line, and
parses clean under the old defaults; the skip path reports SKIP and exits 1,
and ALLOW_SKIP=1 accepts it.
Also drops the CI `abi` job, which ran on a bare checkout with no binaries and
so was permanently green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
71cea577c5 |
Drop test-printer-db and test-env
test-printer-db.py is gone entirely. It started as a re-implementation of PrinterDetector's scoring formula and was later narrowed to invariants that do not model anyone's arithmetic; neither version earned the place it took up. The Makefile target, the suite entry and the docs note go with it. test-env.sh was a file whose whole job was to load two other files. Its contents move into sim-image.sh, which every replica launcher already sources or is one exec away from, so the config and the docker plumbing now arrive together instead of as two sources the caller had to remember in order. sim-install.sh and sim-roundtrip.sh source sim-image.sh directly now, which also means a missing docker is reported before a package is staged rather than after. The environment still wins over both config files -- verified in both directions, an explicit PRINTER_IMAGE beats test.env and an unset one falls through to it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
35aa61fccb |
Keep the Orca profile stock; prepare from the file by default
The decision is that a FlashForge Orca profile is not edited at all: no START_PRINT pasted into Machine start G-code, and files sliced before the mod keep printing. So FF_BEFORE_PRINT_START.prepare is now 1, and [ff_print] runs the whole of START_PRINT before the file's first line. The stock block needs exactly that. It prepares nothing: M140 and M104 without waiting, fans, M191, then `G1 Z5 F2400` as its first motion on a machine nothing has homed, then T<n>, M109 and the prime line. No G28, so nothing homes; no M190, so nothing waits for the bed. The touchscreen app did both before handing the file over. Its end block is one move -- no heaters off, no M84, no park -- so FF_AFTER_PRINT_END is what ends the job. assets/orca/*.txt now hold the stock blocks themselves, as the reference for what the mod expects rather than something to paste. The previous contents were a modified start block carrying a START_PRINT call, which is no longer what a user should have. docs/toolchange.md said to paste both files into the profile; it now says to leave the profile alone, and documents prepare=0 plus an explicit START_PRINT as the way to drive the sequence from the slicer instead. Its TOOLS=/TEMPS= paragraph is updated for the paired TOOLS=0:220,2:240 form and for clean temperatures now coming from _FF_FILAMENT.tool_material. The two prepare=0 cases stay documented on the variable: a profile that calls START_PRINT itself, and one that homes for itself -- a generic Orca block runs G28 after M140/M104, and the G28 wrapper docks a mounted tool before homing Z. Untested on hardware. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr Shyshatskyi
|
12bbd1e7dc |
Retire the probe profile: there is one package now
The probe profile was a bring-up aid. It set BUILD_APPLY=0, reinstalled the stock component byte-for-byte and wrote a report to the stick, so you could prove the update chain worked on your own machine -- the filename glob, the DES3 key, the md5 gate, run.sh execution -- before flashing anything that changed a file. That rehearsal has served its purpose. With it gone there was exactly one profile left, so the whole PROFILE indirection goes too: profiles/ is deleted and its six BUILD_* flags are plain defaults in bin/common.sh, which also means config.env now overrides them instead of being silently overridden. make default folds into make build; all-profiles and RECOVERY_PROFILE are gone with it. What is NOT gone is the undo button: flashing the stock FlashForge package still restores every file the mod touches, and case-recovery.sh still proves it. What is gone is the dry run. Note the loss: PROG_MB/DATA_MB used to come out of the probe report. The docs now point at df -h over ssh, which you only have after a successful flash. |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
5bdf78109a |
Retire firmwareExe: own the web stack, and say what goes with it
The mod replaces FlashForge's application outright, so the reasoning that still framed the web stack as coexisting with it was wrong on its own terms. * nginx and moonraker are documented as the only web stack on the printer. MOD_WEB=0 is a debugging switch that leaves ssh and the touchscreen, not a way back to stock -- that is a stock FlashForge package. The own-prefix rule stays, for the reason that is actually true: /usr/prog is the firmware partition and every FlashForge package reinstalls it, so a config written there is undone by the next stock flash. * Configs the mod ships now update when you have not edited them. A live file still byte-identical to the one the last package wrote is replaced; one you changed is kept and the new version lands beside it as .mod-new. $MODDIR/config-installed holds that last-written copy. Without this a config we own could only ever be written once: it exists on the second flash, so it would land as .mod-new forever and updates would never reach the printer. * Dropped the three dead /server/files/ alias blocks from nginx.conf. The regex location above them matches /server/ too and nginx gives regex locations precedence, so they never fired -- and had they fired, serving those paths from disk would 405 the DELETE and POST that Mainsail sends to the very same URLs. * Recorded what dies with firmwareExe, in docs/notes/25 and in the README: the :8898 REST API (FlashPrint, Orca's FlashForge profile, the mobile app) and the :8080 camera stream. The camera is the one that looks restorable and is not -- the MJPEG server is compiled into firmwareExe (its own accept loop, `GET /?action=stream`, boundarydonotcross) rather than being mjpg-streamer, so there is no daemon left to point a [webcam] at. assets/moonraker.conf carries the block, commented, for when a streamer is built for this board. Also in this change: the MOD_UI switch and the stock-UI path are gone, the chamber-heater model gate arrives with ff-model.cfg/ff-chamber.cfg, and the tests follow. make test: 32 passed, 1 failed. The failure is the probe end-to-end, which dies at "mount: failed to setup loop device for /stick.img" -- leaked loop devices on this build host, not the package. Probe installs nothing, and the default profile's install -> re-install -> boot passes in the same run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
22d16f984e |
Run the printer's own boot script for the update test, and cut the suite to 5m
The install test used to re-implement app_startup.sh: it pulled the glob,
MACHINE and PID out of it with sed, then called the installer itself. A
mistake in our reading of the boot script could never be caught that way.
The package now sits on a real FAT filesystem on a loop device exposed as
/dev/sda1, and app_startup.sh runs verbatim through three boots -- stick in,
stick still in, stick pulled. The last one asserts what actually matters:
nginx and moonraker up, klipper launched, the UI in the foreground where
app_startup.sh's own 5-second ps watchdog can see it.
One substitution, declared in .SIMULATED: the printer's kernel has nls_cp936
and a container kernel does not, so /bin/mount retries without codepage= if
the real options are rejected.
Speed. The image carried the partitions but not the stock install, and
test-env.sh sourced the config files after the environment -- so PRINTER_IMAGE
on the command line was overwritten by the empty value in test.env and the
published image was never used. Measured, per replica case: 22s to unpack the
factory image, 37s to install the stock package, 0.7s with the image. The
stock install is now baked in at image build time (bake.sh -- it needs binfmt
and chroot, so it runs privileged and the result is committed) and recorded in
/usr/prog/.BASELINE; a run asking for a different package reinstalls.
make test: ~20m -> 5m26s. Every section header now carries elapsed seconds.
Dropped four checks that could not fail:
* test-abi's execution half was `qemu "$f" -h || [ $? -lt 126 ]`, which
accepts nearly every exit status
* test-ash's md5sum -s probe printed "inconclusive" on every path
* run-tests.sh's per-file `sh -n` "POSIX" loop ran bash, and its bashism
grep flagged `local`, which busybox ash supports
* CI's shellcheck ended in `|| true`
And one that mattered more: the release workflow's final sim-install loop
ignored every exit status, and sim-install.sh exits 0 on a skip -- so the last
gate before publishing could not fail. It now stops on the first failure and
runs with REQUIRE_PRINTER_SIM=1, which turns skips into failures throughout.
Also: the profile lists in run-tests.sh, RECOVERY_PROFILE and release.yml
still said ssh/web/full/helix and would have failed the release job; work/.sim
was shared, so two suites at once truncated each other's packages; and the
fixture half left a 380KB package in work/out where make test-install picks it
up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GgNhSa1Qvwvxb2nQXT6b1Y
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
a5300bde12 |
Split the repo into ship / build / test lanes, and add the printer replica
Two strands, both landing here because they had grown into each other in the
working tree.
The printer replica: the printer's real rootfs.squashfs chrooted under
qemu-mipsel, with /usr/prog and /usr/data from the factory image, so a package
is installed by FlashForge's own updater running on the printer's own busybox,
tar, md5sum and unTar. There are no stubs -- if the real prog partition is
missing the harness refuses to run rather than test against mocks. A release
workflow is blocked on it.
The lanes: a file's directory now says whether it can ever reach a printer.
payload/, assets/ ship
bin/, profiles/ build the package
test/, docs/ neither, ever
* the replica's own infrastructure moved out of bin/ and docker/ and in
beside the harness it belongs to (test/extract-rootfs.sh,
test/build-printer-image.sh, test/printer/Dockerfile{,.full})
* the Makefile has two runners: only RUNSIM gets the docker socket and the
replica's variables, so a build cannot reach the docker daemon at all
* replica settings moved to test.env, which nothing in a package reads
* build-time flags renamed BUILD_* (APPLY, REPORT, KLIPPER, TOOLCHANGE,
MAINSAIL, HELIX). MOD_* now means only the runtime switches written into
anvil.conf and re-read by the printer at every boot
* verify.sh fails if any file in a package is byte-identical to one in
bin/, test/, docker/ or profiles/ -- by content, because a name blacklist
would trip over whatever Mainsail happens to call common.sh
Fixed while mapping it:
* BUILD_REPORT was read only inside the BUILD_APPLY=0 branch, so four
profiles declared it and got nothing. It is honoured on both paths now,
and off outside probe: report.sh copies /etc, passwd and shadow onto the
USB stick, which is a diagnostic, not something to hand out.
* three checks in the brick lint had been quietly dead. Two were guarded on
a payload/boot.sh replaced by the firmwareExe wrapper long ago; the third
was the file glob itself, which matched neither firmwareExe nor the
init.d/S* scripts -- the UI wrapper and every service went unscanned. A
missing file is now a failure rather than a silent pass.
* run-tests.sh overwrote config.env and copied it back, leaving the config
you edited one crashed run away from being a fixture. It uses CONFIG_ENV.
* bin/make-uninstall.sh does not exist and never did; README, run-pre.sh and
full.env all pointed at it. Recovery is the stock package, which is what
test-recovery actually proves.
README is now for someone who wants to flash their printer: what it is, which
file, how to flash, what to do when it goes wrong. The rest moved to
docs/how-it-works.md, docs/building.md and docs/testing.md.
make test: 89 passed, 0 failed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|