firmware/docs/qa-migration.md

The qa suite, and what it replaces

qa/ is the test suite. It was built beside test/ rather than inside it, and the migration is now done: every case-*.sh has been ported or retired, and test/ has since been deleted outright. The machinery that builds the replica moved to tools/replica/, because it was never a test.

This document says why it exists, what it proves, and what is still owed.

The problem it is answering

Three complaints, one cause.

Two frameworks. test/run-tests.py and test/ffsim/ are 1,657 lines that reimplement what pytest already does — skip accounting, gating, reporting, and then parsing pytest's own JUnit XML back out — and they invoke pytest as gate #5 of about twenty.

Assertions in the wrong language. test/integration/printer/case-*.sh was 6,280 lines of POSIX sh executing inside a qemu chroot, and between them they reported 13 boolean bits. case-moonraker.sh was 1,059 lines that collapsed to one pass/fail. A failure tells you a case went wrong, not which of its forty assertions did.

No parallelism, no selection. Each replica gate is its own docker run --privileged, re-registering binfmt and re-assembling the mount layout. Thirteen of those, strictly in sequence, all or nothing.

All three follow from one decision: the replica container is treated as a test runner instead of as a fixture. That is what pushes assertions into ash, which yields one bit per container, which then needs a bespoke framework above pytest to aggregate the bits.

Two rules

1. Shell inside the chroot performs actions. Python on the host makes assertions.

Shell in qemu is irreplaceable for doing things — the printer's busybox, its tar, its OpenSSL, and the fact that our scripts survive its ash is a large part of what the suite proves. It is a bad place to judge things. Where a fact is only observable inside the replica, qa/lib/replica.py exposes a typed probe for it, and a case script never branches on it.

qa/replica/actions/ is what remains of the shell — three files that drive the machine and assert nothing. No ok/bad, no counter, no verdict.

2. The machine under test is one the real installer produced.

The printer fixture is a replica with the package installed by the printer's own /usr/prog/app_startup.sh, off a genuine FAT filesystem on /dev/sda1, exactly as a user installs it from a USB stick.

An earlier version of this tree had install-payload.sh, which copied payload/*.sh and pkgs/anvil-core/payload/init.d/S* into $MODDIR by hand — the same shape case-services.sh used. Two things are wrong with it:

• It is a second implementation of the install, so everything downstream asserts against a layout the harness built rather than one the installer produced. The real installer could break and nothing here would go red.
• It can only place what a recipe keeps in the repo, so anything the build produces is missing. The cross-compiled s6 is the obvious one: it lives in work/.s6 and bin/payload.sh stages it into the package, so a hand-placed payload has no supervisor at all — which is why the tests had to carry a stand-in scanner, and why one of them had to ask whether it was testing the stand-in before it could mean anything.

Installing for real fixes both at once, and deletes the stand-in. Compare what lands in /usr/data/anvil:

hand-placed	really installed
anvil-env.sh anvil-service.sh anvil.conf init.d etc/s6	VERSION anvil-env.sh anvil-service.sh anvil.conf bin config etc helixscreen init.d lib libexec moonraker nginx www

bin, lib and libexec are the cross-built s6 and CPython 3.13. No amount of copying a recipe's files produces them.

This repo has learned this once already, in case-install.sh's own header:

An earlier version of this file replayed app_startup.sh by hand — which meant a bug in our reading of it could never be caught.

What the machine is

Unchanged. qa/ reuses test/integration/printer/'s Dockerfile, entrypoint.sh, assemble.sh, binfmt.sh and seed-prog.sh as they are, and the docker run argv is deliberately the same one test/ffsim/replica.py builds, minus --rm and plus -d. Same image, same --privileged, same read-only squashfs root and writable prog/data partitions, same chroot onto the printer's own busybox under qemu.

This matters more than it looks: while both suites run, they have to be testing the same machine, or neither proves anything about the other.

The only new trick is how the container is held open. entrypoint.sh ends by running the case script it was given; the one qa/ gives it is qa/replica/actions/hold.sh, which touches a marker and then sleeps. Setup runs to completion exactly as it always has, and the container simply does not exit. The marker is also the readiness signal, because setup ranges from under a second (a prebuilt PRINTER_IMAGE) to over a minute (unpacking the factory image and installing the stock package under qemu) and a constant would either waste the fast case or fail the slow one.

Lanes

Lane	Needs	Marker
static	nothing but the checkout	@pytest.mark.static
replica	docker + qemu + the firmware	@pytest.mark.replica

make qa            # both
make qa-static     # seconds, any machine, any clone
make qa-replica    # the gates that decide whether a package bricks

The replica lane needs two things.

A base replica. The image, which carries the firmware, /usr/prog and /usr/data with the stock package already installed — put it in test.env (see test.env.example) or pass it per-run:

PRINTER_IMAGE=monstrofil/creator5-printer:1.9.7-1.2.9-20260810 make qa-replica

There is no second source. The lane used to build a replica locally from work/rootfs when PRINTER_IMAGE was unset, which needed make rootfs and the stock package and cost about a minute of setup per case; that path is gone. make printer-image builds the image from the same public firmware.

A built package, in work/out/*.tgz — because the lane installs it for real. make build produces one; the newest by mtime is the one used, and the run says which. It is never built on demand: make build cross-compiles s6 and CPython and takes minutes, and a fixture that silently started that would turn "the suite is slow today" into a mystery.

The installed image

The install is the machine's own shell under qemu, so it is baked into an image once per package and cached under a tag derived from the package's md5 — the same trick build-printer-image.sh uses for the stock baseline:

qa: baking creator5-printer-anvil:ce5f8271ccca from monstrofil/creator5-printer:latest
    (Creator5Pro-anvil-20260827b.tgz) -- once per package, then cached
install: booting /usr/prog/app_startup.sh with the package on /dev/sda1
install: the machine's own updater installed the package
qa-bake: mod installed, VERSION anvil-env.sh anvil-service.sh ... lib libexec moonraker nginx www

Rebuild the package and the md5 changes, so the next run bakes again rather than testing yesterday's build. That is deliberately not a hand-written staleness check, because those are the ones that eventually get it wrong.

qa/replica/actions/bake.sh refuses to run with USB_STICK unset, and fails if /parts/data/anvil is absent afterwards — a bake whose chroot unmounted early would otherwise commit a stock machine under a name saying it has the mod, and every test built on it would assert against firmware it was not told it had.

Selection is pytest's: -k S62moonraker, -m static, or a single test id.

Parallel runs must use --dist loadscope:

python3 -m pytest ./qa -n 4 --dist loadscope

The printer fixture is module-scoped and xdist's default --dist load hands out individual tests, so four workers pulling from one module would assemble four replicas of the same machine. qa/conftest.py refuses the default rather than letting the suite get mysteriously slower when it is parallelised.

Skips

A missing tool, daemon or firmware image is a failure, raised at the point that needs it, with the fix in the message. There is no flag, no environment variable and nothing to remember to pass.

This replaces REQUIRE_PRINTER_SIM and ALLOW_SKIP, and it preserves the discipline they existed for — a gate that did not run must never look green — by removing the mode in which it can. There is no configuration in which running this suite without shellcheck, or without a replica, is a complete run, so nothing needs to be told that.

An earlier version of this tree had a --strict-skip=<lane|all> flag. It was the wrong shape and it lasted one commit. It was ALLOW_SKIP inverted, and ALLOW_SKIP's defect was never the direction it named things in — it was that the decision lived in the CI invocation. An accept-list has to be edited whenever a gate moves; a strict-list has to be remembered whenever a job is added. Both degrade silently when the command line is wrong, which is the same disease as the printer-sim job that was gated on an unset secret and therefore never ran once in its entire existence.

pytest.skip stays available for what it is actually for: a question that does not apply to this configuration, as opposed to one this machine happens to be unequipped to ask. A test that only means something on a Creator5Pro would earn it. "I could not find the tool" does not.

There are three skips, and they do not meet that bar. qa/static/test_packages.py reads work/modpayload-root, which only exists after bin/payload.sh has run, and skips when it is absent. That is "this machine is not set up", which the rule above says must fail -- but making it fail would put a stock FlashForge package in the way of a lane whose whole point is that it needs nothing.

So make qa-static reads 246 passed on a tree that has been built and 243 passed, 3 skipped on one that has not, with nothing at the end saying which kind of run you just had. Either the payload questions move to a lane that can require a build, or they fail with make build in the message the way the replica lane does.

What is proven so far

The migration is done apart from one gate. Every case-*.sh is either ported or retired, and test/run-tests.py no longer drives a single replica case.

Lane	File	Tests	From
static	test_shell_syntax.py	7, parametrised over every script	run-tests.py's check_shell_syntax, check_no_bashisms, check_undefined_names -- now deleted
static	test_packages.py	32	new -- the packages, the layout and the ABI string
static	test_recipe_layout.py	9	new -- a recipe's fixed shape
static	test_probes.py	17	new -- the harness's own parsers
replica	test_install.py	22	case-install.sh, 579 lines and 1 bit
replica	test_supervisor.py	31	case-supervisor.sh, 166 lines
replica	test_upgrade.py	21	case-upgrade.sh, 223 lines
replica	test_boot_screen.py	17	case-boot-screen.sh, 210 lines
replica	test_s6rc.py	12	new -- what the printer does
replica	test_mcu_bringup.py	8	case-mcu-bringup.sh, 129 lines
replica	test_web.py	7	case-nginx.sh + case-moonraker313-s6.sh, 1,124 lines and 2 bits

Measured on monstrofil/creator5-printer:latest, off a real package built from this tree:

qa/static	246 tests, 8s, no docker
qa/replica	118 tests, 3m40s -- 110 pass, 8 blocked on absent hardware
test/ pytest	28 tests, 0.1s

test/run-tests.py is 322 lines and test/ffsim/ 483, from 1,657 between them. What is left of the old harness runs the packaging build on a synthetic stock package and calls pytest; it holds no replica gate at all.

Three ports found assertions that had stopped meaning anything

Worth recording, because each had been green for months while asserting nothing:

• case-mcu-bringup.sh's negative control had inverted. It asserted the bring-up FAILS without LD_LIBRARY_PATH. Our CPython is built --disable-shared now and needs no library path, so the case's own skip branch was the one firing. The port asserts the direction that is true today -- it runs under env -i -- which still goes red if a future build reintroduces the dependency.
• case-upgrade.sh's headline assertion was vacuous. It proved an update removes init.d/S60web; the installer did rm -rf $MODDIR/init.d unconditionally, so it would have passed on an installer whose entire deletion logic had been removed. The port makes that claim against a path the deletion actually governs, and gives the unconditional removal a test of its own.
• case-boot-screen.sh confirmed its own arithmetic. It computed the expected pixel packing by calling ffscreen._pixel inside the replica, so a swapped R/B would have matched itself perfectly. The port computes the expected bytes on the host.

Two rakes in the replica, recorded so nobody steps on them twice

• The install log double-counts. FlashForge's run.sh runs under set -x and run-append.sh is spliced into it, so every echo appears twice -- once as the + echo ... trace and once as output. Counting a string in /usr/data/anvil-install.log says two on one occurrence.
• /usr/bin/od takes neither -A nor -t, and busybox od resolves to the same binary, so the usual od -An -tx1 returns a usage error -- which, parsed as bytes, is an empty list rather than a loud failure. xxd -p is present and does the job.

The replica cannot boot the whole tree, and that is hardware

Eight of test_s6rc.py's twelve need the WHOLE boot set and cannot pass on a replica. Measured, in this order:

• there are no /dev/ttyS4,5,7, so mcu-bringup fails and klippy never gets past disconnected;
• there is no /dev/video*, so camera hits its 40s timeout-up and s6-rc reports command exited 99;
• ff-startup then waits out its own timeout-up of 300000ms for klipper -- holding the s6-rc lock, which is why a poll during that window gets unable to take locks: Device or resource busy rather than a short list of what did come up.

nginx and moonraker do come up, and moonraker reaches ready -- which is what test_web.py asserts against, by bringing up those two alone before anything takes the lock. So the blocker is not the boot graph; it is that two services need hardware the replica has not got. Closing it is the simulated-MCU lane in Not in scope below, not a longer deadline in the fixture.

The four that ask only about what was installed pass, and one of them was failing for a reason worth recording: s6-rc-init's default deadline is TAIN_INFINITE_RELATIVE, which does not fit this printer's 32-bit time_t, so it died with EOVERFLOW before reaching its subject. Every s6-rc-init in the suite now passes -t, the same fix firmwareExe carries.

test_probes.py has no counterpart in the old suite and is worth explaining: Printer.ps() and Printer.listening() are parsers, and a parser is where a harness goes quietly wrong. A listening() that never matches makes every readiness assertion pass by the absence of contradiction. Neither failure looks like a failure — they look like a green suite. So they are tested against captured output, in the static lane, with no docker involved.

What phase 8 did to both suites

docs/notes/80-s6-migration.md phase 8 deleted payload/init.d/ and payload/anvil-service.sh and made the boot a compiled s6-rc database. That is not a refactor the suites can be patched through: a large part of what they assert no longer exists to be asserted.

Retired. qa/replica/test_services.py is deleted. It was the port of case-services.sh and its subject was the wrapper contract -- seven S* scripts, each loading a shared library, answering status, naming itself, and rejecting an unknown verb. There are no wrapper scripts. Three of its assertions survived the move and live in qa/replica/test_s6rc.py: the scanner is the cross-built ELF we shipped, its log is empty, and every run in the scandir is executable. The rest describe a machine that cannot be built any more.

Replaced by qa/replica/test_s6rc.py, which asks what the printer DOES -- init, one transition, a killed daemon coming back.

There was a static half as well, test_s6rc_source.py: it compiled the source tree and read the graph, bundle, timeouts and generated shebang back out with s6-rc-db, never out of the source tree, because the compiler is free to reject, rewrite or silently drop what it was given. (down is the one that bit us: s6-rc-compile accepts a down file in a definition directory and discards it, producing a byte-identical servicedir.)

It is gone, and what it cost is worth stating. It needed an s6-rc-compile that runs on the build host, and the only way to get one was a second NATIVE build of skalibs, execline, s6 and s6-rc -- 110 lines of bin/payload.sh whose entire hazard was that both stacks had to be --prefix=$MODDIR or the #! baked into the oneshot runner pointed at the build host's execline. The database is compiled in the replica now, by the s6-rc-compile the payload ships, so that hazard cannot exist. The price is that a boot-order mistake is caught by the replica lane rather than in a second, and that the timeout and edge assertions are not currently made anywhere. The oneshot runner's shebang was the one part a host could still check and bin/verify.sh kept it; that file is gone too (see bin/verify.sh is retired below), and qa/replica/test_s6rc.py covers the same ground by booting the database rather than reading it.

The gates phase 8 killed have been retired

These case-*.sh asserted deleted architecture -- they drove $MODDIR/init.d/S* or sourced anvil-service.sh, neither of which exists. The decision was made against real output, not against a reading of the scripts:

make test-services  ->  FAIL  the payload ships no anvil-service.sh -- every service aborts
make test-nginx     ->  FAIL  the payload has no etc/s6/nginx/ -- nothing to supervise
                        cp: can't stat '/tmp/payload/anvil-service.sh'

retired	lines	asserted	where that lives now
case-services.sh	360	the seven-script wrapper contract, force-start	nowhere -- the contract is deleted
case-nginx.sh	416	the MOD_S6=0 unsupervised fallback	nowhere -- the fallback and the flag are deleted
case-moonraker.sh	1,059	the same fallback, plus S62moonraker	nowhere
case-moonraker313-s6.sh	708	Moonraker on 3.13 started by S62moonraker	needs a port once the replica can boot
case-camera.sh	467	S40s6 and S65camera verbs	the readiness question survives; see below
case-priority.sh	108	svc_start_daemon niceness	unasserted; the surviving path is -N inside etc/s6-rc/source/klipper/run

Retired in the same pass, for a different reason -- no gate, no Makefile target, no CI step, never run once: case-pyext.sh (567) and case-moonraker313.sh (487). Both were phase-6 spikes whose questions are settled. A file that looks like coverage and is not is worse than no file.

With them went gates.py's services, camera, nginx, moonraker and moonraker313_s6 (and _prefix_tarball / _moonraker_tarball, which nothing else used), their run-tests.py calls, six make test-* targets, and sim-moonraker313.py. About 4,200 lines of shell; the old suite's 13 replica bits are 8.

Two of the three assertions they owned were carried across, into qa/replica/test_web.py: nginx coming back from a kill and staying stopped after a deliberate stop, and Moonraker answering on :7125 on the cross-built 3.13 rather than FlashForge's 3.8.2. Seven named tests where case-nginx.sh and case-moonraker313-s6.sh were 1,124 lines and two bits.

That module brings up only nginx and moonraker rather than running the whole boot, for the reason in the next section, and it says so in its header. Bringing up a subset is a departure from run the real boot, do not reimplement it; it goes through the same compiled database and the same s6-rc, so only the transition's argument differs, and the alternative was asserting nothing about supervision until somebody builds the MCU lane.

The third -- s6-svwait -U readiness gating rather than meaning "forked", which case-camera.sh asked -- is still owed, and cannot be asked here: the camera service needs a /dev/video* the replica has not got.

case-install.sh was stale, not dead -- and it was the one that mattered

It was the last gate to go red, and the sweep above missed it because its stale assertions name the wrapper scripts WITHOUT the init.d/ prefix, so the grep that classified every other case did not catch them. Its subject -- the printer's own app_startup.sh installing a real package off a real FAT filesystem -- was entirely alive, so it was ported, not retired:

was	now, in qa/replica/test_install.py
grep -q helix on firmwareExe -- the wrapper launched the UI itself	the wrapper starts s6-svscan and asks s6-rc for a transition; the UI is a service in the compiled database
[ -d $MODDIR/init.d ], else "no service dir exists to start Klipper"	s6-rc-db list services names klipper, and s6-rc-db dependencies klipper names mcu-bringup
S60nginx S62moonraker S70klipper S80ui each appear in the boot log	not replaced by a grep for the new names -- see below

That last row is the one to be careful about. The boot log line reads s6-rc: bringing up: camera klipper moonraker nginx ui ff-startup wifi and names every service in the boot set INCLUDING THE ONES THAT THEN FAIL, so grepping it for a service name yields an assertion that passes on a machine where that service never started -- green, and unable to go red. The compiled database is asked instead, which is a fact about what was installed rather than an intention.

Most of the 579 lines were DRIVING the install -- attach the stick, run app_startup.sh, wait, repeat. None of that survives, because the printer fixture already is the result. What is left is the part that was always the point: the assertions.

Two of its three boots could not come across. The re-install (boot 2) needs a second package on a stick and the bake deletes /stick.img once the install lands -- but the property it existed for, an update keeps what the user edited and drops only what the last package shipped, is asserted directly in test_upgrade.py. Boot 3's UI liveness needs ok-all, which is unreachable on a replica; its no-stick half -- app_startup.sh must not go looking for an update that is not there -- did come across, since the baked machine IS the stick-pulled case.

What was stale in the pytest half

test_startup.py built its argparse.Namespace by hand and had not been taught --only-bringup / --no-bringup, so 43 of its tests failed with AttributeError inside run() -- the legacy suite was already red on this branch. Its FakeKlipper also still modelled start.sh plus a separate stop; klipper is restarted with one s6-svc -wr -t on its live servicedir now, so the fake models that instead and FF_SKIP_MCU_BRINGUP -- recorded but never asserted -- is gone. test/ pytest is green again: 185 passed, 1 skipped.

run-tests.py's shell globs had rotted the other way, and this one had teeth: pkgs/*/prog/*.sh and pkgs/*/prog/firmwareExe matched nothing, because prog/ moved under payload/. firmwareExe -- the boot -- and start.sh were being syntax-checked and bashism-checked by nothing at all. check_shell_syntax guards only the union of its patterns, so it stayed green while covering less than it claimed. qa/static/test_shell_syntax.py had the paths right already and refuses any glob that matches nothing, which is the discipline that would have caught it.

What is left

Nothing. test/run-tests.py and test/ffsim/report.py are deleted, and with them the JUnit-XML round trip that turned 185 pytest results into one gate. make test is gone; make qa is the suite.

Three gates were dropped rather than ported, as a coverage decision:

dropped	was	consequence
the pytest gate	run-tests.py ran pytest and re-parsed its XML	none at the time -- make test-py and CI ran pytest directly; both went when test/ was deleted
the rootfs extraction	ran unpack.sh + extract_rootfs inline	none, and now moot -- extract_rootfs and make rootfs are both gone, and the replica image does its own extraction at build time
the packaging build on a synthetic stock package	make-stock-fixture.sh, then unpack/patch/pack/verify.sh	real: bin/unpack.sh, payload.sh and pack.sh now have no test at all. verify.sh has since been retired outright -- see below

That last row is the one to be uneasy about, and it is recorded here rather than buried in a commit message. The build path is what produces the .tgz a user flashes, and nothing exercises it end to end any more. qa/static/test_packages.py covers package building -- the .apk layout, the ABI string, reproducibility -- but nothing runs the outer build. The CI job with no firmware has lost most of its content as a result.

test/integration/make-stock-fixture.sh was kept for a while as the input a replacement would need -- a synthetic FlashForge package built from the versions.env pins, so the build could be tested with no proprietary firmware -- and has since been dropped: it fed a gate that no longer exists, and a fixture generator with no consumer rots exactly as silently as a test that asserts nothing. Whoever ports the build path will have to write it again, and should know two things it had learnt. make_fixture staged into work/.fixture INSIDE the repo on purpose: sibling containers resolve their mounts host-side, so a path under the build container's /tmp does not exist as far as the daemon is concerned. And the shape it reproduced -- the software component's run.sh ending in tar -xf $WORK_DIR/klipper/chelper.tar -- is what bin/payload.sh depended on while the klippy tree still went out on the firmware partition; the tree lives at $MODDIR/klipper now and neither that section nor the chelper.tar exists. Recover both from git history rather than from a fresh reading of a stock package.

bin/verify.sh is retired

Forty checks, run on the build host against a decrypted copy of the .tgz. Four survive, in qa/replica/test_what_ships.py. The rest went because of what the word simulate was doing in the file's own description -- "simulate every check the printer performs".

The replica lane does not simulate the install. qa/replica/actions/install-package.sh puts the package on a genuine FAT filesystem at /dev/sda1 and runs the machine's own /usr/prog/app_startup.sh over it, verbatim, under qemu. So the decrypt, the plain-tar components, md5sum.list, the MACHINE= gate and the /mnt/<Model>-*.tgz glob are all preconditions of the lane now rather than assertions in it: break any of them and the install fails, installed_image() raises with the install log attached, and every replica test goes red at once. A host-side re-reading of the same rules is a second implementation, and a second implementation can agree with itself while disagreeing with the printer -- the lesson qa/replica/conftest.py records this repo learning twice about hand-placed payloads.

A second block was asking what the suite already answers, and answers of an installed filesystem rather than of a tar -t listing: the klippy tree and c_helper.so (test_install.py, test_abi.py), the compiled s6-rc database and the oneshot runner's execline (test_s6rc.py), s6 and s6-ftrigrd (test_supervisor.py), libsodium's bare .so symlink and the chamber configs (static/test_packages.py), and every shipped script's syntax (static/test_shell_syntax.py).

The four that moved are the ones nothing else asks, and each got stronger for the move -- a listing cannot tell you that a shared object loads, or what a name resolves to on a PATH:

• the shipped interpreter imports sqlite3, lmdb and _cffi_backend, rather than a grep finding three .so files. This is the check that catches a cross build resolving to an x86_64 manylinux wheel, which is present on disk and raises on import.
• nothing in $MODDIR/bin shadows FlashForge's python3, asked with command -v after sourcing anvil-env.sh -- which is the file that prepends $MODDIR/bin to PATH and therefore creates the hazard.
• CPython's dev half (headers, pkgconfig, config-3.13-*) did not ship.
• the ship boundary: no file on the installed printer is byte-identical to one under bin/ or docker/. Asked of the machine rather than of the .tgz, so it also sees anything a maintainer script wrote as root at install time.

What genuinely left with it, and is recorded here rather than in a commit message: there is no longer any check you can run on the build host, in seconds and without docker, against a package you are about to flash. In particular nothing warns you before you copy the file to a stick that it was built from the wrong model's stock package. The printer still refuses it ("Firmware does not match machine type") and make qa-replica still catches it, but the fast host-side warning that make verify gave the hardware checklist is gone.

What survived in test/, and then did not

Two things did, for a while. Both are gone now and test/ with them:

• test/integration/test_*.py -- 28 host-side unit tests in five files, down from 186; see The second cut below. Deleted, with nothing taking over the tool-frame arithmetic, the chamber macros' per-model branching, the stamp discipline in ff-startup.py, or the verification G-code.
• test/test-chelper.py -- in neither suite; pkgs/klipper/build.sh and bin/verify.sh called it directly at build time. Deleted with the tree, and its callers with it. What keeps a stale c_helper.so out now is that pkgs/klipper compiles the .so from the chelper sources of the tree it ships.

And two things that were under test/ and are not tests, now at tools/replica/:

• ffsim/ -- no longer a test framework and now smaller again, with extract_rootfs and its two tar helpers gone along with make rootfs. Replica is what make boot-screen-sim and, more to the point, bin/payload.sh use -- the payload is assembled by running the printer's own apk inside a replica, so this is build-path code that happened to live in the test tree.
• printer/ -- the Dockerfile.full, entrypoint.sh, assemble.sh, binfmt.sh and seed-prog.sh that BUILD the replica. qa/lib/replica.py uses them unmodified; they were never the problem. Moving them out of test/ changed no content, only qa/lib/replica.py's build directory and three launcher paths.

The second cut

The migration left test/ at 186 tests across twelve files. Seven of those files tested our own Python against fakes so thoroughly that the fake, not the printer, was the subject, and were dropped on that ground:

dropped	tests	asserted	what is lost
test_ff_legacy.py	6	FF_IMPORT_FIRMWARE_CONFIG against a hand-built klippy stub	the command's JSON reading is unasserted
test_ff_mcu_bringup.py	5	the port tuple and the every-port-fails path	nothing asks about either; the handshake was never covered anywhere -- see One thing found on the way
test_config_ownership.py	5	the DO-NOT-EDIT banner on mod-owned configs, and its absence on moonraker.conf	a banner can go stale without anything noticing
test_harness.py	4	repo_root, that every harness file compiles, and that every shebanged script is executable in git	the compile half is covered better by qa/static/test_shell_syntax.py's pyflakes pass; the executable-bit check is not covered anywhere -- qa/ checks only qa/replica/actions/*.sh
test_includes.py	8	printer.base.cfg's include set and order	a new ff-*.cfg can be shipped and never included
test_paths.py	7	every literal /bin,/sbin path in a payload script exists in the real rootfs	the branches no replica reaches -- S50wifi's udhcpc, wpa_supplicant -- are unasserted. It was also the last user of the rootfs fixture, now gone from test/conftest.py
test_tool_offset_sampling.py	15	the SAMPLES/SAMPLES_TOLERANCE parameter set in ff_tool_offset._estop	the defaults-match-the-fork property is unasserted

The four rows in bold-ish -- the executable bit, the include set, the rootfs paths -- are static questions that a replica cannot answer any better, so if they come back they belong in qa/static, not here.

The five survivors were then cut from 136 tests to 28 -- the 20% worth keeping -- on one question: would this failure be silent?

file	was	now	what the survivors are
test_startup.py	49	8	the stamp discipline, and the handover order. A stamp written before the values are verifiably saved is a printer that has lost its factory calibration for good, and no later boot will try again
test_ffscreen.py	33	5	the four ways it can do harm -- scribble outside the buffer, scribble a format it does not understand, raise into the migration, or turn the picture
test_tool_transform.py	24	7	every kept test is a move to the WRONG PLACE: a sign, a frame left applied while probing, a stale gcode_move cache, a klippy that will not parse its config on a calibrated printer
test_chamber.py	20	4	the three chamber mistakes that are silent until they have cost a print, plus the parse gate they rest on
test_gcode.py	10	4	a renamed macro, and the safe file's promise that it is cold and stays above Z50

What went with them, in one sentence each: the panel narration (words a person reads while waiting, and qa/replica/test_boot_screen.py renders the real thing on the real machine); the geometry probe and parse_geometry (a panel that looks wrong, not a printer that breaks); z_adjust staging and the per-job Z term (wrong numbers reported to a person); the config layout checks (a config that does not load, which the printer says loudly the first time); and the G-code bounds checks (Klipper refuses the move at the first offending line, before the toolhead has gone anywhere).

Each of the five files says in its own header what it dropped and why. That is deliberate: a test file that has been cut hard should say so where the next person opens it, not only in a document they may not read.

Still owed

• The build path, above. The largest gap. Its fixture generator, make-stock-fixture.sh, has been dropped too; recover it from git history.
• The rollback, case-recovery.sh's subject: install the mod, flash the stock package, be back byte-for-byte. Retired without a replacement.
• s6-svwait -U readiness on the shipped camera definition: the service needs a /dev/video* the replica has not got. test_supervisor.py asks the same question of a service it builds itself, which is close but not the same.
• The MCU lane, which would unblock the eight test_s6rc.py tests and let test_install.py assert a booted machine rather than an installed one.

One thing found on the way

case-mcu-bringup.sh:4 says the handshake is "tested elsewhere against a pty". There is no pty anywhere in the repo. test_ff_mcu_bringup.py checks the port tuple and the every-port-fails path, and its own comment concedes it. Every documented past bug in ff_mcu_bringup.py lived in the code that comment excuses from testing — and the missing /dev/ttyS* is now also what stops the replica booting, so one lane would close both.

Not in scope

A third lane — real klippy in batch mode against a simulated MCU — would close the gap docs/testing.md names: "the replica cannot, because it never starts klippy." The fork already ships the machinery (scripts/test_klippy.py, klippy's -i/-o/-d batch mode) unused. It is a separate bet with its own unknown (an MCU data dictionary for four MCUs, with no hardware to capture one from), and it was deliberately left out so this change has one variable.

Note for whoever picks it up: work/klipper-fork/klippy/chelper/c_helper.so is a MIPS binary. Host klippy cannot load it. Extract the fork tarball to a separate tree and let klippy build the host one there; never build it inside work/klipper-fork, whose artifact is a gated build output.