Oleksandr ShyshatskyiandClaude Opus 5 91604c2dfb Cut the suite to four gates, and rewrite the Python half as pytest ...

Twelve gates, most of them the weaker copy of a gate that survives. What is
left: pytest for the Klipper config, and three replica gates -- the end-to-end
install, MCU bring-up, and stock recovery.

Fix first, because the cut depends on it: a skip counted as a pass. Every
launcher exits 0 when it decides it cannot run, and sub() reported that as
`ok`. That is how test-abi.sh sat in the suite printing green while checking
nothing -- run-tests.sh deleted work/modpayload immediately before it ran and
CI set KLIPPER_FORK="", so it had no targets on any run, ever. Skips are now
counted apart, the summary says so, and the suite exits non-zero unless
ALLOW_SKIP=1 says the gap is deliberate. With four gates instead of twelve, a
gate that silently does not run is the failure mode that matters most.

Dropped, each because something else already does the job properly:

  test-abi.sh              bin/patch.sh refuses to build a non-nan2008
                           c_helper.so, bin/verify.sh re-checks the packaged
                           one, and case-install.sh reads the ELF header of
                           the file that actually landed on the machine.
  test-ash-conformance.sh  case-install.sh runs `sh -n` over every installed
                           script with the same qemu'd busybox and then greps
                           the boot log for "Syntax error".
  test-model-gate.sh       verify.sh SS8b/SS9 checks the same invariant and
                           pack.sh exits on MODEL MISMATCH before a package
                           exists. Its header claimed it proved the two models
                           ship different files; no such check was in it.
  test-macros.py           It used jinja2.Environment() -- default '{{ }}'.
                           The configs have no '{{' at all, only single-brace
                           expressions, so it checked '{% %}' block structure
                           and never looked inside an expression. Folded into
                           the chamber tests, which had the delimiters right.
  test-base-cfg.py         It read work/software/klipper/config/printer.base.cfg
                           as its stock reference -- the file bin/patch.sh
                           overwrites with ours. It was diffing our file
                           against itself: green on a cold tree, red on a
                           second run, blind to real drift either way. The
                           comparison moves to bin/unpack.sh, where a pristine
                           tree exists, as a warning rather than a gate.
  lint-danger.sh           Its structural greps were satisfiable by comments:
                           firmwareExe:44 and S80ui:16 *mention* helix and
                           SAFE-MODE, so they passed with the logic deleted.
  case-ui.sh               Goes with the helix-only decision.
  applets.allow            Ten entries, none of which the extractor had asked
                           about in a long time.

The command-word half of test-applets went too. It extracted only the first
word of a simple unprefixed command, so `if timeout 5 foo; then` yielded
['if','echo','fi'] -- the one failure its docstring named was invisible to it.
The absolute-path half survives: S50wifi's binaries sit behind a
[ -d /sys/class/net/wlan0 ] guard no simulation ever reaches, so nothing else
checks they exist.

The Python half is pytest now. Real assertions instead of hand-rolled
PASS/FAIL counters, underscored filenames so the modules import normally
(ffcfg keeps only the parser; the dash in test-macros.py was why one test used
to reach the other's parser through importlib), and a `rootfs` marker that
splits the tests needing the proprietary package from those that do not --
deselected in the PR lane, run in the replica half, so neither lane reports
skips it did not expect. test_default_delimiters_are_blind pins the jinja2 bug
so it cannot come back.

The config gate also moves out of the replica half into the half that runs on
a plain pull request. It needs python3, jinja2 and files in the repo, and it
is the only safety-relevant check in the suite -- a chamber heater declared on
a Creator 5 that has no element for it -- so it had no business being gated
behind the proprietary firmware.

Verified: pytest 21 passed / 5 rootfs-marked; the fixture pipeline
(unpack, patch, pack, verify) passes end to end; injecting a broken
single-brace expression fails test_every_macro_parses with the exact line, and
parses clean under the old defaults; the skip path reports SKIP and exits 1,
and ALLOW_SKIP=1 accepts it.

Also drops the CI `abi` job, which ran on a bare checkout with no binaries and
so was permanently green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-24 13:00:12 +03:00

..

Dockerfile.build

Cut the suite to four gates, and rewrite the Python half as pytest

2026-08-24 13:00:12 +03:00