Commit Graph
4 Commits
Author SHA1 Message Date
Oleksandr ShyshatskyiandClaude Opus 5 57c776ee63 test/ is deleted; qa/ is the only suite
The eight files were already gone from the working tree. What this commit does
is everything that had to follow them, because nothing else had been updated
and three of the four things pointing at test/ were on a build or release path.

WHAT WAS STILL WIRED TO IT

  pkgs/klipper/build.sh   ran test/test-chelper.py under `|| pkg_die`. With the
                          file gone `make packages` would have died on every
                          klipper build.
  bin/verify.sh           ran the same script and read a non-zero exit as
                          "c_helper.so does not match the shipped klippy tree
                          (stale build)" -- a false red on every release verify
                          rather than an error about a missing file. Its
                          ship-boundary check also globbed `find bin test
                          docker`; that is `find bin docker` now.
  pytest.ini              testpaths = test, so a bare `pytest` collected
                          nothing at all. It is `qa` now, which is what the
                          comment above it was already describing.
  qa/static/              PY_GLOBS listed test/*.py and test/integration/*.py,
    test_shell_syntax.py  and _files() makes an empty glob a collection error
                          on purpose -- so the whole static lane failed to
                          collect, taking 93 unrelated tests with it.

Also dropped: the `pytest -- the host-side tree` step in ci.yml, the `test-py`
target with its .PHONY entry and two help lines, and qa/conftest.py's opening
paragraph explaining why qa/ sat beside test/ rather than inside it.

THE CHELPER SYMBOL CHECK IS GONE, NOT MOVED

It compared the .so's dynamic symbols against what klippy cdefs, and it was the
only check for a c_helper.so built from older sources than the klippy tree
beside it -- the failure that shipped once and bricked klippy startup on
hardware, because cffi resolves lazily and a stale .so imports cleanly.
qa/replica/test_install.py covers the ELF arch and nan2008 and nothing else.

What stands in its place is structural rather than a gate: pkgs/klipper
compiles the .so from the chelper sources of the very tree it ships, so the
mismatch is not something the recipe can produce. The comments in
pkgs/klipper/build.sh and docs/building.md now say that is the whole defence.

DOCS

docs/testing.md's "what is left in test/" section is now "test/ is gone", and
it names what went uncovered rather than leaving it to be found: the tool-frame
arithmetic in ff_toolchange, the chamber macros' per-model branching, the stamp
discipline in ff-startup.py, and the verification G-code against the shipped
macros. docs/hardware-testing.md said `make test-py` keeps the two
gcode/creator5-*.gcode files honest; it now says nothing does, and to read them
by hand when a macro is renamed. Paths corrected in docs/building.md,
docs/printer-replica.md and docs/qa-migration.md's intro.

Verified: `pytest` collects 366; qa/static is 136 passed, up from 43 because
the lane could not collect before this. Its 110 errors are this checkout's
missing vendor/opkg-utils, the same cause before and after. bin/verify.sh,
pkgs/klipper/build.sh and the Makefile parse; ci.yml is valid YAML.

Not touched, and stale before this commit: tools/python/README.md and
tools/supervisor/README.md still document test/integration/printer/case-*.sh
and a printer-exec.py that earlier commits removed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016bLGYpVAeVzAX6NeAxcU42
2026-08-30 01:10:24 +03:00
Oleksandr ShyshatskyiandClaude Opus 5 43baf4054e Drop the unit lane; one suite, in test/integration
The split existed so a plain pull request had something to run. This repo has
one maintainer who always has the firmware to hand, so it was a boundary kept
in sync for a contributor who never arrives. One directory now.

The tests moved rather than went away. test_chamber.py is the only check in
the suite that can catch a printer heating something it has no element for,
and the replica cannot cover it: case-install.sh touches .cfg files at exactly
two lines, both a grep for a sentinel proving the user config survived, and it
asserts klippy.py exists and c_helper.so has the right ABI without ever
starting klippy. Declaring [heater_generic chamber_heater] on the plain
Creator 5 fails four of these tests, including M191 waiting on a target the
machine can never reach; that same package installs, boots, and passes every
replica gate. make-stock-fixture.sh came too, since the packaging pipeline is
built on it.

Ordering changed, and that is the part to look at. With two lanes, pytest ran
twice -- once per lane, each with its own -m expression or path. With one it
runs once, so it has to run at the point where it can see the most: the stock
package is now resolved and the rootfs extracted BEFORE pytest rather than
after it. On a configured machine that is 28 tests in one pass; on a bare tree
the five that read the rootfs skip.

config.env is read in a subshell while doing that. It is the BUILD config, and
it is now consulted upstream of the fixture build, where the entire point is
that the fixture config wins -- sourcing it into this shell would put MOD_NAME
and TARGET_MACHINE where that build could see them. Verified by pointing a
config.env at MOD_NAME=LEAKED and confirming the fixture still builds
Creator5Pro-anvil-ci.tgz.

One thing gets worse, and it is inherent to having a single lane: on a tree
with no rootfs, pytest reports 5 skips, so sub() marks the whole gate SKIP
rather than ok, and the summary reads 6 passed instead of 7. Those 23 passes
are real and the line above the SKIP says so. This is the existing policy
working as designed -- a gate that did not fully run must not read as success
-- but the summary is coarser than it was when a lane existed that never
skipped.

Verified: 28 passed with a real extracted rootfs, 23 passed / 5 skipped
without; the full suite gives 7 passed / 1 failed / 1 skipped with a rootfs
present and 6 / 1 / 2 without, the failure being shellcheck, which lives in
the build image; no test/unit reference is left outside the passages that
explain why it is gone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 13:37:58 +03:00
Oleksandr ShyshatskyiandClaude Opus 5 2ae9736e97 Name the two lanes unit and integration, and fix what the move broke
test/repo -> test/unit, test/replica -> test/integration. The old names said
where the input comes from; these say what the tests are, which is the thing
you already have a word for.

The rename is also how the bug below surfaced, because "integration" is one
character-count away from nothing and running the launchers by hand is not.

Every launcher that moved out of test/ into a subdirectory opened with

    ROOT="$(cd "$(dirname "$0")/.." && pwd)"

and one `..` was correct at test/ and wrong a directory deeper. $ROOT became
test/, so the first thing each of them did -- source $ROOT/test/integration/
sim-image.sh -- went looking for test/test/integration/. Five scripts:
printer-exec.sh, sim-install.sh, sim-roundtrip.sh, build-printer-image.sh and
extract-rootfs.sh, which reached bin/common.sh the same way.

The previous commit shipped this. The path-existence sweep that was supposed
to catch it only checked paths written as literals; these are assembled at
runtime from $0. Nothing else caught it either, and that is the part worth
naming: the integration half cannot run without docker and the stock package,
so on any machine missing either -- this one, and a fork's CI without the
secrets -- five broken scripts still reported a clean SKIP.

So test/unit/test_script_roots.py now checks it statically: for every .sh
under test/, count the `..` after `dirname "$0"` and compare it against how
deep the file actually sits. It needs no docker, no firmware and no shell, so
it runs in the lane that always runs, which is the whole point -- the failure
it pins only appears in the lane that often does not.

Verified: the guard fails with "climbs 1, is 2 deep" against a reintroduced
bug and passes once it is reverted; all four launchers now reach their own
preconditions (docker daemon, config.env) instead of dying on a missing file;
21 in test/unit and 7 in test/integration against a real extracted rootfs;
run-tests.sh reports 7 passed / 1 failed / 1 skipped, the same as before the
rename (shellcheck lives in the build image, and no stock package is
configured in this worktree); no test/repo or test/replica reference is left
anywhere in the tree.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 13:23:18 +03:00
Oleksandr ShyshatskyiandClaude Opus 5 8337359a68 Split test/ into the two lanes it already had
The suite has run in two halves for a long time, and the rest of the repo is
built around that: RUN vs RUNSIM in the Makefile, `test` vs `printer-sim` in
CI, two halves in run-tests.sh. Only test/ itself was flat, so which half a
file belonged to was something you learned by reading its header.

    test/repo/      needs nothing but this checkout
    test/replica/   needs the printer's real rootfs
    test/           run-tests.sh, and the fixtures both lanes share

The axis is the precondition, not the runner. test/replica is everything that
cannot run without the proprietary package -- the qemu'd sim launchers and the
replica image, but also test_paths.py, which only reads the extracted rootfs
as files and needs no docker at all. Grouping by "does it use docker" would
have put that one in the wrong place; grouping by what has to exist first puts
every file where the question "can this run on a public checkout?" is answered
by the directory name.

The `rootfs` pytest marker goes with it. It existed to let run-tests.sh pass
`-m 'not rootfs'` in the PR lane and `-m rootfs` in the replica lane; the
directories say the same thing without anyone having to remember to tag a new
test, and a file in the wrong lane now shows up in `git status`. The two
lanes are `pytest ./test/repo` and `pytest ./test/replica`.

pytest.ini is new and is load-bearing: without a pinned rootdir, naming a lane
on the command line makes pytest take that lane as the root and never load
test/conftest.py, so the fixtures both lanes share would vanish depending on
which one you asked for.

Renames only, plus the path references: test/fixtures/make-stock-fixture.sh ->
test/repo/, the sim launchers and test/printer/ -> test/replica/, and
test_applets.py -> test/replica/test_paths.py, which is what it has actually
checked since the command-word half was dropped.

Verified: 19 in test/repo and 7 in test/replica against a real extracted
rootfs, the same 26 as before the split; run-tests.sh reports 7 passed /
1 failed / 1 skipped, identical to the pre-split run on this host (shellcheck
is absent here and lives in the build image, and no stock package is
configured in this worktree); every referenced path in the repo resolves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 13:13:45 +03:00