mirror of
https://github.com/Klipper4FlashForge/firmware.git
synced 2026-10-22 21:32:16 -05:00
Oleksandr ShyshatskyiandClaude Opus 5
e0dad3513c
The shell suite decided "this gate did not run" by grepping the child's
stdout:
if "$@" >"$out" 2>&1; then
if grep -qE '^[[:space:]]*SKIP:|[0-9]+ skipped' "$out"
then skip "$name"; else pass "$name"; fi
Text on stdout cannot tell you what happened inside a process. A launcher that
had ALREADY failed reached that same line by another road: the file that
sources config.env failed to load, so STOCK_TGZ_* was never set, so the
launcher concluded there was nothing to test, printed those five characters
and exited 0 -- on a machine with docker and the firmware sitting right there.
That is how five broken launchers shipped and stayed broken.
So the host half is Python and a skip is an exception. Output cannot imitate
one, and a gate that fails cannot claim it was skipped.
test/ffsim/ config loading, docker plumbing, gate reporting
test/run-tests.py the orchestrator
test/integration/ sim-install.py, sim-roundtrip.py, printer-exec.py,
extract-rootfs.py
test/integration/printer/*.sh and the whole payload stay shell. They are
executed by the printer's own busybox ash under qemu, and surviving that is a
large part of what the suite proves; rewriting them would test a Python the
printer does not have. build-printer-image.sh and make-stock-fixture.sh stay
too: both are `set -euo pipefail`, neither is a gate, and one is a release
tool run by hand.
Four things that used to report as skips are now failures -- a config.env that
does not parse, a missing case script, a missing package, a missing baseline.
A genuinely absent precondition is still a skip, because that is still true.
Three more bug classes are gone by construction rather than by being tested
for. The repo root is found by searching upward for bin/common.sh instead of
counting `..` from $0, so a launcher can move to any depth. Sourcing a config
with a syntax error is checked -- `.` returns non-zero and does NOT stop the
shell, so the old code carried on with half a file applied, which is exactly
the broken-looks-absent confusion again. And pytest counts come from the JUnit
XML rather than from grepping "N skipped".
test_script_roots.py becomes test_harness.py: the `..` check now has two
remaining shell targets rather than seven, and it gains a byte-compile of the
whole harness. That last one matters -- sim-install.py and friends are only
executed by `make test-install`, which needs docker and the firmware, so
without it a typo would wait quietly until release day. It also pins the
property that replaced the `..` counting, from four different depths.
VERIFIED, and the verification is the point, because this machine has no
docker daemon and these are exactly the scripts that cannot run here:
- Stubbed the docker binary and diffed the recorded `docker run` argv,
shell against Python, across five paths: install with a prebuilt image,
install with a local build, local build plus PROG_DUMP, roundtrip, and
printer-exec. IDENTICAL in all five.
- Ran the old orchestrator and the new one in the same tree: both give
6 passed, 1 failed, 2 skipped, exit 1, with the same gates in the same
order. (The failure is shellcheck, which lives in the build image.)
- pytest: 24 passed / 5 skipped bare, 29 passed against a real extracted
rootfs.
- Compared failure modes directly. The four cases above go from "exit 0,
reported skipped" to "exit 1, named". Case 2 is the sharpest: with a
missing case script the shell launcher exited 0 in silence, having cp'd
nothing and mounted a directory over /case.sh.
Still unproven here: the gates that need a live daemon. `make test-install`,
`test-mcu` and `test-recovery` want one run on a docker-capable machine. The
argv diff says they will be handed the same command as before; it cannot say
what docker does with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
56 lines
2.1 KiB
Python
56 lines
2.1 KiB
Python
"""Shared fixtures for both Python lanes.
|
|
|
|
It sits at test/ rather than beside the tests because that is what lets
|
|
test/integration import ffcfg by name. The old scripts were called test-chamber.py and test-macros.py, and
|
|
the dash made `import test_macros` impossible, so one of them reached the
|
|
other's parser through importlib machinery. Underscores and a conftest remove the
|
|
whole problem. pytest.ini pins the rootdir so this file is found no matter
|
|
which lane is named on the command line.
|
|
|
|
Everything lives in test/integration. There was briefly a second directory,
|
|
test/unit, for the tests needing nothing but this checkout, and before that a
|
|
`rootfs` marker doing the same job through -m expressions. Both existed to
|
|
keep a plain pull request supplied with something to run; this repo has one
|
|
maintainer who always has the firmware, so neither was earning the split. The
|
|
`rootfs` fixture below skips instead, which is the honest report anyway.
|
|
"""
|
|
import os
|
|
import sys
|
|
|
|
import pytest
|
|
|
|
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
|
|
|
# TARGET_MACHINE -> the suffix bin/patch.sh derives from it. A model that is
|
|
# buildable but missing here would otherwise go untested.
|
|
MODELS = {"Creator5Pro": "creator5pro", "Creator5": "creator5"}
|
|
|
|
HEATER = "heater_generic chamber_heater"
|
|
|
|
|
|
@pytest.fixture(scope="session")
|
|
def root():
|
|
return ROOT
|
|
|
|
|
|
@pytest.fixture(scope="session")
|
|
def cfgdir():
|
|
"""payload/klipper/config -- in the repo, so this needs no firmware."""
|
|
return os.path.join(ROOT, "payload", "klipper", "config")
|
|
|
|
|
|
@pytest.fixture(scope="session")
|
|
def rootfs():
|
|
"""The printer's real extracted rootfs, or skip.
|
|
|
|
This is the one Python fixture that needs the proprietary package, which
|
|
is why everything using it lives in test/integration. A skip here is a gate
|
|
that did not run: run-tests.py reports it as SKIP rather than as a pass,
|
|
and refuses to call the suite clean.
|
|
"""
|
|
path = os.path.join(ROOT, "work", "rootfs")
|
|
if not os.path.isdir(os.path.join(path, "bin")):
|
|
pytest.skip("no printer rootfs -- run 'make rootfs' first")
|
|
return path
|