mirror of
https://github.com/Klipper4FlashForge/firmware.git
synced 2026-10-22 05:12:16 -05:00
worktree-timelapse-test-gcode
24
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Oleksandr ShyshatskyiandClaude Opus 5
|
ba01b8f9f0 |
The release e2e installed both models into one Creator5Pro replica
`for p in dist/*.tgz` fed the replica the Creator5 package first. That machine's own app_startup.sh globs /mnt/Creator5Pro-*.tgz, found nothing, and fell through to a normal boot -- so all 123 replica tests errored with "could not install the package into a replica", about a package that was built correctly and whose model gate was working exactly as designed. The loop is older than the pytest lane and used to be fine: sim-install.py resolved a machine per package. The replica lane takes PRINTER_IMAGE from test.env, and there is one of those. So the end-to-end runs on the model the replica IS, and a Creator5 end-to-end waits on a Creator5 replica image that nothing publishes yet. Both models are still built and both are still attached to the release; only the install test narrows. upload-artifact moves above that step while I am here, so a run that produced packages hands them over even when the lane fails. The release gate is unchanged: Publish comes after, and a failed step still stops the job before it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S64uRMdZbj9xPMns2dqMG8 |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
b98552386b |
No FlashForge component ships; patch.sh is payload.sh
The software component existed to give the mod one exec hook. It is not needed for that any more, so a release now carries our installer and our payload and nothing else: 26MB smaller, 21MB of which was FlashForge's own firmwareExe travelling back to the partition it was already on, along with their klippy tree, their wifi module and their configs. WHAT IT GIVES BACK. /usr/prog/PROGRAM/software is no longer rewritten, so the version directory there stays FlashForge's with their genuine firmwareExe inside it -- and app_startup.sh's five-second watchdog, which copies one out of that directory when nothing by that name is running, has something to copy again. A printer whose payload fails to install now comes up on the stock UI instead of a blank screen. That recovery has been broken since we started deleting their binary out of a component we shipped; docs/how-it-works.md claimed it worked throughout. THE ROOT PASSWORD moves with it, because it used to ride in the component's shadow file. bin/pack.sh bakes ROOT_PW_HASH into the installer, single-quoted -- an unquoted $6$... would have the printer's shell expand $6 as a positional parameter and ship a truncated hash -- and the installer writes it to /usr/prog/etc/shadow. The random-password path asks one question instead of four: FlashForge ship the same root hash on every machine, so a live shadow still carrying it means nobody has set a password and it is ours to set. The .prev-root-hash recording, the shipped-vs-live comparison and the restore-it- afterwards step were all machinery for surviving a file that nothing overwrites any more, and are gone. pack.sh checks the hardcoded constant against the stock package it built from, because getting it wrong is silent and total: every printer would read as "already set" and none would ever get a password. bin/patch.sh is bin/payload.sh. It patched nothing any more -- no splice, no shadow edit, no firmwareExe deletion -- and what is left is the payload: the feed installed by the printer's own opkg, the seeded config, the manifest. Its sections are renumbered 1-5 with no gaps, and the five tombstone comments marking where removed sections used to be are gone with them. test_recipe_layout.py loses six of its nine tests. Each compared the tree against a description of the tree -- no fourth directory kind, each kind still used, 3rdparty carries no files of ours, installer/ holds exactly two named files -- and none could go red for a reason a printer would notice, while all six went red the moment the layout legitimately changed. The two that bite are kept: a payload/ path escaping $MODDIR, and a recipe name at both levels, which makes pkg_dir silently build the wrong package. SW_VER is deleted from config.env.example and both workflows: it stamped a component that is not built. Gates: 112 static tests pass. The generated installer is verified after substitution -- all four lines present, and `sh -n` over the result, because a hash with a stray character in it would otherwise reach a printer as a syntax error whose only symptom is a package that installs nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CH7Yb4wuPhnppHAB25UX7e |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
56b5b6c334 |
The printer image is the only replica; make rootfs was a second way nothing took
`make rootfs` unsquashed rootfs.squashfs out of the stock package into
work/rootfs, and a thin container bind-mounted it with PROG_DUMP supplying
/usr/prog from the factory image. Every caller set PRINTER_IMAGE anyway --
both workflows wrote it into test.env BEFORE running `make rootfs`, so CI
unpacked a 93MB package, unsquashed a rootfs and downloaded a 182MB factory
image on every push and read none of it. build-printer-image.sh never used
work/rootfs either: Dockerfile.full does its own extraction from the same
public firmware and needs no stock package at all.
So the fallback went, and PROG_DUMP with it -- it was only ever read when the
replica was built locally, and both implementations already passed it empty
otherwise. Gone: the target, extract-rootfs.py, gates.extract_rootfs and its
two tar helpers, the thin printer/Dockerfile, the `prebuilt` flag threaded
through both Replica implementations, and Replica.start's want_output, whose
only message was that the local build was about to be slow.
PRINTER_IMAGE is pinned to 1.9.7-1.2.9-20260810 rather than :latest. It is
now the entire substrate the brick-safety gates run on and the only input
nothing checksums, so a floating tag would let it change under a green suite
-- the drift the sha256 pins in both workflows exist to prevent.
THREE THINGS THAT COULD NOT HAVE PASSED, found while doing it:
* Neither workflow built the feed. bin/patch.sh has refused to start
without one since
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
6aa622d2ff |
6,280 lines of case scripts reported 13 bits. They are 118 tests now.
test/run-tests.py is gone, and with it the bespoke harness that wrapped
pytest, re-parsed its JUnit XML, and drove thirteen replica case scripts to
get one boolean back from each. Every gate it ran is either a module under
qa/replica/ or deliberately retired.
PORTED. case-install.sh (579 lines, 1 bit) is test_install.py, 22 tests, asked
of the machine the printer's own app_startup.sh built rather than one the
harness assembled. case-upgrade.sh is 21, case-supervisor.sh 31,
case-boot-screen.sh 17, case-mcu-bringup.sh 8, and case-nginx.sh plus
case-moonraker313-s6.sh are test_web.py's 7.
RETIRED, subject deleted by the s6-rc switch: case-services.sh, case-nginx.sh,
case-moonraker.sh, case-moonraker313-s6.sh, case-camera.sh and
case-priority.sh drove $MODDIR/init.d/S* or sourced anvil-service.sh, neither
of which exists. Decided against real output -- `make test-services` failed
with "the payload ships no anvil-service.sh". case-pyext.sh and
case-moonraker313.sh went too: no gate, no target, no CI step, never run once.
RETIRED as a coverage decision: case-python.sh (test_web.py runs real Moonraker
on that interpreter, which is harder), case-libpath.sh, and case-recovery.sh.
The last is the brick-safety rollback and has no replacement; it is recorded in
docs/qa-migration.md rather than left to be discovered.
THREE ASSERTIONS HAD STOPPED MEANING ANYTHING, and each had been green for
months:
* case-mcu-bringup.sh's negative control had INVERTED. It asserted the
bring-up fails without LD_LIBRARY_PATH; CPython is --disable-shared now and
needs no library path, so the case's own skip branch was the one firing.
* case-upgrade.sh's headline assertion was VACUOUS. run-append.sh:150 does
`rm -rf $MODDIR/init.d` unconditionally, outside the manifest branch, so
"the update removed the stale S60web" passed regardless of the manifest
logic it was written to test.
* case-boot-screen.sh CONFIRMED ITS OWN ARITHMETIC, computing expected pixel
packing by calling ffscreen._pixel inside the replica -- a swapped R/B
matched itself.
case-install.sh's boot-log grep is not reproduced for the same reason: the log
line names every service in the boot set including the ones that then fail, so
grepping it cannot go red. The compiled database is asked instead.
TWO REAL BUGS FOUND ON THE WAY.
assemble.sh attached /stick.img with `losetup` and wrote the device to
/stick.loop, which NOTHING HAS EVER READ. Loop devices are the host's, not the
container's, so every USB_STICK=1 run leaked one permanently and a WSL2 kernel
has thirteen. Once they were gone every replica run failed with "failed to
setup loop device", which looks like a docker fault and is not. Detached now in
entrypoint.sh on the way out, and in qa's stop() for the held-open containers
that never take that path.
test_startup.py was 43 tests red: it built its argparse.Namespace by hand and
had never been taught --only-bringup / --no-bringup, and its FakeKlipper still
modelled start.sh plus a separate stop when klipper is restarted with one
`s6-svc -wr -t`.
CI AND RELEASE. The replica job was getting its package as a side effect of
run-tests.py running unpack/patch/pack; that is an explicit step now, as the
comment there predicted it would have to be. release.yml tested the exact files
it ships through sim-install.py, whose exit status the loop was discarding --
it is `REAL_PKG=<pkg> pytest ./qa/replica` per model now, 22 named results each.
WHAT IS NOT TESTED ANY MORE: bin/unpack.sh, patch.sh, pack.sh and verify.sh.
The packaging build on a synthetic stock package went with the runner and has
no counterpart in qa/. That is the path producing the .tgz a user flashes.
make-stock-fixture.sh is kept as the input a replacement needs.
Measured: qa/static 246 passed, qa/replica 110 passed and 8 blocked on hardware
the replica has not got (no /dev/ttyS*, no /dev/video*, so ok-all is
unreachable), test/ pytest 185 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BTAXgjtF8GdhqSxvdXCFW
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
5b463b889a |
Five switches that were only ever set to 1
MOD_WEB, MOD_CAM, MOD_UI, MOD_SSH and MOD_WIFI all defaulted to on, and what
each one really offered was a half-installed printer: Mainsail with no
moonraker behind it, a camera that would not stream, a screen dark on purpose,
a root password deliberately not set on a machine whose entire recovery story
is "ssh in". Nobody wants those states. What someone who sets one actually
wants is a stock printer, and that is one FlashForge package away.
What they cost was a second answer to every question. Each init script carried
a gate, each `down` file argued its existence from a flag rather than from the
ordering that was the real reason, and anvil.conf had five lines that told a
user they had a choice.
MOD_SSH was the clearest case: it never turned ssh off. dropbear is the
rootfs's own and listens either way, so all the flag did was decline to set a
password.
Each gate is replaced by the fact it was standing in front of, not by a
constant:
S60nginx, S62moonraker nothing -- the web stack is what this mod is
S65camera there is no video device, which the script already
finds out and reports
S50wifi there is no $WIFI_CONF, likewise
S80ui HelixScreen is not installed, which was already the
other half of the condition
To leave a piece out, leave it out: BUILD_MAINSAIL, BUILD_MOONRAKER and
BUILD_HELIX decide what goes into a build, and `opkg remove` decides what stays
on a printer. What survives in anvil.conf is what has a range rather than a
state -- MOD_CAM_RES/FPS/PORT and the NICE_* priorities -- plus MOD_SPLASH,
MOD_STARTUP, MOD_IMPORT and MOD_S6, which are untouched here.
The `down` files stay. They were argued from MOD_WEB and MOD_CAM, but the
argument never depended on them: s6-svscan starts a service the instant the
directory appears, which is before any init script has checked a prerequisite.
Their text now says so.
Measured: `make build` produces a payload with no live read of any of the five,
and qa-static is 265 passed with shellcheck over the edited init scripts. The
test suite still exercises the removed flags and has not been updated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gKRUCqj18TVF1Vz1oMpmC
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
f67e7a269f |
ls pkg/ was thirty-eight entries and four of them mattered
pkg/ -> pkgs/, and the thirty-four recipes that build a pinned tarball and
carry no files of this repo move down a level into pkgs/3rdparty/. What is
left at the top is the four components a person actually edits: anvil-core,
klipper, moonraker, helixscreen.
THE LINE IS MECHANICAL, NOT EDITORIAL, and that is why it can be a directory
rather than a field in pkg.conf. "Do we actively modify it" drifts -- the day
somebody patches zlib nobody would agree which side it belongs on, and the
answer would be a rename touching CI and docs decided by taste. "Does it carry
files of ours" is a fact about the tree: it is exactly whether the recipe has
a payload/, prog/ or seed/, checkable in one assertion, and
test_recipe_layout.py now demands the move in both directions -- a 3rdparty
recipe that grows one of the three must come up, and a top-level recipe with
none of them must go down, or the top level slowly refills and the split stops
meaning anything.
ONE FUNCTION KNOWS THE TWO DEPTHS. pkgs/lib.sh's pkg_dir resolves a recipe id
to its directory and nothing else spells either path -- not a recipe, not
bin/build-packages.sh, not a test. Names stay unique across both levels
because pkg_out, pkg_stamp and the .ipk filename are all keyed by the bare
name; a duplicate would resolve to whichever level is searched first and
build the wrong thing silently, so that is a test too.
Recipes now source `. ./bin/common.sh` instead of counting `..` from $0.
Depth-counting would have needed two spellings for two levels, and it was
never buying anything: `. pkg/lib.sh` on the next line already required the
working directory to be the repo root.
A SECOND SILENT-SHRINK BUG, and this one was live. qa/static's _files()
expanded a list of globs and returned whatever matched, so "pkgs/*/build.sh"
quietly stopped covering thirty-four recipes the moment they moved -- the lane
stayed green, and the only visible sign was a test count dropping by 34 in a
run nobody was counting. _files() now refuses a pattern that matches nothing.
That is the same failure the ash-compatibility check had a guard for and the
syntax check did not, which is why the guard reads the way it does.
AND ONE PACKAGE WAS SHIPPING PYTHON BYTECODE. anvil-core stages a directory of
.py helpers wholesale, so any test that imported one left a __pycache__ beside
it -- gitignored, invisible in a diff, copied by cp -a like anything else. The
package's contents depended on whether pytest had been run on the machine that
built it: two builds of one commit, different sha256, cause nowhere in the
tree. What it shipped was worthless as well as unstable, compiled by the build
image's python 3.11 for a printer running the 3.13 this repo cross-builds. The
sweep is in pkg_ship now, once, and the two hand-written copies in
pkgs/klipper and pkgs/moonraker are deleted -- the trap is set for any recipe
that stages a tree of .py files, and the two recipes that had hit it had each
solved it locally while the third, which had not, was the one shipping.
Measured:
* the rename changes no package bytes. Same cached build trees, feed built
with the diff stashed and unstashed: all 41 .ipk match.
* with the bytecode sweep, anvil-core returns to exactly the bytes the
previous commit produced.
* no package in the feed contains a .pyc or a __pycache__.
* make qa-static: 247 passed (236 before this work + 11 layout tests).
* make test: 8 passed, 0 failed, including unpack/patch/pack/verify on the
synthetic stock package. make test-py: 181 passed.
Not explained: anvil-python-dev produced two different sha256 from an
UNCHANGED work/pkg/python at two points today, under code that was identical
both times -- so it is not reproducible from a cached tree, and it is not this
commit's doing (the stashed and unstashed builds above agree on it). Worth a
look; the dev split in bin/build-packages.sh is where to start.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
09a9f868e3 |
klippy compiles c_helper.so at first run, and this printer has no compiler
So the "build system" for that .so genuinely is one gcc line -- the argument
list in klippy/chelper/__init__.py -- and pkg_build could not express it,
because it always ran make. That is why Klipper was the last cross-build
still living in bin/patch.sh, which unpacked the Ingenic toolchain into
work/.mips-toolchain, wrote its own gcc invocation, cached on an mtime
comparison and gated the result itself: every one of the things pkg/lib.sh
exists to do once.
PKG_CC_SHARED is that line, appended to `$CC -shared -fPIC`. A knob and not
a verb, for the reason pkg_build's own header gives about the three
mechanisms with one user each: the recipe says WHAT to link, pkg_build owns
HOW, and -EL -mnan=2008 stay spelled once in pkg_toolchain where the wrapper
is gated before anything is built on it.
pkg/klipper -> anvil-klipper_0.13.0+gitb722a4c8-1_mipsel_xburst2.ipk
Both gates moved into the recipe, next to the ship list they check -- the
same move the s6 presence checks made. They now run on `make packages` too,
so a c_helper.so that is not nan2008, or one missing a symbol the klippy
beside it cdefs, fails the build that produced the .ipk rather than only a
full firmware build. The symbol gate is the one that matters: cffi resolves
lazily, so a stale .so imports cleanly and dies at connect on the printer.
KLIPPER_FORK is gone. It pointed the build at a local checkout instead of
the pin, which is a second source for a recipe allowed exactly one -- and it
is the same variable that let v20260824-nova-kakhovka ship a klippy tree
nobody had built, set to "" in CI. To build your own fork, put its tarball
in vendor/ and re-pin KLIPPER_VERSION and KLIPPER_SHA256; the sha is then
the guarantee every other source in this repo carries. bin/fetch-assets.sh
loses two of its conditions with it, and test/test-chelper.py loses the
config.env fallback that printed "SKIP: no KLIPPER_FORK configured" -- a
green line for a check that had not run.
bin/patch.sh section 1 is now staging and nothing else: a cp -a into the
SOFTWARE component, where the stock klipperDaemon looks, plus the
chelper.tar its run.sh extracts. That tar stays out of the .ipk, where it
would be a second copy of a file the package already contains. The package
installs the tree under $MODDIR/klipper, inert until phase 7 of
docs/notes/80-s6-migration.md makes the installed copy the live one -- at
which point these lines are deleted rather than moved.
Measured:
* two cold builds of pkg/klipper byte-identical, work/pkg deleted between
* c_helper.so is 0x70001407 -- nan2008, o32, mips32r2
* all 66 functions klippy declares are exported, checked after pkg_ship
strips rather than before
* the whole feed still builds: 38 recipes, 41 packages
* 236 static tests, including the staged-recipe list that now names klipper
* BUILD_KLIPPER=stock removes the recipe from pkg_recipes entirely
Not verified here: bin/patch.sh end to end. It needs the stock FlashForge
package, which the packaging lane deliberately does without; bin/verify.sh's
klippy-tree and chelper.tar assertions are what cover the staging rewrite.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
8b58bfc831 |
PRINTER_IMAGE was optional and unset, so it was never used
vars.PRINTER_IMAGE has apparently never been set on this repository, so every replica case in ci.yml and release.yml has been paying the local replica build -- about a minute -- on every single case, for the whole life of both workflows. Defaults to the public monstrofil/creator5-printer:latest now; vars.PRINTER_IMAGE still overrides it for a fork building its own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Fable 5
|
3b37302c2a |
TOOL_OFFSET_CALIBRATE stops shutting the printer down
_current_tool_or_none() returns (tool, why) -- every call site unpacks it, except the one reapply_offsets_after_external_zero added: it bound the whole tuple to `tool`, so the `is None` guard never fired (no tool mounted is (None, why), a truthy tuple) and off_z[tool] died with "list indices must be integers or slices, not tuple", taking the printer to shutdown at the end of every TOOL_OFFSET_CALIBRATE. Unpack it, and treat -1 (nothing on the carriage) as nothing-to-reapply, same as the runout call site does. Also let a release tag carry the same-day suffix common.sh already documents (v20260825b-...): the installer never reads MOD_VER -- it compares the software component's own version, which stays at stock's -- so the letter exists only to keep the outer filenames distinct. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Fable 5
|
212d3a1efd |
A fork build without a fork is now impossible to ship
v20260824-nova-kakhovka went out without the Klipper fork: release.yml set KLIPPER_FORK="", bin/patch.sh answered with "(skip) Klipper: keeping stock tree", and every gate stayed green because every Klipper check was conditional on files the broken package simply did not contain. On an already-modded printer the stock 0.12-era overlay half-overwrote the v0.13 tree and klippy died at connect: "'void(*)(struct stepper_kinematics *, double, double, double)' expects 4 arguments, got 3" -- a v0.12 caller against the v0.13 cdef of extruder_set_pressure_advance (upstream c84d78f3f). Note for the next reader of such a traceback: an arg-count TypeError is a MIXED KLIPPY TREE, python against python; a stale c_helper.so shows up as AttributeError instead, because cffi resolves symbols lazily. The fork now travels like every other asset. versions.env pins the creator5 commit and the Ingenic gcc 7.2.0/glibc 2.29 toolchain (the X2000's own; Debian's mips crosses are big-endian or legacy-nan), fetch-assets.sh downloads both by sha256, and patch.sh extracts the pinned tarball whenever KLIPPER_FORK is not a local checkout -- which is exactly what CI now does, with the KLIPPER_FORK="" lines gone from both workflows and vendor/ cached on the pin file. c_helper.so is cross-compiled from the chelper sources of the very tree being shipped whenever it is missing or older than them, so the stale-.so class dies too; test/test-chelper.py returns from 3a28fdf's grave to prove every cdef'd function is exported, and runs both in patch.sh and in verify.sh against the extracted package. BUILD_KLIPPER=fork with no tree is a hard build failure now; stock has to be asked for by name. verify.sh also stops trusting `echo "$LIST" | grep -q`: under pipefail, grep exiting at an early match SIGPIPEs the echo and the check reports 141 -- a race that stayed invisible until the fork tree tripled the payload listing, then intermittently declared Mainsail and Moonraker missing from a package that carried both. Herestrings have no pipe to break. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Fable 5
|
b1c2a0553d |
release.yml stops promising a SAFE-MODE that no longer exists
The generated release notes still told a dark-screen user that the UI had
latched SAFE-MODE after three failed boots -- removed in
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
1c0dd83c61 |
Release notes carry the city the release is named for
The fixed half of the notes is the same every time; the half that says which city and what happened there is not, and it was living only in a hand-edited release page. docs/releases/<tag>.md holds it now, at the tagged commit, and the workflow appends it -- falling back to the bare line, with a warning in the job log, when a tag arrives without one. Nova Kakhovka's is the text now on the v20260824 release. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
04d4509b0a |
The shipping-package gate runs in the build image too
Every step before it runs as root inside that image, so work/ and dist/ end up root-owned and the replica could not create its work/.sim-* stage as the runner user: PermissionError, after the brick-safety suite and both builds had already passed. Same container, same docker socket as the suite step above it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
07d99b2748 |
release.yml: two spaces, not a tidy column, before the checksummed path
sha256sum -c takes everything after the first two spaces as the filename, so the aligned lines asked for a file whose name started with spaces and the release died on its second download. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
ecb7fb76e4 |
Release on a tag, with nothing secret in the way
The stock packages and the factory image are large, not confidential -- ci.yml already fetches them from ghzserg/FF by public URL with an enforced sha256. release.yml asked for the same three things as repository secrets instead, so a tag on a fresh clone failed the first step with "missing secrets" and published nothing. Same URLs, same checksums, same enforcement as CI now. Tags are v<YYYYMMDD>-<city>: the release date and a Ukrainian city under occupation. Only the date reaches MOD_VER -- the installer compares the version field as a number -- so the codename rides in the tag and the release title. Releases are marked pre-release, and the notes say where to go when the screen is dark and how to get back to stock. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
f70369155b |
Run the brick-safety gates in CI, and stop skipping quietly
The replica gates -- the only ones that decide whether a package bricks a
printer -- had never run in CI. They were gated on a STOCK_URL_CREATOR5PRO
secret that was never set, so printer-sim rendered as a grey dash beside
two green ticks and read as "CI passed".
The packages it needs are already public. ghzserg/FF publishes both stock
packages and the factory image, and this repo already points at that
release in docs/printer-replica.md. All three are now pinned in ci.yml by
sha256 and verified byte-for-byte against the copies used locally:
Creator5Pro-1.9.7-1.2.9-20260810.tgz 1f3252fe... 93MB
Creator5-1.9.7-1.2.9-20260810.tgz 72009be1... 93MB
Creator5Pro-factory.tar.xz 45657924... 190MB
Nothing proprietary is redistributed from here; the workflow fetches what
is published elsewhere, cached on their checksums so an unchanged pin
costs that archive nothing. Secrets still override for a fork with its own
firmware -- but a URL override without a matching checksum now refuses to
run rather than downloading something nobody verified. have-firmware is
gone with the `if:` it fed.
Rehearsing that job locally found the reason it would have failed anyway.
The replica gates call ./bin/patch.sh directly, and patch.sh expects
vendor/ already populated -- bin/build.sh is what fetches Mainsail and
HelixScreen, and the gates do not go through build.sh. No local run ever
noticed, because vendor/ was always already there. Both workflows now
fetch assets before the suite; in release.yml that step has to precede the
gates, since `make release` would otherwise fetch them one step too late.
Second half: a skip must be legible.
ALLOW_SKIP=1 silenced the explanation as well as the exit code -- the
summary printed "2 skipped" and then nothing about what or why. That is
the disease this harness exists to treat, reintroduced by the flag meant
to manage it. Now:
- Every skip is listed again in the summary with its reason and whether
it was accepted, whatever ALLOW_SKIP says, and becomes a GitHub
annotation. Nobody reads a green job's log.
- ALLOW_SKIP takes names: ALLOW_SKIP="pytest,the printer replica". `1`
still accepts everything, but it is a standing promise never to notice
a skip again, so CI uses the list and a third skip fails there.
- The pytest gate names the tests that skipped instead of counting them.
That immediately showed the exec-bit test skipping wherever .git is
absent -- a bare "6 skipped" would not have.
VERIFIED:
- printer-sim rehearsed end to end on the actual published bytes, real
git checkout, REQUIRE_PRINTER_SIM=1 and no ALLOW_SKIP: 12 passed,
0 failed, 0 skipped, exit 0, in about four minutes. Before the vendor
step it failed at patch.sh, which is what CI would have done.
- The four ALLOW_SKIP cases, in the build image with no firmware: none
set -> exit 1 naming both gaps; "pytest" only -> exit 1, replica still
NOT ACCEPTED; both named -> exit 0, both still printed; and under
GITHUB_ACTIONS the ::warning:: annotations appear.
- pytest 30 passed. Both workflows parse.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
e0dad3513c |
Rewrite the host half of the harness in Python, so a broken gate fails
The shell suite decided "this gate did not run" by grepping the child's
stdout:
if "$@" >"$out" 2>&1; then
if grep -qE '^[[:space:]]*SKIP:|[0-9]+ skipped' "$out"
then skip "$name"; else pass "$name"; fi
Text on stdout cannot tell you what happened inside a process. A launcher that
had ALREADY failed reached that same line by another road: the file that
sources config.env failed to load, so STOCK_TGZ_* was never set, so the
launcher concluded there was nothing to test, printed those five characters
and exited 0 -- on a machine with docker and the firmware sitting right there.
That is how five broken launchers shipped and stayed broken.
So the host half is Python and a skip is an exception. Output cannot imitate
one, and a gate that fails cannot claim it was skipped.
test/ffsim/ config loading, docker plumbing, gate reporting
test/run-tests.py the orchestrator
test/integration/ sim-install.py, sim-roundtrip.py, printer-exec.py,
extract-rootfs.py
test/integration/printer/*.sh and the whole payload stay shell. They are
executed by the printer's own busybox ash under qemu, and surviving that is a
large part of what the suite proves; rewriting them would test a Python the
printer does not have. build-printer-image.sh and make-stock-fixture.sh stay
too: both are `set -euo pipefail`, neither is a gate, and one is a release
tool run by hand.
Four things that used to report as skips are now failures -- a config.env that
does not parse, a missing case script, a missing package, a missing baseline.
A genuinely absent precondition is still a skip, because that is still true.
Three more bug classes are gone by construction rather than by being tested
for. The repo root is found by searching upward for bin/common.sh instead of
counting `..` from $0, so a launcher can move to any depth. Sourcing a config
with a syntax error is checked -- `.` returns non-zero and does NOT stop the
shell, so the old code carried on with half a file applied, which is exactly
the broken-looks-absent confusion again. And pytest counts come from the JUnit
XML rather than from grepping "N skipped".
test_script_roots.py becomes test_harness.py: the `..` check now has two
remaining shell targets rather than seven, and it gains a byte-compile of the
whole harness. That last one matters -- sim-install.py and friends are only
executed by `make test-install`, which needs docker and the firmware, so
without it a typo would wait quietly until release day. It also pins the
property that replaced the `..` counting, from four different depths.
VERIFIED, and the verification is the point, because this machine has no
docker daemon and these are exactly the scripts that cannot run here:
- Stubbed the docker binary and diffed the recorded `docker run` argv,
shell against Python, across five paths: install with a prebuilt image,
install with a local build, local build plus PROG_DUMP, roundtrip, and
printer-exec. IDENTICAL in all five.
- Ran the old orchestrator and the new one in the same tree: both give
6 passed, 1 failed, 2 skipped, exit 1, with the same gates in the same
order. (The failure is shellcheck, which lives in the build image.)
- pytest: 24 passed / 5 skipped bare, 29 passed against a real extracted
rootfs.
- Compared failure modes directly. The four cases above go from "exit 0,
reported skipped" to "exit 1, named". Case 2 is the sharpest: with a
missing case script the shell launcher exited 0 in silence, having cp'd
nothing and mounted a directory over /case.sh.
Still unproven here: the gates that need a live daemon. `make test-install`,
`test-mcu` and `test-recovery` want one run on a docker-capable machine. The
argv diff says they will be handed the same command as before; it cannot say
what docker does with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
2ae9736e97 |
Name the two lanes unit and integration, and fix what the move broke
test/repo -> test/unit, test/replica -> test/integration. The old names said
where the input comes from; these say what the tests are, which is the thing
you already have a word for.
The rename is also how the bug below surfaced, because "integration" is one
character-count away from nothing and running the launchers by hand is not.
Every launcher that moved out of test/ into a subdirectory opened with
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
and one `..` was correct at test/ and wrong a directory deeper. $ROOT became
test/, so the first thing each of them did -- source $ROOT/test/integration/
sim-image.sh -- went looking for test/test/integration/. Five scripts:
printer-exec.sh, sim-install.sh, sim-roundtrip.sh, build-printer-image.sh and
extract-rootfs.sh, which reached bin/common.sh the same way.
The previous commit shipped this. The path-existence sweep that was supposed
to catch it only checked paths written as literals; these are assembled at
runtime from $0. Nothing else caught it either, and that is the part worth
naming: the integration half cannot run without docker and the stock package,
so on any machine missing either -- this one, and a fork's CI without the
secrets -- five broken scripts still reported a clean SKIP.
So test/unit/test_script_roots.py now checks it statically: for every .sh
under test/, count the `..` after `dirname "$0"` and compare it against how
deep the file actually sits. It needs no docker, no firmware and no shell, so
it runs in the lane that always runs, which is the whole point -- the failure
it pins only appears in the lane that often does not.
Verified: the guard fails with "climbs 1, is 2 deep" against a reintroduced
bug and passes once it is reverted; all four launchers now reach their own
preconditions (docker daemon, config.env) instead of dying on a missing file;
21 in test/unit and 7 in test/integration against a real extracted rootfs;
run-tests.sh reports 7 passed / 1 failed / 1 skipped, the same as before the
rename (shellcheck lives in the build image, and no stock package is
configured in this worktree); no test/repo or test/replica reference is left
anywhere in the tree.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
8337359a68 |
Split test/ into the two lanes it already had
The suite has run in two halves for a long time, and the rest of the repo is
built around that: RUN vs RUNSIM in the Makefile, `test` vs `printer-sim` in
CI, two halves in run-tests.sh. Only test/ itself was flat, so which half a
file belonged to was something you learned by reading its header.
test/repo/ needs nothing but this checkout
test/replica/ needs the printer's real rootfs
test/ run-tests.sh, and the fixtures both lanes share
The axis is the precondition, not the runner. test/replica is everything that
cannot run without the proprietary package -- the qemu'd sim launchers and the
replica image, but also test_paths.py, which only reads the extracted rootfs
as files and needs no docker at all. Grouping by "does it use docker" would
have put that one in the wrong place; grouping by what has to exist first puts
every file where the question "can this run on a public checkout?" is answered
by the directory name.
The `rootfs` pytest marker goes with it. It existed to let run-tests.sh pass
`-m 'not rootfs'` in the PR lane and `-m rootfs` in the replica lane; the
directories say the same thing without anyone having to remember to tag a new
test, and a file in the wrong lane now shows up in `git status`. The two
lanes are `pytest ./test/repo` and `pytest ./test/replica`.
pytest.ini is new and is load-bearing: without a pinned rootdir, naming a lane
on the command line makes pytest take that lane as the root and never load
test/conftest.py, so the fixtures both lanes share would vanish depending on
which one you asked for.
Renames only, plus the path references: test/fixtures/make-stock-fixture.sh ->
test/repo/, the sim launchers and test/printer/ -> test/replica/, and
test_applets.py -> test/replica/test_paths.py, which is what it has actually
checked since the command-word half was dropped.
Verified: 19 in test/repo and 7 in test/replica against a real extracted
rootfs, the same 26 as before the split; run-tests.sh reports 7 passed /
1 failed / 1 skipped, identical to the pre-split run on this host (shellcheck
is absent here and lives in the build image, and no stock package is
configured in this worktree); every referenced path in the repo resolves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr Shyshatskyi
|
12bbd1e7dc |
Retire the probe profile: there is one package now
The probe profile was a bring-up aid. It set BUILD_APPLY=0, reinstalled the stock component byte-for-byte and wrote a report to the stick, so you could prove the update chain worked on your own machine -- the filename glob, the DES3 key, the md5 gate, run.sh execution -- before flashing anything that changed a file. That rehearsal has served its purpose. With it gone there was exactly one profile left, so the whole PROFILE indirection goes too: profiles/ is deleted and its six BUILD_* flags are plain defaults in bin/common.sh, which also means config.env now overrides them instead of being silently overridden. make default folds into make build; all-profiles and RECOVERY_PROFILE are gone with it. What is NOT gone is the undo button: flashing the stock FlashForge package still restores every file the mod touches, and case-recovery.sh still proves it. What is gone is the dry run. Note the loss: PROG_MB/DATA_MB used to come out of the probe report. The docs now point at df -h over ssh, which you only have after a successful flash. |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
5bdf78109a |
Retire firmwareExe: own the web stack, and say what goes with it
The mod replaces FlashForge's application outright, so the reasoning that still framed the web stack as coexisting with it was wrong on its own terms. * nginx and moonraker are documented as the only web stack on the printer. MOD_WEB=0 is a debugging switch that leaves ssh and the touchscreen, not a way back to stock -- that is a stock FlashForge package. The own-prefix rule stays, for the reason that is actually true: /usr/prog is the firmware partition and every FlashForge package reinstalls it, so a config written there is undone by the next stock flash. * Configs the mod ships now update when you have not edited them. A live file still byte-identical to the one the last package wrote is replaced; one you changed is kept and the new version lands beside it as .mod-new. $MODDIR/config-installed holds that last-written copy. Without this a config we own could only ever be written once: it exists on the second flash, so it would land as .mod-new forever and updates would never reach the printer. * Dropped the three dead /server/files/ alias blocks from nginx.conf. The regex location above them matches /server/ too and nginx gives regex locations precedence, so they never fired -- and had they fired, serving those paths from disk would 405 the DELETE and POST that Mainsail sends to the very same URLs. * Recorded what dies with firmwareExe, in docs/notes/25 and in the README: the :8898 REST API (FlashPrint, Orca's FlashForge profile, the mobile app) and the :8080 camera stream. The camera is the one that looks restorable and is not -- the MJPEG server is compiled into firmwareExe (its own accept loop, `GET /?action=stream`, boundarydonotcross) rather than being mjpg-streamer, so there is no daemon left to point a [webcam] at. assets/moonraker.conf carries the block, commented, for when a streamer is built for this board. Also in this change: the MOD_UI switch and the stock-UI path are gone, the chamber-heater model gate arrives with ff-model.cfg/ff-chamber.cfg, and the tests follow. make test: 32 passed, 1 failed. The failure is the probe end-to-end, which dies at "mount: failed to setup loop device for /stick.img" -- leaked loop devices on this build host, not the package. Probe installs nothing, and the default profile's install -> re-install -> boot passes in the same run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
a9299397f4 |
Build the toolchanger from this repo instead of a TOOLCHANGE= path
bin/patch.sh now copies the ff_*.py extras, the ff-*.cfg configs and the HelixScreen printer-database entry out of payload/, the way it already handled report.sh and the nginx/moonraker assets. Both call sites also lose a silent-skip guard. They tested for the files under $TOOLCHANGE and quietly shipped a package without the toolchanger if the path was wrong or unset -- worth having when the source was a checkout that might not be there, wrong now that the files are committed next to the script. A missing file here is a broken checkout, and cp should say so. CI and the test fixture had TOOLCHANGE="" set precisely to trigger that skip, so drop the line; both build the probe profile, which never reaches this step anyway. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GgNhSa1Qvwvxb2nQXT6b1Y |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
22d16f984e |
Run the printer's own boot script for the update test, and cut the suite to 5m
The install test used to re-implement app_startup.sh: it pulled the glob,
MACHINE and PID out of it with sed, then called the installer itself. A
mistake in our reading of the boot script could never be caught that way.
The package now sits on a real FAT filesystem on a loop device exposed as
/dev/sda1, and app_startup.sh runs verbatim through three boots -- stick in,
stick still in, stick pulled. The last one asserts what actually matters:
nginx and moonraker up, klipper launched, the UI in the foreground where
app_startup.sh's own 5-second ps watchdog can see it.
One substitution, declared in .SIMULATED: the printer's kernel has nls_cp936
and a container kernel does not, so /bin/mount retries without codepage= if
the real options are rejected.
Speed. The image carried the partitions but not the stock install, and
test-env.sh sourced the config files after the environment -- so PRINTER_IMAGE
on the command line was overwritten by the empty value in test.env and the
published image was never used. Measured, per replica case: 22s to unpack the
factory image, 37s to install the stock package, 0.7s with the image. The
stock install is now baked in at image build time (bake.sh -- it needs binfmt
and chroot, so it runs privileged and the result is committed) and recorded in
/usr/prog/.BASELINE; a run asking for a different package reinstalls.
make test: ~20m -> 5m26s. Every section header now carries elapsed seconds.
Dropped four checks that could not fail:
* test-abi's execution half was `qemu "$f" -h || [ $? -lt 126 ]`, which
accepts nearly every exit status
* test-ash's md5sum -s probe printed "inconclusive" on every path
* run-tests.sh's per-file `sh -n` "POSIX" loop ran bash, and its bashism
grep flagged `local`, which busybox ash supports
* CI's shellcheck ended in `|| true`
And one that mattered more: the release workflow's final sim-install loop
ignored every exit status, and sim-install.sh exits 0 on a skip -- so the last
gate before publishing could not fail. It now stops on the first failure and
runs with REQUIRE_PRINTER_SIM=1, which turns skips into failures throughout.
Also: the profile lists in run-tests.sh, RECOVERY_PROFILE and release.yml
still said ssh/web/full/helix and would have failed the release job; work/.sim
was shared, so two suites at once truncated each other's packages; and the
fixture half left a 380KB package in work/out where make test-install picks it
up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GgNhSa1Qvwvxb2nQXT6b1Y
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
a5300bde12 |
Split the repo into ship / build / test lanes, and add the printer replica
Two strands, both landing here because they had grown into each other in the
working tree.
The printer replica: the printer's real rootfs.squashfs chrooted under
qemu-mipsel, with /usr/prog and /usr/data from the factory image, so a package
is installed by FlashForge's own updater running on the printer's own busybox,
tar, md5sum and unTar. There are no stubs -- if the real prog partition is
missing the harness refuses to run rather than test against mocks. A release
workflow is blocked on it.
The lanes: a file's directory now says whether it can ever reach a printer.
payload/, assets/ ship
bin/, profiles/ build the package
test/, docs/ neither, ever
* the replica's own infrastructure moved out of bin/ and docker/ and in
beside the harness it belongs to (test/extract-rootfs.sh,
test/build-printer-image.sh, test/printer/Dockerfile{,.full})
* the Makefile has two runners: only RUNSIM gets the docker socket and the
replica's variables, so a build cannot reach the docker daemon at all
* replica settings moved to test.env, which nothing in a package reads
* build-time flags renamed BUILD_* (APPLY, REPORT, KLIPPER, TOOLCHANGE,
MAINSAIL, HELIX). MOD_* now means only the runtime switches written into
anvil.conf and re-read by the printer at every boot
* verify.sh fails if any file in a package is byte-identical to one in
bin/, test/, docker/ or profiles/ -- by content, because a name blacklist
would trip over whatever Mainsail happens to call common.sh
Fixed while mapping it:
* BUILD_REPORT was read only inside the BUILD_APPLY=0 branch, so four
profiles declared it and got nothing. It is honoured on both paths now,
and off outside probe: report.sh copies /etc, passwd and shadow onto the
USB stick, which is a diagnostic, not something to hand out.
* three checks in the brick lint had been quietly dead. Two were guarded on
a payload/boot.sh replaced by the firmwareExe wrapper long ago; the third
was the file glob itself, which matched neither firmwareExe nor the
init.d/S* scripts -- the UI wrapper and every service went unscanned. A
missing file is now a failure rather than a silent pass.
* run-tests.sh overwrote config.env and copied it back, leaving the config
you edited one crashed run away from being a fixture. It uses CONFIG_ENV.
* bin/make-uninstall.sh does not exist and never did; README, run-pre.sh and
full.env all pointed at it. Recovery is the stock package, which is what
test-recovery actually proves.
README is now for someone who wants to flash their printer: what it is, which
file, how to flash, what to do when it goes wrong. The rest moved to
docs/how-it-works.md, docs/building.md and docs/testing.md.
make test: 89 passed, 0 failed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|