mirror of
https://github.com/Klipper4FlashForge/firmware.git
synced 2026-10-19 03:49:50 -05:00
master
50
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Oleksandr Shyshatskyi
|
80a4d64ffd |
Prepare next firmware release
Add Fluidd and custom boot scripts, migrate DC24V control, update the web UIs and filament macros, improve startup status reporting, and expand upgrade and replica coverage. |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
9973019ec8 |
A printer ships knowing where its updates come from
Phase 3 of docs/notes/85-packaging.md was three things: signing, a URL the
printer asks, and a tree at that URL. The migration landed the first. This is
the second, and the tooling for the third.
THE POINTER HAS TO SHIP BEFORE THE HOST EXISTS, which is why this lands with
nothing serving yet. A feed cannot tell a printer where the feed is: a machine
installed without etc/apk/repositories needs a USB stick before it can ever
fetch anything, which is the thing packages were adopted to stop. So the file
travels in the .tgz, naming a host that does not answer yet, and every printer
installed from today starts fetching the day one does.
The cost of that is bounded, and was read out of apk's source rather than
assumed: an unreachable repository fails `apk upgrade` and nothing else --
apk_db_repository_check is called from app_upgrade.c and not from app_add.c,
so `apk add ./file.apk` does not consult a repository at all.
http://reforge.8941973.xyz/apk/mipsel_xburst2/anvil.adb
bin/common.sh's FEED_URL is the base, config.env can override it for a fork,
and pkgs/anvil-core generates the file from it beside the public key it
already shipped. Three things about that URL:
* IT NAMES THE INDEX FILE, not a directory. src/repoparser.c reads a line
ending .adb as a v3 index in place, with the packages as siblings named by
the pkgname-spec mkndx wrote into it; anything else sends apk looking for
<url>/<arch>/APKINDEX.tar.gz. That is the layout this repo already builds
and the stick already uses, so the network feed is the same shape --
exercised by every payload build rather than only in production. The v3
directory form would work too, but only with
`mkndx --pkgname-spec '${arch}/${name}-${version}.apk'`.
* THE ARCH IS A DIRECTORY. Package file names carry no architecture, so two
architectures served from one directory would be two files with one name.
* PLAIN HTTP. libfetch defaults to SSL_VERIFY_PEER and this printer has no
certificate bundle -- the same gap tools/python/README.md records for
CPython. Every package is signed, and the signature rather than the
transport is what decides whether a printer installs one.
bin/publish-feed.sh writes the matching tree and optionally rsyncs it.
Packages go first and the index last, because the index names every package by
sha256 and a mirror carrying a new index with old packages hands printers a
checksum for a file that 404s. Nothing is deleted at the far end. It refuses
an unsigned feed unless ALLOW_UNSIGNED=1: a printer verifies against the key
that shipped in its own anvil-core, so an unsigned feed is one no printer
installs from. The release workflow attaches the same tree as
reforge-apk-feed.tar.gz, so a host can be filled from any release -- including
ones tagged before the host existed -- without rebuilding.
THE REPLICA STOPPED WRITING THE FILE IT USED TO DELETE.
case-build-payload.sh wrote its own etc/apk/repositories naming /mnt and
removed it before tarring; it now passes --repositories-file with a path on
the stick, which apk reads INSTEAD of the default (database.c consults the
default file and repositories.d only when no file is named). So what ships is
whatever anvil-core installed and the build never reads it. The `apk info`
line takes the same flag: without it that one command reads the shipped file
and reaches for the network, which a build's output must not depend on.
Measured: 47 packages built from a cold cache, the payload assembled in the
replica twice, qa/static at 319 passed. Two of those are new -- the default
URL in common.sh (read from the file, not resolved, so a fork setting
FEED_URL in config.env still passes) and the built payload carrying a
package-owned entry that ends /mipsel_xburst2/anvil.adb.
What is left is operational: a host answering at $FEED_URL, and who runs
publish-feed.sh -- a release step with a deploy secret, or a person with the
tarball.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017rgrDeu9YAUuMM2G7gKTpL
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
f11b63b6c1 |
Give apk a version it can parse in CI, and say so when it cannot
ci.yml writes MOD_VER=ci into config.env, and MOD_VER is anvil-core's PKG_VERSION. apk's version grammar wants a digit first, so mkpkg refused it -- with "info field 'version' has invalid value", naming neither the package nor the value nor where it came from. A free-form version was fine for .ipk, which is why the line survived the migration unread. MOD_VER=0 now. It is pinned rather than left to the date default so two runs of one tree produce the same bytes, and 0 says "not a release" as plainly as the word did while ordering below every real version. emit_apk checks the version before calling mkpkg and names the package, the offending value and the config.env setting to fix. The grammar test covers the recipes, and could not have caught this: the value comes from config.env, so anvil-core answers with whatever the machine running the test happens to say -- a date, locally, always valid. The new gate reads MOD_VER out of the workflow files themselves, which is the only place that value is written down, with a companion asserting a workflow still pins one at all so the check cannot pass by having nothing to check. busybox's pkg.conf expanded $BUSYBOX_BIN bare and let a failed sha256sum through a pipeline. Sourced under `set -euo pipefail` it therefore failed -- correctly skipping the recipe, since pkg_recipes evaluates PKG_WHEN in a subshell, but printing "BUSYBOX_BIN: unbound variable" twice into every CI log immediately above the first real failure, where it reads like the cause. With that fixed every recipe reads under the build's own flags, so the unsigned-recipe gate now covers all of them rather than only the ones a build would make: a recipe that cannot be read is dropped by that subshell rather than reported, so the failure mode is a package silently leaving the feed. Verified with the production key: 290 static, 135 replica, 47 signed packages and a payload. MOD_VER=ci reproduced locally against the new message, and MOD_VER=0 built and pruned clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013rbKEyLXrzSE6wm3iNApVp |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
734a04db45 |
Make the apk feed buildable by CI, and signable by a release
PR #10 was green on this machine and red on both CI jobs. Three separate faults, none of which a local signed build could show. apk-tools is cloned from gitlab.alpinelinux.org, which answers a GitHub runner with HTTP 418 or with nothing until the connection times out. It clones from the GitHub mirror now. Which host serves it cannot change what is fetched: the pin is a commit sha, and git names a commit by the hash of its content, so a mirror answering with this sha is serving this tree or is not answering. get_git also sets the remote url on every run, so a checkout made before this commit does not keep fetching from the host that refuses. anvil-core's cache key hashed the feed's public key as sha256sum "${APK_SIGN_KEY:-/dev/null}.pub" 2>/dev/null | cut ... With no key that file is absent, sha256sum is non-zero, pipefail makes the pipeline non-zero, and build-packages.sh -- which sources every pkg.conf under `set -euo pipefail` -- died before emitting a single package. The 2>/dev/null ate the reason, so `make packages` exited 1 having said only that it had sealed a cache entry. Every signed build passed. Every CI build is unsigned. It is now pkg_signkey_hash, which answers `unsigned` and succeeds. The build container mounts the checkout and nothing else, so a key at the documented ~/.anvil/anvil.rsa was visible in the developer's shell and invisible to apk. The Makefile reads APK_SIGN_KEY out of $(CONFIG_ENV) and bind-mounts its directory read-only. This survived review because the key used while writing the migration sat under work/, inside the one directory already mounted. Gates for all three, each verified against its own bug by reverting the fix under it. The recipe-reads-unsigned test runs pkg_conf under the same flags build-packages.sh uses -- the existing _conf helper does not, and would have stayed green throughout. Its companion asserts the recipe SET does not change with the key, because pkg_recipes evaluates PKG_WHEN in a subshell and a recipe that fails to source is dropped rather than reported: broken only when unsigned, a package would leave the feed with nothing turning red. Both are scoped to what pkg_recipes lists, so busybox -- gated off whenever there is no stock firmware to take one from -- is not asked a question it cannot answer. release.yml takes the private key PEM from secrets.APK_SIGN_KEY, writes it to $RUNNER_TEMP outside the checkout where no later step can archive it into a release, and derives the public half rather than storing a second secret that could drift out of step with the first. Unset, the release builds unsigned with a warning: a fork has no such secret, and firmware nobody else can build is worse than a feed untrusted on a stick the printer was handed by hand. Verified from a cold work/host and work/packages, three ways: no key (the CI case), a key inside the tree, and a key outside it at the documented location. Each gives 47 packages plus the index, a payload, 286 static and 135 replica tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013rbKEyLXrzSE6wm3iNApVp |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
f8b925e48f |
Replace opkg with apk, and sign the feed
docs/notes/85-packaging.md chose opkg in 2026-08 on one argument: every package
is cross-built here anyway, so the winner is whichever format is cheapest to
PRODUCE -- and .ipk had an upstream tool where .apk meant abuild, a keypair and
an Alpine host. Two lines of that comparison were wrong.
apk mkpkg is an applet of apk itself: -I key:value metadata, -F directory, no
keypair, no abuild, no Alpine host. The "making a package" row described
apk-tools 2. And the on-device client was counted a wash when it is better than
one -- openssl and zlib were already cross-built for CPython, and libarchive
existed in this tree only for opkg, so adopting apk REMOVES a cross-build.
What made it possible at all is that apk resolves its database relative to
--root and uses the same root as the file destination, while / here is a
read-only squashfs -- measured: neither /lib/apk nor /etc/apk can be created.
pkgs/3rdparty/apk-tools/prefix.patch splits the two. Upstream already carries
root_fd and dest_fd separately and merely aliases them in context.c, so the
patch defaults the database root to $MODDIR and leaves files extracting at /.
That is opkg's compile-time VARDIR plus `dest root /`, in apk's spelling.
THE FEED IS SIGNED NOW, which it never was: opkg signed the index and
bin/build-packages.sh signed nothing, so nothing was verified on a printer.
apk verifies with raw PEM public keys, not X.509 -- measured on the machine,
where every certificate directory is empty. ./bin/apk-keygen.sh makes an RSA
pair, APK_SIGN_KEY in config.env points at the private half, and the public
half both seeds the USB stick (so the first install can verify) and ships in
anvil-core (so the printer keeps trusting). An empty APK_SIGN_KEY still builds
an unsigned feed, deliberately: `make packages` has to run on a bare checkout
or the packaging lane stops being a CI gate.
WHAT IT COST is a carried patch, against an internal with one consumer, on a
branch mid v2->v3 transition, on an architecture with no upstream apk port to
share the rebasing. That is the standing risk and it does not go away.
Four bugs this shook out, all silent, all found by a gate rather than by
reading:
* The prune deleted seventeen packages it had just built. The all -> noarch
mapping went into the emitter and not into pkg_pkgfile, so the prune looked
for a name nothing wrote. `make packages` exited 0.
* Whitespace-separated Depends dropped nineteen packages. opkg's pkg_parse.c
splits that field on ',' and nothing else, so the whole list became one
dependency whose name contained spaces and resolved to nothing -- and opkg
neither warned nor failed. `make build` shipped a payload with no Tornado
in it. test_the_payload_is_dependency_closed is what catches that now.
* Ownership was recorded as `nobody`. apk resolves an owner to a NAME against
<--root>/etc/passwd and the build lane runs --user with no passwd entry, so
every file in every package would have installed as uid 65534, with no
error at build or install time. build-packages.sh writes a two-line
synthetic passwd naming the build user root and points --root at it.
* Four recipes carried versions apk cannot parse -- two spelled a commit
+git<sha> where the grammar's hash slot is ~, x264 had a snapshot name
whose -2245 reads as a malformed release, and helixscreen had
0.99.115.creator5.3 where apk allows one trailing letter and then only a
suffix from a closed set of nine words. All four were valid opkg versions,
which is why nothing had noticed. pkg_version_ok gates every recipe now.
A v3 .apk begins with ADBd magic and is not a tar stream, so there is no apk
equivalent of the `ar -p | gunzip | tar` that hand-unpacked anvil-opkg: you
cannot open a package without a package manager. bin/build-payload.py puts the
cross-built binary on the stick at /mnt/apk instead. It is the same build
anvil-apk-tools contains, that package installs normally, and the bootstrap
copy never lands.
Two gates retired rather than moved. apk's installed database is plain text
(F: sets a directory, R: names a file in it), so the payload-ownership check
reads one file where it used to glob info/*.list. And it carries no clock at
all -- apk records no install time and mkpkg sets no build-time -- so the
Installed-Time normalisation case-build-payload.sh did on every build is gone,
and the test asserts the ABSENCE of a t: line instead. Clock-free by
construction rather than by repair.
Also removed: PKG_SECTION, which 45 recipes declared and nothing read once
apk's schema had no field for it, and the `make apk-host` target -- the
packager is built on demand by bin/build-packages.sh and stamped on the pin,
the way pkg_toolchain and pkg_buildpython already work.
Verified from a cold work/host and work/packages: 47 packages built and signed,
`make build` installs 35 and verifies them against the shipped public key,
`make qa-static` 195 passed, `make qa-replica` 135 passed -- the printer
installing off a simulated USB stick and booting. The installed set was checked
against the declared closure of the eleven roots: 35 expected, 35 installed,
nothing extra and nothing missing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013rbKEyLXrzSE6wm3iNApVp
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
ba01b8f9f0 |
The release e2e installed both models into one Creator5Pro replica
`for p in dist/*.tgz` fed the replica the Creator5 package first. That machine's own app_startup.sh globs /mnt/Creator5Pro-*.tgz, found nothing, and fell through to a normal boot -- so all 123 replica tests errored with "could not install the package into a replica", about a package that was built correctly and whose model gate was working exactly as designed. The loop is older than the pytest lane and used to be fine: sim-install.py resolved a machine per package. The replica lane takes PRINTER_IMAGE from test.env, and there is one of those. So the end-to-end runs on the model the replica IS, and a Creator5 end-to-end waits on a Creator5 replica image that nothing publishes yet. Both models are still built and both are still attached to the release; only the install test narrows. upload-artifact moves above that step while I am here, so a run that produced packages hands them over even when the lane fails. The release gate is unchanged: Publish comes after, and a failed step still stops the job before it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S64uRMdZbj9xPMns2dqMG8 |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
73279f66f3 |
A backtick in a comment ran make vendor into config.env
The printer-sim job wrote its config.env from an UNQUOTED heredoc, and one
of the comments inside it says
# `make vendor` puts the pinned download. Setting them here would be
Those backticks are command substitution. The step ran `make vendor`,
captured its stdout, and pasted the fetch log into the config file it was
supposed to be writing -- so config.env line 6 came out as
>> fetch https://github.com/ghzserg/FF/releases/download/R/...
which sources as a redirect into a file named "fetch" plus a URL for a
command. Every run of that job died at 127 before it fetched anything.
The delimiter is quoted now and ROOT_PW_HASH loses the backslashes it only
needed to survive expansion. release.yml keeps its unquoted heredoc: it
interpolates $(case ...) and $PWD deliberately, and has no backticks inside.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S64uRMdZbj9xPMns2dqMG8
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
c3a64e0394 |
The stock firmware is a pinned download like everything else
config.env.example opened with DESKTOP="$HOME/Desktop", and four settings hung off it. Nothing in bin/, pkgs/, qa/ or installer/ ever read it: three of the four were commented-out overrides for files `make vendor` already fetches, and the fourth, BUSYBOX_BIN, names a binary with no source at all. What DESKTOP really did was carry the stock FlashForge .tgz -- the one input a person had to go and find before they could build anything. It did not have to be. .github/workflows/ci.yml has been fetching that exact file for as long as printer-sim has run, from ghzserg/FF, pinned by sha256, with its own get(), its own cache and its own secrets -- a second spelling of "download the firmware and check the hash" that no human path shared. The pins here are those pins, verified byte-for-byte against a fresh download of both assets. So versions.env pins the stock package the way it pins everything else, bin/fetch-assets.sh downloads it into vendor/, and bin/common.sh defaults STOCK_TGZ_* there. A cold checkout is now `cp config.env.example config.env && make build`, with nothing to edit. --stock is a separate flag from --all, not folded into it. The packaging lane wants no FlashForge package -- that is most of the point of it being runnable in CI on a bare checkout -- so `--all` stays "every payload piece, no firmware", `make build` asks for the one model it is building, and `make vendor` (--all --stock) takes both. The one download that will not overwrite an override. HELIX_TGZ and friends name a public release anyone can re-fetch in a minute; STOCK_TGZ_* names 93MB of proprietary firmware someone downloaded by hand, and clobbering it on a hash mismatch would destroy the only copy on the machine. It is skipped instead, and unchecked -- their file, their problem. CI loses its duplicate fetcher, its stock/ cache and the STOCK_URL_* secrets. A fork testing its own firmware re-pins versions.env, which works locally too; the secrets only ever worked in CI. qa/lib/config.py reads versions.env for the pinned filenames so stock_for() looks where the download actually lands. An explicit STOCK_TGZ_* still wins. An existing config.env keeps working untouched: its paths are now overrides, so the build skips the download and uses the file it always used. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DPU6RR4UJH2Qf9DR5hb2gg |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
b98552386b |
No FlashForge component ships; patch.sh is payload.sh
The software component existed to give the mod one exec hook. It is not needed for that any more, so a release now carries our installer and our payload and nothing else: 26MB smaller, 21MB of which was FlashForge's own firmwareExe travelling back to the partition it was already on, along with their klippy tree, their wifi module and their configs. WHAT IT GIVES BACK. /usr/prog/PROGRAM/software is no longer rewritten, so the version directory there stays FlashForge's with their genuine firmwareExe inside it -- and app_startup.sh's five-second watchdog, which copies one out of that directory when nothing by that name is running, has something to copy again. A printer whose payload fails to install now comes up on the stock UI instead of a blank screen. That recovery has been broken since we started deleting their binary out of a component we shipped; docs/how-it-works.md claimed it worked throughout. THE ROOT PASSWORD moves with it, because it used to ride in the component's shadow file. bin/pack.sh bakes ROOT_PW_HASH into the installer, single-quoted -- an unquoted $6$... would have the printer's shell expand $6 as a positional parameter and ship a truncated hash -- and the installer writes it to /usr/prog/etc/shadow. The random-password path asks one question instead of four: FlashForge ship the same root hash on every machine, so a live shadow still carrying it means nobody has set a password and it is ours to set. The .prev-root-hash recording, the shipped-vs-live comparison and the restore-it- afterwards step were all machinery for surviving a file that nothing overwrites any more, and are gone. pack.sh checks the hardcoded constant against the stock package it built from, because getting it wrong is silent and total: every printer would read as "already set" and none would ever get a password. bin/patch.sh is bin/payload.sh. It patched nothing any more -- no splice, no shadow edit, no firmwareExe deletion -- and what is left is the payload: the feed installed by the printer's own opkg, the seeded config, the manifest. Its sections are renumbered 1-5 with no gaps, and the five tombstone comments marking where removed sections used to be are gone with them. test_recipe_layout.py loses six of its nine tests. Each compared the tree against a description of the tree -- no fourth directory kind, each kind still used, 3rdparty carries no files of ours, installer/ holds exactly two named files -- and none could go red for a reason a printer would notice, while all six went red the moment the layout legitimately changed. The two that bite are kept: a payload/ path escaping $MODDIR, and a recipe name at both levels, which makes pkg_dir silently build the wrong package. SW_VER is deleted from config.env.example and both workflows: it stamped a component that is not built. Gates: 112 static tests pass. The generated installer is verified after substitution -- all four lines present, and `sh -n` over the result, because a hash with a stray character in it would otherwise reach a printer as a syntax error whose only symptom is a package that installs nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CH7Yb4wuPhnppHAB25UX7e |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
56b5b6c334 |
The printer image is the only replica; make rootfs was a second way nothing took
`make rootfs` unsquashed rootfs.squashfs out of the stock package into
work/rootfs, and a thin container bind-mounted it with PROG_DUMP supplying
/usr/prog from the factory image. Every caller set PRINTER_IMAGE anyway --
both workflows wrote it into test.env BEFORE running `make rootfs`, so CI
unpacked a 93MB package, unsquashed a rootfs and downloaded a 182MB factory
image on every push and read none of it. build-printer-image.sh never used
work/rootfs either: Dockerfile.full does its own extraction from the same
public firmware and needs no stock package at all.
So the fallback went, and PROG_DUMP with it -- it was only ever read when the
replica was built locally, and both implementations already passed it empty
otherwise. Gone: the target, extract-rootfs.py, gates.extract_rootfs and its
two tar helpers, the thin printer/Dockerfile, the `prebuilt` flag threaded
through both Replica implementations, and Replica.start's want_output, whose
only message was that the local build was about to be slow.
PRINTER_IMAGE is pinned to 1.9.7-1.2.9-20260810 rather than :latest. It is
now the entire substrate the brick-safety gates run on and the only input
nothing checksums, so a floating tag would let it change under a green suite
-- the drift the sha256 pins in both workflows exist to prevent.
THREE THINGS THAT COULD NOT HAVE PASSED, found while doing it:
* Neither workflow built the feed. bin/patch.sh has refused to start
without one since
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
57c776ee63 |
test/ is deleted; qa/ is the only suite
The eight files were already gone from the working tree. What this commit does
is everything that had to follow them, because nothing else had been updated
and three of the four things pointing at test/ were on a build or release path.
WHAT WAS STILL WIRED TO IT
pkgs/klipper/build.sh ran test/test-chelper.py under `|| pkg_die`. With the
file gone `make packages` would have died on every
klipper build.
bin/verify.sh ran the same script and read a non-zero exit as
"c_helper.so does not match the shipped klippy tree
(stale build)" -- a false red on every release verify
rather than an error about a missing file. Its
ship-boundary check also globbed `find bin test
docker`; that is `find bin docker` now.
pytest.ini testpaths = test, so a bare `pytest` collected
nothing at all. It is `qa` now, which is what the
comment above it was already describing.
qa/static/ PY_GLOBS listed test/*.py and test/integration/*.py,
test_shell_syntax.py and _files() makes an empty glob a collection error
on purpose -- so the whole static lane failed to
collect, taking 93 unrelated tests with it.
Also dropped: the `pytest -- the host-side tree` step in ci.yml, the `test-py`
target with its .PHONY entry and two help lines, and qa/conftest.py's opening
paragraph explaining why qa/ sat beside test/ rather than inside it.
THE CHELPER SYMBOL CHECK IS GONE, NOT MOVED
It compared the .so's dynamic symbols against what klippy cdefs, and it was the
only check for a c_helper.so built from older sources than the klippy tree
beside it -- the failure that shipped once and bricked klippy startup on
hardware, because cffi resolves lazily and a stale .so imports cleanly.
qa/replica/test_install.py covers the ELF arch and nan2008 and nothing else.
What stands in its place is structural rather than a gate: pkgs/klipper
compiles the .so from the chelper sources of the very tree it ships, so the
mismatch is not something the recipe can produce. The comments in
pkgs/klipper/build.sh and docs/building.md now say that is the whole defence.
DOCS
docs/testing.md's "what is left in test/" section is now "test/ is gone", and
it names what went uncovered rather than leaving it to be found: the tool-frame
arithmetic in ff_toolchange, the chamber macros' per-model branching, the stamp
discipline in ff-startup.py, and the verification G-code against the shipped
macros. docs/hardware-testing.md said `make test-py` keeps the two
gcode/creator5-*.gcode files honest; it now says nothing does, and to read them
by hand when a macro is renamed. Paths corrected in docs/building.md,
docs/printer-replica.md and docs/qa-migration.md's intro.
Verified: `pytest` collects 366; qa/static is 136 passed, up from 43 because
the lane could not collect before this. Its 110 errors are this checkout's
missing vendor/opkg-utils, the same cause before and after. bin/verify.sh,
pkgs/klipper/build.sh and the Makefile parse; ci.yml is valid YAML.
Not touched, and stale before this commit: tools/python/README.md and
tools/supervisor/README.md still document test/integration/printer/case-*.sh
and a printer-exec.py that earlier commits removed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016bLGYpVAeVzAX6NeAxcU42
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
c185ef3285 |
The replica is a build tool; test/ is 28 unit tests
Two changes that turned out to be one.
MOVED, NOT DELETED. `test/ffsim/` and `test/integration/printer/` are
`tools/replica/`. They were never tests: bin/patch.sh calls
bin/build-payload.py, which starts a replica and assembles the payload by
running the PRINTER'S OWN opkg against / inside it -- so the "test harness"
was on the build path, and deleting it as asked would have broken `make
build`. qa/lib/replica.py drives the same directory unmodified. No content
changed; what changed is qa/lib/replica.py's build directory, three launcher
sys.path lines, the Makefile targets and the CI shellcheck glob.
tools/replica/README.md says who drives it and why it is not under test/.
CUT TO 28. The five surviving modules were 136 tests; they are 28, chosen on
one question: would this failure be SILENT? What is kept is the mistakes that
run perfectly and are wrong --
test_startup 8 no stamp unless the values are verifiably saved (a
stamp written early is a printer that has lost its
factory calibration for good), and the handover order
test_tool_transform 7 every one is a move to the wrong place: a sign, a
frame left applied while probing, a stale gcode_move
cache, a config a calibrated printer cannot parse
test_ffscreen 5 the four ways it can do harm rather than look wrong
test_chamber 4 the chamber mistakes that cost a print before anyone
sees them, plus the parse gate they rest on
test_gcode 4 a renamed macro, and the safe file being cold
What went: the panel narration, the geometry probe and parse_geometry, the
z_adjust and per-job-Z bookkeeping, the config layout checks, the G-code
bounds checks. Each is either a wrong number shown to a person, or a failure
the printer reports loudly the first time -- and qa/replica/test_boot_screen.py
already renders the real frames on the real machine.
The rewrite is not a delete: assertions were folded so a kept test asks its
question in every direction it can be got wrong (both models, both rotations,
all four ways the migration can fail), and each file's header now says what it
dropped and why, where the next person will open it.
Also gone: conftest's `rootfs` fixture, orphaned two commits ago. Nothing under
test/ needs the firmware, docker, or a network any more.
28 host-side tests pass in 0.15s. qa/static unchanged at 218. `make rootfs`
re-verified end to end from its new path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzLebCrvc8ssWR5JuEsha4
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
465c9902f1 |
test/: drop the seven modules whose subject was the fake
Eight files, 46 tests and make-stock-fixture.sh. Each of the seven test modules built a stand-in -- a klippy stub, an argparse Namespace, a config tree -- elaborate enough that what they pinned down was the stand-in's behaviour rather than the printer's. What is left under test/ is the Klipper config gate and the three modules whose subject is our own Python: ff-startup, ffscreen, and the tool transform. make-stock-fixture.sh goes with them. docs/qa-migration.md argued for keeping it as the input a future build-path test would need, but it fed a gate that no longer exists, and a fixture generator with no consumer rots exactly as silently as a test that asserts nothing. What it had learnt is recorded in the doc so whoever ports the build path does not have to find it twice. The coverage this gives up is written down rather than glossed: a new section in docs/qa-migration.md names every dropped file, what it asserted, and what now asks the question instead -- for four of them, nothing. The one worth watching is the shebang-executable check, which qa/ makes only of qa/replica/actions/*.sh. Falling out of it: the rootfs fixture in test/conftest.py had no users left, and nothing under test/ needs the proprietary firmware any more. Comments in qa/static/test_probes.py, qa/replica/test_mcu_bringup.py, bin/patch.sh and ci.yml that pointed at the deleted files say what is true now; the tree listing in docs/building.md was stale from the run-tests.py removal and is rewritten. 136 host-side tests pass. qa/static is unchanged at 218. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzLebCrvc8ssWR5JuEsha4 |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
6aa622d2ff |
6,280 lines of case scripts reported 13 bits. They are 118 tests now.
test/run-tests.py is gone, and with it the bespoke harness that wrapped
pytest, re-parsed its JUnit XML, and drove thirteen replica case scripts to
get one boolean back from each. Every gate it ran is either a module under
qa/replica/ or deliberately retired.
PORTED. case-install.sh (579 lines, 1 bit) is test_install.py, 22 tests, asked
of the machine the printer's own app_startup.sh built rather than one the
harness assembled. case-upgrade.sh is 21, case-supervisor.sh 31,
case-boot-screen.sh 17, case-mcu-bringup.sh 8, and case-nginx.sh plus
case-moonraker313-s6.sh are test_web.py's 7.
RETIRED, subject deleted by the s6-rc switch: case-services.sh, case-nginx.sh,
case-moonraker.sh, case-moonraker313-s6.sh, case-camera.sh and
case-priority.sh drove $MODDIR/init.d/S* or sourced anvil-service.sh, neither
of which exists. Decided against real output -- `make test-services` failed
with "the payload ships no anvil-service.sh". case-pyext.sh and
case-moonraker313.sh went too: no gate, no target, no CI step, never run once.
RETIRED as a coverage decision: case-python.sh (test_web.py runs real Moonraker
on that interpreter, which is harder), case-libpath.sh, and case-recovery.sh.
The last is the brick-safety rollback and has no replacement; it is recorded in
docs/qa-migration.md rather than left to be discovered.
THREE ASSERTIONS HAD STOPPED MEANING ANYTHING, and each had been green for
months:
* case-mcu-bringup.sh's negative control had INVERTED. It asserted the
bring-up fails without LD_LIBRARY_PATH; CPython is --disable-shared now and
needs no library path, so the case's own skip branch was the one firing.
* case-upgrade.sh's headline assertion was VACUOUS. run-append.sh:150 does
`rm -rf $MODDIR/init.d` unconditionally, outside the manifest branch, so
"the update removed the stale S60web" passed regardless of the manifest
logic it was written to test.
* case-boot-screen.sh CONFIRMED ITS OWN ARITHMETIC, computing expected pixel
packing by calling ffscreen._pixel inside the replica -- a swapped R/B
matched itself.
case-install.sh's boot-log grep is not reproduced for the same reason: the log
line names every service in the boot set including the ones that then fail, so
grepping it cannot go red. The compiled database is asked instead.
TWO REAL BUGS FOUND ON THE WAY.
assemble.sh attached /stick.img with `losetup` and wrote the device to
/stick.loop, which NOTHING HAS EVER READ. Loop devices are the host's, not the
container's, so every USB_STICK=1 run leaked one permanently and a WSL2 kernel
has thirteen. Once they were gone every replica run failed with "failed to
setup loop device", which looks like a docker fault and is not. Detached now in
entrypoint.sh on the way out, and in qa's stop() for the held-open containers
that never take that path.
test_startup.py was 43 tests red: it built its argparse.Namespace by hand and
had never been taught --only-bringup / --no-bringup, and its FakeKlipper still
modelled start.sh plus a separate stop when klipper is restarted with one
`s6-svc -wr -t`.
CI AND RELEASE. The replica job was getting its package as a side effect of
run-tests.py running unpack/patch/pack; that is an explicit step now, as the
comment there predicted it would have to be. release.yml tested the exact files
it ships through sim-install.py, whose exit status the loop was discarding --
it is `REAL_PKG=<pkg> pytest ./qa/replica` per model now, 22 named results each.
WHAT IS NOT TESTED ANY MORE: bin/unpack.sh, patch.sh, pack.sh and verify.sh.
The packaging build on a synthetic stock package went with the runner and has
no counterpart in qa/. That is the path producing the .tgz a user flashes.
make-stock-fixture.sh is kept as the input a replacement needs.
Measured: qa/static 246 passed, qa/replica 110 passed and 8 blocked on hardware
the replica has not got (no /dev/ttyS*, no /dev/video*, so ok-all is
unreachable), test/ pytest 185 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013BTAXgjtF8GdhqSxvdXCFW
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
c7b3a2a6c4 |
Catch up to master: the docs are a built site now
master rewrote the documentation into a nine-page manual with an mkdocs build, and this branch forked before it. Five commits, and the conflicts were all places where both sides had edited the same page. .gitignore and .github/workflows/ci.yml keep BOTH sides: master adds the docs/index.md generation and a `docs` job that runs `mkdocs build --strict`, this branch adds the qa suite step. They are independent. docs/hardware-testing.md and docs/toolchange.md take master's: it moved the safety prose into docs/printing.md rather than deleting it -- checked, the text is there -- so keeping our copy would have duplicated a page that CI now link-checks. The chamber-config and s6-rc rows this branch added to docs/toolchange.md survived; they are in the table, which master did not touch. Left for a follow-up rather than folded in here, so this merge stays readable: docs/building.md still lists the seven init.d/S* scripts anvil-core no longer ships, and testing.md and qa-migration.md still name S62moonraker, S60nginx and S80ui. mkdocs --strict will not catch those -- it checks links, not claims. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VpUmBb4xtmnmtTosonqeNi |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
5b463b889a |
Five switches that were only ever set to 1
MOD_WEB, MOD_CAM, MOD_UI, MOD_SSH and MOD_WIFI all defaulted to on, and what
each one really offered was a half-installed printer: Mainsail with no
moonraker behind it, a camera that would not stream, a screen dark on purpose,
a root password deliberately not set on a machine whose entire recovery story
is "ssh in". Nobody wants those states. What someone who sets one actually
wants is a stock printer, and that is one FlashForge package away.
What they cost was a second answer to every question. Each init script carried
a gate, each `down` file argued its existence from a flag rather than from the
ordering that was the real reason, and anvil.conf had five lines that told a
user they had a choice.
MOD_SSH was the clearest case: it never turned ssh off. dropbear is the
rootfs's own and listens either way, so all the flag did was decline to set a
password.
Each gate is replaced by the fact it was standing in front of, not by a
constant:
S60nginx, S62moonraker nothing -- the web stack is what this mod is
S65camera there is no video device, which the script already
finds out and reports
S50wifi there is no $WIFI_CONF, likewise
S80ui HelixScreen is not installed, which was already the
other half of the condition
To leave a piece out, leave it out: BUILD_MAINSAIL, BUILD_MOONRAKER and
BUILD_HELIX decide what goes into a build, and `opkg remove` decides what stays
on a printer. What survives in anvil.conf is what has a range rather than a
state -- MOD_CAM_RES/FPS/PORT and the NICE_* priorities -- plus MOD_SPLASH,
MOD_STARTUP, MOD_IMPORT and MOD_S6, which are untouched here.
The `down` files stay. They were argued from MOD_WEB and MOD_CAM, but the
argument never depended on them: s6-svscan starts a service the instant the
directory appears, which is before any init script has checked a prerequisite.
Their text now says so.
Measured: `make build` produces a payload with no live read of any of the five,
and qa-static is 265 passed with shellcheck over the edited init scripts. The
test suite still exercises the removed flags and has not been updated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gKRUCqj18TVF1Vz1oMpmC
|
||
|
Oleksandr Shyshatskyi
|
21cf19f2e2 |
BUILD_KLIPPER=stock kept a Klipper nothing else was written for
The flag chose between the Klipper4FlashForge fork and FlashForge's own v0.12 tree. Every other thing the mod ships had already stopped being neutral about that choice: the ff_*.py extras are the fork's package's own files, the ff-*.cfg declare sections only those extras implement, and c_helper.so is compiled from the tree it ships beside. A stock build was a printer with the mod's configuration and none of the code behind it -- and the previous commit turned that into a hard error, which is what showed the flag had nothing left to select. So it is gone rather than guarded. pkgs/klipper loses its PKG_WHEN, patch.sh section 1 loses the case and is straight-line, verify.sh section 7 stops being conditional -- a package with no klippy tree is now unrepresentable rather than merely gated -- and fetch-assets.sh pulls the fork tarball on every build, the way it already does s6 and CPython. Two fixture-lane repairs come with it. test/run-tests.py said BUILD_KLIPPER=stock to keep the lane off the network. That reason had already expired: since the payload became the feed installed, the lane runs build-packages first, and that needs the ~203MB Ingenic toolchain from vendor/ for CPython alone. The Klipper tarball is 27MB beside it, cached on versions.env like everything else. The lane now builds the fork, so verify.sh's ABI and symbol checks on c_helper.so run there for the first time. And the synthetic Moonraker tarball unpacked to moonraker-fixture/, while pkgs/moonraker/build.sh looks for moonraker-$MOONRAKER_VERSION/ -- the shape GitHub generates. That lane could therefore never build anvil-moonraker at all. It passed only while work/pkg/moonraker held a tree from a REAL build, because pkg_stamp keys on the version and not on the tarball; the first run against a cold cache died with "no moonraker/moonraker.py", a true statement about a tarball nobody could have made work. CI caches work/pkg on versions.env, so the first run after any pin bump would have hit it. Confirmed pre-existing by running the parent commit against the same cold cache: it fails identically. make qa-static 261 passed; make test 9 gates, 0 failed, with the fixture lane building Klipper and Moonraker from its own assets for the first time; CI's shellcheck gate clean. |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
7347f56418 |
docs/ rendered on GitHub and nowhere else. It is a built, versioned site now.
The alternative was a wiki, and a wiki is a separate git repo: a page would stop moving in the same commit and the same review as the code it describes, with nothing able to catch the drift, and there would be exactly one live version of it. This project ships a package per tag and tells owners which checkboxes have touched hardware; the docs for the build someone flashed have to be readable, so Read the Docs builds this repo per tag and `latest` off master, with the version switcher that follows from that. Nothing moved. The audience split that the pages already have -- owner pages against build-and-test pages -- is the nav in mkdocs.yml, so a reader lands in their own section and never falls out of it, while the 30+ relative links between README.md, docs/*.md and docs/notes/ keep working in the repo view and the links people have already shared keep resolving. notes/70-error-codes.md sits in the owner section because README.md was already sending owners there. Two things had to hold for that to be true, and hooks/mkdocs_readme.py is both. Links out of docs/ -- `../payload/klipper/extras/ff_tool.py` and 18 others in toolchange.md -- are written relative because that is what works where a contributor reads them, in the repo; they become GitHub URLs at build time, and a link whose target does not exist is left alone so the build still fails on it. README.md is the front door and stays it: docs/index.md is generated from it on every build and gitignored, rather than being a second front page free to drift from the first. Anchors use GitHub's slug rules for the same reason. `#runout--clog` is what GitHub makes of `## Runout / clog`, python-markdown would have made `#runout-clog`, and toolchange.md:340 already pointed at the first. CI builds it with --strict, which is the cheap half of keeping docs true: a rename in payload/ now breaks the build instead of quietly leaving a page pointing at nothing. It cannot check that the prose is still right. Only a reader can. The site is not up yet -- the project has to be created on readthedocs.org and its versions activated -- so nothing links to it from README.md. The slug is free; mkdocs.yml claims it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S3XZh4SqPdXAowVndz6tMr |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
f67e7a269f |
ls pkg/ was thirty-eight entries and four of them mattered
pkg/ -> pkgs/, and the thirty-four recipes that build a pinned tarball and
carry no files of this repo move down a level into pkgs/3rdparty/. What is
left at the top is the four components a person actually edits: anvil-core,
klipper, moonraker, helixscreen.
THE LINE IS MECHANICAL, NOT EDITORIAL, and that is why it can be a directory
rather than a field in pkg.conf. "Do we actively modify it" drifts -- the day
somebody patches zlib nobody would agree which side it belongs on, and the
answer would be a rename touching CI and docs decided by taste. "Does it carry
files of ours" is a fact about the tree: it is exactly whether the recipe has
a payload/, prog/ or seed/, checkable in one assertion, and
test_recipe_layout.py now demands the move in both directions -- a 3rdparty
recipe that grows one of the three must come up, and a top-level recipe with
none of them must go down, or the top level slowly refills and the split stops
meaning anything.
ONE FUNCTION KNOWS THE TWO DEPTHS. pkgs/lib.sh's pkg_dir resolves a recipe id
to its directory and nothing else spells either path -- not a recipe, not
bin/build-packages.sh, not a test. Names stay unique across both levels
because pkg_out, pkg_stamp and the .ipk filename are all keyed by the bare
name; a duplicate would resolve to whichever level is searched first and
build the wrong thing silently, so that is a test too.
Recipes now source `. ./bin/common.sh` instead of counting `..` from $0.
Depth-counting would have needed two spellings for two levels, and it was
never buying anything: `. pkg/lib.sh` on the next line already required the
working directory to be the repo root.
A SECOND SILENT-SHRINK BUG, and this one was live. qa/static's _files()
expanded a list of globs and returned whatever matched, so "pkgs/*/build.sh"
quietly stopped covering thirty-four recipes the moment they moved -- the lane
stayed green, and the only visible sign was a test count dropping by 34 in a
run nobody was counting. _files() now refuses a pattern that matches nothing.
That is the same failure the ash-compatibility check had a guard for and the
syntax check did not, which is why the guard reads the way it does.
AND ONE PACKAGE WAS SHIPPING PYTHON BYTECODE. anvil-core stages a directory of
.py helpers wholesale, so any test that imported one left a __pycache__ beside
it -- gitignored, invisible in a diff, copied by cp -a like anything else. The
package's contents depended on whether pytest had been run on the machine that
built it: two builds of one commit, different sha256, cause nowhere in the
tree. What it shipped was worthless as well as unstable, compiled by the build
image's python 3.11 for a printer running the 3.13 this repo cross-builds. The
sweep is in pkg_ship now, once, and the two hand-written copies in
pkgs/klipper and pkgs/moonraker are deleted -- the trap is set for any recipe
that stages a tree of .py files, and the two recipes that had hit it had each
solved it locally while the third, which had not, was the one shipping.
Measured:
* the rename changes no package bytes. Same cached build trees, feed built
with the diff stashed and unstashed: all 41 .ipk match.
* with the bytecode sweep, anvil-core returns to exactly the bytes the
previous commit produced.
* no package in the feed contains a .pyc or a __pycache__.
* make qa-static: 247 passed (236 before this work + 11 layout tests).
* make test: 8 passed, 0 failed, including unpack/patch/pack/verify on the
synthetic stock package. make test-py: 181 passed.
Not explained: anvil-python-dev produced two different sha256 from an
UNCHANGED work/pkg/python at two points today, under code that was identical
both times -- so it is not reproducible from a cached tree, and it is not this
commit's doing (the stashed and unstashed builds above agree on it). Worth a
look; the dev split in bin/build-packages.sh is where to start.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
54a1e72cc4 |
A file that ships belongs to a package, and the path should say which
payload/ was organised by DESTINATION -- init.d/, bin/, etc/s6/,
klipper/config/ -- plus an assets/ directory of three .conf files. That was
right when there was one owner. With 38 recipes it held three different kinds
of file behind identical-looking paths:
* files a recipe stages into its .ipk,
* files that go to /usr/prog and CANNOT be in a package of ours, because
every path in one lands under $MODDIR,
* files that never ship as files at all -- run-pre.sh and run-append.sh are
spliced into FlashForge's own run.sh at build time.
The only thing recording which package owned any of it was a thirty-line
comment at the top of pkg/anvil-core/build.sh. A comment cannot be checked.
Files now live with the recipe that owns them, and the subtree says how they
ship: payload/ is staged into the .ipk laid out exactly as it lands under
$MODDIR, prog/ is placed on /usr/prog by patch.sh, seed/ is templated or
seeded user state. installer/ sits outside pkg/ and holds the two files that
are never files on a printer. qa/static/test_recipe_layout.py fails if a
fourth kind of directory appears, which is the failure this is actually for:
a misfiled file is caught by a build, an unruled directory is caught by
nothing.
anvil-core's recipe is a loop over a directory now. Six pkg_stage lines with
hand-written destinations plus a glob became one loop, because payload/ IS the
overlay -- a file added to that tree ships without an edit to build.sh.
A REAL BUG FELL OUT. anvil-core's PKG_STAMP_EXTRA was an eight-path list
naming payload/ and assets/ subtrees by hand, and one of those paths was
payload/helixscreen/ -- HelixScreen's printer-database entry, hashed into the
cache key of the package that does NOT contain it. Editing that json rebuilt
anvil-core and left pkg/helixscreen sitting in the cache. pkg_payload_hash is
now the one way a recipe keys its own files and it can only hash its own.
WHAT ELSE MOVED, and why each is not where it looks like it should be:
* ff-*.cfg stay with anvil-core rather than pkg/klipper. They are the
package that has always shipped them; ownership moves in the next commit,
with the two model-config packages. This one is a move, not a repackaging.
* printer.base.cfg is pkg/klipper's prog/, not payload/: it is FlashForge's
file with one block replaced and it lands beside their Klipper tree.
* moonraker.conf is anvil-core's for now; moonraker-custom.conf is
pkg/moonraker's seed/, because it is the user's and a package member is
overwritten on every upgrade by definition.
The cfgdir fixture is a merged copy of both config directories rather than a
path into the checkout. The split is a fact about which package ships what --
on the printer there is no split, the stock run.sh force-copies
printer.base.cfg into /usr/data/config/ beside the ff-*.cfg, which is why its
includes are bare and relative. A test about include resolution has to model
where files END UP or it is testing the repository's filing system.
qa/lib/replica.py's one payload mount is three, and entrypoint.sh reassembles
/tmp/payload out of them byte-for-byte as it was. Not re-pointed: every
case-*.sh copies out of /tmp/payload with 2>/dev/null, so a path that stops
resolving does not fail -- it copies nothing and the case runs green against
an empty $MODDIR.
Measured:
* all 41 .ipk byte-identical. anvil-core and helixscreen were rebuilt from
HEAD and from this tree and compared directly; the other 39 match the
feed built before the move.
* that comparison caught a mistake in this commit: the HelixScreen json was
mirrored one directory too high (helixscreen/printer_database.d/ instead
of helixscreen/config/printer_database.d/). A move that changes no bytes
is only a claim until it is diffed.
* anvil-core: two cold builds byte-identical.
* make test: 8 passed, 0 failed -- including unpack/patch/pack/verify on the
synthetic stock package, which is the gate that covers patch.sh end to end.
* make qa-static: 244 passed (236 + 8 new). make test-py: 181 passed.
Known gap, pre-existing and left alone: pkg/*/payload/bin/*.sh is in no ash
glob, so wifi-action.sh's dialect is unchecked. The old glob was payload/*.sh,
one level above it, so this is not a regression -- it belongs with the fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
09a9f868e3 |
klippy compiles c_helper.so at first run, and this printer has no compiler
So the "build system" for that .so genuinely is one gcc line -- the argument
list in klippy/chelper/__init__.py -- and pkg_build could not express it,
because it always ran make. That is why Klipper was the last cross-build
still living in bin/patch.sh, which unpacked the Ingenic toolchain into
work/.mips-toolchain, wrote its own gcc invocation, cached on an mtime
comparison and gated the result itself: every one of the things pkg/lib.sh
exists to do once.
PKG_CC_SHARED is that line, appended to `$CC -shared -fPIC`. A knob and not
a verb, for the reason pkg_build's own header gives about the three
mechanisms with one user each: the recipe says WHAT to link, pkg_build owns
HOW, and -EL -mnan=2008 stay spelled once in pkg_toolchain where the wrapper
is gated before anything is built on it.
pkg/klipper -> anvil-klipper_0.13.0+gitb722a4c8-1_mipsel_xburst2.ipk
Both gates moved into the recipe, next to the ship list they check -- the
same move the s6 presence checks made. They now run on `make packages` too,
so a c_helper.so that is not nan2008, or one missing a symbol the klippy
beside it cdefs, fails the build that produced the .ipk rather than only a
full firmware build. The symbol gate is the one that matters: cffi resolves
lazily, so a stale .so imports cleanly and dies at connect on the printer.
KLIPPER_FORK is gone. It pointed the build at a local checkout instead of
the pin, which is a second source for a recipe allowed exactly one -- and it
is the same variable that let v20260824-nova-kakhovka ship a klippy tree
nobody had built, set to "" in CI. To build your own fork, put its tarball
in vendor/ and re-pin KLIPPER_VERSION and KLIPPER_SHA256; the sha is then
the guarantee every other source in this repo carries. bin/fetch-assets.sh
loses two of its conditions with it, and test/test-chelper.py loses the
config.env fallback that printed "SKIP: no KLIPPER_FORK configured" -- a
green line for a check that had not run.
bin/patch.sh section 1 is now staging and nothing else: a cp -a into the
SOFTWARE component, where the stock klipperDaemon looks, plus the
chelper.tar its run.sh extracts. That tar stays out of the .ipk, where it
would be a second copy of a file the package already contains. The package
installs the tree under $MODDIR/klipper, inert until phase 7 of
docs/notes/80-s6-migration.md makes the installed copy the live one -- at
which point these lines are deleted rather than moved.
Measured:
* two cold builds of pkg/klipper byte-identical, work/pkg deleted between
* c_helper.so is 0x70001407 -- nan2008, o32, mips32r2
* all 66 functions klippy declares are exported, checked after pkg_ship
strips rather than before
* the whole feed still builds: 38 recipes, 41 packages
* 236 static tests, including the staged-recipe list that now names klipper
* BUILD_KLIPPER=stock removes the recipe from pkg_recipes entirely
Not verified here: bin/patch.sh end to end. It needs the stock FlashForge
package, which the packaging lane deliberately does without; bin/verify.sh's
klippy-tree and chelper.tar assertions are what cover the staging rewrite.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
8cc92e8266 |
A wheel could only be built while three scratch trees happened to coexist
That is why bin/patch.sh section 5c was 825 lines and why a Pillow bump
rebuilt CPython. Cross-building an extension module needs the untrimmed
staging tree (INCLUDEPY points into it), the private sysroot of static C
libraries (pillow links zlib, cffi links libffi) and an x86-64 build-python
of the same version -- and all three existed only mid-section, so the wheels
had to be built there too, on a cache with two stamps that invalidated each
other.
Each of those three is now a durable thing: anvil-python-dev carries the
headers, pkg_deps fills each recipe's own sysroot from the feed, and
pkg_buildpython caches one build-python for all nineteen recipes. So the
coupling goes away with them -- `make packages PKG=python-pillow` builds
Pillow.
pkg/python -> anvil-python + anvil-python-dev
pkg/python-<name> x18 -> anvil-python-distro, anvil-python-pillow, ...
Three verbs in pkg/lib.sh, shared by all nineteen rather than one per awkward
project: pkg_buildpython provides the build-python, pkg_pytarget points its
sysconfig at the target headers in the sysroot, pkg_pywheel builds one wheel
and unpacks it into the staging site-packages. greenlet's gcc 7.2 designated
initializers, lmdb's LMDB_FORCE_CPYTHON and pillow's build_ext options are
recipe lines and a PKG_PY_SETUP_ARGS knob, not three more verbs.
Eighteen versions that were invisible are now in the feed index, anvil-moonraker
declares the thirteen packages it imports, and a printer that does not want
gcode thumbnails can leave Pillow uninstalled.
PY_STAMP, PYPKG_STAMP and PY_BUILD are gone: pkg_stamp derives the key from
PKG_BUILD_DEPENDS, and bin/fetch-assets.sh asks pkg_needs instead of
re-deriving a stamp bin/patch.sh also derived. Two of its four toolchain
conditions were that second spelling.
Measured:
* 40 packages; 959 site-packages files, none owned by two packages
* 12 mipsel extension modules -- the count the old gate demanded
* every Depends resolves inside the feed (24 declare one, 0 unsatisfiable)
* two cold builds byte-identical, with work/pkg, work/.py-host and the
unpacked toolchain deleted in between
* 233 static tests, 4 new: the PYPKG_LIST/recipe agreement both ways, no
version pinned twice, patch.sh builds every package, and no Depends
naming a package no recipe produces
Not verified here: the replica suite. It needs the stock FlashForge package,
which the packaging lane deliberately does without.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
6e97f946b0 |
execline was 2.1MB unstripped against musl; it is 1008KB and mandatory
Eight recipes: skalibs, execline, s6, s6-rc, mainsail, moonraker,
helixscreen, anvil-core. bin/patch.sh loses 183 lines and stops being a
build system for any of them.
The supervision stack moves to the Ingenic glibc toolchain and links libc
dynamically, so the musl toolchain -- the only second libc in this tree --
is deleted along with MUSL_TOOLCHAIN_*, S6_STAMP and S6_BUILD. The
measurement that chose musl compared two STATIC builds; dynamic glibc was
never in it. Measured now: s6's shipped tree is 696K against ~930K for the
static musl one, and the whole stack including execline and s6-rc is 2.0M.
Three things that were not guesses:
* s6 and s6-rc do not link against glibc 2.29 without -lpthread. skalibs'
pthread_mutex_tailock reaches pthread_mutex_timedlock, which glibc kept
in libpthread until 2.34. Upstream never sees it and musl never had the
split. PKG_MAKE_ARGS is the lever, because s6's PTHREAD_LIB is wired
only to --enable-nsss and its Makefile appends LDLIBS last.
* s6 ships 21 binaries, not 13. s6-rc's generated scripts exec s6-sudo,
s6-sudoc, the s6-ipcserver chain, s6-sudod and s6-fdholder-daemon --
found by running a real up/down cycle with a restricted PATH, not by
reading the documentation.
* --enable-static-libc has never been a skalibs option. Its configure
warns on unknown flags and patch.sh sent that to /dev/null, so it was a
no-op for as long as section 5b existed.
execline ships whole. A subset picked from today's services fails the day
somebody adds a pipeline, at boot, naming a program nobody chose to leave
out. tools/supervisor/README.md and 80-s6-migration.md said it was not
needed; both now say why that stopped being true.
PY_STAMP was spelled in three places and happened to agree -- the same
shape as the s6 stamp bug, caught before it inverted. It is in common.sh
now, and the test that guarded S6_STAMP by name is rewritten around the
rule so it covers both and dies with neither.
HelixScreen's binaries reach mips_abi_gate for the first time: patch.sh
never gated them. They pass, 3 objects, nan2008/o32/mips32r2.
anvil-core carries a content hash in PKG_STAMP_EXTRA because its version is
a release date: pinned, an edit to payload/ would read as current; unpinned,
it rebuilds daily. The version says which release, the stamp says what is
in it. anvil.conf stays out -- it is user state that run-append.sh
preserves, and a package member is overwritten on every upgrade.
Twelve packages, byte-identical across two cold builds. 152 static tests.
The replica suite has NOT been run: docker is unavailable here, and the s6
change ships without its behavioural gate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
c8e33a5604 |
zlib was compiled twice because it belonged to whoever needed it first
pkg/opkg/build.sh unpacked zlib, built it into a private sysroot, unpacked
libarchive, built that against it, and then built the one binary it shipped.
Section 5c did the same with its own zlib, from the same pinned tarball with
the same flags. Neither copy could see the other, because a library built
inside somebody else's recipe has no version, no package, and no way to be
reused.
One build.sh now produces exactly one package. A recipe names what it builds
against in PKG_BUILD_DEPENDS and pkg_deps unpacks those out of the feed with
opkg-unbuild -- upstream's inverse of opkg-build, from the checkout that
already provides both. That dissolves the bootstrap: nothing needs a working
opkg in order to build packages, so opkg is an ordinary recipe that waits on
nothing except its own dependencies, and no qemu enters the build path.
Building against the package rather than the build tree is the part worth
paying for. A package that forgets a header fails the next recipe's configure,
on the build that produced it.
One libc. skalibs, s6 and opkg were musl because a STATIC glibc s6 tree
measured 73MB -- a static-to-static comparison that never considered dynamic.
Two libcs also meant two of every library both worlds wanted, which a feed
cannot express without lying about one of them. opkg moves to the Ingenic
toolchain and links libc.so.6 dynamically with zlib and libarchive inside it;
it got smaller (212K against 332K). Section 5b stays musl until it becomes a
recipe too, and is now the only place left where one script builds two
libraries.
Found on the way, each of which had been true for a while:
* ar archives are not reproducible. Every member header carries an mtime and
the builder's uid/gid, and the symbol index carries another timestamp --
none of it reachable by SOURCE_DATE_EPOCH or opkg-build's -o 0 -g 0,
because it is inside a file the tarball merely contains. objcopy -D fixes
the members; the cross binutils is 2.27 and its ranlib -D writes a fresh
current timestamp into the index instead of zeroing it, so the index is
rewritten with the build image's ranlib.
* The ABI gate accepted exactly 0x70001405 and 0x70001407, both measured from
linked binaries where crt objects set NOREORDER. Object files need not:
libarchive's xxhash.o is 0x70001406, identical ABI, and the gate called it
wrong. It also read one Flags: line, while readelf -h on an archive prints
one per member -- 122 for libarchive.a. It now masks the three non-ABI bits
and checks every header.
* bin/fetch-assets.sh compared work/.s6/.version against two fields where
patch.sh wrote three, so the test could never be false and 71MB was
re-hashed every run. The comment above it described a fast path that had
never once been taken.
Gate: 127 static tests (29 packaging), and `make packages` from a cold cache
producing four byte-reproducible .ipk files plus the index. The end-to-end
install under qemu has NOT been re-run since opkg went dynamic -- a dynamic
binary needs qemu -L <sysroot>, and the command is written down in
docs/notes/85-packaging.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
16d00be525 |
case-services.sh is 360 lines and one exit code. That contract is 44 tests now.
test/ has two test frameworks. run-tests.py + ffsim is 1,657 lines reimplementing what pytest already does -- skip accounting, gating, reporting, then parsing pytest's own JUnit XML back out -- and it calls pytest as gate #5 of about twenty. Underneath it, 6,280 lines of case-*.sh execute inside the qemu chroot and report 13 booleans between them; case-moonraker.sh is 1,059 lines that collapse to one pass/fail. Nothing runs in parallel and nothing can be selected: each replica gate is its own `docker run --privileged`, re-registering binfmt and re-assembling the mount layout, thirteen of them in sequence. All three follow from one decision -- THE REPLICA CONTAINER IS TREATED AS A TEST RUNNER INSTEAD OF AS A FIXTURE. That is what pushes assertions into ash, which yields one bit per container, which then needs a bespoke framework above pytest to aggregate the bits. qa/ is the replacement, built beside test/ rather than inside it, with two lanes and one framework. Both suites run in CI; test/ is still the release gate; nothing has been deleted. docs/qa-migration.md has the order the rest moves in, and the rule: a case script is deleted only once its replacement here has been green, so there is never a window where coverage drops. THE MACHINE IS UNCHANGED. qa/ reuses test/integration/printer/'s Dockerfile, entrypoint.sh, assemble.sh, binfmt.sh and seed-prog.sh as they are, and its `docker run` argv is deliberately ffsim's, minus --rm and plus -d. While both suites run they have to be testing the same machine or neither proves anything about the other. The only new trick is how the container is held open: entrypoint.sh ends by running the case script it was given, and the one it is given here touches a marker and sleeps, so setup completes exactly as it always has and the container simply does not exit. The marker is also the readiness signal, because setup ranges from under a second to over a minute and a constant would either waste the fast case or fail the slow one. TESTS RUN AGAINST A MACHINE THE REAL INSTALLER PRODUCED. The `printer` fixture is a replica with the package installed by the printer's own app_startup.sh, off a genuine FAT filesystem on /dev/sda1. The first version of this tree did what case-services.sh does -- copy payload/*.sh and payload/init.d/S* into $MODDIR by hand -- and two things were wrong with it. It is a second implementation of the install, so everything downstream asserted against a layout the harness built rather than one the installer produced, and the real installer could have broken without a single test noticing. And it can only place what is in payload/, so everything the BUILD produces was missing: s6 is a build output that bin/patch.sh stages into the package, so a hand-placed payload has no supervisor at all, which is why the tests had to carry a stand-in scanner and gate one assertion on whether it was in use. Installing for real fixes both and deletes the stand-in. Hand-placed, /usr/data/anvil held anvil-env.sh, anvil-service.sh, anvil.conf, init.d and etc/s6. Really installed it holds VERSION, backup, bin, config, config-installed, helixscreen, lib, libexec, moonraker, nginx and www as well -- bin/lib/libexec being the cross-built s6 and CPython 3.13. The file is SHORTER than the case script it replaces while reporting 44 results instead of one. case-install.sh's header records this same lesson from the first time somebody learned it: "An earlier version of this file replayed app_startup.sh by hand -- which meant a bug in our reading of it could never be caught." The install is the machine's own shell under qemu, so it is baked into an image once per package and cached under a tag derived from the package's md5 -- build-printer-image.sh's trick for the stock baseline. Rebuild the package and the tag changes, so the next run bakes again instead of testing yesterday's build; deliberately not a hand-written staleness check, because those are the ones that eventually get it wrong. A MISSING TOOL OR RESOURCE FAILS, and there is no flag. This tree briefly had --strict-skip=<lane|all>, which was ALLOW_SKIP inverted -- and ALLOW_SKIP's defect was never the direction it named things in, it was that the decision lived in the CI invocation. An accept-list has to be edited whenever a gate moves; a strict-list has to be remembered whenever a job is added; both degrade silently when the command line is wrong, which is the same disease as the printer-sim job that was gated on an unset secret and never ran once in its existence. So the verdict lives with the knowledge: no shellcheck, no pyflakes, no docker, no daemon, no replica image, no built package -- each fails where it is needed, with the fix in the message. There are no skips in this suite. MEASURED, on monstrofil/creator5-printer:latest, both suites agreeing on the same machine: 131 tests in 40s, all green; qa/static alone 1.3s with no docker; one assertion selected by node id 2.9s, against the whole 360-line case being the smallest unit test/ can run. Granularity checked by injecting the regression case-services.sh exists to catch -- deleting SVC_EXTRA_VERBS="|force-start" from S70klipper -- which failed one named test, printed the usage line that came back, left the other 41 reporting, and took 4s to re-run alone. Two bugs the port found in its own harness, both only visible by running it. bake.sh shipped without its executable bit, so the lane died at container init with 43 errors all saying "could not install the package" -- a file mode wearing an installer failure's clothes; test_action_scripts_are_executable now guards it, the same check the payload's own s6 run scripts already have. And an ELF magic assertion used `od -An`, which the printer's busybox od does not have -- the replica catching host-vs-printer drift in the test code itself, which is the entire reason it exists. Also here: qa/static/test_probes.py, which has no counterpart in test/. Printer.ps() and Printer.listening() are parsers, and a parser is where a harness goes quietly wrong -- a listening() that never matches makes every readiness assertion pass by the absence of contradiction. Neither failure looks like a failure; they look like a green suite. So they are tested against captured output, in the static lane, with no docker involved. Two things found while surveying, neither fixed here. case-pyext.sh is 567 lines with no gate in gates.py, no Makefile target and no CI step: it has never run. And case-mcu-bringup.sh:4 says the handshake is "tested elsewhere against a pty" -- there is no pty anywhere in the repo, and every documented past bug in ff_mcu_bringup.py lived in the code that comment excuses from testing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
8b58bfc831 |
PRINTER_IMAGE was optional and unset, so it was never used
vars.PRINTER_IMAGE has apparently never been set on this repository, so every replica case in ci.yml and release.yml has been paying the local replica build -- about a minute -- on every single case, for the whole life of both workflows. Defaults to the public monstrofil/creator5-printer:latest now; vars.PRINTER_IMAGE still overrides it for a fork building its own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
9b8387f3b1 |
CI's test job never had pyflakes, and check_undefined_names needs it
Pre-existing gap, not something this branch's changes introduced --
python3-pyflakes was never in the test job's apt-get line, only
python3-pytest and python3-jinja2. check_undefined_names() in
test/run-tests.py hard-fails without it ("pyflakes not installed"),
which is the exact error the pushed CI run just hit. Added it beside
the other two.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
c870d8417d |
FF_PYTHON moves to our own 3.13, and CI stops rehearsing around it
The gate this switch was waiting on now exists and passes:
case-moonraker313-s6.sh drives Moonraker through the real boot path --
S40s6's scandir, S62moonraker, readiness gated on :7125 actually
listening, a kill -9 respawn, a stop that stays stopped -- on the
interpreter this commit ships. anvil-env.sh now points FF_PYTHON at
$MODDIR/bin/python3.13 and drops /usr/prog/libsodium/lib from
ANVIL_LIBS, since nothing left running under FF_PYTHON imports libnacl
from there any more (Moonraker carries its own libsodium; klippy is not
on this list at all -- it stays on FlashForge's 3.8.2, started
independently by /usr/prog/klipper/start.sh).
Three real bugs surfaced chasing this through the replica, none of them
about the switch itself:
* The Phase-1 manifest-based install can only delete files it once
tracked, so an init.d/S60web surviving from a release that predates
the manifest system would never be cleaned -- caught by
case-install.sh, which had never run against real stock packages
before today. run-append.sh now clears $MODDIR/init.d unconditionally
on every update: those scripts are 100% mod-owned and get executed,
so a diff against tracked history is not a strong enough guarantee.
* case-moonraker313-s6.sh's own readiness-ordering check flaked once:
its bind-time sampler polled at 1-second resolution while s6-svwait
is event-driven and returns within a fraction of a second, so
`date +%s` truncation could show readiness before the bind though the
real events were in order. Tightened to 0.2s.
* case-mcu-bringup.sh and case-boot-screen.sh source anvil-env.sh but
named their own directory variable $MOD, not $MODDIR -- harmless
while FF_PYTHON was a literal path, and silently wrong the moment it
started reading $MODDIR. Aliased.
Every other test/integration file touched here is the same story:
FF_PYTHON's callers were hardcoded to FlashForge's 3.8.2 throughout the
suite, and each one either now stages our interpreter as a tarball
(gates.py's mcu_bringup/boot_screen/moonraker, matching the pattern
already used for python()/moonraker313_s6) or asserts against the new
target. case-libpath.sh's premise changes with it: the three ANVIL_LIBS
entries that remain are read as what klippy's process tree still needs,
not what every mod process needs -- Moonraker maps zero /usr/prog
libraries now, measured.
payload/anvil-service.sh's SVC_S6_SCANDIR line tripped test_paths.py's
absolute-path regex, which reads a closing brace immediately followed by
a suffix as a stock rootfs path in its own right; split onto its own
line so a word character precedes it instead, which the regex already
excludes.
bin/fetch-assets.sh had a real latent bug of its own, independent of
this switch: the Ingenic toolchain fetch was gated on BUILD_KLIPPER=fork
alone, a condition that predates CPython 3.13's unconditional need for
the same compiler. A BUILD_KLIPPER=stock build from a clean vendor/
would reach the Python cross-build with sources and no compiler. Also
added curl --retry -- musl.cc flaked mid-download in CI while this was
being tracked down.
.github/workflows/ci.yml's fixture-only `test` job now actually
cross-builds s6 and CPython 3.13 instead of never exercising that path:
it fetches just those two build inputs (Mainsail/HelixScreen/Klipper
stay stubbed, unchanged) and caches vendor/, work/.s6, work/.py313 and
work/.sodium on versions.env.
Full suite, replica required, against real stock packages fetched for
this run: 23 passed, 0 failed, 0 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Fable 5
|
3b37302c2a |
TOOL_OFFSET_CALIBRATE stops shutting the printer down
_current_tool_or_none() returns (tool, why) -- every call site unpacks it, except the one reapply_offsets_after_external_zero added: it bound the whole tuple to `tool`, so the `is None` guard never fired (no tool mounted is (None, why), a truthy tuple) and off_z[tool] died with "list indices must be integers or slices, not tuple", taking the printer to shutdown at the end of every TOOL_OFFSET_CALIBRATE. Unpack it, and treat -1 (nothing on the carriage) as nothing-to-reapply, same as the runout call site does. Also let a release tag carry the same-day suffix common.sh already documents (v20260825b-...): the installer never reads MOD_VER -- it compares the software component's own version, which stays at stock's -- so the letter exists only to keep the outer filenames distinct. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Fable 5
|
212d3a1efd |
A fork build without a fork is now impossible to ship
v20260824-nova-kakhovka went out without the Klipper fork: release.yml set KLIPPER_FORK="", bin/patch.sh answered with "(skip) Klipper: keeping stock tree", and every gate stayed green because every Klipper check was conditional on files the broken package simply did not contain. On an already-modded printer the stock 0.12-era overlay half-overwrote the v0.13 tree and klippy died at connect: "'void(*)(struct stepper_kinematics *, double, double, double)' expects 4 arguments, got 3" -- a v0.12 caller against the v0.13 cdef of extruder_set_pressure_advance (upstream c84d78f3f). Note for the next reader of such a traceback: an arg-count TypeError is a MIXED KLIPPY TREE, python against python; a stale c_helper.so shows up as AttributeError instead, because cffi resolves symbols lazily. The fork now travels like every other asset. versions.env pins the creator5 commit and the Ingenic gcc 7.2.0/glibc 2.29 toolchain (the X2000's own; Debian's mips crosses are big-endian or legacy-nan), fetch-assets.sh downloads both by sha256, and patch.sh extracts the pinned tarball whenever KLIPPER_FORK is not a local checkout -- which is exactly what CI now does, with the KLIPPER_FORK="" lines gone from both workflows and vendor/ cached on the pin file. c_helper.so is cross-compiled from the chelper sources of the very tree being shipped whenever it is missing or older than them, so the stale-.so class dies too; test/test-chelper.py returns from 3a28fdf's grave to prove every cdef'd function is exported, and runs both in patch.sh and in verify.sh against the extracted package. BUILD_KLIPPER=fork with no tree is a hard build failure now; stock has to be asked for by name. verify.sh also stops trusting `echo "$LIST" | grep -q`: under pipefail, grep exiting at an early match SIGPIPEs the echo and the check reports 141 -- a race that stayed invisible until the fork tree tripled the payload listing, then intermittently declared Mainsail and Moonraker missing from a package that carried both. Herestrings have no pipe to break. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Fable 5
|
b1c2a0553d |
release.yml stops promising a SAFE-MODE that no longer exists
The generated release notes still told a dark-screen user that the UI had
latched SAFE-MODE after three failed boots -- removed in
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
1c0dd83c61 |
Release notes carry the city the release is named for
The fixed half of the notes is the same every time; the half that says which city and what happened there is not, and it was living only in a hand-edited release page. docs/releases/<tag>.md holds it now, at the tagged commit, and the workflow appends it -- falling back to the bare line, with a warning in the job log, when a tag arrives without one. Nova Kakhovka's is the text now on the v20260824 release. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
04d4509b0a |
The shipping-package gate runs in the build image too
Every step before it runs as root inside that image, so work/ and dist/ end up root-owned and the replica could not create its work/.sim-* stage as the runner user: PermissionError, after the brick-safety suite and both builds had already passed. Same container, same docker socket as the suite step above it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
07d99b2748 |
release.yml: two spaces, not a tidy column, before the checksummed path
sha256sum -c takes everything after the first two spaces as the filename, so the aligned lines asked for a file whose name started with spaces and the release died on its second download. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
ecb7fb76e4 |
Release on a tag, with nothing secret in the way
The stock packages and the factory image are large, not confidential -- ci.yml already fetches them from ghzserg/FF by public URL with an enforced sha256. release.yml asked for the same three things as repository secrets instead, so a tag on a fresh clone failed the first step with "missing secrets" and published nothing. Same URLs, same checksums, same enforcement as CI now. Tags are v<YYYYMMDD>-<city>: the release date and a Ukrainian city under occupation. Only the date reaches MOD_VER -- the installer compares the version field as a number -- so the codename rides in the tag and the release title. Releases are marked pre-release, and the notes say where to go when the screen is dark and how to get back to stock. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
ac558f6686 |
Document the Moonraker swap honestly, and finish its doc coverage
The reversibility claim was the strongest promise in the README and
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
d05565284d |
Sections E, F, G: wrong values, stale claims, overclaims
E -- values and counts. The START_PRINT prologue has no M400 and an idle timeout of 864000, not 1800000. The nozzle-clean temperature comes from the material table FIRST, so NOZZLE=240 with no per-tool temp cleans at 220 -- documented backwards before. TOOLS is the paired 0:220,2:240 form. The wipe Z is the raw eddy frame, not the homed one. The runout M117 strings are "T<n> clog" / "T<n> out of filament", not E-codes. Grab-verify raises E0051+tool, not E0143. Docks are at X~297; 250/280 are staging waypoints. The grab sequence is documented in the order _grab actually emits it. ff_print exposes seven status fields, not nine, and never emits TOOLS=. Package sizes reconciled against the real artifacts (62MB built, ~93MB stock, ~370MB of CI assets, ~60s not 90s). testing.md's sample output block matched no header the code prints, so it is replaced with the real ones; its pytest row listed two of six suites. F -- stale claims. _FF_PREFLIGHT is not "missing". SDCARD_SET_CHANNEL is not still issued ( |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
f70369155b |
Run the brick-safety gates in CI, and stop skipping quietly
The replica gates -- the only ones that decide whether a package bricks a
printer -- had never run in CI. They were gated on a STOCK_URL_CREATOR5PRO
secret that was never set, so printer-sim rendered as a grey dash beside
two green ticks and read as "CI passed".
The packages it needs are already public. ghzserg/FF publishes both stock
packages and the factory image, and this repo already points at that
release in docs/printer-replica.md. All three are now pinned in ci.yml by
sha256 and verified byte-for-byte against the copies used locally:
Creator5Pro-1.9.7-1.2.9-20260810.tgz 1f3252fe... 93MB
Creator5-1.9.7-1.2.9-20260810.tgz 72009be1... 93MB
Creator5Pro-factory.tar.xz 45657924... 190MB
Nothing proprietary is redistributed from here; the workflow fetches what
is published elsewhere, cached on their checksums so an unchanged pin
costs that archive nothing. Secrets still override for a fork with its own
firmware -- but a URL override without a matching checksum now refuses to
run rather than downloading something nobody verified. have-firmware is
gone with the `if:` it fed.
Rehearsing that job locally found the reason it would have failed anyway.
The replica gates call ./bin/patch.sh directly, and patch.sh expects
vendor/ already populated -- bin/build.sh is what fetches Mainsail and
HelixScreen, and the gates do not go through build.sh. No local run ever
noticed, because vendor/ was always already there. Both workflows now
fetch assets before the suite; in release.yml that step has to precede the
gates, since `make release` would otherwise fetch them one step too late.
Second half: a skip must be legible.
ALLOW_SKIP=1 silenced the explanation as well as the exit code -- the
summary printed "2 skipped" and then nothing about what or why. That is
the disease this harness exists to treat, reintroduced by the flag meant
to manage it. Now:
- Every skip is listed again in the summary with its reason and whether
it was accepted, whatever ALLOW_SKIP says, and becomes a GitHub
annotation. Nobody reads a green job's log.
- ALLOW_SKIP takes names: ALLOW_SKIP="pytest,the printer replica". `1`
still accepts everything, but it is a standing promise never to notice
a skip again, so CI uses the list and a third skip fails there.
- The pytest gate names the tests that skipped instead of counting them.
That immediately showed the exec-bit test skipping wherever .git is
absent -- a bare "6 skipped" would not have.
VERIFIED:
- printer-sim rehearsed end to end on the actual published bytes, real
git checkout, REQUIRE_PRINTER_SIM=1 and no ALLOW_SKIP: 12 passed,
0 failed, 0 skipped, exit 0, in about four minutes. Before the vendor
step it failed at patch.sh, which is what CI would have done.
- The four ALLOW_SKIP cases, in the build image with no firmware: none
set -> exit 1 naming both gaps; "pytest" only -> exit 1, replica still
NOT ACCEPTED; both named -> exit 0, both still printed; and under
GITHUB_ACTIONS the ::warning:: annotations appear.
- pytest 30 passed. Both workflows parse.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
cd36ff584c |
Restore the exec bit on 31 scripts, so CI can run any of them
CI has never once run this suite. Four runs, four failures, each about
twenty seconds long, each ending the same way:
./test/run-tests.sh: Permission denied exit 126
Not one tracked file in the repo was 100755. `core.filemode` is false on
my checkout and has to be -- the repo lives on a Windows drive where
every file reads back as 0777, so turning it on would mark the entire
tree executable. The consequence is that `chmod +x` here changes nothing
git can see. Locally everything ran anyway, because make invokes the
launchers through bash and because the working tree claims 0777, so the
only machine that could notice was the one nobody was reading.
The payload was never at risk: patch.sh chmods each file as it installs
it onto the printer. This is about the scripts the host executes.
The exec bit was only the first of two. With modes fixed the fixture job
still exits 1 -- honestly this time. It has no stock package by design,
so two gates cannot run: the tests that read the real rootfs, and the
replica. The suite refuses to call a gate it did not run a pass, which
is the rule the whole harness exists to enforce, so the fixture job is
exactly the case ALLOW_SKIP=1 is for. printer-sim and release.yml keep
REQUIRE_PRINTER_SIM=1 and no ALLOW_SKIP, because there a skip must fail.
test_harness.py gains the guard. It reads the mode from `git ls-files
-s`, not from the filesystem: asking this filesystem answers a question
about Windows, not about what was committed. It names each offender and
prints the fix. It passes `-c safe.directory=*`, because git will not
read a repository owned by another uid and every docker run here is root
over a checkout that is not -- without it the test skips inside the
build image, and release.yml runs the suite there with no ALLOW_SKIP, so
a skip would fail the release.
Writing that test turned up a flaw in the neighbouring one.
compileall.compile_file writes the .pyc beside the source and returns
False when it cannot, which under quiet=2 is indistinguishable from a
syntax error and carries an empty message -- a root-owned __pycache__
reported "python that does not compile:" and then nothing at all. It now
compiles through py_compile with an explicit cfile under tmp_path, so it
touches nothing outside the temp dir and reports the real exception.
VERIFIED:
- Reproduced the ci.yml test job against the committed tree: a fresh
clone on ext4, no config.env, no firmware, run in the build image.
Before, exit 126 on the first line. After: shellcheck ok, 7 passed
0 failed 2 skipped, exit 0 -- and without ALLOW_SKIP, exit 1, which
is the behaviour being kept deliberately.
- Full make test with the real firmware: 11 passed, 0 failed, all
three replica gates including recovery.
- The harness tests inside the build image, root over a uid-1000
checkout -- the release.yml shape: 4 passed, nothing skipped.
- Mutation: chmod -x a file and the new test fails naming it; append a
syntax error and the compile test fails with SyntaxError rather than
an empty string; restore and both pass.
- Checked every path, make target and shellcheck glob the workflows
reference: all present, no glob matching zero files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
e0dad3513c |
Rewrite the host half of the harness in Python, so a broken gate fails
The shell suite decided "this gate did not run" by grepping the child's
stdout:
if "$@" >"$out" 2>&1; then
if grep -qE '^[[:space:]]*SKIP:|[0-9]+ skipped' "$out"
then skip "$name"; else pass "$name"; fi
Text on stdout cannot tell you what happened inside a process. A launcher that
had ALREADY failed reached that same line by another road: the file that
sources config.env failed to load, so STOCK_TGZ_* was never set, so the
launcher concluded there was nothing to test, printed those five characters
and exited 0 -- on a machine with docker and the firmware sitting right there.
That is how five broken launchers shipped and stayed broken.
So the host half is Python and a skip is an exception. Output cannot imitate
one, and a gate that fails cannot claim it was skipped.
test/ffsim/ config loading, docker plumbing, gate reporting
test/run-tests.py the orchestrator
test/integration/ sim-install.py, sim-roundtrip.py, printer-exec.py,
extract-rootfs.py
test/integration/printer/*.sh and the whole payload stay shell. They are
executed by the printer's own busybox ash under qemu, and surviving that is a
large part of what the suite proves; rewriting them would test a Python the
printer does not have. build-printer-image.sh and make-stock-fixture.sh stay
too: both are `set -euo pipefail`, neither is a gate, and one is a release
tool run by hand.
Four things that used to report as skips are now failures -- a config.env that
does not parse, a missing case script, a missing package, a missing baseline.
A genuinely absent precondition is still a skip, because that is still true.
Three more bug classes are gone by construction rather than by being tested
for. The repo root is found by searching upward for bin/common.sh instead of
counting `..` from $0, so a launcher can move to any depth. Sourcing a config
with a syntax error is checked -- `.` returns non-zero and does NOT stop the
shell, so the old code carried on with half a file applied, which is exactly
the broken-looks-absent confusion again. And pytest counts come from the JUnit
XML rather than from grepping "N skipped".
test_script_roots.py becomes test_harness.py: the `..` check now has two
remaining shell targets rather than seven, and it gains a byte-compile of the
whole harness. That last one matters -- sim-install.py and friends are only
executed by `make test-install`, which needs docker and the firmware, so
without it a typo would wait quietly until release day. It also pins the
property that replaced the `..` counting, from four different depths.
VERIFIED, and the verification is the point, because this machine has no
docker daemon and these are exactly the scripts that cannot run here:
- Stubbed the docker binary and diffed the recorded `docker run` argv,
shell against Python, across five paths: install with a prebuilt image,
install with a local build, local build plus PROG_DUMP, roundtrip, and
printer-exec. IDENTICAL in all five.
- Ran the old orchestrator and the new one in the same tree: both give
6 passed, 1 failed, 2 skipped, exit 1, with the same gates in the same
order. (The failure is shellcheck, which lives in the build image.)
- pytest: 24 passed / 5 skipped bare, 29 passed against a real extracted
rootfs.
- Compared failure modes directly. The four cases above go from "exit 0,
reported skipped" to "exit 1, named". Case 2 is the sharpest: with a
missing case script the shell launcher exited 0 in silence, having cp'd
nothing and mounted a directory over /case.sh.
Still unproven here: the gates that need a live daemon. `make test-install`,
`test-mcu` and `test-recovery` want one run on a docker-capable machine. The
argv diff says they will be handed the same command as before; it cannot say
what docker does with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
43baf4054e |
Drop the unit lane; one suite, in test/integration
The split existed so a plain pull request had something to run. This repo has one maintainer who always has the firmware to hand, so it was a boundary kept in sync for a contributor who never arrives. One directory now. The tests moved rather than went away. test_chamber.py is the only check in the suite that can catch a printer heating something it has no element for, and the replica cannot cover it: case-install.sh touches .cfg files at exactly two lines, both a grep for a sentinel proving the user config survived, and it asserts klippy.py exists and c_helper.so has the right ABI without ever starting klippy. Declaring [heater_generic chamber_heater] on the plain Creator 5 fails four of these tests, including M191 waiting on a target the machine can never reach; that same package installs, boots, and passes every replica gate. make-stock-fixture.sh came too, since the packaging pipeline is built on it. Ordering changed, and that is the part to look at. With two lanes, pytest ran twice -- once per lane, each with its own -m expression or path. With one it runs once, so it has to run at the point where it can see the most: the stock package is now resolved and the rootfs extracted BEFORE pytest rather than after it. On a configured machine that is 28 tests in one pass; on a bare tree the five that read the rootfs skip. config.env is read in a subshell while doing that. It is the BUILD config, and it is now consulted upstream of the fixture build, where the entire point is that the fixture config wins -- sourcing it into this shell would put MOD_NAME and TARGET_MACHINE where that build could see them. Verified by pointing a config.env at MOD_NAME=LEAKED and confirming the fixture still builds Creator5Pro-anvil-ci.tgz. One thing gets worse, and it is inherent to having a single lane: on a tree with no rootfs, pytest reports 5 skips, so sub() marks the whole gate SKIP rather than ok, and the summary reads 6 passed instead of 7. Those 23 passes are real and the line above the SKIP says so. This is the existing policy working as designed -- a gate that did not fully run must not read as success -- but the summary is coarser than it was when a lane existed that never skipped. Verified: 28 passed with a real extracted rootfs, 23 passed / 5 skipped without; the full suite gives 7 passed / 1 failed / 1 skipped with a rootfs present and 6 / 1 / 2 without, the failure being shellcheck, which lives in the build image; no test/unit reference is left outside the passages that explain why it is gone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
2ae9736e97 |
Name the two lanes unit and integration, and fix what the move broke
test/repo -> test/unit, test/replica -> test/integration. The old names said
where the input comes from; these say what the tests are, which is the thing
you already have a word for.
The rename is also how the bug below surfaced, because "integration" is one
character-count away from nothing and running the launchers by hand is not.
Every launcher that moved out of test/ into a subdirectory opened with
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
and one `..` was correct at test/ and wrong a directory deeper. $ROOT became
test/, so the first thing each of them did -- source $ROOT/test/integration/
sim-image.sh -- went looking for test/test/integration/. Five scripts:
printer-exec.sh, sim-install.sh, sim-roundtrip.sh, build-printer-image.sh and
extract-rootfs.sh, which reached bin/common.sh the same way.
The previous commit shipped this. The path-existence sweep that was supposed
to catch it only checked paths written as literals; these are assembled at
runtime from $0. Nothing else caught it either, and that is the part worth
naming: the integration half cannot run without docker and the stock package,
so on any machine missing either -- this one, and a fork's CI without the
secrets -- five broken scripts still reported a clean SKIP.
So test/unit/test_script_roots.py now checks it statically: for every .sh
under test/, count the `..` after `dirname "$0"` and compare it against how
deep the file actually sits. It needs no docker, no firmware and no shell, so
it runs in the lane that always runs, which is the whole point -- the failure
it pins only appears in the lane that often does not.
Verified: the guard fails with "climbs 1, is 2 deep" against a reintroduced
bug and passes once it is reverted; all four launchers now reach their own
preconditions (docker daemon, config.env) instead of dying on a missing file;
21 in test/unit and 7 in test/integration against a real extracted rootfs;
run-tests.sh reports 7 passed / 1 failed / 1 skipped, the same as before the
rename (shellcheck lives in the build image, and no stock package is
configured in this worktree); no test/repo or test/replica reference is left
anywhere in the tree.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
8337359a68 |
Split test/ into the two lanes it already had
The suite has run in two halves for a long time, and the rest of the repo is
built around that: RUN vs RUNSIM in the Makefile, `test` vs `printer-sim` in
CI, two halves in run-tests.sh. Only test/ itself was flat, so which half a
file belonged to was something you learned by reading its header.
test/repo/ needs nothing but this checkout
test/replica/ needs the printer's real rootfs
test/ run-tests.sh, and the fixtures both lanes share
The axis is the precondition, not the runner. test/replica is everything that
cannot run without the proprietary package -- the qemu'd sim launchers and the
replica image, but also test_paths.py, which only reads the extracted rootfs
as files and needs no docker at all. Grouping by "does it use docker" would
have put that one in the wrong place; grouping by what has to exist first puts
every file where the question "can this run on a public checkout?" is answered
by the directory name.
The `rootfs` pytest marker goes with it. It existed to let run-tests.sh pass
`-m 'not rootfs'` in the PR lane and `-m rootfs` in the replica lane; the
directories say the same thing without anyone having to remember to tag a new
test, and a file in the wrong lane now shows up in `git status`. The two
lanes are `pytest ./test/repo` and `pytest ./test/replica`.
pytest.ini is new and is load-bearing: without a pinned rootdir, naming a lane
on the command line makes pytest take that lane as the root and never load
test/conftest.py, so the fixtures both lanes share would vanish depending on
which one you asked for.
Renames only, plus the path references: test/fixtures/make-stock-fixture.sh ->
test/repo/, the sim launchers and test/printer/ -> test/replica/, and
test_applets.py -> test/replica/test_paths.py, which is what it has actually
checked since the command-word half was dropped.
Verified: 19 in test/repo and 7 in test/replica against a real extracted
rootfs, the same 26 as before the split; run-tests.sh reports 7 passed /
1 failed / 1 skipped, identical to the pre-split run on this host (shellcheck
is absent here and lives in the build image, and no stock package is
configured in this worktree); every referenced path in the repo resolves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
91604c2dfb |
Cut the suite to four gates, and rewrite the Python half as pytest
Twelve gates, most of them the weaker copy of a gate that survives. What is
left: pytest for the Klipper config, and three replica gates -- the end-to-end
install, MCU bring-up, and stock recovery.
Fix first, because the cut depends on it: a skip counted as a pass. Every
launcher exits 0 when it decides it cannot run, and sub() reported that as
`ok`. That is how test-abi.sh sat in the suite printing green while checking
nothing -- run-tests.sh deleted work/modpayload immediately before it ran and
CI set KLIPPER_FORK="", so it had no targets on any run, ever. Skips are now
counted apart, the summary says so, and the suite exits non-zero unless
ALLOW_SKIP=1 says the gap is deliberate. With four gates instead of twelve, a
gate that silently does not run is the failure mode that matters most.
Dropped, each because something else already does the job properly:
test-abi.sh bin/patch.sh refuses to build a non-nan2008
c_helper.so, bin/verify.sh re-checks the packaged
one, and case-install.sh reads the ELF header of
the file that actually landed on the machine.
test-ash-conformance.sh case-install.sh runs `sh -n` over every installed
script with the same qemu'd busybox and then greps
the boot log for "Syntax error".
test-model-gate.sh verify.sh SS8b/SS9 checks the same invariant and
pack.sh exits on MODEL MISMATCH before a package
exists. Its header claimed it proved the two models
ship different files; no such check was in it.
test-macros.py It used jinja2.Environment() -- default '{{ }}'.
The configs have no '{{' at all, only single-brace
expressions, so it checked '{% %}' block structure
and never looked inside an expression. Folded into
the chamber tests, which had the delimiters right.
test-base-cfg.py It read work/software/klipper/config/printer.base.cfg
as its stock reference -- the file bin/patch.sh
overwrites with ours. It was diffing our file
against itself: green on a cold tree, red on a
second run, blind to real drift either way. The
comparison moves to bin/unpack.sh, where a pristine
tree exists, as a warning rather than a gate.
lint-danger.sh Its structural greps were satisfiable by comments:
firmwareExe:44 and S80ui:16 *mention* helix and
SAFE-MODE, so they passed with the logic deleted.
case-ui.sh Goes with the helix-only decision.
applets.allow Ten entries, none of which the extractor had asked
about in a long time.
The command-word half of test-applets went too. It extracted only the first
word of a simple unprefixed command, so `if timeout 5 foo; then` yielded
['if','echo','fi'] -- the one failure its docstring named was invisible to it.
The absolute-path half survives: S50wifi's binaries sit behind a
[ -d /sys/class/net/wlan0 ] guard no simulation ever reaches, so nothing else
checks they exist.
The Python half is pytest now. Real assertions instead of hand-rolled
PASS/FAIL counters, underscored filenames so the modules import normally
(ffcfg keeps only the parser; the dash in test-macros.py was why one test used
to reach the other's parser through importlib), and a `rootfs` marker that
splits the tests needing the proprietary package from those that do not --
deselected in the PR lane, run in the replica half, so neither lane reports
skips it did not expect. test_default_delimiters_are_blind pins the jinja2 bug
so it cannot come back.
The config gate also moves out of the replica half into the half that runs on
a plain pull request. It needs python3, jinja2 and files in the repo, and it
is the only safety-relevant check in the suite -- a chamber heater declared on
a Creator 5 that has no element for it -- so it had no business being gated
behind the proprietary firmware.
Verified: pytest 21 passed / 5 rootfs-marked; the fixture pipeline
(unpack, patch, pack, verify) passes end to end; injecting a broken
single-brace expression fails test_every_macro_parses with the exact line, and
parses clean under the old defaults; the skip path reports SKIP and exits 1,
and ALLOW_SKIP=1 accepts it.
Also drops the CI `abi` job, which ran on a bare checkout with no binaries and
so was permanently green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr Shyshatskyi
|
12bbd1e7dc |
Retire the probe profile: there is one package now
The probe profile was a bring-up aid. It set BUILD_APPLY=0, reinstalled the stock component byte-for-byte and wrote a report to the stick, so you could prove the update chain worked on your own machine -- the filename glob, the DES3 key, the md5 gate, run.sh execution -- before flashing anything that changed a file. That rehearsal has served its purpose. With it gone there was exactly one profile left, so the whole PROFILE indirection goes too: profiles/ is deleted and its six BUILD_* flags are plain defaults in bin/common.sh, which also means config.env now overrides them instead of being silently overridden. make default folds into make build; all-profiles and RECOVERY_PROFILE are gone with it. What is NOT gone is the undo button: flashing the stock FlashForge package still restores every file the mod touches, and case-recovery.sh still proves it. What is gone is the dry run. Note the loss: PROG_MB/DATA_MB used to come out of the probe report. The docs now point at df -h over ssh, which you only have after a successful flash. |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
5bdf78109a |
Retire firmwareExe: own the web stack, and say what goes with it
The mod replaces FlashForge's application outright, so the reasoning that still framed the web stack as coexisting with it was wrong on its own terms. * nginx and moonraker are documented as the only web stack on the printer. MOD_WEB=0 is a debugging switch that leaves ssh and the touchscreen, not a way back to stock -- that is a stock FlashForge package. The own-prefix rule stays, for the reason that is actually true: /usr/prog is the firmware partition and every FlashForge package reinstalls it, so a config written there is undone by the next stock flash. * Configs the mod ships now update when you have not edited them. A live file still byte-identical to the one the last package wrote is replaced; one you changed is kept and the new version lands beside it as .mod-new. $MODDIR/config-installed holds that last-written copy. Without this a config we own could only ever be written once: it exists on the second flash, so it would land as .mod-new forever and updates would never reach the printer. * Dropped the three dead /server/files/ alias blocks from nginx.conf. The regex location above them matches /server/ too and nginx gives regex locations precedence, so they never fired -- and had they fired, serving those paths from disk would 405 the DELETE and POST that Mainsail sends to the very same URLs. * Recorded what dies with firmwareExe, in docs/notes/25 and in the README: the :8898 REST API (FlashPrint, Orca's FlashForge profile, the mobile app) and the :8080 camera stream. The camera is the one that looks restorable and is not -- the MJPEG server is compiled into firmwareExe (its own accept loop, `GET /?action=stream`, boundarydonotcross) rather than being mjpg-streamer, so there is no daemon left to point a [webcam] at. assets/moonraker.conf carries the block, commented, for when a streamer is built for this board. Also in this change: the MOD_UI switch and the stock-UI path are gone, the chamber-heater model gate arrives with ff-model.cfg/ff-chamber.cfg, and the tests follow. make test: 32 passed, 1 failed. The failure is the probe end-to-end, which dies at "mount: failed to setup loop device for /stick.img" -- leaked loop devices on this build host, not the package. Probe installs nothing, and the default profile's install -> re-install -> boot passes in the same run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
a9299397f4 |
Build the toolchanger from this repo instead of a TOOLCHANGE= path
bin/patch.sh now copies the ff_*.py extras, the ff-*.cfg configs and the HelixScreen printer-database entry out of payload/, the way it already handled report.sh and the nginx/moonraker assets. Both call sites also lose a silent-skip guard. They tested for the files under $TOOLCHANGE and quietly shipped a package without the toolchanger if the path was wrong or unset -- worth having when the source was a checkout that might not be there, wrong now that the files are committed next to the script. A missing file here is a broken checkout, and cp should say so. CI and the test fixture had TOOLCHANGE="" set precisely to trigger that skip, so drop the line; both build the probe profile, which never reaches this step anyway. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GgNhSa1Qvwvxb2nQXT6b1Y |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
22d16f984e |
Run the printer's own boot script for the update test, and cut the suite to 5m
The install test used to re-implement app_startup.sh: it pulled the glob,
MACHINE and PID out of it with sed, then called the installer itself. A
mistake in our reading of the boot script could never be caught that way.
The package now sits on a real FAT filesystem on a loop device exposed as
/dev/sda1, and app_startup.sh runs verbatim through three boots -- stick in,
stick still in, stick pulled. The last one asserts what actually matters:
nginx and moonraker up, klipper launched, the UI in the foreground where
app_startup.sh's own 5-second ps watchdog can see it.
One substitution, declared in .SIMULATED: the printer's kernel has nls_cp936
and a container kernel does not, so /bin/mount retries without codepage= if
the real options are rejected.
Speed. The image carried the partitions but not the stock install, and
test-env.sh sourced the config files after the environment -- so PRINTER_IMAGE
on the command line was overwritten by the empty value in test.env and the
published image was never used. Measured, per replica case: 22s to unpack the
factory image, 37s to install the stock package, 0.7s with the image. The
stock install is now baked in at image build time (bake.sh -- it needs binfmt
and chroot, so it runs privileged and the result is committed) and recorded in
/usr/prog/.BASELINE; a run asking for a different package reinstalls.
make test: ~20m -> 5m26s. Every section header now carries elapsed seconds.
Dropped four checks that could not fail:
* test-abi's execution half was `qemu "$f" -h || [ $? -lt 126 ]`, which
accepts nearly every exit status
* test-ash's md5sum -s probe printed "inconclusive" on every path
* run-tests.sh's per-file `sh -n` "POSIX" loop ran bash, and its bashism
grep flagged `local`, which busybox ash supports
* CI's shellcheck ended in `|| true`
And one that mattered more: the release workflow's final sim-install loop
ignored every exit status, and sim-install.sh exits 0 on a skip -- so the last
gate before publishing could not fail. It now stops on the first failure and
runs with REQUIRE_PRINTER_SIM=1, which turns skips into failures throughout.
Also: the profile lists in run-tests.sh, RECOVERY_PROFILE and release.yml
still said ssh/web/full/helix and would have failed the release job; work/.sim
was shared, so two suites at once truncated each other's packages; and the
fixture half left a 380KB package in work/out where make test-install picks it
up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GgNhSa1Qvwvxb2nQXT6b1Y
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
a5300bde12 |
Split the repo into ship / build / test lanes, and add the printer replica
Two strands, both landing here because they had grown into each other in the
working tree.
The printer replica: the printer's real rootfs.squashfs chrooted under
qemu-mipsel, with /usr/prog and /usr/data from the factory image, so a package
is installed by FlashForge's own updater running on the printer's own busybox,
tar, md5sum and unTar. There are no stubs -- if the real prog partition is
missing the harness refuses to run rather than test against mocks. A release
workflow is blocked on it.
The lanes: a file's directory now says whether it can ever reach a printer.
payload/, assets/ ship
bin/, profiles/ build the package
test/, docs/ neither, ever
* the replica's own infrastructure moved out of bin/ and docker/ and in
beside the harness it belongs to (test/extract-rootfs.sh,
test/build-printer-image.sh, test/printer/Dockerfile{,.full})
* the Makefile has two runners: only RUNSIM gets the docker socket and the
replica's variables, so a build cannot reach the docker daemon at all
* replica settings moved to test.env, which nothing in a package reads
* build-time flags renamed BUILD_* (APPLY, REPORT, KLIPPER, TOOLCHANGE,
MAINSAIL, HELIX). MOD_* now means only the runtime switches written into
anvil.conf and re-read by the printer at every boot
* verify.sh fails if any file in a package is byte-identical to one in
bin/, test/, docker/ or profiles/ -- by content, because a name blacklist
would trip over whatever Mainsail happens to call common.sh
Fixed while mapping it:
* BUILD_REPORT was read only inside the BUILD_APPLY=0 branch, so four
profiles declared it and got nothing. It is honoured on both paths now,
and off outside probe: report.sh copies /etc, passwd and shadow onto the
USB stick, which is a diagnostic, not something to hand out.
* three checks in the brick lint had been quietly dead. Two were guarded on
a payload/boot.sh replaced by the firmwareExe wrapper long ago; the third
was the file glob itself, which matched neither firmwareExe nor the
init.d/S* scripts -- the UI wrapper and every service went unscanned. A
missing file is now a failure rather than a silent pass.
* run-tests.sh overwrote config.env and copied it back, leaving the config
you edited one crashed run away from being a fixture. It uses CONFIG_ENV.
* bin/make-uninstall.sh does not exist and never did; README, run-pre.sh and
full.env all pointed at it. Recovery is the stock package, which is what
test-recovery actually proves.
README is now for someone who wants to flash their printer: what it is, which
file, how to flash, what to do when it goes wrong. The rest moved to
docs/how-it-works.md, docs/building.md and docs/testing.md.
make test: 89 passed, 0 failed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr Shyshatskyi
|
4415905fb3 |
Initial commit: USB-installable custom firmware builder for Creator 5
Builds a modified FlashForge update package that the printer installs by itself from a USB stick, reusing FlashForge's own updater rather than a parallel one. Integration is a single file: /usr/prog/PROGRAM/software/firmwareExe is replaced with a shell script. Everything funnels through that binary, so app_startup.sh, rcS and the init chain stay completely stock. The genuine binary is kept as firmwareExe.stock and remains the default UI. Key findings from the stock firmware, each enforced by a test: - components named *.tar.xz are PLAIN tar; the installer runs bare `tar -xvf` - the .tgz is DES3-encrypted plain tar, not gzip - nothing in init starts Klipper -- firmwareExe runs start.sh, so replacing the UI without S70klipper leaves a printer that cannot move - the installer wipes the software dir before run.sh, so an uninstall package must ship the genuine firmwareExe rather than back one up - ssh already works: stock ships dropbear + an enabled S50dropbear, only the root password is missing Brick-safety is designed in, not bolted on: a five-rung profile ladder starting with a probe that changes nothing, a container-based simulation that actually runs the installer and asserts the printer would still boot, an uninstall round-trip test, and a SAFE-MODE latch for UI crash loops. 75 checks pass. Not yet run on hardware. |