mirror of
https://github.com/Klipper4FlashForge/firmware.git
synced 2026-10-22 05:12:16 -05:00
worktree-timelapse-test-gcode
19
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Oleksandr ShyshatskyiandClaude Opus 5
|
28eb20f3da |
Package numpy, because klippy does not start without it
extras/stepper_resonance_tester.py opens with a bare `import numpy as np`; klippy.py:122 walks EVERY config section through a load_object whose importlib.import_module is not wrapped in except ImportError; printer.base.cfg includes FlashForge's printer.vibration.cfg, which declares [stepper_resonance_tester]. Since klippy moved to our own CPython 3.13 that has not been a printer missing resonance testing. It has been a klippy that never reaches ready. So this is not a scientific extra. anvil-klipper depends on anvil-python-numpy the way it depends on cffi, except that the others mean "klippy imports this" and this one means "klippy starts". It also gives back SHAPER_CALIBRATE and TEST_RESONANCES, dead since the same switch, and the VFA calibration behind STEPPER_RESONANCE_FACTORY_CALIBRATE that docs/notes/44-vfa-calibration.md traces end to end. WHAT WAS NOT OBVIOUS, in the order it bit. numpy 2.x builds with meson-python, and meson has never heard of _PYTHON_SYSCONFIGDATA_NAME -- the trick every other python-* recipe here leans on. pkg_mesoncross writes a real cross file instead, naming the $PKG_XW wrappers rather than the raw toolchain: the wrappers carry -EL -mnan=2008, and a cross file that reaches past them builds a tree that passes every gate which only reads symbols and is refused at exec() on the printer. pkg_pywheel grew a PKG_PY_PIP_ARGS branch to pass the file as --config-settings, which is the one thing PEP 517 does standardise. The cross file's `python =` is the BUILD python, and that is not a mistake: meson introspects an interpreter by running it and asking sysconfig, and pkg_pytarget has already made that sysconfig answer for mipsel. The old trick does reach meson, just not by its own mechanism. numpy's meson.build hard-errors below gcc 8.4 and ours is 7.2. The gate is the SciPy Toolchain Roadmap's support statement, not a requirement of the code: this toolchain compiles if constexpr, structured bindings, std::optional, std::string_view, and pocketfft_hdronly.h -- 3635 lines of the heaviest template C++ in the tree, and the half np.fft.rfft needs. Probed before the floor was lowered, and lowered rather than removed, behind a guard that fails loudly if upstream moves the check. 19 extension modules, every one at e_flags 0x70001405/0x70001407, 14MB shipped after pruning tests/, f2py/, stubs and static libs. meson-python is self-hosting, so pkg_buildpython runs meson and ninja just to install the backend -- hence ninja-build in the image and PYPKG_HOST_LIST in dependency order rather than alphabetical. And pip puts meson and cython in work/.py-host/bin, which nothing had ever looked in: without that on PATH the backend fails to install with `meson executable "meson" not found` while pip list shows it right there. THE GATE THAT WOULD HAVE CAUGHT ALL OF IT is qa/replica/test_klippy_extras_import.py: walk the real /usr/data/config include graph, map every section to its extra, import each on the printer's own interpreter. It needs no MCU, because config load happens before klippy:mcu_identify. Nothing asked that question before, which is how two comments in this repo came to assert the module guarded its own import. test_numpy.py asks for numeric answers rather than a module object -- linalg.lstsq specifically, because -Dallow-noblas selects the bundled lapack_lite and _r2_sinusoidal_fit calls it every calibration round. And test_install.py's include check read only ff-*.cfg, so FlashForge's four -- printer.vibration.cfg among them -- were never asked about at all. ff-toolchange.cfg wraps the two search commands in _FF_SHAPER_PREP, as SHAPER_CALIBRATE already was. STEPPER_RESONANCE_FACTORY_CALIBRATE is deliberately NOT wrapped: it is a gcode_macro, not a registered command, and same-named sections merge under RawConfigParser(strict=False) -- a wrapper would not wrap it, it would BE it, with rename_existing pointing at itself. It needs none, since its body is two calls to a command that is wrapped. No SAVE_CONFIG in any macro either; it restarts klippy and stays the operator's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EJr2x7aDaY889qwSxbdLt8 |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
6a4067e863 |
VFA calibration is one import away, and that import stops klippy booting
The stock app's "magic" VFA calibration is not app-side math at all. Traced
from CommMgr::calibration_vfa (0x0075e694), it sends five G-code strings and
the last one is STEPPER_RESONANCE_FACTORY_CALIBRATE -- a macro in FlashForge's
printer.vibration.cfg over klippy/extras/stepper_resonance_tester.py, which
searches amplitude and phase for the motor torque-ripple compensation the
closed-loop driver applies at 1x/2x/4x the electrical frequency. Every piece is
already here: the module is byte-identical in our vendored fork, the mclib
sections are in our printer.base.cfg, and mclib_set_resonance_damp is in the
stock mainBoardGD dictionary we never rebuild.
WHAT THE TRACE TURNED UP ON THE WAY. Two comments in this repo said the numpy
gap was a small item because the module guards its own import. It does not:
line 1 of stepper_resonance_tester.py is a bare `import numpy as np`, and
klippy.py:122 loads EVERY config section through a load_object that does not
catch ImportError -- the default=None argument only covers a missing file.
printer.base.cfg:19 includes printer.vibration.cfg, which declares
[stepper_resonance_tester]. So on the CPython 3.13 klippy has run on since
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
bd0df7c28d |
FF_PA_CALIBRATE runs FlashForge's PA sweep and reports the number
The host half of the touchscreen's automatic pressure advance, ported from CommMgr::paTestMgr. ff_pa.py drives the recovered sequence verbatim -- the same seven candidates in the same scrambled order, the same Y ladder, the same 5-sweep / 3-winner / mean-of-3 selection, the same "only verdict 9 passes" -- and prints the result. It saves nothing and applies nothing; the measurement itself stays where it already works, in the closed eBoard firmware we ship unchanged, reached through [pa_adjust]'s existing command wrappers. It borrows those wrappers rather than declaring its own. Declaring our own would be safe (lookup_query_command builds a fresh wrapper per call and SerialRetryCommand registers per send), but it would duplicate a closed MCU's protocol strings and would need a config-time callback on [mcu eboard], letting a report-only tool block startup on a machine whose eBoard is unplugged. Deliberate divergences, all commented at the point they matter: the extruder's own pressure advance is restored on exit, which the app never does because SET_PA_ADVANCE overrides it and we do not; a run where every line came back with the same verdict is refused, because "smallest passing wins" then yields the smallest candidate wearing a measurement's clothes; cold extruder, empty tool and out-of-range geometry are refused before anything moves rather than failing mid-line with the eBoard armed; SET_KINEMATIC_POSITION is not replayed, since our sweep is plain G1 under Klipper's own kinematics. What is NOT changed is the timing. pa_action is an immediate MCU command and there is no M400 between the last G1 and ACTION=0; the scorer was tuned against that, and Klipper's own input throttle is what bounds the window. FF_PA_PROBE draws one line and prints one verdict. It exists because the first real question on hardware is whether the eBoard discriminates between candidates at all, and answering that should not cost a full run. Checked: pyflakes clean; qa/static 143 passed; both packages build and both files land in the .ipk; the include set and the shipped ff-*.cfg set match the way the replica gate checks them. The selection logic was exercised against stubs -- it picks 0.0150/0.0200/0.0150 -> 0.016667 from a canned verdict table, refuses an all-9 run, and refuses fewer than three winners. Nothing in the repo can load klippy off-hardware, so behaviour on a machine is still unproven. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Kw2rehvQJ7LSc28hPikfqk |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
2e9f2214a0 |
Merge master into the comment trim
Master moved twelve commits while this branch ran: patch.sh became
payload.sh, run-pre.sh and run-append.sh became runFirmwareExe.sh, the
install manifest and anvil.conf are gone, and versions.env was trimmed
harder there than here (53 comment lines against 185).
Resolved so the merge contributes comments and nothing else. Verified
mechanically: every one of the 270 files in the tree now carries
exactly master's code, and no file this branch touched is resurrected.
- master's deletions accepted: bin/patch.sh, bin/verify.sh,
installer/run-{pre,append}.sh, anvil.conf.in
- versions.env: master's, which is the better trim
- 26 files master only renamed through: this branch's trim kept, with
patch.sh -> payload.sh applied
- 14 files master rewrote to describe the new installer: master's
text taken wholesale, to be re-trimmed separately
- stale references to the manifest and run-append.sh corrected in
pkgs/3rdparty/busybox/build.sh and pkgs/moonraker/build.sh
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NMMpQTajN7RDJySrhQ2cTE
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
b98552386b |
No FlashForge component ships; patch.sh is payload.sh
The software component existed to give the mod one exec hook. It is not needed for that any more, so a release now carries our installer and our payload and nothing else: 26MB smaller, 21MB of which was FlashForge's own firmwareExe travelling back to the partition it was already on, along with their klippy tree, their wifi module and their configs. WHAT IT GIVES BACK. /usr/prog/PROGRAM/software is no longer rewritten, so the version directory there stays FlashForge's with their genuine firmwareExe inside it -- and app_startup.sh's five-second watchdog, which copies one out of that directory when nothing by that name is running, has something to copy again. A printer whose payload fails to install now comes up on the stock UI instead of a blank screen. That recovery has been broken since we started deleting their binary out of a component we shipped; docs/how-it-works.md claimed it worked throughout. THE ROOT PASSWORD moves with it, because it used to ride in the component's shadow file. bin/pack.sh bakes ROOT_PW_HASH into the installer, single-quoted -- an unquoted $6$... would have the printer's shell expand $6 as a positional parameter and ship a truncated hash -- and the installer writes it to /usr/prog/etc/shadow. The random-password path asks one question instead of four: FlashForge ship the same root hash on every machine, so a live shadow still carrying it means nobody has set a password and it is ours to set. The .prev-root-hash recording, the shipped-vs-live comparison and the restore-it- afterwards step were all machinery for surviving a file that nothing overwrites any more, and are gone. pack.sh checks the hardcoded constant against the stock package it built from, because getting it wrong is silent and total: every printer would read as "already set" and none would ever get a password. bin/patch.sh is bin/payload.sh. It patched nothing any more -- no splice, no shadow edit, no firmwareExe deletion -- and what is left is the payload: the feed installed by the printer's own opkg, the seeded config, the manifest. Its sections are renumbered 1-5 with no gaps, and the five tombstone comments marking where removed sections used to be are gone with them. test_recipe_layout.py loses six of its nine tests. Each compared the tree against a description of the tree -- no fourth directory kind, each kind still used, 3rdparty carries no files of ours, installer/ holds exactly two named files -- and none could go red for a reason a printer would notice, while all six went red the moment the layout legitimately changed. The two that bite are kept: a payload/ path escaping $MODDIR, and a recipe name at both levels, which makes pkg_dir silently build the wrong package. SW_VER is deleted from config.env.example and both workflows: it stamped a component that is not built. Gates: 112 static tests pass. The generated installer is verified after substitution -- all four lines present, and `sh -n` over the result, because a hash with a stray character in it would otherwise reach a printer as a syntax error whose only symptom is a package that installs nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CH7Yb4wuPhnppHAB25UX7e |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
9b5b562a22 |
Comments: keep the contract and the trap, drop the narration
A sweep over every commented file in the tree. What is kept is what a reader needs to change the code safely: what a thing is, its signature, the invariants other files depend on, and the traps that fail silently (find's non-zero exit under set -e, s6's baked-in libexecdir, the -lm that decides whether _sqlite3 exists, opkg's compile-time prefix). What goes is the narration around it: incident histories, arguments against alternatives already rejected, restatements of the code below, and paragraphs re-explaining a rule three files already state. One paragraph in pkgs/3rdparty/opkg/build.sh was duplicated outright. Comment lines 7971 -> 6158. Every change is comment-only: no non-comment line differs anywhere in the 86 files, checked mechanically. Commented-out config directives (#fast_feed: 30000 and friends) are documentation of defaults and are kept. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NMMpQTajN7RDJySrhQ2cTE |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
57c776ee63 |
test/ is deleted; qa/ is the only suite
The eight files were already gone from the working tree. What this commit does
is everything that had to follow them, because nothing else had been updated
and three of the four things pointing at test/ were on a build or release path.
WHAT WAS STILL WIRED TO IT
pkgs/klipper/build.sh ran test/test-chelper.py under `|| pkg_die`. With the
file gone `make packages` would have died on every
klipper build.
bin/verify.sh ran the same script and read a non-zero exit as
"c_helper.so does not match the shipped klippy tree
(stale build)" -- a false red on every release verify
rather than an error about a missing file. Its
ship-boundary check also globbed `find bin test
docker`; that is `find bin docker` now.
pytest.ini testpaths = test, so a bare `pytest` collected
nothing at all. It is `qa` now, which is what the
comment above it was already describing.
qa/static/ PY_GLOBS listed test/*.py and test/integration/*.py,
test_shell_syntax.py and _files() makes an empty glob a collection error
on purpose -- so the whole static lane failed to
collect, taking 93 unrelated tests with it.
Also dropped: the `pytest -- the host-side tree` step in ci.yml, the `test-py`
target with its .PHONY entry and two help lines, and qa/conftest.py's opening
paragraph explaining why qa/ sat beside test/ rather than inside it.
THE CHELPER SYMBOL CHECK IS GONE, NOT MOVED
It compared the .so's dynamic symbols against what klippy cdefs, and it was the
only check for a c_helper.so built from older sources than the klippy tree
beside it -- the failure that shipped once and bricked klippy startup on
hardware, because cffi resolves lazily and a stale .so imports cleanly.
qa/replica/test_install.py covers the ELF arch and nan2008 and nothing else.
What stands in its place is structural rather than a gate: pkgs/klipper
compiles the .so from the chelper sources of the very tree it ships, so the
mismatch is not something the recipe can produce. The comments in
pkgs/klipper/build.sh and docs/building.md now say that is the whole defence.
DOCS
docs/testing.md's "what is left in test/" section is now "test/ is gone", and
it names what went uncovered rather than leaving it to be found: the tool-frame
arithmetic in ff_toolchange, the chamber macros' per-model branching, the stamp
discipline in ff-startup.py, and the verification G-code against the shipped
macros. docs/hardware-testing.md said `make test-py` keeps the two
gcode/creator5-*.gcode files honest; it now says nothing does, and to read them
by hand when a macro is renamed. Paths corrected in docs/building.md,
docs/printer-replica.md and docs/qa-migration.md's intro.
Verified: `pytest` collects 366; qa/static is 136 passed, up from 43 because
the lane could not collect before this. Its 110 errors are this checkout's
missing vendor/opkg-utils, the same cause before and after. bin/verify.sh,
pkgs/klipper/build.sh and the Makefile parse; ci.yml is valid YAML.
Not touched, and stale before this commit: tools/python/README.md and
tools/supervisor/README.md still document test/integration/printer/case-*.sh
and a printer-exec.py that earlier commits removed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016bLGYpVAeVzAX6NeAxcU42
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
7dda2fee0d |
ABI gate: one check, on the filesystem the kernel loads from
mips_abi_gate answered "did this recipe stage the right ABI", once per recipe, over a PKG_ROOT. Four implementations of that one rule had grown up around it -- the function in bin/common.sh, the call at the packaging boundary, two recipes that re-ran it on their own output, and a readelf|grep nan2008 in bin/verify.sh reading a single .so -- and between them they still could not see the largest binary we ship: bin/patch.sh unpacks HelixScreen into /usr/prog and no gate ever read that tree. pkgs/helixscreen/pkg.conf had carried a note saying exactly that. qa/replica/test_abi.py replaces all of it. It walks the installed filesystem on the printer's own CPython -- one qemu process, ~24000 files, 8s -- and reads the ELF header of everything it finds. That is the only place where what the recipes staged, what opkg unpacked, what patch.sh copied and what FlashForge shipped are all visible at once, and it is the thing the question is actually about: the kernel does not read a PKG_ROOT. MEASURED, not assumed: 603 ELF objects on a real install. 569 the loader handles and 568 of those conform. The exemptions are the other two, both narrow. ET_REL objects (33 stock .ko files, plus a stray python.o in FlashForge's CPython 3.8 tree) are not asked about the FP ABI, because insmod and the linker read them rather than the ELF loader and kernel code has no FP ABI to agree about -- by e_type, so the python.o needs no case of its own. And /usr/prog/PROGRAM/ffstartup-arm is an ARM binary FlashForge ships that nothing in the stock tree names; exempt by path, so an ARM binary anywhere else still fails. The architecture is settled before anything is classified, and that ordering came from planting a counterexample rather than from reasoning: a big-endian object decodes its own e_type big-endian, so ET_DYN reads back as 768, lands in neither bucket, and an earlier draft looked straight at it and passed. Every gate here was driven red on purpose before it was kept -- an x86-64 .so in $MODDIR, a real mipsel object with EF_MIPS_NAN2008 cleared, an ARM binary outside the exempt path, a big-endian object, an x86-64 object among the .ko files, and a truncated sweep. test_install.py's two chelper header tests go with it: they read one file out of the several hundred on the same fixture, a strict subset of what now runs. Whether the package ships a chelper at all stays test_install.py's question and is not asked twice. test_ipk.py's assertion that the gate exists becomes an assertion that a second one has not grown back beside it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NpH87TJJEjDyRhQPu18pgp |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
214a4e58b9 |
anvil-core owns firmwareExe and start.sh; kill the 3.8.2 myth
Three things, all downstream of klippy having moved to $MODDIR and our own
CPython 3.13.
SECTIONS 8 AND 9 ARE GONE. Both files install at $MODDIR/prog/ and
anvil-link-prog.sh points the stock paths at them -- from run-append.sh on a
flash and from anvil-core's postinst on `opkg upgrade anvil-core`. Nothing
execs either between run.sh distributing the component and link-prog running,
so the component copy bought no ordering. It cost, though: our firmwareExe
with no payload behind it logs "no anvil-env.sh", finds no s6-svscan and then
`while true; do sleep 3600; done` -- a printer that boots to a blank screen for
ever and never reaches app_startup.sh's own recovery. Absent, that recovery
restores a firmwareExe from a version directory. So patch.sh now REMOVES both
from the component: the stock component carries them, and deleting the two cp
lines alone would have shipped FlashForge's originals.
SECTION 5c IS GONE. It did no work -- two du prints and a comment whose first
line was the falsehood below.
THE FALSEHOOD. "klippy runs on the stock 3.8.2, started by
/usr/prog/klipper/start.sh" had spread well past patch.sh, and it had reached
things people follow:
- docs/toolchange.md told owners to scp the ff_*.py extras to
/usr/prog/klipper/klippy/extras/ -- the wrong partition -- and warned that
an OTA would delete them there. Same wrong path in three ff-*.cfg headers
and in ff_toolchange.py.
- versions.env labelled our CPython 3.13 and its eighteen packages
"SHIPPED BUT NOT USED".
- docs/how-it-works.md said Moonraker runs on python3.8, contradicting line
126 of the same file.
All corrected. Historical notes keep their history and get a [Superseded: ...]
marker instead -- including 80-s6-migration.md's "s6-rc-compile is not
shipped", which pkgs/3rdparty/s6-rc/build.sh contradicts deliberately.
Found while reviewing: every docs/ reference to the ff-*.cfg pointed at
pkgs/anvil-core/payload/config/, which does not exist -- they are
anvil-klipper-config's. 28 references across 9 files, several of them broken
markdown links on the published site. All 24 relative links under docs/ now
resolve.
Gates: verify.sh syntax-checks prog/start.sh and prog/firmwareExe out of the
payload (firmwareExe was never syntax-checked before) and fails if the
component carries either. A new replica test asserts both stock paths are
symlinks into $MODDIR/prog and do not dangle. One existing replica test stubbed
firmwareExe by redirecting into $FE, which now writes THROUGH the symlink into
the packaged wrapper; it moves the link aside instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01La5MaDKqTW675k5fsXsP6S
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
53292b4d78 |
klippy runs from $MODDIR, so patch.sh stops staging a second copy
The klipper s6-rc service already execs klippy under $FF_PYTHON -- our own CPython 3.13 -- but it read the tree from /usr/prog/klipper/klippy: a copy bin/patch.sh staged into the software component out of the very anvil-klipper package that also installed it under $MODDIR. Two copies of one tree, and the one that ran was the one a stock flash could overwrite. One line moves it: KLIPPY=$MODDIR/klipper/klippy/klippy.py. That deletes sections 1 and 2 of bin/patch.sh -- the cp -a of the klippy tree, the chelper.tar wrapped around a .so the package already contained, the rm -rf of the stock extras/ and kinematics/, and the ff_*.py copy. The software component now carries firmwareExe, start.sh and run.sh and nothing else of ours, which is the phase 7 end state in docs/notes/80-s6-migration.md. It also lets pkgs/klipper declare the dependencies it has had since the interpreter moved. cffi and greenlet reached the printer only through a hardcoded list in patch.sh, so every build had them and no `opkg install anvil-klipper` did; pyserial and jinja2 came only as anvil-moonraker's Depends, so a BUILD_MOONRAKER=0 build shipped a klippy that dies the first time it opens an MCU. Both are real Depends now and MOD_ROOTS drops to ten. Also repointed: run-append.sh's __pycache__ sweep, which was clearing the tree nothing imports while klippy writes bytecode into the one it does -- and those runtime-created directories are in no .install-manifest, so the manifest delete walks past them. verify.sh reads the klippy tree and c_helper.so out of the payload (extracting ./klipper alone), and gains a check that the component does NOT carry a klippy tree. New replica tests assert the service execs the payload tree under $FF_PYTHON and that cffi/greenlet/serial/jinja2 import on it. Comments in anvil-env.sh, verify.sh and pkgs/klipper/pkg.conf that still said klippy runs on FlashForge's 3.8.2 are corrected; that stopped being true when FF_PYTHON moved. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01La5MaDKqTW675k5fsXsP6S |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
5f36cd14e3 |
Say what the code does, not what it used to
Comment cleanup over the files this branch touched, plus the dead code the
last two commits left:
* pkgs/lib.sh's pkg_buildopkg, 89 lines building the x86-64 opkg nothing
calls any more, and the qa test that checked its configure flags
* patch.sh's "5d. what is no longer here" section, and its mkdir of the
prefix directories -- build-payload.py replaces $PAYLOAD_DIR wholesale, so
they were created and then thrown away
* the two emptied creator5*-config recipe directories
The rest is prose. Comments that narrated a previous design -- what the host
opkg did, which package used to own the chamber config, that prog/ was once a
sibling of payload/ -- are gone rather than shortened; the ones explaining
constraints that still hold are trimmed to the constraint.
docs/building.md and docs/toolchange.md describe the packages as they are.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VpUmBb4xtmnmtTosonqeNi
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
26e3388895 |
The printer picks its own chamber config
anvil-klipper-config carries printer.base.cfg and BOTH chamber configs, under config/chamber/<Machine>.cfg, and anvil-link-prog.sh symlinks the right one into /usr/data/config. The model comes from FlashForge's own app_startup.sh, which carries MACHINE= at its top -- so the machine answers the question about itself instead of a build being told the answer and having to be right. THAT REMOVES A PACKAGE PAIR THAT COULD NOT COEXIST. anvil-klipper-creator5-config and -creator5pro-config each owned config/printer.chamber.cfg, so they Conflicted, opkg refused the pair with exit 255, and bin/patch.sh had to pick one from TARGET_MACHINE. The payload was model-specific for the sake of one file; it is not any more. TARGET_MACHINE still names the output, because runFirmwareExe.sh gates on it. THE SOFTWARE COMPONENT'S klipper/config IS NOW BYTE-IDENTICAL TO STOCK -- diffed, not assumed. printer.base.cfg was the last thing patch.sh wrote there. The seven configs we do NOT modify stay stock and unpackaged, and that works because Klipper resolves [include] against the directory of the file doing the including, by the path as opened rather than the resolved target (configfile.py: dirname = os.path.dirname(source_filename)). So a symlinked printer.base.cfg still finds printer.filament.cfg and friends beside it in /usr/data/config. run-append.sh no longer copies the files the linker links -- copying would only put a real file in the way of the symlink about to replace it. moonraker.conf and moonraker-custom.conf keep their branches: those are user-editable, and a symlink would lose the .mod-new preservation a tuned trusted_clients block depends on. anvil-klipper now Depends on anvil-klipper-config, which is where that dependency always belonged -- it used to arrive only because the per-model packages pulled it in, and naming it in MOD_ROOTS would answer a missing dependency in the one place a printer's own `opkg install` never reads. prog/ is gone as a recipe subtree. Its own test said an empty one was the good outcome; it emptied, so the rule went with it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VpUmBb4xtmnmtTosonqeNi |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
722804364a |
The boot path follows the package, not the last .tgz
firmwareExe and start.sh have to sit on /usr/prog because FlashForge's own scripts look for them by absolute path. They are package files now (anvil-core, under payload/prog/) and anvil-link-prog.sh replaces the copies on /usr/prog with symlinks into $MODDIR, so what the printer executes is what the package owns. WHAT THAT BUYS: `opkg upgrade anvil-core` on a running printer used to update $MODDIR and change nothing the boot path reads -- the new firmwareExe sat in the payload while the old copy kept being run, and only a full .tgz could land it. The postinst calls the same script, so an upgrade is now enough. THE LINKS ARE MADE AFTER EXTRACTION, and that is the whole ordering. Stock run.sh distributes the component at its lines 107-179 and run-append.sh does not extract anvil.tar.xz until line 180, so the component cannot carry the links itself: on a first install they would dangle, and on an upgrade they would resolve to the payload being replaced and silently install the previous release's files. The component still carries real copies. That is what keeps a printer bootable when the payload is not there, and patch.sh now takes them from the installed payload rather than from a directory beside the recipe -- the same move it already makes for the klippy fork. A postinst needed the recipe system to have maintainer scripts at all, so build-packages.sh copies pkgs/<name>/control/* into CONTROL/. Note what that exposes: patch.sh installs with a host opkg under --offline-root --force-postinstall, and that combination DOES run maintainer scripts, with IPKG_INSTROOT empty, so their absolute paths are taken as written. Inside the build image that is harmless -- there is no /usr/prog there -- and the script's guard is what makes it a quiet no-op rather than an error. pkg_payload_hash covers control/ too, or an edited postinst would read as "already current". It takes two find calls because `find a b` with b missing exits non-zero, which under the recipes' set -euo pipefail killed every build whose recipe has no control/ -- silently, the existing 2>/dev/null swallowing the one line that said so. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VpUmBb4xtmnmtTosonqeNi |
||
|
Oleksandr Shyshatskyi
|
21cf19f2e2 |
BUILD_KLIPPER=stock kept a Klipper nothing else was written for
The flag chose between the Klipper4FlashForge fork and FlashForge's own v0.12 tree. Every other thing the mod ships had already stopped being neutral about that choice: the ff_*.py extras are the fork's package's own files, the ff-*.cfg declare sections only those extras implement, and c_helper.so is compiled from the tree it ships beside. A stock build was a printer with the mod's configuration and none of the code behind it -- and the previous commit turned that into a hard error, which is what showed the flag had nothing left to select. So it is gone rather than guarded. pkgs/klipper loses its PKG_WHEN, patch.sh section 1 loses the case and is straight-line, verify.sh section 7 stops being conditional -- a package with no klippy tree is now unrepresentable rather than merely gated -- and fetch-assets.sh pulls the fork tarball on every build, the way it already does s6 and CPython. Two fixture-lane repairs come with it. test/run-tests.py said BUILD_KLIPPER=stock to keep the lane off the network. That reason had already expired: since the payload became the feed installed, the lane runs build-packages first, and that needs the ~203MB Ingenic toolchain from vendor/ for CPython alone. The Klipper tarball is 27MB beside it, cached on versions.env like everything else. The lane now builds the fork, so verify.sh's ABI and symbol checks on c_helper.so run there for the first time. And the synthetic Moonraker tarball unpacked to moonraker-fixture/, while pkgs/moonraker/build.sh looks for moonraker-$MOONRAKER_VERSION/ -- the shape GitHub generates. That lane could therefore never build anvil-moonraker at all. It passed only while work/pkg/moonraker held a tree from a REAL build, because pkg_stamp keys on the version and not on the tarball; the first run against a cold cache died with "no moonraker/moonraker.py", a true statement about a tarball nobody could have made work. CI caches work/pkg on versions.env, so the first run after any pin bump would have hit it. Confirmed pre-existing by running the parent commit against the same cold cache: it fails identically. make qa-static 261 passed; make test 9 gates, 0 failed, with the fixture lane building Klipper and Moonraker from its own assets for the first time; CI's shellcheck gate clean. |
||
|
Oleksandr Shyshatskyi
|
53a2bc010b |
Five extras were on the firmware partition because of a flag
pkgs/klipper/prog/klippy/extras/ff_*.py sat on /usr/prog for one reason, and it was not a technical one: anvil-klipper answers to BUILD_KLIPPER and the extras answered to BUILD_TOOLCHANGE, and a package cannot hold files gated differently from itself. So they stayed outside every package, copied to the firmware partition by bin/patch.sh. They are anvil-klipper's now, at payload/klipper/klippy/extras/, and BUILD_TOOLCHANGE no longer gates them. An extra is inert until a config section instantiates it, and those sections are anvil-klipper-config's, where the flag still applies -- so the gate bought nothing and cost the mismatch. Inside the tree is the only place they can go: klippy resolves an extra as dirname(klippy.py)/extras/<name>.py (klippy.py:90-103), with no search path and no environment variable. bin/patch.sh still copies them into the software component -- klippy runs from /usr/prog until the s6 service lands -- but reads them out of the payload now, the way section 1 already reads the klippy tree, so the recipe directory is not a second source for a file a package holds. It names the five rather than globbing ff_*.py: the fork ships an ff_eddy.py of its own and the glob could not tell them apart. Verified the component still gets exactly the same five files. The flags are coupled by this, and patch.sh says so rather than shipping a broken printer: BUILD_TOOLCHANGE=1 with BUILD_KLIPPER=stock now fails with the fix in the message. It has to -- the extras are written against the fork's v0.13 klippy, and dropping them onto FlashForge's v0.12 is the half-overwritten mixture that shipped as v20260824. The fixture lane in test/run-tests.py used exactly that pair (BUILD_KLIPPER=stock is a no-network convenience there) and now sets BUILD_TOOLCHANGE=0. pkgs/klipper/pkg.conf gains PKG_STAMP_EXTRA=$(pkg_payload_hash): KLIPPER_VERSION says nothing about payload/, so without it an edit to ff_toolchange.py would read as "already current". make qa-static 261 passed; make test 9 gates, 0 failed. The .ipk carries all five under klipper/klippy/extras/. |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
8f54d87513 |
Comments that restated the code, and a few that had stopped being true
A pass over what this branch added, on the same rule as the branch merged before it: a comment earns its place by saying WHY, and the ones here had grown into essays. bin/patch.sh 781 -> 555 lines, its comments 523 -> 297. The install section keeps the three things a reader cannot get from the code -- the set comes from pkg_recipes so the BUILD_ flags are read once, -dev is filtered by package name because nine recipes ARE the dev package, --force-postinstall is what stops the database saying "unpacked" about files that are in place -- and loses the narration around them. Sections 3 and 4 were mostly an account of the sections that used to be there; they are four lines now. The CPython and libsodium headers keep the facts that took measuring (FF_PYTHON's callers, the missing _sqlite3 that pins Moonraker, libnacl's __file__ arithmetic and its dependence on "lib" appearing in the prefix) and drop the rest. Same for pkg_buildopkg, the payload-path block in bin/common.sh, and the four test docstrings. Also fixed, found while reading rather than looked for: bin/build-packages.sh still called itself "the proof-of-concept half" and said bin/patch.sh "still stages libsodium into the payload exactly as it did and `make build` is untouched". None of that survived the migration. The -maxdepth 1 comment had been wedged between `# shellcheck disable` and the command it applies to. Eleven "bin/patch.sh stages ..." references across pkgs/, test/ and qa/ described a script that stages nothing now. Payload bytes unchanged except S40s6, whose comment was one of the eleven. qa-static 259; make test 9 gates, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
51f524b5cf |
Merge worktree-comment-cleanup
The branch tightens comments across the tree -- removing the ones that
explained the code's past rather than the code. It branched before both the
pkg/ -> pkgs/ rename and the migration of bin/patch.sh onto the feed, so most
of its conflicts were with text describing code that has since moved or gone.
Resolved by taking its brevity and correcting what it could not have known:
bin/patch.sh ours, entirely. Its only non-comment change there deletes a
trailing comment from a line this branch rewrote, and every
comment it edits annotates one of the eight staging sections
that no longer exist.
bin/fetch-assets.sh ours. Theirs reinstates a KLIPPER_FORK condition; that
knob was deleted with the move to pkgs/ -- a recipe names its
source exactly once.
pkgs/lib.sh theirs, except the two hunks where our side carries code it
never saw (pkg_payload_hash and pkg_dir; the PKG_CC_SHARED
branch of pkg_build). Its prose, our functions.
bin/common.sh theirs throughout, with pkg/ -> pkgs/ and $MOD_PAYLOAD ->
$PAYLOAD_DIR, and minus its claim that the toolchain is used
"by bin/patch.sh for Klipper's c_helper.so" -- patch.sh
compiles nothing now.
pkg/anvil-core/build.sh deleted. The file is pkgs/anvil-core/build.sh and
was rewritten as a loop; its comments are superseded.
Two of its deletions were right about things this branch had wrong:
bzip2 named a pkg_make verb that was retired into pkg_build.
zlib claimed a ./configure line of its own as "the one sanctioned
exception". It has not had one since PKG_CONFIGURE_AUTO=0 and CHOST; the
rule in qa/static/test_ipk.py is absolute, and its docstring says so. I had
kept that paragraph in an earlier pass of this merge and was wrong to.
Three stale hints in the Makefile went with them: work/pkg is filled by
`make packages`, not by bin/patch.sh, in the two gate targets and one error
message.
Verified: the payload rebuilds to the same 31 packages and 154 ELF objects,
and the six files whose bytes changed are five payload scripts this branch
comment-edited (zero non-comment lines between them) plus the opkg status
file, which records their sizes. qa-static 259; make test 9 gates, 0 failed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
dff6fe3198 |
anvil-core was holding four other components' configuration
It held all of it for one reason: it was the first recipe written, so
everything of ours started there. With the files visible per recipe the
misfilings read straight off the tree.
moonraker.conf -> pkgs/moonraker, with the server it configures
ff-*.cfg, printer.base.cfg -> pkgs/klipper-config (new)
printer.chamber.cfg -> pkgs/klipper-creator5-config and
pkgs/klipper-creator5pro-config (new)
PRINTER.BASE.CFG WAS IN pkgs/klipper AND THAT WAS WRONG TWICE. It is Klipper
CONFIG rather than any part of the Klipper program -- it is the file carrying
[include ff-toolchange.cfg] and [include printer.chamber.cfg], so it is the
entry point to everything in the config package -- and pkgs/klipper is gated
on BUILD_KLIPPER while this file is installed under BUILD_TOOLCHANGE. A
BUILD_KLIPPER=stock build has no pkgs/klipper recipe at all and was still
reaching into its prog/ directory for a file.
THE ff-*.cfg ARE NOT IN pkgs/klipper EITHER, for the same gate: ask for
BUILD_KLIPPER=stock and that recipe does not exist, and the toolchanger would
have become a silent casualty of a flag about which klippy is shipped.
FlashForge's own klippy loads ff_*.py and reads these .cfg perfectly well.
They travel with the feature they configure and the feature's flag is
BUILD_TOOLCHANGE.
THE MODEL IS A PACKAGE NAME NOW, not a build flag. printer.chamber.cfg.creator5
and .creator5pro used to be picked by patch.sh from TARGET_MACHINE and copied
under their real name, which made the model a property of the BUILD: two
builds of one commit produced different bytes under one version number, and
nothing in the feed or on the printer recorded which machine a package was
for. Both are built on every build; patch.sh stages the one this tarball is
for. This is the feed's first use of Provides/Conflicts, both on a virtual
anvil-klipper-chamber-config -- Provides so anvil-klipper-config can depend on
"a chamber config" and either satisfies it, Conflicts so a printer cannot
install both and have two packages own one path with opkg letting whichever
unpacked last win. Neither field alone expresses "exactly one", and
printer.base.cfg's [include printer.chamber.cfg] is unconditional, so "none"
is a printer whose klippy does not start.
printer.chamber.cfg also CHANGED WHERE IT LANDS: $MODDIR/config, like the
ff-*.cfg, instead of riding the software component to /usr/prog and reaching
/usr/data/config only because the stock run.sh copies that whole directory.
run-append.sh overwrites it on the same mod-owned terms, and by this repo's
own rule -- files the update overwrites must say so -- it now carries the
DO NOT EDIT banner the ff-*.cfg have always had. test_config_ownership.py is
what demanded that, which is what it is for.
TWO FLAGS BECAME HONEST, and both are behaviour changes:
* BUILD_TOOLCHANGE=0 shipped the toolchanger's .cfg anyway, because they
rode in anvil-core while everything else the flag controls was gated.
It no longer does.
* BUILD_MOONRAKER=0 overwrote the printer's moonraker.conf with one written
for a server it was not going to install -- [webcam anvil] and all. It no
longer does, which is what docs/building.md already said the flag meant.
WHAT DELIBERATELY DID NOT MOVE. init.d/ and etc/s6/ look like they belong to
the services they start, and moving them would brick a printer: those services
are gated at RUNTIME by flags in anvil.conf, while the packages that would own
the scripts are gated at BUILD time. A BUILD_KLIPPER=stock tarball has no
anvil-klipper package, so it would carry no S70klipper and nothing would start
Klipper at all. The script that reads a flag must not sit behind a second,
different flag. nginx.conf stays for a plainer reason: nginx is the rootfs's
binary and there is no package here to give it to.
Measured:
* 44 packages (41 + three config packages), feed builds clean.
* anvil-moonraker carries config/moonraker.conf; anvil-klipper-config the
seven ff-*.cfg; each model package one printer.chamber.cfg. Checked by
listing the built .ipk, which is how the first attempt was caught staging
moonraker.conf and never shipping it -- pkg_ship takes an explicit list.
* make test: 8 passed, 0 failed, including unpack/patch/pack/verify.
* bin/verify.sh gained two gates over the payload -- a chamber config and
the ff-*.cfg -- and BOTH WERE PROVEN BY BREAKING THE BUILD: deleting the
staging line fails patch.sh's own check, and bypassing that too fails
verify with "mod payload has no config/printer.chamber.cfg". Which is
also the proof that a normal build carries them.
* make qa-static: 256 passed. make test-py: 181 passed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
f67e7a269f |
ls pkg/ was thirty-eight entries and four of them mattered
pkg/ -> pkgs/, and the thirty-four recipes that build a pinned tarball and
carry no files of this repo move down a level into pkgs/3rdparty/. What is
left at the top is the four components a person actually edits: anvil-core,
klipper, moonraker, helixscreen.
THE LINE IS MECHANICAL, NOT EDITORIAL, and that is why it can be a directory
rather than a field in pkg.conf. "Do we actively modify it" drifts -- the day
somebody patches zlib nobody would agree which side it belongs on, and the
answer would be a rename touching CI and docs decided by taste. "Does it carry
files of ours" is a fact about the tree: it is exactly whether the recipe has
a payload/, prog/ or seed/, checkable in one assertion, and
test_recipe_layout.py now demands the move in both directions -- a 3rdparty
recipe that grows one of the three must come up, and a top-level recipe with
none of them must go down, or the top level slowly refills and the split stops
meaning anything.
ONE FUNCTION KNOWS THE TWO DEPTHS. pkgs/lib.sh's pkg_dir resolves a recipe id
to its directory and nothing else spells either path -- not a recipe, not
bin/build-packages.sh, not a test. Names stay unique across both levels
because pkg_out, pkg_stamp and the .ipk filename are all keyed by the bare
name; a duplicate would resolve to whichever level is searched first and
build the wrong thing silently, so that is a test too.
Recipes now source `. ./bin/common.sh` instead of counting `..` from $0.
Depth-counting would have needed two spellings for two levels, and it was
never buying anything: `. pkg/lib.sh` on the next line already required the
working directory to be the repo root.
A SECOND SILENT-SHRINK BUG, and this one was live. qa/static's _files()
expanded a list of globs and returned whatever matched, so "pkgs/*/build.sh"
quietly stopped covering thirty-four recipes the moment they moved -- the lane
stayed green, and the only visible sign was a test count dropping by 34 in a
run nobody was counting. _files() now refuses a pattern that matches nothing.
That is the same failure the ash-compatibility check had a guard for and the
syntax check did not, which is why the guard reads the way it does.
AND ONE PACKAGE WAS SHIPPING PYTHON BYTECODE. anvil-core stages a directory of
.py helpers wholesale, so any test that imported one left a __pycache__ beside
it -- gitignored, invisible in a diff, copied by cp -a like anything else. The
package's contents depended on whether pytest had been run on the machine that
built it: two builds of one commit, different sha256, cause nowhere in the
tree. What it shipped was worthless as well as unstable, compiled by the build
image's python 3.11 for a printer running the 3.13 this repo cross-builds. The
sweep is in pkg_ship now, once, and the two hand-written copies in
pkgs/klipper and pkgs/moonraker are deleted -- the trap is set for any recipe
that stages a tree of .py files, and the two recipes that had hit it had each
solved it locally while the third, which had not, was the one shipping.
Measured:
* the rename changes no package bytes. Same cached build trees, feed built
with the diff stashed and unstashed: all 41 .ipk match.
* with the bytecode sweep, anvil-core returns to exactly the bytes the
previous commit produced.
* no package in the feed contains a .pyc or a __pycache__.
* make qa-static: 247 passed (236 before this work + 11 layout tests).
* make test: 8 passed, 0 failed, including unpack/patch/pack/verify on the
synthetic stock package. make test-py: 181 passed.
Not explained: anvil-python-dev produced two different sha256 from an
UNCHANGED work/pkg/python at two points today, under code that was identical
both times -- so it is not reproducible from a cached tree, and it is not this
commit's doing (the stashed and unstashed builds above agree on it). Worth a
look; the dev split in bin/build-packages.sh is where to start.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|