mirror of
https://github.com/Klipper4FlashForge/firmware.git
synced 2026-10-21 21:02:16 -05:00
worktree-timelapse-test-gcode
30
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Oleksandr ShyshatskyiandClaude Opus 5
|
28eb20f3da |
Package numpy, because klippy does not start without it
extras/stepper_resonance_tester.py opens with a bare `import numpy as np`; klippy.py:122 walks EVERY config section through a load_object whose importlib.import_module is not wrapped in except ImportError; printer.base.cfg includes FlashForge's printer.vibration.cfg, which declares [stepper_resonance_tester]. Since klippy moved to our own CPython 3.13 that has not been a printer missing resonance testing. It has been a klippy that never reaches ready. So this is not a scientific extra. anvil-klipper depends on anvil-python-numpy the way it depends on cffi, except that the others mean "klippy imports this" and this one means "klippy starts". It also gives back SHAPER_CALIBRATE and TEST_RESONANCES, dead since the same switch, and the VFA calibration behind STEPPER_RESONANCE_FACTORY_CALIBRATE that docs/notes/44-vfa-calibration.md traces end to end. WHAT WAS NOT OBVIOUS, in the order it bit. numpy 2.x builds with meson-python, and meson has never heard of _PYTHON_SYSCONFIGDATA_NAME -- the trick every other python-* recipe here leans on. pkg_mesoncross writes a real cross file instead, naming the $PKG_XW wrappers rather than the raw toolchain: the wrappers carry -EL -mnan=2008, and a cross file that reaches past them builds a tree that passes every gate which only reads symbols and is refused at exec() on the printer. pkg_pywheel grew a PKG_PY_PIP_ARGS branch to pass the file as --config-settings, which is the one thing PEP 517 does standardise. The cross file's `python =` is the BUILD python, and that is not a mistake: meson introspects an interpreter by running it and asking sysconfig, and pkg_pytarget has already made that sysconfig answer for mipsel. The old trick does reach meson, just not by its own mechanism. numpy's meson.build hard-errors below gcc 8.4 and ours is 7.2. The gate is the SciPy Toolchain Roadmap's support statement, not a requirement of the code: this toolchain compiles if constexpr, structured bindings, std::optional, std::string_view, and pocketfft_hdronly.h -- 3635 lines of the heaviest template C++ in the tree, and the half np.fft.rfft needs. Probed before the floor was lowered, and lowered rather than removed, behind a guard that fails loudly if upstream moves the check. 19 extension modules, every one at e_flags 0x70001405/0x70001407, 14MB shipped after pruning tests/, f2py/, stubs and static libs. meson-python is self-hosting, so pkg_buildpython runs meson and ninja just to install the backend -- hence ninja-build in the image and PYPKG_HOST_LIST in dependency order rather than alphabetical. And pip puts meson and cython in work/.py-host/bin, which nothing had ever looked in: without that on PATH the backend fails to install with `meson executable "meson" not found` while pip list shows it right there. THE GATE THAT WOULD HAVE CAUGHT ALL OF IT is qa/replica/test_klippy_extras_import.py: walk the real /usr/data/config include graph, map every section to its extra, import each on the printer's own interpreter. It needs no MCU, because config load happens before klippy:mcu_identify. Nothing asked that question before, which is how two comments in this repo came to assert the module guarded its own import. test_numpy.py asks for numeric answers rather than a module object -- linalg.lstsq specifically, because -Dallow-noblas selects the bundled lapack_lite and _r2_sinusoidal_fit calls it every calibration round. And test_install.py's include check read only ff-*.cfg, so FlashForge's four -- printer.vibration.cfg among them -- were never asked about at all. ff-toolchange.cfg wraps the two search commands in _FF_SHAPER_PREP, as SHAPER_CALIBRATE already was. STEPPER_RESONANCE_FACTORY_CALIBRATE is deliberately NOT wrapped: it is a gcode_macro, not a registered command, and same-named sections merge under RawConfigParser(strict=False) -- a wrapper would not wrap it, it would BE it, with rename_existing pointing at itself. It needs none, since its body is two calls to a command that is wrapped. No SAVE_CONFIG in any macro either; it restarts klippy and stays the operator's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EJr2x7aDaY889qwSxbdLt8 |
||
|
Monstrofil
|
85615447c5 |
Merge pull request #6 from Klipper4FlashForge/pin-helixscreen-v0.99.115-creator5.3
Pin HelixScreen creator5.3 |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
6bb58a4091 |
The timelapse pin is main's head, and v0.0.1 is not a newer one
Two comments claimed upstream had never cut a tag. It has: v0.0.1, October 2021. The claim came from checking a --depth 1 clone, which fetches none. The pin does not move. v0.0.1 is two years OLDER than c7fff11e, which is still the head of main, so "update to the latest release" would drop hyperlapse, the configurable delayed_gcode interval and the webcam-database lookup getWebcamConfig depends on. Both comments now say which release exists and why the pin is a commit anyway, so the next reader does not re-derive it and reach the opposite conclusion. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TSmXJv47Wwq2J7FfgCgpCe |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
58cfd00415 |
FlashForge's ffmpeg cannot render a rotated webcam, so we build our own
moonraker-timelapse emits -vf 'transpose=1', 'hflip,vflip' and friends straight from the webcam's rotation and flip settings. The stock ffmpeg is configured --disable-everything with a narrow enable list, and asking the binary itself for -filters returns format, null, scale and trim -- so every one of those renders fails on a printer whose camera is not upright. So: ffmpeg 4.4.5 and x264, cross-built, static, one 1.8M binary at $MODDIR/bin/ffmpeg. The enable list is FlashForge's plus the filters they left out, since flash is scarce and nothing else is wanted. There is no hardware encoder to use instead, and it is written down in pkgs/3rdparty/ffmpeg/pkg.conf so nobody re-derives it: the X2600's device tree declares jpege (a JPEG encoder), jpegd and felix, and felix is an H.264 DECODER -- the die has no H.264 encode block at all. Only felix has a driver in the stock kernel; jpege and the rotate block have DT nodes and clock gates and nothing bound to them. Software x264 is the only route to an H.264 mp4 here, which is why moonraker.conf tunes it with -preset ultrafast rather than leaving the default. x264 is named anvil-x264-dev outright, as anvil-zlib-dev is: every file in it is a header or an archive, so a PKG_DEV_FILES split would leave an empty anvil-x264 beside it that no printer would ever install. bzip2 joins the build image because x264's only distributed form is a .tar.bz2 and tar could not unpack it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TSmXJv47Wwq2J7FfgCgpCe |
||
|
Oleksandr ShyshatskyiandClaude Sonnet 5
|
2f0d8b8a8c |
Creator5 HelixScreen moved to build 4 for the tool/material axis split
HELIX_VERSION -> v0.99.115-creator5.3, matching the new GitHub release (upstream VERSION.txt is still 0.99.115). SHA256 updated to match. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RFzh33YrxSnms5feP1xMR9 |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
d898d105e0 |
The Moonraker pin was frozen by a missing _sqlite3 we now build ourselves
MOONRAKER_VERSION was 9d0d09d, the last commit before 80c7620 moved the
database from lmdb to sqlite, because FlashForge's python 3.8.2 has no
_sqlite3 and every release after it died loading the database component.
This repo builds and ships its own CPython 3.13 now, and it has one. The pin
is v0.11.0 -- a release tag, not a sha, so PKG_VERSION is just 0.11.0 and no
longer the Debian-style 0.8.0+git<sha> snapshot string. GitHub strips the
leading "v" from the directory inside /archive/<tag>.tar.gz, which is what
${MOONRAKER_VERSION#v} in the recipe is for.
The dependency set is the part that actually moved. Read against 0.11.0's
CORE_COMPONENTS plus the sections moonraker.conf enables:
dbus-next became dbus-fast. dbus_manager is a core component and imports
it at module scope, so this is not optional whatever [machine] provider
says. Its sdist cythonises nine modules; SKIP_CYTHON=1 ships the python
they were generated from and keeps the package ARCH all. The wheel still
comes out platform-tagged -- poetry-core marks the distribution impure
for declaring a build script at all -- but it holds 29 .py and no .so,
and the content is what ships.
importlib-metadata is new, imported at module scope by
utils/source_info.py, and brings zipp. Both build through setuptools_scm,
so PYPKG_HOST_LIST grew packaging and setuptools-scm. That list is now
install order: pkg_buildpython pip-installs each with --no-deps against
no index, flit-core builds packaging, setuptools_scm imports it.
pyserial-asyncio is gone. Nothing in 0.11.0 imports serial_asyncio;
utils/async_serial.py drives pyserial over asyncio itself. pyserial stays
-- klippy needs it and declares it.
LMDB STAYS, for a reason that is not the old one. 0.11.0 converts an
existing data.mdb to sqlite on first start and imports lmdb to read it. With
the module missing it tries to pip install one, and a printer has no index
to install from -- the old database would be dropped in silence. It is the
upgrade path now, not the database, and the comment on WANTED_MODULES says
so.
inotify-simple STAYS AT 1.3.5 though 0.11.0 pins ==2.0.1: 2.0.1 declares its
licence the PEP 639 way and needs setuptools>=77 to build, against the 75.6.0
pinned here for lmdb's sake. It fails at metadata generation, not at import,
which is the kind of thing that reads as a broken recipe later. 1.3.5 carries
every name 0.11.0 asks of it, and moonraker.conf sets file_system_observer:
none, so the import is all that is asked. jinja2 goes to 3.1.6 to match.
Not taken: paho-mqtt, zeroconf, apprise, ldap3 and python-periphery, whose
components this printer does not enable, and msgspec and uvloop, which
Moonraker imports under contextlib.suppress(ImportError).
Verified: every changed sha256 downloads and matches at the URL
fetch-assets.sh builds; all five affected wheels build offline
(--no-index --no-build-isolation) on a real 3.13 with the host backends
installed in the declared order; qa/static 122 passed, including the gates
that cross-check PYPKG_LIST against the recipes and every PKG_DEPENDS
against the feed. The mipsel cross-build and `make qa-replica` have not run
yet, and versions.env still says a bump is gated on the latter.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012oRdgKM7scrEsr6waAByaY
|
||
|
Oleksandr Shyshatskyi
|
1b89782ac9 | Merge remote-tracking branch 'origin/master' | ||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
bf7d0f7e06 |
The printer has no clock and nothing was setting it
There is no RTC and no battery, so every boot starts at whatever the kernel was built with and stays there: job timestamps, log ordering and certificate validity are all wrong on a machine that has been unplugged. Nothing on the stock rootfs could fix it -- FlashForge's busybox is v1.31.1 built without the ntpd applet, and its rdate speaks RFC 868. anvil-sntpd is troglobit's sntpd, a fork of Doolittle's ntpclient: one small binary, servers and interval on the command line, so there is no config file, no privsep user and no chroot to place on a read-only rootfs. It ships $MODDIR/sbin/sntpd and nothing else. A LONGRUN, NOT A HOOK ON THE WIFI ONESHOT. That oneshot runs once, at boot, and returns before wlan0 has associated -- so a sync hung off its end would fire too early, never retry, and never fire at all for the owner who joins a network from the HelixScreen panel an hour later. And with no RTC a clock is not set once but maintained. sntpd keeps its schedule with the network down instead of exiting, which is what lets this service need no wifi dependency -- just as well, since nothing may depend on that oneshot. The poll interval is 300 rather than upstream's 600 because it is also the ceiling on the down-network retry backoff, and so bounds how long a printer that joined wifi late keeps the wrong time. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0149y4iyLpEwYKh2PS4fmf4u |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
c3a64e0394 |
The stock firmware is a pinned download like everything else
config.env.example opened with DESKTOP="$HOME/Desktop", and four settings hung off it. Nothing in bin/, pkgs/, qa/ or installer/ ever read it: three of the four were commented-out overrides for files `make vendor` already fetches, and the fourth, BUSYBOX_BIN, names a binary with no source at all. What DESKTOP really did was carry the stock FlashForge .tgz -- the one input a person had to go and find before they could build anything. It did not have to be. .github/workflows/ci.yml has been fetching that exact file for as long as printer-sim has run, from ghzserg/FF, pinned by sha256, with its own get(), its own cache and its own secrets -- a second spelling of "download the firmware and check the hash" that no human path shared. The pins here are those pins, verified byte-for-byte against a fresh download of both assets. So versions.env pins the stock package the way it pins everything else, bin/fetch-assets.sh downloads it into vendor/, and bin/common.sh defaults STOCK_TGZ_* there. A cold checkout is now `cp config.env.example config.env && make build`, with nothing to edit. --stock is a separate flag from --all, not folded into it. The packaging lane wants no FlashForge package -- that is most of the point of it being runnable in CI on a bare checkout -- so `--all` stays "every payload piece, no firmware", `make build` asks for the one model it is building, and `make vendor` (--all --stock) takes both. The one download that will not overwrite an override. HELIX_TGZ and friends name a public release anyone can re-fetch in a minute; STOCK_TGZ_* names 93MB of proprietary firmware someone downloaded by hand, and clobbering it on a hash mismatch would destroy the only copy on the machine. It is skipped instead, and unchecked -- their file, their problem. CI loses its duplicate fetcher, its stock/ cache and the STOCK_URL_* secrets. A fork testing its own firmware re-pins versions.env, which works locally too; the secrets only ever worked in CI. qa/lib/config.py reads versions.env for the pinned filenames so stock_for() looks where the download actually lands. An explicit STOCK_TGZ_* still wins. An existing config.env keeps working untouched: its paths are now overrides, so the build skips the download and uses the file it always used. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DPU6RR4UJH2Qf9DR5hb2gg |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
b98552386b |
No FlashForge component ships; patch.sh is payload.sh
The software component existed to give the mod one exec hook. It is not needed for that any more, so a release now carries our installer and our payload and nothing else: 26MB smaller, 21MB of which was FlashForge's own firmwareExe travelling back to the partition it was already on, along with their klippy tree, their wifi module and their configs. WHAT IT GIVES BACK. /usr/prog/PROGRAM/software is no longer rewritten, so the version directory there stays FlashForge's with their genuine firmwareExe inside it -- and app_startup.sh's five-second watchdog, which copies one out of that directory when nothing by that name is running, has something to copy again. A printer whose payload fails to install now comes up on the stock UI instead of a blank screen. That recovery has been broken since we started deleting their binary out of a component we shipped; docs/how-it-works.md claimed it worked throughout. THE ROOT PASSWORD moves with it, because it used to ride in the component's shadow file. bin/pack.sh bakes ROOT_PW_HASH into the installer, single-quoted -- an unquoted $6$... would have the printer's shell expand $6 as a positional parameter and ship a truncated hash -- and the installer writes it to /usr/prog/etc/shadow. The random-password path asks one question instead of four: FlashForge ship the same root hash on every machine, so a live shadow still carrying it means nobody has set a password and it is ours to set. The .prev-root-hash recording, the shipped-vs-live comparison and the restore-it- afterwards step were all machinery for surviving a file that nothing overwrites any more, and are gone. pack.sh checks the hardcoded constant against the stock package it built from, because getting it wrong is silent and total: every printer would read as "already set" and none would ever get a password. bin/patch.sh is bin/payload.sh. It patched nothing any more -- no splice, no shadow edit, no firmwareExe deletion -- and what is left is the payload: the feed installed by the printer's own opkg, the seeded config, the manifest. Its sections are renumbered 1-5 with no gaps, and the five tombstone comments marking where removed sections used to be are gone with them. test_recipe_layout.py loses six of its nine tests. Each compared the tree against a description of the tree -- no fourth directory kind, each kind still used, 3rdparty carries no files of ours, installer/ holds exactly two named files -- and none could go red for a reason a printer would notice, while all six went red the moment the layout legitimately changed. The two that bite are kept: a payload/ path escaping $MODDIR, and a recipe name at both levels, which makes pkg_dir silently build the wrong package. SW_VER is deleted from config.env.example and both workflows: it stamped a component that is not built. Gates: 112 static tests pass. The generated installer is verified after substitution -- all four lines present, and `sh -n` over the result, because a hash with a stray character in it would otherwise reach a printer as a syntax error whose only symptom is a package that installs nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CH7Yb4wuPhnppHAB25UX7e |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
7768bb8fe6 |
versions.env kept 412 lines of comment for 90 lines of pin
The file had grown into an essay: the history of a musl toolchain that is no longer pinned here, three paragraphs on why execline stopped being optional, the full account of every failure that produced a pin. All of it true, none of it needed at the moment somebody edits a version string. What survives is what changes what you do to a bump -- the traps. OpenSSL must stay on 3.0.x, xz must stay off 5.6.x, greenlet needs fill-designators.py rerun, pillow cannot be 10.3.0, streaming-form-data cannot be 1.13.0, opkg and s6 bake their prefix in at compile time, Moonraker is pinned to the commit before sqlite. The narrative around each one is gone; the instruction is not. Every pin is byte-identical: the file's non-comment lines diff clean against the old one, it still sources, and PYPKG_LIST still matches the 18 recipes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UgtmbEg8rUsQHka8mMiCxW |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
4850be432c |
Docs: say what is there now, not what the last three commits removed
Deleting test/ and anvil.conf left claims behind in nine files. Every one of
these was present tense about something that no longer exists.
THE CHELPER SYMBOL GATE, in three places that each described it as live:
notes/20-klipper-fork.md said pkgs/klipper gates the .so on "both the ELF
flags and test-chelper.py's symbol check". It
gates on neither now: the ABI is asked once of the
installed filesystem in qa/replica/test_abi.py,
and the single-source rule -- the .so is compiled
from the chelper sources of the tree it ships -- is
the whole defence. The page says that instead.
notes/85-packaging.md described the KLIPPER_FORK fallback the script no
longer had; the script itself is gone too.
qa-migration.md "What survives in test/" listed both the 28 unit
tests and test-chelper.py as surviving. It is
"What survived in test/, and then did not", and it
names what went uncovered rather than leaving that
to be discovered.
ANVIL.CONF, in the two notes that still read it at runtime:
notes/80-s6-migration.md is a dated plan, so it keeps its phases and gains
a "Where this ended up" note: anvil.conf and every
MOD_* switch were removed, so the phases that
source it describe an intermediate state, and the
case-*.sh gates they name are qa/replica/ modules.
notes/85-packaging.md predicted seed/ emptying via a postinst that seeds
anvil.conf.default. It emptied by anvil.conf being
deleted instead. Its payload allowlist said six
entries including anvil.conf and bin/busybox; the
real one in test_ipk.py has five and neither.
THE TWO STANDALONE TREES under tools/, stale before any of this and now fixed:
tools/python/README.md and tools/supervisor/README.md both told the reader to
run their gate with `test/integration/printer-exec.py` over a case-*.sh, and
one offered a `make test-python` target. None of the three has existed for
several commits. Both now point at qa/replica/, which installs the real package
and asks more than hand-feeding a tarball to a case script could. The four
places tools/python/README.md narrates what case-python.sh proves are past
tense, since that is now a record of how the measurements were taken.
versions.env's sqlite3 note pointed at the same dead target.
NAV: every notes/ page was listed except 85-packaging.md, and qa-migration.md
was reachable only by following a link out of Testing, Building or How it
works. Both are in the nav now, which also clears the two "not included in the
nav" lines from every build. .venv/ is gitignored -- docs/requirements.txt
tells you to make one right there in the tree.
Verified: `mkdocs build --strict` is clean and no longer reports pages outside
the nav. qa/static is 123 passed, its 110 errors the same missing
vendor/opkg-utils as before. No .md file is read by any lane, so the check that
matters here is the build.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016bLGYpVAeVzAX6NeAxcU42
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
214a4e58b9 |
anvil-core owns firmwareExe and start.sh; kill the 3.8.2 myth
Three things, all downstream of klippy having moved to $MODDIR and our own
CPython 3.13.
SECTIONS 8 AND 9 ARE GONE. Both files install at $MODDIR/prog/ and
anvil-link-prog.sh points the stock paths at them -- from run-append.sh on a
flash and from anvil-core's postinst on `opkg upgrade anvil-core`. Nothing
execs either between run.sh distributing the component and link-prog running,
so the component copy bought no ordering. It cost, though: our firmwareExe
with no payload behind it logs "no anvil-env.sh", finds no s6-svscan and then
`while true; do sleep 3600; done` -- a printer that boots to a blank screen for
ever and never reaches app_startup.sh's own recovery. Absent, that recovery
restores a firmwareExe from a version directory. So patch.sh now REMOVES both
from the component: the stock component carries them, and deleting the two cp
lines alone would have shipped FlashForge's originals.
SECTION 5c IS GONE. It did no work -- two du prints and a comment whose first
line was the falsehood below.
THE FALSEHOOD. "klippy runs on the stock 3.8.2, started by
/usr/prog/klipper/start.sh" had spread well past patch.sh, and it had reached
things people follow:
- docs/toolchange.md told owners to scp the ff_*.py extras to
/usr/prog/klipper/klippy/extras/ -- the wrong partition -- and warned that
an OTA would delete them there. Same wrong path in three ff-*.cfg headers
and in ff_toolchange.py.
- versions.env labelled our CPython 3.13 and its eighteen packages
"SHIPPED BUT NOT USED".
- docs/how-it-works.md said Moonraker runs on python3.8, contradicting line
126 of the same file.
All corrected. Historical notes keep their history and get a [Superseded: ...]
marker instead -- including 80-s6-migration.md's "s6-rc-compile is not
shipped", which pkgs/3rdparty/s6-rc/build.sh contradicts deliberately.
Found while reviewing: every docs/ reference to the ff-*.cfg pointed at
pkgs/anvil-core/payload/config/, which does not exist -- they are
anvil-klipper-config's. 28 references across 9 files, several of them broken
markdown links on the published site. All 24 relative links under docs/ now
resolve.
Gates: verify.sh syntax-checks prog/start.sh and prog/firmwareExe out of the
payload (firmwareExe was never syntax-checked before) and fails if the
component carries either. A new replica test asserts both stock paths are
symlinks into $MODDIR/prog and do not dangle. One existing replica test stubbed
firmwareExe by redirecting into $FE, which now writes THROUGH the symlink into
the packaged wrapper; it moves the link aside instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01La5MaDKqTW675k5fsXsP6S
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
09a9f868e3 |
klippy compiles c_helper.so at first run, and this printer has no compiler
So the "build system" for that .so genuinely is one gcc line -- the argument
list in klippy/chelper/__init__.py -- and pkg_build could not express it,
because it always ran make. That is why Klipper was the last cross-build
still living in bin/patch.sh, which unpacked the Ingenic toolchain into
work/.mips-toolchain, wrote its own gcc invocation, cached on an mtime
comparison and gated the result itself: every one of the things pkg/lib.sh
exists to do once.
PKG_CC_SHARED is that line, appended to `$CC -shared -fPIC`. A knob and not
a verb, for the reason pkg_build's own header gives about the three
mechanisms with one user each: the recipe says WHAT to link, pkg_build owns
HOW, and -EL -mnan=2008 stay spelled once in pkg_toolchain where the wrapper
is gated before anything is built on it.
pkg/klipper -> anvil-klipper_0.13.0+gitb722a4c8-1_mipsel_xburst2.ipk
Both gates moved into the recipe, next to the ship list they check -- the
same move the s6 presence checks made. They now run on `make packages` too,
so a c_helper.so that is not nan2008, or one missing a symbol the klippy
beside it cdefs, fails the build that produced the .ipk rather than only a
full firmware build. The symbol gate is the one that matters: cffi resolves
lazily, so a stale .so imports cleanly and dies at connect on the printer.
KLIPPER_FORK is gone. It pointed the build at a local checkout instead of
the pin, which is a second source for a recipe allowed exactly one -- and it
is the same variable that let v20260824-nova-kakhovka ship a klippy tree
nobody had built, set to "" in CI. To build your own fork, put its tarball
in vendor/ and re-pin KLIPPER_VERSION and KLIPPER_SHA256; the sha is then
the guarantee every other source in this repo carries. bin/fetch-assets.sh
loses two of its conditions with it, and test/test-chelper.py loses the
config.env fallback that printed "SKIP: no KLIPPER_FORK configured" -- a
green line for a check that had not run.
bin/patch.sh section 1 is now staging and nothing else: a cp -a into the
SOFTWARE component, where the stock klipperDaemon looks, plus the
chelper.tar its run.sh extracts. That tar stays out of the .ipk, where it
would be a second copy of a file the package already contains. The package
installs the tree under $MODDIR/klipper, inert until phase 7 of
docs/notes/80-s6-migration.md makes the installed copy the live one -- at
which point these lines are deleted rather than moved.
Measured:
* two cold builds of pkg/klipper byte-identical, work/pkg deleted between
* c_helper.so is 0x70001407 -- nan2008, o32, mips32r2
* all 66 functions klippy declares are exported, checked after pkg_ship
strips rather than before
* the whole feed still builds: 38 recipes, 41 packages
* 236 static tests, including the staged-recipe list that now names klipper
* BUILD_KLIPPER=stock removes the recipe from pkg_recipes entirely
Not verified here: bin/patch.sh end to end. It needs the stock FlashForge
package, which the packaging lane deliberately does without; bin/verify.sh's
klippy-tree and chelper.tar assertions are what cover the staging rewrite.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
8cc92e8266 |
A wheel could only be built while three scratch trees happened to coexist
That is why bin/patch.sh section 5c was 825 lines and why a Pillow bump
rebuilt CPython. Cross-building an extension module needs the untrimmed
staging tree (INCLUDEPY points into it), the private sysroot of static C
libraries (pillow links zlib, cffi links libffi) and an x86-64 build-python
of the same version -- and all three existed only mid-section, so the wheels
had to be built there too, on a cache with two stamps that invalidated each
other.
Each of those three is now a durable thing: anvil-python-dev carries the
headers, pkg_deps fills each recipe's own sysroot from the feed, and
pkg_buildpython caches one build-python for all nineteen recipes. So the
coupling goes away with them -- `make packages PKG=python-pillow` builds
Pillow.
pkg/python -> anvil-python + anvil-python-dev
pkg/python-<name> x18 -> anvil-python-distro, anvil-python-pillow, ...
Three verbs in pkg/lib.sh, shared by all nineteen rather than one per awkward
project: pkg_buildpython provides the build-python, pkg_pytarget points its
sysconfig at the target headers in the sysroot, pkg_pywheel builds one wheel
and unpacks it into the staging site-packages. greenlet's gcc 7.2 designated
initializers, lmdb's LMDB_FORCE_CPYTHON and pillow's build_ext options are
recipe lines and a PKG_PY_SETUP_ARGS knob, not three more verbs.
Eighteen versions that were invisible are now in the feed index, anvil-moonraker
declares the thirteen packages it imports, and a printer that does not want
gcode thumbnails can leave Pillow uninstalled.
PY_STAMP, PYPKG_STAMP and PY_BUILD are gone: pkg_stamp derives the key from
PKG_BUILD_DEPENDS, and bin/fetch-assets.sh asks pkg_needs instead of
re-deriving a stamp bin/patch.sh also derived. Two of its four toolchain
conditions were that second spelling.
Measured:
* 40 packages; 959 site-packages files, none owned by two packages
* 12 mipsel extension modules -- the count the old gate demanded
* every Depends resolves inside the feed (24 declare one, 0 unsatisfiable)
* two cold builds byte-identical, with work/pkg, work/.py-host and the
unpacked toolchain deleted in between
* 233 static tests, 4 new: the PYPKG_LIST/recipe agreement both ways, no
version pinned twice, patch.sh builds every package, and no Depends
naming a package no recipe produces
Not verified here: the replica suite. It needs the stock FlashForge package,
which the packaging lane deliberately does without.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
6e97f946b0 |
execline was 2.1MB unstripped against musl; it is 1008KB and mandatory
Eight recipes: skalibs, execline, s6, s6-rc, mainsail, moonraker,
helixscreen, anvil-core. bin/patch.sh loses 183 lines and stops being a
build system for any of them.
The supervision stack moves to the Ingenic glibc toolchain and links libc
dynamically, so the musl toolchain -- the only second libc in this tree --
is deleted along with MUSL_TOOLCHAIN_*, S6_STAMP and S6_BUILD. The
measurement that chose musl compared two STATIC builds; dynamic glibc was
never in it. Measured now: s6's shipped tree is 696K against ~930K for the
static musl one, and the whole stack including execline and s6-rc is 2.0M.
Three things that were not guesses:
* s6 and s6-rc do not link against glibc 2.29 without -lpthread. skalibs'
pthread_mutex_tailock reaches pthread_mutex_timedlock, which glibc kept
in libpthread until 2.34. Upstream never sees it and musl never had the
split. PKG_MAKE_ARGS is the lever, because s6's PTHREAD_LIB is wired
only to --enable-nsss and its Makefile appends LDLIBS last.
* s6 ships 21 binaries, not 13. s6-rc's generated scripts exec s6-sudo,
s6-sudoc, the s6-ipcserver chain, s6-sudod and s6-fdholder-daemon --
found by running a real up/down cycle with a restricted PATH, not by
reading the documentation.
* --enable-static-libc has never been a skalibs option. Its configure
warns on unknown flags and patch.sh sent that to /dev/null, so it was a
no-op for as long as section 5b existed.
execline ships whole. A subset picked from today's services fails the day
somebody adds a pipeline, at boot, naming a program nobody chose to leave
out. tools/supervisor/README.md and 80-s6-migration.md said it was not
needed; both now say why that stopped being true.
PY_STAMP was spelled in three places and happened to agree -- the same
shape as the s6 stamp bug, caught before it inverted. It is in common.sh
now, and the test that guarded S6_STAMP by name is rewritten around the
rule so it covers both and dies with neither.
HelixScreen's binaries reach mips_abi_gate for the first time: patch.sh
never gated them. They pass, 3 objects, nan2008/o32/mips32r2.
anvil-core carries a content hash in PKG_STAMP_EXTRA because its version is
a release date: pinned, an edit to payload/ would read as current; unpinned,
it rebuilds daily. The version says which release, the stamp says what is
in it. anvil.conf stays out -- it is user state that run-append.sh
preserves, and a package member is overwritten on every upgrade.
Twelve packages, byte-identical across two cold builds. 152 static tests.
The replica suite has NOT been run: docker is unavailable here, and the s6
change ships without its behavioural gate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
5025aac171 |
opkg-utils already builds ipks, and opkg is the second package that proves lib.sh
Two corrections to the packaging PoC, both from review.
FIRST: THERE WAS ALREADY A TOOL. bin/mkipk.sh is deleted. opkg-utils ships
opkg-build and opkg-make-index -- 423 lines of POSIX sh and a python indexer,
upstream, maintained -- and this repo now drives them instead of re-deriving
the format. What is left of build-packages.sh is the part that is genuinely
ours: laying out the tree opkg-build packages. The hand-rolled version worked,
and it was still us reimplementing member ORDER, control validation and the tar
flags that make a build reproducible, all of which fail as "a package that
inspects fine and installs nowhere".
opkg-utils is PINNED BY COMMIT, uniquely in this repo, and not by choice: it
publishes no release tarball anywhere. Upstream's cgit has snapshots disabled,
downloads.yoctoproject.org carries opkg but not opkg-utils, and the GitHub
mirror last saw a commit in 2012 with no tags. A git sha is a hash of the whole
tree -- the same guarantee every sha256 here gives -- and fetch-assets.sh
verifies HEAD against it and refuses a dirty checkout. Both were tested.
SECOND: A SECOND PACKAGE, AND IT IS opkg ITSELF. pkg/lib.sh now holds the part
of a cross-build that is the same every time -- the two toolchains, the gcc
wrappers that bake -EL -mnan=2008 into the DRIVER because autotools link lines
do not all forward CFLAGS, the ABI self-test of those wrappers, autotools,
the build cache, and pkg_ship. Both recipes fit on a screen and share exactly
that file.
opkg is the honest test of it: different toolchain (Bootlin musl, not Ingenic
glibc), different libc, different link mode (static, not shared), and a
dependency chain three builds deep (zlib -> libarchive -> opkg) against
libsodium's none. It is static musl for the same reason s6 is -- a standalone
binary that must match nothing -- so no libarchive.so or libz.so ships beside
it. qa/static/test_ipk.py fails any recipe that sources its own toolchain,
spells its own -mnan=2008, or calls ./configure directly; the gate was checked
by breaking a recipe on purpose. zlib is the one named exception, because its
configure has never accepted --host.
THE LOOP IS CLOSED ON THE TARGET. The opkg this repo builds -- 775KB, static,
nan2008/o32/mips32r2 -- runs under qemu-mipsel and installs both our packages,
including itself, then lists their files and removes them leaving nothing
behind. The opkg it installed then runs and reports its version. Its status
database landed at /usr/data/anvil/var/lib/opkg, which is where pkg/ipk-install
writes, because both agree rather than by coincidence.
Three findings paid for by building it, all in the note:
* opkg BAKES ITS STATE DIRECTORY IN AT COMPILE TIME -- --offline-root does
not move it. Built with any other prefix it believes nothing is installed
and reinstalls the world. Third instance of this trap here after s6's
libexecdir and execline's shebangdir.
* opkg 0.6.3 does not build against musl: two files call basename() without
<libgen.h> and get away with it only under glibc. 0.7.0 fixed both, which
is the difference between a pin bump and carrying a patch.
* libtool SWALLOWS -static. A program configured LDFLAGS=-static comes out
dynamic and the only symptom is a NEEDED entry nobody reads. -all-static,
at make time, is the flag that means it -- hence the explicit check.
Both packages are byte-identical across a full cold rebuild. The musl toolchain
fetch now also triggers on a stale work/.opkg, and its cache key lives in
common.sh so the fetcher and the recipe cannot spell it differently -- the
hazard already written above pypkg_stamp.
Also fixed: two `# shellcheck disable=... -- prose` directives were malformed,
which made shellcheck abandon those files rather than check them. A gate that
examined nothing, in the file arguing against those.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
99706bb5f7 |
s6 was legacy-NaN. The kernel this printer runs refuses that at exec().
The Bootlin swap two commits ago (musl.cc -> mips32el) fixed a
reachability problem and kept a correctness one: mips32el is plain
mips32r1, whose musl crt/libc is legacy-NaN by construction and cannot
be made to emit nan2008 output at all -- forcing -mnan=2008 on it fails
to LINK ("mixing -mnan=2008 module with previous -mnan=legacy modules").
So every s6 binary this repo has shipped read e_flags=0x1007, defended
in five separate places as correct on the theory that a supervisor
doing no floating point cannot care about NaN encoding.
True of the arithmetic. False of exec(): docs/how-it-works.md already
documented, for a different binary, that "the Ingenic X2000 kernel
returns ENOEXEC for legacy-NaN MIPS" -- the ABI check happens before a
single instruction of the program's own runs, which means it was never
about what s6 computes. None of the replica gates that "measured s6
working on the printer" ever caught this, because none of them ran on
the printer: qemu-mipsel-static's user-mode emulation does not enforce
the same ABI check a real kernel's binfmt loader does. A build that
passed case-supervisor.sh, case-services.sh, case-moonraker313-s6.sh and
a full release run was still shipping a supervisor that a real X2000
kernel may refuse to start at all.
THE FIX. Bootlin also publishes mips32r5el: MIPS Release 5 has no
legacy-NaN silicon to be compatible with, so its musl crt/libc is
nan2008 by construction, and its default codegen already targets
mips32r2. Verified outside the repo before touching anything: fetched
the tarball, cross-compiled skalibs+s6 with it, every binary read
nan2008/o32/mips32r2, and s6-svscan ran and supervised under
qemu-mipsel-static. Section 5b now applies the same gcc-wrapper
discipline 5c already uses for the Ingenic toolchain -- -EL -mnan=2008
-march=mips32r2 baked into the driver, not left to CFLAGS, with a probe
gate before building anything on top -- because "the toolchain's default
happens to be right" is exactly the kind of assumption this bug was.
mips_abi_gate moved earlier in the file so 5b can use it, and s6 is IN
that gate now rather than exempt from it -- exempt is how this shipped
wrong for as long as it did. The gate itself grew a real relaxation
alongside the real tightening: a static EXEC from this musl toolchain
legitimately carries EF_MIPS_PIC (musl's crt is unconditionally -fPIC;
no combination of -static/-no-pie/-fno-PIC removes it, measured) and so
reads the DYN flag value anyway, which is a property of the toolchain
and not a sign of anything wrong -- so the gate now accepts either
0x70001405 or 0x70001407 for any object, rather than picking one from
the ELF Type.
Also bumped S6_STAMP to include MUSL_TOOLCHAIN_FILE: without that, a
checkout with an already-built work/.s6 from before this commit would
read its own stale legacy-NaN tree as current and never rebuild it.
Full real build: "s6: 14 ELF objects are nan2008/o32/mips32r2 -- good".
case-supervisor.sh green on the replica with the new binaries.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
4662cd0964 |
HelixScreen bumped to build 3 (v0.99.115-creator5.2)
Published today, after the previous pin. Fetched the real asset and checked it against the release's own SHA256SUMS before touching the pin -- both match what is written here. Full replica suite green with it: 23 passed, 0 failed, 0 skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
6ee6ba63ab |
s6's toolchain moves from musl.cc to Bootlin, which is still up
musl.cc flaked in CI while this session was watching -- the exact same
"Failed to connect... 134000ms" twice, on the exact same file, and once
more live while a human was reading the log. It is a single
volunteer-run host with no CDN behind it. Bootlin's toolchains project
(toolchains.bootlin.com) is a funded, ongoing effort that publishes
prebuilt Buildroot toolchains for a wide target range, including
mips32el+musl.
Verified before touching anything downstream of the pin: fetched
mips32el--musl--stable-2025.08-1, checked its sha256, and cross-compiled
skalibs+s6 with it using the exact recipe bin/patch.sh runs -- all 13
s6 binaries plus s6-ftrigrd came out ELF32 LE MIPS (e_flags carries the
same base 0x1007 as the musl.cc build, plus PIC/CPIC bits from musl's
own -fPIC libc.a, which cost nothing on a static binary that never
dlopens anything), and s6-svscan actually ran under qemu-mipsel-static,
supervising an empty scandir the way it does in case-supervisor.sh. The
real repo fetch-assets.sh path was exercised too, against the real
Bootlin URL with sha256 verification.
Three real differences the swap could not paper over:
* The toolchain prefix is mipsel-buildroot-linux-musl-, not
mipsel-linux-musl- -- S6_HOST changes with it.
* Bootlin ships a .tar.xz, not musl.cc's .tar.gz -- `tar -xf` picks the
decompressor off the file now instead of assuming gzip.
* Bootlin's archive extracts into a directory named after the release
("mips32el--musl--stable-2025.08-1"), not the fixed "$S6_HOST-cross"
musl.cc used -- bin/patch.sh renames it after extraction so nothing
downstream has to know the version string.
The full replica suite was 23/23 immediately before this change (musl.cc
still pinned); it has not finished a clean run since, because the run in
flight was interrupted to bring in svc-nice-20260825 and the other
worktree branches master picked up. That confirmation happens together
with theirs, not separately from them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
a8a46e21d2 |
The build makes the packages Moonraker runs on, from hashed sources
Section 5c grows a second half and section 5d appears. The 18 packages and libsodium that two spikes proved are now built by bin/patch.sh from pins in versions.env, and land in the payload at lib/python3.13/site-packages and lib/libsodium.so. WHERE THE PACKAGE BUILD HAD TO GO, and it is not obvious: inside 5c, against the UNTRIMMED interpreter stage, before the trim runs. The headers INCLUDEPY names, the static dependency libraries pillow and cffi link against, and the x86-64 build-python all exist only at that moment -- the trim deletes include/ and config-3.13-* and then removes the staging tree entirely. The alternative was caching a second untrimmed ~200MB tree. The price of not doing that is that either stamp going stale rebuilds both halves, which is written down at the stamp rather than left to be discovered. The build no longer touches PyPI. pip comes from --with-ensurepip=install on the build-python, out of the same pinned CPython tarball, and every pip wheel runs --no-index --no-build-isolation --no-binary :all: against hashed sdists in vendor/. That last flag is not caution: during the spike pip silently downloaded x86-64 manylinux wheels for cffi, greenlet and tornado, three wrong-architecture objects sailed through the build, and only the ABI gate caught them. Which is why the gate now reaches the STAGED PAYLOAD rather than the build cache. It walked $PY_BUILD/bin and $PY_BUILD/lib, so libsodium and the twelve extension modules staged straight into $MOD_PAYLOAD would have gone unexamined. 69 ELF objects now, plus libsodium's resolved .so. Deliberately not all of $MOD_PAYLOAD: s6 is musl-built and correctly reads e_flags=0x1007, mips1 and legacy-NaN, which is right for a static binary that does no floating point and is measured working on the printer. It has its own gate in 5b. That distinction is recorded in patch.sh, because a sweep that assumes every object must be nan2008 flags all thirteen s6 binaries and looks like a disaster. Two pins carry reasons that cost someone an afternoon: pillow is 11.0.0 because 10.3.0 cannot be built by ANY 3.13 -- its setup.py execs and then reads locals()["__version__"], which PEP 667 broke -- and streaming-form-data is 1.19.1 because Moonraker's own pin ships pregenerated Cython calling the four-argument _PyLong_AsByteArray, which grew a fifth parameter in 3.13. docker/Dockerfile.build needs zlib1g-dev and patch, and the image must be rebuilt. The first is a genuinely nasty one: a build-python without zlib cannot unpack a wheel, or install pip, and ensurepip fails INSIDE a `make install` that reports success. patch.sh now asserts zlib and pip on the build-python rather than failing later and further away. The install manifest goes from 1872 paths to 2940 and case-upgrade.sh still passes, which is the check that the deletion loop scales. Note that the "70 to 728" in the commit that landed the interpreter was wrong -- 728 predates the interpreter itself; HEAD's own patch.sh produces 1872 here, measured. anvil.tar.xz grows 1.6MB packed, about 15MB on the printer. FF_PYTHON still points at FlashForge's 3.8.2 and /usr/prog/libsodium/lib is still in ANVIL_LIBS. Both comments now say what changed -- the packages exist -- and what has not: nothing has run through S62moonraker under s6 on 3.13. The two lines move together, in the commit that has that gate behind it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
813973620c |
The payload carries a CPython 3.13 with a working sqlite3
Nothing runs on it. FF_PYTHON still points at FlashForge's 3.8.2, and the
comment beside it now says at length why changing that line is a one-line way
to break the whole userspace: klippy, Moonraker and ff-startup.py import
third-party C extensions -- tornado, lmdb, cffi, greenlet, pillow, libnacl --
that exist on this printer only as mipsel objects built against 3.8. None has
been cross-built for 3.13 yet. The interpreter ships ahead of the switch for
the same reason s6 did: it puts the artefact on real printers where it can be
measured, and it leaves the switch a one-file change rather than a build
change and a boot change at once.
Why it exists at all: the Moonraker pin is frozen at commit 9d0d09d, December
2023, for exactly one reason -- this firmware's Python has no _sqlite3 and
there is no libsqlite3 anywhere on the box, while Moonraker v0.9.0 moved its
database off lmdb onto sqlite. That is a cost paid at every release. The gate
proves the premise rather than assuming it: the stock 3.8.2, given the library
path it needs to start, imports json and zlib fine and cannot import sqlite3.
Built by bin/patch.sh from eight pinned sources with the Ingenic toolchain
that already builds c_helper.so -- gcc 7.2, glibc 2.29, cross to
mipsel-linux-gnu. Not musl, and this is not a preference: a musl interpreter
cannot dlopen a glibc c_helper.so, which is exactly how klippy loads it.
Three things about the build are load-bearing and all three were found the
hard way:
* -EL -mnan=2008 must reach the LINK lines, not only the compile lines.
They are baked into PATH wrapper scripts around the gcc driver rather than
passed as CFLAGS, because trusting eight separate build systems to forward
flags to their linkers is how you get legacy-NaN objects that the printer
refuses to load.
* _sqlite3 disappears silently without LIBS="-latomic -lm". A static
libsqlite3.a carries no DT_NEEDED on libm, so configure's
sqlite3_bind_double probe fails to link and CPython records the module as
"missing" -- a probe failure, so nothing in the build output says why. The
build now hard-fails when _sqlite3 is absent rather than shipping an
interpreter whose whole purpose is gone.
* OpenSSL's linux-mips32 target hardcodes -mips2, which collides with this
toolchain's -mfp64. -mips32r2 as a user cflag lands after the target's and
wins.
Built --disable-shared, unlike FlashForge's own 3.8.2. Static libpython and
static dependencies mean the shipped tree contains no shared object of ours at
all: nothing to version-skew, nothing that can resolve to a /usr/prog copy by
accident, and no new mandatory LD_LIBRARY_PATH entry -- which matters here
specifically, because removing /usr/prog version strings from anvil-env.sh is
the point. The usual objection to static, that C extensions cannot find the
interpreter's symbols, was tested rather than assumed: an out-of-tree
extension cross-built against the staged include/ imports and runs on the
printer with no libpython mapped.
It installs into $MODDIR itself, --prefix=/usr/data/anvil, the same prefix
root s6 uses: bin/python3.13 beside bin/s6-svscan, stdlib in lib/python3.13
beside lib/. One install, not two trees.
That creates a hazard worth naming, because anvil-env.sh PREPENDS $MODDIR/bin
to PATH so s6-svscan can find s6-supervise. `make install` also writes
bin/python3, and a shipped bin/python3 would then sit in front of
FlashForge's for every process that sources anvil-env.sh -- silently handing
klippy and Moonraker an interpreter with none of their extensions. So the
build deletes it and asserts bin/ holds exactly python3.13, verify.sh fails a
package carrying bin/python3, and the gate asks the shell what `python3`
resolves to on anvil-env.sh's own PATH. It answers
/usr/prog/Python-3.8.2/bin/python3, while python3.13 answers ours.
About 28MB and 600-odd files, which takes the install manifest from 70 entries
to 728 -- case-upgrade.sh still passes, which is the check that the manifest
deletion loop scales.
The build image grows a host gcc, which looks redundant beside two
cross-toolchains and is not: cross-building CPython needs a build-python of
exactly the same version to freeze modules and byte-compile the stdlib, so
patch.sh builds an x86-64 3.13 from the same tarball first.
tools/python/ keeps the measurement build and what it cost, as
tools/supervisor/ does for s6. Two gaps recorded there rather than lost: the
printer has NO CA certificates -- /etc/ssl/certs exists and is empty, so TLS
negotiates and verifies nothing, which is true of the stock interpreter too --
and libatomic.so.1 is a new runtime dependency, present on the rootfs,
measured, and removable by linking it statically if that is ever wanted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
491d0dfe1d |
The payload carries s6, built here from a pinned hash
Nothing starts it yet. This step only puts the supervisor in the payload, so
that the boot change in the next phase is a boot change and not a build
change at the same time.
skalibs and s6 2.15.1.0 are pinned in versions.env beside a musl mipsel
toolchain, and bin/patch.sh cross-compiles them the way it already
cross-compiles c_helper.so: unpack a pinned toolchain into work/, build,
check the result, stage it. Nothing binary is vendored in this repo, so
anything the printer executes is built from a hash we chose.
Three properties of the build are load-bearing, and all three were measured
on the replica rather than reasoned about:
* --prefix=/usr/data/anvil, because s6 bakes its libexecdir in at compile
time. Every waiting verb execs s6-svlisten, which spawns s6-ftrigrd from
that baked-in path. Built for any other prefix, s6-svstat and respawn
keep working and only the waiting verbs fail -- it fails late and
partially, which is the worst way for a supervisor to fail. Moving MODDIR
now means rebuilding s6.
* -D_FILE_OFFSET_BITS=64, because without it readdir() returns EOVERFLOW on
this box: a 32-bit build meeting 64-bit inodes. s6-svscan starts cleanly
and then cannot see its own scandir.
* musl, statically. Static glibc produced a 73MB tree here. musl produces
3.6MB, of which we ship 928K -- the 13 supervision binaries and
libexec/s6-ftrigrd, nothing else.
--disable-execline had to be added: s6 links against execline by default and
the build stops without its headers. Turning it off is what makes "we ship no
execline" true rather than aspirational; our run scripts are plain /bin/sh.
The build is cached on the two version strings, unlike c_helper.so which
rebuilds on mtime. That difference is deliberate: chelper's sources are a
checkout someone may be editing, while these are tarballs that cannot change
without their sha256 changing. Staging into the payload is never cached, so a
stale payload is not a thing that can happen. The ELF check runs outside the
cache branch, on the tree that will actually ship: a cross-build that quietly
produced host, big-endian or 64-bit objects looks like a clean build here and
like ENOEXEC on a printer, with nothing to explain why.
libexec/ joins bin/ as a payload directory. Its name is part of the ABI
rather than a preference, for the compiled-in-prefix reason above. The
install manifest from the previous commit picked it up with no change, which
was verified rather than assumed.
case-supervisor.sh now runs against the tree the build produced instead of a
hand-made tarball, and execs all 13 binaries rather than one. Its runit half
is gone: the comparison is over and recorded in tools/supervisor/README.md,
nothing in the build produces runit any more, and a gate that still wanted it
could only ever be fed by hand. bin/verify.sh checks for s6-svscan and for
libexec/s6-ftrigrd separately, because a payload missing only the latter
supervises perfectly and fails every waiting verb.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
666358df4b |
One environment for the whole mod, and one Moonraker
Every script that started a process carried its own copy of the /usr/prog/*/lib list. They drifted, and the drift was the bug: the installer's pre-flight named ten directories while S60web started moonraker with four, so a build could pass the check that decided whether to install it and then fail to run. payload/anvil-env.sh is now the only place that list exists, along with the interpreter. run-append.sh, firmwareExe, start.sh, S60web and S65camera source it. Sourcing twice is a no-op, so inheriting it and re-sourcing it behaves. The `|| FF_PYTHON=python3` fallbacks are gone -- the rootfs ships no other interpreter, so they either resolved to the same binary or to something untested. The installer no longer chooses between builds. It moved the old tree aside, import-tested ours first, and put FlashForge's 2022 Moonraker back whenever that failed: a printer could be running any of three Moonrakers, nothing said which, and the mod's own Mainsail config assumes the new one. It installs what it ships. S60web's fallback to the payload copy goes with it, and every way it can fail to start now says so in the boot log. moonraker-preflight.py is deleted rather than kept as a reporter. By the time a machine is being flashed it is too late to learn the Moonraker in the package does not load, and there is no second build to pick. That check belongs to the build; it lives in the replica gate now. stop() waits for the process and clears the pidfile -- busybox start-stop-daemon does neither, so status could report a recycled PID and restart raced its own start. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Fable 5
|
212d3a1efd |
A fork build without a fork is now impossible to ship
v20260824-nova-kakhovka went out without the Klipper fork: release.yml set KLIPPER_FORK="", bin/patch.sh answered with "(skip) Klipper: keeping stock tree", and every gate stayed green because every Klipper check was conditional on files the broken package simply did not contain. On an already-modded printer the stock 0.12-era overlay half-overwrote the v0.13 tree and klippy died at connect: "'void(*)(struct stepper_kinematics *, double, double, double)' expects 4 arguments, got 3" -- a v0.12 caller against the v0.13 cdef of extruder_set_pressure_advance (upstream c84d78f3f). Note for the next reader of such a traceback: an arg-count TypeError is a MIXED KLIPPY TREE, python against python; a stale c_helper.so shows up as AttributeError instead, because cffi resolves symbols lazily. The fork now travels like every other asset. versions.env pins the creator5 commit and the Ingenic gcc 7.2.0/glibc 2.29 toolchain (the X2000's own; Debian's mips crosses are big-endian or legacy-nan), fetch-assets.sh downloads both by sha256, and patch.sh extracts the pinned tarball whenever KLIPPER_FORK is not a local checkout -- which is exactly what CI now does, with the KLIPPER_FORK="" lines gone from both workflows and vendor/ cached on the pin file. c_helper.so is cross-compiled from the chelper sources of the very tree being shipped whenever it is missing or older than them, so the stale-.so class dies too; test/test-chelper.py returns from 3a28fdf's grave to prove every cdef'd function is exported, and runs both in patch.sh and in verify.sh against the extracted package. BUILD_KLIPPER=fork with no tree is a hard build failure now; stock has to be asked for by name. verify.sh also stops trusting `echo "$LIST" | grep -q`: under pipefail, grep exiting at an early match SIGPIPEs the echo and the check reports 141 -- a race that stayed invisible until the fork tree tripled the payload listing, then intermittently declared Mainsail and Moonraker missing from a package that carried both. Herestrings have no pipe to break. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
ac558f6686 |
Document the Moonraker swap honestly, and finish its doc coverage
The reversibility claim was the strongest promise in the README and
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
4460488ea9 |
Moonraker: pin the last lmdb build -- this python has no sqlite3
v0.9.3 does not start on the printer. It was built, flashed by hand and
tried, and it died on:
ModuleNotFoundError: No module named '_sqlite3'
FlashForge built python 3.8.2 without that module -- there is no
_sqlite3*.so in lib-dynload and no libsqlite3 anywhere on the image,
only the pure-python sqlite3/ wrapper that cannot work without it.
Moonraker moved its database from lmdb to sqlite in v0.9.0, so every
release from there on gets as far as loading the database component and
exits. The last release still on lmdb is v0.8.0, which predates the
webcam "enabled" flag this whole change exists to get.
No release has both, so the pin is now the newest COMMIT that does:
9d0d09d, 2023-12-05, the one before "database: migrate from lmdb to
sqlite". It has the webcam field and reads the printer's existing lmdb
store in place, so nothing is converted and going back to stock is a
clean round trip. Verified on python 3.8 against the printer's exact
library versions with _sqlite3 removed from the interpreter to match.
Two things follow from having shipped a build that could not start.
run-append.sh now asks the printer's OWN python to import the tree
before it moves anything, and keeps the stock server if it cannot. A bad
pin refuses to install instead of leaving the machine with no web UI --
confirmed on the replica, where the v0.9.3 payload is now declined and
the stock Moonraker keeps running.
And the test that passed this build is fixed. It asked only whether a
moonraker.py process ever appeared, which a build that dies on an import
satisfies for a second or two on its way down. It now requires the
process to still be there after a settle, and the log to be free of
import errors. Re-pinned to v0.9.3 it fails, as it should have.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
a40bc861dd |
Ship our own Moonraker: the stock 2022 build hides the camera
Mainsail has shown no webcam panel on this printer no matter how the stream is configured, and the stream was never the problem: nginx serves /webcam/ correctly and mjpg-streamer answers it. FlashForge's Moonraker is a 2022 build (API 1.0.5). Moonraker only grew the webcam "enabled" flag in April 2023, and Mainsail filters its webcam list on exactly that field -- so every [webcam] entry is discarded and the panel is removed from the dashboard entirely. Nothing on either side of that can be configured around; the server has to be newer. So the mod now ships Moonraker the way it ships Mainsail: pinned in versions.env, fetched to vendor/ against a sha256, installed over the stock tree. v0.9.3 is the newest release that runs on the printer's existing python environment untouched -- v0.10.0 wants dbus-fast, a compiled module that is not there and would have to be cross-compiled for mipsel. Only the python package is replaced; the interpreter, the moonraker-env beside it and moonrakerDaemon stay FlashForge's. It rides in the mod payload rather than the software component because the stock run.sh does not extract that component over /usr/prog -- it copies a hand-written list of paths out of it, and a moonraker/ dropped in beside them would never be looked at. run-append.sh installs it, moving the old tree aside and putting it back if the copy fails: this is the one thing the mod writes to /usr/prog that the stock installer does not, and a printer whose Moonraker did not survive has no web UI at all. The database migrates itself -- v0.9.x moved from lmdb to sqlite, lmdb 1.3.0 is installed, so Mainsail's settings survive the upgrade. Covered end to end on the printer replica: that the swap happened, that the installed webcam.py has the enabled field, and that the shipped tree really runs on the printer's own python3.8 under qemu-mipsel. With BUILD_MOONRAKER=0 those skip and the stock server is left alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
3e997e5a44 |
HelixScreen: bump to v0.99.115-creator5.1 (build 2)
Same upstream 0.99.115, rebuilt from creator5 @ 696c0d2: the setup wizard's Tool Offsets step, the connection step pre-filled with 127.0.0.1:7125, the Dock pill in the tool switcher, and Russian gone from the language list. Tarball name is unchanged, so the stale sha256 is what re-downloads it. Klipper fork is already at the tip of origin/creator5; Mainsail v2.18.2 is still the latest release. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oleksandr ShyshatskyiandClaude Opus 5
|
22d16f984e |
Run the printer's own boot script for the update test, and cut the suite to 5m
The install test used to re-implement app_startup.sh: it pulled the glob,
MACHINE and PID out of it with sed, then called the installer itself. A
mistake in our reading of the boot script could never be caught that way.
The package now sits on a real FAT filesystem on a loop device exposed as
/dev/sda1, and app_startup.sh runs verbatim through three boots -- stick in,
stick still in, stick pulled. The last one asserts what actually matters:
nginx and moonraker up, klipper launched, the UI in the foreground where
app_startup.sh's own 5-second ps watchdog can see it.
One substitution, declared in .SIMULATED: the printer's kernel has nls_cp936
and a container kernel does not, so /bin/mount retries without codepage= if
the real options are rejected.
Speed. The image carried the partitions but not the stock install, and
test-env.sh sourced the config files after the environment -- so PRINTER_IMAGE
on the command line was overwritten by the empty value in test.env and the
published image was never used. Measured, per replica case: 22s to unpack the
factory image, 37s to install the stock package, 0.7s with the image. The
stock install is now baked in at image build time (bake.sh -- it needs binfmt
and chroot, so it runs privileged and the result is committed) and recorded in
/usr/prog/.BASELINE; a run asking for a different package reinstalls.
make test: ~20m -> 5m26s. Every section header now carries elapsed seconds.
Dropped four checks that could not fail:
* test-abi's execution half was `qemu "$f" -h || [ $? -lt 126 ]`, which
accepts nearly every exit status
* test-ash's md5sum -s probe printed "inconclusive" on every path
* run-tests.sh's per-file `sh -n` "POSIX" loop ran bash, and its bashism
grep flagged `local`, which busybox ash supports
* CI's shellcheck ended in `|| true`
And one that mattered more: the release workflow's final sim-install loop
ignored every exit status, and sim-install.sh exits 0 on a skip -- so the last
gate before publishing could not fail. It now stops on the first failure and
runs with REQUIRE_PRINTER_SIM=1, which turns skips into failures throughout.
Also: the profile lists in run-tests.sh, RECOVERY_PROFILE and release.yml
still said ssh/web/full/helix and would have failed the release job; work/.sim
was shared, so two suites at once truncated each other's packages; and the
fixture half left a 380KB package in work/out where make test-install picks it
up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GgNhSa1Qvwvxb2nQXT6b1Y
|