starred/firmware
1
0 Fork 0
mirror of https://github.com/Klipper4FlashForge/firmware.git synced 2026-10-22 13:22:16 -05:00
Code Issues Packages Projects Releases 8 Wiki Activity
192 Commits 8 Branches 8 Tags
057a3a11d7856bc6903c19da7c144a0d19add6ee
Commit Graph
5 Commits
Author SHA1 Message Date
Oleksandr ShyshatskyiandClaude Opus 5 99706bb5f7 s6 was legacy-NaN. The kernel this printer runs refuses that at exec(). 
The Bootlin swap two commits ago (musl.cc -> mips32el) fixed a
reachability problem and kept a correctness one: mips32el is plain
mips32r1, whose musl crt/libc is legacy-NaN by construction and cannot
be made to emit nan2008 output at all -- forcing -mnan=2008 on it fails
to LINK ("mixing -mnan=2008 module with previous -mnan=legacy modules").
So every s6 binary this repo has shipped read e_flags=0x1007, defended
in five separate places as correct on the theory that a supervisor
doing no floating point cannot care about NaN encoding.

True of the arithmetic. False of exec(): docs/how-it-works.md already
documented, for a different binary, that "the Ingenic X2000 kernel
returns ENOEXEC for legacy-NaN MIPS" -- the ABI check happens before a
single instruction of the program's own runs, which means it was never
about what s6 computes. None of the replica gates that "measured s6
working on the printer" ever caught this, because none of them ran on
the printer: qemu-mipsel-static's user-mode emulation does not enforce
the same ABI check a real kernel's binfmt loader does. A build that
passed case-supervisor.sh, case-services.sh, case-moonraker313-s6.sh and
a full release run was still shipping a supervisor that a real X2000
kernel may refuse to start at all.

THE FIX. Bootlin also publishes mips32r5el: MIPS Release 5 has no
legacy-NaN silicon to be compatible with, so its musl crt/libc is
nan2008 by construction, and its default codegen already targets
mips32r2. Verified outside the repo before touching anything: fetched
the tarball, cross-compiled skalibs+s6 with it, every binary read
nan2008/o32/mips32r2, and s6-svscan ran and supervised under
qemu-mipsel-static. Section 5b now applies the same gcc-wrapper
discipline 5c already uses for the Ingenic toolchain -- -EL -mnan=2008
-march=mips32r2 baked into the driver, not left to CFLAGS, with a probe
gate before building anything on top -- because "the toolchain's default
happens to be right" is exactly the kind of assumption this bug was.

mips_abi_gate moved earlier in the file so 5b can use it, and s6 is IN
that gate now rather than exempt from it -- exempt is how this shipped
wrong for as long as it did. The gate itself grew a real relaxation
alongside the real tightening: a static EXEC from this musl toolchain
legitimately carries EF_MIPS_PIC (musl's crt is unconditionally -fPIC;
no combination of -static/-no-pie/-fno-PIC removes it, measured) and so
reads the DYN flag value anyway, which is a property of the toolchain
and not a sign of anything wrong -- so the gate now accepts either
0x70001405 or 0x70001407 for any object, rather than picking one from
the ELF Type.

Also bumped S6_STAMP to include MUSL_TOOLCHAIN_FILE: without that, a
checkout with an already-built work/.s6 from before this commit would
read its own stale legacy-NaN tree as current and never rebuild it.

Full real build: "s6: 14 ELF objects are nan2008/o32/mips32r2 -- good".
case-supervisor.sh green on the replica with the new binaries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-27 17:19:03 +03:00
Oleksandr ShyshatskyiandClaude Opus 5 c870d8417d FF_PYTHON moves to our own 3.13, and CI stops rehearsing around it 
The gate this switch was waiting on now exists and passes:
case-moonraker313-s6.sh drives Moonraker through the real boot path --
S40s6's scandir, S62moonraker, readiness gated on :7125 actually
listening, a kill -9 respawn, a stop that stays stopped -- on the
interpreter this commit ships. anvil-env.sh now points FF_PYTHON at
$MODDIR/bin/python3.13 and drops /usr/prog/libsodium/lib from
ANVIL_LIBS, since nothing left running under FF_PYTHON imports libnacl
from there any more (Moonraker carries its own libsodium; klippy is not
on this list at all -- it stays on FlashForge's 3.8.2, started
independently by /usr/prog/klipper/start.sh).

Three real bugs surfaced chasing this through the replica, none of them
about the switch itself:

  * The Phase-1 manifest-based install can only delete files it once
    tracked, so an init.d/S60web surviving from a release that predates
    the manifest system would never be cleaned -- caught by
    case-install.sh, which had never run against real stock packages
    before today. run-append.sh now clears $MODDIR/init.d unconditionally
    on every update: those scripts are 100% mod-owned and get executed,
    so a diff against tracked history is not a strong enough guarantee.
  * case-moonraker313-s6.sh's own readiness-ordering check flaked once:
    its bind-time sampler polled at 1-second resolution while s6-svwait
    is event-driven and returns within a fraction of a second, so
    `date +%s` truncation could show readiness before the bind though the
    real events were in order. Tightened to 0.2s.
  * case-mcu-bringup.sh and case-boot-screen.sh source anvil-env.sh but
    named their own directory variable $MOD, not $MODDIR -- harmless
    while FF_PYTHON was a literal path, and silently wrong the moment it
    started reading $MODDIR. Aliased.

Every other test/integration file touched here is the same story:
FF_PYTHON's callers were hardcoded to FlashForge's 3.8.2 throughout the
suite, and each one either now stages our interpreter as a tarball
(gates.py's mcu_bringup/boot_screen/moonraker, matching the pattern
already used for python()/moonraker313_s6) or asserts against the new
target. case-libpath.sh's premise changes with it: the three ANVIL_LIBS
entries that remain are read as what klippy's process tree still needs,
not what every mod process needs -- Moonraker maps zero /usr/prog
libraries now, measured.

payload/anvil-service.sh's SVC_S6_SCANDIR line tripped test_paths.py's
absolute-path regex, which reads a closing brace immediately followed by
a suffix as a stock rootfs path in its own right; split onto its own
line so a word character precedes it instead, which the regex already
excludes.

bin/fetch-assets.sh had a real latent bug of its own, independent of
this switch: the Ingenic toolchain fetch was gated on BUILD_KLIPPER=fork
alone, a condition that predates CPython 3.13's unconditional need for
the same compiler. A BUILD_KLIPPER=stock build from a clean vendor/
would reach the Python cross-build with sources and no compiler. Also
added curl --retry -- musl.cc flaked mid-download in CI while this was
being tracked down.

.github/workflows/ci.yml's fixture-only `test` job now actually
cross-builds s6 and CPython 3.13 instead of never exercising that path:
it fetches just those two build inputs (Mainsail/HelixScreen/Klipper
stay stubbed, unchanged) and caches vendor/, work/.s6, work/.py313 and
work/.sodium on versions.env.

Full suite, replica required, against real stock packages fetched for
this run: 23 passed, 0 failed, 0 skipped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-27 13:38:16 +03:00
Oleksandr ShyshatskyiandClaude Opus 5 a8a46e21d2 The build makes the packages Moonraker runs on, from hashed sources 
Section 5c grows a second half and section 5d appears. The 18 packages and
libsodium that two spikes proved are now built by bin/patch.sh from pins in
versions.env, and land in the payload at lib/python3.13/site-packages and
lib/libsodium.so.

WHERE THE PACKAGE BUILD HAD TO GO, and it is not obvious: inside 5c, against
the UNTRIMMED interpreter stage, before the trim runs. The headers INCLUDEPY
names, the static dependency libraries pillow and cffi link against, and the
x86-64 build-python all exist only at that moment -- the trim deletes
include/ and config-3.13-* and then removes the staging tree entirely. The
alternative was caching a second untrimmed ~200MB tree. The price of not
doing that is that either stamp going stale rebuilds both halves, which is
written down at the stamp rather than left to be discovered.

The build no longer touches PyPI. pip comes from --with-ensurepip=install on
the build-python, out of the same pinned CPython tarball, and every pip wheel
runs --no-index --no-build-isolation --no-binary :all: against hashed sdists
in vendor/. That last flag is not caution: during the spike pip silently
downloaded x86-64 manylinux wheels for cffi, greenlet and tornado, three
wrong-architecture objects sailed through the build, and only the ABI gate
caught them.

Which is why the gate now reaches the STAGED PAYLOAD rather than the build
cache. It walked $PY_BUILD/bin and $PY_BUILD/lib, so libsodium and the twelve
extension modules staged straight into $MOD_PAYLOAD would have gone
unexamined. 69 ELF objects now, plus libsodium's resolved .so. Deliberately
not all of $MOD_PAYLOAD: s6 is musl-built and correctly reads e_flags=0x1007,
mips1 and legacy-NaN, which is right for a static binary that does no floating
point and is measured working on the printer. It has its own gate in 5b. That
distinction is recorded in patch.sh, because a sweep that assumes every object
must be nan2008 flags all thirteen s6 binaries and looks like a disaster.

Two pins carry reasons that cost someone an afternoon: pillow is 11.0.0
because 10.3.0 cannot be built by ANY 3.13 -- its setup.py execs and then
reads locals()["__version__"], which PEP 667 broke -- and streaming-form-data
is 1.19.1 because Moonraker's own pin ships pregenerated Cython calling the
four-argument _PyLong_AsByteArray, which grew a fifth parameter in 3.13.

docker/Dockerfile.build needs zlib1g-dev and patch, and the image must be
rebuilt. The first is a genuinely nasty one: a build-python without zlib
cannot unpack a wheel, or install pip, and ensurepip fails INSIDE a `make
install` that reports success. patch.sh now asserts zlib and pip on the
build-python rather than failing later and further away.

The install manifest goes from 1872 paths to 2940 and case-upgrade.sh still
passes, which is the check that the deletion loop scales. Note that the "70 to
728" in the commit that landed the interpreter was wrong -- 728 predates the
interpreter itself; HEAD's own patch.sh produces 1872 here, measured.
anvil.tar.xz grows 1.6MB packed, about 15MB on the printer.

FF_PYTHON still points at FlashForge's 3.8.2 and /usr/prog/libsodium/lib is
still in ANVIL_LIBS. Both comments now say what changed -- the packages exist
-- and what has not: nothing has run through S62moonraker under s6 on 3.13.
The two lines move together, in the commit that has that gate behind it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-27 11:29:52 +03:00
Oleksandr ShyshatskyiandClaude Opus 5 6a750f6bcc Moonraker serves on the 3.13 we built, with nothing from /usr/prog 
The previous commit proved every component IMPORTS. Importing is not serving,
so this runs it. case-moonraker313.sh starts Moonraker on the exact command
line payload/etc/s6/moonraker/run execs -- only with our interpreter --
against the mod's own assets/moonraker.conf, and asks the printer what
happened:

  * bound :7125 in 3s, with /proc/PID/cmdline confirming our interpreter and
    our entry point rather than a coincidence
  * GET /server/info answers 200 with 23 components loaded and none failed.
    /machine/system_info, a different component reading this box's own /proc,
    reports cpu: mips and python 3.13.7 [GCC 7.2.0]
  * the database really works, which is the entire reason for the exercise: a
    POST wrote a namespace, a GET read it back, the namespace appears beside
    Moonraker's own, and data.mdb is 147456 bytes on disk -- through lmdb's
    CPython extension, not the compile-at-import cffi fallback that would try
    to invoke a compiler this printer does not have
  * still alive and still answering 120 seconds later, with no tracebacks in
    its log, and SIGTERM released the port in 2s
  * the RUNNING process maps zero libraries under /usr/prog, with
    LD_LIBRARY_PATH completely unset -- not merely purged of /usr/prog
  * negative controls: nothing on :7125 beforehand, and with site-packages
    moved aside it never binds at all

libsodium was the last /usr/prog string in the picture and is now ours.
libsodium 1.0.20, cross-built with the Ingenic toolchain in 24 seconds, gated
at 0x70001407 nan2008/o32/mips32r2, needing only libpthread, libc and the
loader, 406KB stripped into $MODDIR/lib. It computes rather than merely
loading: sodium_version_string is 1.0.20, its BLAKE2b-256 matches CPython's
own _blake2 on the same kernel, and libnacl.sign.Signer/Verifier -- the exact
ed25519 pair authorization.py imports -- signs, verifies, and rejects a
tampered message.

It costs anvil-env.sh nothing, which was not obvious and had to be measured.
libnacl's third fallback builds a path out of its own __file__, so from
$MODDIR/lib/python3.13/site-packages/libnacl it resolves $MODDIR/lib/libsodium.so
by absolute path -- working with LD_LIBRARY_PATH unset entirely. Recorded in
the README, because it works only because the prefix happens to contain the
string "lib": a prefix without it would break this silently.

klippy was scanned rather than guessed at. Its module-scope third-party
imports are cffi, greenlet, jinja2, pyserial and numpy; four of the five are
built and proven, and cffi already dlopens the printer's real c_helper.so.
numpy is at module scope only in extras/stepper_resonance_tester.py, which
Klipper loads on demand, and it is a multi-day problem that costs input-shaper
calibration rather than printing. There is no removed-stdlib usage anywhere in
klippy -- no imp, distutils, asyncore, cgi or getargspec.

What is left is packaging, not capability, and the README says so in order.
The sharp one: section 5c's trim deletes include/ and config-3.13-*, which the
package build needs, so the package build has to run inside 5c against the
untrimmed stage. And the ABI gate walks the build cache rather than the
payload, so anything staged straight into $MOD_PAYLOAD would bypass it --
coverage, not logic.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-27 10:41:24 +03:00
Oleksandr ShyshatskyiandClaude Opus 5 da65764b86 Moonraker's whole component set imports on the 3.13 we built 
Nineteen packages cross-built for mipsel, twelve of them C extensions, every
one gated at nan2008/o32/mips32r2. Not wired into bin/patch.sh yet -- this
preserves the build and, more importantly, what it cost, the way
tools/supervisor/ does for s6. The measurement lives in
test/integration/printer/case-pyext.sh rather than in a paragraph.

Two results carry the phase.

The first: 3.13's cffi dlopens the printer's OWN
/usr/prog/klipper/klippy/chelper/c_helper.so, resolves stepcompress_alloc,
steppersync_alloc and itersolve_generate_steps, and calls one. That is phase
7's central question -- whether an interpreter we built can drive
FlashForge's C helper -- answered by doing it, and it is the whole reason the
interpreter had to be glibc rather than musl.

The second: all 16 Moonraker components import on 3.13, with the list taken
from CORE_COMPONENTS plus moonraker.conf rather than written down. The
negative control is that moving the tree aside fails every third-party import
and the database component.

The package list is deliberately NOT moonraker-requirements.txt, which
installs every optional component's dependencies. It is the closure of what
the ENABLED set imports plus klippy's, so no apprise, ldap3, paho-mqtt or
zeroconf. msgspec and uvloop are skipped on purpose: Moonraker guards both
with contextlib.suppress(ImportError) and falls back to the stdlib.

greenlet was the expected blocker and blocked for an unexpected reason. Its
MIPS stack-switching assembly was never a problem; greenlet 3.x is C++ and
writes its PyTypeObjects as designated initializers, which gcc 7.2 takes only
in declaration order with no gaps. The exact rule was probed rather than
guessed -- in-order contiguous fine, a trailing gap fine, an interior or
leading gap refused -- and fill-designators.py writes the skipped fields back
as explicit zeros, reading field order from the target's own headers and
refusing to guess when a designator is not in them. Semantically a no-op:
static storage duration, those fields were already zero. It switches out and
back, in order, on the printer.

Three traps worth having in the log. pip silently downloaded x86-64 manylinux
wheels for cffi, greenlet and tornado on the first run -- three
wrong-architecture .so files sailed through the build and only the ABI gate
caught them, so --no-binary :all: is mandatory here rather than cautious.
lmdb chose its cffi backend and produced a py3-none-any wheel with mdb.c
inside, which is phase 6's recorded trap rebuilt from scratch: on the printer
that path invokes a compiler that does not exist. And pillow 10.3.0 cannot be
built by any 3.13, cross or not, because its setup.py execs and then reads
locals()["__version__"], which PEP 667 broke.

One dependency is unsolved and named rather than buried: libnacl ctypes-loads
libsodium, which exists on this box only at /usr/prog/libsodium/lib. Measured
both ways. So the authorization component keeps the last /usr/prog string in
the picture until libsodium is cross-built or that one path entry is kept.

What is left before FF_PYTHON could switch is written down in the README, and
the honest remaining unknown is at the top of it: every component imports and
the entry point loads, but importing is not serving. Nothing has bound :7125
on 3.13 yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-27 10:14:38 +03:00