mirror of
https://github.com/Klipper4FlashForge/firmware.git
synced 2026-10-21 12:52:16 -05:00
Oleksandr ShyshatskyiandClaude Opus 5
b98552386b
The software component existed to give the mod one exec hook. It is not needed for that any more, so a release now carries our installer and our payload and nothing else: 26MB smaller, 21MB of which was FlashForge's own firmwareExe travelling back to the partition it was already on, along with their klippy tree, their wifi module and their configs. WHAT IT GIVES BACK. /usr/prog/PROGRAM/software is no longer rewritten, so the version directory there stays FlashForge's with their genuine firmwareExe inside it -- and app_startup.sh's five-second watchdog, which copies one out of that directory when nothing by that name is running, has something to copy again. A printer whose payload fails to install now comes up on the stock UI instead of a blank screen. That recovery has been broken since we started deleting their binary out of a component we shipped; docs/how-it-works.md claimed it worked throughout. THE ROOT PASSWORD moves with it, because it used to ride in the component's shadow file. bin/pack.sh bakes ROOT_PW_HASH into the installer, single-quoted -- an unquoted $6$... would have the printer's shell expand $6 as a positional parameter and ship a truncated hash -- and the installer writes it to /usr/prog/etc/shadow. The random-password path asks one question instead of four: FlashForge ship the same root hash on every machine, so a live shadow still carrying it means nobody has set a password and it is ours to set. The .prev-root-hash recording, the shipped-vs-live comparison and the restore-it- afterwards step were all machinery for surviving a file that nothing overwrites any more, and are gone. pack.sh checks the hardcoded constant against the stock package it built from, because getting it wrong is silent and total: every printer would read as "already set" and none would ever get a password. bin/patch.sh is bin/payload.sh. It patched nothing any more -- no splice, no shadow edit, no firmwareExe deletion -- and what is left is the payload: the feed installed by the printer's own opkg, the seeded config, the manifest. Its sections are renumbered 1-5 with no gaps, and the five tombstone comments marking where removed sections used to be are gone with them. test_recipe_layout.py loses six of its nine tests. Each compared the tree against a description of the tree -- no fourth directory kind, each kind still used, 3rdparty carries no files of ours, installer/ holds exactly two named files -- and none could go red for a reason a printer would notice, while all six went red the moment the layout legitimately changed. The two that bite are kept: a payload/ path escaping $MODDIR, and a recipe name at both levels, which makes pkg_dir silently build the wrong package. SW_VER is deleted from config.env.example and both workflows: it stamped a component that is not built. Gates: 112 static tests pass. The generated installer is verified after substitution -- all four lines present, and `sh -n` over the result, because a hash with a stray character in it would otherwise reach a printer as a syntax error whose only symptom is a package that installs nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CH7Yb4wuPhnppHAB25UX7e
299 lines
15 KiB
Markdown
299 lines
15 KiB
Markdown
# A Python for the Creator 5 Pro that has sqlite3
|
|
|
|
An adoption, not an investigation -- but it started as one, and this is what
|
|
the investigation measured. It answers one question: can we put an interpreter
|
|
on this printer that is not FlashForge's, and does it have the module whose
|
|
absence has been shaping this repo's decisions for two releases?
|
|
|
|
Everything below was measured on the printer replica (real `rootfs.squashfs`
|
|
under qemu-mipsel) or read out of the objects with `readelf`, not off a
|
|
project page. `build.sh` beside this file rebuilds the whole thing in a
|
|
throwaway container and is where the numbers came from;
|
|
`pkg/python` is the same build wired into the real build lane,
|
|
and `qa/replica/` is what proves the result works on the printer's own kernel
|
|
-- `test_install.py` over the installed interpreter, `test_boot_screen.py`
|
|
running our own code on it, and `test_abi.py` over every ELF it ships.
|
|
|
|
## Why: one missing module, two releases of consequences
|
|
|
|
FlashForge built `/usr/prog/Python-3.8.2` **without `_sqlite3`**, and shipped
|
|
no `libsqlite3` anywhere on the rootfs. Measured on the replica, with the
|
|
library path the interpreter needs to start at all:
|
|
|
|
ModuleNotFoundError: No module named '_sqlite3'
|
|
|
|
That single omission is why `MOONRAKER_VERSION` in `versions.env` is a
|
|
2023 commit rather than a release: every Moonraker from v0.9.0 onward keeps
|
|
its database in sqlite, and v0.9.0 is where it moved off lmdb. The pin is the
|
|
last commit that still uses lmdb *and* has the webcam fix Mainsail needs. No
|
|
release has both. That is not a Moonraker problem; it is this problem, seen
|
|
from downstream.
|
|
|
|
The alternatives were: build a `_sqlite3` for 3.8.2 (a module against an
|
|
interpreter we did not build, whose ABI we would be guessing at), or build a
|
|
whole interpreter. A from-source cross-build costs the same either way, so it
|
|
is a whole interpreter -- and a current one, which also ends 3.8's end-of-life
|
|
status as a thing anyone has to think about.
|
|
|
|
## What it is
|
|
|
|
CPython **3.13.7**, `--disable-shared`, cross-compiled with the repo's pinned
|
|
Ingenic gcc 7.2 / glibc 2.29 toolchain, against seven static C libraries built
|
|
alongside it (OpenSSL 3.0.15, sqlite 3.46.1, zlib 1.3.1, libffi 3.4.6,
|
|
xz 5.4.7, bzip2 1.0.8, expat 2.6.4). Everything is pinned by sha256 in
|
|
`versions.env`.
|
|
|
|
`lib-dynload` comes to 60 extension modules, including the ones this printer
|
|
could never have had: `_sqlite3`, `_ssl`, `_hashlib`, `_ctypes`, `_lzma`,
|
|
`_bz2`, `zlib`, `pyexpat`, `_asyncio`, `_decimal`, `_multiprocessing`.
|
|
|
|
## The ABI, which is the whole game
|
|
|
|
The printer's kernel loads **little-endian, NAN2008, o32, mips32r2** and
|
|
nothing else. It refuses everything else with `ENOEXEC` -- "cannot execute
|
|
binary file" -- and explains nothing.
|
|
|
|
e_flags = 0x70001405 an EXECUTABLE
|
|
e_flags = 0x70001407 a SHARED OBJECT -- 0x2 more, EF_MIPS_PIC
|
|
loader = /lib/ld-linux-mipsn8.so.1 rootfs glibc 2.33
|
|
|
|
Both words are correct and any ABI gate must accept both, or it fails on every
|
|
one of the 56 extension modules in the tree. `qa/replica/test_abi.py` -- the
|
|
only ABI gate there is now -- masks the low three bits off for exactly this
|
|
reason. `c_helper.so` reads `0x70001407`
|
|
too, for the same reason.
|
|
|
|
**`-EL -mnan=2008` are mandatory in CFLAGS *and* in LDFLAGS.** This toolchain
|
|
defaults to big-endian legacy-NaN; the flags are what move it. Passing them in
|
|
`CFLAGS` is not enough, because several of the eight projects here do not
|
|
forward `CFLAGS` to their link line, and an object linked without them is
|
|
legacy-NaN however it was compiled.
|
|
|
|
So the build does not pass the flags at all. It installs **PATH wrapper
|
|
scripts that bake them into the gcc driver**:
|
|
|
|
#!/bin/sh
|
|
exec /toolchain/bin/mips-linux-gnu-gcc -EL -mnan=2008 "$@"
|
|
|
|
and no build system gets a vote. That is the single most important trick in
|
|
either copy of this build. The wrapper is itself gated -- compile
|
|
`int main(void){return 0;}`, read `e_flags`, refuse to continue unless it is
|
|
`0x70001405` -- before 300MB of anything is built on top of it.
|
|
|
|
## musl is forbidden here
|
|
|
|
s6 is built against musl in this same repo, and s6 is 73MB with static glibc
|
|
versus 3.6MB with musl. None of that applies to the interpreter: klippy
|
|
`dlopen`s `c_helper.so`, which is a **glibc** shared object, and a musl-linked
|
|
process cannot dlopen it. So this uses the Ingenic glibc toolchain -- the same
|
|
one that builds `c_helper.so`, which is also what makes them a matched pair.
|
|
|
|
glibc 2.29 (toolchain) against 2.33 (rootfs) is forward compatible, and the
|
|
measured `NEEDED` list is six ordinary sonames:
|
|
|
|
libpthread.so.0 libdl.so.2 libatomic.so.1 libm.so.6 libutil.so.1 libc.so.6
|
|
|
|
`libatomic.so.1` is the one worth naming. 64-bit atomics on mips32 are
|
|
out-of-line calls into libatomic and CPython 3.13's `_Py_atomic_*` needs them.
|
|
It **is** on the rootfs -- `/lib/libatomic.so.1.2.0`, measured -- but it is
|
|
the only one of the six that is not obviously part of a base system, so the
|
|
gate says so out loud.
|
|
|
|
## Static dependencies, and a static libpython
|
|
|
|
All seven C libraries are built `-fPIC` and linked in as `.a`. Nothing of ours
|
|
ships as a `.so`, and `--disable-shared` puts libpython inside the executable
|
|
rather than beside it.
|
|
|
|
The reason is not size, it is **isolation**. A shared build needs an
|
|
`LD_LIBRARY_PATH`, and on this printer that path is a minefield: `/usr/prog`
|
|
carries `libffi.so.8` while the rootfs carries `libffi.so.7`, FlashForge's
|
|
openssl is 1.0.2d, and a stock OTA can replace any of it. Statically linked,
|
|
the interpreter maps *nothing* under `/usr/prog` -- which
|
|
`case-python.sh` section 3 proved by reading `/proc/self/maps` of a process
|
|
that has imported `ssl`, `ctypes`, `sqlite3`, `lzma`, `zlib` and `hashlib`
|
|
first. Zero mappings. It also means it starts with `LD_LIBRARY_PATH` empty,
|
|
which FlashForge's cannot.
|
|
|
|
The cost is a few MB of duplicated libcrypto between `_ssl.so` and
|
|
`_hashlib.so`. At 30MB that is a good trade.
|
|
|
|
A `--disable-shared` interpreter is not automatically usable by third-party
|
|
extension modules -- they resolve Python's symbols out of the running
|
|
executable, which only works if it was linked `-export-dynamic`. That was
|
|
checked, not assumed: an out-of-tree extension module was cross-compiled
|
|
against the staged `include/` and imported successfully on the replica.
|
|
|
|
## The trap that costs a day: `_sqlite3` and `-lm`
|
|
|
|
**The one module this entire build exists for goes missing silently.**
|
|
|
|
A shared `libsqlite3.so` carries its own `DT_NEEDED` on libm. A
|
|
`libsqlite3.a` does not. So CPython's
|
|
|
|
checking for sqlite3_bind_double in -lsqlite3
|
|
|
|
link probe fails with a wall of `undefined reference to floor/log/pow/...`,
|
|
and CPython 3.13 **records `_sqlite3` as "missing" and carries on**. It is a
|
|
probe failure, not a compile failure. Nothing in 400 lines of build output
|
|
says why. You get a clean build, a working interpreter, and no sqlite3.
|
|
|
|
The fix is two exports:
|
|
|
|
LIBS="-latomic -lm"
|
|
LIBSQLITE3_LIBS="-L$DEP/lib -lsqlite3 -lm"
|
|
|
|
the second stating the link line outright so pkg-config cannot reorder `-lm`
|
|
out from under the probe. And because a silent failure fixed by an
|
|
easily-deleted export is a silent failure waiting to come back, **both copies
|
|
of this build hard-fail when `_sqlite3` is absent**, `bin/verify.sh` checks
|
|
the packaged tree for it, and `case-python.sh` did not merely import it: it
|
|
created a database, inserted rows, closed it, and reopened it from disk **in a
|
|
second process**. An import proves the `.so` loaded. Only the second process
|
|
proves there is a database.
|
|
|
|
## Two OpenSSL traps
|
|
|
|
Both hit for real on 3.0.15:
|
|
|
|
* **`no-docs` does not exist before OpenSSL 3.1.** On 3.0.x it is an
|
|
"Unsupported options" *hard error*, not a warning.
|
|
* **The `linux-mips32` target hardcodes `-mips2`** into its cflags. This
|
|
toolchain defaults to `-mfp64`, and gcc rejects the combination below
|
|
mips32r2: *"'-mgp32' and '-mfp64' can only be combined if the target
|
|
supports the mfhc1 and mthc1 instructions"*. User cflags land *after* the
|
|
target's on the command line, so appending `-mips32r2` puts the ISA back
|
|
where the printer is. If a future OpenSSL orders them the other way,
|
|
`linux-generic32` (portable C, no mips assembly) is the fallback -- and both
|
|
copies of the build take it automatically rather than leaving it as a note,
|
|
because the failure is a screenful of assembler errors that says nothing
|
|
about ISA levels.
|
|
|
|
This is why `OPENSSL_VERSION` is pinned to the 3.0 LTS branch and not bumped
|
|
casually.
|
|
|
|
## The gap: there are no CA certificates
|
|
|
|
`--openssldir` points at `/usr/data/anvil/ssl`, which is where the
|
|
interpreter looks for a trust store **on the printer**. Nothing installs one
|
|
there, and the rootfs has none to borrow.
|
|
|
|
So: `import ssl` works, TLS works, and **verifying a certificate chain does
|
|
not**. `ssl.create_default_context()` builds a context with an empty trust
|
|
store and every verified connection fails. Anything that fetches over https on
|
|
this interpreter has to be handed a `cafile` of its own until somebody ships a
|
|
`ca-certificates` bundle into that directory -- which is ~200KB and an open
|
|
piece of work, not a decision.
|
|
|
|
`case-python.sh` printed the trust-store paths and said which of the two states
|
|
the printer was in, rather than asserting either, so the day someone shipped a
|
|
bundle the gate would report it instead of failing.
|
|
|
|
## Sizes and time
|
|
|
|
| | files | on disk |
|
|
|---|---|---|
|
|
| `make install` output | 2909 | 183 MB |
|
|
| trimmed -- what ships | 608 | 30 MB |
|
|
| trimmed, gzipped | | 10.4 MB |
|
|
|
|
The trim drops the CPython test package and `idlelib` (nothing on a printer
|
|
runs either), `tkinter` (there is no X11), `include/` and `config-*/` (they
|
|
exist to *build* extension modules, which happens on a developer's machine),
|
|
every `.a`, and every `__pycache__` -- 12MB of `.pyc` for modules that will be
|
|
imported once if ever, and which the interpreter regenerates on demand anyway.
|
|
The interpreter and every `lib-dynload/*.so` are stripped.
|
|
|
|
It also empties `bin/` of everything except `python3.13` itself -- the
|
|
`python3` symlink, `idle3`, `pydoc3` and the `*-config` scripts -- and drops
|
|
`lib/pkgconfig`. Those are all build-time or convenience files, and in a
|
|
*shared* prefix root they are not neutral: `bin/` and `lib/` here also hold
|
|
s6's binaries and, through `anvil-env.sh`, sit on every mod process's PATH.
|
|
See "Where it lands" above. The build asserts `bin/` holds exactly one name
|
|
afterwards, so a future CPython that installs one more launcher fails the
|
|
build rather than quietly adding a name to PATH.
|
|
|
|
**Cold build: 142 seconds** (`tools/python/build.sh`, 16 cores), of which about
|
|
90 is the x86-64 build-python. In the build lane it is ~100s and it happens
|
|
**once**: `bin/payload.sh` caches the cross-built tree in `work/.py313` under a
|
|
stamp of all eight versions, so every later build copies rather than compiles.
|
|
The stamp is all eight and not just CPython's, because a bumped OpenSSL under
|
|
an unchanged `PY_VERSION` has to rebuild.
|
|
|
|
Cross-building CPython needs a build-python of the **same** version -- the
|
|
Makefile runs it to freeze modules, generate the deepfreeze sources and
|
|
byte-compile the stdlib, and `configure` hard-errors when the version does not
|
|
match. That is why `docker/Dockerfile.build` carries a host C compiler, and why
|
|
`PY_VERSION` moves both halves of the build at once.
|
|
|
|
## Where it lands
|
|
|
|
`$MODDIR` -- `/usr/data/anvil` -- is a `--prefix` root. That is the whole
|
|
reason s6 was configured for it, and it is why the mod's tree has `bin/`,
|
|
`lib/`, `libexec/` and `etc/` directly inside it rather than a directory per
|
|
package. The interpreter is configured `--prefix=/usr/data/anvil` like
|
|
everything else, so it installs as `bin/python3.13` beside `bin/s6-svscan`,
|
|
with its stdlib in `lib/python3.13/`. One prefix, one place to look.
|
|
|
|
That path is **compiled in** -- `sys.prefix` and the stdlib search that follows
|
|
from it -- so moving the directory on the printer breaks the interpreter the
|
|
way moving s6 breaks its waiting verbs. It has to be rebuilt, not renamed;
|
|
which is itself an argument for keeping it in the one prefix the mod already
|
|
guarantees rather than inventing a second one.
|
|
|
|
**One thing does not ship: the `python3` symlink.** `make install` puts it
|
|
beside `python3.13`, along with `idle3`, `pydoc3` and the `*-config` scripts.
|
|
`anvil-env.sh` prepends `$MODDIR/bin` to `PATH` -- it has to, because
|
|
`s6-svscan` execs `s6-supervise` by name -- so a `bin/python3` of ours would
|
|
sit *in front of* FlashForge's for every process that sources it, and anything
|
|
saying `python3` rather than `"$FF_PYTHON"` would change interpreter without
|
|
anyone deciding to. The build deletes it before staging and asserts that
|
|
`bin/` contains `python3.13` and nothing else; `bin/verify.sh` checks the
|
|
packaged tree for it; and `case-python.sh` section 6 asked the *shell*, with
|
|
the mod's own PATH in force, what `python3` resolved to. The answer is still
|
|
FlashForge's binary.
|
|
|
|
## It ships. Nothing runs it.
|
|
|
|
*[Superseded: `FF_PYTHON` is `$MODDIR/bin/python3.13` now, and Moonraker,
|
|
klippy, `ff-startup.py`, `ffscreen.py` and the MCU bring-up all run on it. What
|
|
follows is the state this investigation ended in.]*
|
|
|
|
`payload/anvil-env.sh` still points `FF_PYTHON` at
|
|
`/usr/prog/Python-3.8.2/bin/python3`. That is timing, not preference: klippy,
|
|
Moonraker and `ff-startup.py` import third-party C extensions -- tornado,
|
|
lmdb, cffi, greenlet, pillow, libnacl -- which exist on this printer only as
|
|
mipsel `.so` files built against 3.8. None has been cross-built for 3.13.
|
|
Switching before they exist is a dark screen and an ImportError in a log
|
|
nobody reads.
|
|
|
|
FlashForge's tree is left alone in both directions: nothing writes to
|
|
`/usr/prog`, and our interpreter lives entirely under `/usr/data/anvil` like
|
|
everything else this mod installs. This is deliberately the same shape as
|
|
phase 2, which shipped s6 without starting it -- the artefact reaches real
|
|
printers where it can be measured, and the switch stays a one-line change
|
|
instead of a build change and a boot change arriving together. Cross-building
|
|
the extensions is the actual work in adopting this; not the interpreter.
|
|
|
|
## Reproducing
|
|
|
|
./bin/fetch-assets.sh --all # pins + the Ingenic toolchain
|
|
./tools/python/build.sh # 142s, artefacts in tools/python/out/
|
|
|
|
That is how the numbers above were measured, and it is kept here for the
|
|
record and for trying a version bump before it becomes a pin. It is not how
|
|
the shipped interpreter is built: `pkg/python` compiles it inside the repo's
|
|
own build image, from the sha256-pinned tarballs, against the seven static
|
|
libraries the feed builds as their own `-dev` packages, and caches it in
|
|
`work/pkg/python`. `bin/payload.sh` section 5c runs that recipe and stages what
|
|
it produced; it no longer contains a compiler invocation of its own.
|
|
|
|
The gate, against the tree the build actually produced:
|
|
|
|
make build && make qa-replica
|
|
|
|
`case-python.sh`, `printer-exec.py` and the `make test-python` target that
|
|
drove them are gone. The replica lane installs the real package through the
|
|
printer's own installer and asks its questions of that, which is strictly more
|
|
than hand-feeding a tarball into a case script could.
|