starred/firmware
1
0 Fork 0
mirror of https://github.com/Klipper4FlashForge/firmware.git synced 2026-10-19 03:49:50 -05:00
Code Issues Packages Projects Releases 8 Wiki Activity

Adopt the s6-rc boot graph -- DOES NOT BOOT YET, see below

Browse Source 
Merge of qa-suite-bridgehead into the packaging line. The seven init.d/S*
scripts and the hand-written etc/s6 servicedirs are gone; anvil-core ships
etc/s6-rc/source/ and bin/patch.sh compiles it into etc/s6-rc/compiled/ after
the payload is installed. klipperDaemon joins firmwareExe and start.sh in
payload/prog/, all three linked into place by anvil-link-prog.sh.

WHAT IS KNOWN BROKEN. qa/replica/test_s6rc.py fails: the tree never comes up,
because s6-rc-init dies with

    unable to supervise service directories in /run/s6-rc/servicedirs:
    Value too large for defined data type

That is EOVERFLOW out of the readdir() loop in s6rc_servicedir_manage.c, which
checks errno after every entry. Measured, and these are ruled out:

  * not the live directory's filesystem -- fails identically with -l on
    /usr/data as with the default /run, and the inodes there are 3 and
    1358496, nowhere near 32 bits
  * not the database -- s6-rc-db lists all eleven services out of it, and a
    database recompiled inside the replica by the TARGET s6-rc-compile fails
    exactly like the one compiled natively on x86
  * not a missing -D_FILE_OFFSET_BITS=64 -- both s6-svscan and s6-rc-init
    reference readdir64 and never plain readdir

What is left is the toolchain. qa-suite-bridgehead built s6, s6-rc, execline
and skalibs with the Bootlin musl toolchain, where readdir has no EOVERFLOW
path at all; the packaging line builds them as recipes against the Ingenic
glibc, where it does. s6-svscan looked healthy only because it was started on
an EMPTY scandir, which never reaches the loop.

Everything else is green: 262 static tests, and bin/verify.sh passes on the
73MB package, including the graph's own "compiled s6-rc database" check.

Resolution notes: the five MOD_* switches stay deleted -- firmwareExe defaults
every gate to 1, so nothing changes -- but MOD_CAM is now worth having by hand
on a cameraless printer, because the camera is a longrun that waits out
timeout-up. anvil-service.sh went with init.d; nothing sourced it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VpUmBb4xtmnmtTosonqeNi
This commit is contained in:
Oleksandr ShyshatskyiandClaude Opus 5 committed 2026-08-29 19:27:07 +03:00
2 parents 39b5cb0d0b 1191f87411
commit abd4a01d73
71 files changed
+2212 -3497
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

bin/patch.sh
+114
View File
@@ -197,6 +197,120 @@ fi
# shipped unconditionally, a BUILD_MOONRAKER=0 build still overwrote the
# printer's config with one written for a server it was not installing.
# --------------------------------------------- 5b-2. the s6-rc database
# The boot order, compiled. payload/etc/s6-rc/source/ holds the definition
# directories (one per service, plus the ok-all bundle) and this turns them
# into the binary database s6-rc-init reads at boot.
#
# COMPILED HERE, ON THE BUILD HOST, NOT ON THE PRINTER. That needs a
# s6-rc-compile that runs on x86, which means a second, native build of the
# same four tarballs -- which is why this step exists at all and is not two
# lines. The alternative was shipping the target s6-rc-compile (131KB) and
# compiling at install time, and it was rejected for the reason phase 6 of
# docs/notes/80-s6-migration.md states as "do the gate first, not the
# installer": a database compiled on the printer can fail on a printer in a
# way CI never sees, and the ABI gate above cannot look at a database.
#
# THAT THIS WORKS AT ALL WAS MEASURED, not reasoned from the manual. On
# 2026-08-28, in the replica: the same source tree compiled by the native
# s6-rc-compile and by the target one under qemu produced byte-identical
# db, n and resolve.cdb, and `diff -r` found no difference in the generated
# servicedirs. The database files are host-neutral by construction (every
# integer goes through uint32_pack_big) and the servicedirs are neutral
# BECAUSE both builds are configured --prefix=$MODDIR -- the #! line
# s6-rc-compile writes into the oneshot runner comes from the execline the
# COMPILER was linked against, not from the one we ship. Configure the native
# stack with a different prefix and every oneshot on the printer dies with
# ENOENT while every longrun keeps working.
# The source tree comes out of the INSTALLED payload, not out of the recipe
# directory -- anvil-core ships it, so compiling what shipped is the only way
# the database and the package cannot disagree.
S6RC_SRC="$PAYLOAD_DIR/etc/s6-rc/source"
S6RC_NATIVE="$PWD/work/.s6-native"
# The four versions and nothing else: this compiler is built by the build
# image's own gcc, so there is no toolchain file to key on.
S6RC_NATIVE_STAMP="$SKALIBS_VERSION $EXECLINE_VERSION $S6_VERSION $S6RC_VERSION"
[ -d "$S6RC_SRC" ] || { echo " !! no s6-rc source tree at $S6RC_SRC" >&2; exit 1; }
if [ ! -x "$S6RC_NATIVE/bin/s6-rc-compile" ] \
|| [ "$(cat "$S6RC_NATIVE/.version" 2>/dev/null || true)" != "$S6RC_NATIVE_STAMP" ]; then
say "s6-rc: building a native compiler ($SKALIBS_VERSION/$EXECLINE_VERSION/$S6_VERSION/$S6RC_VERSION)"
rm -rf work/.s6-native work/.s6-native-src
mkdir -p work/.s6-native-src
for t in "$SKALIBS_TGZ" "$EXECLINE_TGZ" "$S6_TGZ" "$S6RC_TGZ"; do
tar -xzf "$t" -C work/.s6-native-src
done
(
# A subshell for the same reason the cross build has one: nothing
# about this compiler's environment may leak into the rest of the
# build. Note the ABSENCE of --host: this one runs here.
set -e
NSRC="$PWD/work/.s6-native-src"
ND="$PWD/work/.s6-native-stage"
rm -rf "$ND"
export CFLAGS="-Os"
JOBS=$(nproc 2>/dev/null || echo 4)
for pkg in "skalibs-$SKALIBS_VERSION" "execline-$EXECLINE_VERSION" \
"s6-$S6_VERSION" "s6-rc-$S6RC_VERSION"; do
cd "$NSRC/$pkg"
if [ "$pkg" = "skalibs-$SKALIBS_VERSION" ]; then
# No --with-sysdep answers here: configure can compile and RUN
# its probes, because the target is this machine.
./configure --prefix="$MODDIR" \
--disable-shared --enable-static >/dev/null
else
./configure --prefix="$MODDIR" \
--with-sysdeps="$ND$MODDIR/lib/skalibs/sysdeps" \
--with-include="$ND$MODDIR/include" \
--with-lib="$ND$MODDIR/lib" \
--disable-shared --enable-static >/dev/null
fi
make -j"$JOBS" >/dev/null
make install DESTDIR="$ND" >/dev/null
done
)
mkdir -p "$S6RC_NATIVE/bin"
# s6-rc-db comes too: qa/static/test_s6rc_source.py reads the boot graph
# back out of the compiled database rather than out of the source tree,
# and this is the only copy of it that runs on the build host.
for b in s6-rc-compile s6-rc-db; do
cp -f "work/.s6-native-stage$MODDIR/bin/$b" "$S6RC_NATIVE/bin/"
done
rm -rf work/.s6-native-src work/.s6-native-stage
echo "$S6RC_NATIVE_STAMP" > "$S6RC_NATIVE/.version"
else
skip "s6-rc: native compiler already built for $S6RC_NATIVE_STAMP"
fi
# The database goes to etc/s6-rc/compiled/<stamp>, with `current` a symlink to
# it -- skarnet's own advice, so the boot command never changes when the
# database does. Not /etc/s6-rc/, which is s6-rc-init's default and is inside
# the read-only squashfs.
S6RC_DB_NAME="db-$MOD_VER"
rm -rf "$PAYLOAD_DIR/etc/s6-rc/compiled"
mkdir -p "$PAYLOAD_DIR/etc/s6-rc/compiled"
"$S6RC_NATIVE/bin/s6-rc-compile" \
"$PAYLOAD_DIR/etc/s6-rc/compiled/$S6RC_DB_NAME" "$S6RC_SRC" || {
echo " !! s6-rc-compile refused $S6RC_SRC" >&2; exit 1; }
ln -sfn "$S6RC_DB_NAME" "$PAYLOAD_DIR/etc/s6-rc/compiled/current"
# The shebang the compiler baked in is the one assertion that catches a native
# stack built with the wrong prefix, and it is worth making here rather than
# discovering on a printer: this file is generated, so it is the only place
# the build can see what the database will ask the printer to exec.
S6RC_RUNNER="$PAYLOAD_DIR/etc/s6-rc/compiled/$S6RC_DB_NAME/servicedirs/s6rc-oneshot-runner/run"
[ -f "$S6RC_RUNNER" ] || {
echo " !! s6-rc-compile produced no oneshot runner -- has the source tree no oneshots?" >&2
exit 1; }
S6RC_SHEBANG=$(head -1 "$S6RC_RUNNER")
case "$S6RC_SHEBANG" in
"#!$MODDIR/bin/execlineb"*) ;;
*) echo " !! the s6-rc database asks for '$S6RC_SHEBANG', not $MODDIR/bin/execlineb --" >&2
echo " the native stack was configured with the wrong --prefix" >&2
exit 1 ;;
esac
say "s6-rc: database $S6RC_DB_NAME compiled -- $(ls "$S6RC_SRC" | wc -l) definitions"
# -------------------------------------------------- 5c. CPython 3.13 (shipped)
# A second Python for the printer, installed by section 0 as anvil-python plus
# the eighteen anvil-python-* packages. How they are built is in
bin/verify.sh
+7 -5
View File
@@ -132,10 +132,12 @@ if [ -f "$WORK/anvil.tar.xz" ]; then
# 141 -- a race that stayed invisible until the fork klippy tree tripled
# the listing, then intermittently reported Mainsail/Moonraker missing
# from packages that carried both.
grep -q 'init.d/S80ui' <<<"$LIST" && ok "mod payload has the service scripts" \
|| bad "mod payload missing init.d services"
grep -q 'init.d/S70klipper' <<<"$LIST" && ok "mod payload owns Klipper startup" \
|| bad "mod payload missing S70klipper -- Klipper would never start"
# The compiled s6-rc database IS the service set: a payload without it
# boots to a scanner supervising nothing.
grep -q 'etc/s6-rc/compiled/' <<<"$LIST" && ok "mod payload has the compiled s6-rc database" \
|| bad "mod payload has no s6-rc database -- no service would ever start"
grep -q 'servicedirs/klipper/run' <<<"$LIST" && ok "mod payload owns Klipper startup" \
|| bad "mod payload has no klipper service -- Klipper would never start"
# THE KLIPPER CONFIG. printer.base.cfg has a bare
# [include printer.chamber.cfg] and Klipper treats a missing include as
# fatal, so a payload without a chamber config is a printer that comes up
@@ -190,7 +192,7 @@ if [ -f "$WORK/anvil.tar.xz" ]; then
# ff-startup.py, ffscreen.py and ff_mcu_bringup.py all run on it -- so a
# package missing it breaks the boot, not just a future release. klippy is
# not among FF_PYTHON's callers: it stays on FlashForge's 3.8.2, started by
# /usr/prog/klipper/start.sh independently (see init.d/S70klipper).
# the klipper s6-rc service independently.
#
# Two checks, not one, and the second is the one worth having. The
# interpreter can be present and perfectly runnable while _sqlite3 is
docs/building.md
+9 -4
View File
@@ -211,12 +211,17 @@ directory inside that recipe says how they get there. Three names, no fourth
```
pkgs/<recipe>/payload/ staged into the .ipk, laid out as it lands under
$MODDIR: payload/init.d/S60nginx installs as
$MODDIR/init.d/S60nginx and no recipe says so
pkgs/<recipe>/prog/ placed on /usr/prog by patch.sh -- somebody else's
filesystem, so it cannot be in a package of ours
$MODDIR: payload/etc/s6-rc/source/nginx installs as
$MODDIR/etc/s6-rc/source/nginx and no recipe says so
pkgs/<recipe>/payload/prog/
package files the printer reads from somebody else's
filesystem: anvil-link-prog.sh symlinks them to the
absolute paths under /usr/prog that FlashForge's own
scripts open
pkgs/<recipe>/seed/ templated or seeded user state: not a package member,
because a member is overwritten on every upgrade
pkgs/<recipe>/control/ maintainer scripts, copied verbatim into the .ipk's
CONTROL/ -- metadata, not content
installer/ run-pre.sh, run-append.sh -- never files on the
printer at all: patch.sh splices them into the stock
docs/hardware-testing.md
+9 -9
View File
@@ -102,11 +102,11 @@ Understand the safety net first, because FlashForge's UI is gone from the
screen and there is nothing to fall back to:
- ssh and Mainsail do not depend on the screen. `/etc/init.d/S50dropbear` is
stock and runs long before the UI, and `init.d/S60nginx` and
`init.d/S62moonraker` start the web stack independently of it — and of each
other, so `S62moonraker restart` over ssh leaves Mainsail served. They are
your way in when the screen is dark.
- `init.d/S70klipper` owns Klipper startup, because on stock firmware it was
stock and runs long before the UI, and the `nginx` and `moonraker` services
are independent of it — and of each other, so restarting moonraker over ssh
(`s6-svc -r /usr/data/anvil/etc/s6/moonraker`) leaves Mainsail served. They
are your way in when the screen is dark.
- The `klipper` service owns Klipper startup, because on stock firmware it was
`firmwareExe` — not any init script — that ran `/usr/prog/klipper/start.sh`.
Without this the printer would boot to a working screen and be unable to
move.
@@ -254,9 +254,9 @@ machine will make it light up again:
```sh
ssh root@PRINTER
/usr/data/anvil/init.d/S80ui status # what did it choose, and why
# chose "none" -> HelixScreen is not installed
# chose "helix" -> it was started and failed on its own; see its log
s6-svstat /usr/data/anvil/etc/s6/ui # is the UI service up?
# no such servicedir / "unable to open" -> anvil-helixscreen is not installed
# "down" -> it started and failed; see its log
```
`opkg remove anvil-helixscreen` only stops it being started at all — useful to
@@ -271,7 +271,7 @@ interface.
| Symptom | Do this |
|---|---|
| Printer boots, screen blank | ssh in; `/usr/data/anvil/init.d/S80ui status` says whether the UI was even started. No on-device repair — reflash the mod, or flash the stock package to get FlashForge's UI back. ssh, Mainsail and printing are unaffected |
| Printer boots, screen blank | ssh in; `s6-svstat /usr/data/anvil/etc/s6/ui` says whether the UI was even started. No on-device repair — reflash the mod, or flash the stock package to get FlashForge's UI back. ssh, Mainsail and printing are unaffected |
| No ssh, no screen | flash the stock package for your model |
| Recovery stick does not help | try a newer stock FlashForge package for your model |
| Still broken | factory package (`Creator5Pro-factory-*.tgz` **plus** the separate `Creator5Pro-factory.tar.xz` on the same stick; needs 800 MB free) |
docs/how-it-works.md
+41 -36
View File
@@ -38,49 +38,54 @@ Two properties come for free from keeping that name and that place:
## Services
Services are ordinary init.d-style scripts in `/usr/data/anvil/init.d/`, run in
filename order and individually restartable over ssh:
Services are **s6-rc services**, defined in `payload/etc/s6-rc/source/`,
compiled into a database at build time and supervised by `s6-svscan`:
```
S50wifi wlan0 + wpa_supplicant + udhcpc
S60nginx nginx (Mainsail) on :80
S62moonraker moonraker on :7125
S65camera mjpg-streamer on :8080 (nginx proxies it at /webcam/)
S70klipper Klipper
S80ui decides whether the UI runs (is HelixScreen installed?)
mcu-bringup oneshot toolhead boards out of their bootloaders
klipper longrun klippy (depends on mcu-bringup)
wifi oneshot wlan0 + wpa_supplicant + udhcpc
nginx longrun nginx (Mainsail) on :80
moonraker longrun moonraker on :7125
camera longrun mjpg-streamer on :8080 (nginx proxies it at /webcam/)
ff-startup oneshot waits until the printer is usable
(depends on klipper, moonraker)
ui longrun HelixScreen (depends on ff-startup)
ok-all bundle everything except mcu-bringup, which klipper pulls in
```
nginx and moonraker were one script, `S60web`, until they were split. They fail
separately and are debugged separately — nginx not starting is a missing config
or a port already bound, and you know within a second; moonraker not starting
is a component that failed to import, and it takes a background check and a log
tail to even notice. One script had to say `web: stopped` for both, and
`status` reported on two unrelated things at once. Now `S62moonraker restart`
over ssh — the one you run over and over while chasing a moonraker config — no
longer takes the web UI down with it. The numbering keeps the order: nginx
first, because it proxies the other two, so a browser gets a page saying the
backend is not up yet rather than a refused connection.
`firmwareExe` starts `s6-svscan`, runs `s6-rc-init`, and asks for one machine
state — that transition is the whole boot order. There is no `init.d/`
directory and no wrapper scripts: **s6-rc is the command line**.
Two sourced libraries sit beside them, both installed to `/usr/data/anvil` and
neither executable:
```sh
s6-rc -u change moonraker # start it (and its dependencies)
s6-rc -d change moonraker # stop it (and whatever depends on it)
s6-svc -r /usr/data/anvil/etc/s6/moonraker # restart, leaving s6-rc's view intact
s6-svstat /usr/data/anvil/etc/s6/moonraker # is it up, and for how long
s6-rc -a list # what s6-rc believes is up
```
Two things are worth knowing before you type them. `-d change` follows
dependencies, so stopping moonraker also stops `ff-startup` and therefore the
screen — that is correct, there is nothing for a printer UI to show, and
`-u change` brings them back. And `moonraker` and `camera` are the only
services with a `notification-fd`, so only their "up" means *usable* rather
than *forked*; `klipper`'s does not, because klippy's readiness is a state on
moonraker's API rather than something klippy reports.
nginx is deliberately **not** a dependency of moonraker even though it proxies
it: an edge would mean stopping nginx also stopped the API, which is the thing
splitting the old single `S60web` script in two was meant to prevent.
One sourced library sits beside them, installed to `/usr/data/anvil` and not
executable:
* **`anvil-env.sh`** — `PATH`, `LD_LIBRARY_PATH` and `FF_PYTHON`, in the one
place that defines them. `run-append.sh`, `firmwareExe`, `start.sh` and every
service script source it. A private copy per script is how the installer's
check and the boot script came to disagree about the library list, with
moonraker dying on `libpython3.8.so.1.0: cannot open shared object file`.
* **`anvil-service.sh`** — the shape every service script has: `svc_say` /
`svc_warn` for the boot log, `svc_pid_alive` and `svc_proc_alive` for
liveness, `svc_start_daemon` / `svc_stop_daemon` around busybox
`start-stop-daemon`, `svc_detach` for a wait that must not hold the boot up,
and `svc_dispatch` for the `start|stop|restart|status` block. The scripts
were written months apart and had arrived at four different answers to "is it
still running?" and two to `restart`; none of those differences were
decisions. The library also carries the two corrections busybox needs —
`start-stop-daemon -K` returns before the process is dead, and does not
remove the pidfile, which is how `restart` raced its own start and how
`status` came to report a recycled pid as a running service. A script that
cannot find this file says so and exits rather than half-starting.
Everything the mod installs lives under `/usr/data/anvil` on the **data**
partition, which a FlashForge OTA cannot delete. The software component itself
@@ -104,7 +109,7 @@ payload unpacks to `/usr/data/anvil`, and that is the whole installation —
there is no separate Moonraker step that can fail on its own. Nothing is
written to `/usr/prog`: FlashForge's tree at `/usr/prog/moonraker/moonraker/`
is left exactly where it is and simply never used again, and
`S62moonraker` starts ours by absolute path with no fallback.
the moonraker service starts ours by absolute path with no fallback.
An earlier release copied the same tree over `/usr/prog/moonraker` as well.
That is gone. It put a second, byte-identical Moonraker on the one partition
@@ -116,7 +121,7 @@ Both things thought to require that location turned out not to: the
`moonraker-env` virtualenv beside it was never on `sys.path` (checked by
running the printer's own interpreter against the real image), and
`moonrakerDaemon`, the one thing that did exec that tree by absolute path, is
never invoked — `S62moonraker` starts the server itself.
never invoked — the moonraker service starts the server itself.
**FlashForge's python 3.8.2 is not what this runs on any more.** `FF_PYTHON`
in `anvil-env.sh` names a CPython 3.13 of our own, cross-built for mipsel by
@@ -124,7 +129,7 @@ in `anvil-env.sh` names a CPython 3.13 of our own, cross-built for mipsel by
`$MODDIR/lib/python3.13/site-packages` — eighteen packages, one `pkgs/3rdparty/python-*`
recipe and one `.ipk` each — and libsodium in `$MODDIR/lib`; none of it
borrowed from `/usr/prog`. Measured through the
real boot path (S40s6's scandir, `S62moonraker`, readiness gating on `:7125`
real boot path (the scandir, the moonraker service, readiness gating on `:7125`
actually listening, a `kill -9` respawn, a stop that stays stopped) in
`test/integration/printer/case-moonraker313-s6.sh`. klippy is not part of
this: it stays on FlashForge's 3.8.2, started independently by
@@ -214,7 +219,7 @@ One thing it does not cover, worth knowing before you rely on it:
* **Moonraker.** The mod's Moonraker lives on the data partition at
`/usr/data/anvil/moonraker`, and it is started by
`/usr/data/anvil/init.d/S62moonraker` — which a stock flash removes along
the mod's own moonraker service — which a stock flash removes along
with the rest of the mod's boot path. So a stock reflash goes back to
FlashForge's own 2022 tree at `/usr/prog/moonraker/moonraker/`, which was
never touched and is still exactly what the factory image shipped. The mod's
docs/notes/80-s6-migration.md
+461
View File
@@ -154,6 +154,15 @@ directory under `etc/s6/` with a `run` script, and a `down` file when its
people's hands; the script becomes a thin wrapper over `s6-svc`. Changing the
supervisor is not a reason to change the interface.
> **REVERSED IN PHASE 8, deliberately.** The rule was right while a wrapper
> still did something -- prerequisites, the `MOD_*` gate, a liveness check.
> Once every one of those moved into the `run` script (where it has to live
> anyway, because s6 restarts a service without going near a shell script),
> the wrapper was verb translation and nothing else, in a directory whose
> `S<NN>` numbering encoded an ordering that no longer existed. `init.d/` is
> deleted and `s6-rc` is the interface. The cost is real and is paid once:
> `S60nginx restart` is gone from people's hands and from the docs.
Camera is the interesting one: its 30-second `/dev/video0` poll becomes a
readiness notification, so whatever waits for the camera waits for *ready*
rather than for *forked*.
@@ -328,6 +337,451 @@ mod that depends on no FlashForge version string, so a stock OTA cannot break a
modded printer between our releases, and a single place to look when something
does not start. That is worth having. "Survives a flash" is not the reason.
## Phase 8 -- hand the boot to s6-rc
**Numbered last because it was written last, not because it must run last.**
Its hard predecessors are phases 1, 3, 4 and 5: it needs the manifest (shipped
-- `payload/run-append.sh` is on the manifest path today), a scanner (shipped
-- `payload/init.d/S40s6`), and the three services already living in the
scandir. It needs nothing from phase 6 and nothing from phase 7, and phase 7
does not need it. It can go before either, after either, or between them.
**Klipper came in after all, and phase 7 did not get bigger.** The plan said
Klipper was explicitly out of this phase. It is in it: an ordinary longrun
whose `run` script is `klipperDaemon`'s command line with the `-b` removed,
exactly as the fact list above predicted it would be. What phase 7 was actually
about -- shipping the klippy tree into `$MODDIR`, owning the Python, moving
`/usr/data/config` -- is untouched and still ahead. Supervising a program is
not the same as owning where it lives, and conflating the two is what kept
Klipper out of a phase it belonged in.
**What it is for.** `firmwareExe` runs seven `init.d/S*` scripts one at a time
in the foreground and that loop IS the boot order (`payload/firmwareExe` lines
63-69). Three of those seven are already thin -- `S60nginx`, `S62moonraker` and
`S65camera` are `svc_s6_*` wrappers over a supervisor that is already running.
What is left in the loop is not supervision, it is *sequencing*, and it is
sequencing expressed as filenames. This phase moves the sequencing into a
compiled s6-rc database, so that `firmwareExe` asks for a machine STATE
(`s6-rc -u change ok-all`) instead of running seven programs and hoping.
### The graph, and what its edges do NOT promise
**There is a real dependency graph now, and it is small.** Written out, with
the reason for every edge that exists and every one that does not:
mcu-bringup oneshot no dependencies
klipper longrun dependencies.d/mcu-bringup
wifi oneshot no dependencies
nginx longrun no dependencies
moonraker longrun no dependencies
camera longrun no dependencies
ff-startup oneshot dependencies.d/{klipper,moonraker}
ui longrun dependencies.d/ff-startup
ok-all bundle contents.d/{wifi,nginx,moonraker,camera,klipper,
ff-startup,ui}
`mcu-bringup` is not in the bundle: it is pulled in as klipper's dependency,
which is the point of having one.
**The three edges are ORDERING, and only one of them is also readiness.** This
distinction is the whole honesty of the section and it is easy to lose:
* `s6-rc` counts a **longrun** up as soon as it is forked, *unless* its
servicedir carries a `notification-fd`, in which case it waits for the
service to say it is ready.
* `moonraker` and `camera` have one. Their readiness is real: `moonraker` is
counted up when it has BOUND :7125, the camera when the streamer has bound
:8080 with a device behind it.
* `klipper` does **not**. klippy's "ready" is a state on Moonraker's API --
there is no "Printer is ready" line even in `printer.log`, which is why
`S70klipper` used to tail for `Stats <n>:` -- so `ff-startup` depending on
`klipper` means "after klippy was LAUNCHED", not "after klippy is usable".
`bin/ff-startup.py` therefore keeps its own poll of `klippy_state` and its
own retry loop. Giving klipper a real `notification-fd` means teaching
klippy to write to one, and that is a separate change; it is not attempted
here and it must not be claimed.
So `ui` depending on `ff-startup` genuinely does mean "the screen appears when
the printer is ready", because `ff-startup` is a oneshot and a oneshot is not
done until its `up` has returned -- and its `up` is the program that waits.
That is the edge the phase was worth doing for.
**nginx before moonraker is still NOT an edge, for the reason it never was.**
`S60nginx`'s header argues that nginx should start first because it proxies
both moonraker and the camera, and a browser then gets a page saying the
backend is not answering rather than a refused connection. That is a
PREFERENCE, not a dependency, and encoding it as one has a price: a
`dependencies.d/nginx` under moonraker means `s6-rc -d change nginx` also stops
moonraker, so `S60nginx stop` would take the API down with the UI -- the exact
thing splitting `S60web` into two scripts was done to prevent. So the edge is
not written, s6-rc brings those two up in parallel, and the cost is a second or
two in which a reloading browser gets a refused connection instead of a 502.
**What the edges cost, said plainly.** `s6-rc -d change moonraker` now also
stops `ff-startup` and therefore the UI, because both depend on it. That is
correct -- there is nothing for a printer UI to show when there is no printer
-- but it is a behaviour change from the days when each `S*` script stopped one
thing, and both `S62moonraker` and `S70klipper` say so in their `stop()`.
**And a boot that waits.** The transition is what `firmwareExe` blocks on, so
readiness that used to be reported by a detached background job is now
something the boot pays for: moonraker's up to `timeout-up` = 120s, the
camera's 40s on a printer that has no camera, `ff-startup`'s up to 300s. Every
one of those services has a `timeout-up`, and that is not tidiness -- without
one, a camera that never appears would hold the transition open for ever and
the UI would never start.
### Does s6-rc need execline? Yes, and here is the bill
Phase 2 says "execline is **not** needed: `run` scripts are plain `#!/bin/sh`".
That survives for longruns and fails for everything else, and both halves were
read out of the s6-rc 0.7.0.0 source rather than assumed.
* **Longruns keep plain `#!/bin/sh`.** `s6-rc-compile` only wraps a `run`
script in an execlineb wrapper when the service is *pipelined*
(`src/s6-rc/s6-rc-compile.c` lines 1375-1379, guarded on `ispipelined`).
Pipelines are s6-rc's logger mechanism and we ship no loggers, so our three
existing `run` scripts are copied verbatim. Nothing about phase 4 or 5 has to
be rewritten.
* **execline is a build-time AND run-time requirement anyway.** skarnet says so
outright on <https://skarnet.org/software/s6-rc/>, `package/deps-build` lists
`execline 2.9.9.2`, and `s6-rc-compile` links `-lexecline`
(`package/deps.mak` line 136). The runtime half is concrete rather than
theoretical: `write_oneshot_runner` is called unconditionally
(`s6-rc-compile.c` line 1592) and emits a servicedir whose `run` begins
`#!<execlinebprefix>execlineb -s1` and then execs `fdmove`,
`s6-ipcserver-socketbinder`, `s6-ipcserverd`, `s6-ipcserver-access`,
`s6-sudod` and `libexec/s6-rc-oneshot-run` (lines 1083-1092). Every oneshot's
`up` runs through that service. We are adding a oneshot, so we are adding
execline.
* **The oneshot `up` file itself does not have to be execline.** It is lexed by
execlineb at compile time and stored as an argv, and skarnet documents
`/bin/sh -c "script"` and "just call a script stored somewhere else" as
supported shapes. Every `up` we write will be the latter, so no execline
command beyond `execlineb` and `fdmove` is ever exec'd.
The bill, against the ~813KB + 116KB s6 already costs: two more pinned tarballs
with sha256s (`execline`, `s6-rc`), two more cross-builds in `bin/patch.sh`
section 5b -- the same toolchain, the same wrapper discipline, the same cache
stamp -- and more entries in the ABI gate, whose expected-count assertion
(`bin/patch.sh`, the `S6_ELF` check) has to move with them.
**The binary set was MEASURED, and it is bigger than reading the oneshot
runner alone suggests.** The estimate above -- four new s6 names and two
execline binaries -- was made from `write_oneshot_runner`. `s6-rc-compile` also
emits an `s6rc-fdholder` servicedir, and that one execs `pipeline`, `if`,
`forstdin`, `exit`, `redirfd`, `s6-ipcclient`, `s6-fdholder-daemon`, `s6-svc`
and `libexec/s6-rc-fdholder-filler`. Read off the generated `servicedirs/*/run`
of a real compile, the additions are **seven** execline binaries (`execlineb`,
`fdmove`, `pipeline`, `if`, `forstdin`, `exit`, `redirfd`), six s6 binaries we
build but do not ship today (`s6-ipcserver-socketbinder`, `s6-ipcserverd`,
`s6-ipcserver-access`, `s6-sudod`, `s6-fdholder-daemon`, `s6-ipcclient`),
`s6-rc`, `s6-rc-init` and `s6-rc-db`, and two `libexec` helpers
(`s6-rc-oneshot-run`, `s6-rc-fdholder-filler`). Stripped, for this target:
today, the s6 subset + s6-ftrigrd 1434 KB
added by s6-rc 1309 KB
So s6-rc roughly DOUBLES what supervision costs on the data partition, and it
does so to buy a boot order that is three real edges wide (see the section
above) and was previously spelled as filenames. That is the honest trade and it should be read next to the
free-space unknown phase 6 records: nobody has ever run `df` on a real printer.
`s6-rc-compile` (131KB) is host-side only and is not shipped.
**Both new packages must be configured `--prefix=/usr/data/anvil`, for a reason
one level worse than the `s6-ftrigrd` trap.** execline's `--shebangdir`
defaults to `$bindir` and is baked into `EXECLINE_SHEBANGPREFIX`
(`execline-2.9.9.2/configure` lines 152, 587); `s6-rc-compile` pastes that
string into the `#!` line of the oneshot runner it generates. So the prefix
that ends up INSIDE the shipped database is a property of the execline that the
*compiler* was linked against, not of the execline we ship. Get it wrong and
every oneshot fails at exec with ENOENT while every longrun keeps working --
late and partial, exactly as `tools/supervisor/README.md` describes for the
waiting verbs.
### Where the database lives, and when it is compiled
**Compiled on the build host, in `bin/patch.sh`, and shipped.** The database
files are architecture-neutral by construction: `s6-rc-compile` writes every
integer with `uint32_pack_big` (`s6-rc-compile.c` lines 1037-1042, 1199, 1214,
1482) and the runtime reads them back with `uint32_unpack_big`
(`src/libs6rc/s6rc_db_read_uint32.c` line 12), so nothing native-word-sized or
host-endian reaches the file; `resolve.cdb` is djb's fixed cdb format. What is
NOT neutral is the `servicedirs/` subdirectory, because of the shebang above --
which is a configure problem, not a portability problem, and is solved by
building s6-rc twice from one tarball: once for the host to get a
`s6-rc-compile` configured with the printer's prefixes, once for mipsel to get
`s6-rc`, `s6-rc-init`, `s6-rc-db` and `libexec/s6-rc-oneshot-run`. That is the
shape section 5c already uses for CPython, where a throwaway x86-64 3.13.7 is
built from the same tarball as the shipped mipsel one.
Compiling on the printer at install time was the alternative and is rejected:
it means shipping `s6-rc-compile` plus the execline lexer plus the whole source
tree, and it adds a way for `run-append.sh` to fail that ends in a printer with
no services and a log nobody will read. The database is a build artefact. Build
artefacts are built where the build is.
**Paths.** The compiled database ships at
`$MODDIR/etc/s6-rc/compiled/<version>` with
`$MODDIR/etc/s6-rc/compiled/current` a symlink to it -- skarnet's own advice on
the `s6-rc-init` page, so the boot command never has to change when the
database does. It is under `$MODDIR/etc/`, not `/etc/s6-rc/`, which is
`s6-rc-init`'s default and is unreachable: `/etc` is inside the read-only
squashfs (`docs/printer-replica.md`, the mount table). Configure with
`--bootdb=/usr/data/anvil/etc/s6-rc/compiled/current` so a hand-run
`s6-rc-init` over ssh is right too, and pass `-c` explicitly anyway.
**The live directory is `/run/s6-rc`**, which is s6-rc's own default.
`s6-rc-init` needs it writable and skarnet recommends a RAM filesystem; `/run`
is tmpfs on this machine (`docs/printer-replica.md`, mount table, copied from
`/etc/fstab` and `/etc/init.d/S21mount_mmc_ext4`). That gives the reboot answer
for free: tmpfs comes up empty, so `s6-rc-init`'s requirement that the live
directory "should not exist prior" holds by construction on every boot, and
nothing has to clean up after a crash. Across an UPGRADE the live directory
survives, still pointing at a compiled database the manifest pass is about to
delete -- see below.
### What starts the tree, and what happens when it dies
**This is where the plan changed, and it changed for the better.** Phase 3 said
nothing supervises the supervisor, that a respawn loop in `ash` would hide a
real fault, and that the honest fix would be inittab-style respawn. There is
one, and it was already on the printer.
`firmwareExe` starts `s6-svscan` AS ITS OWN CHILD, runs `s6-rc-init`, issues
one `s6-rc -u change` for the boot set, and then blocks in `wait
"$SCANNER_PID"`. That `wait` is what holds the foreground for the stock
`app_startup.sh` watchdog, which greps `ps` for the name `firmwareExe` five
seconds after launch -- the job the old `while true; do sleep 3600; done` tail
did, now done by waiting on something real.
When the scanner dies, `wait` returns, `firmwareExe` exits, and
`app_startup.sh` -- FlashForge's own stock init, which the mod never patches --
re-execs it. The whole tree comes back. That is not a hand-rolled supervisor:
there is no loop, nothing is retried in place, and nothing is hidden.
Three details are load-bearing:
* **The scanner must be `firmwareExe`'s own child.** `wait` only works on a
process the calling shell forked. Backgrounding it inside `S40s6 start`, as
phase 3 did, puts it in a subshell of another program. `S40s6` keeps its own
copy of the sequence for the ssh path, and every step of it is a no-op when
the step has already happened.
* **Not `exec s6-svscan`.** The watchdog greps for the name; `exec` would make
`$0` `s6-svscan`, and busybox `ash` has no `exec -a`. The shell also has to
run `s6-rc-init` *after* the scanner is up, which an `exec` forecloses.
* **`wait "$SCANNER_PID"`, not bare `wait`.** Services and `ff-startup` leave
background children around; a bare `wait` would return on the wrong one.
What it does NOT buy: it is one restart of EVERYTHING, not a targeted restart
of the scanner. klippy goes down with it, mid-print. A printer whose scanner
dies twice has a real fault -- realistically the `-D_FILE_OFFSET_BITS=64` build
trap -- and the second time round this looks like a boot loop, which is what a
boot loop is for.
### `init.d/` is deleted
Phase 4 promised the `S*` scripts would survive as thin wrappers. They did not,
and the reason is worth stating because it is the same argument the `run`
scripts already make: a service that only works when a shell script happened to
run first is not supervised, it is lucky. Every prerequisite a wrapper did --
the `mkdir`, the leftover sweep, the binary check -- has to be in the `run`
script anyway, because s6 restarts a service without going near a shell. Once
they moved, the wrappers were verb translation in a directory whose `S<NN>`
numbering encoded an ordering that no longer existed.
So `s6-rc` is the interface. `s6-rc -u change nginx`, `s6-rc -d change nginx`,
`s6-svc -r $SCANDIR/nginx`, `s6-svstat $SCANDIR/nginx`. `S60nginx restart` is
gone from people's hands and from `docs/`.
* **`payload/anvil-service.sh` is deleted too.** With the wrappers gone its
only caller was `firmwareExe`, and a shared library for one consumer is the
redundancy the wrappers were. What `firmwareExe` uses is inlined there: the
paths, the `PATH` prepend, the scanner fork, `s6-rc-init`, the boot set.
* **The tree-starter exists once**, in `firmwareExe`. There is no `S40s6` and
no replacement helper. The ssh recovery path is plain `s6-rc` commands.
* **`bin/ff-startup.py` is two services**, `mcu-bringup` and `ff-startup`. A
compiled database cannot read `anvil.conf`, so `MOD_STARTUP`, `MOD_IMPORT`,
`MOD_STARTUP_TIMEOUT` and `MOD_KLIPPER_TRIES` become argv inside the two
`up` files, which source `anvil.conf` at runtime.
* **wifi is a oneshot**, not a "do the work then sleep forever" longrun -- s6
would respawn a longrun that had finished. Its `up` returns when the
supplicant is launched, not when there is an address, which is safe only
because NOTHING DEPENDS ON WIFI. A oneshot that returns early lies to its
dependents.
* **klipper loses its retry loop and `force-start`** -- `ff-startup` owns the
retry, with moonraker to ask instead of `printer.log` to tail, and nothing
stands aside from anything any more. It restarts with `s6-svc -wr -t` rather
than two s6-rc transitions, because `ff-startup` calls it from inside one and
`s6-rc` takes an exclusive lock on the live directory.
* **`klipperDaemon` is replaced with a no-op `start`.** It is FlashForge's
script and off-repo callers remain -- stock `start.sh`, the UI's restart
path, habits over ssh -- and every one of them would fork a second,
unsupervised klippy beside the supervised one. `stop`, `restart` and `status`
redirect at the service rather than reimplementing pidfile semantics nothing
writes any more.
* **The UI is a supervised longrun.** The plan said it could not be: the UI had
to hold `firmwareExe`'s foreground for the watchdog. `firmwareExe` holds its
own foreground on `wait` now, so the foreground is not available to lend and
`$MODDIR/.ui-choice` is deleted rather than left unread.
**BLOCKING UNKNOWN: does `helix-screen` run under `s6-supervise`?** Nothing has
proved it. A supervised process gets the descriptors `s6-supervise` hands it,
no controlling terminal, and a session it did not create; until now the UI
inherited `firmwareExe`'s. The environment half is easy and is in
`etc/s6-rc/source/ui/run` -- `HELIX_DATA_DIR`, `HELIX_LOG_DEST`,
`HELIX_LOG_FILE`, `HELIX_DISABLE_AUTO_UPDATES`, exactly what `firmwareExe`
exported. A tty is not something a `run` script can conjure. This has to be
settled on the replica before the change ships; if it turns out a terminal is
needed, the answer is s6's own tty handling or a small wrapper, NOT a return to
the foreground, which no longer exists.
`firmwareExe`'s loop over `$MODDIR/init.d/S*` is gone and nothing replaces it:
the body is the scanner, `s6-rc-init`, one `s6-rc -u change`, and `wait`.
### The `MOD_WEB` / `MOD_CAM` gate survives, and gets better
**The three `down` files are deleted, and the reason is not the one the manual
gives.** MEASURED: `s6-rc-compile` does not refuse a `down` file in a definition
directory -- it ACCEPTS it and SILENTLY DROPS it, producing a byte-identical
servicedir. So "not started by default" in s6-rc is not a file at all; it is
membership of the `ok-all` bundle. The mechanism whose long argument was
written inside `payload/etc/s6/nginx/down` stops existing either way.
What replaces it is strictly better, and it is worth being explicit because the
`down` file's own text spends thirty lines on this exact point: the gate has to
be a RUNTIME read of `anvil.conf`, because the payload is built once and
`anvil.conf` is edited over ssh. Under s6-rc the gate is the ARGUMENT LIST of
`s6-rc -u change` -- `svc_rc_enabled` in `anvil-service.sh` reads the switches
at runtime, `svc_rc_boot_set` expands the `ok-all` bundle with `s6-rc-db
contents` and drops what is switched off, and both `firmwareExe` and each
`S*` script's `start()` ask the same function, so they cannot disagree. `MOD_CAM=0` means camera is never named, so it never starts;
`MOD_WEB=0` means neither nginx nor moonraker is named. A printer upgrading
with `MOD_CAM=0` already set needs no action at all and behaves identically.
That is the whole upgrade path for the flags, and it is why they are kept
rather than dropped.
The one thing genuinely lost is the corner case the `down` file's own "THE
COST" paragraph describes -- and it is lost by being FIXED. There, a killed
`s6-supervise` was replaced by a fresh one that re-read `down` and left the
service stopped; under s6-rc the live servicedir's `down` file is managed
against s6-rc's current state, so the replacement supervisor reads what the
machine is supposed to be doing. The reasoning in the deleted file does not
disappear: it moves to `svc_rc_enabled` and `svc_rc_boot_set` in
`anvil-service.sh`, which is where the decision now lives.
### Upgrade
Phase 1 is already shipped, which is what makes this section short. The
compiled database, the source definitions and everything else are staged under
`$MOD_PAYLOAD` and therefore named in `.install-manifest` (`bin/patch.sh`
section 10b), so `run-append.sh`'s manifest pass deletes the previous
database and extracts the new one with no new code. **`s6-rc-update` is
deliberately not used**: it exists to swap a compiled database under a live
machine, and a flash ends in a reboot. Reaching for it would mean the installer
mutating a running supervision tree on a machine that is about to restart
anyway.
Two things do need writing.
* **The scandir MUST be swept, and the manifest cannot do it.** MEASURED:
`s6-rc-init` creates one symlink per service in the scandir and fails
outright -- `unable to supervise service directories ...: File exists` -- if
a name is taken. A printer upgrading from the pre-s6-rc payload has `nginx`,
`moonraker` and `camera` sitting there as real directories. The manifest pass
deletes every FILE it listed and then `rmdir`s the directories, but
`s6-supervise` created `supervise/` and `event/` inside each of them at
RUNTIME, and no manifest can list a file the previous install did not write
-- so the `rmdir` correctly refuses and leaves exactly the name `s6-rc-init`
collides with. `run-append.sh` therefore does `rm -rf $MODDIR/etc/s6` and
recreates it, after extraction, plus `rm -rf /run/s6-rc` for the person who
runs `sh run.sh` over ssh instead of rebooting.
* **The legacy sweeps stay.** `S62moonraker`'s `/run/moonraker.pid` cleanup
(lines 369-388) and `S65camera`'s `LEGACY_PIDFILE` block (lines 225-238) are
the same class of problem one release further on, and this phase adds a third
instance of it: a printer that boots the new payload while an old
`s6-supervise` from the pre-s6-rc scandir is still alive has two supervisors
fighting for `:80`. The reboot settles it; the ssh path has to sweep it.
### SETTLED: both of this phase's blocking unknowns were measured
Run on 2026-08-28, in the replica, against the real cross-built stack --
skalibs 2.15.1.0, execline 2.9.9.2, s6 2.15.1.0 and s6-rc 0.7.0.0, all four
built twice from ONE set of tarballs (the skalibs and s6 sha256s are the ones
already pinned in `versions.env`): natively, to get an `s6-rc-compile` that
runs on the build host, and for `mipsel-buildroot-linux-musl` with the Bootlin
toolchain `versions.env` pins, both configured `--prefix=/usr/data/anvil`.
**A host-compiled database IS accepted, byte for byte.** The same two-service
source tree -- one oneshot, one longrun, the shapes `S50wifi` and `S60nginx`
become -- was compiled by the host `s6-rc-compile` and by the target one under
qemu. `db`, `n` and `resolve.cdb` are identical (`db` md5
`80f5261437191fa591744ada6087c6f7` both ways) and `diff -r` reports the
generated `servicedirs/` trees identical too. The endianness argument holds.
Then the HOST-compiled database was run: `s6-rc-init` returned 0, `s6-rc -u
change ok-all` returned 0 with no warnings, the longrun came up with a
`supervise/` directory, and the oneshot's `up` actually executed -- its stamp
file was written -- which is the half that goes through execline. `s6-svscan`'s
log stayed empty. **Nothing about the database needs to be compiled on the
printer, and `s6-rc-update` is not needed to make one usable.**
**`/run` is a tmpfs and is writable.** `rw,seclabel,relatime,inode64`, no
`noexec`, and a write to it succeeded. Use `/run/s6-rc` as the live directory
as designed; the `/tmp/s6-rc` fallback is not needed. Note this was measured in
the replica, which models the machine but is not one -- the replica does mount
the chroot root READ-ONLY (`overlay ... (ro)`), which is the property that
matters here, and only `/usr/data`, `/tmp`, `/run` and `/dev` are writable.
**Two traps found by running it, both of which would have cost a day on a
printer.** Neither is a reason not to do the phase; both are things the gate
must assert.
* **`s6-svscan` resolves `s6-supervise` through `PATH`.** Started from a shell
without `$MODDIR/bin` on `PATH`, the scanner answers `s6-svscanctl -a`
perfectly and then fails every spawn with `unable to spawn s6-supervise for
X: No such file or directory`, forever, in a log nobody is reading.
`s6-rc-init` hangs rather than fails while this is true. This is the same
class as the `s6-ftrigrd` trap and `payload/anvil-env.sh` lines 160-174
already documents it for `s6-svlisten` -- but it applies to the scanner
itself, and `S40s6` must not assume its caller's environment.
* **`s6-rc-init` creates one symlink per service in the scandir it is given,
and collides with what is already there.** Pointed at a live
`$MODDIR/etc/s6` that already holds `nginx`, `moonraker` and `camera` from
phases 4 and 5, it fails with `unable to supervise service directories ...:
File exists`. So the cutover is not additive: the phase-4/5 servicedirs must
be REMOVED from the scandir in the same change that introduces the database,
and an upgrade that leaves them behind produces a printer whose supervision
tree never initialises. This is the sharpest edge in the phase.
*Files:* `versions.env` and `bin/fetch-assets.sh` (execline, s6-rc),
`bin/patch.sh` (5b grows a host and a target s6-rc, `S6_BINS` grows by the
fifteen names the measurement above lists, the ABI gate count moves, a new step
compiles the database),
`payload/etc/s6-rc/source/` (new: the definition directories),
`payload/etc/s6/*/down` (deleted), `payload/anvil-service.sh` (`svc_rc_*`),
`payload/init.d/S40s6`, `S50wifi`, `S60nginx`, `S62moonraker`, `S65camera`,
`payload/firmwareExe` (the loop goes).
*Gate:* `qa/replica/test_s6rc.py`, in `qa/` rather than as a new
`case-*.sh` -- this phase's failures are per-assertion and
`docs/qa-migration.md` is explicit about where an assertion belongs. Against a
machine the real installer produced, it must show: the database the BUILD
produced is the one running (`s6-rc-db -c .../current list all` from inside the
chroot); `s6-rc -a list` reports the four services after the boot; with
`MOD_CAM=0` it reports three and there is no `mjpg_streamer`, which is the
negative control; `S60nginx stop` leaves moonraker answering, which is the
property the `S60web` split bought and the property a wrong dependency edge
would silently destroy; `kill -9` on nginx is respawned; the wifi oneshot
reports up and the init sequence still returned promptly; and `S40s6 start`
twice in a row does not fail. Two more, one per trap the experiment found:
`S40s6` brings the tree up when invoked from an environment with an empty
`PATH` (the scanner-spawn trap, which otherwise hangs `s6-rc-init` rather than
failing it), and an upgrade FROM a phase-4/5 payload -- whose scandir still
holds `nginx`, `moonraker` and `camera` -- initialises rather than colliding on
`File exists`. Then one negative control on the phase itself:
move the compiled database aside and assert the boot says so with `!!` and that
Klipper still comes up, because Klipper is not in the database and must not
care.
## How this gets implemented
Phases 1, 2 and 3 are sequential and all touch `run-append.sh`, `patch.sh` and
@@ -340,6 +794,13 @@ script, one service directory and one gate, each in its own worktree.
Phase 5 is one worker again: it touches user data and deserves undivided
attention.
Phase 8 is one worker as well, and in two sittings with a review between them:
the portability probe first (host-compiled database, run in the replica,
nothing else touched), then everything else. It rewrites the boot path and
touches `patch.sh`, `anvil-service.sh` and five init scripts at once, so
fanning it out produces merge conflicts in exactly the files a bad merge is
hardest to notice in.
Every agent follows the rule this project already has -- install the payload
and run the real tools in the replica, assert on behaviour, always with a
negative control. Never grep a shipped script to decide whether it works.
docs/qa-migration.md
+54
View File
@@ -237,6 +237,60 @@ readiness assertion pass by the absence of contradiction. Neither failure looks
like a failure — they look like a green suite. So they are tested against
captured output, in the static lane, with no docker involved.
## What phase 8 did to both suites
`docs/notes/80-s6-migration.md` phase 8 deleted `payload/init.d/` and
`payload/anvil-service.sh` and made the boot a compiled s6-rc database. That is
not a refactor the suites can be patched through: a large part of what they
assert no longer exists to be asserted.
**Retired.** `qa/replica/test_services.py` is deleted. It was the port of
`case-services.sh` and its subject was the wrapper contract -- seven `S*`
scripts, each loading a shared library, answering `status`, naming itself, and
rejecting an unknown verb. There are no wrapper scripts. Three of its
assertions survived the move and live in `qa/replica/test_s6rc.py`: the scanner
is the cross-built ELF we shipped, its log is empty, and every `run` in the
scandir is executable. The rest describe a machine that cannot be built any
more.
**Replaced.** The contract that a `run` script is startable and a service comes
up is now asked two ways, and the split is deliberate:
| lane | file | asks |
|---|---|---|
| static | `test_s6rc_source.py` | what the database SAYS -- graph, bundle, timeouts, the generated shebang, and the shell inside each oneshot |
| replica | `test_s6rc.py` | what the printer DOES -- init, one transition, a killed daemon coming back |
The static half needs no docker and runs in about a second, which is where a
boot-order mistake should be caught. It compiles the real source tree with the
native `s6-rc-compile` that `bin/patch.sh` section 5b-2 builds, and reads the
answers back out with `s6-rc-db` -- never out of the source tree, because the
compiler is free to reject, rewrite, or silently drop what it was given.
`down` is the one that bit us: `s6-rc-compile` accepts a `down` file in a
definition directory and discards it, producing a byte-identical servicedir.
### The old suite is not green and cannot be made green by editing
These `test/integration/printer/case-*.sh` gates assert deleted architecture.
None of them can pass against a package built from this tree, and none of them
can be *run* on a machine without a stock FlashForge package, which is why they
have been left alone rather than edited blind:
| case | what it asserts | where that lives now |
|---|---|---|
| `case-services.sh` | the seven-script wrapper contract, `force-start` | nowhere -- the contract is deleted |
| `case-nginx.sh`, `case-moonraker.sh` | the `MOD_S6=0` unsupervised fallback | nowhere -- the fallback and the flag are deleted |
| `case-priority.sh` | `svc_start_daemon` niceness | unasserted; the surviving path is `-N` inside `etc/s6-rc/source/klipper/run` |
| `test_startup.py` | `FF_SKIP_MCU_BRINGUP`, `--start-sh`, `--daemon` | `ff-startup.py` takes `--only-bringup` / `--no-bringup` now |
| `gates.py` | gates that drive the above | -- |
Retiring them is a coverage decision and it should be made against real
output, not against a reading of the scripts. The next person with a stock
package should: `make build`, run `make test` and `make qa-replica`, and retire
each red gate whose subject is genuinely gone -- moving anything that still
matters into `qa/` first. The rule at the top of this document has not changed;
what changed is that for some of these there is nothing left to replace.
## Moving the rest
**A `case-*.sh` is deleted only when its replacement here has been green in the
docs/testing.md
+11 -8
View File
@@ -2,15 +2,18 @@
> **There is a second suite.** `qa/` is the replacement for everything below,
> built beside it rather than inside it, and both run in CI. It is one
> framework (pytest), and its results are per-assertion rather than per-case:
> `case-services.sh`'s 360 lines and single exit code are 43 named tests there.
> `test/` is still the release gate and nothing has been deleted. See
> **[qa-migration.md](qa-migration.md)** for what moves when, and run
> `make qa` / `make qa-static` / `make qa-replica`.
> framework (pytest), and its results are per-assertion rather than per-case.
> Run `make qa` / `make qa-static` / `make qa-replica`.
>
> The rest of this page describes the suite being migrated away from. It is
> current and accurate for everything `qa/` has not yet replaced, which is
> most of it.
> **Several gates below are now RED and cannot be fixed by editing them.**
> Phase 8 of [notes/80-s6-migration.md](notes/80-s6-migration.md) deleted
> `payload/init.d/` and `payload/anvil-service.sh`, so the cases that assert
> the `S*` wrapper contract or the `MOD_S6=0` fallback are asserting something
> that no longer exists. [qa-migration.md](qa-migration.md) lists which, what
> each one asserted, and where that assertion lives now. Retiring them needs a
> stock package and a real run -- not a reading.
>
> The rest of this page describes the suite being migrated away from.
Everything lives in `test/integration`, and `test/run-tests.py` runs it. There
is still a real seam inside it, though — what needs the proprietary firmware
docs/toolchange.md
+3 -3
View File
@@ -32,11 +32,11 @@ documented in the file headers.
| [`pkgs/anvil-core/payload/config/ff-tool-offset.cfg`](../pkgs/anvil-core/payload/config/ff-tool-offset.cfg) | `/usr/data/config/` | `[ff_tool_offset]` — probe geometry and guards for the calibration commands |
| [`pkgs/anvil-core/payload/config/ff-legacy.cfg`](../pkgs/anvil-core/payload/config/ff-legacy.cfg) | `/usr/data/config/` | `[ff_legacy]` — stays included permanently; declares the section and, optionally, `firmware_config_dir` |
| [`pkgs/anvil-core/payload/bin/ffscreen.py`](../pkgs/anvil-core/payload/bin/ffscreen.py) | `/usr/data/anvil/bin/` | A few lines of text and a progress bar drawn straight onto `/dev/fb0`, geometry read from sysfs. The framebuffer is **portrait 480×800@32** and the panel is that buffer turned 90° clockwise (landscape 800×480) — established from FlashForge's own `/usr/prog/start.img`, 1536000 bytes, which only decodes into a picture read that way. Drawing is done in landscape coordinates and each rectangle is rotated on the way into the buffer, so it costs arithmetic per rectangle and nothing per pixel. All of the first boot happens before HelixScreen starts, so without it the panel is black for the longest wait of the install — which reads as a brick and invites a power cut mid-`SAVE_CONFIG`. `make boot-screen` renders every frame to PNG on the host; `make boot-screen-sim` renders the same frames inside the replica using FlashForge's own python3 on MIPS (they come out byte-identical); `make test-boot-screen` is the gate |
| [`pkgs/anvil-core/payload/bin/ff-startup.py`](../pkgs/anvil-core/payload/bin/ff-startup.py) | `/usr/data/anvil/bin/` | Everything before HelixScreen. **Every boot** it hands the toolhead boards over from their bootloaders (calling `ff_mcu_bringup.py` directly — it owns when klippy opens the ports, so it owns doing this first), starts klipper, and waits for klipper + moonraker to be ready, naming the board or service holding things up and re-handing the boards over on each retry. `init.d/S70klipper` stands aside for it and takes the job back if it cannot run — starting the UI before that is what produces a screen reporting a disconnected printer with no clue which board is missing. **First boot only**, once that has happened, it sends `FF_IMPORT_FIRMWARE_CONFIG` and `SAVE_CONFIG` over the moonraker API and stamps `/usr/data/anvil/.firmware-config-imported`. Only a verified save stamps, so a slow boot retries. `MOD_STARTUP=0` skips the lot; `MOD_IMPORT=0` keeps the wait and skips the migration |
| [`pkgs/anvil-core/payload/bin/ff-startup.py`](../pkgs/anvil-core/payload/bin/ff-startup.py) | `/usr/data/anvil/bin/` | Everything before HelixScreen. **Every boot** it hands the toolhead boards over from their bootloaders (calling `ff_mcu_bringup.py` directly — it owns when klippy opens the ports, so it owns doing this first), starts klipper, and waits for klipper + moonraker to be ready, naming the board or service holding things up and re-handing the boards over on each retry. It runs as two s6-rc oneshots (`mcu-bringup`, then `ff-startup`), and the UI depends on the second — starting the UI before that is what produces a screen reporting a disconnected printer with no clue which board is missing. **First boot only**, once that has happened, it sends `FF_IMPORT_FIRMWARE_CONFIG` and `SAVE_CONFIG` over the moonraker API and stamps `/usr/data/anvil/.firmware-config-imported`. Only a verified save stamps, so a slow boot retries. `MOD_STARTUP=0` skips the lot; `MOD_IMPORT=0` keeps the wait and skips the migration |
| [`pkgs/anvil-core/payload/config/ff-print-macros.cfg`](../pkgs/anvil-core/payload/config/ff-print-macros.cfg) | `/usr/data/config/` | `START_PRINT` / `END_PRINT` / `PAUSE` / `RESUME` / `CANCEL_PRINT`, reconstructed from the app's sequences, plus the `_FF_PREFLIGHT` calibration and tool-presence gate; declares `[ff_print]` and the `FF_BEFORE_PRINT_START` / `FF_AFTER_PRINT_END` entry points it calls |
| [`pkgs/anvil-core/payload/config/ff-filament.cfg`](../pkgs/anvil-core/payload/config/ff-filament.cfg) | `/usr/data/config/` | `LOAD_FILAMENT` / `UNLOAD_FILAMENT` / `PURGE` — the touchscreen's filament-load sequence (grab tool, purge chute, feed) recovered from the binary; unload is a designed retract (the stock app has none) |
| [`pkgs/anvil-core/payload/config/ff-runout.cfg`](../pkgs/anvil-core/payload/config/ff-runout.cfg) | `/usr/data/config/` | Runout / clog handling: gives the stock `fd_ex*` / `fm_ex*` sensors a `runout_gcode` that pauses a Mainsail print when the **mounted** tool runs out or clogs (the app's E0162 / E0163, reported here in plain words); `ff_toolchange` arms only the mounted tool's sensors |
| [`pkgs/klipper-config/prog/config/printer.base.cfg`](../pkgs/klipper-config/prog/config/printer.base.cfg) | `/usr/prog/klipper/config/` | FlashForge's `printer.base.cfg` with the chamber block replaced by `[include printer.chamber.cfg]`. Klipper can override an option but cannot un-declare a section, and the plain Creator 5 has no chamber heating element, so its heater must be **absent** rather than neutralised. `bin/unpack.sh` compares this against each stock package it unpacks and warns if FlashForge's has changed |
| [`pkgs/klipper-config/payload/config/printer.base.cfg`](../pkgs/klipper-config/payload/config/printer.base.cfg) | `$MODDIR/config/` -> symlinked to `/usr/data/config/printer.base.cfg` | FlashForge's `printer.base.cfg` with the chamber block replaced by `[include printer.chamber.cfg]`. Klipper can override an option but cannot un-declare a section, and the plain Creator 5 has no chamber heating element, so its heater must be **absent** rather than neutralised. `bin/unpack.sh` compares this against each stock package it unpacks and warns if FlashForge's has changed |
| [`Creator5.cfg`](../pkgs/klipper-config/payload/config/chamber/Creator5.cfg) · [`Creator5Pro.cfg`](../pkgs/klipper-config/payload/config/chamber/Creator5Pro.cfg) | `$MODDIR/config/chamber/<Machine>.cfg` -> symlinked to `/usr/data/config/printer.chamber.cfg` | The one per-model difference: the Pro gets `[heater_generic chamber_heater]` + `[verify_heater]` verbatim from FlashForge, the Creator 5 gets only `[temperature_sensor chamber]` on the same pin. **Both ship in `anvil-klipper-config`** and `anvil-link-prog.sh` links whichever the printer asks for, reading `MACHINE=` out of FlashForge's own `app_startup.sh`. They used to be a package per model, which Conflicted — each owned `config/printer.chamber.cfg`, so opkg refused the pair and the build had to choose. Nothing is edited at build time, and the payload is no longer model-specific |
| [`pkgs/anvil-core/payload/config/ff-chamber.cfg`](../pkgs/anvil-core/payload/config/ff-chamber.cfg) | `/usr/data/config/` | `M141` / `M191` for the chamber heater (Klipper has neither, and the stock app drove the chamber only from its own UI), plus the gate: the macros ask Klipper whether `heater_generic chamber_heater` exists, so a non-zero chamber target is refused on a machine that does not declare one. Nothing to keep in sync; identical in every package |
| [`docs/notes/`](notes/) | (reference only) | Condensed reverse-engineering notes: what the stock app actually does, with binary addresses |
@@ -175,7 +175,7 @@ scp pkgs/anvil-core/payload/config/ff-*.cfg \
```
Only a save it can verify writes the stamp. A boot where the heater board
needed several klippy restarts (`S70klipper`) simply times out and tries
needed several klippy restarts simply times out and tries
again next boot; `/usr/data/logs/anvil-boot.log` says which service it was
waiting on.
installer/run-append.sh
+57 -16
View File
@@ -57,14 +57,12 @@ if [ -n "$MODTAR" ]; then
# Remove what the PREVIOUS payload installed -- exactly that, and
# nothing else.
#
# THE PROPERTY THIS HAS TO KEEP, and why anything is removed rather
# than just extracted over: the installed set must end up exactly the
# shipped set. Overwriting in place is harmless only while the set of
# filenames never changes, and it does -- S60web was split into
# S60nginx and S62moonraker, and the old S60web survived the update and
# sat next to both. firmwareExe runs every executable $MODDIR/init.d/S*
# in filename order, so nginx and moonraker each started twice, from
# two scripts that disagree about where moonraker lives.
# THE PROPERTY THIS HAS TO KEEP, which is why anything is removed at
# all rather than just extracted over: files used to be overwritten in
# place and never removed, which is harmless only while the set of
# filenames never changes. It does change -- a renamed script survives
# the update and sits next to the one that replaced it. The installed
# set must end up exactly the shipped set.
#
# bin/patch.sh ships a manifest of every path the payload installs --
# one per line, relative to $MODDIR, files and directories both, itself
@@ -134,14 +132,22 @@ if [ -n "$MODTAR" ]; then
rm -rf $MODDIR/bin $MODDIR/www $MODDIR/nginx $MODDIR/helixscreen $MODDIR/config $MODDIR/moonraker $MODDIR/init.d
echo "previous install removed (no manifest -- pre-manifest layout)"
fi
# init.d GETS THIS EVEN IN THE MANIFEST BRANCH: a manifest only knows
# the files IT tracked, so a script from further back (planted by hand,
# restored from a backup, left over from the one pre-manifest jump) is
# invisible to it and survives forever. Tolerable in $MODDIR/bin or
# $MODDIR/config, where something not ours might legitimately live --
# not here, where firmwareExe runs every executable S* in filename
# order and an orphan starts a second copy of whatever it launches.
rm -rf $MODDIR/init.d
# init.d/ AND anvil-service.sh ARE GONE, and go unconditionally rather
# than by manifest. The payload ships neither any more: s6-rc is the
# CLI, and the tree is started by firmwareExe. A leftover S70klipper is
# a script that starts an UNSUPERVISED klippy next to the supervised
# one, and a leftover anvil-service.sh is a library something stale
# could still source, so neither may be left to a diff that only knows
# about files the LAST manifest tracked -- a script planted by hand, or
# surviving the one pre-manifest jump every printer takes, is invisible
# to that.
#
# No hot migration is attempted. The install runs from app_startup.sh
# DURING BOOT, before firmwareExe starts, so there is no supervision
# tree up while this runs and the new one comes up from scratch a
# moment later. A `sh run.sh` typed over ssh instead is the exception:
# that printer needs a reboot, and nothing here forces one.
rm -rf $MODDIR/init.d $MODDIR/anvil-service.sh
mkdir -p $MODDIR
# Try xz first (FlashForge's own factory installer uses `xz -dc`, so
# it exists), then fall back to plain tar in case a build shipped it
@@ -179,6 +185,41 @@ if [ -n "$MODTAR" ]; then
rm -rf $HELIX_KEEP
fi
chmod a+x $MODDIR/bin/* 2>/dev/null
# ---- the s6 scandir, swept -------------------------------------
# MEASURED: s6-rc-init creates one symlink per service in the scandir
# and fails outright -- "unable to supervise service directories ...:
# File exists" -- if a name is taken. A printer upgrading from the
# pre-s6-rc payload has nginx, moonraker and camera in there as real
# directories, and the manifest cannot remove them: it deletes the
# files it listed, but s6-supervise created supervise/ and event/
# inside each at RUNTIME, so the rmdir correctly refuses and leaves
# exactly the name s6-rc-init collides with.
#
# After the extraction, so the payload's own empty etc/s6 has landed.
rm -rf $MODDIR/etc/s6
mkdir -p $MODDIR/etc/s6
# /run is a tmpfs, so this matters only for a `sh run.sh` typed over
# ssh: a live s6-rc state points at the database just replaced.
rm -rf /run/s6-rc
# klipperDaemon, replaced: stock's `start` forks a second, unsupervised
# klippy beside the s6 one. It cannot ride in the software component --
# FlashForge's run.sh copies a fixed list of files and klipperDaemon is
# not on it -- so it is installed from here.
#
# KLIPPER_NICENESS is carried forward off the file being replaced, so
# FlashForge's number survives without being guessed. On later updates
# that reads our own copy, which is how the value persists.
if [ -f $MODDIR/bin/klipperDaemon ] && [ -d /usr/prog/klipper ]; then
KN=`sed -n 's/^ *KLIPPER_NICENESS= *\([-0-9][0-9]*\).*/\1/p' \
/usr/prog/klipper/klipperDaemon 2>/dev/null | head -n 1`
[ -n "$KN" ] || KN=0
sed "s/^KLIPPER_NICENESS=.*/KLIPPER_NICENESS=$KN/" \
$MODDIR/bin/klipperDaemon > /usr/prog/klipper/klipperDaemon.new \
&& mv -f /usr/prog/klipper/klipperDaemon.new /usr/prog/klipper/klipperDaemon \
&& chmod +x /usr/prog/klipper/klipperDaemon \
&& echo "klipperDaemon: replaced (start is a no-op; niceness $KN)" \
|| echo "!! could not replace klipperDaemon -- it can still fork a second klippy"
fi
echo "mod payload installed"
# Point the stock boot path at the payload's own copies. This has to
# be HERE and not earlier: FlashForge's run.sh distributed the
pkgs/anvil-core/build.sh
+20 -40
View File
@@ -3,21 +3,12 @@
#
# THE RECIPE IS A LOOP OVER A DIRECTORY NOW, and that is the whole point of
# the layout. pkgs/anvil-core/payload/ is the $MODDIR overlay this package
# installs, laid out exactly as it lands: payload/init.d/S60nginx becomes
# $MODDIR/init.d/S60nginx and nothing here has to say so. What used to be six
# pkg_stage lines with hand-written destinations and a glob for the .cfg files
# is one loop, and a file added to that tree is shipped without an edit here.
# installs, laid out exactly as it lands: payload/etc/s6-rc/source/nginx
# becomes $MODDIR/etc/s6-rc/source/nginx and nothing here has to say so. A file
# added to that tree is shipped without an edit here.
#
# THE TWO SIBLING DIRECTORIES ARE NOT SHIPPED, and they are named rather than
# explained because a comment cannot be checked and a path can:
# THE ONE SIBLING DIRECTORY THAT IS NOT SHIPPED:
#
# pkgs/anvil-core/prog/ files bin/patch.sh places on /usr/prog -- the
# FIRMWARE partition, which is somebody else's filesystem. firmwareExe
# is the boot entry point app_startup.sh execs, so it cannot live under
# a --prefix root that a package installs into; the test that every
# path in a package lands under $MODDIR would catch it if this tried.
# It becomes a package member the day a postinst symlinks it into place
# from a staging root -- see docs/notes/85-packaging.md phase 2.
# pkgs/anvil-core/seed/ anvil.conf.in, which is TEMPLATED at build time
# from config.env's MOD_* and NICE_* values and then PRESERVED across
# updates by installer/run-append.sh. Those two facts make it user
@@ -36,28 +27,15 @@
# chamber/<Machine>.cfg
# klippy extras pkgs/klipper
#
# WHAT CAME BACK: firmwareExe and start.sh, in payload/prog/. They are the two
# files the printer reads from /usr/prog, and this package owns the script
# that links them there -- see payload/bin/anvil-link-prog.sh.
# payload/prog/ holds the three files the printer reads from /usr/prog --
# firmwareExe, start.sh and klipperDaemon -- and this package owns the script
# that links them there: payload/bin/anvil-link-prog.sh.
#
# WHAT STAYS, AND WHY IT IS NOT THE SAME QUESTION. init.d/ and etc/s6/ look
# like they belong to the services they start -- S62moonraker to Moonraker,
# S70klipper to Klipper -- and moving them would brick a printer. Those
# services are gated at RUNTIME by flags in anvil.conf, while the package that
# would own S80ui is gated at BUILD time by BUILD_HELIX. A BUILD_HELIX=0
# tarball has no anvil-helixscreen package, so it would ship no S80ui and
# nothing would decide about the screen at all. The scripts that read a flag
# must not themselves be behind a second, different flag. nginx.conf stays for a plainer reason: nginx is the rootfs's binary
# and there is no package here to give it to.
#
# THE s6 SERVICE DEFINITIONS ARE HERE, for now, and that is a decision with a
# short shelf life. Today payload/etc/s6/ is a plain s6-svscan scandir: text
# files, no build step, nothing to compile. Once the s6-rc migration lands
# they become a source tree that s6-rc-compile turns into a binary database,
# and at that point they need PKG_BUILD_DEPENDS="s6-rc" -- which this package
# must not acquire, because the repo's own shell scripts should be packageable
# on a checkout that has never run a cross-compiler. That is the moment they
# become anvil-services with a recipe of its own.
# THE s6-rc SERVICE SOURCE IS HERE and is not compiled by this recipe.
# payload/etc/s6-rc/source/ is text; bin/patch.sh runs s6-rc-compile over it
# once the payload is assembled, so this package stays buildable on a checkout
# that has never run a cross-compiler. Giving it PKG_BUILD_DEPENDS="s6-rc"
# would take that away.
set -euo pipefail
. ./bin/common.sh
. pkgs/lib.sh
@@ -81,16 +59,18 @@ done
# a `run` that arrives without one is a service s6 can never start and reports
# only in its own log. cp -a preserved whatever git had; this makes it true
# regardless of what a contributor's umask did.
chmod +x "$PKG_WORK/stage$MODDIR/init.d"/S* "$PKG_WORK/stage$MODDIR/bin"/*
chmod +x "$PKG_WORK/stage$MODDIR"/etc/s6/*/run 2>/dev/null || true
chmod +x "$PKG_WORK/stage$MODDIR/bin"/* "$PKG_WORK/stage$MODDIR/prog"/*
# The service definitions s6-supervise and the oneshot runner exec. `up` and
# `down` are execline command lines rather than files, so they are not here.
chmod +x "$PKG_WORK/stage$MODDIR"/etc/s6-rc/source/*/run 2>/dev/null || true
# shellcheck disable=SC2086
pkg_ship $_top
# anvil-env.sh and anvil-service.sh are SOURCED, not executed, and shipping
# either with an executable bit would invite somebody to run it. Checked
# because the chmod above is a glob and globs grow.
for _f in anvil-env.sh anvil-service.sh; do
# anvil-env.sh is SOURCED, not executed, and shipping it with an executable
# bit would invite somebody to run it. Checked because the chmod above is a
# glob and globs grow.
for _f in anvil-env.sh; do
[ -x "$PKG_OUT/$_f" ] && pkg_die \
"anvil-core: $_f is executable -- it is sourced, never run"
done
pkgs/anvil-core/payload/anvil-service.sh
-322
View File
@@ -1,322 +0,0 @@
# The shape every mod service script has. Sourced, never executed.
#
# MODDIR=/usr/data/anvil
# [ -f $MODDIR/anvil.conf ] && . $MODDIR/anvil.conf
# . $MODDIR/anvil-service.sh
# SVC_NAME=moonraker
#
# WHY THIS FILE EXISTS. Every script in init.d/ has the same four problems: how
# to say something in the boot log, how to tell whether the daemon is still
# there, how to stop it without leaving a stale pidfile, and how to wait on
# hardware without holding up the boot. This is the shared answer, so fixing one
# of them fixes it everywhere rather than in whichever script the bug was
# noticed in -- and so the busybox corrections below (see svc_stop_daemon) live
# in one place where the comment cannot drift away from the code.
#
# THE API, in full:
#
# SVC_NAME the log prefix, set once per script. Required.
# SVC_EXTRA_VERBS optional; extra verbs for the usage line, e.g.
# SVC_EXTRA_VERBS="|force-start"
#
# svc_say MSG -> "name: MSG"
# svc_warn MSG -> "name: !! MSG"
# svc_pid_alive PIDFILE is the pid in this file still a process?
# svc_proc_alive PATTERN is a process matching this pattern running?
# svc_start_daemon PIDFILE EXEC [ARGS...]
# start-stop-daemon -S, backgrounded, with a
# pidfile; warns and returns non-zero if
# nothing was started
# svc_stop_daemon PIDFILE stop it, WAIT for it to actually be gone,
# remove the pidfile
# svc_detach FUNC [ARGS...] run a slow thing in the background and
# return at once; SVC_BG_PID is its pid
# svc_dispatch "$@" the start|stop|restart|status block; calls
# the start(), stop() and status() the script
# defines, and exits
#
# -- talking to s6, for the services that have been moved to it --
# SVC_S6_BIN where the s6 binaries are; $MODDIR/bin
# SVC_S6_SCANDIR the scandir S40s6 scans; $MODDIR/etc/s6
#
# svc_s6_running is the SCANNER alive and listening?
# svc_s6_svc NAME OPT... s6-svc OPT... $SVC_S6_SCANDIR/NAME
# svc_s6_up NAME [SECONDS] bring NAME up and WAIT until it is up
# svc_s6_down NAME [SECONDS] bring NAME down and WAIT until it is down
# svc_s6_stat NAME one line of s6-svstat, on stdout
# svc_s6_ready NAME [SECONDS] block until NAME declares itself READY
#
# The script defines start(), stop() and status(); this file defines
# everything else. A script with an extra verb intercepts it before handing the
# rest over -- see svc_dispatch.
# ---- saying things ---------------------------------------------------------
#
# Every line these scripts print is "name: something", because firmwareExe runs
# them all into one boot log and a bare "started" says nothing about which
# started. Name it once, at the top, in SVC_NAME.
#
# The "!!" on svc_warn is what somebody greps for when a printer boots wrong,
# and means the same everywhere else in the mod (firmwareExe and start.sh).
svc_say() {
echo "${SVC_NAME:-service}: $*"
}
svc_warn() {
echo "${SVC_NAME:-service}: !! $*"
}
# ---- is it alive? ----------------------------------------------------------
#
# Two answers, because there are genuinely two situations.
#
# svc_pid_alive is for a daemon WE started through start-stop-daemon, so the
# pid is known exactly. Prefer it: it cannot be confused by a second copy, by a
# grep matching its own pipeline, or by an unrelated process with a similar
# name. The pidfile can still lie -- pids get recycled -- which is why
# svc_stop_daemon below removes it.
svc_pid_alive() {
[ -f "$1" ] || return 1
kill -0 "`cat "$1" 2>/dev/null`" 2>/dev/null
}
# svc_proc_alive is for a process we did NOT start with a pidfile -- klippy,
# launched by FlashForge's start.sh; wpa_supplicant, which backgrounds itself
# with -B; mjpg_streamer under its own respawn loop. All we have is the name.
#
# pgrep is tried first and is not assumed: this busybox is built with it on the
# printers we have looked at, but that is a build option and a rootfs we do not
# control ships it. The fallback is plain `ps` (busybox ps takes no options
# worth relying on, so this is deliberately bare) piped through grep, with the
# `grep -v grep` that stops the pipeline from matching itself.
#
# Pass the pattern in the form pgrep wants; `klippy\.py` matches both, because
# grep reads the backslash-escaped dot the same way.
svc_proc_alive() {
if command -v pgrep >/dev/null 2>&1; then
pgrep -f "$1" >/dev/null 2>&1
else
ps 2>/dev/null | grep "$1" | grep -v grep >/dev/null 2>&1
fi
}
# ---- starting and stopping -------------------------------------------------
#
# start-stop-daemon -S with -b (background it), -m (write the pidfile) and -p.
# The pidfile is the point: without one there is no way to stop the daemon
# except by name, and by-name is how you kill the wrong python.
#
# The caller announces success itself, with the path or port that makes it
# useful. Failure is announced here with the exit code: busybox
# start-stop-daemon exits non-zero when a process matching the pidfile is
# ALREADY running, which after a `restart` means the old one outlived the wait
# in stop(). Either way nothing new started, and a script that prints nothing
# at that moment looks like it worked.
#
# SVC_NICE, non-empty and non-zero, becomes start-stop-daemon's -N. klippy
# shares two cores with everything the mod added, and Klipper reports a host
# that misses its step deadlines as an MCU "Timer too close" shutdown, so the
# services around it should yield to it. CPU only: this busybox has no ionice
# applet, so eMMC contention has no equivalent knob.
#
# -N rather than a `nice -n` wrapper because --exec is what start-stop-daemon
# matches and records, and wrapping would make that the nice binary rather than
# the program.
svc_start_daemon() {
_svc_pidfile=$1
shift
_svc_exec=$1
shift
# An empty or 0 SVC_NICE means "as before": no -N, normal priority.
if [ -n "${SVC_NICE:-}" ] && [ "${SVC_NICE:-0}" != 0 ]; then
start-stop-daemon -S -b -m -N "$SVC_NICE" \
-p "$_svc_pidfile" --exec "$_svc_exec" -- "$@"
else
start-stop-daemon -S -b -m -p "$_svc_pidfile" --exec "$_svc_exec" -- "$@"
fi
_svc_rc=$?
[ $_svc_rc = 0 ] || svc_warn "FAILED to start -- start-stop-daemon returned $_svc_rc"
unset _svc_pidfile _svc_exec
return $_svc_rc
}
# The correction busybox needs. Its start-stop-daemon -K sends the signal and
# returns immediately, and does not remove the pidfile. Both are wrong here:
#
# * Not waiting makes `restart` race its own start. stop() returns, start()
# runs start-stop-daemon -S, the old process is still holding the pidfile,
# -S refuses, and the service is now stopped rather than restarted.
# * Not removing the pidfile leaves svc_pid_alive testing a pid that the
# kernel is free to hand to something else, so `status` eventually reports
# an unrelated process as a running service.
#
# So: signal, wait for it to actually be gone, then remove the file. 15 seconds
# is long enough for moonraker to finish writing its database and short enough
# that a stuck process does not hold a boot open. A process still there at the
# end is said out loud, because the next start is about to fail and this is the
# line that explains why.
svc_stop_daemon() {
[ -f "$1" ] || return 0
start-stop-daemon -K -p "$1" 2>/dev/null
_svc_waited=0
while svc_pid_alive "$1" && [ $_svc_waited -lt 15 ]; do
sleep 1
_svc_waited=$((_svc_waited + 1))
done
svc_pid_alive "$1" && svc_warn "still running after ${_svc_waited}s"
unset _svc_waited
rm -f "$1"
return 0
}
# ---- not making the boot wait ----------------------------------------------
#
# firmwareExe runs $MODDIR/init.d/S* one at a time, in the FOREGROUND, in
# filename order. So every second a start() spends waiting is a second the
# services after it do not exist -- and the waits in these scripts are all
# hardware: up to 15s for wlan0 to register after the 8821cu insmod and 20s
# more to associate, 30s for a USB camera to enumerate, 90s for klippy to give
# up on an MCU. Inline, the longest delay lands on exactly the machine with no
# network, no camera or no heater board, which is the machine somebody needs to
# ssh into to find out why.
#
# So the waiting happens detached and start() returns at once. The cost is that
# the detached lines arrive in the boot log out of order, after the later
# services'.
#
# It returns 0 unconditionally: letting `func &` be the last statement of
# start() would hand the caller the shell's opinion of a fork rather than a
# verdict on the service. SVC_BG_PID is for a supervisor that has to be
# killable -- S65camera writes it to a pidfile, because killing only the
# streamer it supervises gets the streamer restarted three seconds later.
svc_detach() {
"$@" &
SVC_BG_PID=$!
return 0
}
# ---- talking to s6 ---------------------------------------------------------
#
# Everything above is the hand-rolled supervisor; everything below is how a
# service asks the real one instead. Both sets exist at once: S40s6 starts the
# scanner and the services move into it ONE AT A TIME, each keeping its S*
# script as the interface people already type. A service that has not moved
# uses the functions above and does not know these exist.
#
# WHERE THE PATHS COME FROM. s6 bakes its own prefix in at COMPILE time -- its
# waiting verbs exec s6-svlisten, which spawns s6-ftrigrd out of the libexecdir
# chosen by ./configure -- so the binaries only work from the prefix they were
# built for (/usr/data/anvil; see tools/supervisor/README.md, where shipping
# them elsewhere made every waiting verb fail while status and respawn kept
# working). The scandir is the second thing every script has to agree about
# with S40s6. Both are overridable so a test can point at its own scandir
# without reinstalling the payload.
SVC_S6_BIN=${SVC_S6_BIN:-${MODDIR:-/usr/data/anvil}/bin}
# Not written as one ${MODDIR:-/usr/data/anvil} expression ending in etc/s6:
# test_paths.py's absolute-path regex reads the closing brace immediately
# followed by that suffix as a STOCK rootfs path in its own right (its
# lookbehind excludes a preceding word character or dot, but not a brace),
# and this directory is not something any stock FlashForge rootfs ships -- it
# lives under $MODDIR, on the data partition. Splitting the fallback onto its
# own line puts a word character (the variable name) right before the suffix
# instead, which the same lookbehind does exclude.
_s6_default_scandir=${MODDIR:-/usr/data/anvil}
_s6_default_scandir=$_s6_default_scandir/etc/s6
SVC_S6_SCANDIR=${SVC_S6_SCANDIR:-$_s6_default_scandir}
unset _s6_default_scandir
# Is the SCANNER there? Asked by BEHAVIOUR, not by process name.
# `s6-svscanctl -a` is the request to rescan the scandir: harmless, idempotent,
# and it travels down $SCANDIR/.s6-svscan/control, a FIFO opened non-blocking.
# With nothing reading that FIFO the open fails with ENXIO and s6-svscanctl
# exits 100, "supervisor not listening". A dead scanner leaves the FIFO on
# disk, so this is a distinction `ps | grep` cannot make.
#
# It answers false rather than exploding when s6 is not installed at all,
# because that is a real state on a printer half-way through an update.
svc_s6_running() {
[ -x "$SVC_S6_BIN/s6-svscanctl" ] || return 1
"$SVC_S6_BIN/s6-svscanctl" -a "$SVC_S6_SCANDIR" >/dev/null 2>&1
}
# The raw verb, for anything the four wrappers below do not cover. NAME first
# and the options after, which is the opposite order to s6-svc's own -- s6
# wants `s6-svc OPTIONS servicedir` -- because in ash it is the first argument
# that can be shifted off cleanly, and because every caller here is naming a
# service, not a directory. The directory is this file's business.
svc_s6_svc() {
_svc_s6_name=$1
shift
"$SVC_S6_BIN/s6-svc" "$@" "$SVC_S6_SCANDIR/$_svc_s6_name"
_svc_s6_rc=$?
unset _svc_s6_name
return $_svc_s6_rc
}
# Up and down, each WAITING for what it asked for, which is the reason to
# prefer a supervisor to start-stop-daemon. -wu/-wD do not return until the
# service is really up or really down, and -T bounds that in MILLISECONDS so a
# service that never comes down cannot hold a boot or an ssh session open. The
# 15-second default is svc_stop_daemon's bound and reasoning.
svc_s6_up() {
svc_s6_svc "$1" -wu -T "$(( ${2:-15} * 1000 ))" -u
}
svc_s6_down() {
svc_s6_svc "$1" -wD -T "$(( ${2:-15} * 1000 ))" -d
}
# One line about a service, on stdout, for a status() to print. s6-svstat's own
# output is already the right shape -- "up (pid 1234) 71 seconds" -- so it is
# passed through rather than paraphrased. 2>&1 because "unable to open
# supervise/status" is also an answer.
svc_s6_stat() {
"$SVC_S6_BIN/s6-svstat" "$SVC_S6_SCANDIR/$1" 2>&1
}
# READINESS -- the reason this is s6 and not runit. A service that writes to
# its notification-fd is telling the supervisor it is USABLE, not merely
# forked, and s6-svwait -U blocks until it does; runit's `sv start` returns on
# the fork, which says nothing.
#
# Unused today: S65camera polls /dev/video0 for up to 30 seconds and S70klipper
# retries an MCU handshake for up to 90, both inside their own scripts, and
# both become "declare ready when the device is there" plus a caller that waits
# here. Timeout in seconds, converted to the milliseconds s6 wants; non-zero if
# it ran out, so a caller can carry on degraded rather than block.
svc_s6_ready() {
"$SVC_S6_BIN/s6-svwait" -U -t "$(( ${2:-30} * 1000 ))" "$SVC_S6_SCANDIR/$1"
}
# ---- the verb block --------------------------------------------------------
#
# Every one of these scripts ends in the same case statement, spelled once here.
#
# `restart` has NO sleep between stop and start. A sleep there stands in for a
# wait nobody is doing -- busybox start-stop-daemon -K returns before the
# process is dead -- and svc_stop_daemon waits for the process to actually be
# gone, for as long as it takes. A stop() that does not wait is a bug to fix in
# stop(); a sleep only hides it for two seconds.
#
# EXTRA VERBS. S70klipper has `force-start`, which overrides its hand-off to
# ff-startup.py. A script with a verb of its own handles it first and hands the
# rest here, and names it in SVC_EXTRA_VERBS so the usage line stays true:
#
# SVC_EXTRA_VERBS="|force-start"
# case "$1" in
# force-start) start force ;;
# *) svc_dispatch "$@" ;;
# esac
#
# This exits rather than returns, so it is the last statement of the script.
# The exit code is the one start(), stop() or status() produced -- an unknown
# verb is the only thing this file decides, and it is 1.
svc_dispatch() {
case "$1" in
start|'') start; exit $? ;;
stop) stop; exit $? ;;
restart) stop; start; exit $? ;;
status) status; exit $? ;;
*) echo "usage: $0 {start|stop|restart|status${SVC_EXTRA_VERBS:-}}"; exit 1 ;;
esac
}
pkgs/anvil-core/payload/bin/ff-startup.py
+90 -76
View File
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
# Everything that has to happen before HelixScreen, and the panel that says so.
#
# The firmwareExe wrapper runs this after the init.d services and before the
# UI. It does two jobs, and only the second is once-per-install:
# Run as two s6-rc oneshots, between klipper and the UI. It does two jobs, and
# only the second is once-per-install:
#
# EVERY BOOT wait until the printer is genuinely up -- the toolhead boards
# handed over from their bootloaders, then klipper and
@@ -32,28 +32,43 @@
# 1. Hand the toolhead boards over from their bootloaders, by calling
# ff_mcu_bringup.py directly -- it is a module here, not a subprocess --
# naming on the panel whichever board is still being waited for.
# 2. Launch klipper (start.sh, told not to redo the bring-up) and wait for
# klippy and moonraker to be ready. If a board never answered, klippy
# says so and names it, and so do we; then stop klippy, redo the
# bring-up and try again, which is what reopening the port achieves.
# 2. Wait for klippy and moonraker to be ready. If a board never answered,
# klippy says so and names it, and so do we; then hand the boards over
# again and restart klippy, which is what reopening the port achieves.
# 3. If the install is already stamped, stop here -- the rest is the
# migration, and it has been done.
# 4. Otherwise run FF_IMPORT_FIRMWARE_CONFIG, then SAVE_CONFIG (which
# restarts klippy), wait for it to come back, and only then stamp.
#
# WHY THIS OWNS KLIPPER'S START. The bring-up has to happen before klippy
# opens the serial ports, and the retry that actually fixes a board which
# missed its window is "close the port, hand it over again, reopen" -- so
# whatever does the bring-up has to be the same thing that starts and
# restarts klippy. Splitting them across two processes meant this program
# could only watch, through a file, work it was not allowed to do. init.d/
# S70klipper stands aside when this program is going to run and takes the job
# back when it cannot: klipper starting is the single most important thing
# the mod does, and it must not depend on a python interpreter.
# HOW IT IS RUN NOW: as TWO s6-rc oneshots, both of them this same program.
# `mcu-bringup` is step 1 alone (--only-bringup) and klipper depends on it, so
# the boards are handed over before klippy opens the ports and nothing has to
# remember to do it -- including s6 restarting klippy after a crash, which is a
# path no script of ours is on. `ff-startup` is steps 2-4 (--no-bringup
# --no-klipper) and depends on klipper and moonraker; the UI depends on it. See
# the `up` scripts in payload/etc/s6-rc/source/, which assemble both command
# lines out of anvil.conf.
#
# WHAT THAT DEPENDENCY DOES NOT GIVE US, and why the waiting below stays. s6-rc
# counts a longrun up as soon as it is forked unless its servicedir carries a
# notification-fd, and klipper has none: klippy's "ready" is a state on
# moonraker's API, which is exactly what this program polls for. So depending
# on klipper means "after klippy was LAUNCHED", not "after klippy is usable".
# moonraker and the camera do have a notification-fd, so their readiness is
# real -- which is why depending on moonraker genuinely does mean the API is
# answering.
#
# WHY IT STILL RESTARTS KLIPPER. The retry that actually fixes a board which
# missed its window is "close the port, hand it over again, reopen", so
# whatever does the bring-up has to be the same thing that restarts klippy.
# That has not changed; only the way it asks has -- `s6-svc -wr -t`, which
# terminates klippy and waits for the supervisor to bring it back. NOT `s6-rc`:
# this program runs INSIDE an s6-rc transition, which holds an exclusive lock on
# the live directory, so an s6-rc underneath it would deadlock.
#
# Anything short of a verified success leaves NO stamp: the next boot tries
# again. That is deliberate. The heater board on this machine routinely needs
# several klippy restarts before it answers (see init.d/S70klipper), so "the
# several klippy restarts before it answers, so "the
# stack was not ready in time" is a normal event to retry, not a failure to
# record forever.
#
@@ -73,7 +88,8 @@
#
# usage: ff-startup.py [--stamp F] [--dir D] [--moonraker URL] [--timeout S]
# [--no-import] [--no-klipper] [--klipper-tries N]
# [--start-sh P] [--daemon P] [--mcu-timeout S]
# [--only-bringup] [--no-bringup] [--klipper-svcdir D]
# [--mcu-timeout S]
# [--fb DEV] [--fb-geometry G] [--no-screen] [--dry-run]
#
# Exit 0 = the printer came up, and the migration either was not needed or
@@ -105,8 +121,13 @@ except ImportError:
STAMP = '/usr/data/anvil/.firmware-config-imported'
JSON_DIR = '/usr/data/firmwareRes/config'
MOONRAKER = 'http://127.0.0.1:7125'
START_SH = '/usr/prog/klipper/start.sh'
DAEMON = '/usr/prog/klipper/klipperDaemon'
# How klipper is restarted: `s6-svc -wr -t` on its live servicedir. NOT s6-rc --
# this program runs as an s6-rc oneshot, and s6-rc holds an exclusive lock on
# the live directory for the length of a transition, so an s6-rc underneath it
# would wait for a lock its own parent holds. `-t` also leaves the WANTED state
# alone, so s6-rc goes on believing klipper is up, which stays true.
S6_SVC = '/usr/data/anvil/bin/s6-svc'
KLIPPER_SVCDIR = '/usr/data/anvil/etc/s6/klipper'
UDS = '/tmp/uds'
TIMEOUT = 240.0
MCU_TIMEOUT = 30.0
@@ -337,56 +358,31 @@ def klippy_running():
return False
def start_klipper(args):
"""Run start.sh, telling it the bring-up is already done.
def restart_klipper(args):
"""Terminate klippy and wait for its supervisor to bring it back.
It is not left to run in the background: everything it does after the
bring-up we skipped is quick, and knowing it finished is what makes the
wait that follows meaningful."""
if not os.path.exists(args.start_sh):
log('no %s -- cannot start klipper' % args.start_sh)
Reopening the port is what toggles DTR/RTS and gives a board that missed
its window another chance, so the restart is the fix rather than another
go at the same thing -- and -wr does not return until klippy is actually
running again, which is what makes the next attempt worth making."""
if not os.path.exists(args.s6_svc):
log('no %s -- cannot restart klipper' % args.s6_svc)
return False
env = dict(os.environ)
env['FF_SKIP_MCU_BRINGUP'] = '1'
log('starting klipper (%s)' % args.start_sh)
log('restarting klipper (s6-svc -t %s)' % args.klipper_svcdir)
try:
completed = subprocess.run(['/bin/sh', args.start_sh], env=env,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, timeout=120)
completed = subprocess.run(
[args.s6_svc, '-wr', '-T', '60000', '-t', args.klipper_svcdir],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=120)
except subprocess.TimeoutExpired:
log('start.sh did not return within 120s')
log('s6-svc did not return within 120s')
return False
except Exception as exc:
log('start.sh could not be run (%s)' % exc)
log('s6-svc could not be run (%s)' % exc)
return False
for line in (completed.stdout
or b'').decode('utf-8', 'replace').splitlines():
log(' start.sh: %s' % line)
return True
def stop_klipper(args):
"""Stop klippy and wait for it to actually go.
Reopening the port is what toggles DTR/RTS and gives a board that missed
its window another chance, so the stop has to be real before the next
attempt is worth making."""
try:
subprocess.run([args.daemon, 'stop'], stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL, timeout=30)
except Exception as exc:
log('klipperDaemon stop failed (%s)' % exc)
for _ in range(15):
if not klippy_running():
break
time.sleep(1.0)
# start.sh refuses to run while the socket is there, so clear a stale one
# -- but only once klippy is definitely gone.
if not klippy_running():
try:
os.remove(UDS)
except OSError:
pass
log(' klipper: %s' % line)
return completed.returncode == 0
def klippy_fault(moonraker):
@@ -492,8 +488,19 @@ def stamp(path, note):
def run(args):
started = time.time()
deadline = started + args.timeout
moonraker = Moonraker(args.moonraker)
panel = Panel(args.fb, args.fb_geometry, enabled=not args.no_screen)
if args.only_bringup:
# The `mcu-bringup` oneshot. Step 1 and nothing else: klipper depends
# on this service, so the boards are out of their bootloaders before
# klippy opens the ports, every time klippy starts.
#
# The panel is NOT handed back clean here. This runs seconds before
# klipper and then ff-startup, and blanking the screen between the two
# would be a flicker for no reason -- ff-startup's own panel.done()
# ends the sequence.
hand_over_boards(panel, args.mcu_timeout)
return 0
moonraker = Moonraker(args.moonraker)
panel.say('STARTING SERVICES', 0.05)
try:
return startup(args, moonraker, panel, started, deadline)
@@ -505,32 +512,34 @@ def run(args):
def bring_up_printer(args, moonraker, panel, started, deadline):
"""Boards out of their bootloaders, klippy up, and retry until it is.
This is what init.d/S70klipper's supervise() used to do by tailing
printer.log for markers, which was the only signal a shell script had.
Here the signal is moonraker's, and the retry can do the thing that
actually helps: hand the boards over again before reopening the ports.
The signal is moonraker's rather than a marker tailed out of
printer.log, and the retry does the thing that actually helps: hand the
boards over again before reopening the ports.
"""
for attempt in range(1, args.klipper_tries + 1):
if not klippy_running():
hand_over_boards(panel, min(args.mcu_timeout,
max(1.0, deadline - time.time())))
if not args.no_klipper and not start_klipper(args):
# ATTEMPT 1 NORMALLY DOES NOTHING HERE. klipper is this service's
# dependency, so s6-rc has already started it and the boards have
# already been handed over by the mcu-bringup oneshot; there is
# nothing to do but wait. The branch is for the two cases where that
# is not true: a retry, and a klippy that is not there at all because
# its run script is failing.
if attempt > 1 or not klippy_running():
if attempt > 1 or not args.no_bringup:
hand_over_boards(panel, min(args.mcu_timeout,
max(1.0, deadline - time.time())))
if not args.no_klipper and not restart_klipper(args):
panel.failed('KLIPPER COULD NOT BE STARTED')
return False
if wait_for_stack(moonraker, panel, deadline, started, quiet=True):
return True
if time.time() >= deadline or attempt == args.klipper_tries:
break
# Reopening the port toggles DTR/RTS, which is what gives a board
# that missed its window another chance -- so the restart is the
# fix, not merely another go at the same thing.
board = klippy_fault(moonraker)
log('attempt %d/%d failed (%s) -- restarting klipper'
% (attempt, args.klipper_tries, board or 'no board named'))
panel.say('RETRYING %s' % (board or 'THE PRINTER'), 0.15,
detail='ATTEMPT %d OF %d' % (attempt + 1,
args.klipper_tries))
stop_klipper(args)
report_stack_failure(moonraker, panel)
return False
@@ -665,10 +674,15 @@ def main(argv):
help="how long one bring-up pass may take")
parser.add_argument('--klipper-tries', type=int, default=KLIPPER_TRIES,
help="how many times to hand the boards over and retry")
parser.add_argument('--start-sh', default=START_SH)
parser.add_argument('--daemon', default=DAEMON)
parser.add_argument('--s6-svc', default=S6_SVC)
parser.add_argument('--klipper-svcdir', default=KLIPPER_SVCDIR,
help="klipper's live servicedir, for the restart")
parser.add_argument('--only-bringup', action='store_true',
help="hand the toolhead boards over, then exit")
parser.add_argument('--no-bringup', action='store_true',
help="skip the FIRST bring-up; a retry still redoes it")
parser.add_argument('--no-klipper', action='store_true',
help="do not start klipper; only wait for it")
help="never touch klipper; only wait for it")
parser.add_argument('--no-import', action='store_true',
help="only bring the printer up; never migrate")
parser.add_argument('--fb', default=None,
pkgs/anvil-core/payload/etc/s6/camera/notification-fd → pkgs/anvil-core/payload/etc/s6-rc/source/camera/notification-fd
View File
File renamed without changes.
pkgs/anvil-core/payload/etc/s6-rc/source/camera/run
Executable
+144
View File
@@ -0,0 +1,144 @@
#!/bin/sh
# The camera, as s6 supervises it: mjpg_streamer on :8080, in the FOREGROUND.
#
# THE ONE RULE: never fork the streamer, never background it, never return
# while the camera is meant to be running. `exec` at the bottom makes the
# process s6 supervises BE mjpg_streamer, so s6-svstat reports its pid and a
# stop signals the streamer rather than a shell that has long since gone.
#
# LD_LIBRARY_PATH matters: libjpeg.so.9 lives in the mjpg-streamer directory
# itself. That one directory is prepended here rather than in anvil-env.sh --
# it is a plugin directory with its own libjpeg, and putting it in front of
# every python process on the printer would risk a version conflict to benefit
# one service.
#
# THE BINARY, THE PLUGINS AND -w ARE ABSOLUTE PATHS below even though
# `cd "$CAMDIR"` still happens: relative plugin names worked in the replica
# (glibc's dlopen falls back to LD_LIBRARY_PATH for a bare name) but not on
# real hardware.
MODDIR=/usr/data/anvil
# anvil.conf is read HERE, not inherited. s6-supervise starts this script from
# the SCANNER's environment, which is whatever firmwareExe had at boot -- and
# on a `s6-svc -u` typed over ssh it is whatever that ssh session had, which is
# not the same thing at all. Reading the file in the run script is what makes
# MOD_CAM_RES/FPS/PORT mean the same on both paths, and it is why editing
# anvil.conf and restarting the service is enough to change the resolution.
[ -f $MODDIR/anvil.conf ] && . $MODDIR/anvil.conf
[ -f $MODDIR/anvil-env.sh ] && . $MODDIR/anvil-env.sh
CAMDIR=/usr/prog/mjpg-streamer
VIDEO=/dev/video0
LOG=/usr/data/logs/mjpg.log
RES="${MOD_CAM_RES:-1280x720}"
FPS="${MOD_CAM_FPS:-30}"
PORT="${MOD_CAM_PORT:-8080}"
# The same log file the hand-rolled loop wrote to, for the same reason: it is
# the file the troubleshooting notes name, and mjpg_streamer is chatty enough
# (a banner per start, a line per plugin option) that leaving it on the
# inherited descriptor would put it in the boot log of a boot that finished
# hours ago -- s6-svscan's descriptors outlive firmwareExe's redirection
# exactly as the scanner itself does. Appending, not truncating, so a respawn
# cannot erase the reason for the respawn.
mkdir -p /usr/data/logs
exec >>"$LOG" 2>&1
# The prerequisite. A run script cannot decline to run -- s6 respawns it -- so a
# stock firmware that moved or dropped mjpg-streamer is reported here, with a
# sleep, instead of showing up as a failed exec once a second.
if [ ! -x "$CAMDIR/mjpg_streamer" ]; then
echo "camera: !! no $CAMDIR/mjpg_streamer"
sleep 5
exit 1
fi
# THE UPGRADE PATH. A printer on the pre-s6 payload has our old
# `while true; ... sleep 3` respawn loop alive with its pid in this file; left
# alone it keeps putting mjpg_streamer back on :8080 and fights this service for
# the port, with nothing in any log saying why. The loop dies first -- killing
# only the streamer gets it restarted three seconds later. Delete once no
# pre-s6 install can be upgraded in the field.
if [ -f "$MODDIR/mjpg.pid" ]; then
echo "camera: !! a pre-s6 respawn loop is still running -- stopping it"
kill "`cat "$MODDIR/mjpg.pid" 2>/dev/null`" 2>/dev/null
rm -f "$MODDIR/mjpg.pid"
killall mjpg_streamer 2>/dev/null
sleep 1
fi
# ---- the 30-second wait for /dev/video0 ------------------------------------
#
# The wait is HERE rather than in a caller because only here does giving up
# turn into retrying: the script exits non-zero, s6 runs it again, and a camera
# plugged in ten minutes after boot comes up on its own. Thirty seconds is how
# long a USB camera can take to enumerate on this box.
#
# THE BOOT PAYS FOR IT. s6-rc waits for this service's readiness notification,
# bounded by `timeout-up` (40s) next to this file, so a printer with no camera
# spends that on every boot. Nothing depends on the camera, so nothing else is
# held back; MOD_CAM=0 is the answer for a printer that has none. The
# alternative was a service that lies about being ready.
waited=0
while [ ! -e "$VIDEO" ] && [ $waited -lt 30 ]; do
sleep 1
waited=$((waited + 1))
done
if [ ! -e "$VIDEO" ]; then
# No readiness notification on this path, deliberately: a service that
# declared itself ready here would be telling nginx's /webcam/ proxy there
# is a stream to proxy.
echo "camera: !! $VIDEO never appeared after ${waited}s -- no camera plugged in?"
exit 1
fi
export LD_LIBRARY_PATH=$CAMDIR:$LD_LIBRARY_PATH
cd "$CAMDIR" || exit 1
# ---- readiness ------------------------------------------------------------
#
# `notification-fd` holds "3": the service is READY once a line is written to
# fd 3, and s6-rc waits for that before counting it up.
#
# READY IS "THE HTTP PORT ANSWERS", not "the process was forked" (the gap
# between fork and usable is where a camera that enumerated but cannot do
# 1280x720 dies) and not "/dev/video0 exists" (a precondition -- the node
# appears the instant the kernel enumerates the device). The only consumer is
# nginx's /webcam/ proxy, and what a proxy needs is a socket that accepts.
# output_http.so is started last, so a bound port also means the device opened
# and the format was accepted.
#
# /proc/net/tcp because this busybox has no nc applet (measured): field 2 is
# LOCAL_ADDRESS:PORT with the port in uppercase hex, field 4 is the state, 0A
# is TCP_LISTEN. The address half is not matched, so a future `-l 127.0.0.1`
# still counts as listening.
#
# A subshell, because the last thing this script does is exec and an exec'd
# streamer cannot report on itself. It inherits fd 3; the exec below closes
# fd 3 in the STREAMER (3>&-) so the only process holding the notification pipe
# is the one with something to say. It gives up after 60s rather than looping
# for ever -- a prober that never exits is one stray process per respawn.
PORTHEX=`printf '%04X' "$PORT"`
(
n=0
while [ $n -lt 60 ]; do
if awk -v p=":$PORTHEX" '$2 ~ p"$" && $4 == "0A" { found = 1 }
END { exit !found }' /proc/net/tcp 2>/dev/null
then
echo ready >&3
exit 0
fi
sleep 1
n=$((n + 1))
done
) &
# The exact invocation the hand-rolled loop used, argument for argument. Do not
# "tidy" the quoting: mjpg_streamer takes a plugin and all of its options as
# ONE argv entry and splits it itself, so those quotes are load-bearing.
#
# `nice` execs the target in its own place, so s6 still holds the streamer's
# own pid. NICE_CAM (anvil.conf): the streamer is pure background work and the
# most deserving of the services that yield to klippy. 0 or unset = normal.
exec nice -n "${NICE_CAM:-0}" "$CAMDIR/mjpg_streamer" \
-i "$CAMDIR/input_uvc.so -d $VIDEO -r $RES -f $FPS" \
-o "$CAMDIR/output_http.so -p $PORT -w $CAMDIR/www" 3>&-
pkgs/anvil-core/payload/etc/s6-rc/source/camera/timeout-up
+1
View File
@@ -0,0 +1 @@
40000
pkgs/anvil-core/payload/etc/s6-rc/source/camera/type
+1
View File
@@ -0,0 +1 @@
longrun
pkgs/anvil-core/payload/etc/s6-rc/source/ff-startup/dependencies.d/klipper
View File
Whitespace-only changes.
pkgs/anvil-core/payload/etc/s6-rc/source/ff-startup/dependencies.d/moonraker
View File
Whitespace-only changes.
pkgs/anvil-core/payload/etc/s6-rc/source/ff-startup/timeout-up
+1
View File
@@ -0,0 +1 @@
300000
pkgs/anvil-core/payload/etc/s6-rc/source/ff-startup/type
+1
View File
@@ -0,0 +1 @@
oneshot
pkgs/anvil-core/payload/etc/s6-rc/source/ff-startup/up
+40
View File
@@ -0,0 +1,40 @@
/bin/sh -c "
# Wait until the printer is genuinely up -- klippy answering and moonraker
# serving -- retrying klipper when a board did not come back, and on a first
# boot import this unit's factory calibration. The UI depends on this, so
# HelixScreen starts in front of a working printer rather than a connecting one.
#
# anvil.conf is sourced HERE because it is a runtime read: the compiled database
# is built once, on a build host, and the user edits this file over ssh.
#
# IT NEVER EXITS NON-ZERO -- a timeout would leave the printer with no screen,
# which is worse than the thing it would be reporting.
MODDIR=/usr/data/anvil
[ -f \$MODDIR/anvil.conf ] && . \$MODDIR/anvil.conf
[ -f \$MODDIR/anvil-env.sh ] && . \$MODDIR/anvil-env.sh
# MOD_STARTUP=0 cannot mean 'do not run this service': the UI depends on it, so
# s6-rc pulls it in regardless. It means what it always described -- do not wait
# for the printer, start the screen at once. klipper still starts; the retries,
# the panel narration and the first-boot import are what is skipped.
if [ \"\${MOD_STARTUP:-1}\" != 1 ]; then
echo 'ff-startup: MOD_STARTUP=0 -- not waiting for the printer; the UI starts at once'
exit 0
fi
if [ ! -f \$MODDIR/bin/ff-startup.py ] || [ ! -x \"\$FF_PYTHON\" ]; then
echo 'ff-startup: !! no ff-startup.py or no interpreter -- not waiting for the printer'
exit 0
fi
# --no-bringup because mcu-bringup already did it; retries inside the program
# still redo the handover, which is the half that matters. --no-klipper because
# klipper is a dependency now, not this program's to launch -- it still restarts
# it through s6-svc.
ARGS=\"--no-bringup --no-klipper\"
ARGS=\"\$ARGS --timeout \${MOD_STARTUP_TIMEOUT:-240}\"
ARGS=\"\$ARGS --klipper-tries \${MOD_KLIPPER_TRIES:-6}\"
[ \"\${MOD_IMPORT:-1}\" = 1 ] || ARGS=\"\$ARGS --no-import\"
\"\$FF_PYTHON\" \$MODDIR/bin/ff-startup.py \$ARGS \
|| echo \"ff-startup: !! returned \$? -- the next boot will try again\"
exit 0
"
pkgs/anvil-core/payload/etc/s6-rc/source/klipper/dependencies.d/mcu-bringup
View File
Whitespace-only changes.
pkgs/anvil-core/payload/etc/s6-rc/source/klipper/run
Executable
+78
View File
@@ -0,0 +1,78 @@
#!/bin/sh
# Klippy, as s6 supervises it. Exec'd by s6-supervise, never by hand.
#
# This is FlashForge's klipperDaemon command line without the -b: without it
# busybox start-stop-daemon execs the program in place, so the process
# s6-supervise holds IS klippy rather than a shell that started one. The
# pidfile goes with it -- s6-supervise knows the pid.
#
# THE MCU BRING-UP DOES NOT HAPPEN HERE. It is the `mcu-bringup` oneshot, which
# klipper depends on, so s6-rc runs it before this service starts. Doing it
# here as well would hand the same serial ports to two processes at once.
MODDIR=/usr/data/anvil
# klippy is FlashForge's Python 3.8.2 on the firmware partition, linked against
# libraries in four directories nothing on the base rootfs knows about.
# s6-supervise hands this script the scanner's environment, which is already
# correct on the boot path -- but a service must not work only because the
# shell that started the tree happened to be set up right.
if [ -f $MODDIR/anvil-env.sh ]; then
. $MODDIR/anvil-env.sh
else
echo "klipper/run: !! no $MODDIR/anvil-env.sh -- klippy will not find its libraries"
fi
KLIPPY=/usr/prog/klipper/klippy/klippy.py
PRINTER_CFG=/usr/data/config/printer.cfg
PRINTER_LOG=/usr/data/logs/printer.log
UDS=/tmp/uds
DAEMON=/usr/prog/klipper/klipperDaemon
# klippy does not unlink its socket on exit, and everything on this printer
# reads that socket as "klipper is running". Swept here rather than in a stop()
# because s6 restarts this service without going near one.
rm -f "$UDS"
# A klippy killed with SIGKILL -- an OOM -- leaves a process still holding
# /dev/ttyS4 while s6 starts a second one that cannot open the port and reports
# an MCU error for ever. s6-supervise runs one copy of this service at a time,
# so anything running klippy.py now is a leftover.
#
# /proc rather than `ps | grep`: busybox ps truncates to the terminal width, so
# the command line arrives cut off and the grep matches nothing. Our own pid is
# skipped.
for _kl_proc in /proc/[0-9]*; do
[ -r "$_kl_proc/cmdline" ] || continue
[ "${_kl_proc#/proc/}" = "$$" ] && continue
case "`tr '\0' ' ' < "$_kl_proc/cmdline" 2>/dev/null`" in
*klippy.py*) kill -9 "${_kl_proc#/proc/}" 2>/dev/null ;;
esac
done
unset _kl_proc
# klipper_pri.sh puts klippy's main thread on SCHED_FIFO 50. It cannot run
# before the exec below -- there would be no klippy to find -- and this script
# has no "after", so it waits for the socket that says klippy is serving.
# Bounded, and nothing reads the result: a missing chrt is a slower printer.
if [ -x /usr/prog/klipper/klipper_pri.sh ]; then
( _kp=0
while [ $_kp -lt 60 ] && [ ! -S "$UDS" ]; do
sleep 1
_kp=$((_kp + 1))
done
[ -S "$UDS" ] && sh /usr/prog/klipper/klipper_pri.sh
) >/dev/null 2>&1 &
fi
# FlashForge's nice value, read out of the klipperDaemon the mod ships in its
# place -- run-append.sh carries the number forward off the file it replaces,
# so it is never guessed. A grep rather than a source, because sourcing stock's
# klipperDaemon would run the daemon.
KLIPPER_NICENESS=`sed -n 's/^ *KLIPPER_NICENESS= *\([-0-9][0-9]*\).*/\1/p' \
"$DAEMON" 2>/dev/null | head -n 1`
if [ -z "$KLIPPER_NICENESS" ] || [ "$KLIPPER_NICENESS" = 0 ]; then
exec start-stop-daemon -S --exec "$FF_PYTHON" -- \
"$KLIPPY" "$PRINTER_CFG" -l "$PRINTER_LOG" -a "$UDS"
fi
exec start-stop-daemon -S -N "$KLIPPER_NICENESS" --exec "$FF_PYTHON" -- \
"$KLIPPY" "$PRINTER_CFG" -l "$PRINTER_LOG" -a "$UDS"
pkgs/anvil-core/payload/etc/s6-rc/source/klipper/type
+1
View File
@@ -0,0 +1 @@
longrun
pkgs/anvil-core/payload/etc/s6-rc/source/mcu-bringup/timeout-up
+1
View File
@@ -0,0 +1 @@
60000
pkgs/anvil-core/payload/etc/s6-rc/source/mcu-bringup/type
+1
View File
@@ -0,0 +1 @@
oneshot
pkgs/anvil-core/payload/etc/s6-rc/source/mcu-bringup/up
+37
View File
@@ -0,0 +1,37 @@
/bin/sh -c "
# The toolhead boards, out of their bootloaders, before klippy opens the ports.
# klipper DEPENDS ON THIS, so nothing that starts klipper can skip it --
# including s6 restarting klippy after a crash, which no script of ours is on.
#
# The heater board on /dev/ttyS4 is frequently not answering when klippy first
# opens the port; klippy then gives up after 90s with 'MCU error during connect'
# and neither exits nor retries.
#
# IT NEVER EXITS NON-ZERO. s6-rc does not start what depends on a failed
# service, so a bring-up that reported a problem would leave the printer with no
# klipper at all -- worse than the board it is reporting on.
MODDIR=/usr/data/anvil
[ -f \$MODDIR/anvil-env.sh ] && . \$MODDIR/anvil-env.sh
# cmd_mcu is on the base rootfs, version-matched to the firmware; the mod does
# not ship it. This is the main board (/dev/ttyS2); ff-startup.py does the other
# three.
if [ -x /usr/bin/cmd_mcu ]; then
cmd_mcu write_firmware /usr/prog/libmcu-bare.bin
cmd_mcu bootup
# It takes a moment to leave its bootloader, and everything after this talks
# to serial ports it owns.
sleep 2
else
echo 'mcu-bringup: !! no /usr/bin/cmd_mcu -- the main board is not brought up'
fi
if [ ! -f \$MODDIR/bin/ff-startup.py ] || [ ! -x \"\$FF_PYTHON\" ]; then
echo 'mcu-bringup: !! no ff-startup.py or no interpreter -- the toolhead boards'
echo ' are left to klippy, which names the one that did not answer'
exit 0
fi
\"\$FF_PYTHON\" \$MODDIR/bin/ff-startup.py --only-bringup \
|| echo \"mcu-bringup: !! returned \$? -- going on to klipper anyway\"
exit 0
"
pkgs/anvil-core/payload/etc/s6/moonraker/notification-fd → pkgs/anvil-core/payload/etc/s6-rc/source/moonraker/notification-fd
View File
File renamed without changes.
pkgs/anvil-core/payload/etc/s6-rc/source/moonraker/run
Executable
+149
View File
@@ -0,0 +1,149 @@
#!/bin/sh
# The moonraker service, as s6 runs it. Exec'd by s6-supervise, never by hand.
#
# NO start-stop-daemon AND NO `-b`: the process this script turns into IS the
# service, so a fork-and-return would have s6 supervise the process that
# exited, decide the service died a millisecond after starting, and restart it
# for ever while the real moonraker served :7125 supervised by nobody.
#
# AND `exec`. Without it s6 supervises this shell, and every signal it sends --
# the SIGTERM behind `s6-rc -d change moonraker` above all -- goes to a shell that
# ignores it while moonraker carries on.
#
# THIS FILE DOES NOT LOOK AT MOD_WEB: it runs whenever s6 decides the service
# should be up, and a `run` that exits because it disagrees is a respawn loop
# with extra steps. There is no `down` file next to it either -- MEASURED:
# s6-rc-compile accepts one in a definition directory and SILENTLY DROPS it,
# producing a byte-identical servicedir. The gate is the argument list;
# svc_rc_enabled reads MOD_WEB at runtime.
#
# Moving /usr/data/config is phase 7 of docs/notes/80-s6-migration.md; the `-d`
# below is the one line here that changes when it does.
MODDIR=/usr/data/anvil
# Read HERE, not inherited: s6-supervise starts this script from the SCANNER's
# environment, which is firmwareExe's at boot but whatever an ssh session had
# when the service is restarted by hand. moonraker is the process that dies on
# "libpython3.8.so.1.0: cannot open shared object file" when that is wrong.
[ -f $MODDIR/anvil.conf ] && . $MODDIR/anvil.conf
[ -f $MODDIR/anvil-env.sh ] && . $MODDIR/anvil-env.sh
# One moonraker, on the DATA partition -- /usr/prog is what a stock flash
# overwrites.
MOONRAKER_MAIN=$MODDIR/moonraker/moonraker.py
MOONRAKER_DATA=/usr/data
MOONRAKER_TMP=/usr/data/tmp
MOONRAKER_CONF=$MOONRAKER_DATA/config/moonraker.conf
# TMPDIR: /tmp is a ramdisk here, so a large upload without this fills memory
# instead of the data partition. Set in the run script because s6 restarts this
# service without going near any shell of ours.
#
# The wipe needs no "is it running" guard: s6-supervise runs one copy of this
# service at a time and execs this only after the previous one was reaped.
rm -rf $MOONRAKER_TMP
mkdir -p $MOONRAKER_TMP $MOONRAKER_DATA/logs
export TMPDIR=$MOONRAKER_TMP
# The prerequisites. A run script cannot decline to run -- s6 respawns it -- so
# a missing one is reported and the sleep keeps it to one line every 5 seconds.
if [ ! -x "$FF_PYTHON" ]; then
echo "moonraker: !! no interpreter at ${FF_PYTHON:-/usr/prog/Python-3.8.2/bin/python3}"
sleep 5
exit 1
fi
if [ ! -f "$MOONRAKER_MAIN" ]; then
echo "moonraker: !! nothing at $MOONRAKER_MAIN -- the install did not complete"
sleep 5
exit 1
fi
# THE UPGRADE PATH: a printer on the pre-s6 payload has a start-stop-daemon'd
# moonraker whose pid is in this file. The sweep below kills the process; this
# removes the file so nothing reads a pid the kernel has since reused. Delete
# both once no pre-s6 install can be upgraded in the field.
rm -f /run/moonraker.pid
# A moonraker already holding :7125 -- started by hand over ssh, or left by the
# pre-s6 payload on a printer upgraded without a reboot -- makes this service
# fail to bind, exit, get respawned, and loop for as long as the printer is on,
# with s6-svstat saying "up" while an unsupervised leftover answers Mainsail.
#
# /proc rather than `ps | grep`: busybox ps truncates to the terminal width, so
# a long python command line arrives cut in half. The match is the absolute
# path of the entry point, which appears in a moonraker's command line and in
# no other process here -- `s6-supervise moonraker` and this script have the
# word but not the path. Our own pid is skipped.
for _mr_proc in /proc/[0-9]*; do
[ -r "$_mr_proc/cmdline" ] || continue
[ "${_mr_proc#/proc/}" = "$$" ] && continue
_mr_args=`tr '\0' ' ' < "$_mr_proc/cmdline" 2>/dev/null`
case "$_mr_args" in
*"$MOONRAKER_MAIN"*) kill -9 "${_mr_proc#/proc/}" 2>/dev/null ;;
esac
done
unset _mr_proc _mr_args
# ---- readiness ------------------------------------------------------------
#
# `notification-fd` holds "3": the service is READY once a line is written to
# fd 3. s6-rc waits for that before counting this service up, and ff-startup
# depends on it -- so "ready" has to mean something a caller can use.
#
# READY IS "THE HTTP API IS LISTENING", not "the process was forked" (component
# imports fail seconds later) and NOT "klippy_state is ready" -- moonraker
# reporting klippy_state=error IS moonraker doing its job, so that definition
# would never fire on precisely the broken printer whose UI somebody needs.
#
# /proc/net/tcp because this busybox has no nc applet (measured): field 2 is
# LOCAL_ADDRESS:PORT with the port in uppercase hex, field 4 is the state, 0A
# is TCP_LISTEN. The address half is not matched -- a user who sets
# host: 127.0.0.1 should still count as listening.
#
# The port is read from moonraker's own config because it is user-editable and
# a probe watching the wrong port would never fire: s6-rc would burn the whole
# `timeout-up` on a working printer and skip everything that depends on it. The
# awk is section-aware so a `port:` in another component cannot be picked up.
MOONRAKER_PORT=`awk '
/^[[:space:]]*\[/ { section = $0 }
section ~ /\[[[:space:]]*server[[:space:]]*\]/ && /^[[:space:]]*port[[:space:]]*:/ {
sub(/^[^:]*:[[:space:]]*/, "");
gsub(/[^0-9]/, "");
if (length($0)) { print; exit }
}' "$MOONRAKER_CONF" 2>/dev/null`
[ -n "$MOONRAKER_PORT" ] || MOONRAKER_PORT=7125
# A subshell, because the last thing this script does is exec and an exec'd
# moonraker cannot report on itself. It inherits fd 3; the exec below closes
# fd 3 in MOONRAKER (3>&-) so the only process holding the notification pipe is
# the one with something to say -- otherwise s6 never sees the pipe close when
# the prober gives up.
#
# 180s: moonraker imports its whole component tree on a 1GHz mipsel box off
# eMMC, and slower again on the qemu replica. Giving up sends no notification,
# and s6-rc then gives up at `timeout-up` (120s, in the file next to this).
(
_mr_n=0
while [ $_mr_n -lt 180 ]; do
if awk -v p=":`printf '%04X' "$MOONRAKER_PORT"`" \
'$2 ~ p"$" && $4 == "0A" { found = 1 }
END { exit !found }' /proc/net/tcp 2>/dev/null
then
echo ready >&3
exit 0
fi
sleep 1
_mr_n=$((_mr_n + 1))
done
) &
# `-d /usr/data` is moonraker's data path -- config/moonraker.conf,
# logs/moonraker.log and its database. Not optional: moonraker defaults to
# ~/printer_data, a home directory that does not exist here.
#
# stderr goes wherever the scanner's does ($SVC_S6_LOG), which is where a
# moonraker that dies before its own logging is configured explains itself.
#
# `nice` sets its own priority and then execs in place (verified: the pid s6
# holds is the interpreter's, not nice's), so this is still one process.
# NICE_MOONRAKER protects klippy from this process; 0 or unset = normal.
exec nice -n "${NICE_MOONRAKER:-0}" "$FF_PYTHON" "$MOONRAKER_MAIN" -d "$MOONRAKER_DATA" 3>&-
pkgs/anvil-core/payload/etc/s6-rc/source/moonraker/timeout-up
+1
View File
@@ -0,0 +1 @@
120000
pkgs/anvil-core/payload/etc/s6-rc/source/moonraker/type
+1
View File
@@ -0,0 +1 @@
longrun
pkgs/anvil-core/payload/etc/s6-rc/source/nginx/run
Executable
+81
View File
@@ -0,0 +1,81 @@
#!/bin/sh
# The nginx service, as s6 runs it. Exec'd by s6-supervise, never by hand.
#
# THEREFORE `daemon off;`. nginx's default is to fork a master into the
# background and return, which under a supervisor means s6 watches the process
# that forked and exited, decides the service died a millisecond after starting
# it, and restarts it for ever while the real nginx serves on, supervised by
# nobody. `-g` puts the directive on the command line so the config file stays
# shared with a hand-run nginx. Verified on the replica against this printer's
# own nginx (1.5.0, FlashForge's build): the pid we launched is still the
# master five seconds later.
#
# AND `exec`. Without it s6 supervises this shell, and every signal it sends --
# the SIGTERM behind `s6-rc -d change nginx` above all -- goes to a shell that
# ignores it while nginx carries on.
#
# THIS FILE DOES NOT LOOK AT MOD_WEB: it runs whenever s6 decides the service
# should be up, and a `run` that exits because it disagrees is a respawn loop
# with extra steps. There is no `down` file next to it either -- MEASURED:
# s6-rc-compile accepts one in a definition directory and SILENTLY DROPS it,
# producing a byte-identical servicedir. The gate is the argument list;
# svc_rc_enabled reads MOD_WEB at runtime.
MODDIR=/usr/data/anvil
NGINX=/usr/prog/nginx/sbin/nginx
# -p sets the prefix: logs, the pidfile and the temp dirs land under $MODDIR on
# the data partition, never on /usr/prog. nginx will not start if it cannot
# create them and does not create them itself.
mkdir -p $MODDIR/nginx/logs $MODDIR/nginx/tmp
# The prerequisites. A run script cannot decline to run -- s6 respawns it -- so
# a missing one is reported and the sleep keeps it to one line every 5 seconds
# instead of one a second in a log nobody is reading.
if [ ! -x "$NGINX" ]; then
echo "nginx: !! no $NGINX -- this printer's firmware did not ship one"
sleep 5
exit 1
fi
if [ ! -f $MODDIR/nginx/nginx.conf ]; then
echo "nginx: !! no $MODDIR/nginx/nginx.conf -- the install did not complete"
sleep 5
exit 1
fi
# nginx writes this even under `daemon off;` (measured), and nothing reads it.
# Removed so nothing on this printer ever starts to.
rm -f $MODDIR/nginx/logs/nginx.pid
# THE ORPHANED WORKER. nginx is a master that binds :80 and a worker that
# inherits the listening socket. TERM the master -- what `s6-rc -d change`
# does -- and the worker goes with it. SIGKILL it, which is what an OOM does,
# and the worker is orphaned, KEEPS THE SOCKET, and goes on serving: s6 then
# starts a new nginx every second that cannot bind, while s6-svstat says "up"
# and an unsupervised leftover answers the browser. All measured on the replica.
#
# s6-supervise runs one copy of this service at a time, so anything speaking
# for nginx now is a leftover.
#
# /proc rather than `ps | grep`: busybox ps truncates to the terminal width, so
# on the replica the command line arrived cut off at "/usr/prog/nginx/" and the
# grep matched nothing; and busybox kill will not take a negative pid, so
# signalling the process group is not available either. Both were tried.
#
# The match is POSITIVE and narrow, because a wrong match here kills the wrong
# thing: "s6-supervise nginx" and this very script both have
# the word nginx in their command lines, and s6-supervise is the process that
# would otherwise restart us. Only the two shapes nginx itself produces are
# swept -- "nginx: master process ..." / "nginx: worker process" (it rewrites
# its own argv) and anything exec'd from the binary before it does.
for _ng_proc in /proc/[0-9]*; do
[ -r "$_ng_proc/cmdline" ] || continue
[ "${_ng_proc#/proc/}" = "$$" ] && continue
_ng_args=`tr '\0' ' ' < "$_ng_proc/cmdline" 2>/dev/null`
case "$_ng_args" in
"nginx: "*|*"$NGINX"*) kill -9 "${_ng_proc#/proc/}" 2>/dev/null ;;
esac
done
unset _ng_proc _ng_args
# stderr is where nginx puts the reason it would not start, and it lands in the
# scanner's log -- a file that outlives the boot.
exec $NGINX -p $MODDIR/nginx -c $MODDIR/nginx/nginx.conf -g "daemon off;"
pkgs/anvil-core/payload/etc/s6-rc/source/nginx/type
+1
View File
@@ -0,0 +1 @@
longrun
pkgs/anvil-core/payload/etc/s6-rc/source/ok-all/contents.d/camera
View File
Whitespace-only changes.
pkgs/anvil-core/payload/etc/s6-rc/source/ok-all/contents.d/ff-startup
View File
Whitespace-only changes.
pkgs/anvil-core/payload/etc/s6-rc/source/ok-all/contents.d/klipper
View File
Whitespace-only changes.
pkgs/anvil-core/payload/etc/s6-rc/source/ok-all/contents.d/moonraker
View File
Whitespace-only changes.
pkgs/anvil-core/payload/etc/s6-rc/source/ok-all/contents.d/nginx
View File
Whitespace-only changes.
pkgs/anvil-core/payload/etc/s6-rc/source/ok-all/contents.d/ui
View File
Whitespace-only changes.
pkgs/anvil-core/payload/etc/s6-rc/source/ok-all/contents.d/wifi
View File
Whitespace-only changes.
pkgs/anvil-core/payload/etc/s6-rc/source/ok-all/type
+1
View File
@@ -0,0 +1 @@
bundle
pkgs/anvil-core/payload/etc/s6-rc/source/ui/dependencies.d/ff-startup
View File
Whitespace-only changes.
pkgs/anvil-core/payload/etc/s6-rc/source/ui/run
Executable
+58
View File
@@ -0,0 +1,58 @@
#!/bin/sh
# HelixScreen, as s6 supervises it. Exec'd by s6-supervise, never by hand.
#
# `exec` at the bottom is what makes the process s6 supervises BE the UI, so
# s6-svstat reports its pid and `s6-rc -d change ui` signals it rather than a
# shell that has long since gone.
#
# BLOCKING UNKNOWN: nothing has proved helix-screen runs under a supervisor. It
# gets the descriptors s6-supervise hands it, no controlling terminal, and a
# session it did not create. The environment below is the easy half; a tty is
# not something a run script can conjure. If one turns out to be needed, the
# answer is s6's own tty handling or a small wrapper -- the foreground this
# used to run in no longer exists. See docs/notes/80-s6-migration.md.
MODDIR=/usr/data/anvil
if [ -f $MODDIR/anvil-env.sh ]; then
. $MODDIR/anvil-env.sh
else
echo "ui/run: !! no $MODDIR/anvil-env.sh -- HelixScreen may not find its libraries"
fi
# HELIX_DATA_DIR is the root HelixScreen resolves config/settings.json against,
# which is why run-append.sh backs that tree up across an update.
HELIX_DATA_DIR=$MODDIR/helixscreen
HELIX_LOG_DEST=file
HELIX_LOG_FILE=/usr/data/logs/helixscreen.log
# This repo pins and patches the HelixScreen build it ships; an in-app update
# would replace it behind the package's back. HelixScreen documents this
# variable as the vendor opt-out.
HELIX_DISABLE_AUTO_UPDATES=1
export HELIX_DATA_DIR HELIX_LOG_DEST HELIX_LOG_FILE HELIX_DISABLE_AUTO_UPDATES
mkdir -p /usr/data/logs
# The prerequisite. A run script cannot decline to run -- s6 respawns it -- so
# a missing HelixScreen is reported with a sleep instead of showing up as a
# failed exec once a second, which is a printer that got hot and never drew
# anything. MOD_UI=0 is a different thing and is handled in firmwareExe, which
# simply leaves this service out of the transition.
if [ ! -x $MODDIR/helixscreen/bin/helix-launcher.sh ] \
&& [ ! -x $MODDIR/helixscreen/bin/helix-screen ]; then
echo "ui: !! HelixScreen is not installed under $MODDIR/helixscreen"
sleep 5
exit 1
fi
# s6-supervise runs one copy of this service at a time, so anything answering
# to these names is a leftover -- a UI started by hand over ssh -- and two
# processes fighting over /dev/fb0 is a black screen with no explanation.
killall helix-screen helix-watchdog helix-splash 2>/dev/null
# The launcher is what a HelixScreen release ships to set up its own
# environment; skipping it silently would change behaviour on the next release
# rather than on this one.
if [ -x $MODDIR/helixscreen/bin/helix-launcher.sh ]; then
exec $MODDIR/helixscreen/bin/helix-launcher.sh
fi
exec $MODDIR/helixscreen/bin/helix-screen
pkgs/anvil-core/payload/etc/s6-rc/source/ui/type
+1
View File
@@ -0,0 +1 @@
longrun
pkgs/anvil-core/payload/etc/s6-rc/source/wifi/down
+16
View File
@@ -0,0 +1,16 @@
/bin/sh -c "
# By name: nothing here was started with a pidfile, and inventing one for a
# daemon that backgrounds itself would be a second thing that can be wrong.
killall wpa_cli 2>/dev/null
killall udhcpc 2>/dev/null
killall wpa_supplicant 2>/dev/null
# killall returns as soon as the signal is sent, so without this wait a
# reconnect a moment later finds the dying supplicant in the process table and
# does nothing.
n=0
while [ \$n -lt 15 ] && { pgrep -f wpa_supplicant >/dev/null 2>&1 || ps 2>/dev/null | grep wpa_supplican[t] >/dev/null; }; do
sleep 1
n=\$((n + 1))
done
echo 'wifi: stopped'
"
pkgs/anvil-core/payload/etc/s6-rc/source/wifi/type
+1
View File
@@ -0,0 +1 @@
oneshot
pkgs/anvil-core/payload/etc/s6-rc/source/wifi/up
+58
View File
@@ -0,0 +1,58 @@
/bin/sh -c "
# wlan0 + wpa_supplicant + udhcpc. Nothing in /etc/init.d brings the interface
# up: FlashForge's firmwareExe owned the whole network stack, and app_startup.sh
# only insmods the 8821cu driver. So the moment the mod replaces firmwareExe the
# printer boots with no ssh, no Mainsail and no way to diagnose either.
#
# A ONESHOT, not a longrun: wpa_supplicant backgrounds itself with -B and so
# does udhcpc, so there is no process for s6-supervise to hold.
#
# THE WAITS ARE BACKGROUNDED and this returns at once, because a oneshot's up
# runs inside the transition firmwareExe is blocked on -- up to 15s for wlan0 to
# register after the insmod and 20s more to associate, and that delay would land
# on exactly the printer with no network. Returning early is lying to whatever
# depends on this, so NOTHING DEPENDS ON WIFI, deliberately: moonraker and nginx
# bind 0.0.0.0 and klipper talks to serial ports.
#
# The saved networks live in /usr/prog/wifi/wpa_supplicant.conf, which the mod
# never ships or touches, so whatever FlashForge's UI saved is still there.
IFACE=wlan0
CONF=/usr/prog/wifi/wpa_supplicant.conf
if pgrep -f wpa_supplicant >/dev/null 2>&1 || ps 2>/dev/null | grep wpa_supplican[t] >/dev/null; then
echo 'wifi: wpa_supplicant already running'
exit 0
fi
[ -f \"\$CONF\" ] || { echo \"wifi: !! no \$CONF -- nothing to connect to\"; exit 0; }
(
n=0
while [ \$n -lt 15 ] && [ ! -d /sys/class/net/\$IFACE ]; do
sleep 1
n=\$((n + 1))
done
if [ ! -d /sys/class/net/\$IFACE ]; then
echo \"wifi: !! \$IFACE never appeared -- is 8821cu.ko loaded?\"
exit 1
fi
/sbin/ifconfig \$IFACE up 2>/dev/null
mkdir -p /var/run/wpa_supplicant
# nl80211 first, wext as the fallback: the 8821cu is a Realtek USB part and
# which one it registers varies by build.
if /usr/sbin/wpa_supplicant -B -i \$IFACE -c \"\$CONF\" -D nl80211,wext; then
echo 'wifi: wpa_supplicant started'
else
echo 'wifi: !! wpa_supplicant failed to start'
exit 1
fi
n=0
while [ \$n -lt 20 ]; do
/usr/sbin/wpa_cli -i \$IFACE status 2>/dev/null | grep -q '^wpa_state=COMPLETED' && break
sleep 1
n=\$((n + 1))
done
/sbin/udhcpc -i \$IFACE -b -t 5 >/dev/null 2>&1 &
echo \"wifi: udhcpc requesting a lease on \$IFACE\"
) &
echo 'wifi: bringing up wlan0 in the background'
"
pkgs/anvil-core/payload/etc/s6/camera/down
-24
View File
@@ -1,24 +0,0 @@
# The presence of this file is the whole message: s6-supervise starts a service
# whose directory contains a `down` file in the DOWN state. s6 never reads the
# contents, so this text is here purely for whoever finds the file on a printer
# and wants to know why it exists.
#
# WHY THE CAMERA SHIPS DOWN. The moment s6-svscan notices this directory it
# starts an s6-supervise for it, and without this file that supervise would
# launch mjpg-streamer immediately -- before payload/init.d/S65camera has
# looked for the video device, checked that FlashForge's streamer binary is
# still on the firmware partition where it expects it, or read the resolution
# and port out of anvil.conf. Started that early it would fail on a printer
# with no camera, be restarted, and fail again, filling the boot log with the
# one message that is guaranteed not to be the actual problem.
#
# This used to be argued from a switch, MOD_CAM, which no longer exists. The
# argument survives it unchanged, because it was never about the switch: it is
# that the SCANNER cannot make this decision and S65camera can. With this file
# the service sits down and idle, costing one ~30KB supervisor and nothing
# else, until S65camera brings it up with `s6-svc -u`.
#
# The state is not sticky across boots, and that is correct: `down` is consulted
# by s6-supervise when the scandir is scanned, which happens fresh on every
# boot, so a camera brought up by hand for a test does not silently outlive the
# session it was brought up in.
pkgs/anvil-core/payload/etc/s6/camera/run
-179
View File
@@ -1,179 +0,0 @@
#!/bin/sh
# The camera, as s6 supervises it: mjpg_streamer on :8080, in the FOREGROUND.
#
# WHAT THIS REPLACES. Until phase 4 this was a `while true; do mjpg_streamer
# ...; sleep 3; done` inside a background subshell of payload/init.d/S65camera,
# started with svc_detach and remembered by a pidfile that held THE LOOP rather
# than the streamer -- because killing only the streamer got it restarted three
# seconds later. All of that is gone. s6-supervise runs this script, notices
# when it exits, and runs it again; the respawn loop, the pidfile, the "which
# of the two processes do I kill first" ordering and the 15-second
# wait-for-it-to-die in stop() are the supervisor's problem now, in C, once,
# instead of ours in ash, per service.
#
# So the ONE rule this file has to keep is: never fork the streamer, never
# background it, never return while the camera is meant to be running. `exec`
# at the bottom is what makes the process s6 supervises BE mjpg_streamer, so
# that s6-svstat reports the streamer's own pid and `s6-svc -d` signals the
# streamer rather than a shell that has long since gone.
#
# EVERYTHING BELOW THE WAIT IS COPIED FROM THE OLD supervise(), DELIBERATELY.
# LD_LIBRARY_PATH matters: libjpeg.so.9 lives in the mjpg-streamer directory
# itself, so without it the binary dies on a missing shared object. That one
# directory is prepended here rather than in anvil-env.sh, which carries the
# shared list: it is a plugin directory with its own libjpeg, and putting it
# in front of every python process on the printer would risk a version
# conflict to benefit one service.
#
# THE BINARY, THE PLUGINS AND -w ARE ALL ABSOLUTE PATHS BELOW, deliberately,
# even though `cd "$CAMDIR"` still happens: relative plugin names worked in
# the replica (glibc's dlopen falls back to LD_LIBRARY_PATH for a bare name)
# but not on real hardware, reported and confirmed fixed by users running the
# equivalent invocation by hand with every path spelled out. Whatever this
# printer's dlopen actually does with a bare "input_uvc.so", it is not worth
# relying on again -- the cwd is kept only because some future flag may still
# want it, not because anything below reads it.
MODDIR=/usr/data/anvil
# anvil.conf is read HERE, not inherited. s6-supervise starts this script from
# the SCANNER's environment, which is whatever firmwareExe had at boot -- and
# on a `s6-svc -u` typed over ssh it is whatever that ssh session had, which is
# not the same thing at all. Reading the file in the run script is what makes
# MOD_CAM_RES/FPS/PORT mean the same on both paths, and it is why editing
# anvil.conf and restarting the service is enough to change the resolution.
[ -f $MODDIR/anvil.conf ] && . $MODDIR/anvil.conf
[ -f $MODDIR/anvil-env.sh ] && . $MODDIR/anvil-env.sh
CAMDIR=/usr/prog/mjpg-streamer
VIDEO=/dev/video0
LOG=/usr/data/logs/mjpg.log
RES="${MOD_CAM_RES:-1280x720}"
FPS="${MOD_CAM_FPS:-30}"
PORT="${MOD_CAM_PORT:-8080}"
# The same log file the hand-rolled loop wrote to, for the same reason: it is
# the file the troubleshooting notes name, and mjpg_streamer is chatty enough
# (a banner per start, a line per plugin option) that leaving it on the
# inherited descriptor would put it in the boot log of a boot that finished
# hours ago -- s6-svscan's descriptors outlive firmwareExe's redirection
# exactly as the scanner itself does. Appending, not truncating, so a respawn
# cannot erase the reason for the respawn.
mkdir -p /usr/data/logs
exec >>"$LOG" 2>&1
# ---- the 30-second wait for /dev/video0 ------------------------------------
#
# WHY THE WAIT LIVES HERE AND NOT IN S65camera. It used to be the first half of
# the detached supervise(): poll for the device, give up after 30s, print
# "no camera plugged in?" and never try again until somebody re-ran the script.
# The other candidate home for it was a `svc_s6_ready camera 30` in the init
# script -- wait in the caller instead of the callee. This is the callee, and
# the reason is that only here does giving up turn into RETRYING:
#
# * The device is not a boot-time fact. A USB camera plugged in ten minutes
# after boot, or unplugged and replugged mid-print, used to mean a dark
# camera until somebody ssh'd in and typed `S65camera restart`. Here the
# script exits non-zero, s6 runs it again, and the next thirty seconds of
# waiting cost one sleeping shell. The retry is free because it is exactly
# what the supervisor is already doing for a streamer that dies.
# * Waiting in the init script would put a 30-second block on the boot path
# of precisely the printer with no camera -- the machine somebody needs to
# ssh into to find out why. S65camera does still wait, but DETACHED and
# only in order to report; nothing on the boot path blocks on it.
# * A caller-side wait happens once. This one is the service's own
# precondition, and a precondition belongs with the thing it precedes.
#
# 30 seconds is the number the old poll used and there is no reason to move it:
# it is how long a USB camera can take to enumerate on this box.
waited=0
while [ ! -e "$VIDEO" ] && [ $waited -lt 30 ]; do
sleep 1
waited=$((waited + 1))
done
if [ ! -e "$VIDEO" ]; then
# No readiness notification is sent on this path, and that is the point:
# `s6-svwait -U` in the caller keeps blocking, times out, and S65camera
# prints the honest line. A service that declared itself ready here would
# be telling nginx's /webcam/ proxy there is a stream to proxy.
echo "camera: !! $VIDEO never appeared after ${waited}s -- no camera plugged in?"
exit 1
fi
export LD_LIBRARY_PATH=$CAMDIR:$LD_LIBRARY_PATH
cd "$CAMDIR" || exit 1
# ---- readiness: fd 3, and what "ready" is allowed to mean ------------------
#
# The `notification-fd` file next to this script holds "3", which is s6's
# contract: this service is READY once a line is written to file descriptor 3,
# and `s6-svwait -U` blocks until it is. This is the one thing s6 has that
# runit has no concept of, and it is the reason the supervisor choice went the
# way it did -- see tools/supervisor/README.md.
#
# WHEN IS THE CAMERA READY? Three candidate answers, and only one of them is
# worth anything to whoever asks:
#
# "the process was forked" -- worthless. That is what runit's `sv start`
# already gives you and what a pidfile already told us. mjpg_streamer
# forks, opens /dev/video0, negotiates a pixel format and a resolution,
# and only then binds its port; the gap between fork and usable is
# exactly where a camera that enumerated but cannot do 1280x720 dies.
# "/dev/video0 exists" -- better, and it is what the old poll checked,
# but it is a PRECONDITION, not readiness. The node appears the instant
# the kernel enumerates the device, before anything has proved it can be
# opened, and it is equally there for a webcam whose format negotiation
# is about to fail.
# "the HTTP port answers" -- this one. The only consumer of this service
# is nginx's /webcam/ proxy (moonraker.conf's [webcam] entry points at
# the proxy rather than at :8080, so that a tunnel forwarding one port
# still shows the stream), and what a proxy needs is a socket that
# accepts. By the time mjpg_streamer has bound the port it has already
# opened the device and set the format -- output_http.so is started
# last. So this is the strongest claim available and the only one that
# means what a caller thinks it means.
#
# HOW IT IS CHECKED, and why it needs no tool. /proc/net/tcp lists every TCP
# socket on the box: field 2 is LOCAL_ADDRESS:PORT with the port in uppercase
# hex, field 4 is the state, and 0A is TCP_LISTEN. No nc, no curl, no wget --
# measured on the replica, whose busybox has no nc applet at all, so a
# shell-level port test that reached for one would have failed on the printer
# and nowhere else. The address half is deliberately not matched:
# mjpg_streamer binds 0.0.0.0 today, and a future `-l 127.0.0.1` should still
# count as listening.
#
# The prober is a subshell because the last thing this script does is exec, and
# an exec'd mjpg_streamer cannot report on itself. It inherits fd 3 across the
# fork; the exec below closes fd 3 in the STREAMER (3>&-) so the only process
# holding the notification pipe is the one with something to say about it. It
# gives up after 60s rather than looping for ever, because a prober that never
# exits is one stray process per respawn -- and giving up simply means no
# notification is ever sent, which is the same honest outcome as the no-device
# path above.
PORTHEX=`printf '%04X' "$PORT"`
(
n=0
while [ $n -lt 60 ]; do
if awk -v p=":$PORTHEX" '$2 ~ p"$" && $4 == "0A" { found = 1 }
END { exit !found }' /proc/net/tcp 2>/dev/null
then
echo ready >&3
exit 0
fi
sleep 1
n=$((n + 1))
done
) &
# The exact invocation the hand-rolled loop used, argument for argument. Do not
# "tidy" the quoting: mjpg_streamer takes a plugin and all of its options as
# ONE argv entry and splits it itself, so those quotes are load-bearing.
#
# nice -n "${NICE_CAM:-0}" in front: the pre-s6 respawn loop this replaced
# used the same applet for the same reason (NICE_CAM in anvil.conf -- the
# streamer is pure background work and the most deserving of the two
# services that yield to klippy), and `nice` execs the target in its own
# place, so s6 still ends up holding the streamer's own pid, re-niced on
# every respawn because s6-supervise re-execs this whole script each time.
# 0 or unset = normal priority.
exec nice -n "${NICE_CAM:-0}" "$CAMDIR/mjpg_streamer" \
-i "$CAMDIR/input_uvc.so -d $VIDEO -r $RES -f $FPS" \
-o "$CAMDIR/output_http.so -p $PORT -w $CAMDIR/www" 3>&-
pkgs/anvil-core/payload/etc/s6/moonraker/down
-51
View File
@@ -1,51 +0,0 @@
This file's presence, not its contents, is the whole message: s6-supervise
starts a service whose directory contains a file called `down` in the DOWN
state instead of starting it. s6 never opens the file, so the reasoning can
live in it rather than in a comment somewhere else that would drift away from
it. (case-moonraker.sh checks that too, with this text in place, because a
fact this file depends on is worth measuring rather than believing.)
WHY MOONRAKER SHIPS DOWN. It is ORDERING. A service directory that ships
without this file is a service the scanner starts a fraction of a second after
init.d/S40s6 runs -- before init.d/S62moonraker has checked a single one of its
prerequisites, before init.d/S60nginx has put a web server in front of it, and
before init.d/S70klipper has started the klippy it wants to talk to.
So the shipped state is "down", meaning: THE SCANNER DOES NOT DECIDE WHEN
MOONRAKER STARTS. init.d/S62moonraker does, at the point in the boot where the
things it depends on are actually there, by calling svc_s6_up -- which
overrides this file for as long as that s6-supervise lives.
This used to be argued from a switch, MOD_WEB, which no longer exists. Nothing
above depended on it: the reason the scandir cannot decide is that the scandir
runs before the boot does, and that was true before there was a flag and is
true now that there is not one.
IT MATTERS MORE HERE THAN ANYWHERE ELSE, and that is worth saying plainly.
Moonraker is not a service that idles when it is started too early: it opens
its database, binds :7125, and connects to klippy's socket at /tmp/uds. A
scanner-started moonraker would be doing all of that before S60nginx, before
S70klipper and before anything had decided whether klippy is even meant to
run -- and the copy that init.d/S62moonraker started afterwards would then be
the SECOND one, fighting the first for the port, in the exact respawn loop the
sweep at the top of the `run` script exists to prevent. The `down` file is what
keeps that loop hypothetical.
THE COST, stated honestly, because it is the one thing this arrangement gives
up. `down` is consulted when s6-supervise STARTS, not when the service dies:
once S62moonraker has brought moonraker up, s6 keeps restarting it on every
crash for as long as that supervisor lives, which is the entire point of the
migration and is unaffected by this file. What it does mean is that if
s6-supervise ITSELF is killed -- not moonraker, the supervisor -- s6-svscan
starts a fresh one, that fresh one reads this file, and moonraker stays down
until something runs `S62moonraker start` again. That is a rare failure
(s6-supervise is a small C program that blocks in poll() and is not the thing
that crashes) with an honest signal (`S62moonraker status` says it is not
running, and says so from s6-svstat rather than from a pidfile) and a one-line
fix from an ssh session. The alternative -- deleting this file at runtime once
the boot has got far enough -- buys back that corner case at the price of a
payload that rewrites its own install tree, which every path in this mod is
arranged not to do.
This file is not a record of anything having gone wrong: it is the state
moonraker ships in, on every build, and nothing removes it.
pkgs/anvil-core/payload/etc/s6/moonraker/run
-258
View File
@@ -1,258 +0,0 @@
#!/bin/sh
# The moonraker service, as s6 runs it. Exec'd by s6-supervise, never by hand.
#
# WHAT A `run` SCRIPT IS. s6-svscan sees this directory in the scandir and
# starts one s6-supervise for it; that s6-supervise forks, puts the child in a
# process group of its own, and execs this file. The process this script turns
# into IS the service: when it exits, s6-supervise starts the whole thing
# again. Everything below follows from that one sentence.
#
# THEREFORE: NO start-stop-daemon, and no `-b`. init.d/S62moonraker used to
# start moonraker with `start-stop-daemon -S -b -m -p /run/moonraker.pid`,
# which forks the server into the background and returns. Under a supervisor
# that is exactly wrong: s6 would supervise the process that forked and exited,
# decide the service had died a millisecond after starting it, and start it
# again for ever, while the real moonraker it left behind went on serving
# :7125 supervised by nobody. So the invocation at the bottom is the same
# interpreter, the same entry point and the same `-d` argument as the old
# daemon line -- argument for argument -- with the backgrounding removed and
# `exec` in front.
#
# AND `exec`. Without it this shell would stay around as moonraker's parent, s6
# would be supervising the shell, and every signal it sent -- the SIGTERM
# behind `S62moonraker stop` above all -- would go to a shell that ignores it
# while moonraker carried on. exec makes the pid s6 holds the pid of the python
# process itself, which is also what makes `s6-svstat -p` the honest answer to
# "which moonraker is this printer running?" that the pidfile used to give.
#
# THIS FILE DECIDES NOTHING. It runs whenever s6 decides the service should be
# up, and a `run` that exits because it disagrees is a respawn loop with extra
# steps. What decides is the `down` file next to this one plus
# init.d/S62moonraker, which picks the point in the boot where starting a
# server that wants klippy and a web front end means something. See the comment
# inside `down`; the whole story is written there.
#
# NOTHING HERE MOVES. The interpreter, the entry point, the data directory and
# the config moonraker reads out of it are all exactly where init.d/S62moonraker
# had them. Moving /usr/data/config is phase 7 of docs/notes/80-s6-migration.md
# and it is deliberately not this change: FlashForge's own klipperDaemon
# hardcodes /usr/data/config/printer.cfg, so the directory cannot move until we
# own the klippy launch as well. When it does move, the `-d` below is the one
# line in this file that changes.
MODDIR=/usr/data/anvil
# anvil.conf and anvil-env.sh are read HERE, not inherited, for the reason the
# camera's run script gives at the same point: s6-supervise starts this script
# from the SCANNER's environment, which is whatever firmwareExe had at boot --
# and on an `s6-svc -u` typed over ssh it is whatever that ssh session had,
# which is not the same thing at all. anvil-env.sh is the file that defines
# FF_PYTHON and LD_LIBRARY_PATH, and moonraker is the process that dies with
# "libpython3.8.so.1.0: cannot open shared object file" when that is wrong --
# so a supervised respawn that inherited a stranger's environment would be a
# service that works on the boot path and fails on every restart afterwards.
[ -f $MODDIR/anvil.conf ] && . $MODDIR/anvil.conf
[ -f $MODDIR/anvil-env.sh ] && . $MODDIR/anvil-env.sh
# ONE moonraker, at one path, and that path is on the DATA partition. The long
# version is in init.d/S62moonraker and it has not changed: the byte-identical
# second copy under /usr/prog is gone, because /usr/prog is the partition a
# stock FlashForge flash overwrites, and running from there made "which
# moonraker is this printer running?" depend on what was flashed last.
MOONRAKER_MAIN=$MODDIR/moonraker/moonraker.py
MOONRAKER_DATA=/usr/data
MOONRAKER_TMP=/usr/data/tmp
MOONRAKER_CONF=$MOONRAKER_DATA/config/moonraker.conf
# TMPDIR, which is one of the exactly two things FlashForge's moonrakerDaemon
# did for us that were worth keeping -- the other was the pidfile, and s6 has
# made that one redundant. Moonraker writes temporary files, uploads among
# them, and /tmp on this box is a ramdisk, so a large upload without this fills
# memory instead of the data partition. It is exported here rather than left to
# the init script because s6 restarts this service without going anywhere near
# init.d/S62moonraker: a service that only has its environment when a shell
# script happened to run first is not supervised, it is lucky.
#
# The wipe is unconditional here where the init script had to guard it on
# "moonraker is not already running". That guard existed because re-running
# S62moonraker over ssh against a LIVE server would otherwise delete the
# directory it was streaming an upload into. Under s6 it is satisfied by
# construction: s6-supervise runs exactly one copy of this service at a time
# and only execs this script after the previous one has been reaped, so at the
# moment this line runs there is no moonraker of ours anywhere.
rm -rf $MOONRAKER_TMP
mkdir -p $MOONRAKER_TMP $MOONRAKER_DATA/logs
export TMPDIR=$MOONRAKER_TMP
# ---- the leftover from an unsupervised start -------------------------------
#
# WHY THIS IS HERE. init.d/S62moonraker still has a fallback that starts
# moonraker directly when there is no scanner -- see the long argument in that
# file -- so a printer can reach the moment when the supervised service comes
# up with a hand-started moonraker already holding :7125. It can also reach it
# by way of an upgrade: a printer running the pre-phase-5 payload has a
# start-stop-daemon'd moonraker alive with its pid in /run/moonraker.pid, and
# installing this payload and typing `S62moonraker restart` rather than
# rebooting would leave it exactly there.
#
# What follows is not a graceful loss. Moonraker exits when it cannot bind its
# port, s6 starts it again a second later, it cannot bind again, and the loop
# runs for as long as the printer is on -- with `s6-svstat` saying "up" most of
# the time, because the process it is watching really is alive, it is just
# dying -- while what is actually answering Mainsail is an unsupervised
# leftover that nothing will ever restart. That is the same failure the nginx
# run script sweeps for, measured there first, and it is worth the same handful
# of lines here.
#
# It reads /proc rather than `ps | grep`, for the reason written out in the
# nginx run script: busybox `ps` truncates its output to the terminal width, so
# a long python command line arrives cut in half and a grep for the entry point
# matches nothing at all. /proc/PID/cmdline is the whole command line, always.
#
# The match is POSITIVE and narrow -- the absolute path of the entry point we
# are about to run, which appears in the command line of a moonraker and of
# nothing else on this printer. `s6-supervise moonraker`, this script and
# `init.d/S62moonraker start` all have the word moonraker in their command
# lines and none of them has moonraker.py's absolute path in it; our own pid is
# skipped as well, for the same belt-and-braces reason nginx skips it.
for _mr_proc in /proc/[0-9]*; do
[ -r "$_mr_proc/cmdline" ] || continue
[ "${_mr_proc#/proc/}" = "$$" ] && continue
_mr_args=`tr '\0' ' ' < "$_mr_proc/cmdline" 2>/dev/null`
case "$_mr_args" in
*"$MOONRAKER_MAIN"*) kill -9 "${_mr_proc#/proc/}" 2>/dev/null ;;
esac
done
unset _mr_proc _mr_args
# ---- readiness: fd 3, and what "ready" is allowed to mean ------------------
#
# The `notification-fd` file next to this script holds "3": this service is
# READY once a line is written to file descriptor 3, and `s6-svwait -U` blocks
# until it is. This is the capability runit has no concept of and the reason
# the supervisor choice went the way it did -- see tools/supervisor/README.md.
#
# WHY MOONRAKER GETS ONE AT ALL, when nginx does not. Because something already
# waits for it, badly. bin/ff-startup.py -- the program S70klipper hands the
# boot to -- polls http://127.0.0.1:7125/server/info in a retry loop to learn
# klippy_state, and every one of those early polls is against a moonraker that
# has been forked and is not listening yet. init.d/S62moonraker had its own
# version of the same guess: a detached moonraker_check that slept two seconds
# five times and called the service "up" if the pid was still there. A pid that
# is still there is not an API that answers, and the gap between the two is
# exactly the window in which moonraker imports its components and dies
# quietly.
#
# WHEN IS MOONRAKER READY? The candidates, and why only one of them is worth
# telling a caller:
#
# "the process was forked" -- worthless, and it is what the pidfile and the
# old moonraker_check both actually measured. Component-load failures
# happen seconds AFTER the fork, which is why that check had to sleep at
# all, and a moonraker that is still importing components answers
# nothing.
# "klippy_state is ready" -- WRONG, and tempting, because that is what
# ff-startup.py is ultimately waiting for. But it is not a fact about
# THIS service. Moonraker is up and doing its job on a printer whose MCU
# never answered -- reporting klippy_state=error IS the job -- so a
# readiness that waited for klippy would never fire on precisely the
# broken printer whose UI somebody needs, and the caller would time out
# and print a warning about a moonraker that was working perfectly. It
# would also invert the dependency: moonraker would be declaring itself
# ready on behalf of a service it neither owns nor starts.
# "the HTTP API is listening" -- this one. Every consumer of this service --
# Mainsail through nginx's /webcam-and-everything-else proxy,
# HelixScreen, and ff-startup.py -- is an HTTP client, and what an HTTP
# client needs is a socket that accepts. Moonraker binds its port after
# the server object is built and its core components have imported, so a
# bound port is also the strongest available statement about the
# component failures the old check was written to catch.
#
# HOW IT IS CHECKED, and why it needs no tool. /proc/net/tcp lists every TCP
# socket on the box: field 2 is LOCAL_ADDRESS:PORT with the port in uppercase
# hex, field 4 is the state, and 0A is TCP_LISTEN. No nc -- this busybox has no
# nc applet at all, measured on the replica -- and no wget in the hot path of a
# supervised service. The address half is deliberately not matched: moonraker
# binds 0.0.0.0 under the shipped config, and a user who sets host: 127.0.0.1
# should still count as listening.
#
# THE PORT IS READ FROM MOONRAKER'S OWN CONFIG, not written down here. 7125 is
# the shipped default and also moonraker's, but it is a user-editable line in a
# user-editable file, and a probe watching the wrong port would report a
# perfectly healthy moonraker as never ready -- which is worse than having no
# probe at all, because S62moonraker would then print a warning and a log tail
# on every boot of a printer that is working. The awk is section-aware so that
# a `port:` belonging to some other component cannot be picked up by mistake.
MOONRAKER_PORT=`awk '
/^[[:space:]]*\[/ { section = $0 }
section ~ /\[[[:space:]]*server[[:space:]]*\]/ && /^[[:space:]]*port[[:space:]]*:/ {
sub(/^[^:]*:[[:space:]]*/, "");
gsub(/[^0-9]/, "");
if (length($0)) { print; exit }
}' "$MOONRAKER_CONF" 2>/dev/null`
[ -n "$MOONRAKER_PORT" ] || MOONRAKER_PORT=7125
# The prober is a subshell because the last thing this script does is exec, and
# an exec'd moonraker cannot report on itself. It inherits fd 3 across the
# fork; the exec below closes fd 3 in MOONRAKER (3>&-) so that the only process
# holding the notification pipe is the one with something to say about it --
# were moonraker to keep it open, s6 would never see the pipe close when the
# prober gave up.
#
# 180 seconds, which is much longer than the camera's 60 and is not a copied
# number. Moonraker imports its whole component tree on a 1GHz mipsel box off
# eMMC; on the qemu replica the same startup is slower again, and
# case-moonraker.sh has waited up to 105 seconds for the API since long before
# s6 was involved. A prober that gave up first would turn a slow printer into a
# printer that reports itself broken. Giving up simply means no notification is
# ever sent, which is the honest outcome: S62moonraker then prints its warning
# and the tail of moonraker.log, exactly as its old detached check did.
(
_mr_n=0
while [ $_mr_n -lt 180 ]; do
if awk -v p=":`printf '%04X' "$MOONRAKER_PORT"`" \
'$2 ~ p"$" && $4 == "0A" { found = 1 }
END { exit !found }' /proc/net/tcp 2>/dev/null
then
echo ready >&3
exit 0
fi
sleep 1
_mr_n=$((_mr_n + 1))
done
) &
# THE INVOCATION, preserved argument for argument from the start-stop-daemon
# line it replaces:
#
# start-stop-daemon -S -b -m -p /run/moonraker.pid --exec "$FF_PYTHON" \
# -- "$MODDIR/moonraker/moonraker.py" -d /usr/data
#
# `-d /usr/data` is moonraker's data path: it is where it finds
# config/moonraker.conf, where it writes logs/moonraker.log and where its
# database lives. It is not a debug flag and it is not optional -- moonraker
# defaults to ~/printer_data, which on this printer is a home directory that
# does not exist.
#
# stderr is inherited from s6-supervise, which inherited it from the scanner,
# which init.d/S40s6 points at /usr/data/logs/s6.log. That is a genuine
# improvement rather than an accident of the port: under start-stop-daemon -b
# these descriptors went to /dev/null, so a moonraker that died before its own
# logging was configured -- a syntax error in a component, a missing shared
# object -- died with its explanation thrown away, and the only symptom was a
# service that "did not come up". Now it lands in a file that outlives the
# boot. Moonraker's own structured log stays exactly where it has always been,
# at $MOONRAKER_DATA/logs/moonraker.log, because moonraker opens that itself
# from -d.
#
# nice -n "${NICE_MOONRAKER:-0}", not a plain exec of the interpreter: this
# service is supervised now, which init.d/S62moonraker's start_unsupervised()
# fallback is not, and NICE_MOONRAKER (anvil.conf) exists to protect klippy
# from exactly the process this script starts -- reaching only the fallback
# would leave the feature dead for every printer that has s6, which by
# default is all of them. `nice` sets its own priority and then execs the
# given command in its own place (verified: the pid s6 ends up holding is the
# interpreter's, not nice's, exactly as required above), so this is still one
# process and $FF_PYTHON is still what --exec would have matched had this
# been start-stop-daemon. 0 or unset = normal priority, unchanged from before
# this existed.
exec nice -n "${NICE_MOONRAKER:-0}" "$FF_PYTHON" "$MOONRAKER_MAIN" -d "$MOONRAKER_DATA" 3>&-
pkgs/anvil-core/payload/etc/s6/nginx/down
-38
View File
@@ -1,38 +0,0 @@
This file's presence, not its contents, is the whole message: s6-supervise
starts a service whose directory contains a file called `down` in the DOWN
state instead of starting it. s6 never opens the file, so the reasoning can
live in it rather than in a comment somewhere else that would drift away from
it. (The gate checks that too, with this text in place, because a fact this
file depends on is worth measuring rather than believing.)
WHY NGINX SHIPS DOWN. It is ORDERING. s6-svscan starts a service the instant
it notices the directory, which is a fraction of a second after init.d/S40s6
runs -- before init.d/S60nginx has checked that the nginx binary is where it
thinks it is, before its config has been put in place, and before moonraker
is coming up behind it. A web server that binds :80 at that moment serves
Mainsail a backend that does not exist yet, and the first thing a user sees is
a UI blaming Klipper for something nginx did.
This used to be argued from a switch, MOD_WEB, which no longer exists. The
argument was never really about the switch: it was that THE SCANNER DOES NOT
DECIDE WHEN NGINX STARTS. init.d/S60nginx does, at the point in the boot where
its prerequisites have been checked, by calling svc_s6_up -- which overrides
this file for as long as that s6-supervise lives.
THE COST, stated honestly, because it is the one thing this arrangement gives
up. `down` is consulted when s6-supervise STARTS, not when the service dies:
once S60nginx has brought nginx up, s6 keeps restarting it on every crash for
as long as that supervisor lives, which is the entire point of the migration
and is unaffected by this file. What it does mean is that if s6-supervise
ITSELF is killed -- not nginx, the supervisor -- s6-svscan starts a fresh one,
that fresh one reads this file, and nginx stays down until something runs
`S60nginx start` again. That is a rare failure (s6-supervise is a small C
program that blocks in poll() and is not the thing that crashes) with an
honest signal (`S60nginx status` says it is not running, and says so from
s6-svstat rather than from a pidfile) and a one-line fix from an ssh session.
The alternative -- deleting this file at runtime once the boot has got far
enough -- buys back that corner case at the price of a payload that rewrites
its own install tree, which every path in this mod is arranged not to do.
This file is not a record of anything having gone wrong: it is the state nginx
ships in, on every build, and nothing removes it.
pkgs/anvil-core/payload/etc/s6/nginx/run
-105
View File
@@ -1,105 +0,0 @@
#!/bin/sh
# The nginx service, as s6 runs it. Exec'd by s6-supervise, never by hand.
#
# WHAT A `run` SCRIPT IS. s6-svscan sees this directory in the scandir and
# starts one s6-supervise for it; that s6-supervise forks, puts the child in a
# process group of its own, and execs this file. The process this script turns
# into IS the service: when it exits, s6-supervise runs `finish` if there is
# one and then starts the whole thing again. That is the entire contract, and
# every line below follows from it.
#
# THEREFORE: `daemon off;`. nginx's default is to fork a master into the
# background and return, which is what init.d/S60nginx relied on for years and
# is exactly wrong here -- s6 would supervise the process that forked and
# exited, decide the service had died a millisecond after starting it, and
# start it again, for ever, while the real nginx it left behind kept serving
# quite happily and was supervised by nobody. `-g` puts a directive on the
# command line as though it were at the top of nginx.conf, so the config file
# stays the one thing it should be -- shared with the hand-run and fallback
# paths -- and only the way we are launching it changes. Verified on the
# replica against this printer's own nginx (1.5.0, FlashForge's build, from
# /usr/prog): with `daemon off;` the pid we launched is still the master five
# seconds later, and it still writes logs/nginx.pid, which is what lets
# S60nginx's non-supervised fallback keep working unchanged.
#
# AND `exec`. Without it this shell would stay around as the parent of nginx,
# s6 would supervise the shell, and every signal s6 sent -- the SIGTERM behind
# `S60nginx stop` above all -- would go to a shell that ignores it while nginx
# carried on. exec makes the pid s6 is holding the pid of nginx itself.
#
# THIS FILE DECIDES NOTHING. It runs whenever s6 decides the service should be
# up, and a `run` that exits because it disagrees is a respawn loop with extra
# steps. What decides is the `down` file next to this one plus init.d/S60nginx,
# which is the thing that checks nginx's prerequisites and picks the moment.
# See the comment in `down`; the whole story is written there.
MODDIR=/usr/data/anvil
NGINX=/usr/prog/nginx/sbin/nginx
# -p sets the prefix: logs, the pidfile and the temp directories all land
# under $MODDIR on the data partition, never on /usr/prog. nginx will not
# start if it cannot create them and it does not create them itself -- the
# same mkdir S60nginx has always done, repeated here because s6 restarts this
# service without going anywhere near S60nginx, and a service that only works
# when a shell script happened to run first is not supervised, it is lucky.
mkdir -p $MODDIR/nginx/logs $MODDIR/nginx/tmp
# THE ORPHANED WORKER, and why there is a loop in what should be two lines.
#
# nginx is two processes: a master that binds :80 and a worker that inherits
# the listening socket and answers on it. Kill the master politely -- which is
# what `s6-svc -d` does, SIGTERM -- and it shuts the worker down with it;
# measured on the replica, nothing is left and the port is free. Kill the
# master with SIGKILL, which is what an OOM or a `kill -9` does, and the
# worker is orphaned, KEEPS THE LISTENING SOCKET, and goes on serving. s6 then
# does its job perfectly and makes everything worse: it starts a new nginx one
# second later, that nginx cannot bind because the orphan still holds :80, it
# retries five times over five seconds, prints "still could not bind()" and
# exits, and s6 starts it again. For ever. `s6-svstat` says "up" the whole
# time -- the process it is watching really is alive, it is just failing --
# while what is actually answering the browser is an unsupervised leftover.
# All of that was measured on the replica; none of it is a guess.
#
# So: before nginx starts, anything already speaking for it is a leftover, and
# leftovers get killed. That is true by construction -- s6-supervise runs
# exactly one copy of this service at a time, so at the moment this line runs
# there is no nginx of ours anywhere -- and it makes the fallback in S60nginx
# safe as well: a printer that started nginx directly because the scanner was
# not running gets that copy swept the moment the supervised one takes over,
# instead of two nginxes fighting over the port.
#
# It reads /proc rather than `ps | grep`, which is the opposite of what the
# rest of the mod does and is deliberate. busybox `ps` truncates its output to
# the terminal width, so on the replica the command line arrives cut off at
# "/usr/prog/nginx/" and a grep for the binary matches nothing at all; and
# busybox `kill` will not take a negative pid, so signalling the process group
# -- the obvious fix -- is not available on this box either. Both were tried
# and both failed there. /proc/PID/cmdline is the whole command line, always.
#
# The match is POSITIVE and narrow, because the cost of a wrong match here is
# not a missed cleanup, it is killing the wrong thing: "s6-supervise nginx",
# "/usr/data/anvil/init.d/S60nginx start" and this very script all have the
# word nginx in their command lines, and s6-supervise in particular is the
# process that would otherwise restart us. So only two shapes are swept -- the
# ones nginx itself produces:
#
# nginx: master process /usr/prog/nginx/sbin/nginx -p ... (it rewrites
# nginx: worker process its own argv)
# ...anything exec'd from /usr/prog/nginx/sbin/nginx (before it does)
#
# and our own pid is skipped for good measure.
for _ng_proc in /proc/[0-9]*; do
[ -r "$_ng_proc/cmdline" ] || continue
[ "${_ng_proc#/proc/}" = "$$" ] && continue
_ng_args=`tr '\0' ' ' < "$_ng_proc/cmdline" 2>/dev/null`
case "$_ng_args" in
"nginx: "*|*"$NGINX"*) kill -9 "${_ng_proc#/proc/}" 2>/dev/null ;;
esac
done
unset _ng_proc _ng_args
# stderr is where nginx puts the reason it would not start, and s6-supervise
# hands this script the descriptors the scanner had, which is the s6 log --
# /usr/data/logs/s6.log, per init.d/S40s6. That is the right place for it: it
# is a file that outlives the boot, unlike the boot log S60nginx used to print
# into, and "see the error above" finally points at something you can read.
exec $NGINX -p $MODDIR/nginx -c $MODDIR/nginx/nginx.conf -g "daemon off;"
pkgs/anvil-core/payload/init.d/S40s6
-229
View File
@@ -1,229 +0,0 @@
#!/bin/sh
# s6-svscan -- the process supervisor, running and supervising nothing.
#
# WHY THIS EXISTS. Everything the other six scripts do by hand -- "is it still
# there", "stop it and wait until it really is gone", "start it again when it
# dies" -- is what a supervisor does in C, and anvil-service.sh has already
# had two real bugs in exactly that code (busybox start-stop-daemon -K
# returning before the process is dead; a stale socket making a dead klippy
# report itself running). s6 was measured against runit on the replica and won
# on the one thing that matters here, readiness notification: a service can
# declare itself ready and s6-svwait -U blocks until it does, which is what
# S65camera's 30-second poll of /dev/video0 and S70klipper's MCU handshake
# retry are both hand-rolling. See tools/supervisor/README.md for the
# measurements and docs/notes/80-s6-migration.md for the plan.
#
# THIS SCRIPT MIGRATES NOTHING. The scandir it scans is empty, on purpose.
# All this phase buys is that the scanner is alive on every boot before any
# service could want it, so that moving services into it later is one small
# change per service rather than one big one. A scanner with an empty scandir
# costs a single ~30KB process and does nothing at all -- measured on the
# replica: it starts clean, prints not one line, and answers its control
# socket. "Supervising nothing" is a supported state, not a degenerate one.
#
# THE NUMBERING IS LOAD-BEARING, the same way it is for S60nginx. firmwareExe
# runs $MODDIR/init.d/S* in filename order, so 40 puts the scanner before
# S50wifi and therefore before every service that will later be supervised by
# it -- a service coming up and asking for a scanner that does not exist yet
# is the one ordering mistake this file exists to prevent. It is not S00 or
# S10 because those numbers are worth keeping free for something that has to
# run BEFORE the supervisor (a filesystem check, a migration); nothing below
# 40 exists today and this script has no reason to claim the whole range.
#
# IT MUST NOT BLOCK THE BOOT, and this is the sharp edge. s6-svscan NEVER
# RETURNS -- that is its entire job. firmwareExe runs these scripts one at a
# time in the FOREGROUND, so running the scanner inline does not delay the
# boot, it ENDS it: no wifi, no ssh, no Mainsail, no Klipper and no screen,
# for ever, on every boot, with no way in to find out why. So it is
# backgrounded through svc_detach -- exactly how S65camera backgrounds the
# respawn loop it hand-rolls for mjpg_streamer -- and start() returns at once.
#
# WHAT RESTARTS THE SCANNER IF IT DIES? Nothing, and that is a decision, not
# an oversight. We are not PID 1 and there is no supervisor above this one; a
# watchdog for it would be one more hand-rolled respawn loop in ash, which is
# the exact thing we are buying s6 to delete. What exists instead is
# proportionate to how it can actually fail:
#
# * It is one small C program that allocates nothing after startup and
# blocks in poll(). The realistic way it "dies" is not a crash but a bad
# BUILD -- the -D_FILE_OFFSET_BITS=64 trap in tools/supervisor/README.md,
# where it execs fine and then dies on readdir() with EOVERFLOW. A
# watchdog would respawn that forever and change nothing. start() below
# therefore waits for the scanner to answer and SAYS SO when it does not,
# which is the only response that helps.
# * If it does die, the s6-supervise processes it started keep running and
# keep respawning their own services. A dead scanner is a printer that has
# stopped noticing changes to the scandir, not a printer that has stopped
# printing. Today, with an empty scandir, it is not even that.
# * firmwareExe re-checks it once after the init loop and restarts it there
# if it is gone -- the same last-resort shape it already uses for Klipper,
# and for the same reason: the check is cheap, it runs at the one moment
# something else has already gone wrong, and the log line it prints is the
# thing you would otherwise have had to guess.
#
# So: no watchdog, one re-check on the boot path, and a loud start(). If that
# ever proves too little, the honest fix is an inittab-style respawn, not a
# `while true` in a shell script.
MODDIR=/usr/data/anvil
[ -f $MODDIR/anvil.conf ] && . $MODDIR/anvil.conf
# The library path and the interpreter, from the one file that defines them.
# s6 itself needs nothing from it -- it is static-pie musl and links against
# no shared object on this box, which is exactly why it was built that way --
# but a script that gets re-run over ssh should get the same environment on
# every path, and sourcing it twice is a no-op.
if [ -f $MODDIR/anvil-env.sh ]; then
. $MODDIR/anvil-env.sh
else
echo "s6: !! no $MODDIR/anvil-env.sh -- the mod payload is not installed"
fi
# start/stop/status/liveness, shared with the other services. See
# anvil-service.sh for the whole API -- including the svc_s6_* helpers, which
# are only usable because this script is what puts a scanner behind them.
if [ -f $MODDIR/anvil-service.sh ]; then
. $MODDIR/anvil-service.sh
else
echo "s6: !! no $MODDIR/anvil-service.sh -- the mod payload is not installed"
exit 1
fi
SVC_NAME=s6
# SVC_S6_BIN and SVC_S6_SCANDIR come from anvil-service.sh: every script that
# talks to the scanner has to agree with this one about where it is, and the
# way to make that true is to name it once, in the file they all source.
SVSCAN=$SVC_S6_BIN/s6-svscan
SCANDIR=$SVC_S6_SCANDIR
# The pidfile holds the SCANNER, not a supervisor: stop() has to be able to
# take down the one process that would otherwise put everything back. Same
# reasoning as the pidfile in S65camera, which holds the respawn loop rather
# than the streamer it respawns.
PIDFILE=$MODDIR/s6-svscan.pid
# The scanner's own output, in a file of its own. A healthy s6-svscan says
# NOTHING -- verified on the replica, where three seconds on an empty scandir
# produced an empty log -- so anything landing here is a fault worth finding
# on its own. It is not left on the inherited descriptor because the scanner
# OUTLIVES firmwareExe's redirection: it would otherwise keep writing into the
# boot log of a boot that finished hours ago.
LOG=/usr/data/logs/s6.log
scanner() {
mkdir -p /usr/data/logs
# exec, so the pid svc_detach captured IS the scanner's pid rather than
# that of a shell which happens to have started one. svc_stop_daemon
# signals whatever is in the pidfile, and signalling a wrapper that has
# already gone would leave the scanner running with nothing pointing at it.
exec "$SVSCAN" "$SCANDIR" >>"$LOG" 2>&1
}
start() {
[ "${MOD_S6:-1}" = "1" ] || { svc_say "disabled in anvil.conf (MOD_S6=0)"; return 0; }
if [ ! -x "$SVSCAN" ]; then
svc_warn "no $SVSCAN -- this payload did not ship the supervisor"
return 1
fi
if svc_s6_running; then
svc_say "already scanning $SCANDIR"
return 0
fi
# WHY THE mkdir IS HERE AS WELL AS IN THE PAYLOAD. anvil-core ships
# etc/s6/, so the directory lands on a real install and -- the reason for
# shipping it rather than only creating it
# here -- is NAMED IN THE INSTALL MANIFEST, which is what lets a later
# update account for it instead of orphaning it. But it ships EMPTY, and
# an empty directory is precisely the kind of payload path that other
# things legitimately remove: run-append.sh's manifest pass rmdir's every
# empty directory the PREVIOUS payload listed, and it does that before the
# new tarball is extracted. So there is a window; and there are printers
# somebody installed by hand. One mkdir -p costs nothing and deletes a
# whole class of "the scanner did not start and the log says ENOENT".
mkdir -p "$SCANDIR" || { svc_warn "cannot create $SCANDIR"; return 1; }
# svc_detach leaves the scanner's pid in SVC_BG_PID; it is written to the
# pidfile because the scanner is the half of this that has to be killable.
svc_detach scanner
echo "$SVC_BG_PID" > "$PIDFILE"
# Then wait for it to answer -- but only for a thing that takes no time.
# s6-svscan is up once it has created $SCANDIR/.s6-svscan/, taken the lock
# and started reading the control FIFO: a mkdir, a mkfifo and two opens.
# It waits on no hardware, no network and no other service, so the check
# below almost always succeeds on its first pass and costs nothing. What
# it buys is that the "scanning" line is TRUE when it is printed, and that
# a scanner which exec'd and then died -- the EOVERFLOW build trap above
# -- is reported here, by name, instead of silently three phases from now
# when a service is moved into the scandir and never comes up. The 5s
# bound is only ever paid on a printer that is already broken.
_s6_waited=0
while ! svc_s6_running && [ $_s6_waited -lt 5 ]; do
sleep 1
_s6_waited=$((_s6_waited + 1))
done
if svc_s6_running; then
svc_say "scanning $SCANDIR (pid $SVC_BG_PID)"
unset _s6_waited
return 0
fi
svc_warn "started but is not answering on $SCANDIR after ${_s6_waited}s -- see $LOG"
unset _s6_waited
return 1
}
stop() {
# -t is the documented way down, and the only one that keeps the promise
# "stop leaves no s6 processes behind": it tells the scanner to send TERM
# to every s6-supervise it started, reap them, and then exit itself.
# Killing the scanner on its own would leave those supervisors orphaned
# and still respawning their services -- the same mistake as killing
# mjpg_streamer without its respawn loop, which S65camera's stop() is
# written the way it is to avoid. It costs nothing today, with nothing to
# supervise, and it is the line that has to already be right on the day
# phase 4 puts the first service in there.
if [ -x "$SVC_S6_BIN/s6-svscanctl" ] && svc_s6_running; then
"$SVC_S6_BIN/s6-svscanctl" -t "$SCANDIR" 2>/dev/null
_s6_waited=0
while svc_s6_running && [ $_s6_waited -lt 15 ]; do
sleep 1
_s6_waited=$((_s6_waited + 1))
done
svc_s6_running && svc_warn "still answering after ${_s6_waited}s"
unset _s6_waited
fi
# Then the pidfile, which does two things -t does not. It REMOVES the file
# -- svc_pid_alive on a recycled pid is how `status` once came to report an
# unrelated process as a running service -- and it is the way out when the
# scanner is wedged badly enough not to read its own control socket: a
# plain TERM to the pid exits it too, verified on the replica.
svc_stop_daemon "$PIDFILE"
svc_say "stopped"
}
status() {
# Asked of the SCANNER, not of the process table. `s6-svscanctl -a` is a
# request to rescan the scandir -- harmless, idempotent, and it travels
# down the control FIFO, so it can only succeed if something on the other
# end is reading. A dead scanner leaves that FIFO on disk and the ping
# then fails with "supervisor not listening" (exit 100), which is exactly
# the distinction a `ps | grep` cannot make and the distinction that made
# a stale pidfile lie about moonraker.
if svc_s6_running; then
svc_say "scanning $SCANDIR"
else
svc_say "not running"
fi
# And what it is supervising, if anything. Zero is correct today and stays
# correct until phase 4 puts the first service directory in there; saying
# the number rather than nothing is what makes that change visible the day
# it happens. The glob skips .s6-svscan/ -- the scanner's own control
# directory is hidden, which is why it is not a service here.
_s6_n=0
for _s6_d in "$SCANDIR"/*; do
[ -d "$_s6_d" ] || continue
_s6_n=$((_s6_n + 1))
_s6_name=`basename "$_s6_d"`
svc_say " $_s6_name: `svc_s6_stat "$_s6_name" 2>&1`"
done
svc_say "$_s6_n service directories in $SCANDIR"
unset _s6_n _s6_d _s6_name
}
svc_dispatch "$@"
pkgs/anvil-core/payload/init.d/S50wifi
-147
View File
@@ -1,147 +0,0 @@
#!/bin/sh
# WiFi.
#
# WHY THIS EXISTS -- the same trap as Klipper, and for the same reason:
# nothing in /etc/init.d brings up wlan0. FlashForge's firmwareExe owns the
# whole network stack. `strings firmwareExe` shows it running "ifconfig wlan0
# up", driving wpa_supplicant/wpa_cli against /usr/prog/wifi/wpa_supplicant.conf
# and then "udhcpc -i wlan0". app_startup.sh only insmods the 8821cu driver;
# it never configures the interface.
#
# So the moment HelixScreen replaces firmwareExe the printer boots with NO
# network at all: no ssh, no Mainsail, no moonraker. Which also means no way
# to diagnose it. Hence this runs before S60nginx/S62moonraker and S70klipper.
#
# BUT IT MUST NOT MAKE THEM WAIT. firmwareExe runs the S* scripts one at a
# time in the foreground. Everything interesting here is a wait on hardware:
# up to 15s for wlan0 to register after the 8821cu insmod, then up to 20s more
# for the AP to associate. The two waits are sequential, so inline that is up
# to ~35s with no Klipper, no Mainsail and
# no UI whenever the AP is out of range or the password is stale -- i.e. the
# longest delay lands on exactly the machines with no network, where you most
# need ssh to diagnose and where waiting buys nothing. So the waiting happens
# in a background subshell and start() returns at once: the rule firmwareExe
# already states for a service that fails, applied to one that is merely slow.
# The trade is that "wifi:" lines land in the boot log after the later
# services' lines, out of order. That rule is svc_detach now -- see
# anvil-service.sh, where it is stated once for all five scripts.
#
# The saved networks live in /usr/prog/wifi/wpa_supplicant.conf on the
# firmware partition -- we never ship or touch that file, so whatever
# FlashForge's UI saved there before the mod is still there. It sets ctrl_interface=DIR=/var/run/wpa_supplicant,
# which is also one of the paths HelixScreen probes for a control socket, so
# its WiFi panel drives this very daemon (and update_config=1 lets it save new
# networks back).
MODDIR=/usr/data/anvil
[ -f $MODDIR/anvil.conf ] && . $MODDIR/anvil.conf
# start/stop/status/liveness, shared with the other services. See
# anvil-service.sh for the whole API.
if [ -f $MODDIR/anvil-service.sh ]; then
. $MODDIR/anvil-service.sh
else
echo "wifi: !! no $MODDIR/anvil-service.sh -- the mod payload is not installed"
exit 1
fi
SVC_NAME=wifi
IFACE=wlan0
WIFI_CONF=/usr/prog/wifi/wpa_supplicant.conf
CTRL_DIR=/var/run/wpa_supplicant
# wpa_supplicant backgrounds itself with -B, so there is no pid we own and no
# pidfile to keep it in: by name is the only answer available. svc_proc_alive
# is the same `ps | grep | grep -v grep` this script always did, with pgrep
# preferred where the rootfs ships it.
supplicant_running() {
svc_proc_alive wpa_supplicant
}
# The part that waits. Detached from the boot sequence, so nothing reads its
# exit code -- it reports through the boot log only.
connect() {
# app_startup.sh insmods 8821cu just before us, but the interface takes a
# moment to register.
waited=0
while [ $waited -lt 15 ]; do
[ -d /sys/class/net/$IFACE ] && break
sleep 1
waited=$((waited + 1))
done
if [ ! -d /sys/class/net/$IFACE ]; then
svc_warn "$IFACE never appeared -- is 8821cu.ko loaded?"
return 1
fi
/sbin/ifconfig $IFACE up 2>/dev/null
mkdir -p $CTRL_DIR
# -B backgrounds it. Try nl80211 first and fall back to wext: the 8821cu
# is a Realtek USB part and which one it registers varies by build.
if /usr/sbin/wpa_supplicant -B -i $IFACE -c "$WIFI_CONF" -D nl80211,wext; then
svc_say "wpa_supplicant started"
else
svc_warn "wpa_supplicant failed to start"
return 1
fi
# Give association a chance before asking for a lease.
waited=0
while [ $waited -lt 20 ]; do
/usr/sbin/wpa_cli -i $IFACE status 2>/dev/null | grep -q '^wpa_state=COMPLETED' && break
sleep 1
waited=$((waited + 1))
done
/sbin/udhcpc -i $IFACE -b -t 5 >/dev/null 2>&1 &
svc_say "udhcpc requesting a lease on $IFACE"
}
start() {
# No MOD_WIFI check: it is gone, and the line below is what it was standing
# in front of. A printer with nothing to connect to has no $WIFI_CONF and
# this script says so and returns -- which is the same outcome the flag
# bought, without a second place to look when wifi does not come up.
if supplicant_running; then
svc_say "wpa_supplicant already running"
return 0
fi
[ -f "$WIFI_CONF" ] || { svc_warn "no $WIFI_CONF -- nothing to connect to"; return 1; }
# Everything above was instant. Everything below waits on hardware, so it
# goes to the background and the boot carries on to S60nginx, S62moonraker
# and S70klipper.
svc_detach connect
svc_say "bringing up $IFACE in the background"
return 0
}
stop() {
# By name, not by pidfile: nothing here was started with one, and inventing
# one for a daemon that backgrounds itself would only be a second thing
# that can be wrong. wpa_cli first, then the lease, then the daemon that
# owns the association.
killall wpa_cli 2>/dev/null
killall udhcpc 2>/dev/null
killall wpa_supplicant 2>/dev/null
# Then WAIT for the supplicant to actually be gone. This is the thing the
# `sleep 2` in front of the old `restart` was standing in for: killall
# returns as soon as the signal is sent, so a start() running a moment
# later finds the dying supplicant still in the process table, says
# "already running", and does nothing -- leaving the printer stopped
# rather than restarted. Same bound and same reasoning as svc_stop_daemon.
waited=0
while supplicant_running && [ $waited -lt 15 ]; do
sleep 1
waited=$((waited + 1))
done
supplicant_running && svc_warn "wpa_supplicant still running after ${waited}s"
svc_say "stopped"
}
status() {
supplicant_running && svc_say "wpa_supplicant running" || svc_say "not running"
# Raw ifconfig output, deliberately unprefixed: it is the address you came
# here to read, in the shape you would have typed the command to get.
/sbin/ifconfig $IFACE 2>/dev/null | grep -E 'inet |UP'
}
svc_dispatch "$@"
pkgs/anvil-core/payload/init.d/S60nginx
-302
View File
@@ -1,302 +0,0 @@
#!/bin/sh
# nginx -- the Mainsail web UI on :80.
#
# The binary already ships on stock firmware -- FlashForge simply comments it
# out of /usr/prog/klipper/start.sh and serves its own HTTP API from firmwareExe
# instead. The mod replaces firmwareExe, so that API is gone and this is the
# only web server on the printer. Nothing is built or shipped for it beyond the
# config and the Mainsail files under $MODDIR: we start the binary FlashForge
# already put there, pointed at our own prefix so nothing writes to /usr/prog.
#
# THIS AND S62moonraker WERE ONE SCRIPT, S60web. They are separated because
# they fail separately and are debugged separately: nginx not starting is a
# missing config or a port already bound, and is instant; moonraker not
# starting is a component that failed to import, and takes a background check
# and a log tail to even notice. One script had to say "web: stopped" for both,
# and `status` reported on two unrelated things at once. Splitting them means
# `S62moonraker restart` over ssh -- which is the one you actually want, over
# and over, while chasing a moonraker config -- no longer takes the UI down
# with it.
#
# THE NUMBERING IS LOAD-BEARING. firmwareExe runs $MODDIR/init.d/S* in filename
# order, so this runs before S62moonraker, which runs before S65camera. nginx
# first is the useful order: it proxies both of the others (moonraker on 7125
# and the camera's :8080 stream through /webcam/), and when it is up first the
# browser gets a page that says the backend is not answering yet, rather than a
# refused connection that says nothing at all. It also runs AFTER S40s6, which
# is what starts the supervisor this script now hands the work to, and that
# ordering is not a coincidence either -- see below.
#
# THERE IS NO SWITCH ANY MORE. This script and S62moonraker used to read
# MOD_WEB out of anvil.conf and return early if it was 0, which left a printer
# with ssh and the touchscreen only. It was described as a debugging switch and
# it earned that description exactly once; what it cost the rest of the time
# was a second state for every question anyone asked about the web stack --
# every gate, every test, every `down` file argued about both. Mainsail and
# moonraker are what this mod IS. A printer that does not want them wants a
# stock FlashForge package, which is a thing that exists and is one flash away.
#
# WHAT CHANGED WHEN s6 ARRIVED, and what deliberately did not. nginx is now a
# supervised service: $MODDIR/etc/s6/nginx/ holds a `run` script, s6-supervise
# runs it, and when nginx dies s6 starts it again. That is the whole point --
# before this, a master process that fell over stayed fallen until somebody
# noticed the UI was gone and ssh'd in, and this script had no way to do
# anything about it because a shell that has returned cannot restart anything.
#
# THE INTERFACE IS UNCHANGED, on purpose. `S60nginx restart` over ssh is in the
# docs and in people's hands, and changing the supervisor is not a reason to
# change what people type. So start/stop/restart/status all still exist, still
# say "nginx: ...", and still mean what they meant; underneath, each of them is
# now a couple of svc_s6_* calls instead of a start-stop-daemon invocation and
# a pidfile. What that buys is the same thing the shared library was written
# for: the waiting is done by C that s6 maintains rather than by a `while
# kill -0 ...; sleep 1` we maintain, and `restart` cannot race its own start.
#
# WHY $MODDIR/etc/s6/nginx/down STILL SHIPS, now that no flag needs it. It is
# ORDERING, and it always was the larger half of the argument: s6-svscan starts
# a service the instant the directory appears, which is before this script has
# checked that nginx exists, that its config is where it thinks it is, and that
# moonraker is coming up behind it. `down` means the scanner supervises and
# waits, and start() below is what decides the moment. The long version is in
# the `down` file, next to the thing it explains.
MODDIR=/usr/data/anvil
[ -f $MODDIR/anvil.conf ] && . $MODDIR/anvil.conf
# The library path and the interpreter, from the one file that defines them.
# See anvil-env.sh for why naming them here instead was a bug. nginx itself
# needs nothing from it -- it links against the base rootfs -- but a script
# that is re-run over ssh should get the same environment on every path, and
# sourcing it twice is a no-op.
if [ -f $MODDIR/anvil-env.sh ]; then
. $MODDIR/anvil-env.sh
else
echo "nginx: !! no $MODDIR/anvil-env.sh -- the mod payload is not installed"
fi
# start/stop/status/liveness, shared with the other services. See
# anvil-service.sh for the whole API -- including the svc_s6_* helpers, which
# are what the three verbs below are built out of.
if [ -f $MODDIR/anvil-service.sh ]; then
. $MODDIR/anvil-service.sh
else
echo "nginx: !! no $MODDIR/anvil-service.sh -- the mod payload is not installed"
exit 1
fi
SVC_NAME=nginx
NGINX=/usr/prog/nginx/sbin/nginx
NGINX_PID=$MODDIR/nginx/logs/nginx.pid
# The service directory the scanner watches, spelled out of the two variables
# anvil-service.sh defines so that this script and S40s6 cannot disagree about
# where the scandir is. The name is the service name, which is also SVC_NAME,
# which is also what every line this script prints starts with; that is one
# fewer thing to keep in step by hand.
SVCDIR=$SVC_S6_SCANDIR/$SVC_NAME
# Is there a live s6-supervise for our service directory RIGHT NOW?
#
# Two separate questions hide in "is s6 there", and both have to be yes before
# any of the verbs below can use it. svc_s6_running answers the first -- is the
# SCANNER alive and listening -- and does it by behaviour, by pinging the
# control FIFO. This answers the second: has that scanner actually picked our
# directory up and started a supervisor for it? A scandir the scanner has not
# read yet (the payload was extracted after S40s6 ran; somebody dropped the
# directory in by hand) looks exactly like a scandir with no nginx in it, and
# `s6-svc` against it fails with "supervisor not running" -- which is a true
# statement about a state that fixes itself the moment the scanner rescans.
#
# s6-svok is that question in one syscall: it opens the supervise/control FIFO
# and exits 0 only if a supervisor has it open at the other end. Same technique
# as svc_s6_running, one level down, and the same reason for preferring it to a
# test for the directory: a servicedir on disk with no supervisor behind it is
# the stale-pidfile lie all over again.
supervised() {
[ -x "$SVC_S6_BIN/s6-svok" ] || return 1
"$SVC_S6_BIN/s6-svok" "$SVCDIR"
}
# Ask the scanner to pick up the service directory, then wait a moment for the
# supervisor to appear. svc_s6_running IS the rescan request -- `s6-svscanctl
# -a` is what it pings with -- so this is one call doing two jobs, which is
# worth saying out loud because it does not look like it does. The bound is 5
# seconds and is only ever paid on a printer where something is already wrong:
# on a normal boot the payload was extracted long before S40s6 started the
# scanner, so the supervisor is already there and the first test succeeds.
# Anything that spends time here is spending firmwareExe's boot, one script at
# a time in the foreground, which is why it is 5 and not 30.
await_supervisor() {
supervised && return 0
svc_s6_running || return 1
_ng_waited=0
while ! supervised && [ $_ng_waited -lt 5 ]; do
sleep 1
_ng_waited=$((_ng_waited + 1))
done
unset _ng_waited
supervised
}
# ---- the fallback, and why it exists ---------------------------------------
#
# WHAT HAPPENS WHEN THERE IS NO SUPERVISOR. MOD_S6=0 is a switch S40s6 honours,
# so "no scanner" is a state a user can choose; and a payload whose s6 failed
# to build, or an install that got half way, is a state nobody chose but that
# happens anyway. The choice is between refusing to start and starting nginx
# the old way, and this script starts it the old way. The reasoning:
#
# * The failure this would otherwise produce is a printer with no web UI and
# no way to be told why, because the way you would be told is a web UI.
# ssh is the remaining door; a printer that loses Mainsail because a
# SUPERVISOR did not build has lost it for a reason that has nothing to do
# with nginx.
# * The old path is not hypothetical or unmaintained -- it is the code that
# shipped for the whole life of this script, it is nine lines, and
# case-moonraker.sh still exercises it, because that case starts nginx
# without a scanner anywhere in sight.
# * The classic reason NOT to fall back is that you end up with two of them:
# a directly-started nginx, then a scanner arriving later and starting a
# second one on a port that is already bound. That cannot happen here, and
# it is the `down` file that makes it so -- the scanner never starts nginx
# on its own, only this script does. And when this script does, the `run`
# script sweeps any nginx that is already running before exec'ing its own,
# so the hand-over from the fallback to the supervised copy is clean rather
# than a fight over :80.
#
# What it is NOT allowed to be is silent. A printer running the fallback has
# lost automatic restart-on-crash without losing the web UI, which is exactly
# the kind of half-degraded state that gets discovered six months later, so it
# says so every time, with "!!", which is what people grep a boot log for.
start_unsupervised() {
svc_warn "no supervisor -- starting nginx directly, it will NOT be restarted if it dies"
# -p sets the prefix: logs, the pidfile and the temp directories all land
# under $MODDIR on the data partition, never on /usr/prog. nginx will not
# start if it cannot create them, and it does not create them itself.
if $NGINX -p $MODDIR/nginx -c $MODDIR/nginx/nginx.conf; then
svc_say "started (unsupervised)"
return 0
fi
# nginx has already printed its own reason on stderr, which is in the
# boot log right above this line. Do not paraphrase it.
svc_warn "FAILED to start -- see the error above"
return 1
}
start() {
# The two prerequisites this script has always checked, in the same order
# and with the same words, because they are the two things that are
# actually missing when nginx does not start and neither of them is any
# less true under a supervisor. A missing prerequisite is REPORTED and
# returns non-zero; it is not a crash and it is not a stack trace out of
# s6, which is what handing a supervisor a service it cannot run produces.
if [ ! -x "$NGINX" ]; then
svc_warn "no $NGINX -- this printer's firmware did not ship one"
return 1
fi
if [ ! -f $MODDIR/nginx/nginx.conf ]; then
svc_warn "no $MODDIR/nginx/nginx.conf -- the install did not complete"
return 1
fi
mkdir -p $MODDIR/nginx/logs $MODDIR/nginx/tmp
# The third prerequisite, which is new: the service directory itself. It is
# staged into the payload by bin/patch.sh and named in the install
# manifest, so its absence means a payload that was built without it or an
# install that did not finish -- and it is deliberately NOT created here.
# Writing it at runtime would mean the scandir contained a directory no
# manifest lists, which is the exact thing the manifest exists to stop, and
# it would put the `run` script's text inside this file where nobody would
# ever look at it again.
if [ ! -x "$SVCDIR/run" ]; then
svc_warn "no $SVCDIR/run -- this payload did not ship the s6 service"
start_unsupervised
return $?
fi
if ! await_supervisor; then
start_unsupervised
return $?
fi
# And up it goes. svc_s6_up is `s6-svc -wu -u` with a bound: it asks for the
# service to be brought up -- overriding the `down` file, which is how a
# supervised-but-idle service becomes a running server -- and does not
# return until it IS up. 10 rather than the default 15 seconds because nginx
# coming up is a bind() and a fork() and nothing else: it is either up in
# well under a second or it is not coming up at all, and every second spent
# here is a second firmwareExe is not spending on moonraker, the camera and
# Klipper.
if svc_s6_up "$SVC_NAME" 10; then
svc_say "started -- `svc_s6_stat "$SVC_NAME"`"
return 0
fi
# Up it did not come. s6 has kept the reason where a reason belongs: the
# `run` script's stderr is the scanner's stderr, which S40s6 points at
# /usr/data/logs/s6.log, and nginx says precisely why it would not start
# (an unreadable config, a port already bound) on stderr. Say where, rather
# than paraphrasing something we have not read.
svc_warn "FAILED to come up -- `svc_s6_stat "$SVC_NAME"` -- see /usr/data/logs/s6.log"
return 1
}
stop() {
# A stop that the supervisor undoes is the classic mistake of this
# migration, and `s6-svc -d` is what makes it not happen: it does not just
# signal the process, it tells s6 the service is WANTED DOWN, and s6 then
# does not start it again until something says otherwise. Killing the pid
# instead -- which is all this script could do before, and all
# svc_stop_daemon still does below -- would get nginx restarted a second
# later by a supervisor that was doing exactly what it was asked to do.
#
# -wD (inside svc_s6_down) is the other half: it returns when the service
# is really down, not when the signal has been sent. That is the busybox
# start-stop-daemon correction from anvil-service.sh, except in C.
if supervised; then
svc_s6_down "$SVC_NAME" || svc_warn "still not down -- `svc_s6_stat "$SVC_NAME"`"
# nginx writes logs/nginx.pid even with `daemon off;` -- measured on
# the replica -- so a supervised run leaves one behind, and it is a pid
# that the kernel is now free to hand to something else. The fallback
# path below is the only thing that reads this file, and it is exactly
# the stale-pidfile lie anvil-service.sh has two comments about, so it
# goes here rather than being left for it to trip over.
rm -f "$NGINX_PID"
svc_say "stopped"
return 0
fi
# No supervisor: whatever is running was started by start_unsupervised (or
# by a hand-run nginx), so it is the pidfile that has to be signalled. The
# wait matters here too: the master process takes a moment to reap its
# workers, and returning before it has is how `restart` came back to a port
# that was still bound.
svc_stop_daemon "$NGINX_PID"
svc_say "stopped"
}
status() {
# Asked of the supervisor when there is one, because it is the only thing
# that knows. s6-svstat's own wording is passed through -- "up (pid 1234
# pgid 1234) 71 seconds", "down (signal SIGTERM) 4 seconds, normally up" --
# for the reason anvil-service.sh gives for svc_s6_stat: it is the answer
# you came for, in the shape you would have typed the command to get.
#
# The words "running" and "not running" are still the first thing on the
# line, though, and that is not decoration either. It is the vocabulary
# every other service in init.d/ answers in, it is what a human skims a
# boot log for, and it is what case-moonraker.sh looks for when it checks
# that stopping moonraker did not take the web server down with it. The
# s6 detail is appended to it rather than replacing it.
if supervised; then
_ng_stat=`svc_s6_stat "$SVC_NAME"`
case "$_ng_stat" in
up*) svc_say "running -- $_ng_stat" ;;
*) svc_say "not running -- $_ng_stat" ;;
esac
unset _ng_stat
return 0
fi
# No supervisor, so the pidfile is all there is -- and it is worth saying
# that the answer is coming from a weaker source than usual, because "not
# running" from a pidfile and "not running" from s6 are not the same claim.
svc_pid_alive "$NGINX_PID" \
&& svc_say "running (unsupervised, pid `cat "$NGINX_PID" 2>/dev/null`)" \
|| svc_say "not running (no supervisor; asked the pidfile)"
}
svc_dispatch "$@"
pkgs/anvil-core/payload/init.d/S62moonraker
-422
View File
@@ -1,422 +0,0 @@
#!/bin/sh
# moonraker -- the API on :7125 that Mainsail and HelixScreen both talk to.
#
# It already ships on stock firmware -- FlashForge simply comments it out of
# /usr/prog/klipper/start.sh and serves its own HTTP API from firmwareExe
# instead. The mod replaces firmwareExe, so that API is gone: this is the only
# thing on the printer that can answer a UI, and a printer whose moonraker is
# not running is a printer with no UI at all, however healthy klippy is.
#
# THIS AND S60nginx WERE ONE SCRIPT, S60web -- see the header there for why
# they are two now, and why the numbering has to keep this one after S60nginx
# and before S65camera.
#
# NEITHER SCRIPT HAS A SWITCH. MOD_WEB used to gate both and no longer exists;
# the argument is in S60nginx's header, and the short version is that "no web
# stack" was a state nobody wanted and everything had to reason about.
#
# WHAT CHANGED WHEN s6 ARRIVED, and what deliberately did not. moonraker is now
# a supervised service: $MODDIR/etc/s6/moonraker/ holds a `run` script,
# s6-supervise runs it, and when moonraker dies s6 starts it again. That is the
# whole point -- before this, a server that fell over stayed fallen until
# somebody noticed the UI was dead and ssh'd in, and this script had no way to
# do anything about it because a shell that has returned cannot restart
# anything. Three pieces of hand-rolled machinery went with it:
#
# the pidfile -> s6-svstat, asked of the supervisor. /run/moonraker.pid
# is no longer written by anything; it survives below
# only as the upgrade path, see stop().
# moonraker_check -> a real READINESS notification. The old check slept
# two seconds five times and called the service "up"
# if the pid was still there; the run script now
# notifies s6 when the API is LISTENING, and the
# detached reporter below blocks on that. See the long
# note in etc/s6/moonraker/run for why "listening" is
# the right definition and "klippy is ready" is not.
# stop-and-wait -> s6-svc -wD -d, which marks the service down BEFORE
# signalling it and returns only when it is gone.
#
# THE INTERFACE IS UNCHANGED, on purpose. `S62moonraker restart` over ssh -- the
# one you type over and over while chasing a moonraker config -- is in the docs
# and in people's hands, and changing the supervisor is not a reason to change
# what people type. start/stop/restart/status all still exist, still say
# "moonraker: ...", and still mean what they meant.
#
# WHY $MODDIR/etc/s6/moonraker/down STILL SHIPS. Ordering, which was always the
# larger half of the reason: s6-svscan starts a service the instant the
# directory appears, and moonraker must not be started before this script has
# checked its prerequisites -- nor before nginx, which S60nginx runs first so
# that a browser arriving early gets a page rather than a refused connection.
# `down` means the scanner supervises and waits. The long version is in the
# `down` file, next to the thing it explains.
MODDIR=/usr/data/anvil
[ -f $MODDIR/anvil.conf ] && . $MODDIR/anvil.conf
# The library path and the interpreter, from the one file that defines them.
# See anvil-env.sh for why naming them here instead was a bug: this is the
# script that had its own four-directory copy of the list while the installer
# checked with ten, and moonraker is the process that dies on
# "libpython3.8.so.1.0: cannot open shared object file" when it is wrong.
if [ -f $MODDIR/anvil-env.sh ]; then
. $MODDIR/anvil-env.sh
else
echo "moonraker: !! no $MODDIR/anvil-env.sh -- the mod payload is not installed"
fi
# start/stop/status/liveness, shared with the other services. See
# anvil-service.sh for the whole API -- including the svc_s6_* helpers, which
# are what the verbs below are built out of.
if [ -f $MODDIR/anvil-service.sh ]; then
. $MODDIR/anvil-service.sh
else
echo "moonraker: !! no $MODDIR/anvil-service.sh -- the mod payload is not installed"
exit 1
fi
SVC_NAME=moonraker
MOONRAKER_LOG=/usr/data/logs/moonraker.log
MOONRAKER_TMP=/usr/data/tmp
MOONRAKER_DATA=/usr/data
# THE PIDFILE THE OLD SCRIPT WROTE. Nothing here writes it any more -- the
# supervised service is identified by s6-svstat and the fallback path below
# writes it only on a printer that has no supervisor at all. It is named
# unconditionally because stop() has to be able to clean up after a payload
# that DID write it: see the comment there. Same shape, same reason, as
# S65camera's LEGACY_PIDFILE.
MOONRAKER_PID=/run/moonraker.pid
# ONE moonraker, at one path, and that path is on the DATA partition.
#
# Moonraker's python package is installed straight into the payload
# (`cp -a work/.moonraker/moonraker "$MOD_PAYLOAD/moonraker"`), so the tarball
# the installer extracts already puts the entry point here. Nothing else has to
# happen for this to exist, and if it does not exist the install did not
# complete.
#
# It used to run from /usr/prog/moonraker/moonraker/moonraker/moonraker.py --
# a byte-identical SECOND copy of this same tree that run-append.sh made after
# extracting the first one. That copy is gone, and the reasons it seemed
# necessary did not survive being checked:
#
# * /usr/prog is the small firmware partition, and it is what a stock
# FlashForge flash overwrites. Running from there meant that flashing stock
# firmware silently reverted Moonraker to FlashForge's 2022 build while the
# rest of the mod -- which lives on /usr/data and survives -- carried on
# assuming the new one. That is the "which moonraker is this printer
# actually running?" ambiguity we spent a release removing, reintroduced by
# the one file nobody looked at.
# * The moonraker-env virtualenv sitting beside it is unused. Imports resolve
# from /usr/prog/Python-3.8.2/lib/python3.8/site-packages instead --
# verified by running the printer's own interpreter on the real image,
# where moonraker-env is not on sys.path at all.
# * moonrakerDaemon, the one thing that did name the /usr/prog tree by
# absolute path, is never invoked: this script starts moonraker itself.
#
# There is no fallback here and there must not be one. The mod ships exactly
# one moonraker and installs it; if it is not at this path, that is a failed
# install to be reported, not a condition to route around. The `run` script
# names the same path, and the two must not drift.
MOONRAKER_MAIN=$MODDIR/moonraker/moonraker.py
# The service directory the scanner watches, spelled out of the two variables
# anvil-service.sh defines so that this script and S40s6 cannot disagree about
# where the scandir is. S6NAME is a separate variable from SVC_NAME even though
# the two are spelled the same today: SVC_NAME is what this script calls itself
# in the log, S6NAME is what s6 calls the thing it supervises, and keeping them
# apart is what stops a future rename of one silently renaming the other.
S6NAME=moonraker
SVCDIR=$SVC_S6_SCANDIR/$S6NAME
# Is there a live s6-supervise for our service directory RIGHT NOW?
#
# Two separate questions hide in "is s6 there", and both have to be yes before
# any of the verbs below can use it. svc_s6_running answers the first -- is the
# SCANNER alive and listening -- and does it by behaviour, by pinging the
# control FIFO. This answers the second: has that scanner actually picked our
# directory up and started a supervisor for it? A scandir the scanner has not
# read yet (the payload was extracted after S40s6 ran; somebody dropped the
# directory in by hand) looks exactly like a scandir with no moonraker in it,
# and `s6-svc` against it fails with "supervisor not running" -- a true
# statement about a state that fixes itself the moment the scanner rescans.
#
# s6-svok is that question in one syscall: it opens the supervise/control FIFO
# and exits 0 only if a supervisor has it open at the other end. Same technique
# as svc_s6_running, one level down, and the same reason for preferring it to a
# test for the directory: a servicedir on disk with no supervisor behind it is
# the stale-pidfile lie all over again.
supervised() {
[ -x "$SVC_S6_BIN/s6-svok" ] || return 1
"$SVC_S6_BIN/s6-svok" "$SVCDIR"
}
# Ask the scanner to pick up the service directory, then wait a moment for the
# supervisor to appear. svc_s6_running IS the rescan request -- `s6-svscanctl
# -a` is what it pings with -- so this is one call doing two jobs, which is
# worth saying out loud because it does not look like it does. The bound is 5
# seconds and is only ever paid on a printer where something is already wrong:
# on a normal boot the payload was extracted long before S40s6 started the
# scanner, so the supervisor is already there and the first test succeeds.
# Anything that spends time here is spending firmwareExe's boot, one script at
# a time in the foreground, which is why it is 5 and not 30.
await_supervisor() {
supervised && return 0
svc_s6_running || return 1
_mr_waited=0
while ! supervised && [ $_mr_waited -lt 5 ]; do
sleep 1
_mr_waited=$((_mr_waited + 1))
done
unset _mr_waited
supervised
}
# ---- the fallback, and why moonraker has one when the camera does not ------
#
# WHAT HAPPENS WHEN THERE IS NO SUPERVISOR. MOD_S6=0 is a switch S40s6 honours,
# so "no scanner" is a state a user can choose; and a payload whose s6 failed
# to build, or an install that got half way, is a state nobody chose but that
# happens anyway -- the -D_FILE_OFFSET_BITS=64 trap in
# tools/supervisor/README.md produces binaries that exec fine and then go blind
# on readdir(). The two services already migrated answered this question
# differently on purpose: S60nginx falls back to starting nginx directly,
# S65camera refuses and explains. moonraker FALLS BACK, with nginx, and the
# reason is that it is the same decision, not a second one:
#
# * THE WEB STACK IS ONE THING, so "it runs" is the only state anyone asked
# for. A dead supervisor that left nginx serving and moonraker refusing
# would invent a second one that nobody chose and that is the worse of the
# two to be handed: Mainsail loads, connects to nothing, and blames
# Klipper. Every minute spent debugging that is spent in the wrong
# subsystem.
# * MOONRAKER IS NOT ONLY THE UI. bin/ff-startup.py -- which S70klipper hands
# the whole boot to -- drives the panel from moonraker's API and cannot
# learn klippy_state, report an MCU fault, or run the firmware-config
# migration without it. Refusing here would turn a supervisor that did not
# build into a printer that cannot bring itself up or say why, and the way
# you would be told is the screen and the web UI that both just went away.
# That is a strictly worse failure than the camera's, which is why the
# camera's answer is not copied: a printer with no camera is a printer with
# no camera, and a printer with no moonraker is a brick with a login shell.
# * The old path is not hypothetical or unmaintained -- it is the code that
# shipped for the whole life of this script, it is a dozen lines, and
# case-moonraker.sh still exercises it on the no-scanner path.
# * The classic reason NOT to fall back is that you end up with two of them:
# a directly-started moonraker, then a scanner arriving later and starting a
# second one on a port that is already bound. That cannot happen here, and
# it is the `down` file that makes it so -- the scanner never starts
# moonraker on its own, only this script does. And when this script does,
# the `run` script sweeps any moonraker that is already running before
# exec'ing its own, so the hand-over from the fallback to the supervised
# copy is clean rather than a fight over :7125.
#
# What it is NOT allowed to be is silent. A printer running the fallback has
# lost automatic restart-on-crash without losing the API, which is exactly the
# kind of half-degraded state that gets discovered six months later, so it says
# so every time, with "!!", which is what people grep a boot log for.
start_unsupervised() {
svc_warn "no supervisor -- starting moonraker directly, it will NOT be restarted if it dies"
# The wipe is guarded on moonraker not already running: re-running this
# script over ssh against a live server would otherwise delete the
# directory it is streaming an upload into. (The supervised path has no
# such guard and does not need one -- see the comment in the run script.)
if ! svc_pid_alive "$MOONRAKER_PID"; then
rm -rf $MOONRAKER_TMP
fi
mkdir -p $MOONRAKER_TMP
export TMPDIR=$MOONRAKER_TMP
mkdir -p `dirname $MOONRAKER_LOG`
# Yield to klippy on a two-core SoC: see NICE_MOONRAKER in anvil.conf for
# why this is insurance rather than a fix. Unset or 0 = normal priority.
SVC_NICE="${NICE_MOONRAKER:-0}"
svc_start_daemon "$MOONRAKER_PID" "$FF_PYTHON" \
"$MOONRAKER_MAIN" -d "$MOONRAKER_DATA" || return 1
unset SVC_NICE
svc_say "started (unsupervised, $MOONRAKER_MAIN)"
# The pre-s6 liveness check, kept whole for the path that still needs it.
# It is a guess -- a pid that is still there is not an API that answers --
# and it is why the supervised path has a readiness notification instead.
svc_detach moonraker_check
}
# Component-load failures happen within seconds of exec, so a server still
# alive at 10s is up. Confirming and returning beats holding the boot log open
# for a minute: this runs in the background while S65camera, S70klipper and
# S80ui are writing to the same file, and an interleaved log is what made one
# user's boot unreadable at the point they were trying to read it.
moonraker_check() {
waited=0
while [ $waited -lt 5 ]; do
sleep 2
waited=$((waited + 1))
svc_pid_alive "$MOONRAKER_PID" && continue
svc_warn "NOT running after $((waited * 2))s -- last words:"
tail -n 15 "$MOONRAKER_LOG" 2>/dev/null | sed 's/^/ /'
return 1
done
svc_say "up"
return 0
}
# The detached half of the supervised path, and the thing the whole phase was
# for. Nothing reads its exit code; it reports through the boot log only,
# exactly as moonraker_check did.
#
# svc_s6_ready blocks on s6-svwait -U, which returns when the run script writes
# to its notification-fd -- and the run script does that only once moonraker has
# BOUND :7125, not when it was forked. So the "answering" line below is true
# when it is printed, which is the entire difference between this and the old
# check that watched a pid and said "up".
#
# 120 seconds, against the run script's own 180-second probe. A caller-side
# bound equal to the callee's would be a coin flip over which one reports
# first; a shorter one means this says "not answering yet" while the prober is
# still watching, which is the honest way round -- the service is still coming
# up and s6 will keep it, and the next `S62moonraker status` tells the truth.
# The boot never waits on any of it: this runs detached, which is what
# svc_detach is for, and the trade (log lines arriving out of order with the
# services that started after this one) is the one S50wifi and S65camera both
# already make.
ready_report() {
if svc_s6_ready "$S6NAME" 120; then
svc_say "answering on :7125"
return 0
fi
svc_warn "NOT answering on :7125 after 120s -- `svc_s6_stat "$S6NAME"` -- last words:"
tail -n 15 "$MOONRAKER_LOG" 2>/dev/null | sed 's/^/ /'
return 1
}
start() {
# Every branch below says something. There is no stock server left to
# quietly take over, so the boot log is the only place that can explain a
# printer with no UI. A missing prerequisite is REPORTED and returns
# non-zero; it is not a crash and it is not a stack trace out of s6, which
# is what handing a supervisor a service it cannot run produces -- once a
# second, for ever, in a log nobody is reading.
if ! anvil_python_ok; then
svc_warn "no interpreter at ${FF_PYTHON:-/usr/prog/Python-3.8.2/bin/python3} -- not started"
return 1
fi
if [ ! -f "$MOONRAKER_MAIN" ]; then
svc_warn "nothing at $MOONRAKER_MAIN -- the install did not complete"
return 1
fi
mkdir -p `dirname $MOONRAKER_LOG`
# The third prerequisite, which is new: the service directory itself. It is
# staged into the payload by bin/patch.sh and named in the install
# manifest, so its absence means a payload that was built without it or an
# install that did not finish -- and it is deliberately NOT created here.
# Writing it at runtime would mean the scandir contained a directory no
# manifest lists, which is the exact thing the manifest exists to stop, and
# it would put the `run` script's text inside this file where nobody would
# ever look at it again.
if [ ! -x "$SVCDIR/run" ]; then
svc_warn "no $SVCDIR/run -- this payload did not ship the s6 service"
start_unsupervised
return $?
fi
if ! await_supervisor; then
start_unsupervised
return $?
fi
# And up it goes. svc_s6_up is `s6-svc -wu -u` with a bound: it asks for the
# service to be brought up -- overriding the `down` file, which is how a
# supervised-but-idle service becomes a running server -- and does not
# return until it IS up. UP, note, not READY: -wu returns as soon as s6 has forked
# the run script, which is exactly what we want on the boot path, because
# moonraker takes the better part of a minute to import its components on
# this box and firmwareExe is running these scripts one at a time in the
# foreground. 15 seconds is the shared default and is a bound on the FORK,
# not on the startup; anything that takes longer than that to fork is a
# scandir problem, not a moonraker problem.
if ! svc_s6_up "$S6NAME" 15; then
# Up it did not come. s6 has kept the reason where a reason belongs:
# the `run` script's stderr is the scanner's stderr, which S40s6 points
# at /usr/data/logs/s6.log. Say where, rather than paraphrasing
# something we have not read.
svc_warn "FAILED to come up -- `svc_s6_stat "$S6NAME"` -- see /usr/data/logs/s6.log"
return 1
fi
# Forked, not serving. The difference is the whole point of the next two
# lines, and of the notification-fd in the service directory.
svc_detach ready_report
svc_say "started ($MOONRAKER_MAIN); waiting for the API in the background"
return 0
}
stop() {
# A stop that the supervisor undoes is the classic mistake of this
# migration, and `s6-svc -d` is what makes it not happen: it does not just
# signal the process, it tells s6 the service is WANTED DOWN, and s6 then
# does not start it again until something says otherwise. Killing the pid
# instead -- which is all this script could do before, and all
# svc_stop_daemon still does below -- would get moonraker restarted a
# second later by a supervisor doing exactly what it was asked to do.
#
# -wD (inside svc_s6_down) is the other half: it returns when the service is
# really down, not when the signal has been sent. That is the busybox
# start-stop-daemon correction from anvil-service.sh, except in C, and it is
# what makes `restart` safe without a sleep in front of it: the 15-second
# bound is the same one svc_stop_daemon used, long enough for moonraker to
# finish writing its database and short enough not to hang a boot.
if supervised; then
svc_s6_down "$S6NAME" 15 >/dev/null 2>&1 \
|| svc_warn "s6 did not report $S6NAME down: `svc_s6_stat "$S6NAME"`"
fi
# AND THE PIDFILE, which covers two live cases rather than being tidiness.
#
# * The fallback above really did start a moonraker with
# start-stop-daemon on a printer with no scanner, and that copy has no
# supervisor to take it down. The pidfile is the only handle on it.
# * THE UPGRADE PATH. A printer running the pre-phase-5 payload has a
# start-stop-daemon'd moonraker alive with its pid in this file.
# Install this payload, then type `S62moonraker restart` instead of
# rebooting, and without this line that server is still there holding
# :7125 -- and the supervised copy this script is about to start cannot
# bind, exits, gets respawned, and loops for ever while s6-svstat says
# "up". (The run script sweeps for exactly that leftover as a second
# line of defence; this is the first, and it is the polite one.)
#
# svc_stop_daemon returns 0 immediately when the pidfile does not exist,
# which is every printer that has rebooted once since the upgrade, so this
# costs nothing in the normal case. It also removes the file, which is what
# keeps `status` from answering out of a pid the kernel has since handed to
# something else.
svc_stop_daemon "$MOONRAKER_PID"
svc_say "stopped"
}
status() {
# Asked of the supervisor when there is one, because it is the only thing
# that knows. s6-svstat's own wording is passed through -- "up (pid 1234
# pgid 1234) 71 seconds", "down (signal SIGTERM) 4 seconds, normally up" --
# for the reason anvil-service.sh gives for svc_s6_stat: it is the answer
# you came for, in the shape you would have typed the command to get.
#
# The words "running" and "not running" are still the first thing on the
# line, though, and that is not decoration either. It is the vocabulary
# every other service in init.d/ answers in, it is what a human skims a
# boot log for, and it is what case-moonraker.sh looks for. The s6 detail
# is appended to it rather than replacing it.
if supervised; then
_mr_stat=`svc_s6_stat "$S6NAME"`
case "$_mr_stat" in
up*) svc_say "running -- $_mr_stat" ;;
*) svc_say "not running -- $_mr_stat" ;;
esac
unset _mr_stat
return 0
fi
# No supervisor, so the pidfile is all there is -- and it is worth saying
# that the answer is coming from a weaker source than usual, because "not
# running" from a pidfile and "not running" from s6 are not the same claim.
svc_pid_alive "$MOONRAKER_PID" \
&& svc_say "running (unsupervised, pid `cat "$MOONRAKER_PID" 2>/dev/null`)" \
|| svc_say "not running (no supervisor; asked the pidfile)"
}
svc_dispatch "$@"
pkgs/anvil-core/payload/init.d/S65camera
-261
View File
@@ -1,261 +0,0 @@
#!/bin/sh
# Camera -- mjpg-streamer on :8080, supervised by s6.
#
# WHY THIS EXISTS -- the same trap as WiFi. /usr/prog/mjpg-streamer ships on
# stock firmware (mjpg_streamer, input_uvc.so, output_http.so, its own
# libjpeg.so.9 and a www/ dir), and app_startup.sh only ever exports
# LD_LIBRARY_PATH for it -- it never launches it. firmwareExe did, and in fact
# served the stream from its own OpenCV capture, labelling itself
# "Server: MJPG-Streamer/0.2". So the moment the mod replaces firmwareExe the
# camera goes dark, with a perfectly good streamer sitting unused on the disk.
#
# Nothing has to be built or shipped for this: we start the binary FlashForge
# already put there. Mainsail then finds it through the [webcam] entry in
# moonraker.conf, which points at nginx's /webcam/ proxy rather than :8080
# directly, so a tunnel that forwards one port still shows the stream.
#
# WHAT CHANGED IN PHASE 4. This script used to BE the supervisor. It carried a
# `while true; do mjpg_streamer ...; sleep 3; done` respawn loop and a poll
# that waited up to thirty seconds for /dev/video0, both inside a background
# subshell held open by a pidfile -- a pidfile that pointed at THE LOOP rather
# than at the streamer, because killing only the streamer got it restarted
# three seconds later. All of that now lives in s6:
#
# the respawn loop -> s6-supervise, which is C and does not race itself
# the 30s poll -> the top of $MODDIR/etc/s6/camera/run, where a
# timeout means "exit and let s6 try again" instead of
# "give up until a human reruns this script"
# "is it running?" -> s6-svstat, asked of the supervisor rather than
# guessed from `ps | grep mjpg_streamer`
# the stop dance -> s6-svc -wD -d, which marks the service down BEFORE
# signalling it and returns only when it is gone
#
# What is left here is the INTERFACE. `S65camera restart` over ssh is in the
# docs and in people's hands, and changing the supervisor is not a reason to
# change the thing people type. Every verb still works, still says "camera:",
# and still returns promptly.
#
# IT STILL MUST NOT MAKE THE BOOT WAIT, and the shape of that has changed. It
# is no longer this script that waits for the camera at all -- the run script
# does. But somebody has to SAY, in the boot log, whether the camera came up,
# and that answer takes up to thirty seconds to exist. So start() asks s6 to
# bring the service up (fast: -wu returns as soon as the streamer is forked)
# and detaches a reporter that blocks on READINESS and prints the verdict when
# it arrives. The boot never waits on hardware; the log still gets the truth,
# just out of order with the services that started after this one. That is
# svc_detach, and it is the same trade S50wifi makes -- see anvil-service.sh.
MODDIR=/usr/data/anvil
[ -f $MODDIR/anvil.conf ] && . $MODDIR/anvil.conf
[ -f $MODDIR/anvil-env.sh ] && . $MODDIR/anvil-env.sh
# start/stop/status/liveness, shared with the other services -- including the
# svc_s6_* helpers this script is now written almost entirely in. See
# anvil-service.sh for the whole API.
if [ -f $MODDIR/anvil-service.sh ]; then
. $MODDIR/anvil-service.sh
else
echo "camera: !! no $MODDIR/anvil-service.sh -- the mod payload is not installed"
exit 1
fi
SVC_NAME=camera
# The name of the service DIRECTORY inside SVC_S6_SCANDIR, which is a different
# thing from SVC_NAME even though they are spelled the same today: SVC_NAME is
# what this script calls itself in the log, S6NAME is what s6 calls the thing
# it supervises. Keeping them as two variables is what stops a future rename of
# one from silently renaming the other.
S6NAME=camera
S6DIR=$SVC_S6_SCANDIR/$S6NAME
CAMDIR=/usr/prog/mjpg-streamer
VIDEO=/dev/video0
LOG=/usr/data/logs/mjpg.log
RES="${MOD_CAM_RES:-1280x720}"
FPS="${MOD_CAM_FPS:-30}"
PORT="${MOD_CAM_PORT:-8080}"
# The pidfile the OLD supervisor used. Nothing here writes it any more; it is
# named only so stop() can clean up after a payload that did -- see the comment
# there. Once no printer in the field can still be running the pre-phase-4
# payload this and the two lines that use it can go.
LEGACY_PIDFILE=$MODDIR/mjpg.pid
# ---- what to do when there is no supervisor --------------------------------
#
# MOD_S6=0 in anvil.conf turns the scanner off, and a payload whose s6 build
# went wrong has no scanner either -- the -D_FILE_OFFSET_BITS=64 trap in
# tools/supervisor/README.md produces binaries that exec fine and then go blind
# on readdir(). Both are real states on a real printer, and both used to be
# impossible because this script supervised itself.
#
# The choice here is to DEGRADE LOUDLY AND DO NOTHING, rather than to fall back
# to the old hand-rolled loop. Keeping the loop as a fallback would mean two
# implementations of the camera, one of them exercised only on printers that
# are already broken and therefore never tested -- which is precisely how the
# five services grew five different liveness checks in the first place. It
# would also be the worst kind of failure to diagnose: a camera that works
# means nobody notices the supervisor is dead until Klipper needs it too.
#
# So: one line naming the actual problem, and a non-zero exit. firmwareExe
# already prints the exit status of a service that fails, S40s6 has already
# printed its own "not answering" warning by this point in the boot, and the
# two lines together say the whole story. A printer with no camera is a
# printer with no camera; a printer with no supervisor is about to have larger
# problems, and hiding one behind the other helps nobody.
s6_or_complain() {
if svc_s6_running; then
return 0
fi
svc_warn "the s6 supervisor is not running -- nothing can start the camera"
svc_warn "check MOD_S6 in $MODDIR/anvil.conf, then $MODDIR/init.d/S40s6 status"
return 1
}
# Wait for s6-supervise to appear for our service directory, then bring it up.
#
# WHY THIS IS NOT JUST svc_s6_up. s6-svc talks to $S6DIR/supervise/control, a
# FIFO that s6-supervise creates -- and s6-supervise only exists once the
# SCANNER has noticed the directory. On a first boot after an update the
# directory is brand new: the scandir was scanned before the payload landed, or
# scanned when the directory was empty, and the scanner has not looked since.
# svc_s6_running is what pokes it -- `s6-svscanctl -a` is literally the request
# to rescan, which is why the liveness check and the rescan are the same call
# -- but the scan, the fork and the mkfifo take a moment, and a s6-svc that
# arrives during that moment fails with "unable to control" rather than
# waiting. Five one-second tries covers it with room to spare and costs
# nothing on the normal path, where the first attempt succeeds.
s6_up_when_ready() {
_cam_try=0
while [ $_cam_try -lt 5 ]; do
if svc_s6_up "$S6NAME" 15; then
unset _cam_try
return 0
fi
sleep 1
_cam_try=$((_cam_try + 1))
done
unset _cam_try
return 1
}
# The detached half. Nothing reads its exit code; it reports through the boot
# log only, exactly as the old detached supervise() did.
#
# svc_s6_ready blocks on s6-svwait -U, which returns when the run script writes
# to its notification-fd -- and the run script does that only once the streamer
# has BOUND ITS PORT, not when it was forked. So the "streaming" line below is
# true when it is printed, which is the entire difference between this and the
# old "waiting for /dev/video0 in the background" that was printed hopefully
# and never followed up.
#
# 35 seconds, not 30: the run script waits up to 30 for the device and then
# still has to start the streamer and bind the port. A caller-side bound equal
# to the callee's would report "not ready" for a camera that was about to work.
ready_report() {
if svc_s6_ready "$S6NAME" 35; then
svc_say "streaming on :$PORT ($RES @ ${FPS}fps)"
else
svc_warn "not streaming after 35s -- no camera at $VIDEO? see $LOG"
fi
}
start() {
# MOD_CAM USED TO BE READ HERE and is gone. A printer with no camera does
# not need a switch: the checks below already find that out -- there is no
# $VIDEO, mjpg-streamer says so, and the log says which. The switch existed
# to skip a 35-second wait for a camera that was known to be absent, and
# what it actually did was give every question about the camera two
# answers. The `down` file in the service directory stays, for the reason
# written inside it: it is what stops s6 starting the streamer before this
# script has looked for the device.
s6_or_complain || return 1
if [ ! -d "$S6DIR" ]; then
svc_warn "no service directory at $S6DIR -- this payload did not ship it"
return 1
fi
# Kept from the pre-s6 script, and it still earns its place: without it the
# first sign of a stock firmware that moved or dropped mjpg-streamer would
# be s6 respawning a run script that fails on exec, once a second, forever.
# Saying it once here is cheaper to read than that in a log.
[ -x "$CAMDIR/mjpg_streamer" ] || { svc_warn "no $CAMDIR/mjpg_streamer"; return 1; }
if ! s6_up_when_ready; then
svc_warn "s6 would not bring $S6NAME up: `svc_s6_stat "$S6NAME"`"
return 1
fi
# Up, not ready. The difference is the whole point of the next two lines.
svc_detach ready_report
svc_say "starting under s6; waiting for $VIDEO in the background (:$PORT, $RES @ ${FPS}fps)"
return 0
}
stop() {
# THE OLD ORDERING PROBLEM IS GONE, and it is worth saying why rather than
# just deleting the comment that explained it. The old stop() had to kill
# the supervisor FIRST and the streamer second, because the supervisor was
# our own `while true; ... sleep 3` loop: signal the streamer while the
# loop still lived and the loop simply started another one three seconds
# later, so a stop in the wrong order looked like it had worked and had
# not. Then it had to wait fifteen seconds for the streamer to actually
# die, because killall returns as soon as the signal is sent and a start()
# arriving a moment later would see the dying process, say "already
# running", and leave the camera dark.
#
# s6-svc -d makes all three of those disappear, and not by being careful in
# a different order -- by not having the race. It sets the service's wanted
# state to DOWN and only then signals it, so there is no window in which
# something is still trying to bring it back; and -wD does not return until
# the process is really gone, which is the wait we were hand-rolling. The
# supervisor itself is deliberately NOT killed: it stays, idle, holding the
# service in the down state, which is what makes the next `start` a single
# s6-svc -u instead of a rescan.
if svc_s6_running && [ -d "$S6DIR" ]; then
svc_s6_down "$S6NAME" 15 >/dev/null 2>&1 \
|| svc_warn "s6 did not report $S6NAME down: `svc_s6_stat "$S6NAME"`"
fi
# THE UPGRADE PATH, and the one place the deleted code still matters. A
# printer running the pre-phase-4 payload has our old respawn loop alive
# with its pid in $LEGACY_PIDFILE. Install this payload, then type
# `S65camera restart` instead of rebooting, and without these two lines
# that loop is still there -- respawning mjpg_streamer on :8080 and
# fighting the s6 service for the port, with neither side ever winning and
# nothing in any log saying why. svc_stop_daemon returns 0 immediately when
# the pidfile does not exist, which is every printer that has rebooted
# once, so this costs nothing in the normal case.
if [ -f "$LEGACY_PIDFILE" ]; then
svc_warn "found a pre-s6 respawn loop still running -- stopping it"
svc_stop_daemon "$LEGACY_PIDFILE"
killall mjpg_streamer 2>/dev/null
fi
svc_say "stopped"
}
status() {
# Asked of the supervisor, not of the process table. `ps | grep
# mjpg_streamer` was the old answer and it could not tell a streamer that
# is up from one that s6 is about to respawn, nor say anything at all about
# a service that is deliberately down. s6-svstat's own wording -- "up (pid
# 1234) 71 seconds", "down 5 seconds, normally up" -- is already the answer
# you came for, so it is passed through rather than paraphrased, exactly as
# S50wifi passes ifconfig through.
#
# The no-scanner case still prints a line and still names itself: a service
# that answers `status` with silence is indistinguishable from one that
# crashed, and every service in this directory owes the boot log an answer.
if svc_s6_running; then
if [ -d "$S6DIR" ]; then
svc_say "s6: `svc_s6_stat "$S6NAME"`"
else
svc_say "no service directory at $S6DIR"
fi
else
svc_say "not running -- the s6 supervisor is not running either"
fi
# And the hardware, which is the question somebody actually has when the
# stream is black. Kept verbatim from the pre-s6 script.
[ -e "$VIDEO" ] && svc_say "$VIDEO present" || svc_say "no $VIDEO"
[ -f "$LEGACY_PIDFILE" ] && svc_warn "a pre-s6 respawn loop is still around ($LEGACY_PIDFILE)"
return 0
}
svc_dispatch "$@"
pkgs/anvil-core/payload/init.d/S70klipper
-216
View File
@@ -1,216 +0,0 @@
#!/bin/sh
# Klipper.
#
# WHY THIS EXISTS -- the single most important dependency in this whole mod:
# on stock firmware nothing in /etc/init.d starts Klipper. It is FlashForge's
# firmwareExe that runs "/usr/prog/klipper/start.sh &" (verified with
# `strings firmwareExe`). So the moment HelixScreen replaces firmwareExe, the
# printer boots to a working screen with NO klippy behind it: no motion, no
# heaters, no printing.
#
# Therefore Klipper startup is owned here, independently of the UI.
#
# Double-start guard: we only start Klipper if it is not already running, and
# start.sh carries the same guard, so a restart over ssh cannot end up with two
# klippys against the same MCU.
#
# WHY IT ALSO SUPERVISES -- the heater board (mcu 'eheaterboard' on
# /dev/ttyS4) is frequently not answering yet when klippy first opens the
# port. Klipper polls it for 90s (connect_uart) and then gives up with "MCU
# error during connect". klippy does NOT exit and does NOT retry on its own:
# _connect() sets the error state, returns, and fires FlashForge's own
# klippy:notify_mcu_error event over /tmp/uds. On stock, firmwareExe is the
# API client listening for that event, and it restarts the firmware -- which
# is why FlashForge's UI needs two or three attempts before every board is
# found. Nothing in the mod plays that role, so klippy would sit in the error
# state forever with no heaters. Restarting klippy achieves what the stock
# retry achieves: reopening the port toggles DTR/RTS and the board comes up.
#
# The retry is the safety net, not the fix: the actual fix is the bootloader
# handshake (ff_mcu_bringup.py). A board that missed its window still gets
# another chance on each restart here, because start.sh redoes the bring-up
# every time it runs.
#
# WHO NORMALLY DOES ALL THIS NOW. bin/ff-startup.py, which runs from the
# firmwareExe wrapper right after these init scripts. It owns the sequence
# because the bring-up has to happen before klippy opens the ports and the
# retry that fixes a stranded board is "hand it over again, then reopen" --
# so the thing doing the handshake has to be the thing restarting klippy. It
# also has moonraker to ask, instead of tailing printer.log for markers, and
# a screen to say which board is missing.
#
# This script therefore STANDS ASIDE when that program is going to run, and
# does the whole job itself when it cannot -- no interpreter, no script,
# MOD_STARTUP=0. Klipper starting is the single most important thing the mod
# does and it must not come to depend on python. `force-start` ignores the
# hand-off; the wrapper uses it as a last resort if ff-startup returned and
# left no klippy behind.
#
# It is not a GPIO poke: firmwareExe drives only PB07 (wifi module power) and
# gpios 34/35, and that pair is the NRST/BOOT0 of the /dev/ttyS3 board, driven
# only by AppUpdateThread when flashing MCU firmware from the DeviceOTA
# screen. Neither runs on a normal boot.
MODDIR=/usr/data/anvil
[ -f $MODDIR/anvil.conf ] && . $MODDIR/anvil.conf
# start/stop/status/liveness, shared with the other services. See
# anvil-service.sh for the whole API.
if [ -f $MODDIR/anvil-service.sh ]; then
. $MODDIR/anvil-service.sh
else
echo "klipper: !! no $MODDIR/anvil-service.sh -- the mod payload is not installed"
exit 1
fi
SVC_NAME=klipper
# `force-start` is this script's own verb; naming it here keeps the usage line
# svc_dispatch prints true. See the case block at the bottom.
SVC_EXTRA_VERBS="|force-start"
KLIPPER_TRIES=${MOD_KLIPPER_TRIES:-6}
PRINTER_LOG=/usr/data/logs/printer.log
# klippy is started by FlashForge's start.sh, not by us, so there is no pidfile
# and the only handle on it is the name: svc_proc_alive 'klippy\.py', which is
# the pgrep-with-a-ps-fallback this script used to carry itself.
#
# klippy_running is deliberately MORE than that. /tmp/uds is the socket klippy
# serves its API on, and it is what start.sh itself looks at before deciding
# whether to start anything -- so a printer with the socket present is one
# start.sh will refuse to touch, whatever `ps` says. Both questions matter and
# they are not the same question, which is why this one stays here rather than
# moving into anvil-service.sh: no other service has a socket to ask about.
klippy_running() {
[ -S /tmp/uds ] && return 0
svc_proc_alive 'klippy\.py' && return 0
return 1
}
# The two outcomes klippy actually writes to printer.log. There is no
# "Printer is ready" line to look for -- _set_state() does not log -- but a
# healthy klippy emits a "Stats <n>:" line every second.
klippy_verdict() {
tail -n +$(($1 + 1)) "$PRINTER_LOG" 2>/dev/null | awk '
/MCU error during connect/ { print "error"; exit }
/Stats [0-9]/ { print "ready"; exit }'
}
supervise() {
attempt=1
while [ $attempt -le $KLIPPER_TRIES ]; do
# NOT `wc -l < "$PRINTER_LOG" 2>/dev/null`: redirections are applied
# left to right, so the input redirect fails and the shell prints
# "can't open ...: no such file" BEFORE 2>/dev/null is in effect. On
# a first boot printer.log does not exist yet, so that message landed
# in the boot log every time, interleaved with whatever else was
# writing -- which is what made one user's log unreadable at exactly
# the point they were trying to diagnose.
log_lines=`cat "$PRINTER_LOG" 2>/dev/null | wc -l`
[ -n "$log_lines" ] || log_lines=0
# A failed connect takes ~90s to be reported, so wait out more than
# that before concluding anything.
waited=0
verdict=""
while [ $waited -lt 30 ]; do
sleep 5
waited=$((waited + 1))
verdict=`klippy_verdict "$log_lines"`
[ -n "$verdict" ] && break
done
case "$verdict" in
ready)
svc_say "ready (attempt $attempt)"
return 0
;;
"")
# Neither marker showed up. Something other than an MCU
# connect is going on; restarting would only hide it.
svc_say "no verdict after attempt $attempt -- leaving it alone"
return 0
;;
esac
svc_warn "MCU connect failed (attempt $attempt/$KLIPPER_TRIES) -- restarting"
klippy_down
/usr/prog/klipper/start.sh
attempt=$((attempt + 1))
done
svc_warn "still no MCU after $KLIPPER_TRIES attempts -- giving up"
return 1
}
# Take klippy down and leave nothing behind that would stop the next start.
#
# The wait is not optional. klipperDaemon stop signals and returns; klippy is
# still holding /dev/ttyS4 for a moment after that, and start.sh reopening the
# port while the old process still has it is exactly the MCU failure this
# script exists to retry out of. Bounded at 15s for the reason svc_stop_daemon
# is: a stuck klippy should be said out loud, not waited on forever.
#
# And then the socket. start.sh bails out if /tmp/uds is still there, and
# klippy does not unlink it on exit -- so a stale socket is a pidfile that
# lies, in precisely the sense svc_stop_daemon removes pidfiles. It is only
# removed once klippy is definitely gone: removing it out from under a live
# klippy would leave a running printer nothing could talk to.
klippy_down() {
[ -x /usr/prog/klipper/klipperDaemon ] && /usr/prog/klipper/klipperDaemon stop
stopping=0
while svc_proc_alive 'klippy\.py' && [ $stopping -lt 15 ]; do
sleep 1
stopping=$((stopping + 1))
done
if svc_proc_alive 'klippy\.py'; then
svc_warn "klippy still running after ${stopping}s"
return 1
fi
rm -f /tmp/uds
return 0
}
# Is bin/ff-startup.py actually going to run and do this? Every condition the
# wrapper checks, checked the same way -- if any of them is false this script
# keeps the job.
startup_owns_klipper() {
[ "${MOD_STARTUP:-1}" = 1 ] || return 1
[ -f $MODDIR/bin/ff-startup.py ] || return 1
[ -x /usr/prog/Python-3.8.2/bin/python3 ] || command -v python3 >/dev/null 2>&1 || return 1
return 0
}
start() {
if klippy_running; then
svc_say "already running"
return 0
fi
if [ "$1" != force ] && startup_owns_klipper; then
svc_say "left to ff-startup.py (it does the MCU bring-up first)"
return 0
fi
[ -x /usr/prog/klipper/start.sh ] || { svc_warn "start.sh missing"; return 1; }
# start.sh does the MCU bring-up itself, and that costs a few seconds per
# unresponsive board -- so it goes to the background; boot must not stall.
svc_detach /usr/prog/klipper/start.sh
svc_say "start.sh launched"
# In the background: boot must not wait minutes for the heater board.
svc_detach supervise
}
stop() {
klippy_down
svc_say "stopped"
}
status() {
klippy_running && svc_say "running" || svc_say "not running"
}
# force-start and restart are handled here, before svc_dispatch, because both
# need the argument svc_dispatch does not pass: `start force`. This script's
# start() normally STANDS ASIDE for ff-startup.py, so a plain `stop; start` --
# which is what svc_dispatch's restart is -- would stop klippy and then decline
# to start it, leaving the printer with no motion and no heaters. A restart
# asked for by hand means restart it, so it forces. There is no `sleep 2` in
# front of the start any more: stop() now waits for klippy to actually be gone
# and clears the stale socket, which is what the sleep was approximating.
case "$1" in
force-start) start force ;;
restart) stop; start force ;;
*) svc_dispatch "$@" ;;
esac
pkgs/anvil-core/payload/init.d/S80ui
-71
View File
@@ -1,71 +0,0 @@
#!/bin/sh
# Decide WHETHER the UI should run, and record the verdict in $MODDIR/.ui-choice
# for the firmwareExe wrapper to act on.
#
# The decision is separated from the launching because the UI must run in the
# FOREGROUND of the firmwareExe wrapper (so app_startup.sh's `ps | grep
# firmwareExe` watchdog sees a live process). An init script cannot hold that
# foreground, so it only chooses.
#
# That is also why this script uses none of anvil-service.sh's daemon helpers:
# there is no daemon here, no pidfile and nothing to supervise. It takes the
# common shape -- svc_say, svc_warn, svc_dispatch -- and nothing else.
#
# HelixScreen is the only UI: FlashForge's firmwareExe is retired and there is
# nothing to fall back to. There is no crash counter and no automatic headless
# latch, and there is no MOD_UI either -- the UI is started whenever it is
# INSTALLED, which is the one fact that decides it. A build that does not want
# the screen is BUILD_HELIX=0, which ships no anvil-helixscreen and lands in
# have_helix() below; a printer that wants it off can `opkg remove
# anvil-helixscreen` or stop the process. Neither needs a line in anvil.conf,
# and the flag cost every headless question two answers -- "not installed" and
# "installed but switched off" -- that behave identically from the screen.
#
# Whatever happens here, ssh comes from the stock /etc/init.d/S50dropbear,
# which busybox init ran long before this script, so it is up regardless, and
# so are S60nginx/S62moonraker (Mainsail) and S70klipper.
MODDIR=/usr/data/anvil
[ -f $MODDIR/anvil.conf ] && . $MODDIR/anvil.conf
# start/stop/status/liveness, shared with the other services. See
# anvil-service.sh for the whole API.
if [ -f $MODDIR/anvil-service.sh ]; then
. $MODDIR/anvil-service.sh
else
echo "ui: !! no $MODDIR/anvil-service.sh -- the mod payload is not installed"
exit 1
fi
SVC_NAME=ui
CHOICE=$MODDIR/.ui-choice
have_helix() { [ -x $MODDIR/helixscreen/bin/helix-launcher.sh ] || [ -x $MODDIR/helixscreen/bin/helix-screen ]; }
start() {
want=helix
if ! have_helix; then
svc_say "HelixScreen is not installed -- staying headless"
want=none
fi
if [ "$want" = none ]; then
svc_warn "printer will be headless -- no UI on the screen"
# A continuation of the line above, so it is indented rather than
# prefixed: the "ui: !!" that somebody greps for is on the warning.
echo " ssh is still up via /etc/init.d/S50dropbear, and so is Mainsail"
fi
echo "$want" > "$CHOICE"
svc_say "chose $want"
}
stop() {
killall helix-screen helix-watchdog helix-splash 2>/dev/null
svc_say "stopped"
}
status() {
svc_say "choice: `cat $CHOICE 2>/dev/null || echo unset`"
svc_say "installed: `have_helix && echo yes || echo no`"
}
svc_dispatch "$@"
pkgs/anvil-core/payload/prog/firmwareExe
+177 -124
View File
@@ -4,23 +4,22 @@
# The stock boot chain is:
# busybox init -> /etc/init.d/rcS -> S99factory_test_shell
# -> /usr/prog/app_startup.sh -> firmwareExe
# and firmwareExe itself is what launches "/usr/prog/klipper/start.sh &".
# Replacing this one file owns the whole userspace boot, so app_startup.sh, rcS
# and the init chain stay 100% STOCK and are never patched.
#
# Because everything funnels through this one path, replacing this binary
# with a script is the cleanest possible integration point: app_startup.sh,
# rcS and the whole init chain stay 100% STOCK and are never patched.
# THE WATCHDOG CONSTRAINT, which shapes the whole file: app_startup.sh greps
# `ps` for "firmwareExe" 5s after launch and re-execs it if it is gone. So this
# script keeps that name (it IS $0) and must stay in the FOREGROUND. The last
# statement is `wait "$SCANNER_PID"`.
#
# HelixScreen is the only UI. FlashForge's binary is not kept anywhere: the
# installer wipes the software dir before run.sh anyway, so the way back is
# flashing the stock FlashForge package, which still ships it.
# If this script is ever removed, app_startup.sh restores a firmwareExe from a
# version directory -- the stock recovery path still works.
#
# Two further properties we get for free:
# * app_startup.sh's own watchdog greps `ps` for "firmwareExe" 5s after
# launch. This script keeps that name (it IS $0) and stays in the
# FOREGROUND, so the stock watchdog supervises the mod correctly.
# * If this script is ever removed, app_startup.sh restores a firmwareExe
# from a version directory (the last `ls` entry, alphabetical -- and there is
# normally only one) -- the stock recovery path still works.
# WHAT IT DOES: start s6-svscan as its own child, initialise s6-rc, ask for one
# machine state, and block. There is no init.d directory any more and nothing
# runs programs in sequence; the boot order is the compiled s6-rc graph. This
# is also the only script that starts the supervision tree -- over ssh, the
# recovery is plain s6-rc.
MODDIR=/usr/data/anvil
@@ -29,27 +28,50 @@ mkdir -p /usr/data/logs
exec >>/usr/data/logs/anvil-boot.log 2>&1
echo "=== mod firmwareExe `date 2>/dev/null` ==="
# The library path and the interpreter, for this script and everything it
# starts. See anvil-env.sh: app_startup.sh exports these on the boot path, so
# inheriting them looks like it works right up until somebody re-runs a
# service by hand.
# The library path and the interpreter. app_startup.sh exports these on the
# boot path, so inheriting them looks like it works right up until somebody
# re-runs something by hand.
if [ -f $MODDIR/anvil-env.sh ]; then
. $MODDIR/anvil-env.sh
else
echo "!! no $MODDIR/anvil-env.sh -- the mod payload is not installed"
fi
S6BIN=$MODDIR/bin
# The s6 scandir: one live servicedir per service, laid down by s6-rc-init.
SCANDIR=$MODDIR/etc/s6
# The compiled database. `current` is a symlink bin/patch.sh points at the
# stamped directory it built, so this path never changes when the database does.
S6DB=$MODDIR/etc/s6-rc/compiled/current
# The scanner outlives the redirection above, so it gets a file of its own. A
# healthy s6-svscan says nothing; anything here is a fault.
S6LOG=/usr/data/logs/s6.log
# MEASURED: s6-svscan execs s6-supervise BY NAME off PATH, and s6-svc -w execs
# s6-svlisten the same way. Without $MODDIR/bin on PATH the scanner answers
# `s6-svscanctl -a` perfectly and then fails every spawn with ENOENT for ever,
# and s6-rc-init HANGS rather than failing. anvil-env.sh does this too; it is
# repeated because that file is allowed to be missing, and "carries on" must
# not mean "starts a tree that can never supervise anything".
case ":$PATH:" in
*":$S6BIN:"*) ;;
*) PATH="$S6BIN:$PATH"; export PATH ;;
esac
# Is the SCANNER alive? Asked by behaviour: `s6-svscanctl -a` is a rescan
# request travelling down a FIFO opened non-blocking, so nothing reading the
# other end means ENXIO and exit 100. A dead scanner leaves the FIFO on disk,
# which is a distinction `ps | grep` cannot make.
scanner_up() {
[ -x "$S6BIN/s6-svscanctl" ] || return 1
"$S6BIN/s6-svscanctl" -a "$SCANDIR" >/dev/null 2>&1
}
# ---- something on the panel, immediately ---------------------------------
# From here until HelixScreen paints, nothing owns the screen. That gap is
# the whole boot on EVERY start, not just the first: the migration below
# exits at its stamp and draws nothing. FlashForge's own splash cannot fill
# it -- /usr/prog/start.img exists, but the only two lines that would display
# it (both in app_startup.sh) are commented out, so nothing has shown it for
# a long time.
# FF_PYTHON and LD_LIBRARY_PATH come from anvil-env.sh, which is also what
# every service this script goes on to run will source. Exporting them here
# means they are inherited too, so the init.d scripts re-source a set that is
# already correct; anvil-env.sh is written to make that a no-op.
# Nothing owns the screen until HelixScreen paints, on every boot.
# FlashForge's own splash cannot fill it -- /usr/prog/start.img exists, but the
# only two lines that would display it, both in app_startup.sh, are commented
# out. FF_PYTHON and LD_LIBRARY_PATH come from anvil-env.sh.
splash() {
[ "${MOD_SPLASH:-1}" = 1 ] || return 0
[ -f $MODDIR/bin/ffscreen.py ] || return 0
@@ -57,110 +79,141 @@ splash() {
}
splash --title "REFORGE" --status "STARTING THE PRINTER" --progress 0.02
# ---- services ------------------------------------------------------------
# init.d-style scripts, run in filename order. One failing service must never
# stop the rest: Klipper has to come up even if the web stack is broken.
if [ -d $MODDIR/init.d ]; then
for service in $MODDIR/init.d/S*; do
[ -x "$service" ] || continue
echo "--- `basename $service` ---"
"$service" start || echo "`basename $service`: returned $?"
# ---- the supervision tree ------------------------------------------------
#
# THE SCANNER MUST BE THIS SHELL'S OWN CHILD: `wait` only works on a process
# the calling shell forked, and `wait "$SCANNER_PID"` is what holds the
# foreground. That is why the tree is started here and not by some script this
# one calls -- a child of a child cannot be waited on.
#
# NOT `exec s6-svscan`: the watchdog greps for the NAME, and busybox ash has no
# `exec -a`. The shell also has to run s6-rc-init after the scanner is up.
if scanner_up; then
# A scanner we did not fork cannot be waited on, so it goes and comes back
# under us -- the same "restart everything" the recovery below performs.
echo "s6: a scanner is already running -- taking it down so this boot owns it"
"$S6BIN/s6-svscanctl" -t "$SCANDIR" 2>/dev/null
n=0
while scanner_up && [ $n -lt 15 ]; do
sleep 1
n=$((n + 1))
done
fi
if [ ! -x "$S6BIN/s6-svscan" ]; then
echo "s6: !! no $S6BIN/s6-svscan -- this payload did not ship the supervisor"
echo "s6: !! this printer has no services; ssh is up via the stock S50dropbear"
# Exiting would have app_startup.sh re-exec us into the same wall in a loop.
while true; do sleep 3600; done
fi
# The scandir ships empty and is named in the install manifest, so
# run-append.sh's rmdir pass can legitimately remove it before the new tarball
# lands. One mkdir deletes a class of "the scanner did not start, log says
# ENOENT".
mkdir -p "$SCANDIR"
# exec, so SCANNER_PID is the scanner itself rather than a shell that started
# one -- otherwise `wait` would return on a wrapper that had already gone.
( exec "$S6BIN/s6-svscan" "$SCANDIR" >>"$S6LOG" 2>&1 ) &
SCANNER_PID=$!
# ---- is the supervisor still there? --------------------------------------
# S40s6 started s6-svscan a few lines above, backgrounded, and then returned
# -- it has to, because the scanner never returns and these scripts run in the
# foreground. Nothing supervises the supervisor: we are not PID 1, there is no
# inittab we can reach (/etc is inside the read-only squashfs a stock flash
# overwrites), and a `while true` respawn loop in this script would be one
# more hand-rolled supervisor, which is the exact thing s6 is here to delete.
#
# So this is the whole answer, and it is deliberately the same shape as the
# Klipper last-resort further down: check once, at the one moment something
# has already had a chance to go wrong, and say so in the boot log. What it
# actually catches is a scanner that exec'd and then died -- the
# -D_FILE_OFFSET_BITS=64 build trap in tools/supervisor/README.md, where
# readdir() returns EOVERFLOW and the scanner starts fine and then does
# nothing. S40s6 already waits and warns; this catches the case where it died
# in the seconds AFTER that, while wifi and nginx were starting.
#
# Deliberately NOT a loop and NOT a watchdog. If the scanner dies twice the
# printer has a real fault and a respawn would only hide it -- and note that a
# dead scanner leaves the s6-supervise processes it started running, still
# respawning their own services, so it is a printer that has stopped noticing
# changes to the scandir rather than a printer that has stopped printing.
# Today, with an empty scandir, it is not even that. See S40s6's header.
if [ "${MOD_S6:-1}" = 1 ] && [ -x $MODDIR/init.d/S40s6 ]; then
if ! $MODDIR/init.d/S40s6 status | grep -q scanning; then
echo "s6: the scanner is not answering -- one restart, then we leave it"
$MODDIR/init.d/S40s6 start
fi
# The wait is for a thing that takes no time -- a mkdir, a mkfifo and two
# opens, no hardware involved. What it buys is that a scanner which exec'd and
# then DIED is reported here: that is the -D_FILE_OFFSET_BITS=64 trap in
# tools/supervisor/README.md, where readdir() returns EOVERFLOW and the scanner
# starts fine and then does nothing. The 5s is only paid on a broken printer.
n=0
while ! scanner_up && [ $n -lt 5 ]; do
sleep 1
n=$((n + 1))
done
if ! scanner_up; then
echo "s6: !! the scanner is not answering after ${n}s -- see $S6LOG"
echo "s6: !! this printer has no services; ssh is up via the stock S50dropbear"
while true; do sleep 3600; done
fi
echo "s6: scanning $SCANDIR (pid $SCANNER_PID)"
# Once per boot: s6-rc-init creates one symlink per service in the scandir and
# fails outright with "File exists" if a name is taken -- which is why
# run-append.sh sweeps $MODDIR/etc/s6 on upgrade. /run is a fresh tmpfs on a
# normal boot, so the check below is for the printer that got here some other
# way. `s6-rc -a list` needs the live directory, so it answers the question by
# behaviour.
if "$S6BIN/s6-rc" -a list >/dev/null 2>&1; then
echo "s6-rc: already initialised -- leaving the live state alone"
elif [ ! -d "$S6DB" ]; then
echo "s6-rc: !! no compiled database at $S6DB -- this printer has no services"
while true; do sleep 3600; done
elif ! "$S6BIN/s6-rc-init" -c "$S6DB" "$SCANDIR"; then
echo "s6-rc: !! could not initialise -- this printer has no services"
while true; do sleep 3600; done
fi
# ---- wait for the printer, and migrate on the first boot -----------------
# Two jobs, and the panel narrates both (bin/ff-startup.py):
# ---- one transition, and it is the whole boot ----------------------------
#
# every boot wait for the toolhead boards to leave their bootloaders and
# for klipper and moonraker to be ready, naming whichever one
# is holding things up. Starting HelixScreen before that is
# what produces a UI reporting a disconnected printer with no
# clue as to which board is missing.
# first boot move this unit's factory calibration into printer.cfg. That
# runs HERE, ahead of the UI, because the save restarts klippy
# and doing that behind a HelixScreen already showing a
# printer is worse than making the splash wait.
# mcu-bringup -> klipper -> ff-startup -> ui
# ^
# moonraker
#
# MOD_IMPORT=0 keeps the wait and skips the migration.
if [ "${MOD_STARTUP:-1}" = 1 ] && [ -f $MODDIR/bin/ff-startup.py ]; then
STARTUP_ARGS="--timeout ${MOD_STARTUP_TIMEOUT:-240}"
STARTUP_ARGS="$STARTUP_ARGS --klipper-tries ${MOD_KLIPPER_TRIES:-6}"
[ "${MOD_IMPORT:-1}" = 1 ] || STARTUP_ARGS="$STARTUP_ARGS --no-import"
"$FF_PYTHON" $MODDIR/bin/ff-startup.py $STARTUP_ARGS \
|| echo "ff-startup: returned $? -- the next boot will try again"
# A transition carries on past a service it could not start, and only what
# DEPENDS on that service is skipped.
#
# THE BOOT SET is the ok-all bundle minus what anvil.conf switched off. The
# bundle is read out of the compiled database rather than written here, so
# there is one statement of what a default printer runs; the switches are read
# at runtime, because a database built on a build host cannot read a file the
# user edits over ssh. THIS IS THE ONLY PLACE THE MOD_* SERVICE GATES EXIST.
# `contents`, not `list`: `list` takes a class of service.
#
# mcu-bringup is deliberately not in the bundle -- it arrives as klipper's
# dependency, which is the point of having one.
BOOT_SET=""
for name in `"$S6BIN/s6-rc-db" -c "$S6DB" contents ok-all 2>/dev/null`; do
case "$name" in
nginx|moonraker) [ "${MOD_WEB:-1}" = 1 ] || continue ;;
camera) [ "${MOD_CAM:-1}" = 1 ] || continue ;;
wifi) [ "${MOD_WIFI:-1}" = 1 ] || continue ;;
ui) [ "${MOD_UI:-1}" = 1 ] || continue ;;
esac
BOOT_SET="$BOOT_SET $name"
done
if [ -z "$BOOT_SET" ]; then
# No database, or a renamed bundle. Asking for it whole beats a printer
# with no services and an honoured MOD_CAM.
echo "s6-rc: !! cannot read the ok-all bundle from $S6DB -- asking for it whole"
BOOT_SET=ok-all
fi
# Last resort. S70klipper stood aside for ff-startup, so if that program did
# not run at all -- a broken interpreter, a truncated file, a python
# traceback before it got as far as start.sh -- nothing has started klipper
# and the printer would come up with a screen and no motion or heaters. This
# is the one case where the shell has to take the job back.
if ! $MODDIR/init.d/S70klipper status | grep -q running; then
echo "klipper: nothing started it -- falling back to S70klipper"
$MODDIR/init.d/S70klipper force-start
# THIS BLOCKS FOR THE WHOLE BOOT, on purpose: moonraker and the camera are
# waited for until READY, and ff-startup is a oneshot that waits for the boards
# and for klippy to answer. ff-startup.py draws on /dev/fb0 throughout.
#
# `-u change` is ADDITIVE, not `-p` (prune) -- it brings these and their
# dependencies up and leaves everything else alone.
echo "s6-rc: bringing up:$BOOT_SET"
"$S6BIN/s6-rc" -u change $BOOT_SET \
|| echo "s6-rc: !! at least one service did not come up (see above)"
# The one thing a service that is not running cannot do: say so on the panel.
if [ "${MOD_UI:-1}" != 1 ]; then
splash --title "REFORGE" \
--status "NO SCREEN" \
--detail "MAINSAIL AND SSH ARE RUNNING. SET MOD_UI=1 IN ANVIL.CONF TO TURN THE SCREEN ON."
echo "ui: MOD_UI=0 -- headless on purpose"
fi
# ---- the UI, in the foreground so the stock watchdog can see us ----------
# S80ui already decided and wrote its verdict here: "helix" to start the UI,
# "none" to stay headless, which now means one thing: HelixScreen is not
# installed. MOD_UI used to be the other way to get here and is gone.
UI_CHOICE=`cat $MODDIR/.ui-choice 2>/dev/null || echo none`
echo "ui: choice is $UI_CHOICE"
if [ "$UI_CHOICE" = helix ]; then
splash --title "REFORGE" --status "STARTING THE SCREEN" --progress 0.9
HELIX_DATA_DIR=$MODDIR/helixscreen
HELIX_LOG_DEST=file
HELIX_LOG_FILE=/usr/data/logs/helixscreen.log
# This repo pins and patches the HelixScreen build it ships; an in-app
# update would replace it behind the package's back. HelixScreen documents
# this variable as the vendor opt-out: it suppresses the update check, the
# notify modal and the settings-panel update UI.
HELIX_DISABLE_AUTO_UPDATES=1
export HELIX_DATA_DIR HELIX_LOG_DEST HELIX_LOG_FILE HELIX_DISABLE_AUTO_UPDATES
if [ -x $MODDIR/helixscreen/bin/helix-launcher.sh ]; then
$MODDIR/helixscreen/bin/helix-launcher.sh
else
$MODDIR/helixscreen/bin/helix-screen
fi
fi
# If the UI exits -- or was never started at all -- we do NOT exit: dropping
# out of this script makes app_startup.sh think the UI died and re-exec us,
# which would restart every service. Hold the process instead; ssh and
# Mainsail remain available.
splash --title "REFORGE" \
--status "NO SCREEN" \
--detail "MAINSAIL AND SSH ARE RUNNING. INSTALL ANVIL-HELIXSCREEN TO TURN THE SCREEN ON."
echo "ui: nothing left in the foreground -- holding process so services stay up"
while true; do sleep 3600; done
# ---- the foreground, and the recovery path -------------------------------
#
# This is what satisfies app_startup.sh's watchdog: still here, under this
# name, blocked on something real.
#
# When the scanner dies, `wait` returns, this script exits, and app_startup.sh
# re-execs it -- bringing the whole tree back. It is one restart of EVERYTHING,
# not a targeted one: klippy goes down with it, mid-print. A scanner that dies
# twice has a real fault and this looks like a boot loop, which is what a boot
# loop is for.
#
# `wait "$SCANNER_PID"`, not a bare `wait`: services leave background children
# around and a bare `wait` would return on the wrong one.
echo "s6: holding the foreground on the scanner (pid $SCANNER_PID)"
wait "$SCANNER_PID"
echo "s6: !! the scanner exited -- leaving, so app_startup.sh restarts everything"
pkgs/anvil-core/payload/prog/klipperDaemon
Executable
+49
View File
@@ -0,0 +1,49 @@
#!/bin/sh
# /usr/prog/klipper/klipperDaemon -- REPLACED by the mod.
#
# Stock, this forks klippy with start-stop-daemon -b. klippy is an s6-rc longrun
# now, so anything that still calls this -- FlashForge's own code, the UI's
# restart-firmware path, a habit over ssh -- would put a second, unsupervised
# klippy next to the supervised one, fighting for /dev/ttyS4 and /tmp/uds. So
# `start` does nothing, and exits 0 so a caller that treats failure as fatal is
# not broken by it.
#
# KLIPPER_NICENESS is FlashForge's tuning and this file is still where it lives:
# etc/s6-rc/source/klipper/run reads the line below, and payload/run-append.sh
# carries the value forward off the file it replaces, so a printer that has
# never seen the stock number does not get a guessed one.
KLIPPER_NICENESS=0
MODDIR=/usr/data/anvil
S6RC=$MODDIR/bin/s6-rc
case "$1" in
start)
echo "klipperDaemon: klipper is supervised -- use '$S6RC -u change klipper'"
exit 0
;;
stop)
# Redirected rather than reimplemented. Stock semantics would signal a
# pidfile nothing writes any more, so `stop` would silently do nothing and
# s6 would keep klippy up regardless. This also stops what depends on
# klipper: ff-startup, and so the UI.
[ -x "$S6RC" ] || { echo "klipperDaemon: no $S6RC"; exit 0; }
exec "$S6RC" -d change klipper
;;
restart)
# `s6-svc -t` and not two s6-rc transitions: it takes no lock on the live
# directory, so this is safe to call from inside one, and it leaves the
# WANTED state alone -- s6-rc goes on believing klipper is up, which stays
# true. -wr returns when the supervisor has started it again.
[ -x "$MODDIR/bin/s6-svc" ] || { echo "klipperDaemon: no $MODDIR/bin/s6-svc"; exit 0; }
exec "$MODDIR/bin/s6-svc" -wr -T 60000 -t "$MODDIR/etc/s6/klipper"
;;
status)
[ -x "$MODDIR/bin/s6-svstat" ] || { echo "klipperDaemon: no $MODDIR/bin/s6-svstat"; exit 0; }
exec "$MODDIR/bin/s6-svstat" "$MODDIR/etc/s6/klipper"
;;
*)
echo "usage: $0 {start|stop|restart|status}"
exit 1
;;
esac
pkgs/anvil-core/payload/prog/start.sh
+23 -69
View File
@@ -1,86 +1,40 @@
#!/bin/sh
# /usr/prog/klipper/start.sh
#
# Kept as close to stock as possible. The web stack is NOT started here even
# though stock has the (commented-out) lines for it -- that is the job of
# /usr/data/anvil/init.d/S60nginx and /usr/data/anvil/init.d/S62moonraker, so
# there is exactly one place that starts each of them and one place to restart
# them from over ssh. They were a single S60web until the split; nginx and
# moonraker fail independently and are now stopped and started independently.
# Klippy is an s6-rc longrun, so this asks for the service and the graph brings
# the MCU bring-up along. It is kept rather than deleted because it is a path
# other things take -- FlashForge's own tooling knows this filename, and it is
# in every set of notes anybody has written about this printer.
#
# Three changes from stock:
# * klipper_pri.sh is actually invoked. FlashForge ships that script but
# never calls it, so klippy runs at normal priority.
# * ff_mcu_bringup.py hands the heat, eboard and level boards over from
# their bootloaders. Stock never needed it here because firmwareExe did
# all three itself; replacing firmwareExe left two of them stranded, and
# the third was covered by checkEboard.
# * checkEboard is deliberately not called. It is one function, hard wired
# to /dev/ttyS5, and an older build of the routine ff_mcu_bringup.py
# reimplements -- one that treats ANY byte from that port as a bootloader
# banner and so sends 'A' at an eboard already running Klipper. The binary
# is still on the firmware partition; nothing runs it.
# * the /tmp/uds idempotence guard below, so S70klipper's retry loop cannot
# start a second klippy against the same MCU.
cmd_mcu write_firmware /usr/prog/libmcu-bare.bin
cmd_mcu bootup
sleep 2
# PATH, LD_LIBRARY_PATH and FF_PYTHON, from the one file that defines them.
# This script is on the firmware partition and the env file is on the data
# partition, which is the right way round: the list describes /usr/prog and
# the mod owns it. See anvil-env.sh.
# There is deliberately no direct launch left here. One that still ran
# klipperDaemon would be a second way to start klippy: unsupervised, invisible
# to s6-rc, and fighting the supervised copy for /dev/ttyS4.
#
# bin/patch.sh stages this as a software component and FlashForge's own run.sh
# copies it onto /usr/prog/klipper, which is why it lives on the firmware
# partition and why a stock flash replaces it.
MODDIR=/usr/data/anvil
# This script is on the firmware partition and the env file is on the data
# partition, which is the right way round: the list describes /usr/prog and the
# mod owns it.
if [ -f $MODDIR/anvil-env.sh ]; then
. $MODDIR/anvil-env.sh
else
echo "start.sh: !! no $MODDIR/anvil-env.sh -- klippy will not find its libraries"
fi
# Idempotence guard: init.d/S70klipper runs this script, and so does its own
# retry loop. Running it twice would start a second klippy against the same
# MCU.
# /tmp/uds is klippy's API socket. The transition below would also be a no-op
# on a running klippy, but it would not say so.
if [ -S /tmp/uds ]; then
echo "start.sh: klippy already running, nothing to do"
exit 0
fi
# MCU bring-up. Each of these boards answers Klipper only after its
# bootloader is told to start the application:
#
# /dev/ttyS2 mcu cmd_mcu bootup, above
# /dev/ttyS4 eheaterboard ff_mcu_bringup.py <- was nobody's job
# /dev/ttyS5 eboard ff_mcu_bringup.py <- was checkEboard
# /dev/ttyS7 levelboard ff_mcu_bringup.py <- was nobody's job
#
# This runs on every klippy start, including the restarts S70klipper issues
# when a board missed its window.
#
# $FF_PYTHON is the absolute path anvil-env.sh sets, with no `python3`
# fallback: the base rootfs ships no other interpreter, so a fallback would
# resolve either to the same binary or to something untested. LD_LIBRARY_PATH
# matters too -- this interpreter does not start without it.
#
# Test -f, not -x. The script is handed to the interpreter by path, so its
# execute bit is irrelevant -- and it is not always set: the payload carries
# mode 644 in git, and the +x only happens if the file arrived through
# patch.sh or run-append.sh. A hand-copied file is perfectly runnable.
#
# FF_SKIP_MCU_BRINGUP=1 says the caller has already done it. bin/ff-startup.py
# sets that: it owns the boot sequence, so it does the bring-up itself, in
# process, and then runs this script to launch klippy. Nothing else sets it,
# so a start.sh run by hand over ssh -- or by S70klipper's fallback -- still
# does the full job.
if [ "${FF_SKIP_MCU_BRINGUP:-0}" = 1 ]; then
echo "start.sh: MCU bring-up already done by the caller"
elif [ -f /usr/data/anvil/bin/ff_mcu_bringup.py ]; then
"$FF_PYTHON" /usr/data/anvil/bin/ff_mcu_bringup.py \
|| echo "start.sh: MCU bring-up reported a problem ($?)"
else
echo "start.sh: ff_mcu_bringup.py missing -- the toolhead boards are not brought up"
if [ ! -x $MODDIR/bin/s6-rc ]; then
echo "start.sh: !! no $MODDIR/bin/s6-rc -- the mod payload is not installed"
echo "start.sh: !! nothing here can start Klipper; flash the stock package to get it back"
exit 1
fi
/usr/prog/klipper/klipperDaemon start
# Real-time priority for klippy (stock ships this but never runs it).
sh /usr/prog/klipper/klipper_pri.sh >/dev/null 2>&1 &
# The transition pulls mcu-bringup in first, because klipper depends on it.
exec $MODDIR/bin/s6-rc -u change klipper
pkgs/anvil-core/seed/anvil.conf.in
+18 -16
View File
@@ -9,11 +9,17 @@
# The web stack, the screen and wifi now start because they are installed, and
# not starting them is `opkg remove`, or a stock FlashForge package.
#
# The camera is the one worth spelling out, because it looks like a switch is
# missing: mjpg-streamer on :8080, proxied by nginx at /webcam/, from
# FlashForge's own binary on the firmware partition. A printer with no camera
# needs no setting -- init.d/S65camera finds no device, says so in the boot log
# and stops. These tune it when there IS one; defaults shown:
# The camera is the one worth spelling out. mjpg-streamer on :8080, proxied by
# nginx at /webcam/, from FlashForge's own binary on the firmware partition. It
# is an s6-rc longrun that reports ready only once it has a device, so a
# printer without one waits out its timeout-up on every boot.
#
# firmwareExe still reads MOD_CAM when it expands the ok-all bundle, defaulting
# to 1. It is not shipped here because none of these switches is: a
# half-installed mod is not a state to offer. Add MOD_CAM=0 by hand on a
# machine that has no camera and never will.
#
# These tune it when there IS one; defaults shown:
#MOD_CAM_RES=1280x720
#MOD_CAM_FPS=30
#MOD_CAM_PORT=8080
@@ -32,10 +38,13 @@ MOD_SPLASH=1
# boards over again on each retry, which is what gives one that missed its
# window another chance. MOD_KLIPPER_TRIES (below) bounds the retries.
#
# 0 = hand klipper's start back to init.d/S70klipper, which does it the old
# way (background, printer.log tailing, no screen) and starts the UI at once.
# That is also what happens automatically if this script or the interpreter
# is missing -- klipper starting must never depend on python.
# 0 = do not wait for the printer: klipper still starts (it is a service in
# its own right and has never depended on this program), the MCU bring-up still
# happens (it is klipper's dependency), and the screen comes up at once in
# front of a machine that may still be connecting. What is skipped is the wait,
# the retries when a board did not answer, the panel narration and the
# first-boot import. That is also what happens automatically if the script or
# the interpreter is missing -- klipper starting must never depend on python.
#MOD_STARTUP_TIMEOUT=240
MOD_STARTUP=1
@@ -46,13 +55,6 @@ MOD_STARTUP=1
# wait above still happens.
MOD_IMPORT=1
# Process supervision: s6-svscan, started first by init.d/S40s6 and watching
# $MODDIR/etc/s6 for service directories: nginx, the camera and moonraker are
# on it now, so turning it off leaves them running unsupervised -- see the
# init.d fallback for each. It is here so that a printer where the supervisor
# itself is suspect has a switch. 0 = no scanner.
MOD_S6=1
# ---------------------------------------------------------------------------
# Priorities. Nice values (0..19) for the services that share this SoC's two
# cores with klippy. Higher = yields more.
qa/lib/replica.py
-36
View File
@@ -146,39 +146,6 @@ class File:
return "File(%s)" % self.path
class Service:
"""One of anvil-core's init.d/S*, driven through its own dispatcher.
Every method returns a Result. Nothing here decides anything by reading
the script: it runs the service and reports what came back, which is the
property case-services.sh's header insists on and is worth keeping.
"""
def __init__(self, printer, name, initd):
self._p = printer
self.name = name
self.path = "%s/%s" % (initd, name)
def __call__(self, verb):
return self._p.sh("%s %s 2>&1" % (self.path, verb))
def status(self):
return self("status")
def start(self):
return self("start")
def stop(self):
return self("stop")
@property
def installed(self):
return self._p.file(self.path).executable
def __repr__(self):
return "Service(%s)" % self.name
class Process:
"""A row of the process table, read from /proc rather than from `ps`.
@@ -270,9 +237,6 @@ class Printer:
def file(self, path):
return File(self, path)
def svc(self, name, initd="/usr/data/anvil/init.d"):
return Service(self, name, initd)
def ps(self):
"""The process table, from /proc/*/cmdline.
qa/replica/test_s6rc.py
+202
View File
@@ -0,0 +1,202 @@
"""The boot, on a machine the real installer produced.
qa/static/test_s6rc_source.py proves the database SAYS the right thing. This
proves the printer DOES it: the scanner comes up, s6-rc-init lays the live
servicedirs down, one transition brings the boot set up, and s6 puts a killed
daemon back.
NOTHING HERE HAS RUN YET. There is no stock FlashForge package on the machine
this was written on, so `make build` is impossible and the replica image holds
a pre-s6-rc payload. The `box` fixture below fails with one clear message
rather than letting forty assertions fail for the same reason.
"""
import pytest
from lib import replica
pytestmark = pytest.mark.replica
MODDIR = "/usr/data/anvil"
SCANDIR = MODDIR + "/etc/s6"
LIVE = "/run/s6-rc"
DB = MODDIR + "/etc/s6-rc/compiled/current"
BOOT_SET = {"wifi", "nginx", "moonraker", "camera", "klipper", "ff-startup",
"ui"}
@pytest.fixture(scope="module")
def box(printer):
"""The machine, with the package's own s6-rc actually present.
Checked once, here, because every test below asks a question that has no
meaning on a payload built before phase 8 -- and forty identical failures
hide which one thing is wrong.
"""
if not printer.file(MODDIR + "/bin/s6-rc-init").executable:
pytest.fail(
"this package has no s6-rc: %s/bin/s6-rc-init is missing, so the "
"machine under test predates the compiled boot. Build a package "
"from this tree first -- `make build` -- and the replica lane will "
"pick it up (qa/lib/replica.py takes the newest work/out/*.tgz)."
% MODDIR)
return printer
@pytest.fixture(scope="module")
def booted(box):
"""The real boot path, run once, bounded.
firmwareExe IS the boot -- scanner, s6-rc-init, one transition -- so this
runs it rather than reimplementing the three steps, which is the mistake
case-install.sh's header warns about ("an earlier version of this file
replayed app_startup.sh by hand"). It never returns (it holds the
foreground on `wait`), so it goes to the background and we poll for the
tree instead.
"""
box.sh("setsid /usr/prog/PROGRAM/software/firmwareExe >/dev/null 2>&1 &")
for _ in range(60):
if "ok-all" in box.sh("%s/bin/s6-rc -l %s -a list 2>/dev/null"
% (MODDIR, LIVE)).text:
return box
box.sh("sleep 2")
pytest.fail(
"the tree never came up within 120s. Boot log:\n%s"
% box.file("/usr/data/logs/anvil-boot.log").text[-2000:])
def _up(box):
return set(box.sh("%s/bin/s6-rc -l %s -a list" % (MODDIR, LIVE)).out.split())
# ------------------------------------------------------- the tree came up
def test_the_scanner_is_answering(booted):
"""Asked of the scanner over its control FIFO, not of the process table:
a dead scanner leaves the FIFO on disk, so a name match cannot tell the
difference."""
got = booted.sh("%s/bin/s6-svscanctl -a %s" % (MODDIR, SCANDIR))
assert got.ok, "the scanner is not listening on %s: %s" % (SCANDIR, got.text)
def test_the_boot_set_is_up(booted):
assert BOOT_SET <= _up(booted), (
"missing from the transition: %s" % (BOOT_SET - _up(booted)))
def test_the_scandir_was_populated_by_s6_rc_init(booted):
"""It ships EMPTY -- bin/patch.sh stopped copying servicedirs in when the
database took over -- so anything here arrived at boot."""
names = booted.sh("ls -1 %s" % SCANDIR).out.split()
assert "nginx" in names, "s6-rc-init laid nothing down: %s" % names
def test_every_live_run_is_executable(booted):
"""s6 reports a non-executable `run` in its own log and nowhere else, so
the service simply never comes up and nothing says why."""
bad = [n for n in booted.sh("ls -1 %s" % SCANDIR).out.split()
if booted.file("%s/%s/run" % (SCANDIR, n)).exists
and not booted.file("%s/%s/run" % (SCANDIR, n)).executable]
assert not bad, "not executable: %s" % ", ".join(bad)
def test_the_supervisor_is_the_one_we_shipped(box):
"""A cross-built MIPS binary, not a shell script standing in for one.
`head -c 4` rather than `od -An -c`: the printer's busybox od has no -A."""
scanner = box.file(MODDIR + "/bin/s6-svscan")
assert scanner.executable, "no s6-svscan in the installed package"
assert "ELF" in box.sh("head -c 4 %s" % scanner.path).out
def test_execline_shipped_too(box):
"""Every oneshot's `up` runs through s6rc-oneshot-runner, whose `run` is
execline. Without execlineb the wifi oneshot fails at exec and nothing
else does, which reads as a wifi bug."""
assert box.file(MODDIR + "/bin/execlineb").executable
def test_the_scanner_logged_nothing(booted):
"""A healthy s6-svscan says nothing. The one that matters is the
-D_FILE_OFFSET_BITS=64 trap: "unable to readdir .: Value too large for
defined data type" -- a scanner that started and then went blind."""
log = booted.file("/usr/data/logs/s6.log")
assert log.empty, "s6-svscan wrote to its log: %s" % log.lines[:3]
# --------------------------------------------------- s6 does its one job
def test_a_killed_daemon_comes_back(booted):
"""The whole reason for shipping 14 binaries, and neither suite has ever
asserted it. SIGKILL, because that is what an OOM does and it is the case
a `stop` verb cannot cover."""
before = booted.pgrep("nginx")
assert before, "nginx was not running, so this proves nothing"
for proc in before:
booted.sh("kill -9 %s 2>/dev/null" % proc.pid)
booted.sh("sleep 5")
after = booted.pgrep("nginx")
assert after, "s6 did not restart nginx after kill -9"
assert {p.pid for p in after} != {p.pid for p in before}, (
"the same pids are still there -- nothing was actually killed")
# ------------------------------------------------------- the negative controls
def test_a_database_moved_aside_is_reported(box):
"""The boot must say so rather than coming up silently wrong. Run in its
own container: it breaks the machine it runs on."""
box.sh("mv %s %s.moved" % (DB, DB))
try:
got = box.sh("%s/bin/s6-rc-init -c %s -l /run/probe %s 2>&1"
% (MODDIR, DB, SCANDIR))
assert not got.ok, "s6-rc-init accepted a database that is not there"
finally:
box.sh("mv %s.moved %s" % (DB, DB))
def test_a_populated_scandir_is_swept_not_collided_with(box):
"""MEASURED: s6-rc-init creates one symlink per service in the scandir and
dies with "unable to supervise service directories: File exists" if the
name is taken. An upgrade from a phase-4/5 payload has nginx, moonraker
and camera sitting there, so run-append.sh sweeps -- this is that sweep,
asked of the machine."""
box.sh("mkdir -p %s/nginx && rm -rf /run/probe2" % SCANDIR)
got = box.sh("%s/bin/s6-rc-init -c %s -l /run/probe2 %s 2>&1"
% (MODDIR, DB, SCANDIR))
assert "File exists" in got.text or got.ok, (
"expected either a clean init or the documented collision, got: %s"
% got.text)
# ------------------------------------------------------------ the watchdog
def test_firmware_exe_holds_the_foreground(booted):
"""app_startup.sh greps `ps` for "firmwareExe" 5s after launch, so the
script must keep its name and not exit. It holds on `wait $SCANNER_PID`;
an `exec` of the scanner would have renamed the process and busybox ash
has no `exec -a`."""
booted.sh("sleep 5")
assert booted.pgrep("firmwareExe"), (
"nothing named firmwareExe is running -- app_startup.sh's watchdog "
"would re-exec us and restart every service")
# ------------------------------------------------------- the open question
def test_the_ui_runs_under_s6_supervise(booted):
"""THE BLOCKING UNKNOWN from docs/notes/80-s6-migration.md.
Until phase 8 the UI ran in firmwareExe's foreground and inherited its
descriptors and session. A supervised process gets neither. If this fails
because helix-screen wants a controlling terminal, the answer is s6's own
tty handling or a wrapper -- NOT a return to the foreground, which no
longer exists.
"""
if not booted.file(MODDIR + "/helixscreen/bin/helix-screen").exists:
pytest.fail(
"no HelixScreen in this package, so the one question this phase "
"left open cannot be answered here -- build a package with "
"BUILD_HELIX=1")
stat = booted.sh("%s/bin/s6-svstat %s/ui" % (MODDIR, SCANDIR))
assert "up" in stat.text, (
"the UI is not up under s6-supervise: %s" % stat.text)
qa/replica/test_services.py
-417
View File
@@ -1,417 +0,0 @@
"""Do our init.d services all behave like the same kind of thing?
Ported from test/integration/printer/case-services.sh. Same machine, same
assertions, same reasoning -- the header below is the original's, because the
reasons have not changed. What changes is that its ~30 checks were one exit
code and are now 43 named tests.
WHY THIS EXISTS -- the seven services used to be seven one-off scripts. Each
had grown its own way of asking "is it running" (a pidfile and kill -0, a
`ps | grep`, a `pgrep -f` with a ps fallback, a socket test), its own spelling
of the log lines, and its own `case "$1"` block with subtly different verbs and
exit codes. Two real bugs lived in that drift: a stop that returned before the
process was gone, so the restart that followed saw it still in the process
table and declined to start anything; and a status that read a socket klippy
does not unlink on exit, so a dead klippy reported itself running and `restart`
did nothing at all.
anvil-service.sh exists to make those one decision each instead of seven. This
file is what stops them drifting apart again. It is deliberately NOT a grep
over the scripts -- that would pass on a service that cannot start anything and
fail on a rename. It RUNS every service, on the printer's own busybox, and
asserts on what came back.
What it does NOT do is start real daemons: that is case-moonraker.sh's job for
the web stack, and wifi/camera/klipper need hardware the replica has none of.
What is checked here is the contract every service shares -- it loads its
library, it answers `status` without blowing up, it names itself in what it
prints, and it rejects a verb it does not know with a usage line and exit 1.
WHAT THE PORT CHANGED
Two things.
THE MACHINE. The original installs the payload itself, with a `cp -f` of
payload/*.sh and payload/init.d/S* into $MODDIR. Here the `printer` fixture is
a machine that had the real package installed by its own app_startup.sh, off a
real FAT stick. So the install is under test rather than re-implemented, and
the services being run are the ones the installer placed -- including the
cross-built s6, which is a build output that no amount of copying from
payload/ can produce. The stand-in scanner the original needs, and the
skip-or-not branching around it, are gone with the copying. See
qa/replica/conftest.py.
THE STOPWATCH. The original bounds the init sequence with a background
subshell, a marker file and a `while [ ! -f ... ]` counter, because a test that
hangs is a test that reports nothing. That is a real constraint and the
implementation was sound, but a timeout is something the host can simply have:
Printer.sh takes one, and a command that does not come back is a Result whose
`timed_out` is True. The negative control is kept -- see TestInitSequence --
because a stopwatch that cannot catch a hang would let the real check pass no
matter what S40s6 did, and that is as true of this one as of the original.
"""
import pytest
pytestmark = pytest.mark.replica
MODDIR = "/usr/data/anvil"
SERVICES = ["S40s6", "S50wifi", "S60nginx", "S62moonraker",
"S65camera", "S70klipper", "S80ui"]
# What each service must call itself. Every line a service prints is
# "name: something": it is what makes a boot log readable when seven services
# are interleaving their output -- and the services background their slow work
# precisely so that they DO interleave.
PREFIX = {"S40s6": "s6", "S50wifi": "wifi", "S60nginx": "nginx",
"S62moonraker": "moonraker", "S65camera": "camera",
"S70klipper": "klipper", "S80ui": "ui"}
# --------------------------------------------------------------- the fixtures
#
# There is no payload-install fixture and no supervisor-staging fixture. The
# `printer` fixture (qa/replica/conftest.py) is already a machine that had the
# real package installed by its own app_startup.sh, so the payload is where the
# installer put it and s6 is the cross-built one the package ships. Nothing
# here stands in a scanner: hand-placing payload/ cannot deliver a binary that
# only exists as a build output.
@pytest.fixture(scope="module")
def box(printer):
"""The installed machine. Named for brevity; it is just `printer`."""
return printer
# ------------------------------------------------------------- the foundation
def test_payload_ships_anvil_service_sh(box):
"""The library is sourced, never executed, so it is packaged with
a plain cp and no chmod +x. It once did not stage it at all, which would
have made every service abort at boot -- hence checking for it by itself,
before anything tries to use it."""
assert box.file("%s/anvil-service.sh" % MODDIR).exists, (
"the payload ships no anvil-service.sh -- every service aborts")
# ----------------------------------------------------------- the shared contract
@pytest.mark.parametrize("name", SERVICES)
class TestServiceContract:
"""The four properties every service shares, asked of each of the seven.
Parametrized over the class so a failure names both the service and the
property: `test_status_answers[S62moonraker]` is a complete bug report in
its own identifier, which is the thing 360 lines reporting one exit code
could never do.
"""
def test_is_installed_and_executable(self, box, name):
assert box.svc(name, MODDIR + "/init.d").installed, (
"%s is missing or not executable" % name)
def test_status_answers(self, box, name):
"""A service must always answer. An empty status is the failure mode
that matters: it means the dispatcher fell through, and a boot script
that prints nothing is one nobody can debug from a log."""
got = box.svc(name, MODDIR + "/init.d").status()
assert "no /usr/data/anvil/anvil-service.sh" not in got.text, (
"%s cannot find its library at runtime" % name)
assert got.text.strip(), (
"%s status printed nothing (rc=%d) -- a service must always answer"
% (name, got.code))
def test_names_itself(self, box, name):
got = box.svc(name, MODDIR + "/init.d").status()
assert got.first_line.startswith(PREFIX[name]), (
"%s output is unprefixed: %r" % (name, got.first_line))
def test_unknown_verb_is_usage_and_exit_1(self, box, name):
"""One dispatcher means one answer to a verb nobody implements.
Getting this wrong is how a typo in a boot script becomes a silent
no-op."""
got = box.svc(name, MODDIR + "/init.d")("bogusverb")
assert got.code == 1, (
"%s: unknown verb gave rc=%d, output: %s"
% (name, got.code, got.text))
assert "usage:" in got.text, (
"%s: unknown verb printed no usage line: %s" % (name, got.text))
def test_klipper_advertises_force_start(box):
"""S70klipper is the one service with a verb of its own.
force-start exists because start() normally stands aside for
bin/ff-startup.py, and the firmwareExe wrapper needs a way to say "no,
actually start it" when that program returned and left no klippy behind.
If the shared dispatcher ever swallows it, the printer loses its
last-resort path to a running Klipper.
"""
got = box.svc("S70klipper", MODDIR + "/init.d")("bogusverb")
assert "force-start" in got.text, (
"S70klipper lost force-start from its usage line: %s" % got.text)
# ------------------------------------------------- the boot it must not hang
def _run_init_sequence(box, directory, limit):
"""Run a directory of S* scripts the way firmwareExe runs them.
One at a time, in the foreground, in filename order -- and bounded, because
a test that hangs is a test that reports nothing.
The bound lives on Printer.sh (a host-side timeout on the docker exec)
rather than in a background subshell with a marker file. Same guarantee,
and the verdict arrives as `result.timed_out` instead of as the absence of
a file.
"""
return box.sh(
'for service in %s/S*; do\n'
' [ -x "$service" ] || continue\n'
' echo "--- `basename $service` ---"\n'
' "$service" start || echo "`basename $service`: returned $?"\n'
'done\n' % directory,
timeout=limit)
@pytest.fixture(scope="module")
def booted(box):
"""The init sequence, run once, with its result kept.
A fixture rather than a step inside the first test, so that every
assertion about the state afterwards DEPENDS on it having happened rather
than on having been written below it in the file. Run a post-boot test on
its own -- `pytest -k test_scandir_exists` -- and the sequence still runs
first. Without this the test would pass on a machine where nothing had
ever started, which is the vacuous green this suite exists to refuse.
"""
return _run_init_sequence(box, MODDIR + "/init.d", 90)
class TestInitSequence:
"""S40s6 starts s6-svscan, and s6-svscan NEVER RETURNS -- that is its job.
firmwareExe runs $MODDIR/init.d/S* one at a time IN THE FOREGROUND, so a
start() that runs the scanner inline does not slow the boot down, it ends
it: no wifi, no ssh, no Mainsail, no Klipper, no screen, for ever, on every
boot, with no way in to find out why.
"""
def test_negative_control_a_foreground_scanner_hangs(self, box):
"""THE NEGATIVE CONTROL, and it runs first on purpose.
A stopwatch that cannot catch a hang would let the real check below
pass no matter what S40s6 did. So here is the naive implementation
somebody would write -- the scanner in the foreground, which is what
you get by deleting the svc_detach from S40s6 -- run through the SAME
harness. It has to time out.
Its own scandir, because the real s6 takes a lock on the one it is
scanning and a second instance EXITS rather than hangs, which would
make this control pass for entirely the wrong reason.
"""
box.sh("mkdir -p /tmp/naive-init.d %s/etc/s6-negctl" % MODDIR)
box.write("/tmp/naive-init.d/S41hang",
"#!/bin/sh\nexec %s/bin/s6-svscan %s/etc/s6-negctl\n"
% (MODDIR, MODDIR), mode="+x")
try:
got = _run_init_sequence(box, "/tmp/naive-init.d", 15)
assert got.timed_out, (
"an init script that runs the scanner inline RETURNED -- "
"the stopwatch proves nothing, so the real check below is "
"meaningless")
finally:
# It is still sitting there holding the foreground; take it down
# before the real sequence runs, or the process-table checks later
# count it.
box.sh("%s/bin/s6-svscanctl -t %s/etc/s6-negctl 2>/dev/null; "
"sleep 2" % (MODDIR, MODDIR))
for proc in box.pgrep("s6-negctl"):
box.sh("kill -9 %s 2>/dev/null" % proc.pid)
box.sh("rm -rf /tmp/naive-init.d %s/etc/s6-negctl" % MODDIR)
def test_the_real_sequence_returns(self, booted):
"""Every service the payload ships, run the way firmwareExe runs them.
90 seconds is generous -- the whole point of svc_detach is that these
return in well under a second each -- and generous is right for a
bound whose only job is to turn a hang into a failure.
"""
assert not booted.timed_out, (
"the init sequence did NOT return -- the boot would never reach "
"klipper or the UI\n%s" % booted.text)
def test_scanner_is_running_afterwards(self, box, booted):
"""Asked of the scanner itself, not of `ps`.
svc_s6_running pings the control FIFO, which a dead scanner leaves
behind on disk, so this distinguishes "running" from "the socket is
still lying there" the way a name match cannot.
"""
got = box.svc("S40s6", MODDIR + "/init.d").status()
assert "scanning %s/etc/s6" % MODDIR in got.text, (
"the scanner is not running after the init sequence: %s"
% got.text)
def test_scandir_exists(self, box, booted):
"""It ships empty from bin/patch.sh and is mkdir -p'd again at
runtime, so what is checked here is the runtime mkdir."""
assert box.file("%s/etc/s6" % MODDIR).is_dir
def test_scanner_survives(self, box, booted):
"""AN EMPTY SCANDIR IS NOT AN ERROR.
The scanner is alive and supervising possibly nothing, and it has to
be content with that -- if it exited, or complained, every boot would
carry a scary log line and the first service to move in would be
debugged against a scanner that was already unhappy.
"""
box.sh("sleep 3")
got = box.svc("S40s6", MODDIR + "/init.d").status()
assert "scanning" in got.text, "the scanner exited"
def test_the_supervisor_is_the_one_we_shipped(self, box):
"""The scanner under test is the package's own cross-built s6.
Cheap, and it is what makes the next test mean anything. When this
file hand-placed payload/ it could not deliver s6 at all -- s6 is a
build output, not a repo file -- so it carried a stand-in scanner and
gated the log check on whether the stand-in was in use. Installing the
real package removed both. This assertion is what stops that quietly
coming back: a machine whose s6-svscan is a shell script is one where
everything below is testing a mock again.
"""
scanner = box.file("%s/bin/s6-svscan" % MODDIR)
assert scanner.executable, (
"no s6-svscan in the installed package -- anvil-s6 ships it "
"from work/.s6, so this package was built without it")
# A cross-built MIPS binary, not a shell script. The first four bytes
# of an ELF are \x7fELF; Printer.sh decodes with errors="replace", so
# the \x7f arrives as a replacement character and "ELF" survives
# intact. Deliberately NOT `od -An -c`: the printer's busybox od has
# no -A ("invalid option -- 'A'"), which is exactly the class of
# difference this whole replica exists to surface -- the host's
# coreutils and the printer's busybox are not the same tools.
magic = box.sh("head -c 4 %s" % scanner.path)
assert "ELF" in magic.out, (
"%s is not an ELF binary -- it looks like a script, which means "
"something is standing in for the supervisor: %r"
% (scanner.path, magic.out[:40]))
def test_scanner_logged_nothing(self, box, booted):
"""A healthy s6-svscan says nothing at all.
Anything in its log is a fault, and the one that matters is the
-D_FILE_OFFSET_BITS=64 trap from tools/supervisor/README.md: "unable
to readdir .: Value too large for defined data type", which is a
scanner that started and then went blind.
"""
log = box.file("/usr/data/logs/s6.log")
assert log.empty, "s6-svscan wrote to its log: %s" % log.lines[:3]
# ------------------------------------------------ the service directories
def _service_dirs(box):
got = box.sh("ls -1 %s/etc/s6 2>/dev/null" % MODDIR)
return [n for n in got.out.split()
if box.file("%s/etc/s6/%s" % (MODDIR, n)).is_dir]
class TestServiceDirs:
"""Every service directory the payload ships has to be startable BY s6
and has to start DOWN."""
def test_some_were_staged(self, box, booted):
assert _service_dirs(box), (
"no service directories in %s/etc/s6 -- nothing was staged"
% MODDIR)
def test_each_ships_an_executable_run(self, box, booted):
"""The executable bit is the one that goes wrong quietly: s6 reports a
non-executable `run` in its own log and nowhere else, so the service
simply never comes up and nothing says why."""
missing = [n for n in _service_dirs(box)
if not box.file("%s/etc/s6/%s/run" % (MODDIR, n)).executable]
assert not missing, (
"no executable run (s6 would report that only in its own log): %s"
% ", ".join(missing))
def test_each_ships_down(self, box, booted):
"""`down` is what keeps the gate in anvil.conf meaningful -- without
it the scanner starts the service the instant it appears, before any
script has read MOD_WEB or MOD_CAM, so "disabled" would mean "runs for
a moment on every boot"."""
missing = [n for n in _service_dirs(box)
if not box.file("%s/etc/s6/%s/down" % (MODDIR, n)).exists]
assert not missing, (
"no 'down' file, so these would start regardless of their MOD_* "
"gate: %s" % ", ".join(missing))
# ------------------------------------------------------- the supervisor's life
@pytest.fixture(scope="module")
def stopped(box, booted):
"""A scanner that WAS running and has been stopped.
The precondition is asserted, not assumed, and that is the whole reason
this is a fixture. "stop left no s6 processes behind" and "the pidfile is
gone" are both trivially true of a machine where nothing ever started, so
written as bare tests in definition order they would pass for the wrong
reason the moment anyone ran one on its own with -k. Here the fixture
fails loudly instead: there was nothing to stop, so the questions below
have no meaning.
"""
before = box.svc("S40s6", MODDIR + "/init.d").status()
assert "scanning" in before.text, (
"nothing was running to stop, so the assertions that depend on this "
"would pass vacuously: %s" % before.text)
box.svc("S40s6", MODDIR + "/init.d").stop()
box.sh("sleep 2")
return box
class TestSupervisorLifecycle:
"""Four questions about one lifecycle. `stopped` is what orders them --
the second-start test runs against a live scanner, and the three after it
depend on a stop that provably had something to stop."""
def test_a_second_start_is_a_no_op(self, box, booted):
"""What a hand-run `S40s6 start` over ssh does, and what firmwareExe's
last-resort re-check does if it misreads the status.
It must be a no-op with exit 0, not a second scanner: the real
s6-svscan takes a lock on the scandir and the second instance dies,
leaving a pidfile pointing at a corpse and `status` answering for the
wrong process.
"""
got = box.svc("S40s6", MODDIR + "/init.d").start()
assert got.ok and "already" in got.text, (
"a second start did not say already-running (rc=%d): %s"
% (got.code, got.text))
def test_stop_leaves_no_processes_behind(self, stopped):
"""The scanner is the process that would otherwise put everything
back, so a stop that kills the services and not the scanner is not a
stop at all -- the same mistake as killing mjpg_streamer without
S65camera's respawn loop. Asked of the process table, which is the
only thing that can answer it.
"""
left = [p for p in stopped.ps()
if "s6-svscan" in p.cmdline or "s6-supervise" in p.cmdline]
assert not left, (
"s6 processes survived stop: %s"
% "; ".join(p.cmdline for p in left))
def test_status_reports_the_stopped_scanner(self, stopped):
got = stopped.svc("S40s6", MODDIR + "/init.d").status()
assert "not running" in got.text, (
"status still claims a scanner after stop: %s" % got.first_line)
def test_stop_removed_the_pidfile(self, stopped):
"""So the next status cannot be answered by a pid the kernel has since
handed to something else -- the stale-pidfile bug this whole library
has two comments about."""
assert not stopped.file("%s/s6-svscan.pid" % MODDIR).exists, (
"stop left %s/s6-svscan.pid behind" % MODDIR)
qa/static/test_ipk.py
+8
View File
@@ -502,6 +502,14 @@ def test_every_payload_file_is_owned_by_a_package():
# never appear in a list it is still being written into.
if rel.startswith(MODDIR + "/var/lib/opkg"):
continue
# The compiled s6-rc database. bin/patch.sh runs s6-rc-compile
# over the source tree anvil-core ships, AFTER the payload is
# installed, so this is generated for the same reason
# .install-manifest is: it describes the payload and cannot be
# finished before the payload is. The SOURCE it is compiled from
# is package-owned and checked like everything else.
if rel.startswith(MODDIR + "/etc/s6-rc/compiled"):
continue
extra.append(rel)
assert not extra, (
qa/static/test_s6rc_source.py
+166
View File
@@ -0,0 +1,166 @@
"""The boot order, asserted by compiling it.
payload/etc/s6-rc/source/ is the boot: which services exist, which waits for
which, and how long each is allowed to take. Everything here asks the COMPILED
database rather than reading the source tree, because the database is what the
printer runs and the compiler is free to reject, rewrite or silently drop what
it was given -- `down` being the one that bit us (s6-rc-compile accepts a
`down` file and discards it, producing a byte-identical servicedir).
Static rather than replica: it needs no printer, no qemu and no docker, and it
catches the whole class of "the graph says something other than we think" in
about a second.
"""
import subprocess
import pytest
from lib.paths import ROOT
pytestmark = pytest.mark.static
SOURCE = (ROOT / "pkgs" / "anvil-core" / "payload"
/ "etc" / "s6-rc" / "source")
# bin/patch.sh section 5b-2 builds a NATIVE skarnet stack here to compile the
# database at build time. Same path, so a developer who has run a build already
# has what this needs and nothing has to be installed.
NATIVE = ROOT / "work" / ".s6-native" / "bin"
# What the boot brings up. mcu-bringup is deliberately absent -- it arrives as
# klipper's dependency, which is how "down by default" has to be expressed:
# a `down` file in a definition directory is accepted and silently dropped.
BOOT_SET = {"wifi", "nginx", "moonraker", "camera", "klipper", "ff-startup",
"ui"}
# Seconds a transition will wait, per service. The camera's matters most: on a
# printer with no camera this is time the UI does not exist for.
TIMEOUTS = {"moonraker": 120000, "camera": 40000, "mcu-bringup": 60000,
"ff-startup": 300000}
def _tool(name):
path = NATIVE / name
if not path.exists():
# A failure, not a skip. See qa/conftest.py: a machine that cannot
# answer this is not a machine where the question does not apply.
pytest.fail(
"no %s -- the s6-rc database was never compiled here, so nothing "
"checked the boot graph. `make build` builds it (bin/patch.sh, "
"section 5b-2); it lands in %s." % (name, NATIVE))
return str(path)
@pytest.fixture(scope="session")
def db(tmp_path_factory):
"""The compiled database, built from the shipped source tree."""
out = tmp_path_factory.mktemp("s6rc") / "db"
built = subprocess.run([_tool("s6-rc-compile"), str(out), str(SOURCE)],
capture_output=True, text=True)
assert built.returncode == 0, (
"s6-rc-compile refused the source tree:\n%s" % built.stderr.strip())
return str(out)
def _db(db, *args):
got = subprocess.run([_tool("s6-rc-db"), "-c", db] + list(args),
capture_output=True, text=True)
assert got.returncode == 0, (
"s6-rc-db %s failed: %s" % (" ".join(args), got.stderr.strip()))
return got.stdout.split()
def test_the_database_is_internally_consistent(db):
checked = subprocess.run([_tool("s6-rc-db"), "-c", db, "check"],
capture_output=True, text=True)
assert checked.returncode == 0, checked.stderr.strip()
def test_the_bundle_is_the_boot_set(db):
assert set(_db(db, "contents", "ok-all")) == BOOT_SET
def test_mcu_bringup_is_reachable_but_not_in_the_bundle(db):
"""The only way s6-rc expresses "start it, but not because the boot asked
for it": be somebody's dependency and nothing else."""
assert "mcu-bringup" not in _db(db, "contents", "ok-all")
assert "mcu-bringup" in _db(db, "list", "all")
@pytest.mark.parametrize("service,wants", [
("klipper", {"mcu-bringup"}),
("ff-startup", {"moonraker", "klipper"}),
("ui", {"ff-startup"}),
])
def test_the_edges(db, service, wants):
"""s6rc-oneshot-runner is filtered out: s6-rc adds it to every oneshot's
dependencies itself, and it is machinery rather than boot order."""
got = {d for d in _db(db, "dependencies", service)
if not d.startswith("s6rc-")}
assert got == wants
def test_nginx_does_not_gate_moonraker(db):
"""S60nginx's header wants nginx first because it proxies moonraker, and
that is a preference, not a dependency. As an edge it would mean
`s6-rc -d change nginx` also stops moonraker -- taking the API down with
the UI, which is what splitting S60web into two scripts prevented."""
assert "nginx" not in _db(db, "dependencies", "moonraker")
@pytest.mark.parametrize("service,want", sorted(TIMEOUTS.items()))
def test_timeouts_reach_the_database(db, service, want):
"""Without one, a service that never comes up holds the boot transition
open for ever and nothing after it starts."""
assert _db(db, "-u", "timeout", service) == [str(want)]
def test_the_oneshot_runner_runs_our_execline(db):
"""MEASURED, and the reason the native stack is configured with the target
--prefix: execline bakes --shebangdir into the #! line s6-rc-compile
writes here. Built with the wrong prefix, every oneshot dies with ENOENT
on the printer while every longrun keeps working."""
run = open("%s/servicedirs/s6rc-oneshot-runner/run" % db).read()
assert run.startswith("#!/usr/data/anvil/bin/execlineb"), (
"the database asks for %r" % run.splitlines()[:1])
def _oneshot_scripts(db):
found = []
for name in _db(db, "list", "oneshots"):
if name.startswith("s6rc-"):
continue
for verb in ("-u", "-d"):
got = subprocess.run(
[_tool("s6-rc-db"), "-c", db, verb, "script", name],
capture_output=True, text=True)
if got.returncode == 0 and got.stdout.strip():
found.append(("%s %s" % (name, verb.strip("-")), got.stdout))
return found
def test_oneshots_have_scripts(db):
assert _oneshot_scripts(db), "no oneshot up/down scripts -- did they move?"
def test_oneshot_shell_parses(db):
"""The `up` and `down` files are EXECLINE command lines carrying shell
inside a `/bin/sh -c "..."` argument, so the shell checks in
test_shell_syntax.py cannot see this code at all -- they would parse the
wrapper and pass. This is the only thing that reads the body, and it reads
the body the COMPILER stored, not the one on disk.
"""
broken = []
for name, argv in _oneshot_scripts(db):
# s6-rc-db writes the stored argv NUL-separated, which is also the
# proof that the shell arrived as ONE argument -- execline lexing the
# body into several would be a script that runs its own first line.
parts = argv.split("\0")
assert parts[:2] == ["/bin/sh", "-c"], (
"%s is not a /bin/sh -c script: %r" % (name, parts[:3]))
body = parts[2]
parsed = subprocess.run(["sh", "-n"], input=body,
capture_output=True, text=True)
if parsed.returncode != 0:
broken.append("%s: %s" % (name, parsed.stderr.strip()))
assert not broken, "oneshot scripts do not parse:\n " + "\n ".join(broken)
qa/static/test_shell_syntax.py
+19 -13
View File
@@ -45,17 +45,17 @@ pytestmark = pytest.mark.static
# THE PRINTER'S SCRIPTS ARE ADDRESSED BY THE SHAPE OF A RECIPE, not by a
# top-level directory: pkgs/<recipe>/payload/ is what that recipe installs
# under $MODDIR, and pkgs/<recipe>/payload/prog/ is the part of that which
# also gets copied onto /usr/prog by bin/patch.sh. Both run on the printer;
# neither is named recipe by recipe here, so a new recipe with scripts in it
# is covered the day it lands.
# anvil-link-prog.sh points the stock /usr/prog paths at. Neither is named
# recipe by recipe here, so a new recipe with scripts in it is covered the day
# it lands.
#
# prog/ USED TO BE A SIBLING of payload/ and is now inside it. That is the
# whole of the change: those files are package members now, so the recipe's
# own hash covers them and anvil-link-prog.sh can point the boot path at the
# installed copy. Note payload/prog/* is matched by payload/*.sh too only for
# the .sh names -- firmwareExe has no extension and needs its own pattern.
SYNTAX_GLOBS = ("bin/*.sh", "pkgs/*/payload/*.sh", "pkgs/*/payload/init.d/S*",
# The s6-rc `run` scripts are in the list because s6-supervise execs them on
# the printer, under the printer's ash, and a bashism in one is a service that
# never starts and says so only in s6's own log.
SYNTAX_GLOBS = ("bin/*.sh", "pkgs/*/payload/*.sh",
"pkgs/*/payload/prog/*.sh", "pkgs/*/payload/prog/firmwareExe",
"pkgs/*/payload/prog/klipperDaemon",
"pkgs/*/payload/etc/s6-rc/" + "source/*/run",
"installer/*.sh",
"test/integration/printer/*.sh",
"qa/replica/actions/*.sh",
@@ -67,6 +67,12 @@ SYNTAX_GLOBS = ("bin/*.sh", "pkgs/*/payload/*.sh", "pkgs/*/payload/init.d/S*",
# fails instead of quietly shrinking.
"pkgs/*.sh", "pkgs/*/build.sh", "pkgs/3rdparty/*/build.sh")
# NOT the oneshots' `up`/`down`. Those are execline command lines, and the
# shell they carry is inside a `/bin/sh -c "..."` argument -- `bash -n` and
# shellcheck would both parse the wrapper, pass, and check none of the code
# that actually runs. Extracting the body and checking THAT belongs in the
# replica lane beside the rest of the s6-rc assertions.
# The subset executed by the printer's busybox ash. bin/ and test/ are
# deliberately absent: they run on the build image, where bash is the shell and
# a bashism is not a defect.
@@ -86,11 +92,11 @@ SYNTAX_GLOBS = ("bin/*.sh", "pkgs/*/payload/*.sh", "pkgs/*/payload/init.d/S*",
#
# NOT here, and it is a real gap rather than a decision:
# pkgs/*/payload/bin/*.sh. wifi-action.sh runs on the printer and no lane
# checks its dialect. It was not covered before the recipe layout either --
# the old glob was payload/*.sh, one level up from it -- so adding it belongs
# with the fix, not with a move.
ASH_GLOBS = ("pkgs/*/payload/*.sh", "pkgs/*/payload/init.d/S*",
# checks its dialect.
ASH_GLOBS = ("pkgs/*/payload/*.sh",
"pkgs/*/payload/prog/*.sh", "pkgs/*/payload/prog/firmwareExe",
"pkgs/*/payload/prog/klipperDaemon",
"pkgs/*/payload/etc/s6-rc/" + "source/*/run",
"installer/*.sh", "qa/replica/actions/*.sh")
PY_GLOBS = ("bin/*.py", "pkgs/*/payload/bin/*.py",