firmware/qa/conftest.py

"""The qa suite: pytest is the only framework.

This is the only suite. It decides whether a package bricks a printer. The
harness that used to stand beside it, and the host-side test/ tree it wrapped,
are both gone -- see docs/qa-migration.md.

WHAT IS DIFFERENT HERE

The suite this replaced ran one `docker run --privileged` per case script and
got one boolean back, because the assertions lived in POSIX sh inside the qemu
chroot. 6,280 lines of case-*.sh reported 13 bits between them. The rule here
is:

    shell inside the chroot performs actions; python on the host asserts.

Shell in qemu is irreplaceable for DOING things -- the printer's busybox, its
tar, its OpenSSL, and the fact that our scripts survive its ash is a large part
of what the suite proves. It is a bad place to JUDGE things. Where a fact is
only observable inside the replica, qa/lib/replica.py exposes a typed probe for
it; a case script never branches on it.

LANES

    static    needs nothing but the checkout and the lint tools. milliseconds.
    replica   needs docker + qemu + the proprietary firmware. seconds to
              minutes. The gates that decide whether a package bricks.

Select with `-m static` / `-m replica`, or by path.

SKIPS, AND WHY THERE IS NO FLAG CONTROLLING THEM

A missing tool, a missing daemon or a missing firmware image is a FAILURE, at
the point that needs it. Not a skip, and not a skip that some flag can be asked
to promote later.

This suite had a `--strict-skip=<lane|all>` for exactly one commit, and it was
the wrong shape. It was ALLOW_SKIP inverted -- and ALLOW_SKIP's defect was
never which direction it named things in, it was that THE DECISION LIVED IN THE
CI INVOCATION. An accept-list in ci.yml has to be edited whenever a gate moves;
a strict-list in ci.yml has to be remembered whenever a job is added. Both
degrade silently when someone gets the command line wrong, which is the same
disease as the printer-sim job that was gated on an unset secret and therefore
never ran once in its entire existence.

So the verdict lives with the knowledge. `shellcheck` missing means the machine
cannot run the check, and no configuration exists in which that is a complete
run, so the test fails and says how to fix it. Same for pyflakes, for the docker
CLI, for the daemon, and for the replica image.

    A GATE THAT DID NOT RUN MUST NEVER LOOK GREEN -- and the way to guarantee
    that is to not have a mode in which it can.

`pytest.skip` stays available for the case it is actually for: a question that
does not APPLY to this configuration, as opposed to one this machine happens to
be unequipped to ask. A test that only means something on a Creator5Pro is the
shape that earns it. "I could not find the tool" is not.

Three skips do not meet that bar: test_packages.py's payload questions read
work/modpayload-root, which only bin/payload.sh produces. Noted rather than
hidden -- and note that `make test` DELETES that directory when it clears up
after its fixture build, so running the old suite quietly costs the static
lane three tests. See docs/qa-migration.md.
"""
import os
import sys

import pytest

# qa/ on the path so `from lib.replica import Printer` works from any subdir.
# Not a package install: this tree is not shipped and never will be, and a
# setup.py for it would be one more thing that can disagree with reality.
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))

LANES = ("static", "replica", "simulator")


def pytest_addoption(parser):
    parser.addoption('--run-simulator', action='store_true',
                     help='Run desktop Klipper/AFC simulation scenarios')


def pytest_collection_modifyitems(config, items):
    if config.getoption('--run-simulator'):
        return
    excluded = [item for item in items if item.get_closest_marker('simulator')]
    items[:] = [item for item in items if not item.get_closest_marker('simulator')]
    config.hook.pytest_deselected(items=excluded)


def pytest_configure(config):
    for lane in LANES:
        config.addinivalue_line(
            "markers", "%s: the %s lane -- see qa/conftest.py" % (lane, lane))
    _check_distribution(config)


def _check_distribution(config):
    """Refuse the xdist mode that would start a container per worker.

    The `printer` fixture is module-scoped, and xdist's DEFAULT distribution
    (`--dist load`) hands out individual tests. Four workers pulling tests
    from one module each instantiate that module's fixture, so `-n 4` would
    assemble four replicas of the same machine instead of one -- four binfmt
    registrations, four mount layouts, and on a locally built replica four
    stock installs under qemu. It would still pass, slowly, which is why it
    needs catching here rather than in review: the symptom is a suite that
    got mysteriously slower when it was parallelised.

    `loadscope` groups by module (`loadfile` by file, `loadgroup` by an
    explicit mark) -- any of those keeps one container per module, which is
    the whole isolation model. Enforced rather than defaulted, because
    silently rewriting somebody's -n flag is its own kind of surprise.
    """
    dist = getattr(config.option, "dist", "no")
    if dist in ("no", "loadscope", "loadfile", "loadgroup"):
        return
    raise pytest.UsageError(
        "--dist=%s would start one replica per worker per module. Use "
        "--dist=loadscope (or loadfile), which keeps each module on one "
        "worker -- see _check_distribution in qa/conftest.py." % dist)