fix(hopper): verify and repair the Linux embedding runtime (#5041)

Verify the distribution Python embedding library during approved Linux Hopper
setup, conditionally repair the matching Ubuntu dependency, and confirm that
the library loads. Share fresh bounded observations with doctor, preserve
unknown configuration and caller cancellation, and join probe exit through
the existing process supervisor.

Validate clean Ubuntu installation, normal-user real Hopper CLI/MCP workflows,
independent ELF/SQLite oracles, cancellation and cleanup regressions, 154
focused checks and all 40 current-head CI checks. Preserve measured artifact
identity, failed attempts and local host-memory guard limitations.

Closes #5015.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
This commit is contained in:
gensokyo-devandKaoru0822 authored and GitHub committed 2026-10-19 00:11:10 +08:00
1 parent 23f984b1a8
commit b0b485abff
14 files changed
+3653 -60

No files matched your search

+14
View File
@@ -599,6 +599,20 @@ host mount and the rest of `/tmp` remain unchanged; this fallback never invokes
`sudo`. `rea doctor --provider hopper --json` reports the selected
strategy and both host and effective mount facts.
Hopper also embeds Python; the Python executable and standard library alone
are insufficient on Ubuntu. Approved setup checks the system interpreter
at `/usr/bin/python3` and, when necessary, installs its matching
`libpython3.X` package, then verifies that its shared library can load.
APT resolves the concrete package (for example, `libpython3.12t64` on Ubuntu
24.04). Doctor includes these fresh observations in the `embedded_python`
details of its `hopper-demo-runtime` check, alongside the private-display
diagnostic. Failed inspections remain unknown rather than proving a library
absent. Run `rea setup --yes --install-hopper` to repair a supported install.
This prerequisite check does not launch the bridge or validate a custom Python
interpreter configured inside Hopper; that interpreter needs its own matching
shared runtime. Custom installation prefixes or shared-library ABIs require
manual dependency repair.
When the package-manager step fails, setup reports the failing command and its
captured output, so a package conflict is distinguishable from an authorization
failure. On CachyOS, an unversioned compatibility package such as
@@ -4,6 +4,10 @@ For Hopper literal string queries, see the [decoded-record checkpoint](hopper-st
It measures record reuse through real Linux CLI/MCP workflows, preserving
native recognition gaps, guard-stopped attempts and negative whole-task costs.
The [Linux embedding-runtime follow-up](hopper-linux-python-runtime.md) checks
actual clean setup, dependency repair and normal-user analysis, retaining
failed attempts and distinguishing prerequisite correctness from speedups.
For JEB on Linux, see the [CE native audit](jeb-linux-native.md). It qualifies
the direct headless launch route and measures selected function reads through
real CLI and production stdio MCP, preserving timeout attempts and the limits
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,203 @@
# Linux Hopper embedding-runtime qualification
This follow-up to the [string-record audit](hopper-string-records.md) resolves
[#5015](https://github.com/morluto/rea/issues/5015): a successful package install
could leave Hopper unable to start its Python bridge. This is a prerequisite
correctness fix. It does not establish faster equivalent investigations.
## Reproduction and owning correction
Baseline REA `093ce695f0a7c3407c7fde520ed8b20c0981b408` ran the actual public
`rea setup --yes --install-hopper --format json` in a fresh Ubuntu Base root,
with full Linux UID/GID support, successful strict APT metadata refresh,
default APT recommendations and unchanged vendor post-install scripts. Setup
reported `ready` and `installed_hopper`; `dpkg --audit` was empty. Python 3.12
was configured, but its reported `/usr/lib/x86_64-linux-gnu/libpython3.12.so.1.0`
was absent. The signed Hopper package did not declare this dependency.
An actual public MCP control-ELF workflow then timed out at its selected
30-second startup budget. Installing only `libpython3.12` resolved to
`libpython3.12t64` version `3.12.3-1ubuntu0.17`: one new package, no upgrades or
removals. The unchanged REA runtime and target then completed all 13 calls.
The independent earlier bootstrap dialog and controlled package intervention
support the embedding-dependency diagnosis; timing alone does not.
The shared Linux installer now asks `/usr/bin/python3` for its version and
shared-library location after the original package transaction. It attempts an
actual load. For a missing standard Ubuntu ABI/layout it installs the matching
versioned library and verifies loadability again. APT success alone cannot
establish readiness. Unknown metadata, alternate prefixes and alternate SONAMEs
retain an explicit unknown diagnosis instead of guessing a package.
Doctor consumes the same fresh, bounded producer, retaining both private-display
and Python observations. Its Python facts concern the system interpreter;
the interpreter actually selected inside Hopper remains unknown. The query
uses isolated Python mode, disables site hooks and bytecode writes, has a
10-second work deadline and 64-KiB output bound, and is not cached: package/interpreter
changes must be visible immediately. The approval plan discloses the conditional
package change without returning the probe program. Existing MCP analysis
contracts and Evidence semantics are unchanged.
Cancellation prevents starting another install stage and preserves the original
reason. A running package transaction is allowed to settle, avoiding forced
interruption of dpkg. This qualifies the install API's signal behavior through
boundary tests; it does not claim new CLI interrupt plumbing or actual
interruption of a real package transaction.
A review follow-up preserves cancellation through doctor as well. A cancelled
Python probe now raises the existing domain cancellation error with its original
reason and completed private-display observations; the owned doctor workflow
performs cleanup and attaches any cleanup failure. It no longer becomes an
ordinary unknown diagnostic. The probe also opts into the existing
stop-and-wait supervisor: cancellation and timeout wait for Python to exit,
with the shared five-second grace period and forced-kill fallback. A real
Python fixture holds its SIGTERM handler until the test releases it, checking
that the probe retains ownership until exit. The success-path native measurements below precede
this error-branch correction. Its boundary tests cover both successful cleanup
and cleanup refusal; final-head validation is tracked on the PR. A pre-fix red
execution and a final-head native rerun are not claimed.
## Pinned environment and artifacts
Native success measurements pin the initial fix at
`0c70be0030f6bb821c4292c4f5b7f776b365c8ec`, based on main
`23f984b1a884baefc3c44e4ab697f6eff61dffb6`.
The measured runtime and bridge manifest covers 1,733 regular files, 26,537,393
bytes. Its SHA-256 is
`8800edc8e70965ae2132e57ca6d999ba276503f27f1045b33305c83cd0e2ff2a`
(sorted paths, compact sorted-key JSON records containing path, bytes and
SHA-256). The tiny MCP loader alone is not a whole-runtime identity.
Earlier prototype runs have a separately recorded manifest; they are not
presented as executions of that same artifact.
- Node 24.18.0; npm 11.16.0; Debian 12 x86-64 host.
- Official [Hopper 6.6.0 Linux demo](https://www.hopperapp.com:443/downloader/public/Hopper-6.6.0-Linux-demo.deb),
35,748,722 bytes, SHA-256
`59705e50f7ebf9104abe1e72ac3949bed1a8e43ec67a8c9510a1c5056c200d27`.
REA's real publisher verifier authenticated signing fingerprint
`7C2D73BAFAF5B9A710A69DA1A26DF815099B1BCD`; launcher SHA-256
`b2271008fb82bf14cfae5c3644cdde2b11fc6804bd7c99d67032ec1b1942d53c`.
- Private [Ubuntu Base 24.04.5 root](https://cdimage.ubuntu.com/ubuntu-base/releases/24.04/release/ubuntu-base-24.04.5-base-amd64.tar.gz),
SHA-256 `e77b6f10c2590cef872b33ee9f635a0e3fd1f57fb074c0e52b5c7f56147a0c86`,
matched its official HTTPS checksum; its publisher signature was not
independently verified. The host glibc 2.36 cannot run this Hopper package;
the private root supplies glibc 2.39. Related packages were installed only
there. Strict refresh used `APT::Update::Error-Mode=any`.
- Small control ELF: 17,360 bytes, SHA-256
`1a6f55303712e096ff5196c7fd6e8acf75473fbf0f7467d7b315f96f50b3f841`.
GNU objdump independently maps the arithmetic at `0x401126` to
`arg0 * 7 + 13`; the original build flags were not recovered. The known
`rea_jeb_native_probe` literal has one native string hit.
- Shipped Debian SQLite 3.40.1-2+deb12u2, `libsqlite3.so.0.8.6`, 1,437,848
bytes, SHA-256
`2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8`.
Independent objdump identifies `sqlite3_libversion_number` at `0xa1d50`,
returning `0x2e6301` (`3040001`). This is a shipped library qualification,
not a claim about current SQLite versions or whole-program correctness.
## Actual workflows and measurements
The accompanying [receipt](hopper-linux-python-runtime.json) preserves native
stage outcomes, elapsed time, CPU time, peak memory, cleanup and token-estimate
scope, including failed and stopped attempts. Installation uses root inside the
owned fixture. Final analysis also uses a normal UID 1004 with a separate home
and the authenticated public publisher evidence copied unchanged into that
user's cache; production verification still checks it. No target was executed
or modified. Non-root package installation through real pkexec authorization,
custom Hopper Python configuration, native Debian installation and other
operating systems remain unverified.
Fresh final setup reported `ready` / `installed_hopper`, installed the matching
shared library, left `dpkg --audit` empty and passed actual public doctor.
Final normal-user sessions completed the positive, repeated, negative and
decompilation checks: 19 `sqlite_master` matches in 1,437 recognized strings,
74 broader `sqlite` matches and the expected version constant. Both CLI runs
matched their MCP facts and Evidence identity; both target files stayed unchanged.
| Stage / outcome | Elapsed (s) | Native CPU (s) | Native peak (MiB) |
| ---------------------------------------------- | ----------: | -------------: | ----------------: |
| Baseline setup: ready, library absent | 75.353 | 39.002 | 1395.4 |
| Baseline control MCP: startup timeout | 49.541 | 24.158 | 1282.5 |
| Install only matching libPython | 5.012 | 1.925 | 174.8 |
| Unchanged runtime control MCP: passed | 24.791 | 23.608 | 1228.4 |
| Fixed fresh setup: ready | 80.739 | 40.404 | 1382.6 |
| Fixed public doctor: healthy | 6.035 | 5.160 | 560.9 |
| Fixed normal-user control MCP: 13 calls passed | 24.434 | 23.305 | 1195.7 |
| Fixed normal-user SQLite MCP: 16 calls passed | 92.504 | 90.138 | 1507.2 |
| Fixed normal-user control CLI: parity passed | 8.981 | 7.831 | 613.5 |
| Fixed normal-user SQLite CLI: parity passed | 70.207 | 68.704 | 781.6 |
Each control MCP session includes discovery and 13 calls: open, status, first
literal query, six repeats, absent query, procedure listing, decompilation and
close. SQLite uses 16 calls: a known exported address avoids a separate name
lookup, and full native string inventory, broader query, regex, case and absence
checks are included. Applicable returned string bytes are checked against the
file. Warm queries must preserve facts and Evidence ID. CLI searches compare
normalized facts and Evidence ID with their corresponding MCP result. Both
target digests are checked after the task.
“Fresh” means a new installation state or Hopper process, as indicated; OS
caches were not flushed. Installation fixtures preload 247 previously fetched
APT archives (138,023,324 bytes) without installing packages. These are not
cold-network benchmarks. The baseline failure and successful repair are
different outcomes, not a paired speedup. This change adds no analysis cache
and makes no token reduction or complex-investigation advantage claim.
Native stages have a 6-GiB container cap, 5-GiB early stop, no swap, 128 tasks,
one CPU and explicit affinity. Their separate controller has a 512-MiB cap,
no swap and 64 tasks. Native plus controller caps total 6.5 GiB. Heavy tasks
run serially; admission waits for at least 16 GiB host availability and aborts
below 8 GiB. Host build/check stages have an 8-GiB cap and 7-GiB early stop.
No other user's processes were signalled.
Native CPU and kernel memory peaks include the container guardian and observer.
Payload-exit observations verify no analysis children remain before stopping
the container. Controller counters are separate partial snapshots; adding
separate peaks is only an upper bound, not a simultaneous measurement. Shared
Docker daemon, containerd and system-manager costs and final counters after
scope removal are unknown. Stopped attempts without payload-exit observations
retain unknown cleanup/counter fields. Observed zero OOM counters at payload
exit are not a claim about the whole host.
## Failed attempts and limits
- A single-ID namespace could not apply the real vendor/package UID/GID
ownership changes. It did not qualify a successful installation.
- An inherited container-loopback proxy prevented publisher downloads; the
fixture was corrected to forward the same configured proxy through the
owned bridge gateway. No global proxy configuration was changed.
- An early APT refresh exited zero with failed-index warnings. It was not
counted as a successful refresh; the strict clean run establishes the result.
- One earlier setup was stopped by the host-memory guard during unpacking.
Its interrupted package state was retained separately; clean fixtures were
used for the qualified baseline and final installations.
- SQLite exceeded a caller-selected 30-second startup budget. With the normal
120-second budget the bridge started, but an initial `libc` positive oracle
failed: those `.dynstr` bytes were not a recognized Hopper string. The
corrected native positive is `sqlite_master`. These attempts remain in the
receipt. Empty recognized-string results do not prove physical absence.
- Early candidate compilation found a TypeScript signal-narrowing error; the
fix reads live signal state through a helper. Lint found oversized new test
groups; they were split by behavior. Failed checks are retained as failures.
- Full local fast checks were stopped twice when other host workloads reduced
available memory. Those attempts did not pass; complete CI and later executed
checks must be reported separately.
- Local package verification completed installation, update, module loading,
discovery and several CLI stages, then the same host-memory guard stopped it
during setup/uninstall after 336.196 seconds (sampled peak 1,742,426,112
bytes). This is an incomplete package gate, despite the separately completed
real Hopper setup and analysis workflows. Its payload-exit counters and
cleanup observation are unknown. CI must complete this gate before merging.
Offline `cl100k_base` estimates include each discovered full catalog, named
arguments and raw replies once per recorded session, plus selected CLI arguments
and stdout. Raw MCP replies can contain both text and structured copies.
The final control and SQLite MCP validation transcripts estimate 845,126 and
1,064,574 tokens respectively, including 805,330 catalog tokens each. The CLI
rows estimate 695 and 2,905 tokens for one selected search each. Those CLI rows
cover different tasks from the multi-call MCP qualification sessions, which
also include full-inventory validation; they do not form an interface-efficiency
comparison. Failed attempts and retries have separate rows in the receipt.
Actual agent/model usage, host schema filtering, protocol framing, prompts,
rendering/replay, independent oracle context and unrecorded in-flight failures
are unmeasured. This is provider qualification, not an agent benchmark.
+40
View File
@@ -28,6 +28,7 @@ import {
import type { VerifiedLinuxHopperPackage } from "./LinuxHopperPublisher.js";
import { OwnedFileHandle } from "../filesystem/OwnedFileHandle.js";
import * as nativeCommands from "../process/ExecFileOutput.js";
import * as pythonRuntime from "./LinuxHopperPython.js";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
describe("Linux package-manager command display", () => {
@@ -123,6 +124,7 @@ class RecordingLinuxHost implements LinuxHopperInstallHost {
installPackage = (
family: LinuxPackageFamily,
archive: string,
_options?: { readonly signal?: AbortSignal },
): Promise<LinuxPackageInstallOutcome> => {
if (!this.verifiedBeforeInstall)
throw new Error("Package manager ran before publisher verification");
@@ -260,6 +262,24 @@ describe("Linux Hopper installation", () => {
});
describe("Linux Hopper package installation", () => {
it("keeps runtime inspection failure distinct from package-manager failure and cleans the archive", async () => {
const host = new RecordingLinuxHost();
const failure = new pythonRuntime.LinuxHopperPythonError(
"configuration_unknown",
"Fixture unknown runtime",
);
host.installPackage = async () => ({
status: "runtime_unavailable",
failure,
});
expect(await installLinuxHopper(host)).toMatchObject({
status: "failed",
reason: "runtime_dependencies",
pythonRuntimeFailure: failure,
});
expect(host.cleaned).toEqual(["/tmp/rea test"]);
});
it.each(["deb", "rpm", "arch"] as const)(
"downloads, verifies, installs, reads back, and cleans a %s package",
async (family) => {
@@ -426,10 +446,30 @@ describe("Linux Hopper cancellation", () => {
).toEqual({
status: "failed",
reason: "cancelled",
cancellationReason: controller.signal.reason,
});
expect(host.downloads).toEqual([]);
expect(host.installed).toEqual([]);
});
it("forwards cancellation to package stages and retains its original cause through cleanup", async () => {
const host = new RecordingLinuxHost();
const controller = new AbortController();
const cause = new Error("caller cancelled between package transactions");
host.installPackage = async (_family, _archive, options) => {
expect(options?.signal).toBe(controller.signal);
expect(host.verifiedBeforeInstall).toBe(true);
controller.abort(cause);
return { status: "cancelled", cause };
};
const result = await installLinuxHopper(host, {
signal: controller.signal,
});
expect(result).toMatchObject({ status: "failed", reason: "cancelled" });
if (result.status === "failed")
expect(result.cancellationReason).toBe(cause);
expect(host.cleaned).toEqual(["/tmp/rea test"]);
});
});
describe("Linux Hopper temporary cleanup ownership", () => {
+178 -41
View File
@@ -1,11 +1,16 @@
import { access, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { join, posix } from "node:path";
import { execFileBoundedOutput } from "../process/ExecFileBoundedOutput.js";
import { execFileOutput } from "../process/ExecFileOutput.js";
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import { downloadHopperPackage } from "./HopperDownload.js";
import {
inspectLinuxHopperPythonRuntime,
linuxHopperPythonUnavailable,
LinuxHopperPythonError,
} from "./LinuxHopperPython.js";
import { RegularFileCleanupFailure } from "../filesystem/OwnedRegularFile.js";
import { err, type Result } from "../domain/result.js";
import {
@@ -29,7 +34,7 @@ export type { LinuxPackageFamily } from "./HopperReleaseFeed.js";
/** Package-manager output retained per stream without stopping the command. */
const PACKAGE_MANAGER_OUTPUT_BYTES = 64 * 1024;
/** Exact stable external effects disclosed before a Linux Hopper installation. */
/** Package mutations disclosed before a Linux Hopper installation. */
export interface LinuxHopperInstallDisclosure {
readonly networkOrigins: readonly string[];
readonly integrity: string;
@@ -48,11 +53,21 @@ export const linuxHopperInstallDisclosure = (
HOPPER_DOWNLOAD_PREFIX,
],
integrity: `Detached PGP signature · primary ${HOPPER_LINUX_PRIMARY_FINGERPRINT} · package byte identity and installed launcher SHA-256 from the official feed`,
commands: linuxPackageManagerCommands(
family,
`<verified-hopper.${family}>`,
isRoot,
).map(({ executable, args }) => formatCommandForDisplay(executable, args)),
commands: [
...linuxPackageManagerCommands(
family,
`<verified-hopper.${family}>`,
isRoot,
),
...(family === "deb"
? [
debianPythonLibraryCommand(
"<libpython matching /usr/bin/python3, if not loadable>",
isRoot,
),
]
: []),
].map(({ executable, args }) => formatCommandForDisplay(executable, args)),
};
};
@@ -108,13 +123,21 @@ export interface LinuxPackageInstallFailure {
}
/** Whether the package-manager step installed the verified package. */
export type LinuxPackageInstallOutcome =
type LinuxPackageCommandOutcome =
| { readonly status: "installed" }
| {
readonly status: "failed";
readonly failure: LinuxPackageInstallFailure;
};
export type LinuxPackageInstallOutcome =
| LinuxPackageCommandOutcome
| { readonly status: "cancelled"; readonly cause: unknown }
| {
readonly status: "runtime_unavailable";
readonly failure: LinuxHopperPythonError;
};
/** External effects required to install the official Linux package. */
export interface LinuxHopperInstallHost {
distribution(): Promise<LinuxDistribution | undefined>;
@@ -131,6 +154,7 @@ export interface LinuxHopperInstallHost {
installPackage(
family: LinuxPackageFamily,
archive: string,
options?: { readonly signal?: AbortSignal },
): Promise<LinuxPackageInstallOutcome>;
verifyArchive(
family: LinuxPackageFamily,
@@ -179,6 +203,10 @@ export type LinuxHopperInstallResult =
readonly packageFailure?: LinuxPackageInstallFailure;
/** Publisher-owned failure, retained for the public setup diagnostic. */
readonly verificationFailure?: HopperReleaseError;
/** System-interpreter inspection, kept separate from package-manager output. */
readonly pythonRuntimeFailure?: LinuxHopperPythonError;
/** Original caller cancellation, without flattening its value into text. */
readonly cancellationReason?: unknown;
} & { readonly cleanupFailure?: LinuxHopperTemporaryCleanupFailure });
/** Failed removal of an owned temporary install directory. */
@@ -282,8 +310,8 @@ export const installLinuxHopper = async (
readonly signal?: AbortSignal;
} = {},
): Promise<LinuxHopperInstallResult> => {
if (options.signal?.aborted === true)
return { status: "failed", reason: "cancelled" };
if (isAborted(options.signal))
return cancelledInstall(options.signal?.reason);
const distribution = await host.distribution();
if (distribution?.hopperSetupSupported !== true)
return { status: "failed", reason: "unsupported_host" };
@@ -339,18 +367,27 @@ export const installLinuxHopper = async (
? "verification_unavailable"
: retained.error.reason,
};
if (options.signal?.aborted === true)
return { status: "failed", reason: "cancelled" };
if (isAborted(options.signal))
return cancelledInstall(options.signal?.reason);
const install = await host.installPackage(
distribution.packageFamily,
archive,
options,
);
if (install.status === "cancelled")
return cancelledInstall(install.cause);
if (install.status === "failed")
return {
status: "failed",
reason: "authorization_or_package_manager",
packageFailure: install.failure,
};
if (install.status === "runtime_unavailable")
return {
status: "failed",
reason: "runtime_dependencies",
pythonRuntimeFailure: install.failure,
};
const launcherPath = "/opt/hopper/bin/Hopper";
const launcherStatus = await host.launcherStatus(
launcherPath,
@@ -367,9 +404,18 @@ export const installLinuxHopper = async (
} catch (cause: unknown) {
result = {
status: "failed",
reason: isAbortCause(cause) ? "cancelled" : "download",
reason:
isAborted(options.signal) || isAbortCause(cause)
? "cancelled"
: "download",
};
}
if (
result.status === "failed" &&
result.reason === "cancelled" &&
isAborted(options.signal)
)
result = { ...result, cancellationReason: options.signal?.reason };
if (temporary === undefined) return result;
try {
await host.cleanup(temporary);
@@ -418,8 +464,8 @@ export const systemLinuxHopperInstallHost = (
download: downloadHopperPackage,
createTemporaryDirectory: () => mkdtemp(join(tmpdir(), "rea-hopper-")),
writeArchive: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
installPackage: (family, archive) =>
installSystemPackage(family, archive, environment),
installPackage: (family, archive, options) =>
installSystemPackage(family, archive, environment, options?.signal),
verifyArchive: (family, release, archive, signal) =>
publisher.verifyArchive(family, release, archive, signal),
retainVerifiedArchive: (verified, archive, signal) =>
@@ -503,36 +549,130 @@ const installSystemPackage = async (
family: LinuxPackageFamily,
archive: string,
environment: Readonly<NodeJS.ProcessEnv>,
signal?: AbortSignal,
): Promise<LinuxPackageInstallOutcome> => {
for (const command of linuxPackageManagerCommands(
family,
archive,
process.getuid?.() === 0,
)) {
const output = await execFileBoundedOutput(
command.executable,
command.args,
{
env: environment,
limit: PACKAGE_MANAGER_OUTPUT_BYTES,
},
);
if (output.exitCode !== 0) {
return {
const cancelled = (): LinuxPackageInstallOutcome => ({
status: "cancelled",
cause: signal?.reason,
});
const isRoot = process.getuid?.() === 0;
for (const command of linuxPackageManagerCommands(family, archive, isRoot)) {
if (isAborted(signal)) return cancelled();
// Let a running package transaction settle; do not start another after
// the caller cancels. Interrupting dpkg can leave configured state partial.
const result = await runPackageManagerCommand(command, environment);
if (result.status === "failed") return result;
}
if (isAborted(signal)) return cancelled();
if (family !== "deb") return { status: "installed" };
// Ubuntu splits the executable/stdlib from the embedding library. Resolve
// the installed distribution interpreter only after python3 is configured.
const python = await inspectLinuxHopperPythonRuntime(environment, signal);
if (isAborted(signal)) return cancelled();
if (!python.ok) return pythonInspectionFailure(python.error);
if (python.value.library.status === "loadable")
return { status: "installed" };
if (python.value.library.status === "unknown")
return {
status: "runtime_unavailable",
failure: linuxHopperPythonUnavailable(python.value.library),
};
const packageName = "libpython" + python.value.version.join(".");
const libraryPath = python.value.library.path;
// Only the standard Ubuntu shared ABI/layout establishes this APT mapping.
// Preserve alternate prefixes/SONAMEs as unknown instead of guessing a package.
if (
posix.basename(libraryPath) !== packageName + ".so.1.0" ||
!/^\/usr\/lib\/[a-z0-9_]+-linux-gnu[a-z0-9_]*$/u.test(
posix.dirname(libraryPath),
)
)
return {
status: "runtime_unavailable",
failure: new LinuxHopperPythonError(
"configuration_unknown",
"Cannot determine the matching APT runtime package for " +
libraryPath +
". Repair this interpreter's shared runtime before retrying.",
),
};
if (isAborted(signal)) return cancelled();
const installed = await runPackageManagerCommand(
debianPythonLibraryCommand(packageName, isRoot),
environment,
);
if (installed.status === "failed") return installed;
if (isAborted(signal)) return cancelled();
// A successful APT exit is not proof that the selected runtime can load.
const verified = await inspectLinuxHopperPythonRuntime(environment, signal);
if (isAborted(signal)) return cancelled();
if (!verified.ok) return pythonInspectionFailure(verified.error);
if (verified.value.library.status !== "loadable")
return {
status: "runtime_unavailable",
failure: linuxHopperPythonUnavailable(verified.value.library),
};
return { status: "installed" };
};
const isAborted = (signal: AbortSignal | undefined): boolean =>
signal?.aborted === true;
const cancelledInstall = (cause: unknown): LinuxHopperInstallResult => ({
status: "failed",
reason: "cancelled",
cancellationReason: cause,
});
const pythonInspectionFailure = (
failure: LinuxHopperPythonError,
): LinuxPackageInstallOutcome =>
failure.reason === "cancelled"
? { status: "cancelled", cause: failure.cause }
: { status: "runtime_unavailable", failure };
interface PackageCommand {
readonly executable: string;
readonly args: readonly string[];
}
const withPackagePrivilege = (
executable: string,
args: readonly string[],
isRoot: boolean,
): PackageCommand =>
isRoot
? { executable, args }
: { executable: "pkexec", args: [executable, ...args] };
const debianPythonLibraryCommand = (
packageName: string,
isRoot: boolean,
): PackageCommand =>
withPackagePrivilege("apt-get", ["install", "-y", packageName], isRoot);
const runPackageManagerCommand = async (
command: PackageCommand,
environment: Readonly<NodeJS.ProcessEnv>,
): Promise<LinuxPackageCommandOutcome> => {
const output = await execFileBoundedOutput(command.executable, command.args, {
env: environment,
limit: PACKAGE_MANAGER_OUTPUT_BYTES,
});
return output.exitCode === 0
? { status: "installed" }
: {
status: "failed",
failure: {
command: formatCommandForDisplay(command.executable, command.args),
exitCode: output.exitCode,
// The streams stay separate; their original interleaved order
// is not recoverable after capture.
// Preserve the separately captured streams and truncation marker.
stdout: output.stdout,
stderr: output.stderr,
outputTruncated: output.outputTruncated,
},
};
}
}
return { status: "installed" };
};
/** Preserve argument boundaries in a command shown to the user. */
@@ -551,10 +691,7 @@ export const linuxPackageManagerCommands = (
family: LinuxPackageFamily,
archive: string,
isRoot: boolean,
): readonly {
readonly executable: string;
readonly args: readonly string[];
}[] => {
): readonly PackageCommand[] => {
const runtimeDependencies =
family === "deb"
? ["xvfb", "xauth", "python3", "libx11-6", "libxtst6"]
@@ -577,9 +714,9 @@ export const linuxPackageManagerCommands = (
["pacman", "-U", "--noconfirm", archive],
];
return commands.map((command) => {
const executable = isRoot ? command[0] : "pkexec";
const executable = command[0];
if (executable === undefined) throw new Error("package command is empty");
return { executable, args: isRoot ? command.slice(1) : command };
return withPackagePrivilege(executable, command.slice(1), isRoot);
});
};
@@ -0,0 +1,264 @@
import { describe, expect, it, vi } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import * as boundedCommands from "../process/ExecFileBoundedOutput.js";
import { err, ok } from "../domain/result.js";
import * as pythonRuntime from "./LinuxHopperPython.js";
import {
linuxHopperInstallDisclosure,
systemLinuxHopperInstallHost,
} from "./LinuxHopper.js";
const missingPython = {
interpreter: "/usr/bin/python3",
version: [3, 14],
library: {
status: "unavailable",
path: "/usr/lib/x86_64-linux-gnu/libpython3.14.so.1.0",
reason: "No such file",
},
} satisfies pythonRuntime.LinuxHopperPythonRuntime;
const readyPython = {
...missingPython,
library: {
status: "loadable",
path: "/usr/lib/x86_64-linux-gnu/libpython3.14.so.1.0",
},
} satisfies pythonRuntime.LinuxHopperPythonRuntime;
const commandSuccess = {
exitCode: 0,
stdout: "",
stderr: "",
outputTruncated: false,
};
describe("Linux Hopper package runtime", () => {
it.each([
{
name: "installs and verifies the matching future Python library",
inspections: [ok(missingPython), ok(readyPython)],
packageCalls: 2,
status: "installed",
},
{
name: "skips the extra APT command for an already loadable library",
inspections: [ok(readyPython)],
packageCalls: 1,
status: "installed",
},
{
name: "does not guess a package for an unknown library location",
inspections: [
ok({
...missingPython,
library: { status: "unknown", reason: "No INSTSONAME" },
} satisfies pythonRuntime.LinuxHopperPythonRuntime),
],
packageCalls: 1,
status: "runtime_unavailable",
},
{
name: "rejects APT success when the library still cannot load",
inspections: [ok(missingPython), ok(missingPython)],
packageCalls: 2,
status: "runtime_unavailable",
},
{
name: "preserves an interpreter inspection failure",
inspections: [
err(
new pythonRuntime.LinuxHopperPythonError(
"query_failed",
"Fixture interpreter unavailable",
),
),
],
packageCalls: 1,
status: "runtime_unavailable",
},
])("$name", async ({ inspections, packageCalls, status }) => {
const directory = await createTestTempDirectory(
"rea-linux-python-install-",
);
const host = systemLinuxHopperInstallHost({ XDG_CACHE_HOME: directory });
const command = vi
.spyOn(boundedCommands, "execFileBoundedOutput")
.mockResolvedValue(commandSuccess);
const inspect = vi
.spyOn(pythonRuntime, "inspectLinuxHopperPythonRuntime")
.mockRejectedValue(new Error("Unexpected extra Python inspection"));
for (const result of inspections) inspect.mockResolvedValueOnce(result);
try {
expect(
await host.installPackage("deb", "/tmp/verified-hopper.deb"),
).toMatchObject({ status });
expect(command).toHaveBeenCalledTimes(packageCalls);
expect(inspect).toHaveBeenCalledTimes(inspections.length);
if (packageCalls === 2) {
const isRoot = process.getuid?.() === 0;
expect(command.mock.calls[1]?.slice(0, 2)).toEqual([
isRoot ? "apt-get" : "pkexec",
[...(isRoot ? [] : ["apt-get"]), "install", "-y", "libpython3.14"],
]);
}
} finally {
try {
await host.close?.();
} finally {
inspect.mockRestore();
command.mockRestore();
}
}
});
it("preserves a failed runtime APT command and does not verify it as installed", async () => {
const directory = await createTestTempDirectory(
"rea-linux-python-failure-",
);
const host = systemLinuxHopperInstallHost({ XDG_CACHE_HOME: directory });
const failed = {
exitCode: 67,
stdout: "original package output",
stderr: "original package error",
outputTruncated: true,
};
const command = vi
.spyOn(boundedCommands, "execFileBoundedOutput")
.mockResolvedValueOnce(commandSuccess)
.mockResolvedValueOnce(failed);
const inspect = vi
.spyOn(pythonRuntime, "inspectLinuxHopperPythonRuntime")
.mockResolvedValue(ok(missingPython));
try {
expect(
await host.installPackage("deb", "/tmp/verified-hopper.deb"),
).toMatchObject({ status: "failed", failure: failed });
expect(command).toHaveBeenCalledTimes(2);
expect(inspect).toHaveBeenCalledTimes(1);
} finally {
try {
await host.close?.();
} finally {
inspect.mockRestore();
command.mockRestore();
}
}
});
});
describe("Linux Hopper package mapping", () => {
it.each([
"/custom/libpython3.14.so.1.0",
"/usr/lib/x86_64-linux-gnu/libpython3.14d.so.1.0",
"/usr/lib/x86_64-linux-gnu/libpython3.13.so.1.0",
])("keeps an unproven APT mapping unknown: %s", async (path) => {
const directory = await createTestTempDirectory("rea-python-package-map-");
const host = systemLinuxHopperInstallHost({ XDG_CACHE_HOME: directory });
const command = vi
.spyOn(boundedCommands, "execFileBoundedOutput")
.mockResolvedValue(commandSuccess);
const inspect = vi
.spyOn(pythonRuntime, "inspectLinuxHopperPythonRuntime")
.mockResolvedValue(
ok({
...missingPython,
library: { ...missingPython.library, path },
}),
);
try {
expect(
await host.installPackage("deb", "/tmp/verified-hopper.deb"),
).toMatchObject({
status: "runtime_unavailable",
failure: {
reason: "configuration_unknown",
message: expect.stringContaining(path),
},
});
expect(command).toHaveBeenCalledTimes(1);
expect(inspect).toHaveBeenCalledTimes(1);
} finally {
try {
await host.close?.();
} finally {
inspect.mockRestore();
command.mockRestore();
}
}
});
});
describe("Linux Hopper package cancellation and disclosure", () => {
it.each([
{ phase: "before", commands: 0, inspections: 0 },
{ phase: "first transaction", commands: 1, inspections: 0 },
{ phase: "query", commands: 1, inspections: 1 },
{ phase: "runtime transaction", commands: 2, inspections: 1 },
{ phase: "verification", commands: 2, inspections: 2 },
])(
"honors cancellation at $phase without starting the next stage",
async ({ phase, commands, inspections }) => {
const directory = await createTestTempDirectory(
"rea-python-install-cancel-",
);
const host = systemLinuxHopperInstallHost({ XDG_CACHE_HOME: directory });
const controller = new AbortController();
const cause = new Error("caller cancelled at " + phase);
let commandCount = 0;
let inspectionCount = 0;
const command = vi
.spyOn(boundedCommands, "execFileBoundedOutput")
.mockImplementation(async () => {
commandCount++;
if (
(phase === "first transaction" && commandCount === 1) ||
(phase === "runtime transaction" && commandCount === 2)
)
controller.abort(cause);
return commandSuccess;
});
const inspect = vi
.spyOn(pythonRuntime, "inspectLinuxHopperPythonRuntime")
.mockImplementation(async (_environment, signal) => {
expect(signal).toBe(controller.signal);
inspectionCount++;
if (
phase === "query" ||
(phase === "verification" && inspectionCount === 2)
)
controller.abort(cause);
return ok(inspectionCount === 1 ? missingPython : readyPython);
});
if (phase === "before") controller.abort(cause);
try {
const result = await host.installPackage(
"deb",
"/tmp/verified-hopper.deb",
{
signal: controller.signal,
},
);
expect(result).toEqual({ status: "cancelled", cause });
if (result.status === "cancelled") expect(result.cause).toBe(cause);
expect(command).toHaveBeenCalledTimes(commands);
expect(inspect).toHaveBeenCalledTimes(inspections);
} finally {
try {
await host.close?.();
} finally {
inspect.mockRestore();
command.mockRestore();
}
}
},
);
it("discloses the conditional privileged library install without returning probe code", () => {
const disclosure = linuxHopperInstallDisclosure("deb", false);
expect(disclosure.commands).toHaveLength(2);
expect(disclosure.commands[1]).toBe(
'pkexec apt-get install -y "<libpython matching /usr/bin/python3, if not loadable>"',
);
expect(linuxHopperInstallDisclosure("rpm", false).commands).toHaveLength(1);
});
});
+257
View File
@@ -0,0 +1,257 @@
import { existsSync } from "node:fs";
import { access, readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect, it, vi } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { execFileOutput } from "../process/ExecFileOutput.js";
import { inspectLinuxHopperPythonRuntime } from "./LinuxHopperPython.js";
const response = (value: unknown) => {
const stdout = JSON.stringify(value);
return {
stdout,
stderr: "",
stdoutBytes: Buffer.byteLength(stdout),
stderrBytes: 0,
};
};
const available = {
version: [3, 14],
library: { status: "loadable", path: "/usr/lib/libpython3.14.so.1.0" },
};
describe("distribution Python runtime inspection", () => {
it("preserves version and library identity without assuming Python 3.12", async () => {
const execute = vi
.fn<typeof execFileOutput>()
.mockResolvedValue(response(available));
const environment = {
PYTHONHOME: "/caller/python",
PYTHONPATH: "/caller/modules",
};
const result = await inspectLinuxHopperPythonRuntime(
environment,
undefined,
execute,
);
expect(result).toEqual({
ok: true,
value: { ...available, interpreter: "/usr/bin/python3" },
});
expect(execute).toHaveBeenCalledWith(
"/usr/bin/python3",
expect.arrayContaining(["-I", "-S", "-B"]),
expect.objectContaining({
env: environment,
timeout: 10_000,
maxBuffer: 64 * 1024,
stopSignal: "SIGTERM",
}),
);
});
it.each([
{ version: [2, 7], library: available.library },
{ version: [3, "14"], library: available.library },
{
version: [3, 14],
library: { status: "loadable", path: "relative/libpython.so" },
},
{
version: [3, 14],
library: { status: "loadable", path: "C:\\not-a-linux-path\\python.dll" },
},
{
version: [3, 14],
library: { status: "loadable", path: "/usr/lib/python\0.so" },
},
{
version: [3, 14],
library: { status: "unavailable", path: "/lib/python.so" },
},
null,
])(
"refuses malformed facts without reporting library absence: %j",
async (value) => {
const execute = vi
.fn<typeof execFileOutput>()
.mockResolvedValue(response(value));
const result = await inspectLinuxHopperPythonRuntime(
{},
undefined,
execute,
);
expect(result).toMatchObject({
ok: false,
error: { reason: "invalid_response" },
});
},
);
it("retains unknown configuration and observes a later repaired library afresh", async () => {
const unknown = {
version: [3, 14],
library: { status: "unknown", reason: "No INSTSONAME" },
};
const execute = vi
.fn<typeof execFileOutput>()
.mockResolvedValueOnce(response(unknown))
.mockResolvedValueOnce(response(available));
expect(
await inspectLinuxHopperPythonRuntime({}, undefined, execute),
).toMatchObject({ ok: true, value: unknown });
expect(
await inspectLinuxHopperPythonRuntime({}, undefined, execute),
).toMatchObject({ ok: true, value: available });
expect(execute).toHaveBeenCalledTimes(2);
});
it("distinguishes execution and JSON failures from an unavailable library", async () => {
const execute = vi
.fn<typeof execFileOutput>()
.mockRejectedValueOnce(new Error("fixture execution failure"))
.mockResolvedValueOnce({
stdout: "{",
stderr: "",
stdoutBytes: 1,
stderrBytes: 0,
});
expect(
await inspectLinuxHopperPythonRuntime({}, undefined, execute),
).toMatchObject({ ok: false, error: { reason: "query_failed" } });
expect(
await inspectLinuxHopperPythonRuntime({}, undefined, execute),
).toMatchObject({ ok: false, error: { reason: "invalid_response" } });
});
});
describe("distribution Python process cleanup", () => {
it.skipIf(process.platform !== "linux" || !existsSync("/usr/bin/python3"))(
"waits for a delayed Python exit before completing cancellation",
async () => {
const directory = await createTestTempDirectory("rea-python-stop-");
const ready = join(directory, "ready");
const stopping = join(directory, "stopping");
const release = join(directory, "release");
const stopped = join(directory, "stopped");
const program = `from pathlib import Path
import signal, sys, time
def stop(signum, frame):
Path(${JSON.stringify(stopping)}).write_text(str(signum))
while not Path(${JSON.stringify(release)}).exists():
time.sleep(0.01)
Path(${JSON.stringify(stopped)}).write_text(str(signum))
sys.exit(0)
signal.signal(signal.SIGTERM, stop)
Path(${JSON.stringify(ready)}).write_text("ready")
while True:
time.sleep(0.1)
`;
const controller = new AbortController();
const reason = { request: "stop owned Python" };
let settled = false;
const execution = inspectLinuxHopperPythonRuntime(
process.env,
controller.signal,
(command, _arguments, options) =>
execFileOutput(command, ["-I", "-S", "-B", "-c", program], options),
).then((result) => {
settled = true;
return result;
});
try {
await vi.waitFor(() => access(ready), { timeout: 5_000, interval: 10 });
controller.abort(reason);
await vi.waitFor(() => access(stopping), {
timeout: 3_000,
interval: 10,
});
// The child cannot exit until released; early settlement loses ownership.
expect(settled).toBe(false);
} finally {
try {
await writeFile(release, "release");
} finally {
controller.abort(reason);
await execution;
}
await vi.waitFor(() => access(stopped), {
timeout: 5_000,
interval: 10,
});
}
expect(await readFile(stopped, "utf8")).toBe("15");
const result = await execution;
expect(result).toMatchObject({
ok: false,
error: { reason: "cancelled" },
});
if (!result.ok) expect(result.error.cause).toBe(reason);
},
);
});
describe("distribution Python isolation and cancellation", () => {
it("preserves cancellation and starts no interpreter for an already cancelled query", async () => {
const controller = new AbortController();
const reason = new Error("caller cancellation");
controller.abort(reason);
const execute = vi.fn<typeof execFileOutput>();
const result = await inspectLinuxHopperPythonRuntime(
{},
controller.signal,
execute,
);
expect(result).toMatchObject({
ok: false,
error: { reason: "cancelled", cause: reason },
});
expect(execute).not.toHaveBeenCalled();
});
it("observes cancellation that arrives during interpreter execution", async () => {
const controller = new AbortController();
const reason = new Error("cancelled during query");
const execute = vi
.fn<typeof execFileOutput>()
.mockImplementation(async () => {
controller.abort(reason);
return response(available);
});
const result = await inspectLinuxHopperPythonRuntime(
{},
controller.signal,
execute,
);
expect(result).toMatchObject({
ok: false,
error: { reason: "cancelled", cause: reason },
});
});
it.skipIf(process.platform !== "linux" || !existsSync("/usr/bin/python3"))(
"runs the real producer without loading caller Python modules or PYTHONHOME",
async () => {
const directory = await createTestTempDirectory("rea-system-python-");
for (const name of ["ctypes.py", "sysconfig.py", "sitecustomize.py"])
await writeFile(
join(directory, name),
"raise RuntimeError('caller module was loaded')\n",
);
const result = await inspectLinuxHopperPythonRuntime({
...process.env,
PYTHONPATH: directory,
PYTHONHOME: join(directory, "missing-python-home"),
});
expect(result).toMatchObject({
ok: true,
value: {
interpreter: "/usr/bin/python3",
version: [3, expect.any(Number)],
},
});
},
);
});
+157
View File
@@ -0,0 +1,157 @@
import { posix } from "node:path";
import { z } from "zod";
import { err, ok, type Result } from "../domain/result.js";
import { execFileOutput } from "../process/ExecFileOutput.js";
const libraryPath = z
.string()
.refine((value) => posix.isAbsolute(value) && !value.includes("\0"));
const runtimeSchema = z.object({
version: z.tuple([z.literal(3), z.number().int().nonnegative()]),
library: z.discriminatedUnion("status", [
z.object({ status: z.literal("loadable"), path: libraryPath }),
z.object({
status: z.literal("unavailable"),
path: libraryPath,
reason: z.string().min(1),
}),
z.object({ status: z.literal("unknown"), reason: z.string().min(1) }),
]),
});
/** Facts reported by the isolated system interpreter, without a GUI launch. */
export type LinuxHopperPythonRuntime = z.infer<typeof runtimeSchema> & {
readonly interpreter: "/usr/bin/python3";
};
/** A failed inspection does not establish that a shared library is absent. */
export class LinuxHopperPythonError extends Error {
readonly _tag = "LinuxHopperPythonError";
constructor(
readonly reason:
| "query_failed"
| "invalid_response"
| "configuration_unknown"
| "runtime_unavailable"
| "cancelled",
message: string,
cause?: unknown,
) {
super(message, { cause });
}
}
// Isolate caller Python paths/site hooks and disable bytecode writes. Inspect
// the distribution interpreter that the package manager installs, not PATH.
const query = `import ctypes, json, os, sys, sysconfig
directory = sysconfig.get_config_var("LIBDIR")
name = sysconfig.get_config_var("INSTSONAME")
if not isinstance(directory, str) or not os.path.isabs(directory) or not isinstance(name, str) or not name:
library = {"status": "unknown", "reason": "Python did not report an absolute shared-library location"}
else:
path = os.path.join(directory, name)
try:
ctypes.CDLL(path)
library = {"status": "loadable", "path": path}
except OSError as error:
library = {"status": "unavailable", "path": path, "reason": str(error)}
print(json.dumps({"version": list(sys.version_info[:2]), "library": library}))
`;
const runtimeCommand = {
executable: "/usr/bin/python3",
args: ["-I", "-S", "-B", "-c", query],
} as const;
// AbortSignal changes while an awaited process runs, despite its readonly type.
const isAborted = (signal: AbortSignal | undefined): boolean =>
signal?.aborted === true;
/** Fresh, bounded inspection; package or interpreter changes must be observable. */
export const inspectLinuxHopperPythonRuntime = async (
environment: Readonly<NodeJS.ProcessEnv>,
signal?: AbortSignal,
execute: typeof execFileOutput = execFileOutput,
): Promise<Result<LinuxHopperPythonRuntime, LinuxHopperPythonError>> => {
const cancelled = () =>
err(
new LinuxHopperPythonError(
"cancelled",
"Python runtime inspection was cancelled.",
signal?.reason,
),
);
if (isAborted(signal)) return cancelled();
let output: string;
try {
const result = await execute(
runtimeCommand.executable,
runtimeCommand.args,
{
env: environment,
signal,
stopSignal: "SIGTERM",
timeout: 10_000,
maxBuffer: 64 * 1024,
},
);
output = result.stdout;
} catch (cause: unknown) {
if (isAborted(signal)) return cancelled();
return err(
new LinuxHopperPythonError(
"query_failed",
"Could not inspect the shared runtime of /usr/bin/python3.",
cause,
),
);
}
if (isAborted(signal)) return cancelled();
let value: unknown;
try {
value = JSON.parse(output);
} catch (cause: unknown) {
return err(
new LinuxHopperPythonError(
"invalid_response",
"System Python returned an invalid shared-runtime description.",
cause,
),
);
}
const parsed = runtimeSchema.safeParse(value);
if (!parsed.success)
return err(
new LinuxHopperPythonError(
"invalid_response",
"System Python returned an invalid shared-runtime description.",
parsed.error,
),
);
return ok({
...parsed.data,
interpreter: runtimeCommand.executable,
});
};
/** Explain a known failed load separately from an unknown configuration. */
export const linuxHopperPythonUnavailable = (
library: Exclude<
LinuxHopperPythonRuntime["library"],
{ readonly status: "loadable" }
>,
): LinuxHopperPythonError =>
new LinuxHopperPythonError(
library.status === "unknown"
? "configuration_unknown"
: "runtime_unavailable",
library.status === "unavailable"
? "The system Python shared runtime cannot load " +
library.path +
": " +
library.reason
: "System Python did not identify its shared runtime: " + library.reason,
);
+5 -2
View File
@@ -222,10 +222,13 @@ export const systemSetupHost = (
result.packageFailure,
"linux",
);
return result.verificationFailure !== undefined
const detail =
result.verificationFailure?.message ??
result.pythonRuntimeFailure?.message;
return detail !== undefined
? {
...failure,
remediation: `${result.verificationFailure.message} ${failure.remediation}`,
remediation: `${detail} ${failure.remediation}`,
}
: failure;
}
+2 -2
View File
@@ -136,7 +136,7 @@ export const setupInstallFailure = (
status: "failed",
code: "runtime_dependency_unavailable",
remediation:
"Hopper was installed but required shared libraries are missing. Rerun rea setup, then apply the hopper-demo-runtime remediation from rea doctor.",
"Hopper's required runtime could not be verified. Inspect rea doctor --provider hopper; on a supported system use rea setup --install-hopper to repair its dependencies.",
};
case "unsupported_hopper_build":
return {
@@ -150,7 +150,7 @@ export const setupInstallFailure = (
status: "failed",
code: "setup_cancelled",
remediation:
"Setup was cancelled before Hopper was installed. Rerun rea setup when ready.",
"Setup was cancelled. Run rea doctor to inspect the current state before retrying.",
};
case "destination_exists":
return {
+1 -1
View File
@@ -301,7 +301,7 @@ const setupPlan = (input: {
: "system package manager",
detail:
input.platform === "linux"
? `Download the current official Hopper demo, verify its PGP publisher and launcher bytes, and retain archive, public-key.asc and release.json under ${input.hopperCacheDirectory ?? `${input.homeDirectory}/.cache/rea/hopper-releases`}/<launcher-sha256> for offline verification. Preserve replaced damaged evidence beside it as <launcher-sha256>.backup-<id>. Install Hopper plus its Xvfb demo-session dependencies. REA uses a private display and selects Hopper's offered demo mode for each analysis session.`
? `Download the current official Hopper demo, verify its PGP publisher and launcher bytes, and retain archive, public-key.asc and release.json under ${input.hopperCacheDirectory ?? `${input.homeDirectory}/.cache/rea/hopper-releases`}/<launcher-sha256> for offline verification. Preserve replaced damaged evidence beside it as <launcher-sha256>.backup-<id>. Install Hopper plus its Xvfb demo-session dependencies${input.linuxPackageFamily === "deb" ? ", including the shared library matching the distribution Python interpreter" : ""}. REA uses a private display and selects Hopper's offered demo mode for each analysis session.`
: "Download the official Hopper package, verify it, and install it. Hopper may show its demo or license prompt when first opened.",
external: true,
operation: "install" as const,
+190 -1
View File
@@ -1,13 +1,202 @@
import { chmod, mkdir, realpath, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect, it as test } from "vitest";
import { expect, it as test, vi } from "vitest";
import { createSystemDoctorHost } from "./doctorRuntime.js";
import { DOCTOR_PROVIDER_IDS } from "./application/DoctorProviderIds.js";
import { providerRegistrationEnvironment } from "./application/SetupRegistrationEnvironment.js";
import { createTestTempDirectory } from "../tests/fixtures/temporaryDirectory.js";
import { writeJadxJarInventory } from "../tests/fixtures/android/jadxJar.js";
import * as privateDisplay from "./hopper/LinuxPrivateDisplayProbe.js";
import * as pythonRuntime from "./application/LinuxHopperPython.js";
import { err, ok } from "./domain/result.js";
import { AnalysisError } from "./domain/analysisErrorBase.js";
import { AnalysisCancelledError } from "./domain/analysisErrorCore.js";
import { createDoctorHostFixture } from "./application/Doctor.fixture.js";
import { runDoctorOwned } from "./application/Doctor.js";
const readyDisplay = {
ok: true,
strategy: "direct",
diagnostic: {
component: "hopper_private_display",
operation: "probe",
reason: "ready",
socket_directory: "/tmp/.X11-unix",
socket_directory_mode: "1777",
mount_read_only: false,
effective_socket_directory_mode: "1777",
effective_mount_read_only: false,
wsl: false,
strategy: "direct",
fallback_reason: null,
xvfb_stderr_bytes: 0,
status: "ready",
failure_code: null,
},
} as const satisfies privateDisplay.LinuxPrivateDisplaySelection;
test.each([undefined, "controlled close refusal"])(
"propagates runtime-probe cancellation through owned doctor cleanup: %s",
async (cleanupFailure) => {
const controller = new AbortController();
const reason = { request: "cancel Python runtime inspection" };
const display = vi
.spyOn(privateDisplay, "selectLinuxPrivateDisplayStrategy")
.mockResolvedValue(readyDisplay);
const python = vi
.spyOn(pythonRuntime, "inspectLinuxHopperPythonRuntime")
.mockImplementation(async (_environment, signal) => {
expect(signal).toBe(controller.signal);
controller.abort(reason);
return err(
new pythonRuntime.LinuxHopperPythonError(
"cancelled",
"Controlled runtime cancellation",
reason,
),
);
});
const systemHost = createSystemDoctorHost({}, controller.signal);
const close = vi.fn(async () => {
await systemHost.close?.();
return cleanupFailure;
});
try {
const failure = await runDoctorOwned(
undefined,
createDoctorHostFixture({
platform: "linux",
configuredHopperPath: "/fixture/Hopper",
signal: controller.signal,
linuxDemoRuntimeCheck: systemHost.linuxDemoRuntimeCheck,
close,
}),
).then(
() => undefined,
(cause: unknown) => cause,
);
if (!(failure instanceof AnalysisError))
throw new Error("Expected a doctor cancellation failure");
const primary = failure.cleanupPrimary ?? failure;
expect(primary).toBeInstanceOf(AnalysisCancelledError);
expect(primary.cause).toBe(reason);
expect(primary.diagnostics).toMatchObject({
hopper_demo_runtime: {
private_display: readyDisplay.diagnostic,
embedded_python: { status: "unknown", reason: "cancelled" },
},
});
expect(close).toHaveBeenCalledTimes(1);
if (cleanupFailure === undefined) expect(failure).toBe(primary);
else expect(failure.cleanupIncomplete).toBe(true);
} finally {
try {
await systemHost.close?.();
} finally {
python.mockRestore();
display.mockRestore();
}
}
},
);
test.each(["unavailable", "unknown"] as const)(
"does not report a healthy Hopper runtime when Python is %s",
async (state) => {
const display = vi
.spyOn(privateDisplay, "selectLinuxPrivateDisplayStrategy")
.mockResolvedValue(readyDisplay);
const python = vi
.spyOn(pythonRuntime, "inspectLinuxHopperPythonRuntime")
.mockResolvedValue(
state === "unavailable"
? ok({
interpreter: "/usr/bin/python3",
version: [3, 14],
library: {
status: "unavailable",
path: "/usr/lib/libpython3.14.so.1.0",
reason: "No such file",
},
})
: err(
new pythonRuntime.LinuxHopperPythonError(
"query_failed",
"Fixture query failed",
),
),
);
try {
const check = await createSystemDoctorHost({}).linuxDemoRuntimeCheck?.();
expect(check).toMatchObject({
name: "hopper-demo-runtime",
ok: false,
classification:
state === "unavailable" ? "missing_dependency" : "unknown",
details: {
private_display: readyDisplay.diagnostic,
embedded_python: {
scope: "system_interpreter",
hopper_interpreter: "unknown",
...(state === "unavailable"
? { status: "observed", library: { status: "unavailable" } }
: { status: "unknown", reason: "query_failed" }),
},
},
});
expect(check?.remediation).toContain(
state === "unavailable"
? "rea setup --install-hopper"
: "Retry rea doctor",
);
} finally {
python.mockRestore();
display.mockRestore();
}
},
);
test("preserves both X11 and Python failures and their independent remediation", async () => {
const display = vi
.spyOn(privateDisplay, "selectLinuxPrivateDisplayStrategy")
.mockResolvedValue({
ok: false,
strategy: "unavailable",
exitCode: 80,
diagnostic: {
...readyDisplay.diagnostic,
status: "error",
failure_code: "x11_socket_directory_unusable",
},
});
const python = vi
.spyOn(pythonRuntime, "inspectLinuxHopperPythonRuntime")
.mockResolvedValue(
ok({
interpreter: "/usr/bin/python3",
version: [3, 14],
library: { status: "unknown", reason: "No INSTSONAME" },
}),
);
try {
const check = await createSystemDoctorHost({}).linuxDemoRuntimeCheck?.();
expect(check).toMatchObject({
ok: false,
classification: "config_drift",
details: {
private_display: { failure_code: "x11_socket_directory_unusable" },
embedded_python: { status: "observed", library: { status: "unknown" } },
},
});
expect(check?.remediation).toContain("user and mount namespaces");
expect(check?.remediation).toContain("shared-runtime configuration");
} finally {
python.mockRestore();
display.mockRestore();
}
});
const it = test.skipIf(process.platform === "win32");
+68 -13
View File
@@ -12,6 +12,11 @@ import { readIdaConfiguration } from "./ida/IdaConfiguration.js";
import { parseJebMcpEndpoint } from "./jeb/JebMcpEndpoint.js";
import { selectLinuxPrivateDisplayStrategy } from "./hopper/LinuxPrivateDisplayProbe.js";
import { ok } from "./domain/result.js";
import { AnalysisCancelledError } from "./domain/analysisErrorCore.js";
import {
inspectLinuxHopperPythonRuntime,
linuxHopperPythonUnavailable,
} from "./application/LinuxHopperPython.js";
/** Compose provider diagnostics at the outer CLI adapter boundary. */
export const createSystemDoctorHost = (
@@ -60,7 +65,8 @@ export const createSystemDoctorHost = (
fileCloseResources,
}),
],
linuxDemoRuntimeCheck: () => inspectLinuxPrivateDisplay(environment),
linuxDemoRuntimeCheck: () =>
inspectLinuxHopperDemoRuntime(environment, signal),
});
};
@@ -68,13 +74,31 @@ const HOPPER_DEMO_HELPER_PATH = fileURLToPath(
new URL("../scripts/hopper-demo-x11.py", import.meta.url),
);
const inspectLinuxPrivateDisplay = async (
const inspectLinuxHopperDemoRuntime = async (
environment: Readonly<NodeJS.ProcessEnv>,
signal?: AbortSignal,
): Promise<DoctorCheck> => {
const selection = await selectLinuxPrivateDisplayStrategy({
helperPath: HOPPER_DEMO_HELPER_PATH,
environment,
});
const python = await inspectLinuxHopperPythonRuntime(environment, signal);
const pythonFailure = !python.ok
? python.error
: python.value.library.status !== "loadable"
? linuxHopperPythonUnavailable(python.value.library)
: undefined;
const pythonDiagnostic = {
scope: "system_interpreter",
hopper_interpreter: "unknown",
...(python.ok
? { status: "observed", ...python.value }
: {
status: "unknown",
reason: python.error.reason,
message: python.error.message,
}),
};
const value = selection.diagnostic;
const detail = [
`strategy=${selection.strategy}`,
@@ -84,29 +108,60 @@ const inspectLinuxPrivateDisplay = async (
`effective_mode=${value.effective_socket_directory_mode ?? "unknown"}`,
`effective_mount_read_only=${String(value.effective_mount_read_only)}`,
`wsl=${String(value.wsl)}`,
`system_python=${python.ok ? python.value.library.status : "unknown"}`,
].join("; ");
if (selection.ok)
const details = {
private_display: { ...value },
embedded_python: pythonDiagnostic,
};
if (!python.ok && python.error.reason === "cancelled")
throw new AnalysisCancelledError("doctor", {
cause: python.error.cause,
diagnostics: { hopper_demo_runtime: details },
});
if (selection.ok && pythonFailure === undefined)
return {
name: "hopper-demo-runtime",
ok: true,
classification: "healthy",
detail,
details: { private_display: { ...value } },
details,
};
const missingDependency =
value.failure_code === "runtime_dependency_unavailable" ||
value.failure_code === "x11_authorization_failed";
value.failure_code === "x11_authorization_failed" ||
(python.ok && python.value.library.status === "unavailable");
const pythonRemediation =
pythonFailure === undefined
? undefined
: pythonFailure.reason === "runtime_unavailable"
? pythonFailure.message +
" On a supported system run rea setup --install-hopper, then rerun rea doctor --provider hopper. This checks the system interpreter; a custom Hopper Python interpreter needs its own matching shared runtime."
: pythonFailure.reason === "configuration_unknown"
? pythonFailure.message +
" Inspect this interpreter's shared-runtime configuration, then rerun rea doctor --provider hopper."
: pythonFailure.message +
" Retry rea doctor --provider hopper and inspect /usr/bin/python3 if its runtime remains unknown.";
const displayRemediation = selection.ok
? undefined
: value.failure_code === "x11_socket_directory_unusable"
? "Enable unprivileged user and mount namespaces, or provide a writable mode-1777 /tmp/.X11-unix directory. Do not mutate the current host mount with sudo; then rerun rea doctor --provider hopper."
: value.failure_code === "runtime_dependency_unavailable" ||
value.failure_code === "x11_authorization_failed"
? "Install Xvfb, xauth, Python 3, libX11, and libXtst, then rerun rea doctor --provider hopper."
: "Inspect the private-display diagnostic, correct the reported local condition, and rerun rea doctor --provider hopper.";
return {
name: "hopper-demo-runtime",
ok: false,
classification: missingDependency ? "missing_dependency" : "config_drift",
classification: missingDependency
? "missing_dependency"
: selection.ok
? "unknown"
: "config_drift",
detail,
details: { private_display: { ...value } },
remediation:
value.failure_code === "x11_socket_directory_unusable"
? "Enable unprivileged user and mount namespaces, or provide a writable mode-1777 /tmp/.X11-unix directory. Do not mutate the current host mount with sudo; then rerun rea doctor --provider hopper."
: missingDependency
? "Install Xvfb, xauth, Python 3, libX11, and libXtst, then rerun rea doctor --provider hopper."
: "Inspect the private-display diagnostic, correct the reported local condition, and rerun rea doctor --provider hopper.",
details,
remediation: [pythonRemediation, displayRemediation]
.filter((message) => message !== undefined)
.join(" "),
};
};