diff --git a/assets/rgui/font/bitmap10x10_chn.bin b/assets/rgui/font/bitmap10x10_chn.bin new file mode 100644 index 0000000..08097fe Binary files /dev/null and b/assets/rgui/font/bitmap10x10_chn.bin differ diff --git a/assets/rgui/font/bitmap10x10_eng.bin b/assets/rgui/font/bitmap10x10_eng.bin new file mode 100644 index 0000000..25dfaf2 Binary files /dev/null and b/assets/rgui/font/bitmap10x10_eng.bin differ diff --git a/assets/rgui/font/bitmap10x10_jpn.bin b/assets/rgui/font/bitmap10x10_jpn.bin new file mode 100644 index 0000000..8463a93 Binary files /dev/null and b/assets/rgui/font/bitmap10x10_jpn.bin differ diff --git a/assets/rgui/font/bitmap10x10_kor.bin b/assets/rgui/font/bitmap10x10_kor.bin new file mode 100644 index 0000000..6450648 Binary files /dev/null and b/assets/rgui/font/bitmap10x10_kor.bin differ diff --git a/assets/rgui/font/bitmap10x10_rus.bin b/assets/rgui/font/bitmap10x10_rus.bin new file mode 100644 index 0000000..7bf8671 Binary files /dev/null and b/assets/rgui/font/bitmap10x10_rus.bin differ diff --git a/assets/rgui/font/bitmap6x10_eng.bin b/assets/rgui/font/bitmap6x10_eng.bin new file mode 100644 index 0000000..13faaf7 Binary files /dev/null and b/assets/rgui/font/bitmap6x10_eng.bin differ diff --git a/assets/rgui/font/bitmap6x10_lse.bin b/assets/rgui/font/bitmap6x10_lse.bin new file mode 100644 index 0000000..dac5bfd Binary files /dev/null and b/assets/rgui/font/bitmap6x10_lse.bin differ diff --git a/docs/ACTIVE.md b/docs/ACTIVE.md index 91f219f..941dd24 100644 --- a/docs/ACTIVE.md +++ b/docs/ACTIVE.md @@ -9,37 +9,21 @@ _Updated: 2026-09-18_ ## Now -**The title builds.** `bash tools/build-title.sh` compiles RetroArch's own sources -(225 objects), archives them, links them with `src/` through the native pipeline's -CRT, signs the result and assembles `dist/PPSA99169/`: **`eboot.bin` 8,026,239 -bytes**, `container: signed, plaintext`, 12 segments, `integrity: valid`, plus -`sce_module/libc.prx` and the `sce_sys/` metadata. The folder's digests are in -`dist/PPSA99169/manifest.sha256` and `bash tools/check-manifest.sh` passes on it. +**The console loop runs itself.** `bash tools/run-title.sh` builds, publishes, +verifies what the console stored, listens, launches, watches, closes the title +itself and prints the title's own trace. No upload needs a person any more. -**The video driver is registered, and that is verified, not assumed.** The port's -changes to RetroArch live in `patches/series` and are applied by -`tools/apply-port-patches.py` to the configured copy under `build/ra-conf` — -never to `vendor/retroarch`. The evidence is a relocation, not a run: -`readelf -r build/ra/obj/gfx_video_driver.c.o` shows `.rela.data.video_drivers` -holding exactly two entries, `video_ps5` then `video_null`, and -`nm build/llvm-pie.elf` shows `video_ps5`, `video_null`, `rarch_main` and one -`main`. +**The title runs and stays up.** 1500+ frames in 25 s, closed by the script, after +`patches/series` 0004 fixed a null joypad driver that killed the second pass +through the runloop (see `docs/FINDINGS.md`). -**The entry point is reconciled.** RetroArch's `main` is compiled out with -`-DHAVE_MAIN`, the flag RetroArch's own desktop build passes; `src/main.cpp` -supplies `main` and calls `rarch_main` with `-f -c /app0/retroarch.cfg --verbose`, -and the SDK's `_start` remains the process entry. Two `main` symbols would be a -link error, and there is no link error. - -**Everything the abandoned route left behind is gone.** The conversion tools, the -baseline tree, the ports cache, `tools/deploy.py`, `tools/console-launch.sh`, -`tools/check-payload.sh`, `tools/scaffold-native.sh`, `tools/build-native-app.sh`, -`tools/install-title.sh` and `tools/deploy.sh` are deleted; the tools that remain -are the ones `tools/build-title.sh` and `tools/verify.sh` actually call, and every -tool named by another tool exists. - -**`RGUI on screen` is the goal, and it is not met.** The title is built and -manifested; it has not run. +**The menu does not draw yet, and the reason is measured.** RGUI initialises and +holds a 320x240 framebuffer, but `GFX_DISP_FLAG_FB_DIRTY` is never set, so +`rgui_set_texture` returns every frame and the driver reports `no-menu-source`. +That flag is set at the end of `rgui_render`, which is only reached through +`menu->driver_ctx->render` under `if (BIT64_GET(menu->state, MENU_STATE_BLIT))`. +The menu's renderer never runs; finding which condition above that call is false +is the next step, and `docs/FINDINGS.md` records the path there. ## Next diff --git a/docs/FINDINGS.md b/docs/FINDINGS.md index 574e767..ae3a005 100644 --- a/docs/FINDINGS.md +++ b/docs/FINDINGS.md @@ -1059,3 +1059,44 @@ running. **Boundary.** The console's VideoOut ABI as this project's SDK declares it. Anything that changes one of these five values is a console-side change, not a build option. + +## A null joypad driver is this console's normal state, and upstream dereferences it + +**Measured.** With every joypad driver upstream ships unavailable - udev, linuxraw, +SDL, XInput, dinput all need a library or a header this SDK does not carry - +`input_st->primary_joypad` is NULL. `input_driver_collect_system_input` passes it +straight to `input_joypad_analog_axis`, which reads `drv->axis` without checking +`drv`: the function guards every axis member against `AXIS_NONE` and never guards +the struct. + +**What it looked like.** Nothing like a null pointer. The title started, the +display opened, the first frame was presented, and then it died on the *second* +pass through the runloop with SIGSEGV and fault address 0x18. The reason it +survived the first pass is that the call is inside `if (menu_is_alive)`, and +`MENU_ST_FLAG_ALIVE` is set after the first frame. Finding it took a probe in the +caller, then one at the function entry, then reading the loop: the fault address +0x18 is `joypad_info.joy_idx` (0x10) plus the `auto_binds` member (0x18). + +**Fix.** `patches/series` 0004 returns 0 when `drv` is NULL, which is what the +function already answers when there is no axis to read. + +## The menu is alive but its framebuffer is never marked dirty + +**Measured.** `/app0/trace.txt` from a 25-second run: `rgui_fonts_init` completes, +RGUI holds a 320x240 framebuffer, and `rgui_set_texture` is called every frame - +but `GFX_DISP_FLAG_FB_DIRTY` is 0 on every one of those calls, so it returns +before handing anything to the driver. The driver therefore reports +`no-menu-source 4x4` for every frame: nothing is ever drawn, which is what "the +screen stayed black" is. + +**Where the flag should come from.** `GFX_DISP_FLAG_FB_DIRTY` is set at the end of +`rgui_render`, and `rgui_render` is only reached through +`menu->driver_ctx->render` inside `if (BIT64_GET(menu->state, MENU_STATE_BLIT))`. +So the menu's renderer is never running. That is the next thing to measure: which +of the conditions above the call is false. + +**Also measured, and separately true.** RGUI's bitmap fonts are downloaded assets, +not built-ins: `bitmapfont_10x10_load` returns NULL when +`/rgui/font/bitmap10x10_eng.bin` is missing, `rgui_fonts_init` then fails, +and `rgui_init` jumps to its error label with the menu dead. They are now bundled +under `assets/rgui/font/` and shipped in the title folder. diff --git a/docs/PHASE_LOG.md b/docs/PHASE_LOG.md index f22ca63..cc729af 100644 --- a/docs/PHASE_LOG.md +++ b/docs/PHASE_LOG.md @@ -655,3 +655,59 @@ old bytes. The console's FTP accepts the transfer and serves something else, so the console's owner uploads by hand. **Commit.** `eebba81` — Link the title and register the PS5 video driver. + +## 2026-09-18: The loop is automated, the crash is fixed, and the menu's silence is explained + +**The console loop is one command now.** `tools/run-title.sh` builds, publishes +over FTP, verifies what the console stored, starts the kernel-log listener before +launching, launches, watches, closes the title itself, and prints the title's own +trace. It exists because the hand-driven loop had produced two bad readings: runs +overlapped, so a trace from one launch was read as another's result, and a probe +build was left in place and mistaken for a finding. No round after this one +needs a person to upload anything. + +**Verification had to change to match this console.** It converts a signed fake +self into a raw ELF as it stores it, so the file it serves is a different +container from the file sent - the stored image is 8,117,072 bytes against a +signed 8,029,263 and only 12.8% of the bytes agree. A digest comparison therefore +answers the wrong question and was rejecting successful uploads. eboot.bin is now +verified by the strings that only this build carries, which is the same evidence +the debugging used. `sce_module/libc.prx` is the one path this console will not +replace - four uploads, a fresh filename and a full listing all returned the +console's own 1,335,962-byte file - so it is reported and kept, not fatal: the +title runs against the console's copy, and failing on it blocked eboot.bin from +being published at all. + +**The crash is fixed, and it was a null joypad driver.** Every joypad driver +upstream ships needs a library or header this SDK does not carry, so +`primary_joypad` is NULL, and `input_joypad_analog_axis` reads `drv->axis` without +checking `drv`. It only runs when the menu is alive, which is why the first pass +through the runloop survived and the second died: SIGSEGV, fault address 0x18, +which is `joypad_info.joy_idx` plus the `auto_binds` member. `patches/series` 0004 +returns 0 when there is no driver - the answer the function already gives when +there is no axis to read. The title now runs indefinitely: 1500+ frames in 25 +seconds, closed by the script. + +**The menu's silence is explained, and it is not the driver.** RGUI initialises, +loads its fonts from the assets now bundled under `assets/rgui/font/`, holds a +320x240 framebuffer, and `rgui_set_texture` is called every frame - but +`GFX_DISP_FLAG_FB_DIRTY` is 0 on every one of those calls, so it returns before +handing anything over and the driver reports `no-menu-source` for every frame. +That flag is set at the end of `rgui_render`, which is only reached through +`menu->driver_ctx->render` under `if (BIT64_GET(menu->state, MENU_STATE_BLIT))`. +The menu's renderer is never running; finding which condition above it is false is +the next step. + +**The evidence.** `bash tools/run-title.sh` → built, published with eboot.bin +verified by its own markers, ran 25 s, closed by the script; `/app0/trace.txt` +shows `ps5_frame 1500: no-menu-source 4x4 present=1`, i.e. a healthy loop +presenting frames and no menu pixels. `tools/verify.sh` → PASS (format unit build +integration evidence). Port changes against upstream: `retroarch.c` 10 lines, +`gfx/video_driver.{c,h}` 5, `input/input_driver.c` 6, `runloop.c` and +`menu/drivers/rgui.c` 0 - every probe removed, verified by diff. + +**One thing to check later.** `sce_sys/icon0.png` shows as modified and I cannot +account for it: the worktree file and `title/assets/retroarch.png` are both +512x512 but differently encoded, and nothing in this round touches it. It is +committed as it stands rather than reverted, and flagged here so it is not a +silent change. diff --git a/patches/series b/patches/series index 57726ab..289a238 100644 --- a/patches/series +++ b/patches/series @@ -15,3 +15,16 @@ # without declaring, so that --disable-xkbcommon is accepted # qb/config.params.sh 0002-declare-have-xkbcommon +# +# 0003 rename upstream's `main` so the SDK's _start has one definition to reach +# and rarch_main keeps the frontend's main loop (see the note in +# tools/apply-port-patches.py: defining HAVE_MAIN instead compiles the loop +# out, which is a title that initialises everything and exits) +# retroarch.c +0003-rename-upstream-main +# +# 0004 guard input_joypad_analog_axis against a null joypad driver, which is this +# console's normal state: without it the menu's analog read faults on the +# second pass through the runloop (see tools/apply-port-patches.py) +# input/input_driver.c +0004-guard-the-joypad-analog-read diff --git a/sce_sys/icon0.png b/sce_sys/icon0.png index b5ee066..5a5f8f4 100644 Binary files a/sce_sys/icon0.png and b/sce_sys/icon0.png differ diff --git a/src/main.cpp b/src/main.cpp index cfb0056..82780e5 100644 --- a/src/main.cpp +++ b/src/main.cpp @@ -17,10 +17,20 @@ * -f fullscreen, which on a console is the only mode * -c the config to read and write, inside the title's own folder * --verbose so the run is readable in the console's log + * + * Why this file writes a trace. The first run of this title on the console ended + * with the kernel reporting `eboot.bin calls exit() exit_value=0` and nothing + * else: no output, no crash, no message. A program that exits zero and says + * nothing is indistinguishable from one that never reached `main`, and the + * console's log cannot be asked which it was. So the entry point and the video + * driver append a line per step to /app0/trace.txt, and after a run that file + * says how far it got. See src/trace.hpp. */ #include +#include "trace.hpp" + /* RetroArch's entry, in C. */ extern "C" int rarch_main(int argc, char *argv[], void *data); @@ -33,6 +43,10 @@ constexpr const char *config_path = "/app0/retroarch.cfg"; int main() { + /* First thing: prove that control reached this function at all, before + * anything that could fail. */ + ps5::debug::mark("main() entered; static constructors have already run"); + /* argv must be writable and NULL-terminated: RetroArch's option parsing * walks it the way the C runtime would have. */ char arg0[] = "retroarch"; @@ -45,5 +59,13 @@ int main() }; (void)config_path; - return rarch_main(static_cast(sizeof(argv) / sizeof(argv[0])) - 1, argv, nullptr); + ps5::debug::mark("argv built: retroarch -f -c /app0/retroarch.cfg --verbose"); + + const int status = + rarch_main(static_cast(sizeof(argv) / sizeof(argv[0])) - 1, argv, nullptr); + + /* If this line is on the console, the frontend ran and returned by itself. */ + ps5::debug::mark_value("rarch_main returned", status); + + return status; } diff --git a/src/trace.cpp b/src/trace.cpp new file mode 100644 index 0000000..b16d071 --- /dev/null +++ b/src/trace.cpp @@ -0,0 +1,52 @@ +/* + * PS5 RetroArch - the startup trace. See src/trace.hpp for why it exists. + * + * Copyright (C) 2026 Mihawk + * SPDX-License-Identifier: GPL-3.0-or-later + */ + +#include "trace.hpp" + +#include + +namespace ps5::debug +{ +namespace +{ +/* Inside the title's own folder, which the console mounts at /app0 and which the + * trace files written by an earlier build proved is writable. */ +constexpr const char *trace_path = "/app0/trace.txt"; + +void write(const char *line) noexcept +{ + std::FILE *file = std::fopen(trace_path, "a"); + if (file == nullptr) + return; + std::fputs(line, file); + std::fputc('\n', file); + std::fflush(file); + std::fclose(file); +} +} // namespace + +void mark(const char *step) noexcept +{ + write(step); +} + +void mark_value(const char *step, long long value) noexcept +{ + char line[192]; + std::snprintf(line, sizeof(line), "%s = %lld", step, value); + write(line); +} + +void mark_init(const char *step, bool have_video, int width, int height) noexcept +{ + char line[192]; + std::snprintf(line, sizeof(line), "%s (video=%s width=%d height=%d)", step, + have_video ? "present" : "NULL", width, height); + write(line); +} +} // namespace ps5::debug + diff --git a/src/trace.hpp b/src/trace.hpp new file mode 100644 index 0000000..5301252 --- /dev/null +++ b/src/trace.hpp @@ -0,0 +1,43 @@ +/* + * PS5 RetroArch - a startup trace, to be read from the console's title folder. + * + * Copyright (C) 2026 Mihawk + * SPDX-License-Identifier: GPL-3.0-or-later + * + * Why this exists. The title's first runs on the console ended with the kernel + * reporting `eboot.bin calls exit() exit_value=0` and nothing else - no output, no + * crash, no message. `main` had already been shown to run, so the question was + * how far the frontend got before returning. There is no debugger here, no + * console output that reaches the kernel log, and the title's own stdout has no + * reader: files in the title's folder are the only channel. + * + * So each step appends one line to /app0/trace.txt. Appending rather than + * overwriting matters - the sequence is the information, and a file per step + * cannot show order if the program dies between two writes. The file is opened, + * written and closed on every call so a line is on the console even if the + * process is killed immediately afterwards. + * + * This is a development aid. It is small, it is on the title's own writable + * folder, and it is removed when the reason for it is gone; docs/ACTIVE.md says + * when that is. + */ + +#ifndef PS5_RETROARCH_TRACE_HPP +#define PS5_RETROARCH_TRACE_HPP + +namespace ps5::debug +{ +/* Append one line to the trace file. Safe to call before anything is + * initialised: it uses stdio only, opens on each call, and ignores every error, + * because a trace that can fail the program it is tracing is worse than none. */ +void mark(const char *step) noexcept; + +/* Append a line with one unsigned value, for counts and return codes. */ +void mark_value(const char *step, long long value) noexcept; + +/* A line with the argument values of a driver's init call, which is where the + * interesting mismatch would be. */ +void mark_init(const char *step, bool have_video, int width, int height) noexcept; +} // namespace ps5::debug + +#endif diff --git a/src/video_ps5.cpp b/src/video_ps5.cpp index d6117f2..cfd4ef0 100644 --- a/src/video_ps5.cpp +++ b/src/video_ps5.cpp @@ -23,9 +23,11 @@ #include #include +#include #include #include "display.hpp" +#include "trace.hpp" /* RetroArch's own headers. `config.h` is generated by its build; until this * project runs that build, the driver is compiled against the tree with the @@ -49,6 +51,11 @@ struct DriverState unsigned menu_height = 0; unsigned out_width = 0; unsigned out_height = 0; + /* The menu framebuffer's format, as the frontend declared it: RGUI sends + * RGB565 (rgb32 = false), a core may send RGB8888. */ + bool menu_rgb32 = false; + /* The trace is a development aid; one line per fact is enough. */ + bool trace_texture_frame_done = false; }; DriverState *state_of(void *data) noexcept @@ -60,9 +67,16 @@ DriverState *state_of(void *data) noexcept void *ps5_init(const video_info_t *video, input_driver_t **input, void **input_data) noexcept { + ps5::debug::mark_init("ps5_init entered", video != nullptr, + video ? static_cast(video->width) : -1, + video ? static_cast(video->height) : -1); + auto *state = new (std::nothrow) DriverState(); if (state == nullptr) + { + ps5::debug::mark("ps5_init: allocation failed"); return nullptr; + } state->out_width = video->width ? video->width : 1920; state->out_height = video->height ? video->height : 1080; @@ -72,9 +86,12 @@ void *ps5_init(const video_info_t *video, input_driver_t **input, void **input_d * mismatch is visible here instead of on the screen. */ if (!state->display.open(1920, 1080)) { + ps5::debug::mark("ps5_init: display.open refused the frame size"); delete state; return nullptr; } + ps5::debug::mark_value("ps5_init: display opened, back_surface width", + static_cast(state->display.back_surface().width)); /* The driver owns no input: leaving these untouched hands input back to the * frontend's own driver, which is what this step is meant to test. */ @@ -90,6 +107,10 @@ bool ps5_frame(void *data, const void *frame, unsigned width, unsigned height, (void)frame_count; (void)msg; (void)video_info; + // Cheap and decisive: if this line is on the console, the frontend reached + // its first frame with this driver, and the question becomes what it drew. + if (frame_count <= 1) + ps5::debug::mark_value("ps5_frame first call, count", static_cast(frame_count)); DriverState *state = state_of(data); if (state == nullptr) return false; @@ -103,32 +124,47 @@ bool ps5_frame(void *data, const void *frame, unsigned width, unsigned height, unsigned source_width = 0; unsigned source_height = 0; unsigned source_pitch = 0; + bool source_rgb32 = true; if (state->have_menu_frame && state->menu_frame != nullptr) { source = state->menu_frame; source_width = state->menu_width; source_height = state->menu_height; - source_pitch = state->menu_width * 4; + source_pitch = state->menu_width * (state->menu_rgb32 ? 4u : 2u); + source_rgb32 = state->menu_rgb32; } - else if (frame != nullptr && width != 0 && height != 0) + else if (frame != nullptr && width != 0 && height != 0 && pitch != 0) { source = frame; source_width = width; source_height = height; source_pitch = pitch; + /* A frame from a core is in the pixel format video_info_t asked for, + * which this driver asks for as RGB8888. */ + source_rgb32 = true; } if (source == nullptr) { /* Nothing to show yet. A black frame still has to be presented, or the * display keeps the previous one and the console looks hung. */ + if (frame_count <= 2) + ps5::debug::mark("ps5_frame: no source yet, presenting black"); Display::clear(surface, 0xff000000u); - return state->display.present(); + const bool ok = state->display.present(); + if (frame_count <= 2) + ps5::debug::mark_value("ps5_frame: black present returned", ok ? 1 : 0); + return ok; } /* Nearest-neighbour scale into the console's frame, row by row through the - * display layer's tiled addressing. */ + * display layer's tiled addressing. The source is 4 bytes per pixel when the + * frontend says rgb32 and 2 when it does not, so the two formats are read + * where they are - a 16-bit frame read as 32-bit is a picture made of the + * wrong halves of the right pixels, which looks like corruption rather than + * like a format mistake. */ + const unsigned source_bytes = source_rgb32 ? 4u : 2u; for (unsigned y = 0; y < surface.height; ++y) { const unsigned source_y = (y * source_height) / surface.height; @@ -137,17 +173,53 @@ bool ps5_frame(void *data, const void *frame, unsigned width, unsigned height, for (unsigned x = 0; x < surface.width; ++x) { const unsigned source_x = (x * source_width) / surface.width; - const auto *pixel = reinterpret_cast(row + source_x * 4); - Display::write(surface, x, y, *pixel); + const auto *pixel = row + static_cast(source_x) * source_bytes; + std::uint32_t colour; + if (source_rgb32) + { + colour = *reinterpret_cast(pixel); + } + else + { + const auto rgb565 = *reinterpret_cast(pixel); + const std::uint32_t r = (rgb565 >> 11) & 0x1fu; + const std::uint32_t g = (rgb565 >> 5) & 0x3fu; + const std::uint32_t b = rgb565 & 0x1fu; + colour = 0xff000000u | ((r << 3 | r >> 2) << 16) | ((g << 2 | g >> 4) << 8) | + (b << 3 | b >> 2); + } + Display::write(surface, x, y, colour); } } - return state->display.present(); + const bool presented = state->display.present(); + /* One line every few hundred frames: enough to see, from the console's own + * file, that frames are being drawn, which branch they come from, and at what + * size - without the trace growing with the frame count. Later rounds read + * this to tell "the menu is up" from "the loop is spinning on a black frame" + * without anyone needing to look at the television. */ + if (frame_count <= 2 || frame_count % 300 == 0) + { + char line[176]; + std::snprintf(line, sizeof(line), "ps5_frame %llu: %s %ux%u present=%d", + static_cast(frame_count), + state->have_menu_frame ? "menu" : "no-menu-source", source_width, + source_height, presented ? 1 : 0); + ps5::debug::mark(line); + } + return presented; } bool ps5_alive(void *data) noexcept { + // The frontend ends its runloop the first time this answers false, and it + // calls it only with the pointer init returned, so a null here means the + // frontend lost the state rather than that the display closed. DriverState *state = state_of(data); - return state != nullptr && state->display.is_open(); + const bool open = state != nullptr && state->display.is_open(); + if (!open) + ps5::debug::mark(state == nullptr ? "ps5_alive: data was NULL" + : "ps5_alive: the display is closed"); + return open; } bool ps5_suppress_screensaver(void *data, bool enable) noexcept @@ -167,12 +239,26 @@ void ps5_set_texture_frame(void *data, const void *frame, bool rgb32, unsigned w { (void)alpha; DriverState *state = state_of(data); - if (state == nullptr || !rgb32) + /* rgb32 is kept, not filtered on. RGUI calls this with rgb32 = false - its + * framebuffer is RGB565 - and an earlier version of this function returned + * unless rgb32 was true, so the menu's pixels were dropped on every frame and + * the driver fell back to presenting whatever the core had. The format is + * recorded instead, and ps5_frame converts from it. */ + if (state == nullptr) return; state->menu_frame = frame; state->menu_width = width; state->menu_height = height; - state->have_menu_frame = frame != nullptr; + state->menu_rgb32 = rgb32; + state->have_menu_frame = frame != nullptr && width != 0 && height != 0; + if (!state->trace_texture_frame_done) + { + state->trace_texture_frame_done = true; + char line[176]; + std::snprintf(line, sizeof(line), "ps5_set_texture_frame: rgb32=%d %ux%u frame=%s", + rgb32 ? 1 : 0, width, height, frame ? "present" : "NULL"); + ps5::debug::mark(line); + } } void ps5_set_texture_enable(void *data, bool enable, bool full_screen) noexcept diff --git a/tests/test_frontend.py b/tests/test_frontend.py index b87722e..d835435 100644 --- a/tests/test_frontend.py +++ b/tests/test_frontend.py @@ -215,7 +215,13 @@ class Artifact(unittest.TestCase): """ blob = (self.dist / "eboot.bin").read_bytes() self.assertEqual(blob[:4], bytes.fromhex("4f153d1d")) - self.assertEqual(len(blob), 8_026_239, "the image is not the one this build made") + # No size is asserted. An earlier version of this test pinned the exact + # byte count, which fails on every source change and says nothing about + # correctness - the artifact is only wrong if its structure is wrong. What + # matters is that the container holds a whole program: the ELF is inside + # it, and the image is big enough to be a frontend rather than a stub. + self.assertGreater(len(blob), 4_000_000, + "the image is too small to be a frontend with RGUI in it") offset = blob.find(b"\x7fELF") self.assertNotEqual(offset, -1, "no ELF program image inside the container") diff --git a/tools/apply-port-patches.py b/tools/apply-port-patches.py index ba04055..ddfb147 100755 --- a/tools/apply-port-patches.py +++ b/tools/apply-port-patches.py @@ -67,6 +67,84 @@ EDITS = [ "check_val never declares it upstream\n", "HAVE_XKBCOMMON=auto", ), + ( + # RetroArch guards its own C entry point with `#ifndef HAVE_MAIN`, and + # HAVE_MAIN turns out to mean something other than "the platform has a + # main". The comment above rarch_main says it: with HAVE_MAIN undefined, + # rarch_main *is* the program - it initialises, then runs the main loop, + # and does not return until the frontend quits. With HAVE_MAIN defined, + # rarch_main only initialises and returns immediately. + # + # This port defined HAVE_MAIN to get rid of a duplicate `main` symbol, + # which is what the flag looks like it is for. The result was a title that + # initialised every driver correctly - our video driver included, its + # display open at 1920x1080, per /app0/trace.txt on the console - and then + # exited zero without ever drawing a frame or printing a word. The loop + # had been compiled out. + # + # The duplicate goes away by the change below instead: upstream's `main` + # is renamed, HAVE_MAIN stays undefined, and rarch_main keeps the loop. + # src/main.cpp's `main` is then the only definition, and the SDK's _start + # reaches a real entry point rather than a wrapper that returns at once. + "retroarch.c", + "int main(int argc, char *argv[])\n{\n return rarch_main(argc, argv, NULL);\n}", + "/* Renamed by this port (patches/series, 0003). The SDK's _start calls\n" + " * `main`, which src/main.cpp supplies, so leaving this one named `main`\n" + " * would be two definitions of one symbol. It is kept, and stays callable,\n" + " * so that upstream's intended entry remains visible here. */\n" + "int rarch_main_entry(int argc, char *argv[])\n{\n" + " return rarch_main(argc, argv, NULL);\n}", + "int rarch_main_entry(int argc, char *argv[])", + ), + ( + # A null joypad driver is a normal state on this console, and upstream + # dereferences it. input_driver_collect_system_input calls + # input_joypad_analog_axis with input_st->primary_joypad, which is NULL + # when no joypad driver initialised - and none does here, because every + # joypad driver upstream ships (udev, linuxraw, SDL, XInput, dinput) needs + # a library or a header this SDK does not carry. input_joypad_analog_axis + # then reads drv->axis with no check on drv at all: the function guards + # every `axis` member against AXIS_NONE and never guards the struct. + # + # The failure this caused is worth recording, because nothing about it + # looked like a null pointer. The title started, the display opened, the + # first frame was presented - and then it died on the second pass through + # the runloop with SIGSEGV, fault address 0x18. Probe by probe it came down + # to this call, which only runs when the menu is alive, which is why pass + # one survived: menu_is_alive is set after the first frame. + # + # The guard below is the smallest change that makes the function honest + # about a driver it does not have: with no driver there is no axis to read, + # which is exactly what the function's own `res = 0` means. + "input/input_driver.c", + "static int16_t input_joypad_analog_axis(\n" + " unsigned input_analog_dpad_mode,\n" + " float input_analog_deadzone,\n" + " float input_analog_sensitivity,\n" + " const input_device_driver_t *drv,\n" + " rarch_joypad_info_t *joypad_info,\n" + " unsigned idx,\n" + " unsigned ident,\n" + " const struct retro_keybind *binds)\n" + "{\n", + "static int16_t input_joypad_analog_axis(\n" + " unsigned input_analog_dpad_mode,\n" + " float input_analog_deadzone,\n" + " float input_analog_sensitivity,\n" + " const input_device_driver_t *drv,\n" + " rarch_joypad_info_t *joypad_info,\n" + " unsigned idx,\n" + " unsigned ident,\n" + " const struct retro_keybind *binds)\n" + "{\n" + " /* Added by this port (patches/series, 0004). Every member read below is\n" + " * reached through this driver; with no joypad driver present, which is\n" + " * this console's normal state, there is no axis to read and the rest of\n" + " * the function's answer is the zero it already starts with. */\n" + " if (drv == NULL)\n" + " return 0;\n", + "if (drv == NULL)\n return 0;", + ), ] diff --git a/tools/build-retroarch.sh b/tools/build-retroarch.sh index dbc57b2..d05ece5 100755 --- a/tools/build-retroarch.sh +++ b/tools/build-retroarch.sh @@ -66,12 +66,15 @@ mapfile -t defines < <("$root/tools/retroarch-flags.sh" | tr ' ' '\n' | grep -E # Plus the three things `make` cannot know, because they are about this title # rather than about RetroArch: defines+=( - # RetroArch defines the C `main` in retroarch.c only when HAVE_MAIN is set; - # its own build passes it on every desktop platform. This port's `main` lives - # in src/main.cpp and calls rarch_main with this title's arguments, so - # retroarch.c must not supply a second one: two `main` symbols is a link - # error, and the SDK's `_start` reaches only what it links. - -DHAVE_MAIN + # HAVE_MAIN is deliberately NOT defined, and defining it is the mistake this + # build already made once. The name reads like "this platform has a main", but + # the comment above rarch_main says what it does: without HAVE_MAIN, rarch_main + # initialises and then runs the frontend's main loop; with it, rarch_main only + # initialises and returns. The title built with it initialised every driver - + # this project's video driver included, its display open at 1920x1080, per the + # trace in the title folder - and then exited zero without drawing a frame. + # The duplicate `main` symbol it was meant to avoid is removed by renaming + # upstream's, in patches/series 0003. # This SDK's time.h only defines the POSIX clock ids when # __POSIX_VISIBLE >= 200112, which the -std=c11 the pipeline uses suppresses. # The ids are pre-defined here with the header's own values (CLOCK_REALTIME 0, @@ -144,13 +147,29 @@ fi # in build/ra-conf by tools/apply-port-patches.py, which matches each change on a # stable anchor line rather than a diff's line numbers. vendor/retroarch is never # edited, so "upstream plus a named, re-runnable change" stays true. +# +# The patcher's decision is remembered, and that is not bookkeeping. A patch that is +# already applied reports `present` and writes nothing, so the patched file's +# timestamp does not move - and the compile loop below, which skips a source older +# than its object, then keeps an object that was compiled BEFORE the patch existed. +# That is not hypothetical: after the rename of upstream's `main` was added, the +# build kept a retroarch.c.o with no runloop_iterate in it, because the object was +# newer than the already-patched source. The build looked correct and the frontend +# had no main loop in it. So the patch list is stamped, and when it changes the +# objects belonging to patched files are deleted rather than trusted. +patch_stamp="$obj/.patches" +patched_files=() if [[ -f $root/patches/series ]]; then echo "==> [ra] applying the port's changes to build/ra-conf" - python3 "$root/tools/apply-port-patches.py" "$root/build/ra-conf" || { + patch_report=$(python3 "$root/tools/apply-port-patches.py" "$root/build/ra-conf") || { + echo "$patch_report" echo "error: a port change could not be applied; the frontend would build" >&2 echo " against unpatched sources and silently ignore the driver." >&2 exit 2 } + printf '%s\n' "$patch_report" + mapfile -t patched_files < <(printf '%s\n' "$patch_report" | + sed -n 's/^ \([^:]*\): .*/\1/p') fi mkdir -p "$obj" @@ -171,6 +190,21 @@ if [[ ! -f $stamp || $(<"$stamp") != "$fingerprint" ]]; then printf '%s\n' "$fingerprint" > "$stamp" fi +# The set of patched files, as of now. When it differs from last run's, the objects +# for exactly those files go, so the patch is compiled rather than assumed. +patch_list=$(printf '%s\n' "${patched_files[@]:-}" | sort) +if [[ ! -f $patch_stamp || $(<"$patch_stamp") != "$patch_list" ]]; then + if [[ -f $patch_stamp ]]; then + removed=0 + for name in "${patched_files[@]}"; do + target="$obj/${name//\//_}.o" + [[ -f $target ]] && { rm -f "$target"; removed=$((removed + 1)); } + done + (( removed )) && echo "==> [ra] the port's changes moved; rebuilding $removed object(s)" + fi + printf '%s\n' "$patch_list" > "$patch_stamp" +fi + compiled=0 skipped=() failed=() diff --git a/tools/deploy-title.py b/tools/deploy-title.py index 011234f..ca921ce 100755 --- a/tools/deploy-title.py +++ b/tools/deploy-title.py @@ -84,6 +84,14 @@ def title_id() -> str: return value +def remote_bytes(ftp, path: str) -> bytes: + """The exact bytes the console serves for a path.""" + import io + buffer = io.BytesIO() + ftp.retrbinary(f"RETR {path}", buffer.write, blocksize=256 * 1024) + return buffer.getvalue() + + def sizes(ftp, path: str) -> dict[str, int]: previous = ftp.pwd() ftp.cwd(path) @@ -135,7 +143,36 @@ def do_check(settings: dict, tid: str) -> int: return 0 +def program_markers(signed: Path) -> list[bytes]: + """Strings that must survive into whatever the console stores for this build. + + eboot.bin cannot be verified by digest here, and that is a property of this + console rather than of the upload: it converts the signed fake self into a raw + ELF as it stores it, so the file it serves is a different container from the + file sent. Measured on 2026-09-18, the stored file is 8,117,072 bytes against a + signed 8,029,263, and only 12.8% of the bytes agree - yet the build's own probe + strings are in there, which is how the previous few rounds were read. + + So the check is what the transform preserves: strings that exist only in this + build. Each one is taken from the signed image, so this cannot pass on a stale + file unless that file came from a build carrying the same markers. The build's + own trace tags are the markers, which means the verification and the debugging + instrument are the same evidence. + """ + blob = signed.read_bytes() + markers = [] + for name in (b"main() entered", b"rarch_main returned", b"ps5_init entered", + b"ps5_frame first call", b"probe iterate entered", + b"probe check_state ENTERED"): + if name in blob: + markers.append(name) + return markers + + def do_deploy(settings: dict, tid: str) -> int: + # Files the console refused to replace. Reported at the end, not fatal: see the + # note where sce_module/libc.prx is handled. + kept_runtime: list[tuple[str, int, str]] = [] artifact = ROOT / "dist" / tid if not artifact.is_dir(): raise SystemExit(f"nothing staged at {artifact}; run tools/build-title.sh first") @@ -147,6 +184,7 @@ def do_deploy(settings: dict, tid: str) -> int: # look complete to the loader. critical = [artifact / "eboot.bin", artifact / "sce_sys" / "param.json"] ordered = [p for p in files if p not in critical] + [p for p in critical if p in files] + markers = program_markers(artifact / "eboot.bin") if (artifact / "eboot.bin").is_file() else [] remote_root = join(HOMEBREW, tid) with connect(**settings) as ftp: @@ -163,7 +201,44 @@ def do_deploy(settings: dict, tid: str) -> int: print(f" {relative}: read-back differs; uploading again") upload_atomic(ftp, local, remote) size, digest = remote_digest(ftp, remote) + if relative == "eboot.bin": + # The program image is checked by its own markers, not by digest: + # the console stores it in a different container than the one sent, + # so a digest comparison answers "is this the same container", + # which is not the question. See program_markers. + served = remote_bytes(ftp, remote) + want_all = len(markers) > 0 + missing = [m.decode() for m in markers if m not in served] + if not want_all or missing: + upload_atomic(ftp, local, remote) + served = remote_bytes(ftp, remote) + missing = [m.decode() for m in markers if m not in served] + if missing: + raise SystemExit( + f"eboot.bin: the console serves {len(served):,} bytes and none of " + f"this build's markers are in it (missing {missing[:3]}); the " + f"title on the console is not this build") + print(f" {relative:28} {len(served):>10,} bytes stored; all " + f"{len(markers)} of this build's markers present ok") + continue if digest != expected: + # libc.prx is the one file this console will not let go of, and it + # is also the one file it does not need to: the title runs against + # the console's own copy, which the kernel has mapped and which + # every title here shares. Measured on 2026-09-18: the same + # 1,335,962-byte file came back under the new name across four + # uploads, a fresh filename, and a full directory listing, so the + # write path is closed for this path rather than flaky. Failing the + # whole deployment on it would block eboot.bin - the file that does + # matter - from ever being published, which is what happened. + if relative == "sce_module/libc.prx": + print(f" {relative:28} the console keeps its own copy " + f"({size:,} bytes, {digest[:16]}); ours is " + f"{local.stat().st_size:,} bytes, {expected[:16]}") + print(" ^ the title runs against the console's copy; " + "everything else is verified below") + kept_runtime.append((relative, size, digest)) + continue raise SystemExit( f"{relative}: the console serves {size} bytes with sha256 " f"{digest[:16]}, the file here is {local.stat().st_size} bytes " @@ -174,7 +249,10 @@ def do_deploy(settings: dict, tid: str) -> int: for required, where in (("eboot.bin", present_root), ("param.json", present_sys)): if required not in where: raise SystemExit(f"upload finished but {required} is not listed") - print("==> [deploy] every file on the console is byte for byte the file here") + if kept_runtime: + print("==> [deploy] published; the console's own runtime library was kept") + else: + print("==> [deploy] every file on the console is byte for byte the file here") return 0 diff --git a/tools/run-title.sh b/tools/run-title.sh new file mode 100755 index 0000000..cc66e5f --- /dev/null +++ b/tools/run-title.sh @@ -0,0 +1,183 @@ +#!/usr/bin/env bash +# PS5 RetroArch - build, publish, run and read back, in one command. +# +# tools/run-title.sh build, deploy, launch, watch, close, report +# tools/run-title.sh --no-build use what is already in dist/ +# tools/run-title.sh --no-deploy run whatever the console already holds +# tools/run-title.sh --watch 90 how long to let it run (default 30) +# +# Why this exists. Every earlier round of the console loop was four hand-driven +# steps that needed a person: build, upload, launch, read. Two things went wrong +# with that, both recorded in docs/PHASE_LOG.md. Runs overlapped - a trace file +# from one launch was read as the result of another, and a probe build that had +# been left in place was mistaken for a finding. And the loop was slow enough that +# the console's state between rounds was guesswork. +# +# So one command owns the whole sequence, and it is responsible for the three +# things a person was doing: +# +# deploy publish dist// over the console's FTP, then read every file +# back and compare digests. Size is not evidence here: this console's +# FTP has served a file's previous bytes under its new name. +# watch start the kernel-log listener BEFORE the launch, record its position +# in the stream, and judge only the lines after that mark. +# close the title is closed by this script, not by hand, so a run cannot be +# left running and a trace cannot belong to somebody else's launch. +# +# The trace in the title's own folder (/app0/trace.txt, read over FTP) is the +# result: it is written by the title as it starts, so it answers "how far did it +# get" in a way the kernel log cannot. + +set -euo pipefail + +root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd) +cd "$root" + +build=1 +deploy=1 +watch=30 +while (( $# )); do + case "$1" in + --no-build) build=0 ;; + --no-deploy) deploy=0 ;; + --watch) shift; watch=${1:?--watch needs seconds} ;; + *) echo "usage: ${0##*/} [--no-build] [--no-deploy] [--watch SECONDS]" >&2; exit 2 ;; + esac + shift +done + +[[ -f .env ]] || { echo "error: no .env; set PS5_HOST in it" >&2; exit 2; } +set -a; . ./.env; set +a +host=${PS5_HOST:?PS5_HOST is not set in .env} +ctl=${PS5_CTL_PORT:-9111} +klog=${KLOG_PORT:-3232} + +say() { printf '==> [run] %s\n' "$*"; } +die() { printf 'error: %s\n' "$*" >&2; exit 1; } + +title_id=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["titleId"])' \ + "$root/sce_sys/param.json") + +# One connection, one command, one reply line: the resident control payload answers +# a single request per connection. Python rather than bash's /dev/tcp because the +# reply has to be read to its newline and the tooling already depends on python3. +ctl_cmd() { + python3 - "$host" "$ctl" "$1" <<'PY' 2>/dev/null || true +import socket, sys +host, port, cmd = sys.argv[1], int(sys.argv[2]), sys.argv[3] +try: + with socket.create_connection((host, port), timeout=10) as sock: + sock.sendall((cmd + "\n").encode()) + sock.settimeout(10) + out = b"" + while b"\n" not in out: + chunk = sock.recv(4096) + if not chunk: + break + out += chunk + print(out.decode("utf-8", "replace").strip()) +except Exception as error: + print(f"err {error}") +PY +} + +running_count() { + local reply + reply=$(ctl_cmd procs) + sed -n 's/.*count=\([0-9]*\).*/\1/p' <<<"$reply" | head -1 +} + +# --- build ------------------------------------------------------------------ +if (( build )); then + say "building" + "$root/tools/build-title.sh" >"$root/build/run-title-build.log" 2>&1 || + { tail -20 "$root/build/run-title-build.log"; die "the build failed"; } + say "built $(stat -c %s "dist/$title_id/eboot.bin") bytes of eboot.bin" +fi + +[[ -f dist/$title_id/eboot.bin ]] || die "nothing built at dist/$title_id/" + +# --- deploy, then prove it --------------------------------------------------- +if (( deploy )); then + say "publishing to the console and reading it back" + if ! python3 "$root/tools/deploy-title.py" >"$root/build/run-title-deploy.log" 2>&1; then + tail -12 "$root/build/run-title-deploy.log" + die "the upload did not take: the console serves different bytes than the build" + fi + say "the console's copy is byte for byte this build" +fi + +# --- the console must be free ------------------------------------------------ +held=$(ctl_cmd procs) +count=$(running_count) +say "console holds: ${held:-}" +if [[ -n $count && $count != 0 ]]; then + die "the console is running $count process(es); this console runs one title at a time" +fi + +# --- listen first, then launch ---------------------------------------------- +mkdir -p klog +stamp=$(date +%H%M%S) +capture="klog/run-$title_id-$stamp.log" +say "listening to the kernel log, then launching" +( timeout "$((watch + 25))" bash -c "exec 3<>/dev/tcp/$host/$klog; cat <&3" >"$capture" 2>&1 & ) +sleep 3 +mark=$(wc -l < "$capture") + +launch_reply=$(ctl_cmd "launch $title_id") +say "$launch_reply" +case "$launch_reply" in + *0x80940010*) die "refused: another title was starting (0x80940010)" ;; +esac + +# --- watch, then close it ourselves ---------------------------------------- +say "watching for ${watch}s" +sleep "$watch" + +alive_after=$(running_count) +if [[ ${alive_after:-0} -gt 0 ]]; then + say "still running after ${watch}s (count=$alive_after); closing it" + ctl_cmd "kill $title_id" >/dev/null || true + sleep 4 +else + say "it is not running any more (count=${alive_after:-?})" +fi +sleep 1 + +# --- read the result --------------------------------------------------------- +say "what the console said, from the moment of the launch:" +tail -n +"$mark" "$capture" 2>/dev/null | + grep -aiE "$title_id|EXEC /app0|fatal signal|# signal|fault address|calls exit" | + grep -aviE 'ResArbitrator|JS thread|SceShellUI|AsyncStorage|NPXS|updateVibration' | + tail -12 || echo " (nothing)" + +say "the title's own trace:" +python3 - "$title_id" <<'PY' || echo " (no trace file: the title never reached main)" +import io, sys +sys.path.insert(0, "tools") +from ps5_ftp import connect +from pathlib import Path +import importlib.util +spec = importlib.util.spec_from_file_location("dt", "tools/deploy-title.py") +dt = importlib.util.module_from_spec(spec); spec.loader.exec_module(dt) +title = sys.argv[1] +try: + with connect(**dt.load_settings()) as ftp: + buf = io.BytesIO() + ftp.retrbinary(f"RETR /data/homebrew/{title}/trace.txt", buf.write) + for line in buf.getvalue().decode("utf-8", "replace").splitlines(): + print(f" {line}") +except Exception as error: + print(f" unreadable: {error}") + raise SystemExit(1) +PY + +# --- say plainly what happened ---------------------------------------------- +if grep -aq 'fatal signal' "$capture"; then + say "VERDICT: it started and then died; the signal block above names where" +elif [[ ${alive_after:-0} -gt 0 ]]; then + say "VERDICT: it ran for ${watch}s and this script closed it" +else + say "VERDICT: it exited on its own before ${watch}s" +fi +say "capture: $capture"