Link the Vulkan driver into the title, and resolve the 24 symbols that stopped it

../PS5_Vulkan's driver stops being a library RetroArch loads at run time and
becomes ordinary symbols in eboot.bin, because a PS5 title cannot dlopen one
(9d29a1f measured ENOEXEC, ENOENT, NULL and ESRCH). This is the commit where
that link completes.

The first link failed on 24 undefined symbols, which is three problems rather
than one. The count is only knowable because the link line now carries
--error-limit=0; before that lld stopped at twenty.

 - Four __eh_frame_* boundaries. The driver's shader compiler is C++ and links
   the SDK's libunwind, which finds unwind tables through these boundary symbols
   rather than dl_iterate_phdr. tooling/native/ps5-pie.ld provides the same four
   ../PS5_Vulkan/tooling/psbc/ps5-pie-unwind.ld defines, in this project's own
   copy of the layout.
 - Eight Mesa utility symbols (u_thread_create, u_thread_setname,
   util_barrier_init/destroy/wait, util_thread_get_time_nano,
   os_create_anonymous_file, os_read_file). That project's PS5 object list
   filters src/util/{u_thread,anon_file,os_file}.c out of the compiler archive,
   and its own libvulkan.so.1 links only because a shared object may leave
   symbols undefined. tools/build-mesa-util.sh compiles Mesa's own definitions
   with that project's PS5 configuration, because u_thread.c's util_barrier
   layout has to match the u_queue.ps5.o already inside the archive.
 - Twelve sceAgc* entry points. The console provides libSceAgc and
   libSceAgcDriver and the SDK stubs neither, so they are declared for the
   converter to record as imports. They live in tooling/ps5-stubs/ rather than
   vendor/, which is not in the diff.

Evidence:

  bash tools/build-title.sh   exit 0, no undefined symbols; the artifact grew
                              from 8,225,706 to 29,859,722 bytes, which is the
                              driver and its shader compiler
  bash tools/verify.sh        PASS (format unit build integration evidence)
  unittest discover -s tests  18 tests, two new: every sceAgc* the three driver
                              archives reference is declared by the stubs (20
                              referenced, 21 declared, none missing), and the
                              linked build/llvm-pie.elf defines the four
                              __eh_frame_* symbols

Not proven: the driver has not been on the console since it was linked.
This commit is contained in:
Mihawk committed 2026-09-18 22:50:12 -04:00
1 parent 9d29a1f6a5
commit be552f1460
9 files changed
+584 -173

No files matched your search

+79 -170
View File
@@ -9,196 +9,105 @@ _Updated: 2026-09-18_
## Now
**The pad works and the menu is live.** `bash tools/run-title.sh --watch 20` with the
console owner working the pad: the title runs, the driver reads the DualSense, and
the menu answers. From the title's own trace on the shipping build:
**The Vulkan driver is linked into the title, and the link completes.** This is the
pivot the project was for: `../PS5_Vulkan`'s driver is no longer something RetroArch
dlopens - a PS5 title cannot, which that project measured - but ordinary symbols in
`eboot.bin`. `bash tools/build-title.sh` reaches a clean link and the artifact grew
from 8,225,706 to 29,859,722 bytes, which is the driver and its shader compiler.
input: pad opened, user=515310723 handle=51119872
input: press, pad=0x00004000 retropad=0x00000001 CROSS -> RetroPad B
input: press, pad=0x00002000 retropad=0x00000100 CIRCLE -> RetroPad A
menu: framebuffer commit 2 is a new picture (2 of 2 changed so far)
ps5_frame 600: menu commits=76 changes=14 presented=yes
The first link failed on **24 undefined symbols**, and they were three different
problems rather than one:
Both goals of the input round are met. Input works, with CIRCLE confirming and CROSS
cancelling as a PlayStation player expects. And the picture is **not** one frozen
frame: 76 framebuffer commits, 14 of them a different picture, against 1 and 1 in
every earlier run. The image was never frozen - the menu had nothing to redraw for,
which is what input supplies.
1. **Four `__eh_frame_*` boundaries.** The shader compiler is C++ and links the SDK's
libunwind, which finds unwind tables through those symbols, not through
`dl_iterate_phdr`. `tooling/native/ps5-pie.ld` now provides them, the same four
`../PS5_Vulkan/tooling/psbc/ps5-pie-unwind.ld` defines.
2. **Eight Mesa utility symbols** (`u_thread_create`, `u_thread_setname`,
`util_barrier_init/destroy/wait`, `util_thread_get_time_nano`,
`os_create_anonymous_file`, `os_read_file`). Their PS5 object list filters
`src/util/{u_thread,anon_file,os_file}.c` out of the compiler archive, and its own
`libvulkan.so.1` only links because a *shared* object may leave symbols undefined.
An executable may not. `tools/build-mesa-util.sh` compiles those three sources with
their PS5 configuration, because `u_thread.c`'s `util_barrier` layout has to match
the `u_queue.ps5.o` already inside the archive.
3. **Twelve `sceAgc*` entry points.** The console provides libSceAgc and
libSceAgcDriver and the SDK stubs neither, so they are declared in
`tooling/ps5-stubs/` for the converter to record as imports.
**Why it never worked before: one comparison.** `ps5_input_init` had never run.
`video_driver_init_input` opens with `if (*input) return true;`, which upstream means
for a video driver that pre-initialised an input driver of its own, and
`video_driver_init_internal` assigns `tmp = current_driver` *before* calling
`video_driver_find_driver` - so after the pre-initialisation pass selects a driver,
`tmp` **is** that selection. The early return fired, the wrap below was dead code,
`current_data` stayed NULL and every button read answered 0. Measured:
`probe INV: entered tmp=ba41e0 *input=ba41e0 configured="ps5"`. `patches/series` 0009
fixes it with `if (*input != NULL && *input == tmp)`.
**The AGC stubs moved out of `vendor/`.** `vendor/` is not in the diff, so the
declarations the link needs would have been lost with the next checkout. They are
tracked under `tooling/ps5-stubs/` now, which is also where `tools/build.sh` reads
them.
**A list of things that were NOT the reason**, each measured: the driver's code, its
registration, its button map, `HAVE_TEST_DRIVERS` (necessary but not sufficient), the
compiled input default 0008 (the driver was named; naming was never the problem), and
the controller-port guard 0007. Three rounds of probes placed *around*
`video_driver_init_input` could not see it; the probes inside it found it in one run.
**Not yet run on the console.** What is proven here is that the title links with the
driver embedded and that every gate passes. Whether the driver initialises on the
console is the next run, and `--log-file=/app0/retroarch.log` is already in argv for
it.
## Next
1. **The config file is still not read.** Content loading discards the title's `-c`,
and the fix for that (0006) is parked in `parked/config-path.patch.py` because
reading the config crashes the launch inside `command_event`. That crash is
narrowed to the dispatch and is the next task. The input driver does not depend on
it any more - `0008` names it as the compiled default - so the pad works either
way, which means this is now about configurability and the file logger, not about
whether RetroArch runs.
2. **Then measurement before more features.** The pad cannot be verified unattended:
the trace reports presses, but nothing in the pipeline presses a button. Deciding
how (a recorded input file, or the console owner) is worth doing explicitly.
3. **Then the menu's size.** RGUI renders 320x240 into a 1920x1080 frame and the
driver scales it up whole.
## Working notes
- **`bash tools/run-title.sh` is the only way to run the title.** It owns the whole
sequence and prints the title's own trace. Do not hand-deploy.
- **Runtime probes are kept: `tools/apply-runtime-probes.py`.** A rebuild wipes any
probe written into `build/ra-conf`, so they live in that script now - applied with
one command, `--revert` to remove, `--list` to see what each one is for. They are
not in the shipping build; `tools/build-title.sh` does not call it.
- **Never grep a compiled object for a patch's comment text**, and remember
`strings(1)` ignores anything shorter than four characters. Both mistakes made
correctly-built fixes look missing. The instruction stream (`objdump -dr`) is the
only reliable evidence that a code change compiled.
- **Place probes at statements, never inside a multi-line call or an `#if` block.**
Two rounds were lost to a split brace (`undefined symbol: rarch_main`) and to a
probe inside a declaration list.
- **Do not park a patch by slicing the script by text.** It emptied the parked file
and desynced the script from the tree; `git checkout` recovered it. Edit the
script, then verify the parked file is non-empty.
- **`/app0/trace.txt` is append-only** and grows across runs; read the tail or slice
from the last `main() entered`.
- **`src/` must be compiled with the frontend's `-D` flags**, which
`tools/build-title.sh` does by asking `tools/retroarch-flags.sh`; a bare
`make app` produces a driver table the frontend reads at the wrong offsets.
- **Compile from `build/ra-conf`, never from `vendor/retroarch`**; upstream is never
edited and `patches/series` is the only record of this port's changes.
- **`-DHAVE_MAIN` must never be defined**: it compiles the frontend's main loop out.
- The console's address and credentials come from the ignored `.env`; console
captures stay in the ignored `klog/` tree.
## Last verified
| Check | Result |
| --- | --- |
| `bash tools/run-title.sh --watch 20` | PASS: pad read (`input: pad opened`), 6 button mappings correct, `menu commits=76 changes=14`, no fatal signal |
| `bash tools/verify.sh` (all gates) | PASS (format unit build integration evidence) |
| `python3 -m unittest discover -s tests` | PASS: 13 tests, including the input driver's registration and button map |
| `python3 tools/apply-runtime-probes.py build/ra-conf` | PASS: all 7 probes apply; `--revert` removes them |
| Config path (0006) | **PARKED**: reading the config still crashes in `command_event`; see `parked/config-path.patch.py` |
## Open findings
- The config-path crash is the last thing between this title and a readable
`retroarch.cfg`, and with it the file logger, which is the instrument the next
debugging round will want.
- The console's FTP will not replace `sce_module/libc.prx`; the title runs against
the console's copy. A release has to solve this.
- RGUI's assets: the fonts are bundled under `assets/rgui/font/`; whether the rest of
RetroArch's asset tree should ship is undecided.
- Pad input cannot be verified unattended yet: nothing in the pipeline presses a
button.
## Working notes
- **`bash tools/run-title.sh` is the only way to run the title.** It owns the whole
sequence and prints the title's own trace. Do not hand-deploy.
- **Trace files are append-only and grow without bound.** One session took
`/app0/trace.txt` past 70,000 lines. Read the tail, or slice from the last
`main() entered`.
- **A probe is not gone until `build/ra/obj` is deleted too.** Reconfiguring alone
reuses stale objects and ships old probes into `eboot.bin`.
- **Do not patch the configured tree with multi-line scripted replacements.** Two
rounds were lost to a replaced brace that produced
`undefined symbol: input_config_reset` at link time. Single-line insertions,
verified by brace count, are safe; blocks are not.
- **The console's signal block prints `rip`.** `rip=0` is a call through a null
function pointer, not a bad data read, and reading it first saves a round.
- **`src/` must be compiled with the frontend's `-D` flags**, which
`tools/build-title.sh` does by asking `tools/retroarch-flags.sh`; a bare
`make app` produces a driver table the frontend reads at the wrong offsets.
- **Compile from `build/ra-conf`, never from `vendor/retroarch`**; upstream is never
edited and `patches/series` is the only record of this port's changes.
- **`-DHAVE_MAIN` must never be defined**: it compiles the frontend's main loop out.
- The console's address and credentials come from the ignored `.env`; console
captures stay in the ignored `klog/` tree.
## Last verified
| Check | Result |
| --- | --- |
| `bash tools/run-title.sh --watch 12` | PASS: title runs, RGUI menu visible, no fatal signal, closed by the script |
| `bash tools/verify.sh` (all gates) | PASS (format unit build integration evidence) |
| `python3 -m unittest discover -s tests` | PASS: 13 tests, including the input driver's registration and button map |
| `nm -S build/obj/src_input_ps5.cpp.o` | PASS: `input_ps5` is 0x58 bytes, the frontend's `sizeof(input_driver_t)` |
| `nm -u build/ra/obj/input_input_driver.c.o` | PASS: the frontend references `input_ps5` |
| Input driver selected on the console | **FAILS**: `SIGSEGV`, `rip=0`, before `ps5_input_init`; parked |
## Open findings
- The config-path crash above is the blocker for input, and therefore for the
frozen-frame question. It happens with any input driver selected, so it is the
config being read - not the driver - that exposes it.
- The console's FTP will not replace `sce_module/libc.prx`; the title runs against
the console's copy. A release has to solve this.
- RGUI's assets: the fonts are bundled under `assets/rgui/font/`; whether the rest
of RetroArch's asset tree should ship is undecided.
- A pad-driven build cannot be verified unattended yet: the trace reports the first
press, but nothing in the pipeline exercises a button.
1. **Run it.** `bash tools/run-title.sh --watch 20`, then read `/app0/retroarch.log`
and `/app0/trace.txt`. The three answers to look for: does `video_vulkan` come up,
does it find a device, and does the picture reach the screen.
2. **The config file is still not read.** Content loading discards the title's `-c`,
and the fix (0006) is parked in `parked/config-path.patch.py` because reading the
config crashes the launch inside `command_event`. The file logger above is inert
until this is fixed, so on the Vulkan run the title's own trace is still the only
instrument.
3. **Then the menu's size.** RGUI renders 320x240 into a 1920x1080 frame.
## Working notes
- **`bash tools/run-title.sh` is the only way to run the title.** It owns the whole
sequence, including starting the klog listener before the launch and closing the
title itself, and it prints the title's own trace. Do not hand-deploy.
- **Nothing this project's `src/` compiles may be built without the frontend's
defines.** `tools/build-title.sh` is the only correct way to build the title: it
asks `tools/retroarch-flags.sh` for the flags the archive uses and passes them
through `APP_DEFINITIONS`. A bare `make app` produces a driver table the frontend
reads at the wrong offsets, and the only symptom is a menu that never appears.
- **A probe is not gone until the object is.** Deleting `build/ra-conf` and
reconfiguring is not enough: `tools/build-retroarch.sh` reuses
`build/ra/obj/*.o` and will archive a stale object carrying yesterday's probes
into `eboot.bin`. Delete `build/ra/obj` as well when a probe must really go.
- **`/app0/trace.txt` is append-only and is never rotated.** Every run adds to it;
one session took it to 72,904 lines. Read the tail, or slice from the last
`main() entered`, rather than reading it whole.
- **Compile from `build/ra-conf`, never from `vendor/retroarch`.** The configured
tree is rebuilt from upstream plus the four patches in `patches/series`; upstream
is never edited.
- **`vendor/` is not in the diff.** Anything a build needs from it must either be
regenerable or live in a tracked directory; `tooling/ps5-stubs/` is the second case.
- **A link that fails on twenty symbols stops there.** `--error-limit=0` is on the
link line, and it turned three rounds of guesswork into one list of 24.
- **Runtime probes are kept: `tools/apply-runtime-probes.py`.** A rebuild wipes any
probe written into `build/ra-conf`, so they live in that script - applied with one
command, `--revert` to remove, `--list` to see what each one is for. They are not in
the shipping build; `tools/build-title.sh` does not call it.
- **Never grep a compiled object for a patch's comment text**, and remember
`strings(1)` ignores anything shorter than four characters. The instruction stream
(`objdump -dr`) is the only reliable evidence that a code change compiled.
- **Place probes at statements, never inside a multi-line call or an `#if` block.**
- **A probe is not gone until `build/ra/obj` is deleted too.** Reconfiguring alone
reuses stale objects and ships old probes into `eboot.bin`.
- **`src/` must be compiled with the frontend's `-D` flags**, which
`tools/build-title.sh` does by asking `tools/retroarch-flags.sh`. A bare `make app`
produces a driver table the frontend reads at the wrong offsets.
- **Compile from `build/ra-conf`, never from `vendor/retroarch`**; upstream is never
edited and `patches/series` is the only record of this port's changes.
- **`-DHAVE_MAIN` must never be defined**: it compiles the frontend's main loop out.
- The console's address and credentials come from the ignored `.env`; anything read
from the console stays in the ignored `klog/` tree.
- The console's address and credentials come from the ignored `.env`; console
captures stay in the ignored `klog/` tree.
## Last verified
| Check | Result |
| --- | --- |
| `bash tools/run-title.sh --watch 20` | PASS: no fatal signal, menu on screen, `ps5_frame 1: menu 320x240 pitch=640 present=1`, `display: flip 1200 of buffer 1, status=0 marker=1` |
| `bash tools/build-title.sh` | PASS: clean link with the driver embedded; `eboot.bin` 29,859,722 bytes (was 8,225,706); 0 undefined symbols |
| `bash tools/build-mesa-util.sh` | PASS: `u_thread.o`, `anon_file.o`, `os_file.o`, each checked to define the symbols it is for |
| `bash tools/verify.sh` (all gates) | PASS (format unit build integration evidence) |
| `python3 -m unittest discover -s tests` | PASS: 11 tests |
| `tests/test_frontend.py` ABI check, built without the defines | FAILS as it must: `video_ps5 is 136 bytes ... video_driver_t is 144` |
| `nm -S build/obj/src_video_ps5.cpp.o` | PASS: `video_ps5` is `0x90` bytes, the frontend's `sizeof(video_driver_t)` |
| `diff -rq build/ra-conf vendor/retroarch` | PASS: only the four files `patches/series` names differ; no probes |
| `python3 tools/evidence.py compare evidence/` | PASS: 3 captures replayed, 0 failed |
| `python3 -m unittest discover -s tests` | PASS: 18 tests, including the AGC import surface and the unwind boundaries in the linked image |
| `bash tools/run-title.sh` on this build | **NOT RUN**: the driver is linked but has not been on the console yet |
| Config path (0006) | **PARKED**: reading the config still crashes in `command_event`; see `parked/config-path.patch.py` |
## Open findings
- The console's FTP will not replace `sce_module/libc.prx`: four uploads, a fresh
filename and a full listing all served the console's own 1,335,962-byte file.
`tools/deploy-title.py` reports it and carries on, so the title runs against the
console's copy. It has to be solved before anything ships.
- RGUI's assets: the menu reads `ASSETS_DIR`, compiled as `/app0/assets`, and the
RGUI fonts are bundled under `assets/rgui/font/`. Whether the rest of RetroArch's
asset tree should ship with the title is still undecided.
- The blit is nearest-neighbour for both formats and converts RGB565 per source
pixel. If a core ever presents a 1080p frame, the line-by-line scale is the part
to look at first.
- **`../PS5_Vulkan`'s PS5 object list has a hole**: `libpsbc_driver.ps5.a` references
eight symbols nothing in that tree defines. Its `libvulkan.so.1` links only because a
shared link may leave symbols undefined, so the hole is invisible there. This
project compiles them instead of reimplementing them; if that project adds
`u_thread.c`, `anon_file.c` and `os_file.c` to the archive, `tools/build-mesa-util.sh`
can go.
- The config-path crash is the last thing between this title and a readable
`retroarch.cfg`, and with it the file logger.
- The console's FTP will not replace `sce_module/libc.prx`; the title runs against the
console's copy. A release has to solve this.
- RGUI's assets: the fonts are bundled under `assets/rgui/font/`; whether the rest of
RetroArch's asset tree should ship is undecided.
- Pad input cannot be verified unattended yet: nothing in the pipeline presses a
button.
+59
View File
@@ -955,3 +955,62 @@ duplicate pairs, three legitimate edits to `input_driver.c` at different anchors
**State.** `bash tools/build-title.sh` from a wiped tree, then
`bash tools/run-title.sh --watch 12` -> title runs, no fatal signal;
`bash tools/verify.sh` -> PASS; 16 host tests.
## 2026-09-18: The driver is linked into the title, and the link completes
**The step.** `../PS5_Vulkan`'s driver stops being a library RetroArch loads at run
time and becomes ordinary symbols in `eboot.bin`. The alternative was measured and
closed in `0cc661d`: a PS5 title cannot dlopen a driver. This is the commit that makes
the replacement actually link.
**What the first link said.** Twenty-four undefined symbols, and the count is only
knowable because the link line now carries `--error-limit=0`; before that lld stopped
at twenty and the tail of the list was invisible. The 24 are three unrelated problems:
- **Four `__eh_frame_*` boundaries** (`__eh_frame_start/end`,
`__eh_frame_hdr_start/end`). Their shader compiler is C++ and links the SDK's
libunwind, which finds the unwind tables through these boundary symbols rather than
through `dl_iterate_phdr`. `../PS5_Vulkan/tooling/psbc/ps5-pie-unwind.ld` defines
them; this project's `tooling/native/ps5-pie.ld` is byte-identical to the
`ps5-pie.ld` that script includes, so the same four `PROVIDE`s were added to it
rather than adopting their `-T` path.
- **Eight Mesa utility symbols** from `u_queue.ps5.o`, `os_memory_fd.ps5.o` and
`ac_spm_config.ps5.o`: `u_thread_create`, `u_thread_setname`,
`util_barrier_init/destroy/wait`, `util_thread_get_time_nano`,
`os_create_anonymous_file`, `os_read_file`. Their
`toolchain/Makefile.opengnm-psbc-ps5` filters `src/util/{u_thread,anon_file,os_file}.c`
out of `UTIL_SRCS`, and `tools/build-psbc-ps5.sh` compiles only a five-file support
list, so nothing in that tree defines them. Their own `libvulkan.so.1` links because
a shared object may leave symbols undefined; a title's link may not. Mesa's own
definitions are compiled here (`tools/build-mesa-util.sh`) with their PS5
configuration rather than reimplemented, because `u_thread.c`'s `util_barrier` and
its `mtx_t`/`cnd_t` come from that tree's `c11/threads.h`, the same header the
`u_queue.ps5.o` inside the archive was compiled against. Two of the three needed a
flag that configuration does not set: `HAVE_PTHREAD_NP_H` for
`pthread_set_name_np`, and for `anon_file.c` the `-D_XOPEN_SOURCE=700` drop that
`tooling/psbc/support.mk` already documents for `usleep` and `getprogname`, because
`SHM_ANON` is declared under `__BSD_VISIBLE`. `os_file.c` needed `-D__ORBIS__`, the
same switch their mak uses for `futex.ps5.o`: its FreeBSD branch walks the kernel's
file table through `sysctl(KERN_FILE)` and a `kvaddr_t` the SDK does not have.
- **Twelve `sceAgc*` entry points.** The console provides libSceAgc and
libSceAgcDriver and the SDK stubs neither. The declarations come from
`../PS5_Vulkan/vendor/ps5/sdk/stubs/agc_canary_link_stub.c`, signatures unchanged,
because those are the imports that project's own titles already record.
**The stubs moved.** They had been edited in `vendor/ps5/sdk/stubs/`, which
`.gitignore` excludes, so the fix would have built here and vanished at the next
checkout. They are `tooling/ps5-stubs/agc_link_stub.c` and
`tooling/ps5-stubs/agc_driver_link_stub.c` now, and `tools/build.sh` reads them there.
**Evidence.** `bash tools/build-title.sh` -> exit 0, no undefined symbols,
`dist/PPSA99169/eboot.bin` 29,859,722 bytes against 8,225,706 before: 21.6 MB of
driver and shader compiler. `bash tools/verify.sh` -> PASS (format unit build
integration evidence). `python3 -m unittest discover -s tests` -> 18 tests, two new:
every `sceAgc*` the three driver archives reference is declared by the stubs (20
referenced, 21 declared, none missing), and the linked `build/llvm-pie.elf` defines
the four `__eh_frame_*` symbols - the check is on the image rather than on the script,
so regenerating the script from the boilerplate fails the test instead of the link.
**Not proven.** The driver has not been on the console since it was linked. The next
run is `bash tools/run-title.sh --watch 20` and the answers are in
`/app0/retroarch.log`.
+90
View File
@@ -26,6 +26,7 @@
from __future__ import annotations
import os
import re
import shutil
import struct
@@ -639,6 +640,95 @@ class FrameLayout(unittest.TestCase):
f"{self.table[key]:#x} instead of {expected:#x}")
class LinkedDriver(unittest.TestCase):
"""What linking ../PS5_Vulkan's driver into this title needs and used to lack.
Two facts, both of which failed a link that had already compiled: the
libraries' imports are declared, and the layout defines the boundary symbols
libunwind looks for.
"""
vulkan = Path(os.environ.get("PS5_VULKAN_DIR", ROOT.parent / "PS5_Vulkan"))
archives = (
vulkan / "build/driver/ps5/libps5vk.ps5.a",
vulkan / "build/driver/ps5/libpsbc_driver.ps5.a",
vulkan / ".deps/native/vulkan-runtime/lib/libvk_runtime.ps5.a",
)
stubs = (
ROOT / "tooling/ps5-stubs/agc_link_stub.c",
ROOT / "tooling/ps5-stubs/agc_driver_link_stub.c",
)
def nm(self) -> str | None:
"""The SDK's llvm-nm, or any nm that can read these archives."""
sdk = ROOT / ".deps/native/ps5-payload-sdk/bin/llvm-nm"
if sdk.is_file():
return str(sdk)
return shutil.which("llvm-nm") or shutil.which("nm")
def test_every_agc_entry_point_the_driver_calls_is_declared(self) -> None:
"""The AGC stubs cover the driver's whole import surface.
The console provides libSceAgc and libSceAgcDriver and the SDK ships no
stub for either, so tooling/ps5-stubs/ declares what the driver calls.
A missing declaration is not a warning: it stops the link, which is how
the first Vulkan build of this port ended, on twelve entry points at once.
The lists are compared rather than a count, because rebuilding the driver
can add one.
"""
if not all(archive.is_file() for archive in self.archives):
self.skipTest("the driver archives are not built; run ../PS5_Vulkan/tools/build-driver.sh")
nm = self.nm()
if nm is None:
self.skipTest("no llvm-nm or nm on PATH to read the archives with")
declared = set()
# A definition line starts with its return type; a comment that mentions
# one of these names starts with an asterisk, which is the difference.
definition = re.compile(r"^[A-Za-z_][\w ]*?\**\s*(sceAgc\w+)\s*\(", re.M)
for stub in self.stubs:
declared |= set(definition.findall(stub.read_text(encoding="utf-8")))
self.assertTrue(declared, "the AGC stubs declare nothing at all")
referenced = set()
for archive in self.archives:
done = subprocess.run([nm, "-u", str(archive)], capture_output=True, text=True, check=True)
for line in done.stdout.splitlines():
fields = line.split()
# ` U sceAgcCreateShader`, and a member heading
# line that ends in a colon rather than a symbol name.
if fields and fields[-1].startswith("sceAgc"):
referenced.add(fields[-1])
self.assertTrue(referenced, "no AGC entry point is referenced at all; the archives changed shape")
missing = referenced - declared
self.assertEqual(missing, set(),
f"the driver calls {sorted(missing)}, which tooling/ps5-stubs/ does not declare")
def test_the_linked_title_defines_the_unwind_boundaries(self) -> None:
"""libunwind finds the unwind tables through four boundary symbols.
The shader compiler is C++ and links the SDK's libunwind, which cannot use
dl_iterate_phdr in a title, so tooling/native/ps5-pie.ld provides the four
symbols the sibling project's tooling/psbc/ps5-pie-unwind.ld defines. The
check is on the linked image rather than on the script, so regenerating
the script from the boilerplate - which is how they went missing - fails
here instead of at the next link.
"""
image = ROOT / "build" / "llvm-pie.elf"
if not image.is_file():
self.skipTest("nothing linked; run tools/build-title.sh")
nm = self.nm()
if nm is None:
self.skipTest("no llvm-nm or nm on PATH to read the image with")
done = subprocess.run([nm, str(image)], capture_output=True, text=True, check=True)
defined = {line.split()[-1] for line in done.stdout.splitlines() if len(line.split()) >= 2}
for symbol in ("__eh_frame_start", "__eh_frame_end",
"__eh_frame_hdr_start", "__eh_frame_hdr_end"):
self.assertIn(symbol, defined,
f"{symbol} is not in the linked image; the layout no longer provides it")
class Artifact(unittest.TestCase):
"""The built folder is a title the console can be asked to run."""
+10
View File
@@ -30,6 +30,16 @@ SECTIONS {
KEEP(*(.eh_frame_hdr))
} : ro
.eh_frame : { KEEP(*(.eh_frame)) } : ro
/* The SDK's libunwind finds the unwind tables through these four boundary
* symbols rather than through dl_iterate_phdr, which a title does not have.
* The Vulkan driver's shader compiler is C++ and links libunwind, so the
* link fails on them without this. Same four definitions as ../PS5_Vulkan's
* tooling/psbc/ps5-pie-unwind.ld, kept here so the layout stays this
* project's own file. */
PROVIDE(__eh_frame_start = ADDR(.eh_frame));
PROVIDE(__eh_frame_end = ADDR(.eh_frame) + SIZEOF(.eh_frame));
PROVIDE(__eh_frame_hdr_start = ADDR(.eh_frame_hdr));
PROVIDE(__eh_frame_hdr_end = ADDR(.eh_frame_hdr) + SIZEOF(.eh_frame_hdr));
.gcc_except_table : { *(.gcc_except_table .gcc_except_table.*) } : ro
.rodata : { *(.rodata .rodata.*) } : ro
+26
View File
@@ -0,0 +1,26 @@
/*
* PS5 RetroArch - AGC driver link stub.
* SPDX-License-Identifier: GPL-3.0-or-later
*
* Host-link declarations for system libSceAgcDriver; never packaged or executed.
* Under tooling/ for the reason tooling/ps5-stubs/agc_link_stub.c gives.
*/
#include <stdint.h>
uint32_t sceAgcDriverGetWaitRenderingPacketSizeInDwords(void) { return 0; }
/* Added by this project: the submission half of the AGC flip path, mirroring
* ../PS5_Vulkan's vendor/ps5/sdk/stubs/agc_driver_canary_link_stub.c. */
int32_t sceAgcDriverSubmitDcb(void *description) { (void)description; return -1; }
/* ../PS5_Vulkan's draw path waits for the previous frame's rendering to finish
* before it writes the next command buffer; signature from that project's
* agc_driver_canary_link_stub.c. Host-link only. */
uint32_t sceAgcDriverWaitUntilSafeForRendering(uint32_t **command, uint32_t packet_size,
uint32_t reserved, uint32_t handle,
int buffer_index)
{
(void)command; (void)packet_size; (void)reserved; (void)handle; (void)buffer_index;
return 0;
}
+105
View File
@@ -0,0 +1,105 @@
/*
* PS5 RetroArch - AGC link stub.
* SPDX-License-Identifier: GPL-3.0-or-later
*
* Host-link declarations for system libSceAgc; never packaged or executed.
*
* Under tooling/ rather than in vendor/, where the boilerplate kept it, because
* vendor/ is not in the diff: declarations the driver needs would be lost with
* the next checkout. tools/build.sh compiles this into the link stub the native
* converter reads the imports from, and the bodies never run on the console.
*/
#include <stdint.h>
int32_t sceAgcInit(void *state, uint32_t size) { (void)state; (void)size; return -1; }
void *sceAgcGetRegisterDefaults(void) { return 0; }
uint32_t *sceAgcDcbSetCxRegistersIndirect(void *command, const void *registers, uint32_t count) { (void)command; (void)registers; (void)count; return 0; }
uint32_t *sceAgcDcbDrawIndexAuto(void *command, uint32_t count, uint64_t modifier) { (void)command; (void)count; (void)modifier; return 0; }
/* Added by this project, mirroring ../PS5_Vulkan's vendor/ps5/sdk/stubs
* (agc_canary_link_stub.c), because the flip this port needs is submitted as an
* AGC command buffer rather than through sceVideoOutSubmitFlip. These bodies are
* host-link declarations for the native converter to record as imports; they are
* never packaged and never run on the console, exactly like the five above. */
int32_t sceAgcSuspendPoint(void) { return -1; }
uint32_t *sceAgcDcbSetFlip(void *command, uint32_t handle, int buffer_index, uint32_t flip_mode,
int64_t flip_argument)
{
(void)command; (void)handle; (void)buffer_index; (void)flip_mode; (void)flip_argument;
return 0;
}
/* The GPU command-processor context. ../PS5_Vulkan calls this before it submits
* anything (driver/ps5vk_device.c: sceAgcInit(PS5VK_AGC_VERSION)); submitting a
* DCB without it is what produced a write fault at 0x202210000 in this port. The
* version is the one that project's native runtime ABI uses. */
int32_t sceAgcInitVersioned(uint32_t version) { (void)version; return -1; }
/* The rest of the AGC surface ../PS5_Vulkan's driver references, so that linking
* its archives into this title resolves: its shader compiler builds ISA through
* sceAgcCreateShader/sceAgcLinkShaders, and its draw path writes the register
* and index packets below. Signatures are ../PS5_Vulkan's
* vendor/ps5/sdk/stubs/agc_canary_link_stub.c, which the converter already
* records as imports for that project's own titles. Host-link only. */
int32_t sceAgcCreateShader(void **shader, void *header, void *code)
{
(void)shader; (void)header; (void)code;
return -1;
}
int32_t sceAgcLinkShaders(void *cx, void *uc, void *reserved, void *vertex_shader,
void *pixel_shader, uint32_t primitive_type)
{
(void)cx; (void)uc; (void)reserved; (void)vertex_shader;
(void)pixel_shader; (void)primitive_type;
return -1;
}
uint32_t *sceAgcDcbSetUcRegistersIndirect(void *command, const void *registers, uint32_t count)
{
(void)command; (void)registers; (void)count;
return 0;
}
uint32_t *sceAgcDcbSetShRegistersIndirect(void *command, const void *registers, uint32_t count)
{
(void)command; (void)registers; (void)count;
return 0;
}
uint32_t *sceAgcDcbSetIndexSize(void *command, uint8_t index_size, uint8_t reserved)
{
(void)command; (void)index_size; (void)reserved;
return 0;
}
uint32_t *sceAgcCbReleaseMem(void *command, uint8_t event, int16_t control, uint64_t arg3,
int8_t arg4, void *arg5, uint32_t arg6, uint64_t arg7,
uint16_t arg8, uint16_t arg9, int8_t arg10, int32_t arg11)
{
(void)command; (void)event; (void)control; (void)arg3; (void)arg4; (void)arg5; (void)arg6;
(void)arg7; (void)arg8; (void)arg9; (void)arg10; (void)arg11;
return 0;
}
uint32_t *sceAgcCbSetShRegisterRangeDirect(void *command, uint32_t offset, const uint32_t *values,
uint32_t count)
{
(void)command; (void)offset; (void)values; (void)count;
return 0;
}
uint32_t *sceAgcDcbSetNumInstances(void *command, uint32_t count)
{
(void)command; (void)count;
return 0;
}
uint32_t *sceAgcDcbSetIndexBuffer(void *command, void *indices)
{
(void)command; (void)indices;
return 0;
}
uint32_t *sceAgcDcbSetIndexCount(void *command, uint32_t count)
{
(void)command; (void)count;
return 0;
}
uint32_t *sceAgcDcbDrawIndex(void *command, uint32_t count, void *indices, uint64_t modifier)
{
(void)command; (void)count; (void)indices; (void)modifier;
return 0;
}
+181
View File
@@ -0,0 +1,181 @@
#!/usr/bin/env bash
# The three Mesa utility sources the driver's archives reference but that
# ../PS5_Vulkan's PS5 build leaves out.
#
# Its tools/build-psbc-ps5.sh compiles the archive's own files plus a short
# support list (os_time, u_process, u_format_s3tc, ps5_agc_package,
# psbc_ps5_shims), and third_party/opengnm-psbc/toolchain/
# Makefile.opengnm-psbc-ps5 additionally filters these three out of UTIL_SRCS:
#
# src/util/u_thread.c u_thread_create, u_thread_setname, util_barrier_init,
# util_barrier_destroy, util_barrier_wait,
# util_thread_get_time_nano
# src/util/anon_file.c os_create_anonymous_file
# src/util/os_file.c os_read_file
#
# Nothing in that tree defines them, which no executable link survives: a shared
# object may leave symbols undefined - ../PS5_Vulkan's libvulkan.so.1 links that
# way, and its shader compiler archive is only ever linked into one - while a
# title's link may not, which is why this project's link stopped on them.
#
# Mesa's own definitions are compiled here rather than reimplemented, and with
# the sibling's PS5 configuration (toolchain/opengnm-psbc-ps5.mak, whose
# SHARED_FLAGS are reproduced below): u_thread.c's util_barrier and its mtx_t and
# cnd_t come from that tree's own c11/threads.h, and the u_queue.ps5.o already
# inside libpsbc_driver.ps5.a was compiled against exactly those, so a
# reimplementation would have to match its layout by luck.
#
# Writes build/obj/mesa-util/*.o, prints their paths one per line, and checks
# each object defines the symbols it is named for.
#
# tools/build-mesa-util.sh [sdk_root]
#
# sdk_root defaults to $PS5_PAYLOAD_SDK, then to .deps/native/ps5-payload-sdk.
# PS5_VULKAN_DIR overrides the sibling project's root.
set -euo pipefail
cd "$(dirname -- "${BASH_SOURCE[0]}")/.."
root=$PWD
sdk=${1:-${PS5_PAYLOAD_SDK:-$root/.deps/native/ps5-payload-sdk}}
# Same default as tools/build-title.sh, which passes this through to make app.
export PS5_CLANG=${PS5_CLANG:-/usr/bin/clang}
vulkan_dir=${PS5_VULKAN_DIR:-$root/../PS5_Vulkan}
third_party="$vulkan_dir/.deps/work/psbc-ps5/third_party"
mesa="$third_party/opengnm-psbc"
out="$root/build/obj/mesa-util"
sources=(
src/util/u_thread.c
src/util/anon_file.c
src/util/os_file.c
)
# One line per source, the symbols the link needs from it.
symbols_for() {
case ${1##*/} in
u_thread.c)
printf '%s\n' u_thread_create u_thread_setname util_barrier_init \
util_barrier_destroy util_barrier_wait util_thread_get_time_nano
;;
anon_file.c) printf '%s\n' os_create_anonymous_file ;;
os_file.c) printf '%s\n' os_read_file ;;
*) return 1 ;;
esac
}
missing=()
for source in "${sources[@]}"; do
[[ -f $mesa/$source ]] || missing+=("$mesa/$source")
done
[[ -d $third_party/opengnm/include ]] || missing+=("$third_party/opengnm/include")
[[ -d $third_party/Vulkan-Headers/include ]] || missing+=("$third_party/Vulkan-Headers/include")
[[ -x $sdk/bin/prospero-clang ]] || missing+=("$sdk/bin/prospero-clang")
if (( ${#missing[@]} )); then
printf 'error: the driver sources this step compiles are missing.\n' >&2
printf ' Build ../PS5_Vulkan first (tools/build-psbc-ps5.sh), or set\n' >&2
printf ' PS5_VULKAN_DIR and PS5_PAYLOAD_SDK.\n' >&2
printf ' missing: %s\n' "${missing[@]}" >&2
exit 2
fi
# SHARED_FLAGS and CFLAGS from the sibling's toolchain/opengnm-psbc-ps5.mak,
# whose include paths are relative to the tree being compiled, hence the cd.
shared_flags=(
-Iinclude/
-Ilibpsbc/
-I"$third_party/opengnm/include"
-I"$third_party/Vulkan-Headers/include"
-Isrc/
-Isrc/amd
-Isrc/amd/common
-Isrc/amd/common/nir
-Isrc/amd/compiler
-Isrc/amd/vulkan
-Isrc/amd/vulkan/nir
-Isrc/vulkan/runtime
-Isrc/vulkan/runtime/bvh
-Isrc/vulkan/util
-Isrc/compiler
-Isrc/compiler/nir
-Isrc/compiler/spirv
-Isrc/gallium/include
-Isrc/mesa
-Isrc/mesa/main
-Isrc/util
-Icmd/psbc
-Iinclude/mesa
-D_GNU_SOURCE
-D_XOPEN_SOURCE=700
-DUTIL_ARCH_LITTLE_ENDIAN=1
-DUTIL_ARCH_BIG_ENDIAN=0
-DHAVE_STRUCT_TIMESPEC=1
-DHAVE_PTHREAD=1
# The sibling's mak does not set this because its PS5 build never compiles
# u_thread.c at all. The console is FreeBSD, where that file's u_thread_setname
# calls pthread_set_name_np, declared in <pthread_np.h> and exported by
# libkernel - the SDK has both.
-DHAVE_PTHREAD_NP_H=1
-DHAVE_SYSCONF=1
-DHAVE_FUNC_ATTRIBUTE_PACKED=1
-Dalloca=__builtin_alloca
-include
strings.h
-DBLAKE3_NO_SSE2
-DBLAKE3_NO_SSE41
-DBLAKE3_NO_AVX2
-DBLAKE3_NO_AVX512
)
mkdir -p "$out"
objects=()
for source in "${sources[@]}"; do
object="$out/${source##*/}"
object=${object%.c}.o
file_flags=("${shared_flags[@]}")
case ${source##*/} in
anon_file.c)
# os_create_anonymous_file's FreeBSD path is shm_open(SHM_ANON, ...),
# and sys/mman.h declares SHM_ANON only under __BSD_VISIBLE, which
# -D_XOPEN_SOURCE=700 clears. ../PS5_Vulkan's tooling/psbc/support.mk
# drops that define for os_time.c's usleep and u_process.c's
# getprogname for the same reason, so this is its rule, one file on.
kept=()
for flag in "${file_flags[@]}"; do
[[ $flag == -D_XOPEN_SOURCE=700 ]] || kept+=("$flag")
done
file_flags=(${kept+"${kept[@]}"})
;;
os_file.c)
# os_same_file_description's FreeBSD branch walks the kernel's file
# table through sysctl(KERN_FILE) and struct xfile, which needs the
# kernel-only kvaddr_t the SDK's headers do not have - a title cannot
# read that table anyway. __ORBIS__ selects the branch after it, which
# answers "cannot tell" with -1, and is the same switch the sibling's
# mak uses (`src/util/futex.ps5.o: CFLAGS += -D__ORBIS__`). os_read_file
# itself is unguarded.
file_flags+=(-D__ORBIS__)
;;
esac
(cd "$mesa" && PS5_PAYLOAD_SDK="$sdk" sh "$root/tooling/prospero-clang18" \
-std=gnu11 -O2 -g -Wall -fPIC -DOPENGNM_PSBC_ORBIS=1 \
-Dstatic_assert=_Static_assert \
-Wno-unused-function -Wno-unused-variable \
-Wno-unreachable-code-generic-assoc \
${file_flags+"${file_flags[@]}"} \
-c "$source" -o "$object") 2>&1 | sed "s|^|[$source] |" >&2
[[ -f $object ]] || { echo "error: $source did not compile" >&2; exit 1; }
# The link resolves these by name, so check the object really defines them
# instead of trusting that the compile produced what this step is for.
while read -r symbol; do
if ! "$sdk/bin/llvm-nm" --defined-only "$object" | grep -qE " [TtWw] $symbol\$"; then
echo "error: $object does not define $symbol" >&2
exit 1
fi
done < <(symbols_for "$source")
objects+=("$object")
done
printf '%s\n' "${objects[@]}"
+16
View File
@@ -129,6 +129,21 @@ fi
# survive the archive boundary (--whole-archive), which is how the sibling links it.
vulkan_flags="--no-dynamic-linker -z nodynamic-undefined-weak"
# Three Mesa utility sources the archives above reference but do not carry:
# ../PS5_Vulkan's PS5 object list filters u_thread.c, anon_file.c and os_file.c
# out, and its own libvulkan.so.1 only links because a shared object may leave
# symbols undefined. A title may not, so they are compiled here from that
# project's sources with its PS5 configuration and linked as plain objects.
# tools/build-mesa-util.sh says which symbols each one is for. It prints the
# object paths on stdout, so a compile failure has to be caught here: a process
# substitution would let the link fail later on symbols this step was to supply.
if ! vulkan_object_list=$(PS5_VULKAN_DIR="$vulkan_dir" PS5_PAYLOAD_SDK="$sdk" \
PS5_CLANG=/usr/bin/clang bash "$root/tools/build-mesa-util.sh"); then
echo "error: the driver's Mesa utility objects did not build" >&2
exit 2
fi
mapfile -t vulkan_objects <<< "$vulkan_object_list"
echo "==> [title] step 2/3: the title"
PS5_PAYLOAD_SDK="$sdk" \
PS5_CLANG=/usr/bin/clang \
@@ -137,6 +152,7 @@ APP_DEFINITIONS="${title_definition_names[*]}" \
APP_INCLUDE_PATHS="build/ra-conf build vendor/retroarch vendor/retroarch/libretro-common/include vendor/retroarch/deps vendor/retroarch/deps/stb" \
APP_STATIC_ARCHIVES="build/ra/libretroarch.a" \
APP_VULKAN_ARCHIVES="${vulkan_archives[*]}" \
APP_EXTRA_OBJECTS="${vulkan_objects[*]}" \
APP_LINK_FLAGS="$vulkan_flags" \
make app
+18 -3
View File
@@ -204,8 +204,8 @@ build_system_link_stub() {
printf '%s\n' "$output"
}
agc_stub=$(build_system_link_stub libSceAgc vendor/ps5/sdk/stubs/agc_link_stub.c)
agc_driver_stub=$(build_system_link_stub libSceAgcDriver vendor/ps5/sdk/stubs/agc_driver_link_stub.c)
agc_stub=$(build_system_link_stub libSceAgc tooling/ps5-stubs/agc_link_stub.c)
agc_driver_stub=$(build_system_link_stub libSceAgcDriver tooling/ps5-stubs/agc_driver_link_stub.c)
link_inputs=("$build/obj/app_crt.o" "$build/obj/app_cpp_runtime.o" "${objects[@]}")
link_inputs+=("$agc_stub" "$agc_driver_stub")
@@ -223,6 +223,19 @@ if [[ -n ${APP_VULKAN_ARCHIVES:-} ]]; then
}
done
link_inputs+=(--whole-archive "${vulkan_archives[@]}" --no-whole-archive)
# Plain objects the driver needs and its archives do not carry: Mesa's
# u_thread.c, anon_file.c and os_file.c, which ../PS5_Vulkan's PS5 object list
# filters out and only a shared link can survive without. tools/build-title.sh
# compiles them with tools/build-mesa-util.sh and passes them here.
if [[ -n ${APP_EXTRA_OBJECTS:-} ]]; then
read -r -a extra_objects <<< "$APP_EXTRA_OBJECTS"
for object in "${extra_objects[@]}"; do
[[ $object =~ ^[A-Za-z0-9_./+-]+(/[A-Za-z0-9_.+-]+)*\.o$ && -f $object ]] || {
echo "invalid extra object: $object" >&2; exit 2;
}
done
link_inputs+=("${extra_objects[@]}")
fi
# The shader compiler and Mesa's runtime are C++, so they need libc++,
# libc++abi and libunwind - the same three the sibling project links, plus the
# compiler's own builtins archive. They are grouped because the dependency runs
@@ -252,7 +265,9 @@ done
if (( ${#pacbrew_libs[@]} > 0 )); then
link_inputs+=(--start-group "${pacbrew_libs[@]}" --end-group)
fi
"$sdk_root/bin/prospero-lld" -T "$native/ps5-pie.ld" --eh-frame-hdr \
# --error-limit=0 lists every unresolved symbol instead of stopping at twenty,
# which is the difference between one link-failure diagnosis and several.
"$sdk_root/bin/prospero-lld" -T "$native/ps5-pie.ld" --eh-frame-hdr --error-limit=0 \
${APP_LINK_FLAGS:-} \
--version-script "$native/app-symbols.map" \
--exclude-libs=ALL -L "$build/obj" \