starred/PS5_RetroArch
1
0 Fork 0
mirror of https://github.com/mihawk-99/PS5_RetroArch.git synced 2026-10-18 23:09:29 -05:00
Code Issues Packages Projects Releases 11 Wiki Activity

Get the Vulkan driver to a created pipeline, a running runloop and a first frame

Browse Source 
The console run now reaches a frame. In order: video_vulkan selected, the console's
khr_display WSI, a device, a swapchain at 3840x2160, four textures, a vertex buffer,
the stock shader reflected, both stages compiled by the driver's own compiler,
vkCreateGraphicsPipelines returning 0, vulkan_init handing back a live pointer, the
input driver opening the pad, and runloop_iterate reaching vulkan_frame -> vkQueueSubmit.
Nothing is on screen yet: the first frame's command buffer already has a render pass
open when the backbuffer pass begins, and Mesa's common CmdBeginRenderPass2 asserts.

Three of the four fixes are port patches, all of them requests that driver refuses:

  0016  its pipeline check accepts only VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST, and its
        draw path refuses a non-indexed draw whose first vertex is not zero; the two
        quads are now six-vertex lists, the second triangle reached through the
        binding's own offset. This is what turned VK_ERROR_UNKNOWN into a pipeline.
  0017  it refuses any sampler that is not clamp-to-edge and leaves the output handle
        untouched, while CommonResources destroys every handle that is not
        VK_NULL_HANDLE - sixteen refused samplers were destroyed as if real and the
        driver asserted. The array is cleared before it is filled.
  0018  the khr_display context reported quit from the frontend's signal-handler
        state, so the runloop ended on its first iteration and rarch_main returned 0
        with no frame drawn. A console title has no terminal and no SIGTERM sender.

The fourth is this port's own CRT: _start went from _init_env straight to the static
constructors and never cleared the BSS, whose tail is 0xc7040 bytes of a segment that
is 0x104b4 bytes in the file. tooling/native/ps5-pie.ld marks __bss_start/__bss_end
and _start clears that range; every run prints the check that says so. Whether the
loader also zeroed it was never measured - the invariant is what this asserts.

Two instruments, both because nothing else could be read:

  src/main.cpp sends stderr to /app0/trace.txt and makes it unbuffered. The driver's
  __assert prints the expression, file and line to stderr and then aborts, and the
  console's libc buffers that stream, so every assertion arrived as a bare `abort is
  called(system)`. The trace now carries the assert text, which is what identified the
  render-pass state as the current blocker.

  tools/apply-runtime-probes.py carries thirty-nine probes, and tests/test_frontend.py
  checks the set itself: an insert that ends with its own anchor duplicates the line it
  is inserted before, which broke the build three times in this round. Every probe must
  carry a marker in its own text, a note and a trailing newline.

Evidence: bash tools/run-title.sh with the probe set applied (captures under klog/,
probes reverted before this commit); the trace's own lines quoted above;
bash tools/verify.sh PASS with a probe-free image (`strings build/llvm-pie.elf |
grep -c 'probe PIPE2|probe VDRV|...' ` = 0 after deleting build/ra/obj, which is what
a revert needs); 20 unit tests including the probe-set checks.

Next: mark the span between the offscreen-passes call and the backbuffer pass - the
first-frame history/feedback clear and the menu texture upload - and fix whichever of
the two leaves a render pass open.
This commit is contained in:
Mihawk committed 2026-09-19 01:03:05 -04:00
1 parent 1327ad1da7
commit 34d9200ecf
8 files changed
+496 -185
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/ACTIVE.md
+91 -113
View File
@@ -9,83 +9,101 @@ _Updated: 2026-09-19_
## Now
**The Vulkan driver runs on the console, and the first frame stops one call short.**
A `tools/run-title.sh` run shows RetroArch selecting `video_vulkan`, resolving the
console's `khr_display` WSI, creating a device, a swapchain at 3840x2160, its
textures (4x4 blank, two 512x512 fonts, 1x1 default), a vertex buffer, reflecting the
stock shader's SPIR-V and compiling both of its stages into shader modules - and then
`vkCreateGraphicsPipelines` answering `VK_ERROR_UNKNOWN`. `vulkan_init` returns NULL,
the title exits 1. Nothing renders yet; everything up to that call is new.
**The Vulkan driver initialises on the console, builds the stock shader's pipeline,
and runs the runloop - the first frame dies on a render pass left open.** A
`tools/run-title.sh` run now shows: `selected="vulkan"`, the console's `khr_display`
WSI resolved, a device and a swapchain at 3840x2160, four textures, a vertex buffer,
the stock shader's SPIR-V reflected, **both shader modules compiled and
`vkCreateGraphicsPipelines` returning 0**, `vulkan_init` handing back a live pointer,
the input driver opening the pad, and `runloop_iterate` reaching
`vulkan_frame` -> `vkQueueSubmit`. Nothing has reached the screen yet.
**Where the refusal is.** Not in the frontend's pipeline state, which was dumped and
compared against the driver's own checks (R32G32_SFLOAT attributes, TRIANGLE_STRIP,
blend off, one sample, two descriptor bindings at the strides that driver assigns),
and not in the shader compiler: both stock shaders compile through the *host* build of
`../PS5_Vulkan`'s compiler with the driver's own options. What is left is the step no
host test can reach - `sceAgcCreateShader`/`sceAgcLinkShaders` on the console inside
that driver's `ps5vk_graphics_pipeline_create`, whose failure is the
`AGC shader creation or linking failed: 0x%08x` that returns the error. That project's
own log is compiled out (`#if !MESA_DEBUG` in its `src/vulkan/runtime/vk_log.c`), so
the code it holds has to be read from their side: it is the one question this port
cannot answer for itself.
**The exact remaining failure**, read from the trace because assertions now say what
they are:
**Five frontend faults were fixed to get this far**, each a request that driver
refuses (patches/series 0013-0015, all in `gfx/`): the swapchain's usage bits masked
with the surface's `supportedUsageFlags`; an image's usage masked to its format's
advertised features, with a format that cannot be sampled replaced by
`R8G8B8A8_UNORM`; and `vulkan_format_to_bpp()` taught that format, without which the
staging buffer it sized came out zero bytes long and the driver aborted in
`vk_buffer_init`.
probe SPAN: offscreen passes
probe SPAN: menu upload
probe SPAN: about to begin the backbuffer pass
probe REC: begin render pass
assertion failed: cmd_buffer->render_pass == NULL
(.../vulkan-runtime/src/vulkan/runtime/vk_render_pass.c:2648,
vk_common_CmdBeginRenderPass2)
**And one file had to go: `src/video_filters_stub.cpp`.** It defined the whole Vulkan
filter chain as NULL-returning stubs, from when this port had video filters disabled.
`src/` is linked before the archive, so the stubs won and the driver read "no chain" -
no pipeline was ever created, and neither the log nor the trace could say so. Removing
it brought in glslang and SPIRV-Cross, and with them FreeBSD's xlocale interface:
`src/locale_shims.c` provides the thirty-four `_l` functions the console SDK does not
export, each the C-locale answer its unsuffixed counterpart gives.
The frame's command buffer already has a render pass open when the backbuffer pass
begins, and the marks bracket the span to the first-frame history/feedback clear
(`vulkan_filter_chain_build_offscreen_passes`) and the menu texture upload.
**Five things were fixed to get here**, four of them in this repository:
- **0016 - the topology.** That driver's pipeline check accepts only
`VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST`, and its draw path refuses a non-indexed draw
whose first vertex is not zero; RetroArch drew its quads as a four-vertex *strip*.
The quads are now six-vertex lists, the second triangle drawn through the binding's
own offset. This is what turned `VK_ERROR_UNKNOWN` into a created pipeline.
- **0017 - the samplers.** The driver refuses any sampler that is not clamp-to-edge
and leaves the output handle untouched, and `CommonResources` destroys every handle
that is not `VK_NULL_HANDLE`: sixteen refused samplers were destroyed as if they were
real, and the driver asserted. The array is cleared before it is filled.
- **0018 - the quit.** The display context reported `quit` from the frontend's signal
handler state, so the runloop ended on its first iteration and `rarch_main` returned
0 with no frame. A console title has no terminal and no SIGTERM sender, so this
context no longer treats that state as a quit request.
- **The BSS is cleared by this port's CRT.** `_start` went from `_init_env` straight to
the static constructors, and the image's writable segment is 0x104b4 bytes in the
file against 0xc7040 in memory. `tooling/native/ps5-pie.ld` now marks
`__bss_start`/`__bss_end` and `_start` clears the range; every run prints
`bss check=0 (must be 0), data check=7 (must be 7)`. Whether the loader also zeroed
it was never measured - the invariant is what this asserts, and it is cheap.
- **Assertions are readable.** The driver's `__assert` prints through stderr and then
aborts, and the console's libc buffers stderr, so every assertion used to arrive as a
bare `abort is called(system)`. `src/main.cpp` sends stderr to the trace **and makes
it unbuffered**, which is what produced the assert text above.
## Next
1. **Ask `../PS5_Vulkan` for the `sceAgcCreateShader`/`sceAgcLinkShaders` result code**
for this pipeline, or for a build whose log is not compiled out. That single value
is the difference between a shader package the console rejects and an AGC call made
in the wrong state, and this port has no way to see it.
2. **Then the first frame.** The swapchain, the textures and the pipeline are the last
three things between here and a menu on screen; `vulkan_init` returning non-NULL is
the milestone to watch for in `/app0/trace.txt`.
1. **Mark the span between the offscreen-passes call and the backbuffer pass**: the
first-frame `clear_history_and_feedback` and the menu texture upload. Whichever one
opens a render pass is the bug; `vulkan_framebuffer_clear` and
`vulkan_copy_staging_to_dynamic` are the two candidates, and the driver's own
`ps5vk_CmdClearColorImage` is CPU work at a submission split point, so the copy path
is the likelier one.
2. **Then the first frame**, and with it the objective: `probe FRAME` lines in
`/app0/trace.txt` every thirty frames, the menu on screen, and the driver's own
present path.
3. **Then the config file.** Content loading still discards the title's `-c`, and the
fix (0006) is parked in `parked/config-path.patch.py`.
## Working notes
- **`bash tools/run-title.sh` is the only way to run the title.** It owns the whole
sequence, including starting the klog listener before the launch and closing the
title itself, and it prints the title's own trace. Do not hand-deploy.
- **Read a console crash with `python3 tools/symbolize-crash.py <capture>`.** A
backtrace is bare addresses and `--exclude-libs` leaves the driver, the compiler and
the SDK with no symbol table; `build/title.map`, written by every link, is what turns
them into names - it is how `0x41a7c4` became `ps5vk_CreateSwapchainKHR +0x634`.
- **A console round is only worth its minutes when the mark is placed after the
assignments.** A dump inserted before a struct is filled prints garbage; one inserted
inside a declaration list does not compile; one inside a function body cannot include
a header that opens an `extern "C"` block. All three happened.
sequence and prints the title's own trace. Do not hand-deploy.
- **`python3 tools/symbolize-crash.py <capture>` reads a console backtrace**, but only
approximately: the addresses are the *converted* image's and `build/title.map` is the
linked one, so a frame can land one function off. The trace's `assertion failed:`
line is exact - prefer it.
- **Assertions only speak because stderr is unbuffered.** If that line vanishes from
the trace, check `src/main.cpp`'s `setvbuf` before anything else.
- **A probe is not gone until the objects are.** Reverting the probe set leaves the
strings in `build/ra/obj`; `strings build/llvm-pie.elf | grep 'probe '` is the check
and `rm -rf build/ra/obj` is the fix. Two stale probe strings survived a revert this
round and needed a clean rebuild.
- **An insert must not repeat its anchor.** The probe tool writes `insert + anchor`, so
an insert ending with its anchor duplicates that line - which broke the build three
times in one round. `tests/test_frontend.py`'s `ProbeSet` now checks every probe for
that, for a marker in its own text, for a note and for a trailing newline.
- **Do not hand-edit the configured tree.** Slicing a probe out of `build/ra-conf` by
text removed a declaration line and a brace's partner this round, and the file only
went back together by diffing it against `vendor/retroarch`. Revert probes with the
tool; if the tree is damaged, diff it against its vendor copy (it carries no port
patches beyond the ones the patcher names).
- **`vendor/` is not in the diff.** Anything a build needs from it must be regenerable
or live in a tracked directory; `tooling/ps5-stubs/` and `src/` are the second case.
- **Runtime probes are kept: `tools/apply-runtime-probes.py`.** A rebuild wipes probes
written into `build/ra-conf`, so they live in that script - applied with one command,
`--revert` to remove, `--list` to see what each is for. They are not in the shipping
build; revert them before the gate run, and check `strings build/llvm-pie.elf` says 0.
or live in a tracked directory.
- **`/app0/...` is not an FTP path.** The title's files are at
`/data/homebrew/PPSA99169/`; `RETR /app0/trace.txt` answers 550.
- **`src/` is linked before the archives**, so a function defined there silently
overrides the same symbol in `libretroarch.a` or in the driver's archives - that is
what the filter-chain stub was doing. It is also what `src/locale_shims.c` relies on.
- **`src/` must be compiled with the frontend's `-D` flags**, which
`tools/build-title.sh` does by asking `tools/retroarch-flags.sh`.
- **Compile from `build/ra-conf`, never from `vendor/retroarch`**; upstream is never
edited and `patches/series` is the only record of this port's changes.
- **`-DHAVE_MAIN` must never be defined**: it compiles the frontend's main loop out.
`/data/homebrew/PPSA99169/`.
- **`src/` is linked before the archives**, so a definition there wins over the same
symbol in `libretroarch.a` or the driver's archives - that is how
`src/locale_shims.c` supplies the `_l` locale functions, and why a libc symbol cannot
be overridden (the native converter refuses to publish an application export).
- The console's address and credentials come from the ignored `.env`; console captures
stay in the ignored `klog/` tree.
@@ -93,19 +111,20 @@ export, each the C-locale answer its unsuffixed counterpart gives.
| Check | Result |
| --- | --- |
| `bash tools/run-title.sh` (probed build) | **Partial**: `selected="vulkan"`, `context driver=khr_display`, 4 textures, `BUF bound`, `REFL entered`, `CHAIN both modules created`, then `vkCreateGraphicsPipelines -> -13` |
| Host compile of both stock shaders | PASS: `build/host/opengnm-psbc-probe` accepts the extracted SPIR-V with the driver's options (vertex `--ngg` and `--raw`, fragment with UBO stride 16 + sampler binding 2 stride 48) |
| `bash tools/verify.sh` (all gates) | PASS (format unit build integration evidence), probes reverted |
| `strings build/llvm-pie.elf \| grep -c probe` | PASS: 0 - the shipping image carries no probes |
| `python3 -m unittest discover -s tests` | PASS: 18 tests |
| `bash tools/run-title.sh` (probed build) | **Partial**: `selected="vulkan"`, `khr_display`, 4 textures, `vkCreateGraphicsPipelines -> 0`, `vulkan_init` live, pad opened, `vulkan_frame` -> `vkQueueSubmit`, then `assertion failed: cmd_buffer->render_pass == NULL` |
| `bash tools/verify.sh` (all gates) | PASS (format unit build integration evidence) |
| `python3 -m unittest discover -s tests` | PASS: 20 tests, including the probe-set checks and the patch count |
| `strings build/llvm-pie.elf \| grep 'probe '` | Only the driver's own messages; no probe text (after a clean rebuild) |
| Config path (0006) | **PARKED**: reading the config still crashes in `command_event`; see `parked/config-path.patch.py` |
## Open findings
- **The pipeline refusal is on `../PS5_Vulkan`'s side of the line**: its
`ps5vk_graphics_pipeline_create` creates the stages through `sceAgcCreateShader` and
links them with `sceAgcLinkShaders`, and its error message is compiled out of its own
build. The result code it holds is the missing fact.
- **The first frame leaves a render pass open** before the backbuffer pass; the marks
narrow it to the offscreen-passes call and the menu upload, and the next measurement
splits those two.
- **The frontend's own log never reaches disk.** `--log-file=/app0/retroarch.log` is in
argv and the file exists, but it is a stale 1200 bytes: RetroArch never closes the
logger on this path, so its buffer is lost. The trace is the instrument that works.
- `src/locale_shims.c` exists because glslang and SPIRV-Cross are written against
FreeBSD's xlocale interface and the console SDK exports only the unsuffixed
functions. If the SDK ever ships the `_l` set, that file goes.
@@ -113,44 +132,3 @@ export, each the C-locale answer its unsuffixed counterpart gives.
console's copy. A release has to solve this.
- Pad input cannot be verified unattended yet: nothing in the pipeline presses a
button.
The first link failed on **24 undefined symbols**, and they were three different
problems rather than one:
1. **Four `__eh_frame_*` boundaries.** The shader compiler is C++ and links the SDK's
libunwind, which finds unwind tables through those symbols, not through
`dl_iterate_phdr`. `tooling/native/ps5-pie.ld` now provides them, the same four
`../PS5_Vulkan/tooling/psbc/ps5-pie-unwind.ld` defines.
2. **Eight Mesa utility symbols** (`u_thread_create`, `u_thread_setname`,
`util_barrier_init/destroy/wait`, `util_thread_get_time_nano`,
`os_create_anonymous_file`, `os_read_file`). Their PS5 object list filters
`src/util/{u_thread,anon_file,os_file}.c` out of the compiler archive, and its own
`libvulkan.so.1` only links because a *shared* object may leave symbols undefined.
An executable may not. `tools/build-mesa-util.sh` compiles those three sources with
their PS5 configuration, because `u_thread.c`'s `util_barrier` layout has to match
the `u_queue.ps5.o` already inside the archive.
3. **Twelve `sceAgc*` entry points.** The console provides libSceAgc and
libSceAgcDriver and the SDK stubs neither, so they are declared in
`tooling/ps5-stubs/` for the converter to record as imports.
**The AGC stubs moved out of `vendor/`.** `vendor/` is not in the diff, so the
declarations the link needs would have been lost with the next checkout. They are
tracked under `tooling/ps5-stubs/` now, which is also where `tools/build.sh` reads
them.
**Not yet working on the console.** `bash tools/run-title.sh --watch 20` on this build:
the title launches, the console reports it running, and every one of its eight attempts
ends in `rarch_main returned = 1` with nothing in between - no `ps5_init`, no frame, no
menu. `ps5_init` is expected to be absent, because the compiled video default is now
`vulkan` and the config that names `ps5` is the one content loading discards (0006), but
*something* returns 1, and the frontend's own log (`/app0/retroarch.log`, fetched over
FTP) stops after the audio fallback with no fatal-error line: the setjmp handler in
`rarch_main` logs `Fatal error received in: "<what>"`, and that line is not there, so
the return is one of the silent paths after `drivers_init` - the leading candidate is
`task_push_load_content_from_cli` returning false (`retroarch.c:6036`).
**Read the log, not the trace, for this one.** `/app0/retroarch.log` is fetched with
`RETR /data/homebrew/<title>/retroarch.log`; `/app0/...` is not an FTP path. It proves
the frontend's own startup ran: input driver `ps5` found, audio fell back to null,
display server `null`, core geometry 320x240.
docs/PHASE_LOG.md
+65
View File
@@ -1131,3 +1131,68 @@ leaves the driver, the compiler and the SDK runtime with no symbol table);
that names an uncaught exception's type. `tools/apply-runtime-probes.py` carries
the probes that measured all of it - twenty-nine, applied with one command and
reverted before the shipping build.
## 2026-09-19: The driver initialises, the pipeline is created, and the first frame stops on an open render pass
**Where this round started.** A `tools/run-title.sh` run showed RetroArch selecting
`video_vulkan`, creating a device, a swapchain and four textures, compiling both stock
shaders into modules - and then `vkCreateGraphicsPipelines` answering
`VK_ERROR_UNKNOWN`, with `vulkan_init` returning NULL. Nothing rendered.
**What it took, in order.**
- **The topology.** `../PS5_Vulkan`'s pipeline check accepts
`VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST` alone (and Mesa's meta rectangle list), and its
draw path refuses a non-indexed draw whose first vertex is not zero. RetroArch's
Vulkan filter chain drew its two quads as a four-vertex triangle *strip*. Patches
0016 converts them to six-vertex lists, says triangle list, moves the final pass's
quad to the new offset, and draws each quad's second triangle through the binding's
own offset. `vkCreateGraphicsPipelines` then returned 0 and `vulkan_init` handed back
a real pointer.
- **Samplers.** This driver refuses any sampler whose address mode is not clamp-to-edge
and leaves the output handle untouched; `CommonResources` destroys every handle that
is not `VK_NULL_HANDLE`, so the sixteen refused ones were destroyed as if they were
real samplers and the driver asserted on the first. Patch 0017 clears the array first.
- **The runloop quit before its first frame.** `rarch_main` returned 0 with no frame
drawn. The display context's `check_window` sets `quit` when the frontend's signal
handler state is non-zero, and RetroArch's khr_display context read that as "the user
asked to quit". Patch 0018 stops this context treating it as one: a console title has
no terminal and no SIGTERM sender, and the platform's own lifecycle ends the process.
- **The value behind that state was garbage, and that is the round's real find.** It
read `-285230512`, stable across runs, before any signal had been delivered - because
**this port's CRT never zeroed the BSS**. The image's writable segment is
`0x104b4` bytes in the file and `0xc7040` in memory, and `_start`
(tooling/native/app_crt.cpp) went straight from `_init_env` to the static
constructors. Every zero-initialised object in the frontend, in the driver and in the
shader compiler therefore started with whatever the memory held: a counter, a
flag, a pointer. `tooling/native/ps5-pie.ld` now marks `__bss_start`/`__bss_end` and
`_start` clears that range before anything else. A run prints
`bss check=0 (must be 0), data check=7 (must be 7)`.
- **Assertions now say what they are.** `../PS5_Vulkan`'s `__assert` prints the
expression, file and line to stderr and then calls `abort`, which does not flush -
and the console's libc buffers stderr, so every assertion arrived as a bare
`abort is called(system)`. `src/main.cpp` points stderr at `/app0/trace.txt` **and
makes it unbuffered**. The trace now reads, for the current blocker:
probe SPAN: offscreen passes
probe SPAN: menu upload
probe SPAN: about to begin the backbuffer pass
probe REC: begin render pass
assertion failed: cmd_buffer->render_pass == NULL
(.../vulkan-runtime/src/vulkan/runtime/vk_render_pass.c:2648,
vk_common_CmdBeginRenderPass2)
**Where it stands.** The driver initialises, both shaders compile, the pipeline is
created, the runloop runs, and the first frame records commands until
`vkCmdBeginRenderPass` - which asserts because the frame's command buffer already has
a render pass open. The marks bracket it to the span between
`vulkan_filter_chain_build_offscreen_passes` and the backbuffer pass, which is the
first-frame history/feedback clear and the menu texture upload. Nothing reaches the
screen yet.
**Probes and their guard.** `tools/apply-runtime-probes.py` carries thirty-nine probes
and `tests/test_frontend.py` now checks the set itself: an insert that ends with its
own anchor duplicates the line it is inserted before, which broke the build three times
in this round (a duplicated `vkCreateImage`, a duplicated `switch (`, a duplicated
`vulkan_filter_chain_build_offscreen_passes(`); every probe must carry a marker, a
note and a trailing newline. All of it is checked before a probe can reach a build.
src/main.cpp
+19 -2
View File
@@ -94,6 +94,19 @@ int main()
{
/* First thing: prove that control reached this function at all, before
* anything that could fail. */
{
/* A zero-initialised static: it lives in the BSS, which this port's own
* CRT now clears before main. Printed so a run says whether that happened -
* the alternative was to read it from the far side of the frontend, where
* the same question cost a round. */
static long ps5_bss_check;
static long ps5_data_check = 7;
char line[128];
std::snprintf(line, sizeof line, "bss check=%ld (must be 0), data check=%ld (must be 7)",
ps5_bss_check, ps5_data_check);
ps5::debug::mark(line);
}
ps5::debug::mark("main() entered; static constructors have already run");
/* Everything the libraries say goes to stderr, and a title's stderr reaches
@@ -108,8 +121,12 @@ int main()
ps5::debug::mark("could not send stderr to the trace file");
else
{
std::fputs("stderr is the trace file\n", stderr);
std::fflush(stderr);
/* Unbuffered, because everything that writes an error and then aborts -
* an assertion, a C++ terminate - would otherwise lose it: the console's
* libc buffers this stream and abort does not flush. The assert message is
* what a console run needs most and it was the one thing never printed. */
std::setvbuf(stderr, nullptr, _IONBF, 0);
std::fputs("stderr is the trace file (unbuffered)\n", stderr);
}
std::set_terminate(on_terminate);
tests/test_frontend.py
+39 -1
View File
@@ -294,7 +294,7 @@ class PortPatches(unittest.TestCase):
change deliberately, in a commit that says why.
"""
self.assertEqual(
len(self.blocks()), 15,
len(self.blocks()), 21,
"the patch count changed: if a block was added or removed on purpose, "
"update this number in the same commit and say why in its message")
@@ -729,6 +729,44 @@ class LinkedDriver(unittest.TestCase):
f"{symbol} is not in the linked image; the layout no longer provides it")
class ProbeSet(unittest.TestCase):
"""The probe script's inserts are insertions, and are applied safely.
tools/apply-runtime-probes.py writes `insert + anchor`, so an insert that ends
with its own anchor duplicates that line - which is how a probe broke the build
three times in one session (a duplicated `vkCreateImage`, a duplicated
`switch (runloop_check_state(`, a duplicated
`vulkan_filter_chain_build_offscreen_passes(`). It is cheaper to check than to
find again.
"""
script = ROOT / "tools/apply-runtime-probes.py"
def probes(self) -> list[tuple[str, str, str, str, str]]:
import ast
tree = ast.parse(self.script.read_text(encoding="utf-8"))
for node in ast.walk(tree):
if isinstance(node, ast.Assign) and getattr(node.targets[0], "id", "") == "PROBES":
return ast.literal_eval(node.value)
raise AssertionError(f"{self.script} no longer defines PROBES")
def test_no_insert_repeats_its_own_anchor(self) -> None:
for name, anchor, insert, marker, _note in self.probes():
self.assertFalse(
insert.endswith(anchor),
f"{marker!r} in {name} ends with its own anchor, which would "
f"duplicate the line it is inserted before")
def test_every_probe_has_a_marker_and_a_note(self) -> None:
for name, _anchor, insert, marker, note in self.probes():
self.assertIn(marker, insert,
f"{marker!r} is not in its own insert text")
self.assertTrue(note.strip(), f"{marker!r} has no note")
self.assertTrue(insert.endswith("\n"),
f"{marker!r} does not end with a newline")
class Artifact(unittest.TestCase):
"""The built folder is a title the console can be asked to run."""
tooling/native/app_crt.cpp
+26
View File
@@ -20,6 +20,11 @@ extern "C"
[[noreturn]] void exit(int status);
int application_main(int argc, char **argv, char **envp) __asm__("main");
/* Provided by tooling/native/ps5-pie.ld: the range app_crt.cpp clears before
* anything can read a zero-initialised static. */
extern char __bss_start[];
extern char __bss_end[];
extern Initializer __preinit_array_start[] __attribute__((weak));
extern Initializer __preinit_array_end[] __attribute__((weak));
extern Initializer __init_array_start[] __attribute__((weak));
@@ -30,6 +35,23 @@ extern "C"
namespace
{
/* The console's loader does not zero the BSS, and the tail of this image's
* writable segment is BSS: the file image is 0x104b4 bytes where the segment is
* 0xc7040, so about three quarters of a megabyte of zero-initialised objects -
* RetroArch's, the Vulkan driver's and the shader compiler's - started with
* whatever the previous owner of that memory left behind.
*
* Found through one of those objects: RetroArch's signal-handler counter, whose
* value the display context reads to decide whether the user asked to quit, read
* -285230512 before any signal had been delivered. The runloop ended on its first
* iteration and rarch_main returned 0 without drawing a frame. Now the range the
* linker script marks is cleared before anything can read it. */
void zero_bss() noexcept
{
for (char *at = __bss_start; at != __bss_end; at++)
*at = 0;
}
void run_forward(Initializer *first, Initializer *last) noexcept
{
if (first == nullptr || last == nullptr)
@@ -70,6 +92,10 @@ _start(void *process_parameters, Destructor loader_teardown)
auto *parameters = static_cast<std::uint8_t *>(process_parameters);
auto **argv = reinterpret_cast<char **>(parameters + sizeof(std::uint64_t));
/* Before anything else: every static this program will read lives in the BSS
* the loader left alone, and _init_env may well be the first code to use one. */
zero_bss();
_init_env(process_parameters);
if (loader_teardown != nullptr)
(void)atexit(loader_teardown);
tooling/native/ps5-pie.ld
+5
View File
@@ -82,7 +82,12 @@ SECTIONS {
.tdata : { *(.tdata .tdata.*) } : data
.tbss : { *(.tbss .tbss.*) } : data
.bss : {
/* The console's loader does not zero the tail of this segment - the file
* image is 0x104b4 bytes and the segment is 0xc7040 - so app_crt.cpp
* needs its boundaries and zeroes it before anything reads a static. */
PROVIDE_HIDDEN(__bss_start = .);
*(.bss .bss.*)
*(COMMON)
PROVIDE_HIDDEN(__bss_end = .);
} : data
}
tools/apply-port-patches.py
+121
View File
@@ -480,6 +480,127 @@ EDITS = [
" default: /* Unknown format */\n",
"patches/series, 0015",
),
(
# ../PS5_Vulkan's pipeline check accepts only VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST
# (and Mesa's meta rectangle list), and its draw path refuses a non-indexed
# draw whose first vertex is not zero. RetroArch's Vulkan filter chain draws
# its two quads as a four-vertex triangle strip starting at vertex 0, so its
# pipeline is refused with VK_ERROR_UNKNOWN and no log line - that project
# compiles its own Mesa log out. The three edits below answer both
# conditions: the quads become six explicit vertices each in list order, the
# pipeline says triangle list, the final pass reads its quad at the new
# offset, and the second triangle is drawn through the binding's own offset
# because a first vertex other than zero is refused.
"gfx/drivers_shader/shader_vulkan.cpp",
" input_assembly.primitiveRestartEnable = VK_FALSE;\n",
" /* Added by this port (patches/series, 0016): this driver takes triangle\n"
" * lists only. */\n"
" input_assembly.topology = VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST;\n"
" input_assembly.primitiveRestartEnable = VK_FALSE;\n",
"VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST;\n"
" input_assembly.primitiveRestartEnable",
),
(
"gfx/drivers_shader/shader_vulkan.cpp",
" vbo->unmap();\n",
" /* Added by this port (patches/series, 0016): the strip above cannot carry a\n"
" * list's six vertices, so the buffer is rebuilt in list order: each quad as\n"
" * two triangles, the second one reachable at a three-vertex offset because a\n"
" * non-indexed draw may not start anywhere but vertex zero. */\n"
" vbo.reset();\n"
" {\n"
" static const float ps5_triangle_list[] = {\n"
" /* Offscreen: (-1,-1) (-1,+1) (1,-1), then (1,-1) (-1,+1) (1,+1) */\n"
" -1.0f, -1.0f, 0.0f, 0.0f,\n"
" -1.0f, +1.0f, 0.0f, 1.0f,\n"
" +1.0f, -1.0f, 1.0f, 0.0f,\n"
" +1.0f, -1.0f, 1.0f, 0.0f,\n"
" -1.0f, +1.0f, 0.0f, 1.0f,\n"
" +1.0f, +1.0f, 1.0f, 1.0f,\n"
" /* Final: (0,0) (0,1) (1,0), then (1,0) (0,1) (1,1) */\n"
" 0.0f, 0.0f, 0.0f, 0.0f,\n"
" 0.0f, +1.0f, 0.0f, 1.0f,\n"
" +1.0f, 0.0f, 1.0f, 0.0f,\n"
" +1.0f, 0.0f, 1.0f, 0.0f,\n"
" 0.0f, +1.0f, 0.0f, 1.0f,\n"
" +1.0f, +1.0f, 1.0f, 1.0f,\n"
" };\n"
" vbo = std::unique_ptr<Buffer>(new Buffer(device, memory_properties,\n"
" sizeof(ps5_triangle_list), VK_BUFFER_USAGE_VERTEX_BUFFER_BIT));\n"
" ptr = vbo->map();\n"
" memcpy(ptr, ps5_triangle_list, sizeof(ps5_triangle_list));\n"
" vbo->unmap();\n"
" }\n"
" vbo->unmap();\n",
"ps5_triangle_list",
),
(
"gfx/drivers_shader/shader_vulkan.cpp",
" vkCmdBindVertexBuffers(cmd, 0, 1,\n",
" /* Added by this port (patches/series, 0016): the final quad is the second\n"
" * of the two list quads, so it starts three vertices later than it did as a\n"
" * strip. */\n"
" if (final_pass)\n"
" offset = 24 * sizeof(float);\n"
" vkCmdBindVertexBuffers(cmd, 0, 1,\n",
"offset = 24 * sizeof(float);",
),
(
"gfx/drivers_shader/shader_vulkan.cpp",
" vkCmdDraw(cmd, 4, 1, 0, 0);\n",
" /* Added by this port (patches/series, 0016): a triangle list uses three of\n"
" * these four vertices, and the quad's other triangle is drawn here through the\n"
" * binding's offset - naming it as a first vertex would be refused. */\n"
" {\n"
" const VkDeviceSize base = final_pass ? 24 * sizeof(float) : 0;\n"
" const VkDeviceSize second = base + 3 * 4 * sizeof(float);\n"
" VkBuffer buffer = common->vbo->get_buffer();\n"
" vkCmdBindVertexBuffers(cmd, 0, 1, &buffer, &second);\n"
" vkCmdDraw(cmd, 3, 1, 0, 0);\n"
" vkCmdBindVertexBuffers(cmd, 0, 1, &buffer, &base);\n"
" }\n"
" vkCmdDraw(cmd, 4, 1, 0, 0);\n",
"vkCmdDraw(cmd, 3, 1, 0, 0);",
),
(
# This driver refuses a sampler whose address mode is not clamp-to-edge, and
# a refused vkCreateSampler leaves its output handle untouched. RetroArch's
# CommonResources destroys every handle that is not VK_NULL_HANDLE, so the
# sixteen samplers this driver refuses are destroyed as if they were real
# samplers and the driver asserts on the first one. Zeroing the array first
# makes a refused creation the NULL it is.
"gfx/drivers_shader/shader_vulkan.cpp",
" info.unnormalizedCoordinates = VK_FALSE;\n",
" info.unnormalizedCoordinates = VK_FALSE;\n"
" /* Added by this port (patches/series, 0017): ../PS5_Vulkan supports the\n"
" * clamp-to-edge samplers alone and returns an error for the rest, leaving\n"
" * the handle it was given untouched - so the array is cleared before it is\n"
" * filled, and a refused creation stays VK_NULL_HANDLE for the destructor. */\n"
" memset(samplers, 0, sizeof(samplers));\n",
"memset(samplers, 0, sizeof(samplers));",
),
(
# A console title has no terminal and no SIGTERM sender, and this console
# delivers SIGINT/SIGTERM while a title starts. RetroArch's khr_display
# context read that state as "the user asked to quit", so the runloop ended
# on its first iteration: the trace showed a successful vulkan_init, a
# created pipeline, and rarch_main returning 0 without a single frame. The
# platform's own lifecycle ends the process, so this context stops asking
# the runloop to.
"gfx/drivers_context/khr_display_ctx.c",
"}\n"
"\n"
"static bool gfx_ctx_khr_display_set_resize(void *data,\n",
" /* Added by this port (patches/series, 0018): the signal-handler state is\n"
" * not a quit request on a console. */\n"
" *quit = false;\n"
"}\n"
"\n"
"static bool gfx_ctx_khr_display_set_resize(void *data,\n",
"the signal-handler state is",
),
]
tools/apply-runtime-probes.py
+130 -69
View File
@@ -349,7 +349,7 @@ PROBES = [
" { FILE *f = fopen(\"/app0/trace.txt\", \"a\");\n"
" if (f) {\n"
" unsigned bi, ai;\n"
" fprintf(f, \"probe PIPE2: renderPass=%p stages=%u verts=%u attrs=%u topo=%d \"\n"
" fprintf(f, \"probe PIPE3: renderPass=%p stages=%u verts=%u attrs=%u topo=%d \"\n"
" \"blendAtt=%u blendEnable=%d writeMask=0x%x depthTest=%d samples=0x%x\\n\",\n"
" (void*)pipe.renderPass, (unsigned)pipe.stageCount,\n"
" (unsigned)(pipe.pVertexInputState ? pipe.pVertexInputState->vertexBindingDescriptionCount : 0),\n"
@@ -363,13 +363,13 @@ PROBES = [
" (int)(pipe.pDepthStencilState ? pipe.pDepthStencilState->depthTestEnable : 0),\n"
" (unsigned)(pipe.pMultisampleState ? pipe.pMultisampleState->rasterizationSamples : 0));\n"
" for (bi = 0; pipe.pVertexInputState && bi < pipe.pVertexInputState->vertexBindingDescriptionCount; bi++)\n"
" fprintf(f, \"probe PIPE2: binding %u = {%u, %u}\\n\", bi,\n"
" fprintf(f, \"probe PIPE3: binding %u = {%u, %u}\\n\", bi,\n"
" (unsigned)pipe.pVertexInputState->pVertexBindingDescriptions[bi].binding,\n"
" (unsigned)pipe.pVertexInputState->pVertexBindingDescriptions[bi].stride);\n"
" for (ai = 0; pipe.pVertexInputState && ai < pipe.pVertexInputState->vertexAttributeDescriptionCount; ai++)\n"
" { const VkVertexInputAttributeDescription *a =\n"
" &pipe.pVertexInputState->pVertexAttributeDescriptions[ai];\n"
" fprintf(f, \"probe PIPE2: attr %u = {loc %u, bind %u, fmt %d, off %u}\\n\",\n"
" fprintf(f, \"probe PIPE3: attr %u = {loc %u, bind %u, fmt %d, off %u}\\n\",\n"
" ai, (unsigned)a->location, (unsigned)a->binding, (int)a->format, (unsigned)a->offset); }\n"
" fclose(f);\n"
" } }\n",
@@ -400,74 +400,135 @@ PROBES = [
"reads, which is the other refusal it can give before compiling anything",
),
(
# The compiler's header, at file scope: it opens an extern "C" block, which
# cannot be included inside a function body.
"gfx/drivers_shader/shader_vulkan.cpp",
"#include \"../include/vulkan/vk_sdk_platform.h\"\n",
"/* Added by this port (probes): the shader compiler's header, so the probe that\n"
" * asks it directly about the stock shader can name its types. It belongs to\n"
" * ../PS5_Vulkan and is only used by that probe. */\n"
"#include \"../../../PS5_Vulkan/.deps/native/psbc/include/psbc_compile.h\"\n"
"#include \"../include/vulkan/vk_sdk_platform.h\"\n",
"#include \"../../../PS5_Vulkan/.deps/native/psbc/include/psbc_compile.h\"",
"the compiler's own types, for the probe that calls it",
# The instrument the objective itself needs: with the Vulkan driver the trace
# has no frame line at all - src/display.cpp belongs to the CPU driver - so a
# run could only say that init succeeded or failed. This counts the frames the
# frontend hands the driver and writes one line every thirty, which is what
# "frames reaching the screen" looks like from inside the title.
"gfx/drivers/vulkan.c",
" int i, j, k;\n"
" VkSubmitInfo submit_info;\n",
" { static unsigned long probe_frames = 0;\n"
" static unsigned long probe_reported = 0;\n"
" probe_frames++;\n"
" if (probe_frames - probe_reported >= 30)\n"
" {\n"
" FILE *f;\n"
" probe_reported = probe_frames;\n"
" f = fopen(\"/app0/trace.txt\", \"a\");\n"
" if (f) { fprintf(f, \"probe FRAME: %lu handed to the driver\\n\", probe_frames);\n"
" fclose(f); }\n"
" } }\n",
"probe FRAME:",
"how many frames the Vulkan path is actually presenting",
),
(
# Both stock shaders compile with the host build of this compiler, using
# the driver's own options (UBO stride 16, sampler at binding 2, stride 48).
# The driver refuses the pipeline anyway, and its message is compiled out,
# so the compiler is asked here on the console with the same options - the
# one thing a host test cannot reproduce. psbc_init() is left alone: the
# driver has already initialised the library by the time a pipeline is
# built, and calling it again is what crashed the first version of this
# probe.
"gfx/drivers_shader/shader_vulkan.cpp",
" { VkPipeline probe_pipeline = VK_NULL_HANDLE;\n",
" {\n"
" PsbcCompileOptions probe_options;\n"
" PsbcShaderOutput probe_output;\n"
" PsbcResult probe_result;\n"
" FILE *f;\n"
"\n"
" memset(&probe_options, 0, sizeof(probe_options));\n"
" memset(&probe_output, 0, sizeof(probe_output));\n"
" probe_options.target = PSBC_TARGET_PS5;\n"
" probe_options.optimise = true;\n"
" probe_options.entrypoint = \"main\";\n"
" probe_options.address32_hi = 2u;\n"
" probe_options.vertex_attributes[0] = (PsbcVertexAttribute){\n"
" .location = 0, .binding = 0, .format = PSBC_VERTEX_FORMAT_R32G32_FLOAT,\n"
" .offset = 0, .stride = 16, .alignment = 4};\n"
" probe_options.vertex_attributes[1] = (PsbcVertexAttribute){\n"
" .location = 1, .binding = 0, .format = PSBC_VERTEX_FORMAT_R32G32_FLOAT,\n"
" .offset = 8, .stride = 16, .alignment = 4};\n"
" probe_options.vertex_attribute_count = 2;\n"
" probe_options.descriptor_bindings[0] = (PsbcDescriptorBinding){\n"
" .set = 0, .binding = 0, .type = PSBC_DESCRIPTOR_UNIFORM_BUFFER,\n"
" .array_size = 1, .offset = 0, .stride = 16};\n"
" probe_options.descriptor_bindings[1] = (PsbcDescriptorBinding){\n"
" .set = 0, .binding = 2, .type = PSBC_DESCRIPTOR_COMBINED_IMAGE_SAMPLER,\n"
" .array_size = 1, .offset = 16, .stride = 48};\n"
" probe_options.descriptor_binding_count = 2;\n"
"\n"
" probe_options.stage = PSBC_STAGE_VERTEX;\n"
" probe_result = psbc_compile_shader(vertex_shader.data(), vertex_shader.size(),\n"
" &probe_options, &probe_output);\n"
" f = fopen(\"/app0/trace.txt\", \"a\");\n"
" if (f) { fprintf(f, \"probe PSBC: vertex -> %d (%s)\\n\", (int)probe_result,\n"
" psbc_result_string(probe_result)); fclose(f); }\n"
" psbc_free_output(&probe_output);\n"
" memset(&probe_output, 0, sizeof(probe_output));\n"
" probe_options.stage = PSBC_STAGE_FRAGMENT;\n"
" probe_result = psbc_compile_shader(fragment_shader.data(), fragment_shader.size(),\n"
" &probe_options, &probe_output);\n"
" f = fopen(\"/app0/trace.txt\", \"a\");\n"
" if (f) { fprintf(f, \"probe PSBC: fragment -> %d (%s)\\n\", (int)probe_result,\n"
" psbc_result_string(probe_result)); fclose(f); }\n"
" psbc_free_output(&probe_output);\n"
" }\n",
"probe PSBC:",
"the console's own answer for the stock shader, with the driver's options",
# The runloop quit on its first iteration and rarch_main returned 0 without a
# frame. runloop_check_state's only reachable quit path is the core-shutdown
# flag, and the video driver's alive() turns false when the display context's
# check_window sets quit - which it does when the frontend's signal handler
# state is non-zero. Both are printed here.
"gfx/drivers/vulkan.c",
" if (quit)\n"
" vk->flags |= VK_FLAG_QUITTING;\n",
" { FILE *f = fopen(\"/app0/trace.txt\", \"a\");\n"
" if (f) { fprintf(f, \"probe ALIVE: quit=%d resize=%d width=%u height=%u\\n\",\n"
" (int)quit, (int)resize, temp_width, temp_height); fclose(f); } }\n",
"probe ALIVE:",
"whether the display context is asking the runloop to stop, and why",
),
(
# The quit on the first iteration comes from this flag, and this is the only
# place that sets it: a libretro core asking the frontend to shut down. With
# no content loaded the core is the dummy one the menu runs on, so the line
# that follows says which environment call did it.
"runloop.c",
"#ifdef HAVE_MENU\n"
" /* Ensure that menu stack is flushed appropriately\n",
" { FILE *f = fopen(\"/app0/trace.txt\", \"a\");\n"
" if (f) { fprintf(f, \"probe SHUTDOWN: a core asked to shut down\\n\"); fclose(f); } }\n"
" runloop_st->flags |= RUNLOOP_FLAG_CORE_SHUTDOWN_INITIATED\n",
"probe SHUTDOWN:",
"which trigger quit the runloop before the first frame",
),
(
# Which signal, and how early: the answer to why the runloop stopped before
# its first frame, and whether the console sends it or something in the
# title does.
"frontend/drivers/platform_unix.c",
" (void)sig;\n",
" { FILE *f = fopen(\"/app0/trace.txt\", \"a\");\n"
" if (f) { fprintf(f, \"probe SIGNAL: %d caught, count now %d\\n\",\n"
" sig, (int)(unix_sighandler_quit + 1)); fclose(f); } }\n"
" (void)sig;\n"
" unix_sighandler_quit++;\n",
"probe SIGNAL:",
"which signal ends the runloop, and how early",
),
(
'gfx/drivers/vulkan.c',
' vkCmdBeginRenderPass(vk->cmd, &rp_info, VK_SUBPASS_CONTENTS_INLINE);\n',
' { FILE *f = fopen("/app0/trace.txt", "a");\n if (f) { fprintf(f, "probe REC: begin render pass\\n"); fclose(f); } }\n',
'probe REC:',
"how far the first frame's recording gets",
),
(
'gfx/drivers/vulkan.c',
' vkCmdEndRenderPass(vk->cmd);\n',
' { FILE *f = fopen("/app0/trace.txt", "a");\n if (f) { fprintf(f, "probe REC: end render pass\\n"); fclose(f); } }\n',
'probe REC:',
'the far side of the pass: present means the refusal is after it',
),
(
'gfx/drivers/vulkan.c',
' vulkan_filter_chain_end_frame((vulkan_filter_chain_t*)filter_chain, vk->cmd);\n',
' { FILE *f = fopen("/app0/trace.txt", "a");\n if (f) { fprintf(f, "probe REC: chain end frame\\n"); fclose(f); } }\n',
'probe REC:',
"the chain's own recording, which is where the offscreen passes and the menu copy are",
),
(
'gfx/drivers/vulkan.c',
' vkEndCommandBuffer(vk->cmd);\n',
' { FILE *f = fopen("/app0/trace.txt", "a");\n if (f) { fprintf(f, "probe REC: ending the command buffer\\n"); fclose(f); } }\n',
'probe REC:',
'the last mark before the submit that asserts',
),
(
# The frame's command buffer already has a render pass open when
# vkCmdBeginRenderPass runs, so something between the begin and this line
# opened one and did not end it. These four marks bracket that span.
"gfx/drivers/vulkan.c",
" vkBeginCommandBuffer(vk->cmd, &begin_info);\n",
" { FILE *f = fopen(\"/app0/trace.txt\", \"a\");\n"
" if (f) { fprintf(f, \"probe SPAN: command buffer recording\\n\"); fclose(f); } }\n",
"probe SPAN: command buffer recording",
"the start of the span",
),
(
"gfx/drivers/vulkan.c",
" vulkan_filter_chain_build_offscreen_passes(\n",
" { FILE *f = fopen(\"/app0/trace.txt\", \"a\");\n"
" if (f) { fprintf(f, \"probe SPAN: offscreen passes\\n\"); fclose(f); } }\n",
"probe SPAN: offscreen passes",
"the chain's offscreen passes: a pass begun and not ended here would be the one",
),
(
"gfx/drivers/vulkan.c",
" if (vk->flags & VK_FLAG_MENU_ENABLE)\n",
" { FILE *f = fopen(\"/app0/trace.txt\", \"a\");\n"
" if (f) { fprintf(f, \"probe SPAN: menu upload\\n\"); fclose(f); } }\n",
"probe SPAN: menu upload",
"the menu texture copy, recorded into this same command buffer",
),
(
"gfx/drivers/vulkan.c",
" rp_info.sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO;\n",
" { FILE *f = fopen(\"/app0/trace.txt\", \"a\");\n"
" if (f) { fprintf(f, \"probe SPAN: about to begin the backbuffer pass\\n\"); fclose(f); } }\n",
"probe SPAN: about to begin the backbuffer pass",
"the last mark before the assert",
),
]