Add opt-in XMB allocation diagnostics for manual console testing

Keep native/mapped allocation policy intact; observe aligned Vulkan host allocations and texture/queue-idle lifetimes with allocation-free failure logging and periodic summaries. Snapshot driver archives locally for an attributable diagnostic link.

Verified: PS5_MEMORY_DIAGNOSTICS=1 bash tools/verify.sh; final bash tools/verify.sh format unit (68 tests); python3 tools/check-memory-diagnostics.py. Evidence: evidence/xmb-memory-diagnostic/{capture,expectation,inspection}.json. Console validation pending owner intervention; no upload or launch and no crash fix claimed.
This commit is contained in:
Mihawk committed 2026-09-19 18:30:27 -04:00
1 parent a18cec63ef
commit 971534cd37
17 files changed
+1089 -54

No files matched your search

+34 -29
View File
@@ -4,16 +4,38 @@ _Updated: 2026-09-19_
## Now
**Genesis Plus GX is ported and owner-confirmed working.** The owner reports
“Sega Genesis works flawlessly” and explicitly confirms “Yes, transitions and
next game are clean.” Build and console evidence: `evidence/genesis-plus-gx-native/`.
No next core is selected.
**XMB allocation diagnostic is built on `codex/xmb-allocation-diagnostics`.**
This is instrumentation, not a crash fix. Owner explicitly requests no automatic
upload or launch while working on another project. Wait for their intervention.
The baseline's next-core work is not the assignment; this crash investigation is.
The public README now describes this verified baseline, with a table of contents,
scoped core coverage, roadmap, native build/install guidance and upstream credits.
This documentation update does not change the accepted binaries.
- Build identity: `000adf3edc2538bc563f833f0ccd2e91a01053aa6dac19dd568a580cdbbbb3cb`.
- Output: `dist/PPSA99169/`; preserved executable/symbols/map/inspection under
`klog/xmb-memory-000adf3edc25/`. Evidence: `evidence/xmb-memory-diagnostic/`.
- `PS5_MEMORY_DIAGNOSTICS=1 bash tools/verify.sh` passed all five host gates.
Final format/unit check passed; 68 tests. Raw: `klog/xmb-memory-verify.log`,
`klog/xmb-memory-final-checks.log`. `tools/check-memory-diagnostics.py` passes.
- No upload, launch, console settings change or PS5_Vulkan modification.
Current libps5vk differs from the crashing build; local archive snapshots and
their hashes identify the diagnostic's inputs. Other three archives match.
- Log: `/app0/memory-diagnostics.log`; periodic requested live bytes/counts/peaks,
caller groups, image/queue-idle counters; immediate allocation failure records.
Metadata saturation/coverage gaps are explicit. No allocator policy change.
- Reproduction and interpretation: `docs/MEMORY_DIAGNOSTICS.md`. Console validation
remains pending, including whether the crash reproduces with the newer driver.
## Verified build and evidence
## Crash evidence
Owner reproduced rapid XMB navigation crash. Raw capture:
`klog/xmb-crash-20260919-181233/` (kernel, frontend, trace, config, diagnosis).
Matching baseline symbols locate SIGSEGV at runtime `0x45b1ae` in
`__vk_log_impl`, reached through `vk_sync_create` allocation failure inside
`vkQueueWaitIdle`, called from XMB texture unload. Logger's second allocation
returns NULL and is dereferenced. Why allocation failed is unresolved: heap
exhaustion, fragmentation, retention/leak, unobserved usage or corruption remain
possible. Do not call this a GPU hang or a resolved frontend/driver leak.
## Previous console-verified baseline (Genesis Plus GX)
- Frontend identity: `ecfcddd57febbb484c2e0724e95f43bbffb3ddd9b9c22aa9c1cf3e37af4a9445`.
- Genesis Plus GX source: `c2838c7dc4236fc2fe94e5dbd08b41486067918e`.
@@ -44,21 +66,6 @@ This documentation update does not change the accepted binaries.
- PS5_Vulkan was not modified. Four archive hashes matched before/after build;
exact linked hashes are in the evidence. The owner develops it concurrently.
## What changed
- `make genesis-plus-gx`: pinned official source and metadata using explicit
project SDK wrappers, native import union and .info in both lookup locations.
Attached metadata was reference only. No ROM or BIOS is shipped.
- Preserve RGB565 software rendering, filters/cursors and cached bitmap; convert
into a separate bounded XRGB8888 callback buffer. Honour cropped byte offsets
and the 720-pixel row pitch. The frontend's existing RGBA upload/cropping path
remains unchanged, as does the Vulkan driver.
- Disable host-detected physical CD-ROM support and optional zstd weak tracing
hooks. Keep CHD/disc-image codecs and useful logs. Correct update_geometry's
unused return type. Reasons/pins and procedures are in REFERENCE/TESTING/DEPLOYMENT.
- Host tests cover every RGB565 colour through upstream packing and the actual
GPU upload helper, viewport changes/crops, cached frames and rejected bounds.
## Named errors and remaining limits
- First frontend snapshot has five intentional missing-core recovery errors and
@@ -93,9 +100,7 @@ See `docs/DEPLOYMENT.md` for procedures and metadata-listed BIOS filenames.
## Operating notes
Uploads/runs are authorized without further permission. Never interrupt an
existing title; the runner checks idle before deployment and launch. Core/game
selection is manual. Keep private filenames and console details in ignored logs.
Separate appended trace launches by bss check and build identity. Preserve live
logs before relaunch. Owner supplies visual confirmation; camera cadence can
hide flicker. Do not infer manual closure solely from the final process count.
Console testing is paused by the owner's explicit instruction for this step.
When they are ready, start passive klog before reproduction and preserve current
logs. Never interrupt another title. After a crash, capture logs before relaunch.
No new core or unrelated driver work is authorized by the active-file context.
+110
View File
@@ -0,0 +1,110 @@
# XMB allocation diagnostic build
This is an opt-in diagnostic, not a fix for the XMB crash. It observes allocation
lifetimes without increasing the heap, redirecting additional allocations to
`mmap`, or suppressing Vulkan errors. PS5_Vulkan is read only for this work.
## Build and host verification
```sh
PS5_MEMORY_DIAGNOSTICS=1 bash tools/verify.sh
```
The normal build (variable absent or `0`) compiles out the observer. The diagnostic
adds `PS5_MEMORY_DIAGNOSTICS` to the title definitions and
`--wrap=posix_memalign` to the existing malloc/calloc/realloc/free wrappers because
Mesa's default Vulkan host allocator uses aligned native allocations. Mode and
archive contents participate in the runtime build identity.
The diagnostic build copies the four driver archives into
`build/memory-diagnostic-inputs/` and links those copies. `archives.json` records
their SHA-256 digests. A copy that changes during the read is rejected. This does
not rebuild or write into the sibling project; it also does not imply the copied
archives are the same driver as an earlier console run. Mesa utility sources are
still read from the configured driver source tree, and the resulting object
bytes participate in the identity.
Output is `dist/PPSA99169/`. Preserve `build/llvm-pie.elf`, `build/title.map`,
`build/title_build_identity.h`, the manifest, and the archive digests before any
subsequent build. Caller addresses must be decoded against the exact matching ELF.
Host tests exercise allocation/free accounting, mapped/native realloc migration,
failed realloc retention, foreign pointers, aligned allocation semantics,
concurrency, metadata saturation, five-second sampling with an injected clock,
and logging while the wrapped allocator fails. Vulkan mocks check that image and
queue-idle observers preserve arguments, return values and repeat registration.
These tests do not establish console stability or identify the memory consumer.
## Manual console test — only when the owner is ready
Do not upload, launch, change live settings, or stop another title automatically.
Start a passive klog listener before the owner's reproduction and preserve the
existing logs first. Deploy the diagnostic using the project's normal folder
procedure, preserving live configuration and user content. The owner launches it.
1. Leave XMB idle for 30 seconds.
2. Browse rapidly for 30 seconds, then leave it idle for 30 seconds. Repeat.
3. Repeat the RGUI to XMB transition that triggered the crash. Note the approximate
elapsed time of each phase, including whether switching required restarting.
4. On a crash, leave the title closed until logs have been downloaded.
Download `/data/homebrew/PPSA99169/memory-diagnostics.log`, `retroarch.log`,
`trace.txt`, and the live configuration, along with the kernel capture. Keep raw
files under ignored `klog/`; they may contain user paths and process information.
In the title the same new log is `/app0/memory-diagnostics.log`. It appends a
`session` line with build identity and PID each launch rather than erasing the
previous run. No runner or deployment behavior is changed by this diagnostic.
## Reading the capture
`initial`, `sample`, `failure-summary` and `final` rows contain:
- `native_bytes/count/peak`: requested live bytes, live count and peak live bytes
from observed ordinary native allocations.
- `aligned_bytes/count/peak`: the same for `posix_memalign`; this is also native
memory, not a separate heap. Native pressure includes both native and aligned.
- `mapped_bytes/count/peak`: requests routed through the existing large-buffer
mappings. These exclude mapping headers, page rounding and allocator overhead.
- `failures`: cumulative failures; `failure` rows immediately record operation,
requested bytes, alignment (or element count for calloc overflow), error and
caller return address (`pc`). `calloc-overflow` is an arithmetic rejection,
not measured heap exhaustion.
- `dropped`: allocation records omitted because the fixed table was full. Any
nonzero value makes live totals incomplete for the rest of that session.
- `foreign_frees/reallocs`: pointers whose allocation was not observed (including
records dropped on saturation). Never inspect private libc allocation headers.
- `image_create/destroy/failed`: successful frontend Vulkan image creations,
non-null destruction calls and failed creations. These include images other
than menu textures and exclude calls internal to the driver.
- `idle_begin/end/failed`: queue-idle calls started, returned and returned failure.
A missing completion in a failure summary helps locate the failing operation.
Every five seconds while Vulkan presents, a summary and up to eight largest live
allocation caller groups are emitted. Groups contain `pc`, requested live bytes
and count; `site_records_omitted` reports overflow of the separate 2,048-site
aggregation table. They are not full stack traces. No frame means no periodic
sample; allocation failures still write synchronously. Clean frontend return
writes `final`; a crash need not.
Logging uses stack formatting, a preopened descriptor and `write`, without stdio
or heap allocation. A fixed 131,072-entry pointer table plus state bytes uses
about 4.13 MiB of static storage. Mutexes, table operations, scans and synchronous
writes add overhead, so use this build to diagnose lifetimes, not benchmark FPS.
The allocation route and payload size remain unchanged.
Coverage is title/core/static-library references intercepted by the linker.
Allocations performed internally by system libraries, direct mappings outside the
existing large-buffer wrapper, GPU direct memory, and allocator bookkeeping are
not measured. Requested live bytes are not the console's total memory usage.
For a `pc` from this title, subtract the image load base recorded in klog (normally
`0x400000`) and one byte for a return address, then use
`llvm-addr2line -C -f -e <matching-symbols.elf> <offset>`. A fault instruction
address is decoded without the one-byte adjustment. Keep symbols locally; never
publish raw memory addresses or local paths as sanitized evidence by accident.
Compare settled idle samples from the same session. Sustained growth is a reason
to investigate retained allocations; it alone does not prove a leak. Stable
observed totals plus allocation failures leave unobserved native usage,
fragmentation, heap limits and corruption open. The crash's Mesa null dereference
while reporting an allocation failure remains unfixed by this instrumentation.
+34
View File
@@ -2313,3 +2313,37 @@ Documentation-only validation:
existing committed artifacts, especially evidence/genesis-plus-gx-native/.
- `git diff --check`: PASS. No source, dependency, binary or console changes;
the accepted gameplay build remains 93a1332's recorded build identity.
## 2026-09-19 — XMB allocation diagnostic branch, host verified only
Owner requested a branch and diagnostic build, explicitly withholding automatic
upload/launch while working on another project. Created
`codex/xmb-allocation-diagnostics`. Added opt-in live allocation accounting to the
existing native/mapped wrappers, plus observation of `posix_memalign`, frontend
Vulkan image lifetimes and queue-idle calls. Fixed-capacity metadata and stack-only
`write` logging preserve OOM evidence without allocating. Allocation policy is
unchanged. Five-second summaries include counts, requested bytes, peaks, largest
caller groups, failure counts and explicit coverage/saturation limits.
The prior capture (`klog/xmb-crash-20260919-181233/`) maps the null write to Mesa's
error-reporting allocation following `vk_sync_create` allocation failure in
`vkQueueWaitIdle` during XMB texture unload. That diagnoses the immediate crash,
not the reason memory allocation failed. No driver fix or console result is claimed.
Command: `PS5_MEMORY_DIAGNOSTICS=1 bash tools/verify.sh` — PASS all five host gates.
The first unit pass had 67 tests; the added Vulkan dispatch test ran in the final
integration pass and final `bash tools/verify.sh format unit` (68 tests, PASS).
`python3 tools/check-memory-diagnostics.py` — PASS required symbols and archive
hashes. Machine-readable inspection and capture/expectation:
`evidence/xmb-memory-diagnostic/`. Raw host logs are in `klog/xmb-memory-verify.log`
and `klog/xmb-memory-final-checks.log`. Existing upstream core/compiler warnings
remain in the full build log; no diagnostic-source warning was observed.
Build identity: `000adf3edc2538bc563f833f0ccd2e91a01053aa6dac19dd568a580cdbbbb3cb`.
Output: `dist/PPSA99169/`; matching executable, symbol ELF and map preserved under
`klog/xmb-memory-000adf3edc25/`. The current libps5vk archive differs from the
crashing baseline; the other three archives match its recorded digests. The
opt-in build snapshots all four locally to avoid mutable archive inputs while
the owner develops PS5_Vulkan. Mode and linked bytes participate in identity.
No sibling writes, console access, upload, launch or settings change occurred.
Manual protocol and limitations are in `docs/MEMORY_DIAGNOSTICS.md`.
+7
View File
@@ -330,3 +330,10 @@ Capture frontend/trace/kernel logs, match the exact build identity, and check fo
Vulkan refusals, GPU API failures, invalid bitmap views and unexpected runtime
errors before committing. Record systems actually tested and preserve user
confirmation separately from machine-readable loader and runtime evidence.
## XMB allocation diagnostics
The opt-in build, host allocator/Vulkan-dispatch tests, raw log fields, ownership
coverage limits and manual reproduction protocol are specified in
[MEMORY_DIAGNOSTICS.md](MEMORY_DIAGNOSTICS.md). Its build inspection is host-only;
it must not be cited as a passing console reproduction or a fixed crash.
@@ -0,0 +1,19 @@
{
"step": "xmb-memory-diagnostic",
"recorded": "2026-09-19",
"revision": "a18cec6 + allocation diagnostics",
"command": "PS5_MEMORY_DIAGNOSTICS=1 bash tools/verify.sh; bash tools/verify.sh format unit; python3 tools/check-memory-diagnostics.py",
"console_run": null,
"raw_capture": "klog/xmb-memory-verify.log",
"lines": [
"Scope: host-verified diagnostic build only; no console upload or launch.",
"Diagnostic build identity: 000adf3edc2538bc563f833f0ccd2e91a01053aa6dac19dd568a580cdbbbb3cb",
"Allocation semantics and OOM logging: PASS; concurrency and saturation: PASS.",
"Five-second sampling with injected clock: PASS; Vulkan dispatch preservation: PASS.",
"Host verification: PASS format unit build integration evidence; final suite 68 tests.",
"Diagnostic symbols and pinned driver snapshots: PASS.",
"Console reproduction: PENDING. XMB crash is not fixed by this change."
],
"raw_lines": 6265,
"raw_bytes": 3051670
}
@@ -0,0 +1,17 @@
{
"step": "xmb-memory-diagnostic",
"note": "Host-only diagnostic readiness. Console test deliberately pending owner intervention.",
"must_contain": [
"Scope: host-verified diagnostic build only; no console upload or launch.",
"Diagnostic build identity: 000adf3edc2538bc563f833f0ccd2e91a01053aa6dac19dd568a580cdbbbb3cb",
"Allocation semantics and OOM logging: PASS; concurrency and saturation: PASS.",
"Five-second sampling with injected clock: PASS; Vulkan dispatch preservation: PASS.",
"Host verification: PASS format unit build integration evidence; final suite 68 tests.",
"Diagnostic symbols and pinned driver snapshots: PASS.",
"Console reproduction: PENDING. XMB crash is not fixed by this change."
],
"must_not_contain": [
"Console reproduction: PASS"
],
"human_check": "Not performed: owner requested no upload or launch."
}
@@ -0,0 +1,30 @@
{
"kind": "host-only-diagnostic-inspection",
"passed": true,
"build_identity": "000adf3edc2538bc563f833f0ccd2e91a01053aa6dac19dd568a580cdbbbb3cb",
"symbols_present": [
"__wrap_malloc",
"__wrap_calloc",
"__wrap_realloc",
"__wrap_free",
"__wrap_posix_memalign",
"ps5::memory::init(",
"ps5::memory::failure(",
"ps5::memory::tick(",
"memory_create(",
"memory_destroy(",
"memory_idle("
],
"artifacts_sha256": {
"build/llvm-pie.elf": "8f89351d91d0ad9bc05b09d3d28ad22b1798d00ed29b9df88df79575ebde1c45",
"build/title.map": "bd743e225dc222a59ec9632a574f59e003e0b07d8ea0428043cb2de3c608e03d",
"dist/PPSA99169/eboot.bin": "dbb71d62520f4baae487a8c7dd7e0425ac5662f7e541b69577ea21cb8bce43e7"
},
"driver_archives_sha256": {
"libps5vk.ps5.a": "900d496a9eb725857e4a70540d34ef262dcb1d5c85b7ed9325a28d16b460e246",
"libvk_runtime.ps5.a": "4106a2c56b269bc52a3976b6bc0134c00b5006ff5358a4b45355afce4a907aa7",
"libpsbc_driver.ps5.a": "a7605370c28895a3b5a13d0718ebfea2abf1b9cf842633de07533775745b21b8",
"libpsbc_support.ps5.a": "e36d3e2a3fa93b0a74db15b9efce7aad4ed7ac4e024de98d5f23dc5fee90acf8"
},
"console_test": "pending; owner requested no automatic upload or launch"
}
+3
View File
@@ -40,6 +40,7 @@
#include <cxxabi.h>
#include "trace.hpp"
#include "memory_diagnostics.hpp"
#include "../build/title_build_identity.h"
/* RetroArch's entry, in C. */
@@ -143,6 +144,7 @@ int main()
std::set_terminate(on_terminate);
ps5::debug::mark(PS5_RETROARCH_BUILD_ID);
ps5::memory::init("/app0/memory-diagnostics.log", PS5_RETROARCH_BUILD_ID);
ps5_vulkan_profile_init();
/* The shell's splash covers the title until it explicitly dismisses it.
@@ -227,6 +229,7 @@ int main()
/* If this line is on the console, the frontend ran and returned by itself. */
ps5::debug::mark_value("rarch_main returned", status);
ps5::memory::finish();
return status;
}
+318
View File
@@ -0,0 +1,318 @@
/* Fixed-size metadata and stack-only formatting: diagnostics must survive OOM.
* No native allocator calls, stdio, dynamic C++ containers or worker threads. */
#include "memory_diagnostics.hpp"
#ifdef PS5_MEMORY_DIAGNOSTICS
#include <cerrno>
#include <cstring>
#include <fcntl.h>
#include <pthread.h>
#include <time.h>
#include <unistd.h>
namespace ps5::memory
{
namespace
{
constexpr size_t capacity = 131072;
Record records[capacity];
// 0 = never used, 1 = live, 2 = tombstone. Pointer values remain unmodified.
unsigned char states[capacity];
Stats stats;
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
int output = -1;
uint64_t start_ns = 0, next_ns = 0, sequence = 0;
uint64_t image_create = 0, image_destroy = 0, image_failed = 0;
uint64_t idle_begin = 0, idle_end = 0, idle_failed = 0;
uint64_t now()
{
timespec t{};
clock_gettime(CLOCK_MONOTONIC, &t);
return uint64_t(t.tv_sec) * 1000000000ULL + uint64_t(t.tv_nsec);
}
struct Line
{
char data[4096];
size_t length = 0;
void text(const char *s)
{
while (*s && length < sizeof(data) - 1)
data[length++] = *s++;
}
void number(uint64_t n, unsigned base = 10)
{
char digits[32];
size_t count = 0;
do
{
digits[count++] = "0123456789abcdef"[n % base];
n /= base;
} while (n);
while (count && length < sizeof(data) - 1)
data[length++] = digits[--count];
}
void field(const char *name, uint64_t n)
{
text(" ");
text(name);
text("=");
number(n);
}
void emit()
{
if (output < 0)
return;
data[length++] = '\n';
size_t done = 0;
while (done < length)
{
const auto written = write(output, data + done, length - done);
if (written < 0 && errno == EINTR)
continue;
if (written <= 0)
break;
done += size_t(written);
}
}
};
size_t hash(void *p)
{
return ((uintptr_t(p) >> 4) * uintptr_t(11400714819323198485ULL)) & (capacity - 1);
}
void summary(const char *kind)
{
Line line;
line.text(kind);
line.field("seq", ++sequence);
line.field("ms", (now() - start_ns) / 1000000);
const char *bytes[] = {"native_bytes", "mapped_bytes", "aligned_bytes"};
const char *counts[] = {"native_count", "mapped_count", "aligned_count"};
const char *peaks[] = {"native_peak", "mapped_peak", "aligned_peak"};
for (unsigned i = 0; i < 3; ++i)
{
line.field(bytes[i], stats.bytes[i]);
line.field(counts[i], stats.count[i]);
line.field(peaks[i], stats.peak[i]);
}
line.field("failures", stats.failures);
line.field("dropped", stats.dropped);
line.field("foreign_frees", stats.foreign_frees);
line.field("foreign_reallocs", stats.foreign_reallocs);
line.field("image_create", image_create);
line.field("image_destroy", image_destroy);
line.field("image_failed", image_failed);
line.field("idle_begin", idle_begin);
line.field("idle_end", idle_end);
line.field("idle_failed", idle_failed);
line.emit();
}
// Fixed aggregation table: caller attribution may overflow independently of
// the live-pointer table. Report that explicitly, never silently lose counts.
void callers()
{
struct Site
{
uintptr_t caller = 0;
uint64_t bytes = 0, count = 0;
};
Site sites[2048]{};
uint64_t omitted = 0;
for (size_t i = 0; i < capacity; ++i)
{
if (states[i] != 1)
continue;
const Record &r = records[i];
size_t j = (r.caller >> 4) & 2047, tries = 0;
while (sites[j].count && sites[j].caller != r.caller && tries < 2048)
{
j = (j + 1) & 2047;
++tries;
}
if (tries == 2048)
{
++omitted;
continue;
}
sites[j].caller = r.caller;
sites[j].bytes += r.bytes;
++sites[j].count;
}
for (unsigned rank = 0; rank < 8; ++rank)
{
size_t best = 0;
for (size_t i = 1; i < 2048; ++i)
if (sites[i].bytes > sites[best].bytes)
best = i;
if (!sites[best].count)
break;
Line line;
line.text("caller");
line.field("seq", sequence);
line.text(" pc=0x");
line.number(sites[best].caller, 16);
line.field("bytes", sites[best].bytes);
line.field("count", sites[best].count);
line.field("site_records_omitted", omitted);
line.emit();
sites[best] = {};
}
}
} // namespace
void add(Record record)
{
if (!record.pointer)
return;
const int saved = errno;
pthread_mutex_lock(&mutex);
size_t slot = hash(record.pointer), vacant = capacity;
for (size_t n = 0; n < capacity; ++n, slot = (slot + 1) & (capacity - 1))
{
if (states[slot] != 1)
{
vacant = slot;
break;
}
}
if (vacant == capacity)
++stats.dropped;
else
{
states[vacant] = 1;
records[vacant] = record;
const auto route = unsigned(record.route);
stats.bytes[route] += record.bytes;
++stats.count[route];
if (stats.bytes[route] > stats.peak[route])
stats.peak[route] = stats.bytes[route];
}
pthread_mutex_unlock(&mutex);
errno = saved;
}
Record take(void *pointer, bool resizing)
{
Record result;
if (!pointer)
return result;
const int saved = errno;
pthread_mutex_lock(&mutex);
size_t slot = hash(pointer);
for (size_t n = 0; n < capacity && states[slot]; ++n, slot = (slot + 1) & (capacity - 1))
{
if (states[slot] == 1 && records[slot].pointer == pointer)
{
result = records[slot];
states[slot] = 2;
stats.bytes[unsigned(result.route)] -= result.bytes;
--stats.count[unsigned(result.route)];
break;
}
}
if (!result.pointer)
{
if (resizing)
++stats.foreign_reallocs;
else
++stats.foreign_frees;
}
pthread_mutex_unlock(&mutex);
errno = saved;
return result;
}
void failure(const char *operation, size_t bytes, size_t alignment, uintptr_t caller, int error)
{
const int saved = errno;
pthread_mutex_lock(&mutex);
++stats.failures;
Line line;
line.text("failure op=");
line.text(operation);
line.field("ms", (now() - start_ns) / 1000000);
line.field("bytes", bytes);
line.field("alignment", alignment);
line.field("error", unsigned(error));
line.text(" pc=0x");
line.number(caller, 16);
line.emit();
summary("failure-summary");
pthread_mutex_unlock(&mutex);
errno = saved;
}
Stats snapshot()
{
pthread_mutex_lock(&mutex);
Stats result = stats;
pthread_mutex_unlock(&mutex);
return result;
}
void init(const char *path, const char *identity)
{
const int saved = errno;
pthread_mutex_lock(&mutex);
if (output >= 0)
close(output);
output = open(path, O_WRONLY | O_CREAT | O_APPEND, 0666);
start_ns = now();
next_ns = start_ns + 5000000000ULL;
Line line;
line.text("session ");
line.text(identity);
line.field("pid", uint64_t(getpid()));
line.field("table_capacity", capacity);
line.field("table_bytes", sizeof(records) + sizeof(states));
line.emit();
summary("initial");
pthread_mutex_unlock(&mutex);
errno = saved;
}
void tick()
{
const int saved = errno;
pthread_mutex_lock(&mutex);
const uint64_t time = now();
if (output >= 0 && time >= next_ns)
{
next_ns = time + 5000000000ULL;
summary("sample");
callers();
}
pthread_mutex_unlock(&mutex);
errno = saved;
}
void event(const char *name, bool success)
{
const int saved = errno;
pthread_mutex_lock(&mutex);
if (!std::strcmp(name, "image_create"))
{
if (success)
++image_create;
else
++image_failed;
}
else if (!std::strcmp(name, "image_destroy"))
++image_destroy;
else if (!std::strcmp(name, "idle_begin"))
++idle_begin;
else if (!std::strcmp(name, "idle_end"))
{
++idle_end;
if (!success)
++idle_failed;
}
pthread_mutex_unlock(&mutex);
errno = saved;
}
void finish()
{
const int saved = errno;
pthread_mutex_lock(&mutex);
summary("final");
callers();
if (output >= 0)
close(output);
output = -1;
pthread_mutex_unlock(&mutex);
errno = saved;
}
} // namespace ps5::memory
#endif
+59
View File
@@ -0,0 +1,59 @@
/* Optional host-allocation observation; no allocator policy changes. */
#pragma once
#include <cstddef>
#include <cstdint>
namespace ps5::memory
{
enum class Route : unsigned
{
Native,
Mapped,
Aligned
};
struct Record
{
void *pointer = nullptr;
size_t bytes = 0;
uintptr_t caller = 0;
Route route = Route::Native;
};
struct Stats
{
uint64_t bytes[3]{}, count[3]{}, peak[3]{};
uint64_t failures = 0, dropped = 0, foreign_frees = 0, foreign_reallocs = 0;
};
#ifdef PS5_MEMORY_DIAGNOSTICS
void init(const char *path, const char *identity);
void finish();
void tick();
void event(const char *name, bool success = true);
void add(Record record);
Record take(void *pointer, bool resizing = false);
void failure(const char *operation, size_t bytes, size_t alignment, uintptr_t caller, int error);
Stats snapshot();
#else
inline void init(const char *, const char *)
{
}
inline void finish()
{
}
inline void tick()
{
}
inline void event(const char *, bool = true)
{
}
inline void add(Record)
{
}
inline Record take(void *, bool = false)
{
return {};
}
inline void failure(const char *, size_t, size_t, uintptr_t, int)
{
}
#endif
} // namespace ps5::memory
+83 -22
View File
@@ -7,6 +7,7 @@
#include <cstring>
#include <pthread.h>
#include <sys/mman.h>
#include "memory_diagnostics.hpp"
extern "C"
{
@@ -38,9 +39,8 @@ Mapping **find(void *pointer)
entry = &(*entry)->next;
return entry;
}
} // namespace
extern "C" void *__wrap_malloc(size_t size)
void *allocate(size_t size)
{
if (size < threshold)
return __real_malloc(size ? size : 1);
@@ -63,7 +63,7 @@ extern "C" void *__wrap_malloc(size_t size)
return entry + 1;
}
extern "C" void __wrap_free(void *pointer)
void release(void *pointer)
{
if (!pointer)
return;
@@ -79,27 +79,13 @@ extern "C" void __wrap_free(void *pointer)
__real_free(pointer);
}
extern "C" void *__wrap_calloc(size_t count, size_t size)
{
if (size && count > SIZE_MAX / size)
{
errno = ENOMEM;
return nullptr;
}
const size_t bytes = count * size;
if (bytes < threshold)
return bytes ? __real_calloc(count, size) : __real_calloc(1, 1);
/* Anonymous mappings are zero-initialized by the kernel. */
return __wrap_malloc(bytes);
}
extern "C" void *__wrap_realloc(void *pointer, size_t size)
void *resize(void *pointer, size_t size)
{
if (!pointer)
return __wrap_malloc(size);
return allocate(size);
if (!size)
{
__wrap_free(pointer);
release(pointer);
return nullptr;
}
pthread_mutex_lock(&lock);
@@ -110,10 +96,85 @@ extern "C" void *__wrap_realloc(void *pointer, size_t size)
* this ABI. Never guess it or read a private libc allocation header. */
if (!entry)
return __real_realloc(pointer, size);
void *replacement = __wrap_malloc(size);
void *replacement = allocate(size);
if (!replacement)
return nullptr;
std::memcpy(replacement, pointer, old_size < size ? old_size : size);
__wrap_free(pointer);
release(pointer);
return replacement;
}
} // namespace
extern "C" void *__wrap_malloc(size_t size)
{
const auto caller = uintptr_t(__builtin_return_address(0));
void *p = allocate(size);
if (p)
ps5::memory::add(
{p, size, caller,
size < threshold ? ps5::memory::Route::Native : ps5::memory::Route::Mapped});
else
ps5::memory::failure("malloc", size, 0, caller, errno);
return p;
}
extern "C" void __wrap_free(void *pointer)
{
ps5::memory::take(pointer);
release(pointer);
}
extern "C" void *__wrap_calloc(size_t count, size_t size)
{
const auto caller = uintptr_t(__builtin_return_address(0));
if (size && count > SIZE_MAX / size)
{
errno = ENOMEM;
ps5::memory::failure("calloc-overflow", size, count, caller, errno);
return nullptr;
}
const size_t bytes = count * size;
void *p = bytes < threshold ? (bytes ? __real_calloc(count, size) : __real_calloc(1, 1))
: allocate(bytes);
if (p)
ps5::memory::add(
{p, bytes, caller,
bytes < threshold ? ps5::memory::Route::Native : ps5::memory::Route::Mapped});
else
ps5::memory::failure("calloc", bytes, 0, caller, errno);
return p;
}
extern "C" void *__wrap_realloc(void *pointer, size_t size)
{
const auto caller = uintptr_t(__builtin_return_address(0));
const auto old = ps5::memory::take(pointer, true);
void *p = resize(pointer, size);
if (p)
{
// Realloc of a native pointer stays native even when it crosses 1 MiB.
const bool mapped =
(!pointer || old.route == ps5::memory::Route::Mapped) && size >= threshold;
ps5::memory::add(
{p, size, caller, mapped ? ps5::memory::Route::Mapped : ps5::memory::Route::Native});
}
else if (size)
{
ps5::memory::add(old); // Failed realloc retains ownership and contents.
ps5::memory::failure("realloc", size, 0, caller, errno);
}
return p;
}
#ifdef PS5_MEMORY_DIAGNOSTICS
extern "C" int __real_posix_memalign(void **, size_t, size_t);
extern "C" int __wrap_posix_memalign(void **out, size_t alignment, size_t size)
{
const auto caller = uintptr_t(__builtin_return_address(0));
const int saved = errno;
const int result = __real_posix_memalign(out, alignment, size);
if (!result && *out)
ps5::memory::add({*out, size, caller, ps5::memory::Route::Aligned});
else if (result)
ps5::memory::failure("posix_memalign", size, alignment, caller, result);
errno = saved; // POSIX reports its error through the return value.
return result;
}
#endif
+56
View File
@@ -9,6 +9,7 @@
#include <cstring>
#include <cstdint>
#include <ctime>
#include "memory_diagnostics.hpp"
namespace
{
@@ -260,6 +261,7 @@ VKAPI_ATTR VkResult VKAPI_CALL traced_present(VkQueue queue, const VkPresentInfo
{
static Results results;
static Results images;
ps5::memory::tick();
ApiTimer timer(Present);
const VkResult result = queue_present(queue, present);
const uint64_t completed = timer.finish();
@@ -354,8 +356,62 @@ extern "C" void ps5_vulkan_profile_end()
/* Called by both frontend symbol loaders. Keep the driver's real function,
* preserve NULL lookups and return every result unchanged. Reloading a device
* must neither reset the run totals nor wrap our own wrapper recursively. */
#ifdef PS5_MEMORY_DIAGNOSTICS
namespace
{
PFN_vkCreateImage memory_create_image;
PFN_vkDestroyImage memory_destroy_image;
PFN_vkQueueWaitIdle memory_queue_idle;
VKAPI_ATTR VkResult VKAPI_CALL memory_create(VkDevice device, const VkImageCreateInfo *info,
const VkAllocationCallbacks *alloc, VkImage *image)
{
const VkResult result = memory_create_image(device, info, alloc, image);
ps5::memory::event("image_create", result == VK_SUCCESS);
return result;
}
VKAPI_ATTR void VKAPI_CALL memory_destroy(VkDevice device, VkImage image,
const VkAllocationCallbacks *alloc)
{
memory_destroy_image(device, image, alloc);
if (image)
ps5::memory::event("image_destroy");
}
VKAPI_ATTR VkResult VKAPI_CALL memory_idle(VkQueue queue)
{
ps5::memory::event("idle_begin");
const VkResult result = memory_queue_idle(queue);
ps5::memory::event("idle_end", result == VK_SUCCESS);
return result;
}
} // namespace
#endif
extern "C" void ps5_vulkan_trace_symbol(const char *name, PFN_vkVoidFunction *symbol)
{
#ifdef PS5_MEMORY_DIAGNOSTICS
if (symbol && *symbol)
{
if (!std::strcmp(name, "vkCreateImage"))
{
if (*symbol != reinterpret_cast<PFN_vkVoidFunction>(memory_create))
memory_create_image = reinterpret_cast<PFN_vkCreateImage>(*symbol);
*symbol = reinterpret_cast<PFN_vkVoidFunction>(memory_create);
}
else if (!std::strcmp(name, "vkDestroyImage"))
{
if (*symbol != reinterpret_cast<PFN_vkVoidFunction>(memory_destroy))
memory_destroy_image = reinterpret_cast<PFN_vkDestroyImage>(*symbol);
*symbol = reinterpret_cast<PFN_vkVoidFunction>(memory_destroy);
}
else if (!std::strcmp(name, "vkQueueWaitIdle"))
{
if (*symbol != reinterpret_cast<PFN_vkVoidFunction>(memory_idle))
memory_queue_idle = reinterpret_cast<PFN_vkQueueWaitIdle>(*symbol);
*symbol = reinterpret_cast<PFN_vkVoidFunction>(memory_idle);
}
}
#endif
if (!*symbol)
return;
if (std::strcmp(name, "vkEndCommandBuffer") == 0)
+131
View File
@@ -0,0 +1,131 @@
#include "src/memory_diagnostics.hpp"
#include <cassert>
#include <cerrno>
#include <cstdlib>
#include <cstring>
#include <thread>
#include <vector>
#include <time.h>
static uint64_t clock_ns = 1;
extern "C" int __wrap_clock_gettime(clockid_t, timespec *t)
{
t->tv_sec = clock_ns / 1000000000ULL;
t->tv_nsec = clock_ns % 1000000000ULL;
return 0;
}
static bool fail_malloc = false, fail_realloc = false;
extern "C"
{
void *__wrap_malloc(size_t);
void *__wrap_calloc(size_t, size_t);
void *__wrap_realloc(void *, size_t);
void __wrap_free(void *);
int __wrap_posix_memalign(void **, size_t, size_t);
void *__real_malloc(size_t n)
{
if (fail_malloc)
{
errno = ENOMEM;
return nullptr;
}
return std::malloc(n);
}
void *__real_calloc(size_t n, size_t s)
{
return std::calloc(n, s);
}
void *__real_realloc(void *p, size_t n)
{
if (fail_realloc)
{
errno = ENOMEM;
return nullptr;
}
return std::realloc(p, n);
}
void __real_free(void *p)
{
std::free(p);
}
int __real_posix_memalign(void **p, size_t a, size_t n)
{
return posix_memalign(p, a, n);
}
}
int main(int argc, char **argv)
{
assert(argc == 2);
using namespace ps5::memory;
init(argv[1], "host-test");
auto *p = static_cast<char *>(__wrap_malloc(123));
assert(p);
std::memset(p, 37, 123);
assert(snapshot().bytes[0] == 123 && snapshot().count[0] == 1);
tick(); // Before the five-second boundary, no sample is emitted.
clock_ns += 5000000000ULL;
tick();
tick(); // Same instant must not emit a duplicate.
fail_realloc = true;
assert(!__wrap_realloc(p, 456) && errno == ENOMEM);
assert(p[122] == 37 && snapshot().bytes[0] == 123);
fail_realloc = false;
p = static_cast<char *>(__wrap_realloc(p, 2 * 1024 * 1024));
assert(p && p[122] == 37);
assert(snapshot().bytes[0] == 2 * 1024 * 1024 && !snapshot().bytes[1]);
__wrap_free(p);
void *mapped = __wrap_malloc(2 * 1024 * 1024);
assert(mapped && snapshot().bytes[1] == 2 * 1024 * 1024);
mapped = __wrap_realloc(mapped, 47);
assert(mapped && snapshot().bytes[0] == 47 && !snapshot().bytes[1]);
__wrap_free(mapped);
void *aligned = nullptr;
errno = EDOM;
assert(!__wrap_posix_memalign(&aligned, 256, 321) && errno == EDOM);
assert(uintptr_t(aligned) % 256 == 0 && snapshot().bytes[2] == 321);
__wrap_free(aligned);
aligned = reinterpret_cast<void *>(uintptr_t(99));
assert(__wrap_posix_memalign(&aligned, 3, 321) == EINVAL && errno == EDOM);
assert(aligned == reinterpret_cast<void *>(uintptr_t(99)));
fail_malloc = true;
assert(!__wrap_malloc(66) && errno == ENOMEM);
// The OOM logger must work while wrapped allocation fails.
failure("forced", 77, 0, 0x1234, ENOMEM);
fail_malloc = false;
assert(!__wrap_calloc(SIZE_MAX, 2));
void *foreign = std::malloc(5);
foreign = __wrap_realloc(foreign, 9);
__wrap_free(foreign);
__wrap_free(strdup("foreign"));
std::vector<std::thread> workers;
for (int t = 0; t < 4; ++t)
workers.emplace_back(
[]
{
for (int i = 0; i < 2000; ++i)
{
void *q = __wrap_calloc(1, 59);
assert(q);
q = __wrap_realloc(q, 101);
assert(q);
__wrap_free(q);
}
});
for (auto &worker : workers)
worker.join();
// Saturation is explicitly reported; it never changes allocation results.
for (uintptr_t i = 1; i <= 131073; ++i)
add({reinterpret_cast<void *>(i * 16), 1, 0x1000, Route::Native});
assert(snapshot().dropped == 1);
for (uintptr_t i = 1; i <= 131073; ++i)
take(reinterpret_cast<void *>(i * 16));
event("image_create");
event("image_destroy");
event("idle_begin");
event("idle_end", false);
const auto stats = snapshot();
assert(!stats.count[0] && !stats.count[1] && !stats.count[2]);
assert(!stats.bytes[0] && !stats.bytes[1] && !stats.bytes[2]);
assert(stats.failures == 5 && stats.foreign_reallocs == 1 && stats.foreign_frees == 2);
finish();
}
+55
View File
@@ -0,0 +1,55 @@
#include "src/vulkan_trace.cpp"
#include <cassert>
static VkDevice device = reinterpret_cast<VkDevice>(1);
static VkQueue queue = reinterpret_cast<VkQueue>(2);
static VkImage image_handle = reinterpret_cast<VkImage>(3);
static VkImageCreateInfo info{};
static VkAllocationCallbacks allocator{};
static unsigned created = 0, destroyed = 0, waited = 0;
static VkResult VKAPI_CALL create_image(VkDevice d, const VkImageCreateInfo *i,
const VkAllocationCallbacks *a, VkImage *image)
{
assert(d == device && i == &info && a == &allocator);
if (++created == 1)
{
*image = image_handle;
return VK_SUCCESS;
}
return VK_ERROR_OUT_OF_HOST_MEMORY;
}
static void VKAPI_CALL destroy_image(VkDevice d, VkImage i, const VkAllocationCallbacks *a)
{
assert(d == device && (i == image_handle || !i) && a == &allocator);
++destroyed;
}
static VkResult VKAPI_CALL queue_idle(VkQueue q)
{
assert(q == queue);
++waited;
return VK_ERROR_DEVICE_LOST;
}
int main(int argc, char **argv)
{
assert(argc == 2);
ps5::memory::init(argv[1], "dispatch-test");
PFN_vkVoidFunction fn = reinterpret_cast<PFN_vkVoidFunction>(create_image);
ps5_vulkan_trace_symbol("vkCreateImage", &fn);
ps5_vulkan_trace_symbol("vkCreateImage", &fn);
VkImage image{};
assert(reinterpret_cast<PFN_vkCreateImage>(fn)(device, &info, &allocator, &image) ==
VK_SUCCESS);
assert(image == image_handle);
assert(reinterpret_cast<PFN_vkCreateImage>(fn)(device, &info, &allocator, &image) ==
VK_ERROR_OUT_OF_HOST_MEMORY);
fn = reinterpret_cast<PFN_vkVoidFunction>(destroy_image);
ps5_vulkan_trace_symbol("vkDestroyImage", &fn);
ps5_vulkan_trace_symbol("vkDestroyImage", &fn);
reinterpret_cast<PFN_vkDestroyImage>(fn)(device, image, &allocator);
reinterpret_cast<PFN_vkDestroyImage>(fn)(device, VK_NULL_HANDLE, &allocator);
fn = reinterpret_cast<PFN_vkVoidFunction>(queue_idle);
ps5_vulkan_trace_symbol("vkQueueWaitIdle", &fn);
ps5_vulkan_trace_symbol("vkQueueWaitIdle", &fn);
assert(reinterpret_cast<PFN_vkQueueWaitIdle>(fn)(queue) == VK_ERROR_DEVICE_LOST);
assert(created == 2 && destroyed == 2 && waited == 1);
ps5::memory::finish();
}
+49
View File
@@ -0,0 +1,49 @@
"""Diagnostic counts, failure-path durability and allocator semantics."""
from pathlib import Path
import subprocess
import tempfile
import unittest
ROOT = Path(__file__).resolve().parent.parent
class MemoryDiagnostics(unittest.TestCase):
def test_counts_failure_logging_and_ownership(self):
with tempfile.TemporaryDirectory() as directory:
binary = Path(directory) / 'diagnostics-test'
capture = Path(directory) / 'memory.log'
subprocess.run(['c++', '-std=c++20', '-pthread', '-I.',
'-DPS5_MEMORY_DIAGNOSTICS', '-Wl,--wrap=clock_gettime',
'tests/memory_diagnostics_test.cpp',
'src/memory_ps5.cpp', 'src/memory_diagnostics.cpp',
'-o', str(binary)], cwd=ROOT, check=True)
subprocess.run([str(binary), str(capture)], check=True)
text = capture.read_text()
self.assertIn('session host-test', text)
self.assertEqual(sum(line.startswith('sample ') for line in text.splitlines()), 1)
self.assertIn('caller seq=', text)
self.assertIn('ms=5000', text)
self.assertEqual(text.count('failure op='), 5)
self.assertIn('failure op=forced', text)
final = next(line for line in text.splitlines() if line.startswith('final '))
fields = dict(item.split('=') for item in final.split()[1:])
for field in ('native_bytes', 'mapped_bytes', 'aligned_bytes',
'native_count', 'mapped_count', 'aligned_count'):
self.assertEqual(fields[field], '0')
self.assertEqual(fields['dropped'], '1')
self.assertEqual(fields['image_create'], fields['image_destroy'])
self.assertEqual(fields['idle_failed'], '1')
def test_vulkan_image_and_idle_dispatch(self):
with tempfile.TemporaryDirectory() as directory:
binary = Path(directory) / 'dispatch-test'
capture = Path(directory) / 'memory.log'
subprocess.run(['c++', '-std=c++20', '-pthread', '-I.', '-Ivendor/retroarch',
'-DPS5_MEMORY_DIAGNOSTICS', 'tests/memory_vulkan_test.cpp',
'src/memory_diagnostics.cpp', '-o', str(binary)], cwd=ROOT, check=True)
subprocess.run([str(binary), str(capture)], check=True)
final = next(line for line in capture.read_text().splitlines() if line.startswith('final '))
fields = dict(item.split('=') for item in final.split()[1:])
for field in ('image_create', 'image_destroy', 'image_failed',
'idle_begin', 'idle_end', 'idle_failed'):
self.assertEqual(fields[field], '1')
+42 -3
View File
@@ -36,6 +36,11 @@ set -euo pipefail
root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)
cd "$root"
# Opt-in diagnostics do not change allocation routing or normal builds.
memory_diagnostics=${PS5_MEMORY_DIAGNOSTICS:-0}
[[ $memory_diagnostics == 0 || $memory_diagnostics == 1 ]] || {
echo "PS5_MEMORY_DIAGNOSTICS must be 0 or 1" >&2; exit 2;
}
stage=false
case "${1:-}" in
'') ;;
@@ -89,6 +94,12 @@ for define in "${title_defines[@]}"; do
title_definition_names+=("${define#-D}")
done
(( ${#title_definition_names[@]} > 0 )) || { echo "error: no feature defines to pass" >&2; exit 2; }
memory_wrap_flags=""
if [[ $memory_diagnostics == 1 ]]; then
title_definition_names+=(PS5_MEMORY_DIAGNOSTICS)
# Mesa's default Vulkan host allocator uses posix_memalign, not malloc.
memory_wrap_flags="--wrap=posix_memalign"
fi
echo "==> [title] compiling src/ with ${#title_definition_names[@]} frontend defines"
# RetroArch's headers reach their generated config as "../../config.h", a relative
@@ -132,6 +143,33 @@ if (( ${#vulkan_missing[@]} )); then
printf ' missing: %s\n' "${vulkan_missing[@]}" >&2
exit 2
fi
# The driver may be developed concurrently. A diagnostic link uses stable local
# archive copies; hashes describe exactly which driver went into this build.
if [[ $memory_diagnostics == 1 ]]; then
if ! snapshot_list=$(python3 - "$root" "${vulkan_archives[@]}" <<'PY_SNAPSHOT'
import hashlib, json, pathlib, shutil, sys
out = pathlib.Path(sys.argv[1]) / "build/memory-diagnostic-inputs"
out.mkdir(parents=True, exist_ok=True)
records = {}
for argument in sys.argv[2:]:
source = pathlib.Path(argument)
before = hashlib.sha256(source.read_bytes()).hexdigest()
target = out / source.name
shutil.copyfile(source, target)
copied = hashlib.sha256(target.read_bytes()).hexdigest()
after = hashlib.sha256(source.read_bytes()).hexdigest()
if before != copied or before != after:
raise SystemExit("Driver archive changed during snapshot; retry when its build finishes")
records[source.name] = copied
print(target)
(out / "archives.json").write_text(json.dumps(records, indent=2) + "\n")
PY_SNAPSHOT
); then
echo "error: driver snapshot failed" >&2; exit 2
fi
mapfile -t vulkan_archives <<< "$snapshot_list"
fi
# Mesa's weak entry points resolve at link time, and the driver's own symbols must
# survive the archive boundary (--whole-archive), which is how the sibling links it.
vulkan_flags="--no-dynamic-linker -z nodynamic-undefined-weak"
@@ -153,7 +191,7 @@ mapfile -t vulkan_objects <<< "$vulkan_object_list"
# Bind the running trace and FTP readback to these exact source/archive inputs.
# The console transforms the SELF container, so its whole-file digest differs.
python3 - "$root" "${vulkan_archives[@]}" "${vulkan_objects[@]}" <<'PY'
python3 - "$root" "$memory_diagnostics" "${vulkan_archives[@]}" "${vulkan_objects[@]}" <<'PY'
import hashlib, pathlib, sys
root = pathlib.Path(sys.argv[1])
inputs = sorted(p for p in (root / "src").rglob("*") if p.is_file())
@@ -165,8 +203,9 @@ inputs += [root / name for name in (
"build/cores/stage/cores/fbneo_libretro.so",
"build/cores/stage/cores/genesis_plus_gx_libretro.so",
"tools/build.sh", "tools/retroarch-flags.sh")]
inputs += [pathlib.Path(name) for name in sys.argv[2:]]
inputs += [pathlib.Path(name) for name in sys.argv[3:]]
digest = hashlib.sha256()
digest.update(b"memory-diagnostics=" + sys.argv[2].encode() + b"\0")
for path in inputs:
digest.update(path.name.encode() + b"\0")
digest.update(hashlib.sha256(path.read_bytes()).digest())
@@ -186,7 +225,7 @@ APP_INCLUDE_PATHS="build/ra-conf build vendor/retroarch build/ra-conf/libretro-c
APP_STATIC_ARCHIVES="build/ra/libretroarch.a" \
APP_VULKAN_ARCHIVES="${vulkan_archives[*]}" \
APP_EXTRA_OBJECTS="${vulkan_objects[*]}" \
APP_LINK_FLAGS="$vulkan_flags --wrap=malloc --wrap=calloc --wrap=realloc --wrap=free" \
APP_LINK_FLAGS="$vulkan_flags --wrap=malloc --wrap=calloc --wrap=realloc --wrap=free $memory_wrap_flags" \
make app
title_id=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["titleId"])' \
+42
View File
@@ -0,0 +1,42 @@
#!/usr/bin/env python3
"""Inspect a locally built diagnostic; never contacts the console.
PS5_MEMORY_DIAGNOSTICS=1 bash tools/verify.sh
python3 tools/check-memory-diagnostics.py > build/memory-diagnostics-inspection.json
"""
import hashlib
import json
from pathlib import Path
import re
import subprocess
ROOT = Path(__file__).resolve().parent.parent
def inspect():
elf = ROOT / 'build/llvm-pie.elf'
symbols = subprocess.check_output(['llvm-nm', '-C', str(elf)], text=True)
required = ('__wrap_malloc', '__wrap_calloc', '__wrap_realloc', '__wrap_free',
'__wrap_posix_memalign', 'ps5::memory::init(', 'ps5::memory::failure(',
'ps5::memory::tick(', 'memory_create(', 'memory_destroy(', 'memory_idle(')
missing = [name for name in required if name not in symbols]
if missing:
raise SystemExit('Diagnostic symbol missing: ' + ', '.join(missing))
identity = re.search(r'build identity: ([a-f0-9]{64})',
(ROOT / 'build/title_build_identity.h').read_text()).group(1)
hashes = {}
for relative in ('build/llvm-pie.elf', 'build/title.map', 'dist/PPSA99169/eboot.bin'):
hashes[relative] = hashlib.sha256((ROOT / relative).read_bytes()).hexdigest()
archives = json.loads((ROOT / 'build/memory-diagnostic-inputs/archives.json').read_text())
for name, expected in archives.items():
actual = hashlib.sha256((ROOT / 'build/memory-diagnostic-inputs' / name).read_bytes()).hexdigest()
if actual != expected:
raise SystemExit('Driver snapshot digest mismatch: ' + name)
return {'kind': 'host-only-diagnostic-inspection', 'passed': True,
'build_identity': identity, 'symbols_present': list(required),
'artifacts_sha256': hashes, 'driver_archives_sha256': archives,
'console_test': 'pending; owner requested no automatic upload or launch'}
if __name__ == '__main__':
print(json.dumps(inspect(), indent=2))