mirror of
https://github.com/mihawk-99/PS5_RetroArch.git
synced 2026-10-18 23:09:29 -05:00
Add opt-in XMB allocation diagnostics for manual console testing
Keep native/mapped allocation policy intact; observe aligned Vulkan host allocations and texture/queue-idle lifetimes with allocation-free failure logging and periodic summaries. Snapshot driver archives locally for an attributable diagnostic link.
Verified: PS5_MEMORY_DIAGNOSTICS=1 bash tools/verify.sh; final bash tools/verify.sh format unit (68 tests); python3 tools/check-memory-diagnostics.py. Evidence: evidence/xmb-memory-diagnostic/{capture,expectation,inspection}.json. Console validation pending owner intervention; no upload or launch and no crash fix claimed.
This commit is contained in:
1 parent
a18cec63ef
commit
971534cd37
17 files changed
+1089
-54
No files matched your search
+34
-29
@@ -4,16 +4,38 @@ _Updated: 2026-09-19_
|
||||
|
||||
## Now
|
||||
|
||||
**Genesis Plus GX is ported and owner-confirmed working.** The owner reports
|
||||
“Sega Genesis works flawlessly” and explicitly confirms “Yes, transitions and
|
||||
next game are clean.” Build and console evidence: `evidence/genesis-plus-gx-native/`.
|
||||
No next core is selected.
|
||||
**XMB allocation diagnostic is built on `codex/xmb-allocation-diagnostics`.**
|
||||
This is instrumentation, not a crash fix. Owner explicitly requests no automatic
|
||||
upload or launch while working on another project. Wait for their intervention.
|
||||
The baseline's next-core work is not the assignment; this crash investigation is.
|
||||
|
||||
The public README now describes this verified baseline, with a table of contents,
|
||||
scoped core coverage, roadmap, native build/install guidance and upstream credits.
|
||||
This documentation update does not change the accepted binaries.
|
||||
- Build identity: `000adf3edc2538bc563f833f0ccd2e91a01053aa6dac19dd568a580cdbbbb3cb`.
|
||||
- Output: `dist/PPSA99169/`; preserved executable/symbols/map/inspection under
|
||||
`klog/xmb-memory-000adf3edc25/`. Evidence: `evidence/xmb-memory-diagnostic/`.
|
||||
- `PS5_MEMORY_DIAGNOSTICS=1 bash tools/verify.sh` passed all five host gates.
|
||||
Final format/unit check passed; 68 tests. Raw: `klog/xmb-memory-verify.log`,
|
||||
`klog/xmb-memory-final-checks.log`. `tools/check-memory-diagnostics.py` passes.
|
||||
- No upload, launch, console settings change or PS5_Vulkan modification.
|
||||
Current libps5vk differs from the crashing build; local archive snapshots and
|
||||
their hashes identify the diagnostic's inputs. Other three archives match.
|
||||
- Log: `/app0/memory-diagnostics.log`; periodic requested live bytes/counts/peaks,
|
||||
caller groups, image/queue-idle counters; immediate allocation failure records.
|
||||
Metadata saturation/coverage gaps are explicit. No allocator policy change.
|
||||
- Reproduction and interpretation: `docs/MEMORY_DIAGNOSTICS.md`. Console validation
|
||||
remains pending, including whether the crash reproduces with the newer driver.
|
||||
|
||||
## Verified build and evidence
|
||||
## Crash evidence
|
||||
|
||||
Owner reproduced rapid XMB navigation crash. Raw capture:
|
||||
`klog/xmb-crash-20260919-181233/` (kernel, frontend, trace, config, diagnosis).
|
||||
Matching baseline symbols locate SIGSEGV at runtime `0x45b1ae` in
|
||||
`__vk_log_impl`, reached through `vk_sync_create` allocation failure inside
|
||||
`vkQueueWaitIdle`, called from XMB texture unload. Logger's second allocation
|
||||
returns NULL and is dereferenced. Why allocation failed is unresolved: heap
|
||||
exhaustion, fragmentation, retention/leak, unobserved usage or corruption remain
|
||||
possible. Do not call this a GPU hang or a resolved frontend/driver leak.
|
||||
|
||||
## Previous console-verified baseline (Genesis Plus GX)
|
||||
|
||||
- Frontend identity: `ecfcddd57febbb484c2e0724e95f43bbffb3ddd9b9c22aa9c1cf3e37af4a9445`.
|
||||
- Genesis Plus GX source: `c2838c7dc4236fc2fe94e5dbd08b41486067918e`.
|
||||
@@ -44,21 +66,6 @@ This documentation update does not change the accepted binaries.
|
||||
- PS5_Vulkan was not modified. Four archive hashes matched before/after build;
|
||||
exact linked hashes are in the evidence. The owner develops it concurrently.
|
||||
|
||||
## What changed
|
||||
|
||||
- `make genesis-plus-gx`: pinned official source and metadata using explicit
|
||||
project SDK wrappers, native import union and .info in both lookup locations.
|
||||
Attached metadata was reference only. No ROM or BIOS is shipped.
|
||||
- Preserve RGB565 software rendering, filters/cursors and cached bitmap; convert
|
||||
into a separate bounded XRGB8888 callback buffer. Honour cropped byte offsets
|
||||
and the 720-pixel row pitch. The frontend's existing RGBA upload/cropping path
|
||||
remains unchanged, as does the Vulkan driver.
|
||||
- Disable host-detected physical CD-ROM support and optional zstd weak tracing
|
||||
hooks. Keep CHD/disc-image codecs and useful logs. Correct update_geometry's
|
||||
unused return type. Reasons/pins and procedures are in REFERENCE/TESTING/DEPLOYMENT.
|
||||
- Host tests cover every RGB565 colour through upstream packing and the actual
|
||||
GPU upload helper, viewport changes/crops, cached frames and rejected bounds.
|
||||
|
||||
## Named errors and remaining limits
|
||||
|
||||
- First frontend snapshot has five intentional missing-core recovery errors and
|
||||
@@ -93,9 +100,7 @@ See `docs/DEPLOYMENT.md` for procedures and metadata-listed BIOS filenames.
|
||||
|
||||
## Operating notes
|
||||
|
||||
Uploads/runs are authorized without further permission. Never interrupt an
|
||||
existing title; the runner checks idle before deployment and launch. Core/game
|
||||
selection is manual. Keep private filenames and console details in ignored logs.
|
||||
Separate appended trace launches by bss check and build identity. Preserve live
|
||||
logs before relaunch. Owner supplies visual confirmation; camera cadence can
|
||||
hide flicker. Do not infer manual closure solely from the final process count.
|
||||
Console testing is paused by the owner's explicit instruction for this step.
|
||||
When they are ready, start passive klog before reproduction and preserve current
|
||||
logs. Never interrupt another title. After a crash, capture logs before relaunch.
|
||||
No new core or unrelated driver work is authorized by the active-file context.
|
||||
@@ -0,0 +1,110 @@
|
||||
# XMB allocation diagnostic build
|
||||
|
||||
This is an opt-in diagnostic, not a fix for the XMB crash. It observes allocation
|
||||
lifetimes without increasing the heap, redirecting additional allocations to
|
||||
`mmap`, or suppressing Vulkan errors. PS5_Vulkan is read only for this work.
|
||||
|
||||
## Build and host verification
|
||||
|
||||
```sh
|
||||
PS5_MEMORY_DIAGNOSTICS=1 bash tools/verify.sh
|
||||
```
|
||||
|
||||
The normal build (variable absent or `0`) compiles out the observer. The diagnostic
|
||||
adds `PS5_MEMORY_DIAGNOSTICS` to the title definitions and
|
||||
`--wrap=posix_memalign` to the existing malloc/calloc/realloc/free wrappers because
|
||||
Mesa's default Vulkan host allocator uses aligned native allocations. Mode and
|
||||
archive contents participate in the runtime build identity.
|
||||
|
||||
The diagnostic build copies the four driver archives into
|
||||
`build/memory-diagnostic-inputs/` and links those copies. `archives.json` records
|
||||
their SHA-256 digests. A copy that changes during the read is rejected. This does
|
||||
not rebuild or write into the sibling project; it also does not imply the copied
|
||||
archives are the same driver as an earlier console run. Mesa utility sources are
|
||||
still read from the configured driver source tree, and the resulting object
|
||||
bytes participate in the identity.
|
||||
|
||||
Output is `dist/PPSA99169/`. Preserve `build/llvm-pie.elf`, `build/title.map`,
|
||||
`build/title_build_identity.h`, the manifest, and the archive digests before any
|
||||
subsequent build. Caller addresses must be decoded against the exact matching ELF.
|
||||
|
||||
Host tests exercise allocation/free accounting, mapped/native realloc migration,
|
||||
failed realloc retention, foreign pointers, aligned allocation semantics,
|
||||
concurrency, metadata saturation, five-second sampling with an injected clock,
|
||||
and logging while the wrapped allocator fails. Vulkan mocks check that image and
|
||||
queue-idle observers preserve arguments, return values and repeat registration.
|
||||
These tests do not establish console stability or identify the memory consumer.
|
||||
|
||||
## Manual console test — only when the owner is ready
|
||||
|
||||
Do not upload, launch, change live settings, or stop another title automatically.
|
||||
Start a passive klog listener before the owner's reproduction and preserve the
|
||||
existing logs first. Deploy the diagnostic using the project's normal folder
|
||||
procedure, preserving live configuration and user content. The owner launches it.
|
||||
|
||||
1. Leave XMB idle for 30 seconds.
|
||||
2. Browse rapidly for 30 seconds, then leave it idle for 30 seconds. Repeat.
|
||||
3. Repeat the RGUI to XMB transition that triggered the crash. Note the approximate
|
||||
elapsed time of each phase, including whether switching required restarting.
|
||||
4. On a crash, leave the title closed until logs have been downloaded.
|
||||
|
||||
Download `/data/homebrew/PPSA99169/memory-diagnostics.log`, `retroarch.log`,
|
||||
`trace.txt`, and the live configuration, along with the kernel capture. Keep raw
|
||||
files under ignored `klog/`; they may contain user paths and process information.
|
||||
In the title the same new log is `/app0/memory-diagnostics.log`. It appends a
|
||||
`session` line with build identity and PID each launch rather than erasing the
|
||||
previous run. No runner or deployment behavior is changed by this diagnostic.
|
||||
|
||||
## Reading the capture
|
||||
|
||||
`initial`, `sample`, `failure-summary` and `final` rows contain:
|
||||
|
||||
- `native_bytes/count/peak`: requested live bytes, live count and peak live bytes
|
||||
from observed ordinary native allocations.
|
||||
- `aligned_bytes/count/peak`: the same for `posix_memalign`; this is also native
|
||||
memory, not a separate heap. Native pressure includes both native and aligned.
|
||||
- `mapped_bytes/count/peak`: requests routed through the existing large-buffer
|
||||
mappings. These exclude mapping headers, page rounding and allocator overhead.
|
||||
- `failures`: cumulative failures; `failure` rows immediately record operation,
|
||||
requested bytes, alignment (or element count for calloc overflow), error and
|
||||
caller return address (`pc`). `calloc-overflow` is an arithmetic rejection,
|
||||
not measured heap exhaustion.
|
||||
- `dropped`: allocation records omitted because the fixed table was full. Any
|
||||
nonzero value makes live totals incomplete for the rest of that session.
|
||||
- `foreign_frees/reallocs`: pointers whose allocation was not observed (including
|
||||
records dropped on saturation). Never inspect private libc allocation headers.
|
||||
- `image_create/destroy/failed`: successful frontend Vulkan image creations,
|
||||
non-null destruction calls and failed creations. These include images other
|
||||
than menu textures and exclude calls internal to the driver.
|
||||
- `idle_begin/end/failed`: queue-idle calls started, returned and returned failure.
|
||||
A missing completion in a failure summary helps locate the failing operation.
|
||||
|
||||
Every five seconds while Vulkan presents, a summary and up to eight largest live
|
||||
allocation caller groups are emitted. Groups contain `pc`, requested live bytes
|
||||
and count; `site_records_omitted` reports overflow of the separate 2,048-site
|
||||
aggregation table. They are not full stack traces. No frame means no periodic
|
||||
sample; allocation failures still write synchronously. Clean frontend return
|
||||
writes `final`; a crash need not.
|
||||
|
||||
Logging uses stack formatting, a preopened descriptor and `write`, without stdio
|
||||
or heap allocation. A fixed 131,072-entry pointer table plus state bytes uses
|
||||
about 4.13 MiB of static storage. Mutexes, table operations, scans and synchronous
|
||||
writes add overhead, so use this build to diagnose lifetimes, not benchmark FPS.
|
||||
The allocation route and payload size remain unchanged.
|
||||
|
||||
Coverage is title/core/static-library references intercepted by the linker.
|
||||
Allocations performed internally by system libraries, direct mappings outside the
|
||||
existing large-buffer wrapper, GPU direct memory, and allocator bookkeeping are
|
||||
not measured. Requested live bytes are not the console's total memory usage.
|
||||
|
||||
For a `pc` from this title, subtract the image load base recorded in klog (normally
|
||||
`0x400000`) and one byte for a return address, then use
|
||||
`llvm-addr2line -C -f -e <matching-symbols.elf> <offset>`. A fault instruction
|
||||
address is decoded without the one-byte adjustment. Keep symbols locally; never
|
||||
publish raw memory addresses or local paths as sanitized evidence by accident.
|
||||
|
||||
Compare settled idle samples from the same session. Sustained growth is a reason
|
||||
to investigate retained allocations; it alone does not prove a leak. Stable
|
||||
observed totals plus allocation failures leave unobserved native usage,
|
||||
fragmentation, heap limits and corruption open. The crash's Mesa null dereference
|
||||
while reporting an allocation failure remains unfixed by this instrumentation.
|
||||
@@ -2313,3 +2313,37 @@ Documentation-only validation:
|
||||
existing committed artifacts, especially evidence/genesis-plus-gx-native/.
|
||||
- `git diff --check`: PASS. No source, dependency, binary or console changes;
|
||||
the accepted gameplay build remains 93a1332's recorded build identity.
|
||||
|
||||
## 2026-09-19 — XMB allocation diagnostic branch, host verified only
|
||||
|
||||
Owner requested a branch and diagnostic build, explicitly withholding automatic
|
||||
upload/launch while working on another project. Created
|
||||
`codex/xmb-allocation-diagnostics`. Added opt-in live allocation accounting to the
|
||||
existing native/mapped wrappers, plus observation of `posix_memalign`, frontend
|
||||
Vulkan image lifetimes and queue-idle calls. Fixed-capacity metadata and stack-only
|
||||
`write` logging preserve OOM evidence without allocating. Allocation policy is
|
||||
unchanged. Five-second summaries include counts, requested bytes, peaks, largest
|
||||
caller groups, failure counts and explicit coverage/saturation limits.
|
||||
|
||||
The prior capture (`klog/xmb-crash-20260919-181233/`) maps the null write to Mesa's
|
||||
error-reporting allocation following `vk_sync_create` allocation failure in
|
||||
`vkQueueWaitIdle` during XMB texture unload. That diagnoses the immediate crash,
|
||||
not the reason memory allocation failed. No driver fix or console result is claimed.
|
||||
|
||||
Command: `PS5_MEMORY_DIAGNOSTICS=1 bash tools/verify.sh` — PASS all five host gates.
|
||||
The first unit pass had 67 tests; the added Vulkan dispatch test ran in the final
|
||||
integration pass and final `bash tools/verify.sh format unit` (68 tests, PASS).
|
||||
`python3 tools/check-memory-diagnostics.py` — PASS required symbols and archive
|
||||
hashes. Machine-readable inspection and capture/expectation:
|
||||
`evidence/xmb-memory-diagnostic/`. Raw host logs are in `klog/xmb-memory-verify.log`
|
||||
and `klog/xmb-memory-final-checks.log`. Existing upstream core/compiler warnings
|
||||
remain in the full build log; no diagnostic-source warning was observed.
|
||||
|
||||
Build identity: `000adf3edc2538bc563f833f0ccd2e91a01053aa6dac19dd568a580cdbbbb3cb`.
|
||||
Output: `dist/PPSA99169/`; matching executable, symbol ELF and map preserved under
|
||||
`klog/xmb-memory-000adf3edc25/`. The current libps5vk archive differs from the
|
||||
crashing baseline; the other three archives match its recorded digests. The
|
||||
opt-in build snapshots all four locally to avoid mutable archive inputs while
|
||||
the owner develops PS5_Vulkan. Mode and linked bytes participate in identity.
|
||||
No sibling writes, console access, upload, launch or settings change occurred.
|
||||
Manual protocol and limitations are in `docs/MEMORY_DIAGNOSTICS.md`.
|
||||
@@ -330,3 +330,10 @@ Capture frontend/trace/kernel logs, match the exact build identity, and check fo
|
||||
Vulkan refusals, GPU API failures, invalid bitmap views and unexpected runtime
|
||||
errors before committing. Record systems actually tested and preserve user
|
||||
confirmation separately from machine-readable loader and runtime evidence.
|
||||
|
||||
## XMB allocation diagnostics
|
||||
|
||||
The opt-in build, host allocator/Vulkan-dispatch tests, raw log fields, ownership
|
||||
coverage limits and manual reproduction protocol are specified in
|
||||
[MEMORY_DIAGNOSTICS.md](MEMORY_DIAGNOSTICS.md). Its build inspection is host-only;
|
||||
it must not be cited as a passing console reproduction or a fixed crash.
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"step": "xmb-memory-diagnostic",
|
||||
"recorded": "2026-09-19",
|
||||
"revision": "a18cec6 + allocation diagnostics",
|
||||
"command": "PS5_MEMORY_DIAGNOSTICS=1 bash tools/verify.sh; bash tools/verify.sh format unit; python3 tools/check-memory-diagnostics.py",
|
||||
"console_run": null,
|
||||
"raw_capture": "klog/xmb-memory-verify.log",
|
||||
"lines": [
|
||||
"Scope: host-verified diagnostic build only; no console upload or launch.",
|
||||
"Diagnostic build identity: 000adf3edc2538bc563f833f0ccd2e91a01053aa6dac19dd568a580cdbbbb3cb",
|
||||
"Allocation semantics and OOM logging: PASS; concurrency and saturation: PASS.",
|
||||
"Five-second sampling with injected clock: PASS; Vulkan dispatch preservation: PASS.",
|
||||
"Host verification: PASS format unit build integration evidence; final suite 68 tests.",
|
||||
"Diagnostic symbols and pinned driver snapshots: PASS.",
|
||||
"Console reproduction: PENDING. XMB crash is not fixed by this change."
|
||||
],
|
||||
"raw_lines": 6265,
|
||||
"raw_bytes": 3051670
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"step": "xmb-memory-diagnostic",
|
||||
"note": "Host-only diagnostic readiness. Console test deliberately pending owner intervention.",
|
||||
"must_contain": [
|
||||
"Scope: host-verified diagnostic build only; no console upload or launch.",
|
||||
"Diagnostic build identity: 000adf3edc2538bc563f833f0ccd2e91a01053aa6dac19dd568a580cdbbbb3cb",
|
||||
"Allocation semantics and OOM logging: PASS; concurrency and saturation: PASS.",
|
||||
"Five-second sampling with injected clock: PASS; Vulkan dispatch preservation: PASS.",
|
||||
"Host verification: PASS format unit build integration evidence; final suite 68 tests.",
|
||||
"Diagnostic symbols and pinned driver snapshots: PASS.",
|
||||
"Console reproduction: PENDING. XMB crash is not fixed by this change."
|
||||
],
|
||||
"must_not_contain": [
|
||||
"Console reproduction: PASS"
|
||||
],
|
||||
"human_check": "Not performed: owner requested no upload or launch."
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"kind": "host-only-diagnostic-inspection",
|
||||
"passed": true,
|
||||
"build_identity": "000adf3edc2538bc563f833f0ccd2e91a01053aa6dac19dd568a580cdbbbb3cb",
|
||||
"symbols_present": [
|
||||
"__wrap_malloc",
|
||||
"__wrap_calloc",
|
||||
"__wrap_realloc",
|
||||
"__wrap_free",
|
||||
"__wrap_posix_memalign",
|
||||
"ps5::memory::init(",
|
||||
"ps5::memory::failure(",
|
||||
"ps5::memory::tick(",
|
||||
"memory_create(",
|
||||
"memory_destroy(",
|
||||
"memory_idle("
|
||||
],
|
||||
"artifacts_sha256": {
|
||||
"build/llvm-pie.elf": "8f89351d91d0ad9bc05b09d3d28ad22b1798d00ed29b9df88df79575ebde1c45",
|
||||
"build/title.map": "bd743e225dc222a59ec9632a574f59e003e0b07d8ea0428043cb2de3c608e03d",
|
||||
"dist/PPSA99169/eboot.bin": "dbb71d62520f4baae487a8c7dd7e0425ac5662f7e541b69577ea21cb8bce43e7"
|
||||
},
|
||||
"driver_archives_sha256": {
|
||||
"libps5vk.ps5.a": "900d496a9eb725857e4a70540d34ef262dcb1d5c85b7ed9325a28d16b460e246",
|
||||
"libvk_runtime.ps5.a": "4106a2c56b269bc52a3976b6bc0134c00b5006ff5358a4b45355afce4a907aa7",
|
||||
"libpsbc_driver.ps5.a": "a7605370c28895a3b5a13d0718ebfea2abf1b9cf842633de07533775745b21b8",
|
||||
"libpsbc_support.ps5.a": "e36d3e2a3fa93b0a74db15b9efce7aad4ed7ac4e024de98d5f23dc5fee90acf8"
|
||||
},
|
||||
"console_test": "pending; owner requested no automatic upload or launch"
|
||||
}
|
||||
@@ -40,6 +40,7 @@
|
||||
#include <cxxabi.h>
|
||||
|
||||
#include "trace.hpp"
|
||||
#include "memory_diagnostics.hpp"
|
||||
#include "../build/title_build_identity.h"
|
||||
|
||||
/* RetroArch's entry, in C. */
|
||||
@@ -143,6 +144,7 @@ int main()
|
||||
|
||||
std::set_terminate(on_terminate);
|
||||
ps5::debug::mark(PS5_RETROARCH_BUILD_ID);
|
||||
ps5::memory::init("/app0/memory-diagnostics.log", PS5_RETROARCH_BUILD_ID);
|
||||
ps5_vulkan_profile_init();
|
||||
|
||||
/* The shell's splash covers the title until it explicitly dismisses it.
|
||||
@@ -227,6 +229,7 @@ int main()
|
||||
|
||||
/* If this line is on the console, the frontend ran and returned by itself. */
|
||||
ps5::debug::mark_value("rarch_main returned", status);
|
||||
ps5::memory::finish();
|
||||
|
||||
return status;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,318 @@
|
||||
/* Fixed-size metadata and stack-only formatting: diagnostics must survive OOM.
|
||||
* No native allocator calls, stdio, dynamic C++ containers or worker threads. */
|
||||
#include "memory_diagnostics.hpp"
|
||||
#ifdef PS5_MEMORY_DIAGNOSTICS
|
||||
#include <cerrno>
|
||||
#include <cstring>
|
||||
#include <fcntl.h>
|
||||
#include <pthread.h>
|
||||
#include <time.h>
|
||||
#include <unistd.h>
|
||||
|
||||
namespace ps5::memory
|
||||
{
|
||||
namespace
|
||||
{
|
||||
constexpr size_t capacity = 131072;
|
||||
Record records[capacity];
|
||||
// 0 = never used, 1 = live, 2 = tombstone. Pointer values remain unmodified.
|
||||
unsigned char states[capacity];
|
||||
Stats stats;
|
||||
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
|
||||
int output = -1;
|
||||
uint64_t start_ns = 0, next_ns = 0, sequence = 0;
|
||||
uint64_t image_create = 0, image_destroy = 0, image_failed = 0;
|
||||
uint64_t idle_begin = 0, idle_end = 0, idle_failed = 0;
|
||||
|
||||
uint64_t now()
|
||||
{
|
||||
timespec t{};
|
||||
clock_gettime(CLOCK_MONOTONIC, &t);
|
||||
return uint64_t(t.tv_sec) * 1000000000ULL + uint64_t(t.tv_nsec);
|
||||
}
|
||||
struct Line
|
||||
{
|
||||
char data[4096];
|
||||
size_t length = 0;
|
||||
void text(const char *s)
|
||||
{
|
||||
while (*s && length < sizeof(data) - 1)
|
||||
data[length++] = *s++;
|
||||
}
|
||||
void number(uint64_t n, unsigned base = 10)
|
||||
{
|
||||
char digits[32];
|
||||
size_t count = 0;
|
||||
do
|
||||
{
|
||||
digits[count++] = "0123456789abcdef"[n % base];
|
||||
n /= base;
|
||||
} while (n);
|
||||
while (count && length < sizeof(data) - 1)
|
||||
data[length++] = digits[--count];
|
||||
}
|
||||
void field(const char *name, uint64_t n)
|
||||
{
|
||||
text(" ");
|
||||
text(name);
|
||||
text("=");
|
||||
number(n);
|
||||
}
|
||||
void emit()
|
||||
{
|
||||
if (output < 0)
|
||||
return;
|
||||
data[length++] = '\n';
|
||||
size_t done = 0;
|
||||
while (done < length)
|
||||
{
|
||||
const auto written = write(output, data + done, length - done);
|
||||
if (written < 0 && errno == EINTR)
|
||||
continue;
|
||||
if (written <= 0)
|
||||
break;
|
||||
done += size_t(written);
|
||||
}
|
||||
}
|
||||
};
|
||||
size_t hash(void *p)
|
||||
{
|
||||
return ((uintptr_t(p) >> 4) * uintptr_t(11400714819323198485ULL)) & (capacity - 1);
|
||||
}
|
||||
void summary(const char *kind)
|
||||
{
|
||||
Line line;
|
||||
line.text(kind);
|
||||
line.field("seq", ++sequence);
|
||||
line.field("ms", (now() - start_ns) / 1000000);
|
||||
const char *bytes[] = {"native_bytes", "mapped_bytes", "aligned_bytes"};
|
||||
const char *counts[] = {"native_count", "mapped_count", "aligned_count"};
|
||||
const char *peaks[] = {"native_peak", "mapped_peak", "aligned_peak"};
|
||||
for (unsigned i = 0; i < 3; ++i)
|
||||
{
|
||||
line.field(bytes[i], stats.bytes[i]);
|
||||
line.field(counts[i], stats.count[i]);
|
||||
line.field(peaks[i], stats.peak[i]);
|
||||
}
|
||||
line.field("failures", stats.failures);
|
||||
line.field("dropped", stats.dropped);
|
||||
line.field("foreign_frees", stats.foreign_frees);
|
||||
line.field("foreign_reallocs", stats.foreign_reallocs);
|
||||
line.field("image_create", image_create);
|
||||
line.field("image_destroy", image_destroy);
|
||||
line.field("image_failed", image_failed);
|
||||
line.field("idle_begin", idle_begin);
|
||||
line.field("idle_end", idle_end);
|
||||
line.field("idle_failed", idle_failed);
|
||||
line.emit();
|
||||
}
|
||||
// Fixed aggregation table: caller attribution may overflow independently of
|
||||
// the live-pointer table. Report that explicitly, never silently lose counts.
|
||||
void callers()
|
||||
{
|
||||
struct Site
|
||||
{
|
||||
uintptr_t caller = 0;
|
||||
uint64_t bytes = 0, count = 0;
|
||||
};
|
||||
Site sites[2048]{};
|
||||
uint64_t omitted = 0;
|
||||
for (size_t i = 0; i < capacity; ++i)
|
||||
{
|
||||
if (states[i] != 1)
|
||||
continue;
|
||||
const Record &r = records[i];
|
||||
size_t j = (r.caller >> 4) & 2047, tries = 0;
|
||||
while (sites[j].count && sites[j].caller != r.caller && tries < 2048)
|
||||
{
|
||||
j = (j + 1) & 2047;
|
||||
++tries;
|
||||
}
|
||||
if (tries == 2048)
|
||||
{
|
||||
++omitted;
|
||||
continue;
|
||||
}
|
||||
sites[j].caller = r.caller;
|
||||
sites[j].bytes += r.bytes;
|
||||
++sites[j].count;
|
||||
}
|
||||
for (unsigned rank = 0; rank < 8; ++rank)
|
||||
{
|
||||
size_t best = 0;
|
||||
for (size_t i = 1; i < 2048; ++i)
|
||||
if (sites[i].bytes > sites[best].bytes)
|
||||
best = i;
|
||||
if (!sites[best].count)
|
||||
break;
|
||||
Line line;
|
||||
line.text("caller");
|
||||
line.field("seq", sequence);
|
||||
line.text(" pc=0x");
|
||||
line.number(sites[best].caller, 16);
|
||||
line.field("bytes", sites[best].bytes);
|
||||
line.field("count", sites[best].count);
|
||||
line.field("site_records_omitted", omitted);
|
||||
line.emit();
|
||||
sites[best] = {};
|
||||
}
|
||||
}
|
||||
} // namespace
|
||||
void add(Record record)
|
||||
{
|
||||
if (!record.pointer)
|
||||
return;
|
||||
const int saved = errno;
|
||||
pthread_mutex_lock(&mutex);
|
||||
size_t slot = hash(record.pointer), vacant = capacity;
|
||||
for (size_t n = 0; n < capacity; ++n, slot = (slot + 1) & (capacity - 1))
|
||||
{
|
||||
if (states[slot] != 1)
|
||||
{
|
||||
vacant = slot;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (vacant == capacity)
|
||||
++stats.dropped;
|
||||
else
|
||||
{
|
||||
states[vacant] = 1;
|
||||
records[vacant] = record;
|
||||
const auto route = unsigned(record.route);
|
||||
stats.bytes[route] += record.bytes;
|
||||
++stats.count[route];
|
||||
if (stats.bytes[route] > stats.peak[route])
|
||||
stats.peak[route] = stats.bytes[route];
|
||||
}
|
||||
pthread_mutex_unlock(&mutex);
|
||||
errno = saved;
|
||||
}
|
||||
Record take(void *pointer, bool resizing)
|
||||
{
|
||||
Record result;
|
||||
if (!pointer)
|
||||
return result;
|
||||
const int saved = errno;
|
||||
pthread_mutex_lock(&mutex);
|
||||
size_t slot = hash(pointer);
|
||||
for (size_t n = 0; n < capacity && states[slot]; ++n, slot = (slot + 1) & (capacity - 1))
|
||||
{
|
||||
if (states[slot] == 1 && records[slot].pointer == pointer)
|
||||
{
|
||||
result = records[slot];
|
||||
states[slot] = 2;
|
||||
stats.bytes[unsigned(result.route)] -= result.bytes;
|
||||
--stats.count[unsigned(result.route)];
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!result.pointer)
|
||||
{
|
||||
if (resizing)
|
||||
++stats.foreign_reallocs;
|
||||
else
|
||||
++stats.foreign_frees;
|
||||
}
|
||||
pthread_mutex_unlock(&mutex);
|
||||
errno = saved;
|
||||
return result;
|
||||
}
|
||||
void failure(const char *operation, size_t bytes, size_t alignment, uintptr_t caller, int error)
|
||||
{
|
||||
const int saved = errno;
|
||||
pthread_mutex_lock(&mutex);
|
||||
++stats.failures;
|
||||
Line line;
|
||||
line.text("failure op=");
|
||||
line.text(operation);
|
||||
line.field("ms", (now() - start_ns) / 1000000);
|
||||
line.field("bytes", bytes);
|
||||
line.field("alignment", alignment);
|
||||
line.field("error", unsigned(error));
|
||||
line.text(" pc=0x");
|
||||
line.number(caller, 16);
|
||||
line.emit();
|
||||
summary("failure-summary");
|
||||
pthread_mutex_unlock(&mutex);
|
||||
errno = saved;
|
||||
}
|
||||
Stats snapshot()
|
||||
{
|
||||
pthread_mutex_lock(&mutex);
|
||||
Stats result = stats;
|
||||
pthread_mutex_unlock(&mutex);
|
||||
return result;
|
||||
}
|
||||
void init(const char *path, const char *identity)
|
||||
{
|
||||
const int saved = errno;
|
||||
pthread_mutex_lock(&mutex);
|
||||
if (output >= 0)
|
||||
close(output);
|
||||
output = open(path, O_WRONLY | O_CREAT | O_APPEND, 0666);
|
||||
start_ns = now();
|
||||
next_ns = start_ns + 5000000000ULL;
|
||||
Line line;
|
||||
line.text("session ");
|
||||
line.text(identity);
|
||||
line.field("pid", uint64_t(getpid()));
|
||||
line.field("table_capacity", capacity);
|
||||
line.field("table_bytes", sizeof(records) + sizeof(states));
|
||||
line.emit();
|
||||
summary("initial");
|
||||
pthread_mutex_unlock(&mutex);
|
||||
errno = saved;
|
||||
}
|
||||
void tick()
|
||||
{
|
||||
const int saved = errno;
|
||||
pthread_mutex_lock(&mutex);
|
||||
const uint64_t time = now();
|
||||
if (output >= 0 && time >= next_ns)
|
||||
{
|
||||
next_ns = time + 5000000000ULL;
|
||||
summary("sample");
|
||||
callers();
|
||||
}
|
||||
pthread_mutex_unlock(&mutex);
|
||||
errno = saved;
|
||||
}
|
||||
void event(const char *name, bool success)
|
||||
{
|
||||
const int saved = errno;
|
||||
pthread_mutex_lock(&mutex);
|
||||
if (!std::strcmp(name, "image_create"))
|
||||
{
|
||||
if (success)
|
||||
++image_create;
|
||||
else
|
||||
++image_failed;
|
||||
}
|
||||
else if (!std::strcmp(name, "image_destroy"))
|
||||
++image_destroy;
|
||||
else if (!std::strcmp(name, "idle_begin"))
|
||||
++idle_begin;
|
||||
else if (!std::strcmp(name, "idle_end"))
|
||||
{
|
||||
++idle_end;
|
||||
if (!success)
|
||||
++idle_failed;
|
||||
}
|
||||
pthread_mutex_unlock(&mutex);
|
||||
errno = saved;
|
||||
}
|
||||
void finish()
|
||||
{
|
||||
const int saved = errno;
|
||||
pthread_mutex_lock(&mutex);
|
||||
summary("final");
|
||||
callers();
|
||||
if (output >= 0)
|
||||
close(output);
|
||||
output = -1;
|
||||
pthread_mutex_unlock(&mutex);
|
||||
errno = saved;
|
||||
}
|
||||
} // namespace ps5::memory
|
||||
#endif
|
||||
@@ -0,0 +1,59 @@
|
||||
/* Optional host-allocation observation; no allocator policy changes. */
|
||||
#pragma once
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
|
||||
namespace ps5::memory
|
||||
{
|
||||
enum class Route : unsigned
|
||||
{
|
||||
Native,
|
||||
Mapped,
|
||||
Aligned
|
||||
};
|
||||
struct Record
|
||||
{
|
||||
void *pointer = nullptr;
|
||||
size_t bytes = 0;
|
||||
uintptr_t caller = 0;
|
||||
Route route = Route::Native;
|
||||
};
|
||||
struct Stats
|
||||
{
|
||||
uint64_t bytes[3]{}, count[3]{}, peak[3]{};
|
||||
uint64_t failures = 0, dropped = 0, foreign_frees = 0, foreign_reallocs = 0;
|
||||
};
|
||||
#ifdef PS5_MEMORY_DIAGNOSTICS
|
||||
void init(const char *path, const char *identity);
|
||||
void finish();
|
||||
void tick();
|
||||
void event(const char *name, bool success = true);
|
||||
void add(Record record);
|
||||
Record take(void *pointer, bool resizing = false);
|
||||
void failure(const char *operation, size_t bytes, size_t alignment, uintptr_t caller, int error);
|
||||
Stats snapshot();
|
||||
#else
|
||||
inline void init(const char *, const char *)
|
||||
{
|
||||
}
|
||||
inline void finish()
|
||||
{
|
||||
}
|
||||
inline void tick()
|
||||
{
|
||||
}
|
||||
inline void event(const char *, bool = true)
|
||||
{
|
||||
}
|
||||
inline void add(Record)
|
||||
{
|
||||
}
|
||||
inline Record take(void *, bool = false)
|
||||
{
|
||||
return {};
|
||||
}
|
||||
inline void failure(const char *, size_t, size_t, uintptr_t, int)
|
||||
{
|
||||
}
|
||||
#endif
|
||||
} // namespace ps5::memory
|
||||
+83
-22
@@ -7,6 +7,7 @@
|
||||
#include <cstring>
|
||||
#include <pthread.h>
|
||||
#include <sys/mman.h>
|
||||
#include "memory_diagnostics.hpp"
|
||||
|
||||
extern "C"
|
||||
{
|
||||
@@ -38,9 +39,8 @@ Mapping **find(void *pointer)
|
||||
entry = &(*entry)->next;
|
||||
return entry;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
extern "C" void *__wrap_malloc(size_t size)
|
||||
void *allocate(size_t size)
|
||||
{
|
||||
if (size < threshold)
|
||||
return __real_malloc(size ? size : 1);
|
||||
@@ -63,7 +63,7 @@ extern "C" void *__wrap_malloc(size_t size)
|
||||
return entry + 1;
|
||||
}
|
||||
|
||||
extern "C" void __wrap_free(void *pointer)
|
||||
void release(void *pointer)
|
||||
{
|
||||
if (!pointer)
|
||||
return;
|
||||
@@ -79,27 +79,13 @@ extern "C" void __wrap_free(void *pointer)
|
||||
__real_free(pointer);
|
||||
}
|
||||
|
||||
extern "C" void *__wrap_calloc(size_t count, size_t size)
|
||||
{
|
||||
if (size && count > SIZE_MAX / size)
|
||||
{
|
||||
errno = ENOMEM;
|
||||
return nullptr;
|
||||
}
|
||||
const size_t bytes = count * size;
|
||||
if (bytes < threshold)
|
||||
return bytes ? __real_calloc(count, size) : __real_calloc(1, 1);
|
||||
/* Anonymous mappings are zero-initialized by the kernel. */
|
||||
return __wrap_malloc(bytes);
|
||||
}
|
||||
|
||||
extern "C" void *__wrap_realloc(void *pointer, size_t size)
|
||||
void *resize(void *pointer, size_t size)
|
||||
{
|
||||
if (!pointer)
|
||||
return __wrap_malloc(size);
|
||||
return allocate(size);
|
||||
if (!size)
|
||||
{
|
||||
__wrap_free(pointer);
|
||||
release(pointer);
|
||||
return nullptr;
|
||||
}
|
||||
pthread_mutex_lock(&lock);
|
||||
@@ -110,10 +96,85 @@ extern "C" void *__wrap_realloc(void *pointer, size_t size)
|
||||
* this ABI. Never guess it or read a private libc allocation header. */
|
||||
if (!entry)
|
||||
return __real_realloc(pointer, size);
|
||||
void *replacement = __wrap_malloc(size);
|
||||
void *replacement = allocate(size);
|
||||
if (!replacement)
|
||||
return nullptr;
|
||||
std::memcpy(replacement, pointer, old_size < size ? old_size : size);
|
||||
__wrap_free(pointer);
|
||||
release(pointer);
|
||||
return replacement;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
extern "C" void *__wrap_malloc(size_t size)
|
||||
{
|
||||
const auto caller = uintptr_t(__builtin_return_address(0));
|
||||
void *p = allocate(size);
|
||||
if (p)
|
||||
ps5::memory::add(
|
||||
{p, size, caller,
|
||||
size < threshold ? ps5::memory::Route::Native : ps5::memory::Route::Mapped});
|
||||
else
|
||||
ps5::memory::failure("malloc", size, 0, caller, errno);
|
||||
return p;
|
||||
}
|
||||
extern "C" void __wrap_free(void *pointer)
|
||||
{
|
||||
ps5::memory::take(pointer);
|
||||
release(pointer);
|
||||
}
|
||||
extern "C" void *__wrap_calloc(size_t count, size_t size)
|
||||
{
|
||||
const auto caller = uintptr_t(__builtin_return_address(0));
|
||||
if (size && count > SIZE_MAX / size)
|
||||
{
|
||||
errno = ENOMEM;
|
||||
ps5::memory::failure("calloc-overflow", size, count, caller, errno);
|
||||
return nullptr;
|
||||
}
|
||||
const size_t bytes = count * size;
|
||||
void *p = bytes < threshold ? (bytes ? __real_calloc(count, size) : __real_calloc(1, 1))
|
||||
: allocate(bytes);
|
||||
if (p)
|
||||
ps5::memory::add(
|
||||
{p, bytes, caller,
|
||||
bytes < threshold ? ps5::memory::Route::Native : ps5::memory::Route::Mapped});
|
||||
else
|
||||
ps5::memory::failure("calloc", bytes, 0, caller, errno);
|
||||
return p;
|
||||
}
|
||||
extern "C" void *__wrap_realloc(void *pointer, size_t size)
|
||||
{
|
||||
const auto caller = uintptr_t(__builtin_return_address(0));
|
||||
const auto old = ps5::memory::take(pointer, true);
|
||||
void *p = resize(pointer, size);
|
||||
if (p)
|
||||
{
|
||||
// Realloc of a native pointer stays native even when it crosses 1 MiB.
|
||||
const bool mapped =
|
||||
(!pointer || old.route == ps5::memory::Route::Mapped) && size >= threshold;
|
||||
ps5::memory::add(
|
||||
{p, size, caller, mapped ? ps5::memory::Route::Mapped : ps5::memory::Route::Native});
|
||||
}
|
||||
else if (size)
|
||||
{
|
||||
ps5::memory::add(old); // Failed realloc retains ownership and contents.
|
||||
ps5::memory::failure("realloc", size, 0, caller, errno);
|
||||
}
|
||||
return p;
|
||||
}
|
||||
#ifdef PS5_MEMORY_DIAGNOSTICS
|
||||
extern "C" int __real_posix_memalign(void **, size_t, size_t);
|
||||
extern "C" int __wrap_posix_memalign(void **out, size_t alignment, size_t size)
|
||||
{
|
||||
const auto caller = uintptr_t(__builtin_return_address(0));
|
||||
const int saved = errno;
|
||||
const int result = __real_posix_memalign(out, alignment, size);
|
||||
if (!result && *out)
|
||||
ps5::memory::add({*out, size, caller, ps5::memory::Route::Aligned});
|
||||
else if (result)
|
||||
ps5::memory::failure("posix_memalign", size, alignment, caller, result);
|
||||
errno = saved; // POSIX reports its error through the return value.
|
||||
return result;
|
||||
}
|
||||
#endif
|
||||
@@ -9,6 +9,7 @@
|
||||
#include <cstring>
|
||||
#include <cstdint>
|
||||
#include <ctime>
|
||||
#include "memory_diagnostics.hpp"
|
||||
|
||||
namespace
|
||||
{
|
||||
@@ -260,6 +261,7 @@ VKAPI_ATTR VkResult VKAPI_CALL traced_present(VkQueue queue, const VkPresentInfo
|
||||
{
|
||||
static Results results;
|
||||
static Results images;
|
||||
ps5::memory::tick();
|
||||
ApiTimer timer(Present);
|
||||
const VkResult result = queue_present(queue, present);
|
||||
const uint64_t completed = timer.finish();
|
||||
@@ -354,8 +356,62 @@ extern "C" void ps5_vulkan_profile_end()
|
||||
/* Called by both frontend symbol loaders. Keep the driver's real function,
|
||||
* preserve NULL lookups and return every result unchanged. Reloading a device
|
||||
* must neither reset the run totals nor wrap our own wrapper recursively. */
|
||||
#ifdef PS5_MEMORY_DIAGNOSTICS
|
||||
namespace
|
||||
{
|
||||
PFN_vkCreateImage memory_create_image;
|
||||
PFN_vkDestroyImage memory_destroy_image;
|
||||
PFN_vkQueueWaitIdle memory_queue_idle;
|
||||
VKAPI_ATTR VkResult VKAPI_CALL memory_create(VkDevice device, const VkImageCreateInfo *info,
|
||||
const VkAllocationCallbacks *alloc, VkImage *image)
|
||||
{
|
||||
const VkResult result = memory_create_image(device, info, alloc, image);
|
||||
ps5::memory::event("image_create", result == VK_SUCCESS);
|
||||
return result;
|
||||
}
|
||||
VKAPI_ATTR void VKAPI_CALL memory_destroy(VkDevice device, VkImage image,
|
||||
const VkAllocationCallbacks *alloc)
|
||||
{
|
||||
memory_destroy_image(device, image, alloc);
|
||||
if (image)
|
||||
ps5::memory::event("image_destroy");
|
||||
}
|
||||
VKAPI_ATTR VkResult VKAPI_CALL memory_idle(VkQueue queue)
|
||||
{
|
||||
ps5::memory::event("idle_begin");
|
||||
const VkResult result = memory_queue_idle(queue);
|
||||
ps5::memory::event("idle_end", result == VK_SUCCESS);
|
||||
return result;
|
||||
}
|
||||
} // namespace
|
||||
#endif
|
||||
|
||||
extern "C" void ps5_vulkan_trace_symbol(const char *name, PFN_vkVoidFunction *symbol)
|
||||
{
|
||||
#ifdef PS5_MEMORY_DIAGNOSTICS
|
||||
if (symbol && *symbol)
|
||||
{
|
||||
if (!std::strcmp(name, "vkCreateImage"))
|
||||
{
|
||||
if (*symbol != reinterpret_cast<PFN_vkVoidFunction>(memory_create))
|
||||
memory_create_image = reinterpret_cast<PFN_vkCreateImage>(*symbol);
|
||||
*symbol = reinterpret_cast<PFN_vkVoidFunction>(memory_create);
|
||||
}
|
||||
else if (!std::strcmp(name, "vkDestroyImage"))
|
||||
{
|
||||
if (*symbol != reinterpret_cast<PFN_vkVoidFunction>(memory_destroy))
|
||||
memory_destroy_image = reinterpret_cast<PFN_vkDestroyImage>(*symbol);
|
||||
*symbol = reinterpret_cast<PFN_vkVoidFunction>(memory_destroy);
|
||||
}
|
||||
else if (!std::strcmp(name, "vkQueueWaitIdle"))
|
||||
{
|
||||
if (*symbol != reinterpret_cast<PFN_vkVoidFunction>(memory_idle))
|
||||
memory_queue_idle = reinterpret_cast<PFN_vkQueueWaitIdle>(*symbol);
|
||||
*symbol = reinterpret_cast<PFN_vkVoidFunction>(memory_idle);
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
if (!*symbol)
|
||||
return;
|
||||
if (std::strcmp(name, "vkEndCommandBuffer") == 0)
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
#include "src/memory_diagnostics.hpp"
|
||||
#include <cassert>
|
||||
#include <cerrno>
|
||||
#include <cstdlib>
|
||||
#include <cstring>
|
||||
#include <thread>
|
||||
#include <vector>
|
||||
#include <time.h>
|
||||
|
||||
static uint64_t clock_ns = 1;
|
||||
extern "C" int __wrap_clock_gettime(clockid_t, timespec *t)
|
||||
{
|
||||
t->tv_sec = clock_ns / 1000000000ULL;
|
||||
t->tv_nsec = clock_ns % 1000000000ULL;
|
||||
return 0;
|
||||
}
|
||||
static bool fail_malloc = false, fail_realloc = false;
|
||||
extern "C"
|
||||
{
|
||||
void *__wrap_malloc(size_t);
|
||||
void *__wrap_calloc(size_t, size_t);
|
||||
void *__wrap_realloc(void *, size_t);
|
||||
void __wrap_free(void *);
|
||||
int __wrap_posix_memalign(void **, size_t, size_t);
|
||||
void *__real_malloc(size_t n)
|
||||
{
|
||||
if (fail_malloc)
|
||||
{
|
||||
errno = ENOMEM;
|
||||
return nullptr;
|
||||
}
|
||||
return std::malloc(n);
|
||||
}
|
||||
void *__real_calloc(size_t n, size_t s)
|
||||
{
|
||||
return std::calloc(n, s);
|
||||
}
|
||||
void *__real_realloc(void *p, size_t n)
|
||||
{
|
||||
if (fail_realloc)
|
||||
{
|
||||
errno = ENOMEM;
|
||||
return nullptr;
|
||||
}
|
||||
return std::realloc(p, n);
|
||||
}
|
||||
void __real_free(void *p)
|
||||
{
|
||||
std::free(p);
|
||||
}
|
||||
int __real_posix_memalign(void **p, size_t a, size_t n)
|
||||
{
|
||||
return posix_memalign(p, a, n);
|
||||
}
|
||||
}
|
||||
int main(int argc, char **argv)
|
||||
{
|
||||
assert(argc == 2);
|
||||
using namespace ps5::memory;
|
||||
init(argv[1], "host-test");
|
||||
auto *p = static_cast<char *>(__wrap_malloc(123));
|
||||
assert(p);
|
||||
std::memset(p, 37, 123);
|
||||
assert(snapshot().bytes[0] == 123 && snapshot().count[0] == 1);
|
||||
tick(); // Before the five-second boundary, no sample is emitted.
|
||||
clock_ns += 5000000000ULL;
|
||||
tick();
|
||||
tick(); // Same instant must not emit a duplicate.
|
||||
fail_realloc = true;
|
||||
assert(!__wrap_realloc(p, 456) && errno == ENOMEM);
|
||||
assert(p[122] == 37 && snapshot().bytes[0] == 123);
|
||||
fail_realloc = false;
|
||||
p = static_cast<char *>(__wrap_realloc(p, 2 * 1024 * 1024));
|
||||
assert(p && p[122] == 37);
|
||||
assert(snapshot().bytes[0] == 2 * 1024 * 1024 && !snapshot().bytes[1]);
|
||||
__wrap_free(p);
|
||||
void *mapped = __wrap_malloc(2 * 1024 * 1024);
|
||||
assert(mapped && snapshot().bytes[1] == 2 * 1024 * 1024);
|
||||
mapped = __wrap_realloc(mapped, 47);
|
||||
assert(mapped && snapshot().bytes[0] == 47 && !snapshot().bytes[1]);
|
||||
__wrap_free(mapped);
|
||||
void *aligned = nullptr;
|
||||
errno = EDOM;
|
||||
assert(!__wrap_posix_memalign(&aligned, 256, 321) && errno == EDOM);
|
||||
assert(uintptr_t(aligned) % 256 == 0 && snapshot().bytes[2] == 321);
|
||||
__wrap_free(aligned);
|
||||
aligned = reinterpret_cast<void *>(uintptr_t(99));
|
||||
assert(__wrap_posix_memalign(&aligned, 3, 321) == EINVAL && errno == EDOM);
|
||||
assert(aligned == reinterpret_cast<void *>(uintptr_t(99)));
|
||||
fail_malloc = true;
|
||||
assert(!__wrap_malloc(66) && errno == ENOMEM);
|
||||
// The OOM logger must work while wrapped allocation fails.
|
||||
failure("forced", 77, 0, 0x1234, ENOMEM);
|
||||
fail_malloc = false;
|
||||
assert(!__wrap_calloc(SIZE_MAX, 2));
|
||||
void *foreign = std::malloc(5);
|
||||
foreign = __wrap_realloc(foreign, 9);
|
||||
__wrap_free(foreign);
|
||||
__wrap_free(strdup("foreign"));
|
||||
std::vector<std::thread> workers;
|
||||
for (int t = 0; t < 4; ++t)
|
||||
workers.emplace_back(
|
||||
[]
|
||||
{
|
||||
for (int i = 0; i < 2000; ++i)
|
||||
{
|
||||
void *q = __wrap_calloc(1, 59);
|
||||
assert(q);
|
||||
q = __wrap_realloc(q, 101);
|
||||
assert(q);
|
||||
__wrap_free(q);
|
||||
}
|
||||
});
|
||||
for (auto &worker : workers)
|
||||
worker.join();
|
||||
// Saturation is explicitly reported; it never changes allocation results.
|
||||
for (uintptr_t i = 1; i <= 131073; ++i)
|
||||
add({reinterpret_cast<void *>(i * 16), 1, 0x1000, Route::Native});
|
||||
assert(snapshot().dropped == 1);
|
||||
for (uintptr_t i = 1; i <= 131073; ++i)
|
||||
take(reinterpret_cast<void *>(i * 16));
|
||||
event("image_create");
|
||||
event("image_destroy");
|
||||
event("idle_begin");
|
||||
event("idle_end", false);
|
||||
const auto stats = snapshot();
|
||||
assert(!stats.count[0] && !stats.count[1] && !stats.count[2]);
|
||||
assert(!stats.bytes[0] && !stats.bytes[1] && !stats.bytes[2]);
|
||||
assert(stats.failures == 5 && stats.foreign_reallocs == 1 && stats.foreign_frees == 2);
|
||||
finish();
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
#include "src/vulkan_trace.cpp"
|
||||
#include <cassert>
|
||||
static VkDevice device = reinterpret_cast<VkDevice>(1);
|
||||
static VkQueue queue = reinterpret_cast<VkQueue>(2);
|
||||
static VkImage image_handle = reinterpret_cast<VkImage>(3);
|
||||
static VkImageCreateInfo info{};
|
||||
static VkAllocationCallbacks allocator{};
|
||||
static unsigned created = 0, destroyed = 0, waited = 0;
|
||||
static VkResult VKAPI_CALL create_image(VkDevice d, const VkImageCreateInfo *i,
|
||||
const VkAllocationCallbacks *a, VkImage *image)
|
||||
{
|
||||
assert(d == device && i == &info && a == &allocator);
|
||||
if (++created == 1)
|
||||
{
|
||||
*image = image_handle;
|
||||
return VK_SUCCESS;
|
||||
}
|
||||
return VK_ERROR_OUT_OF_HOST_MEMORY;
|
||||
}
|
||||
static void VKAPI_CALL destroy_image(VkDevice d, VkImage i, const VkAllocationCallbacks *a)
|
||||
{
|
||||
assert(d == device && (i == image_handle || !i) && a == &allocator);
|
||||
++destroyed;
|
||||
}
|
||||
static VkResult VKAPI_CALL queue_idle(VkQueue q)
|
||||
{
|
||||
assert(q == queue);
|
||||
++waited;
|
||||
return VK_ERROR_DEVICE_LOST;
|
||||
}
|
||||
int main(int argc, char **argv)
|
||||
{
|
||||
assert(argc == 2);
|
||||
ps5::memory::init(argv[1], "dispatch-test");
|
||||
PFN_vkVoidFunction fn = reinterpret_cast<PFN_vkVoidFunction>(create_image);
|
||||
ps5_vulkan_trace_symbol("vkCreateImage", &fn);
|
||||
ps5_vulkan_trace_symbol("vkCreateImage", &fn);
|
||||
VkImage image{};
|
||||
assert(reinterpret_cast<PFN_vkCreateImage>(fn)(device, &info, &allocator, &image) ==
|
||||
VK_SUCCESS);
|
||||
assert(image == image_handle);
|
||||
assert(reinterpret_cast<PFN_vkCreateImage>(fn)(device, &info, &allocator, &image) ==
|
||||
VK_ERROR_OUT_OF_HOST_MEMORY);
|
||||
fn = reinterpret_cast<PFN_vkVoidFunction>(destroy_image);
|
||||
ps5_vulkan_trace_symbol("vkDestroyImage", &fn);
|
||||
ps5_vulkan_trace_symbol("vkDestroyImage", &fn);
|
||||
reinterpret_cast<PFN_vkDestroyImage>(fn)(device, image, &allocator);
|
||||
reinterpret_cast<PFN_vkDestroyImage>(fn)(device, VK_NULL_HANDLE, &allocator);
|
||||
fn = reinterpret_cast<PFN_vkVoidFunction>(queue_idle);
|
||||
ps5_vulkan_trace_symbol("vkQueueWaitIdle", &fn);
|
||||
ps5_vulkan_trace_symbol("vkQueueWaitIdle", &fn);
|
||||
assert(reinterpret_cast<PFN_vkQueueWaitIdle>(fn)(queue) == VK_ERROR_DEVICE_LOST);
|
||||
assert(created == 2 && destroyed == 2 && waited == 1);
|
||||
ps5::memory::finish();
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
"""Diagnostic counts, failure-path durability and allocator semantics."""
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
class MemoryDiagnostics(unittest.TestCase):
|
||||
def test_counts_failure_logging_and_ownership(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
binary = Path(directory) / 'diagnostics-test'
|
||||
capture = Path(directory) / 'memory.log'
|
||||
subprocess.run(['c++', '-std=c++20', '-pthread', '-I.',
|
||||
'-DPS5_MEMORY_DIAGNOSTICS', '-Wl,--wrap=clock_gettime',
|
||||
'tests/memory_diagnostics_test.cpp',
|
||||
'src/memory_ps5.cpp', 'src/memory_diagnostics.cpp',
|
||||
'-o', str(binary)], cwd=ROOT, check=True)
|
||||
subprocess.run([str(binary), str(capture)], check=True)
|
||||
text = capture.read_text()
|
||||
self.assertIn('session host-test', text)
|
||||
self.assertEqual(sum(line.startswith('sample ') for line in text.splitlines()), 1)
|
||||
self.assertIn('caller seq=', text)
|
||||
self.assertIn('ms=5000', text)
|
||||
self.assertEqual(text.count('failure op='), 5)
|
||||
self.assertIn('failure op=forced', text)
|
||||
final = next(line for line in text.splitlines() if line.startswith('final '))
|
||||
fields = dict(item.split('=') for item in final.split()[1:])
|
||||
for field in ('native_bytes', 'mapped_bytes', 'aligned_bytes',
|
||||
'native_count', 'mapped_count', 'aligned_count'):
|
||||
self.assertEqual(fields[field], '0')
|
||||
self.assertEqual(fields['dropped'], '1')
|
||||
self.assertEqual(fields['image_create'], fields['image_destroy'])
|
||||
self.assertEqual(fields['idle_failed'], '1')
|
||||
|
||||
def test_vulkan_image_and_idle_dispatch(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
binary = Path(directory) / 'dispatch-test'
|
||||
capture = Path(directory) / 'memory.log'
|
||||
subprocess.run(['c++', '-std=c++20', '-pthread', '-I.', '-Ivendor/retroarch',
|
||||
'-DPS5_MEMORY_DIAGNOSTICS', 'tests/memory_vulkan_test.cpp',
|
||||
'src/memory_diagnostics.cpp', '-o', str(binary)], cwd=ROOT, check=True)
|
||||
subprocess.run([str(binary), str(capture)], check=True)
|
||||
final = next(line for line in capture.read_text().splitlines() if line.startswith('final '))
|
||||
fields = dict(item.split('=') for item in final.split()[1:])
|
||||
for field in ('image_create', 'image_destroy', 'image_failed',
|
||||
'idle_begin', 'idle_end', 'idle_failed'):
|
||||
self.assertEqual(fields[field], '1')
|
||||
+42
-3
@@ -36,6 +36,11 @@ set -euo pipefail
|
||||
root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)
|
||||
cd "$root"
|
||||
|
||||
# Opt-in diagnostics do not change allocation routing or normal builds.
|
||||
memory_diagnostics=${PS5_MEMORY_DIAGNOSTICS:-0}
|
||||
[[ $memory_diagnostics == 0 || $memory_diagnostics == 1 ]] || {
|
||||
echo "PS5_MEMORY_DIAGNOSTICS must be 0 or 1" >&2; exit 2;
|
||||
}
|
||||
stage=false
|
||||
case "${1:-}" in
|
||||
'') ;;
|
||||
@@ -89,6 +94,12 @@ for define in "${title_defines[@]}"; do
|
||||
title_definition_names+=("${define#-D}")
|
||||
done
|
||||
(( ${#title_definition_names[@]} > 0 )) || { echo "error: no feature defines to pass" >&2; exit 2; }
|
||||
memory_wrap_flags=""
|
||||
if [[ $memory_diagnostics == 1 ]]; then
|
||||
title_definition_names+=(PS5_MEMORY_DIAGNOSTICS)
|
||||
# Mesa's default Vulkan host allocator uses posix_memalign, not malloc.
|
||||
memory_wrap_flags="--wrap=posix_memalign"
|
||||
fi
|
||||
echo "==> [title] compiling src/ with ${#title_definition_names[@]} frontend defines"
|
||||
|
||||
# RetroArch's headers reach their generated config as "../../config.h", a relative
|
||||
@@ -132,6 +143,33 @@ if (( ${#vulkan_missing[@]} )); then
|
||||
printf ' missing: %s\n' "${vulkan_missing[@]}" >&2
|
||||
exit 2
|
||||
fi
|
||||
# The driver may be developed concurrently. A diagnostic link uses stable local
|
||||
# archive copies; hashes describe exactly which driver went into this build.
|
||||
if [[ $memory_diagnostics == 1 ]]; then
|
||||
if ! snapshot_list=$(python3 - "$root" "${vulkan_archives[@]}" <<'PY_SNAPSHOT'
|
||||
import hashlib, json, pathlib, shutil, sys
|
||||
out = pathlib.Path(sys.argv[1]) / "build/memory-diagnostic-inputs"
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
records = {}
|
||||
for argument in sys.argv[2:]:
|
||||
source = pathlib.Path(argument)
|
||||
before = hashlib.sha256(source.read_bytes()).hexdigest()
|
||||
target = out / source.name
|
||||
shutil.copyfile(source, target)
|
||||
copied = hashlib.sha256(target.read_bytes()).hexdigest()
|
||||
after = hashlib.sha256(source.read_bytes()).hexdigest()
|
||||
if before != copied or before != after:
|
||||
raise SystemExit("Driver archive changed during snapshot; retry when its build finishes")
|
||||
records[source.name] = copied
|
||||
print(target)
|
||||
(out / "archives.json").write_text(json.dumps(records, indent=2) + "\n")
|
||||
PY_SNAPSHOT
|
||||
); then
|
||||
echo "error: driver snapshot failed" >&2; exit 2
|
||||
fi
|
||||
mapfile -t vulkan_archives <<< "$snapshot_list"
|
||||
fi
|
||||
|
||||
# Mesa's weak entry points resolve at link time, and the driver's own symbols must
|
||||
# survive the archive boundary (--whole-archive), which is how the sibling links it.
|
||||
vulkan_flags="--no-dynamic-linker -z nodynamic-undefined-weak"
|
||||
@@ -153,7 +191,7 @@ mapfile -t vulkan_objects <<< "$vulkan_object_list"
|
||||
|
||||
# Bind the running trace and FTP readback to these exact source/archive inputs.
|
||||
# The console transforms the SELF container, so its whole-file digest differs.
|
||||
python3 - "$root" "${vulkan_archives[@]}" "${vulkan_objects[@]}" <<'PY'
|
||||
python3 - "$root" "$memory_diagnostics" "${vulkan_archives[@]}" "${vulkan_objects[@]}" <<'PY'
|
||||
import hashlib, pathlib, sys
|
||||
root = pathlib.Path(sys.argv[1])
|
||||
inputs = sorted(p for p in (root / "src").rglob("*") if p.is_file())
|
||||
@@ -165,8 +203,9 @@ inputs += [root / name for name in (
|
||||
"build/cores/stage/cores/fbneo_libretro.so",
|
||||
"build/cores/stage/cores/genesis_plus_gx_libretro.so",
|
||||
"tools/build.sh", "tools/retroarch-flags.sh")]
|
||||
inputs += [pathlib.Path(name) for name in sys.argv[2:]]
|
||||
inputs += [pathlib.Path(name) for name in sys.argv[3:]]
|
||||
digest = hashlib.sha256()
|
||||
digest.update(b"memory-diagnostics=" + sys.argv[2].encode() + b"\0")
|
||||
for path in inputs:
|
||||
digest.update(path.name.encode() + b"\0")
|
||||
digest.update(hashlib.sha256(path.read_bytes()).digest())
|
||||
@@ -186,7 +225,7 @@ APP_INCLUDE_PATHS="build/ra-conf build vendor/retroarch build/ra-conf/libretro-c
|
||||
APP_STATIC_ARCHIVES="build/ra/libretroarch.a" \
|
||||
APP_VULKAN_ARCHIVES="${vulkan_archives[*]}" \
|
||||
APP_EXTRA_OBJECTS="${vulkan_objects[*]}" \
|
||||
APP_LINK_FLAGS="$vulkan_flags --wrap=malloc --wrap=calloc --wrap=realloc --wrap=free" \
|
||||
APP_LINK_FLAGS="$vulkan_flags --wrap=malloc --wrap=calloc --wrap=realloc --wrap=free $memory_wrap_flags" \
|
||||
make app
|
||||
|
||||
title_id=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["titleId"])' \
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Inspect a locally built diagnostic; never contacts the console.
|
||||
|
||||
PS5_MEMORY_DIAGNOSTICS=1 bash tools/verify.sh
|
||||
python3 tools/check-memory-diagnostics.py > build/memory-diagnostics-inspection.json
|
||||
"""
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import subprocess
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
def inspect():
|
||||
elf = ROOT / 'build/llvm-pie.elf'
|
||||
symbols = subprocess.check_output(['llvm-nm', '-C', str(elf)], text=True)
|
||||
required = ('__wrap_malloc', '__wrap_calloc', '__wrap_realloc', '__wrap_free',
|
||||
'__wrap_posix_memalign', 'ps5::memory::init(', 'ps5::memory::failure(',
|
||||
'ps5::memory::tick(', 'memory_create(', 'memory_destroy(', 'memory_idle(')
|
||||
missing = [name for name in required if name not in symbols]
|
||||
if missing:
|
||||
raise SystemExit('Diagnostic symbol missing: ' + ', '.join(missing))
|
||||
identity = re.search(r'build identity: ([a-f0-9]{64})',
|
||||
(ROOT / 'build/title_build_identity.h').read_text()).group(1)
|
||||
hashes = {}
|
||||
for relative in ('build/llvm-pie.elf', 'build/title.map', 'dist/PPSA99169/eboot.bin'):
|
||||
hashes[relative] = hashlib.sha256((ROOT / relative).read_bytes()).hexdigest()
|
||||
archives = json.loads((ROOT / 'build/memory-diagnostic-inputs/archives.json').read_text())
|
||||
for name, expected in archives.items():
|
||||
actual = hashlib.sha256((ROOT / 'build/memory-diagnostic-inputs' / name).read_bytes()).hexdigest()
|
||||
if actual != expected:
|
||||
raise SystemExit('Driver snapshot digest mismatch: ' + name)
|
||||
return {'kind': 'host-only-diagnostic-inspection', 'passed': True,
|
||||
'build_identity': identity, 'symbols_present': list(required),
|
||||
'artifacts_sha256': hashes, 'driver_archives_sha256': archives,
|
||||
'console_test': 'pending; owner requested no automatic upload or launch'}
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
print(json.dumps(inspect(), indent=2))
|
||||
Reference in new issue
Block a user