A shared mapping is a view of a file section created on the descriptor's
handle, read-write when the descriptor is, otherwise read-only. A private
mapping is a copy of the file read at mmap, as POSIX allows. Anonymous
shm_open objects and JIT shared memory descriptors are temporary files on
Windows, so mmap now maps them too (it failed with EOPNOTSUPP before).
mprotect of a mapped view above the view's protection fails with EACCES
before any bookkeeping changes. The cases Windows cannot represent are
listed in the libkernel debt.
Dusklight (PPSA99305) uses SQLite in WAL mode, which maps its -shm file.
On Windows main f0634088b every start logged "Failed to start
transaction: disk I/O error" for its Dawn and pipeline caches (3 of 3
runs); with this change the caches initialise and their WAL files fill
(4 of 4 runs). The title reaches its menu either way.
A shared mapping may end inside the 16 KiB page that holds the end of
the file, as POSIX mmap allows: the view ends with the file's last 4 KiB
host page, which Windows zero-fills past the end, the rest of the guest
page is private zero pages, and the file never grows. A mapping that
reaches a whole page past the end still fails with EOPNOTSUPP.
PS5LM Plus (PPSA99581) loads its model with llama.cpp, which maps the
whole GGUF file shared and read-only. With a 491,400,032-byte model,
Windows main 5de7a46e0 and this change without the last page logged
"llama_model_load: error loading model: mmap failed: Operation not
supported" (3 of 3 runs); now the model loads and the completion of
"The capital of France is" starts with " Paris" (3 of 3 runs).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fopen resolved the random device paths to a host file that does not
exist and failed with ENOENT, while open and sceKernelOpen already serve
them from host entropy. A read-only fopen of either path now returns a
stream whose fread fills the buffer with host entropy. setvbuf, feof,
ferror, clearerr, the stream locks and fclose work on it, fflush returns
0 as on a FreeBSD stream that is not writing, other stream functions
throw, and opening the paths for writing throws as in libkernel.
ProsperoTV (PPSA99003) reads its phone pairing code this way and showed
"Code expired or unavailable".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The console's libSceLibcInternal.sprx (measured on firmware 12.70 in
#7549) prints %p as bare lowercase hex without a 0x prefix, prints 0 for
a null pointer, ignores #, + and the space flag, counts only the digits
towards the width, pads with spaces on the left (on the right for - and
with zeros for 0 when there is no precision), treats the precision as
the minimum number of digits, prints nothing for a null pointer under
%.0p, and accepts the l length. FormatPointer instead followed FreeBSD
and printed 0x..., which is what a title does not see on hardware.
FormatPointer now prints the digits alone, the narrow Windows formatter
accepts %lp, and Linux routes %p and %lp to the guest formatter instead
of glibc, which prints the FreeBSD form and (nil) for a null pointer.
The wide family still refuses %lp, because the console's swprintf was
not measured with it; it returns -1 as it does for any other conversion
the formatter rejects. The debt note records the pointer cases the
console was not measured with and what the Linux routing means for a
format that mixes %p with a refused conversion.
fopen and freopen in an append mode gave the C runtime a descriptor with
_O_APPEND, which seeks to the end of the file and then writes. Streams on
separate descriptors overwrote each other's data. FreeBSD opens these
streams with O_APPEND, so every flush lands at the end of the file.
Open append streams with FILE_APPEND_DATA but without FILE_WRITE_DATA,
so Windows writes each flush at the end of the file. ftruncate and
flock reopen the file to get the access such a handle lacks.
Ship of Harkinian (PPSA99320) logs its heap use with malloc_stats_fast
every 10 s and stopped on the first call, since only an application
allocator could answer it. The default heap now keeps its system and
in-use byte counts with their peaks and fills the same managed-size
block as sceLibcMspaceMallocStatsFast, whose layout moves to a shared
header.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Metal Gear Solid 4 (PPSA18194) aborts on main when its launcher starts
the game: sceSystemServiceLoadExec runs the guest shutdown, a render
thread that then calls into the stopped AGC driver gets ProcessShutdown
back from LibcAwaitExit, and the exception unwinds into title code,
which terminates the process before the new executable starts.
LibcAwaitExit parks a thread only once an exit has been requested, and
LoadExec ran the shutdown without requesting one. It now starts the
shutdown the way exit does, so such threads wait until the process is
replaced.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fopen left the host's errno in place when the host open failed, so on Linux a symlink loop reached the title as 40 and an over-long name as 36, which are EMSGSIZE and EINPROGRESS on FreeBSD. GuestErrnoFromHost_nid_no_patch maps the host value to FreeBSD 11.4's: 1 to 34 are shared except EAGAIN (35), and ELOOP, ENAMETOOLONG, ENOTEMPTY, EDQUOT, EOVERFLOW and EDEADLK get their FreeBSD numbers; anything else is EIO.
Titles that bundle OpenSSL load the console's CA file from the sandbox
directory and fail every HTTPS request without it. ProsperoRadio
(PPSA99001, libcurl 8.18.0 + OpenSSL 3.5.2) on main + #4796 + #4525:
[ProsperoRadio][http] https://all.api.radio-browser.info/json/servers: Problem with the SSL CA cert (path? access rights?) (error adding trust anchors from file: /gkbHtXc3Wq/common/cert/CA_LIST.cer)
SwanStation (PPSA98510) stops on the same file (#5001).
The guest path resolver now accepts exact-path providers, and libkernel
registers one for /<sceKernelGetFsSandboxRandomWord()>/common/cert/CA_LIST.cer.
On first use it writes a PEM bundle once per process to
cert_cache/CA_LIST.cer in the run directory, read-only: on Windows the
current user's ROOT and CA system stores, elsewhere SSL_CERT_FILE or the
first distro bundle that exists. Without a certificate the path keeps
resolving under the guest root and does not exist, as before. Every other
path under the sandbox word is unchanged. The console file's format and
the rest of its common/cert directory are unverified and recorded in
docs/dev/debt/libkernel.md.
With this change, #4796, #4525 and #5025 on Windows 11, 60 s each:
[ProsperoRadio][http] 11846160 bytes in 1458 ms: 1161 passes, 803 waits (679 woken) for 1428 ms
[ProsperoRadio][catalog] page offset=0 bytes=11846160 objects=10000 accepted=9549 total=9549
...
[ProsperoRadio][catalog] page offset=60000 bytes=1712035 objects=1494 accepted=1436 total=59371
[SwanStationPS5] update: latest 1.2.5, this 1.1.3, available
[SwanStationPS5] gamedb: index written
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Unreal titles end in their own libc fatal path with the UTF-16
LowLevelFatalError message in a register, and the crash report shows
only the first 64 bytes of it as hex words, so the message is lost
(#4238: Demon Slayer PPSA03490, Stray PPSA02100).
For every general register that points at readable memory starting
with at least four printable ASCII characters, as UTF-16LE or as single
bytes, the report now prints the text up to a NUL, the first other
character, 512 characters or the end of readable memory (each page is
checked with VirtualQuery before it is read).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A ZArchive named app0.zar beside app0/ is a read-only layer under it. libc opens it at the first path resolution, looks paths missing from app0/ up case-insensitively, reads files across compressed blocks and keeps the archived descriptor table with shared positions.
Resolving an archive-only path through a call that does not read the archive throws instead of failing with ENOENT, as do opendir of a directory present in both places, fdopen of an archived descriptor and chdir into the archive. A path through .. in an archive-only directory resolves.
A GPU hang or a crash in a title leaves no record of what the queues
were last given: APS5_TRACE_GPU prints every submission (too much to
leave on), and the WAIT_REG_MEM timeout message shows only the packet
before the wait on the waiting queue.
Each queue now keeps a ring of its last 16 submissions
(APS5_PM4_RING=<n>, 0 to 4096, 0 disables): the TraceMs time in tenths
of a millisecond, serial, size, guest address and a copy of the first
512 dwords after indirect buffers are inlined, recorded under the
driver lock next to the serial, REWIND tails included. The ring of a
queue and the failure report registration are set up at the top of
Submit, before any output is reserved, so recording under the lock
does not allocate. Disabled, Submit only tests a static flag; enabled,
it costs one bounded copy per submission.
DumpSubmissionRings decodes the kept dwords with Pm4::PacketWords and
the new Pm4::KnownName, which returns the decoder's names as views of
static strings (empty for an unknown opcode, printed as UNKNOWN_0x%02x);
Pm4::Name now builds its std::string from it. Every line is formatted
with integer and string conversions only into a 512-byte stack buffer
and written with one WriteFile / write(2) on the stderr handle, so the
dump does not allocate from the C runtime heap, lines from other
threads do not interleave and the CRT stream lock is not taken (the
rule of #3871). The header gives the number of queues and the ring
size, and each queue line the submissions recorded and kept.
libc gets a small registry of failure reports
(LibcRegisterFailureReport_nid_no_patch, Unregister, Run); Run runs
the registered reports the first time it is called and returns at once
after that. The Windows crash report runs it after the thread dump of
an unhandled exception, after SIGABRT (which also covers the
APS5_HANG_DUMP_SECS timed dump, since it aborts) and after an invalid
CRT parameter, so an abort after a reported exception does not print
the rings twice. The driver registers its dump at the first submission
and unregisters at static destruction; the dump there waits a bounded
time for each ring lock. What the crash-path dump cannot do (a ring
held by the crashed thread, rings corrupted by the crash, Linux) is
recorded in technical debt.
APS5_STALL_WATCHDOG_SECS=<n> (0 to 86400, 0 disables) makes a waiter
that has not progressed for n seconds dump the rings once per wait: a
WAIT_REG_MEM (progress as for APS5_GPU_WAIT_TIMEOUT_MS: another packet
runs or completes), a REWIND waiting for the title, and a compute
submission held behind queue 0. The last one prints under the driver
lock, so submissions and the other queues wait for it (USAGE.md says
so). Both variables are read at the top of Submit, so an invalid value
of either throws at the first submission and never later inside a GPU
worker. The order wait behind queue 0 releases its order hold on scope
exit, so a throwing dump cannot leave it counted.
No packet semantics change; packet sizes and names are those of the
existing decoder (PM4 type-3 header layout as in Mesa
src/amd/common/sid.h). The queue bound of Submit and of the rings is
one constant, SubmissionQueueEnd.
Tests: agc_driver_pm4_tests gets ten modes, each its own ctest:
agc_driver_pm4_ring (APS5_PM4_RING=4, six submissions on queue 0 and
two on 0x20: the dump holds #3..#6 and #1..#2 in order, nothing
older), agc_driver_pm4_ring_empty (one line), _ring_disabled (one
line), _ring_invalid and agc_driver_pm4_stall_watchdog_invalid
(APS5_STALL_WATCHDOG_SECS=1x: the submission throws and is not
executed), _ring_report (LibcRunFailureReports_nid_no_patch dumps the
rings), _ring_report_heap (Windows: with ucrtbase.dll's HeapAlloc and
HeapReAlloc imports counting calls from the reporting thread, the
failure report of a submission holding INDEX_BUFFER_SIZE names it and
allocates nothing, while Pm4::Name of the same header is seen
allocating), and with APS5_STALL_WATCHDOG_SECS=1 a WAIT_REG_MEM
(agc_driver_pm4_stall_watchdog), a REWIND (_stall_watchdog_rewind) and
a compute submission held behind a queue 0 rendering wait
(_stall_watchdog_order), each held for about 2.5 s, dump once and
complete when released. The PM4 catalog test checks KnownName against
every opcode and custom name. windows_backtrace_failure_report checks
that abort() runs the registered reports once and not an unregistered
one; windows_backtrace_failure_report_once that abort() does not run
them again after an earlier run. With the dump decoding through
Pm4::Name, agc_driver_pm4_ring_report_heap fails with "the failure
report allocated from the C runtime heap"; with Run resetting its
flag, windows_backtrace_failure_report_once prints the report twice.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SonyCrt.hpp kept the constraint handler slot in an unnamed namespace,
so every source file that included it had its own slot. A handler
installed with set_constraint_handler_s in WideConversion.cpp was not
the one qsort_s in FileAndHeap.cpp called, and its violations aborted.
The slot and its mutex now have one instance per module, and the
handler type uses the guest calling convention, as the handlers a title
installs and the default handler it gets back are guest functions.
Lines queued behind the logger were lost when the process ended through
abort(), std::terminate, the Windows crash report or a guest _Exit,
because only driver failure, Driver::stop and a normal exit flushed the
queue.
libc keeps one end flush (LibcSetEndFlush_nid_no_patch) and runs it from
those paths. Its first registration chains a SIGABRT handler in front of
the previous one, so every abort() and std::terminate in the process is
covered, on Linux as well; with the default disposition the handler
restores it and raises again, so raise(SIGABRT) still ends the process.
Guest abort() and _Exit, and ReportCrash for the first fatal exception,
run the flush directly. AsyncLog registers a bounded flush (FlushFor, 2 s)
that polls two counters and takes no lock, so a dying process cannot hang
on a blocked stream or on the queue lock; after its first timeout it
returns at once. Flush counts queued and finished entries instead of
watching the queue and the writer flag, so it waits for the lines queued
before the call and not for a moment in which no thread is logging.
agc_async_log_fatal runs the test executable as a child that queues lines
behind a held stream and ends in abort(), std::terminate(), raise(SIGABRT)
(POSIX) or an access violation (Windows), and once more with the stream
never released. The lines of the first four must arrive in order, the last
child must end after the bound, and every child has a watchdog.
agc_async_log gains the bounded flush case.
Texture creation logged one stdout line per texture and per mip through
WriteLogLine, which formatted with the mingw printf and flushed every
line on the queue worker (about 786k lines in a 25 minute cave run).
AsyncLog keeps a bounded queue and a logger thread. AGC_LOG_DEFERRED
captures numeric arguments on the caller and formats them on the logger
thread; WriteLogLine hands finished lines to a per-module sink in
libc/LogMacros.hpp that AgcDriver installs, so all AgcDriver stdout
lines keep their order and are written one flush per batch. A full queue
blocks the caller, so no line is dropped. Flush runs on driver failure,
shutdown and at exit. APS5_LOG_SYNC=1 restores the synchronous path.
Identical "deferred write-back failed" notes are printed three times per
10 s and then counted into one "repeated N more times" line. A
"[log] 10 s" line reports lines, bytes, caller time and logger time
(APS5_LOG_STATS=0 hides it).
GuestWriteWatchCovers says whether a range is watched in full. The
Linux and macOS watches unregister page ranges, so a range can hold
watched pages between unwatched ones, and code that must act on every
watched byte of a store has had no way to find them short of asking
page by page. GuestWriteWatchOverlaps answers with one lookup in the
registered ranges, like Covers.
The macOS write-watch test and agc_write_tracking on Linux check it
around watched pages between unwatched ones, at both edges and for an
empty range.
Retain Pipodi safety guard from closed PR5851 without its resident alias expansion or automatic AGC alias routing. Main still refuses aliases over partial resident runs. Five unmodified-main regression runs fail before this guard; positive validation pending. Fork proposal only; no duplicate upstream PR intended.
(cherry picked from commit 078a5740b4)
Adrià