On Windows the atexit handlers of a DLL run after the process has ended
its other threads, so the logger thread can be gone with lines still
queued and the unbounded Flush waits forever (guest_shader_link timed
out on 4 of the last 7 completed main builds since #6371). Run the bounded
FlushAtEnd there instead.
Co-Authored-By: Claude <noreply@anthropic.com>
A graphics pipeline the store builds or a compute pipeline a dispatch builds is
appended to pipelines.journal next to the shader cache entries, as the
materialized result of each stage (the specialized SPIR-V, bindings and push
constants) plus the recipe the pipeline was built from: the stages'
identities, vertex input layout, descriptor set layout keys and the state
fields the Pipeline constructor reads. When a VulkanDevice is created with a
window (a headless device waits for its first draw or dispatch, since the
window's device usually replaces it first) the journal is loaded and its
pipelines are built on a pool of threads while the title loads, so the first
draw or dispatch of a configuration seen on an earlier run finds its pipeline
resident instead of compiling it on the queue worker.
The pipeline stores key on RecompileResult::Identity, a hash of the module
words, specialization constants, subgroup width and bindings that is the same
on every run (SharedSpirv::ContentHash, computed once per shared module),
rather than on the process-lifetime variant ids; a generated stage is keyed
like any other. CachedPipeline builds outside the store's mutex and the
library parts and link are built outside the library store's mutex, so replay
threads and queue workers build in parallel; a concurrent build of the same
key keeps the first inserted. A replayed pipeline enters the store at its
least recently used end and a full store takes none, and its optimized relink
waits for the first draw that binds it, so the title's own pipelines come
first in both. The compute pipeline creation moves into
VulkanDevice::createComputePipeline, shared by dispatches and the replay. The
store key also covers the color component mapping, the depth target's sample
count and a depth bias clamp, which the constructor reads.
The file names the device it was written for (the properties the driver's
pipeline cache file is named by) and starts over for another; a torn tail is
cut at a record boundary; two runs sharing the directory take turns on a lock
file and keep each other's records; a read failure never starts the file
over. A device cancels its replay before it is replaced or destroyed, and a
replaced device destroys its pipelines then rather than at the driver's stop.
APS5_NO_PIPELINE_JOURNAL=1 disables it; APS5_PIPELINE_JOURNAL_THREADS,
APS5_PIPELINE_JOURNAL_ENTRIES (the first recorded), APS5_PIPELINE_STORE_ENTRIES
and APS5_PIPELINE_JOURNAL_MAX_MIB bound it; APS5_PROFILE_DRAW prints a
[pipeline-journal] summary. APS5_VERIFY_SET_LAYOUTS=1 now also checks a
dispatch's set layout against the one described from its shader. The lavapipe
test suite, which shares one cache directory, runs with the journal off except
for the journal's own tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6hw3kRAuQdfcU9YCZgaoK
The logger thread, the bounded queue that blocks its caller, the LogSink
hook, APS5_LOG_SYNC and APS5_LOG_STATS, where the queue is flushed and
what stays outside it: direct fprintf(stderr) and printf calls are written
at once, so they can appear before older queued lines.
APS5_LAZY_SHADER_PREPARE=1 keeps registration validation and decoding but skips queueing the preparation. Draws and dispatches prepare a deferred snapshot on first use and cache it; explicit ABI resolution still prepares its stages. Titles that register thousands of shaders at boot no longer hold every prepared shader in memory before the first frame.
Cover lazy dispatch, metadata and failure cases in both asynchronous and synchronous modes.
Queue 0 now pipelines draw commits unless APS5_PIPELINED_DRAWS=0, and draw write ranges use the exact
color, DCC, CMASK, depth and stencil sizes unless APS5_EXACT_DRAW_WRITES=0. The base tests now run the
pipeline; agc_driver_pm4_failure_inline and agc_driver_draw_cache_reuse_inline keep the inline path
covered.
Co-Authored-By: Claude <noreply@anthropic.com>
The second queue that settles imports is always created on Windows, and a device without one keeps
the comparisons. The other-family settle path is tested through the recorder test device built
with its settle queue on another family (agc_import_watch_other_family) instead of
APS5_SETTLE_OTHER_FAMILY.
Main's flatVolumeLineCompatible change also took format 34 with another layout, so entries of either would load as the other.
Co-Authored-By: Claude <noreply@anthropic.com>
Move bounded vertex folding and empty-image feedback into separate topics. Keep the replay coverage proof, stage contracts, finite-reference tests and mapped cache regressions; version the stencil layout separately as format 34.
Co-Authored-By: Codex <noreply@openai.com>
A plan copied every control-flow condition with all its operands, though one
that depends on a vector register is never evaluated. Such conditions are
dropped, and the copied values live in one arena without use lists.
Build packs from EncodeEntry output and check the hit and miss paths,
an entry stored under another key, a truncated entry, a damaged index,
an out-of-range record, a foreign compiler source version and an
unknown format. A child process covers pack discovery on a real disk
load, the ANYPS5_NO_PREBUILT_SHADERS opt out and fall-through to the
writable cache. ARCHITECTURE.md describes where the prebuilt lookup
sits.
Plan runtime-indexed T# and S# columns read through a 64-bit pointer.
Preserve the console offset alignment, signed immediate and key bounds.
Track the descriptor dwords consumed by each table in its invocation.
Reject table reads when a writable buffer or storage image overlaps them.
Cover the overlapping-storage fault and isolated second-table behavior.
Materialize runtime-indexed image and sampler records into descriptor
tables while keeping descriptor selection in the shader. Report invalid
active entries through the existing deferred fault buffer.
Bind fragment tables in the fragment set when pipeline libraries split it,
and reserve a physical descriptor for empty tables. Reject unsupported
sampler, border, cube and anisotropic pairings per entry. Keep one column
per heap access shape, one T# shared by several image operations included
(milescoviello's #5550 case), and narrow it to the union of its keys'
domains when every key has one (SleglJan's key-domain changes). Plan
columns read at (key << n) & mask over their whole domain (tibzejoker's
fix). Read records in runs through SleglJan's bulk reader, and reject
entries whose surface has no mapped byte, as Tamalero proposed. Store
table columns in disk plans; update the runtime ABI and disk format. Bound
table-sized entries in the resource cache (65,536 table elements) and the
dispatch cache (256 MiB of captured words), evicting LRU.
Every library under core/libs/prx links the libc and libkernel import
libraries, and every NID patch step depended on the unpatched libc.prx.
GNU ld 2.46 rewrites the import library on every link with new archive
timestamps (the member headers and the symbol map header), so the restat
on the link edge never pruned anything: one libc or libkernel .cpp edit
relinked all 124 libraries and re-patched all of them.
On MinGW the shared library link now writes the import library to
<implib>.new with SOURCE_DATE_EPOCH=0 (ld then writes zero member
timestamps), makes the symbol map deterministic with ranlib -D and copies
the result over the import library only when its bytes differ
(cmake -E copy_if_different). The link edge keeps restat, so a link that
keeps the exports leaves the import library untouched and prunes every
dependent link. The SOURCE_DATE_EPOCH also zeroes the export directory
TimeDateStamp of the DLL; the COFF header TimeDateStamp that the unwinder
keys on is unchanged.
The NID patch steps only use the names libc exports. nid_patcher
--write-export-list writes those names, normalized and sorted, to
libc.exports and leaves the file untouched when they did not change; the
patch steps read it with --preserve-export-list and depend on it instead
of libc.prx, on every platform. A patched library is byte-identical to
one patched against libc.prx.
Measured on Windows (GCC 15.2, -j 4, loaded machine) after touching
libc/src/Strings.cpp: with CMAKE_LINK_DEPENDS_NO_SHARED off, 124 links and
124 patches in 74.9 s before, 1 compile, 1 link, the export list and the
libc patch in 4.3 s after; with it on, 124 patches in 7.8 s before and
the same four steps in 5.3 s after. Touching libkernel/Time/Time.cpp
runs 123 links and 123 patches (89.1 s) before and one compile, link and
patch (3.4 s) after. Removing a libc export still relinks and re-patches
all 124 libraries.
nid_patcher_macho covers --write-export-list (sorted normalized names, no
rewrite when unchanged, rewrite when changed) and --preserve-export-list
(the same exports kept as with --preserve-exports; malformed lists and
two references refused). It fails on main, where the option is unknown.
Sources: SOURCE_DATE_EPOCH, https://reproducible-builds.org/specs/source-date-epoch/;
ranlib -D and ld --out-implib, GNU binutils documentation; ninja restat,
https://ninja-build.org/manual.html#ref_rule.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A registered range larger than 64 MiB is imported one 64 MiB window at a
time for buffer and fill accesses instead of whole, and the live imports
stay under a budget: APS5_HOST_IMPORT_MIB, else the larger of 6 GiB and
half the physical memory, at most the largest host heap less 2 GiB
(imports and type-3 mirrors share it; #167's rule). An import past the budget
retires the least recently used imports not used in the last 256
lookups.
- An access across window edges imports the cells it touches, grown to
the union of the range's imports it overlaps, so uses across
neighbouring edges share one import instead of replacing each other.
- An address-based build (AcquireRegistered) imports the range whole; a
window over its first byte does not serve it.
- Every refusal (alignment, a failed run, a held import, the budget with
what may be evicted) is decided before anything is retired.
- An import made when a range is mapped never evicts: the map observer
holds no GPU lock.
The window size and the budget have setters for the tests.
A texture over a depth surface could only sample one 2D slice of the
plane's own format; 2D arrays, cubes and the R16_UINT view of a D16
plane were refused ("sampling the depth plane of depth surface ... is
not implemented"). A one-slice 2D array now views the depth image as an
array. Several slices, a cube, or an integer view of a D16 plane are
copied into a DepthPlaneCopy image at every use: each slice from the
depth surface bound at its address (base plus one 64 KiB-block depth
slice per layer), otherwise from guest memory, detiled. A depth-bits
R16_UINT view (a depth swizzle) keeps the depth copy, since the shader
converts the sampled depth to its bits itself (GTA V's compute
0x242d4a200 samples a D16 plane array that way, reported by
@tibzejoker). Such a texture is not kept in resource templates.
Storage-image access to depth surfaces is no longer part of this
change; #4500 implements it.
TechnicalDebt (Depth plane copies) and ARCHITECTURE describe the copy and
what still throws.
The memo stamp held TrackerGeneration, which every collect and driver store
advances, so it almost never matched. Driver stores into uncovered ranges
record their generation per 64 KiB block, and NewestStamp of the surface
replaces TrackerGeneration in the stamp.
The guest-access guard covers CB bytes times samples (the single-sample layout size alone missed the later sample planes' tile padding), with a test row in that padding. Remove two TechnicalDebt lines duplicated by the rebase, document the guard bound and the unevicted targets, and add the multisampled targets to ARCHITECTURE.md.
sceAgcFuseShaderHalves returns a fused hull as a copy of its back half
(type 7) that is not registered: type 3, no user data, and shader
registers copied to scratch memory with the checksum, RSRC1/RSRC2 and
the LS program address rewritten. sceAgcCreatePrimState accepts only that
type-3 header, and graphics ABI resolution refused it as a replaced
shader header.
SameHeader now also takes a type-3 header as the registered type-7
header for the same code when every other field is equal and its shader
registers differ only in the fused ones. Graphics ABI resolution then
applies the bound header's shader registers over the registered state.
PPSA26344 fuses its hull halves during its loading screen.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Registration prepares the variants a shader's registered state
predicts; a draw or dispatch whose state needs another one threw
"prepared shader artifact is missing for the requested static ABI".
Titles that link their own libSceAgc build that state themselves and
never call the driver's static ABI resolution, so their draws differ
from the registered prediction. The missing artifact is now prepared at use, logged, and kept
with the snapshot like the deferred and null pixel cases. SourceHandleFor
now checks that the request refers to the registered code, as
InvocationFor already did.
Uncharted: Legacy of Thieves Collection (PPSA05684) still misses on
main 197ffac9 (after the fixed push slots of #2661) for its own pixel
shaders: "prepared shader artifact is missing for the requested static
ABI: address=85899411712 stage=5 wave=64 pushOffset=128
pushCapacity=128".
Decodes SOP1 0x21 s_swappc_b64 (link pair = pc + 4) and resolves its call/return pairing through the existing s_setpc_b64 path: static call regions in GraphBuilder and the vertex-entry fetch-shader call inlined from the parsed fetch-shader plan. s_call_b64 (SOPK 0x16) and s_rfe_b64 (SOP1 0x22) throw at decode. Unmodeled cases throw; see docs/dev/TechnicalDebt.md.
Two diagrams in docs/dev/ARCHITECTURE.md: how the relinker and the NID-patched system libraries produce a native program, and the graphics path from command buffers through the shader recompiler to Vulkan. README only gains a link.
Co-Authored-By: Claude <noreply@anthropic.com>
