Main's #7269 keeps the synchronous settle's buffer, memory and fence
per device in ImportSettle, so clearing one device's imports releases
only that device's objects. This branch's settle batches move there
too: each device's ImportSettle holds its batch target, memory and
batches with their fences, so switching devices no longer drops a
device's batch handles, and clearing a device waits for and destroys
only its own batches. The per-import settle state is renamed
pendingSettles to stay clear of main's per-device map, and a submitted
settle records the device whose batch it is in.
MayEvaluate and the evaluator track the first-lane reads they are inside, so a read reached again through a loop-carried phi or a constant slot is not evaluable instead of recursing.
The deferred settle of small Windows imports could only be turned off
through the test hook. The variable turns it off for a run, so a title can
be compared with and without it on one build; unset, nothing changes.
Main's #5905 settles Linux dma-buf imports with settleImport, which is
now built on both platforms. The batched settle of Windows imports under
1 MiB, its counters and adoptSettleDevice's batch reset stay Windows-only,
and ClearHostImports drains the batches only there.
The automatic window needed 8 logical processors, which a 4-core CPU with SMT passes while the waited-for thread may run on a sibling thread, and ignored the process affinity. It now needs more than 8 processors in the affinity mask (CpuTopology), falling back to hardware_concurrency.
On Windows the atexit handlers of a DLL run after the process has ended
its other threads, so the logger thread can be gone with lines still
queued and the unbounded Flush waits forever (guest_shader_link timed
out on 4 of the last 7 completed main builds since #6371). Run the bounded
FlushAtEnd there instead.
Co-Authored-By: Claude <noreply@anthropic.com>
At the Minecraft Dungeons II title the queue-0 worker, the draw committer and the compute worker
hand work to each other 49,000 times per 10 s through the GPU mutex, the committer's condition
variable and DrawPipeline::Drain, and each hand-off that has to wait costs a kernel wake-up. The
three workers are busy 21-34 % of the time: the GPU idles because the chain of hand-offs is slow,
not because a thread runs out of CPU.
GpuMutexType::lock retries try_lock for a window before it blocks, the committer polls its item
count for the window before it waits on its condition variable, and Drain polls for twice the
window before it waits for the committer. A hand-off that completes within the window costs no
wake-up. The window is 150 us on machines with at least 8 logical processors and 0 below that;
APS5_HANDOFF_SPIN_US sets it (0 turns it off). Ordering and the lock's semantics are unchanged.
The test agc_handoff_spin covers the window policy, the spin helpers and contended locks, and
docs/dev/ARCHITECTURE.md describes the spin.
The stage-A pre-sync of a dispatch listed only the surfaces that stage B reads on the CPU. The DCC
keys of an image that a recorded kernel writes were not in the list, so stage B waited for that
kernel in WaitForKeyWriters with the GPU mutex held and every other worker queued behind the wait.
At the Minecraft Dungeons II title the pre-sync fired once in 296,000 dispatch checks while
WaitForKeyWriters synchronized 400 times per 10 s.
PresyncSurfaces now also returns the key range of every image binding (PresyncSurface::dccKeys).
presync waits for a key range only while recorded work writes it, the key-store memo does not
already give its value (DccKeyWritersPending) and, under the mutex it takes for a moment anyway, the
pending stores do not give the keys either (DccKeyWriterNeedsSync, the test WaitForKeyWriters makes
before it synchronizes). Which keys the dispatch reads, and when, is unchanged: the locked lookup runs
as before and finds nothing pending.
APS5_NO_DCC_PRESYNC=1 leaves the key ranges out. The [presync] line counts key ranges checked,
pre-syncs that only the keys required and the outcomes of WaitForKeyWriters. The test
agc_driver_dcc_key_presync covers a kernel that writes the keys, a recorded fill of the keys and
nothing pending, and skips with the reasons check_skips.py expects when the device has no timeline
semaphores or no host import.
A graphics pipeline the store builds or a compute pipeline a dispatch builds is
appended to pipelines.journal next to the shader cache entries, as the
materialized result of each stage (the specialized SPIR-V, bindings and push
constants) plus the recipe the pipeline was built from: the stages'
identities, vertex input layout, descriptor set layout keys and the state
fields the Pipeline constructor reads. When a VulkanDevice is created with a
window (a headless device waits for its first draw or dispatch, since the
window's device usually replaces it first) the journal is loaded and its
pipelines are built on a pool of threads while the title loads, so the first
draw or dispatch of a configuration seen on an earlier run finds its pipeline
resident instead of compiling it on the queue worker.
The pipeline stores key on RecompileResult::Identity, a hash of the module
words, specialization constants, subgroup width and bindings that is the same
on every run (SharedSpirv::ContentHash, computed once per shared module),
rather than on the process-lifetime variant ids; a generated stage is keyed
like any other. CachedPipeline builds outside the store's mutex and the
library parts and link are built outside the library store's mutex, so replay
threads and queue workers build in parallel; a concurrent build of the same
key keeps the first inserted. A replayed pipeline enters the store at its
least recently used end and a full store takes none, and its optimized relink
waits for the first draw that binds it, so the title's own pipelines come
first in both. The compute pipeline creation moves into
VulkanDevice::createComputePipeline, shared by dispatches and the replay. The
store key also covers the color component mapping, the depth target's sample
count and a depth bias clamp, which the constructor reads.
The file names the device it was written for (the properties the driver's
pipeline cache file is named by) and starts over for another; a torn tail is
cut at a record boundary; two runs sharing the directory take turns on a lock
file and keep each other's records; a read failure never starts the file
over. A device cancels its replay before it is replaced or destroyed, and a
replaced device destroys its pipelines then rather than at the driver's stop.
APS5_NO_PIPELINE_JOURNAL=1 disables it; APS5_PIPELINE_JOURNAL_THREADS,
APS5_PIPELINE_JOURNAL_ENTRIES (the first recorded), APS5_PIPELINE_STORE_ENTRIES
and APS5_PIPELINE_JOURNAL_MAX_MIB bound it; APS5_PROFILE_DRAW prints a
[pipeline-journal] summary. APS5_VERIFY_SET_LAYOUTS=1 now also checks a
dispatch's set layout against the one described from its shader. The lavapipe
test suite, which shares one cache directory, runs with the journal off except
for the journal's own tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6hw3kRAuQdfcU9YCZgaoK
A Windows import under 1 MiB returns to the write watch once its settle
read has completed, though main-queue work submitted before the read may
have used the import first, and an earlier import of the same pages may
not have run its first use yet. On the NVIDIA driver (617.42) neither
marks the pages once the read has completed, so the rewatch needs no wait
for the main queue. The docs said the marking lasts until the import's
first GPU use completes; they now say that a completed GPU use ends it,
the settle read included.
agc_import_settle_first_use submits 4096 copies of a 64 KiB import on
the main queue before its settle read is queued. The import must return
to the write watch while the copies still run, and the driver must mark
none of its pages afterwards.
agc_import_settle_earlier_import records 4096 copies of a 64 KiB window
import, replaces it with the import of its whole 128 KiB range, and
submits the copies only once the range is back in the write watch. The
driver must mark none of the range's pages while they run. A driver that
keeps marking fails either test.
Retiring a Windows import whose settle read was still running waited for
the read's batch under the imports mutex, so lookups on every thread
waited with it. The retired import is now kept with that batch and freed
once the batch has completed. Only a guest unmap of its memory still
waits for the batch, in the pin waiter, which waits for the main queue
there anyway.
The [buffers] profile line counts these retirements and the unmaps that
waited, apart from the imports of 1 MiB or more settled inside their
window, which the old wait counter mixed with the retirement waits.
agc_import_settle_retire replaces a 64 KiB window import with the import
of its whole range, 64 times, right after its settle read was submitted.
No replacement may wait, at least one must find the read running, and
the last import must return to the write watch.
ARCHITECTURE.md and the debt entry describe how imports under 1 MiB are
settled without waiting and return to the write watch, which the debt
entry listed as not implemented.
The logger thread, the bounded queue that blocks its caller, the LogSink
hook, APS5_LOG_SYNC and APS5_LOG_STATS, where the queue is flushed and
what stays outside it: direct fprintf(stderr) and printf calls are written
at once, so they can appear before older queued lines.
APS5_LAZY_SHADER_PREPARE=1 keeps registration validation and decoding but skips queueing the preparation. Draws and dispatches prepare a deferred snapshot on first use and cache it; explicit ABI resolution still prepares its stages. Titles that register thousands of shaders at boot no longer hold every prepared shader in memory before the first frame.
Cover lazy dispatch, metadata and failure cases in both asynchronous and synchronous modes.
The kernel maps a dma-buf import's pages for the GPU at the first submission that uses it. On RADV
that took about 2 s for a 3 GiB import, in the second frame of Hades (PPSA03355), and its staging
pool ran out while the frame was held.
Linux now makes the settle queue from #6060 when the device imports dma-buf memory, and settles
every dma-buf import of 1 MiB or more right after it is made with settleImport's 4-byte copy. The
settle queue needs neither the GPU mutex nor the main queue's work to finish. A dma-buf import's
buffer is shared with the settle queue's family instead of aliased, since a sparse import is bound
to many memory objects. Where the sparse queue already takes that family, the settle queue is a
second queue of it.
Queue 0 now pipelines draw commits unless APS5_PIPELINED_DRAWS=0, and draw write ranges use the exact
color, DCC, CMASK, depth and stencil sizes unless APS5_EXACT_DRAW_WRITES=0. The base tests now run the
pipeline; agc_driver_pm4_failure_inline and agc_driver_draw_cache_reuse_inline keep the inline path
covered.
Co-Authored-By: Claude <noreply@anthropic.com>
The second queue that settles imports is always created on Windows, and a device without one keeps
the comparisons. The other-family settle path is tested through the recorder test device built
with its settle queue on another family (agc_import_watch_other_family) instead of
APS5_SETTLE_OTHER_FAMILY.
Main's flatVolumeLineCompatible change also took format 34 with another layout, so entries of either would load as the other.
Co-Authored-By: Claude <noreply@anthropic.com>
Move bounded vertex folding and empty-image feedback into separate topics. Keep the replay coverage proof, stage contracts, finite-reference tests and mapped cache regressions; version the stencil layout separately as format 34.
Co-Authored-By: Codex <noreply@openai.com>
A plan copied every control-flow condition with all its operands, though one
that depends on a vector register is never evaluated. Such conditions are
dropped, and the copied values live in one arena without use lists.
Build packs from EncodeEntry output and check the hit and miss paths,
an entry stored under another key, a truncated entry, a damaged index,
an out-of-range record, a foreign compiler source version and an
unknown format. A child process covers pack discovery on a real disk
load, the ANYPS5_NO_PREBUILT_SHADERS opt out and fall-through to the
writable cache. ARCHITECTURE.md describes where the prebuilt lookup
sits.
Plan runtime-indexed T# and S# columns read through a 64-bit pointer.
Preserve the console offset alignment, signed immediate and key bounds.
Track the descriptor dwords consumed by each table in its invocation.
Reject table reads when a writable buffer or storage image overlaps them.
Cover the overlapping-storage fault and isolated second-table behavior.
Materialize runtime-indexed image and sampler records into descriptor
tables while keeping descriptor selection in the shader. Report invalid
active entries through the existing deferred fault buffer.
Bind fragment tables in the fragment set when pipeline libraries split it,
and reserve a physical descriptor for empty tables. Reject unsupported
sampler, border, cube and anisotropic pairings per entry. Keep one column
per heap access shape, one T# shared by several image operations included
(milescoviello's #5550 case), and narrow it to the union of its keys'
domains when every key has one (SleglJan's key-domain changes). Plan
columns read at (key << n) & mask over their whole domain (tibzejoker's
fix). Read records in runs through SleglJan's bulk reader, and reject
entries whose surface has no mapped byte, as Tamalero proposed. Store
table columns in disk plans; update the runtime ABI and disk format. Bound
table-sized entries in the resource cache (65,536 table elements) and the
dispatch cache (256 MiB of captured words), evicting LRU.
Every library under core/libs/prx links the libc and libkernel import
libraries, and every NID patch step depended on the unpatched libc.prx.
GNU ld 2.46 rewrites the import library on every link with new archive
timestamps (the member headers and the symbol map header), so the restat
on the link edge never pruned anything: one libc or libkernel .cpp edit
relinked all 124 libraries and re-patched all of them.
On MinGW the shared library link now writes the import library to
<implib>.new with SOURCE_DATE_EPOCH=0 (ld then writes zero member
timestamps), makes the symbol map deterministic with ranlib -D and copies
the result over the import library only when its bytes differ
(cmake -E copy_if_different). The link edge keeps restat, so a link that
keeps the exports leaves the import library untouched and prunes every
dependent link. The SOURCE_DATE_EPOCH also zeroes the export directory
TimeDateStamp of the DLL; the COFF header TimeDateStamp that the unwinder
keys on is unchanged.
The NID patch steps only use the names libc exports. nid_patcher
--write-export-list writes those names, normalized and sorted, to
libc.exports and leaves the file untouched when they did not change; the
patch steps read it with --preserve-export-list and depend on it instead
of libc.prx, on every platform. A patched library is byte-identical to
one patched against libc.prx.
Measured on Windows (GCC 15.2, -j 4, loaded machine) after touching
libc/src/Strings.cpp: with CMAKE_LINK_DEPENDS_NO_SHARED off, 124 links and
124 patches in 74.9 s before, 1 compile, 1 link, the export list and the
libc patch in 4.3 s after; with it on, 124 patches in 7.8 s before and
the same four steps in 5.3 s after. Touching libkernel/Time/Time.cpp
runs 123 links and 123 patches (89.1 s) before and one compile, link and
patch (3.4 s) after. Removing a libc export still relinks and re-patches
all 124 libraries.
nid_patcher_macho covers --write-export-list (sorted normalized names, no
rewrite when unchanged, rewrite when changed) and --preserve-export-list
(the same exports kept as with --preserve-exports; malformed lists and
two references refused). It fails on main, where the option is unknown.
Sources: SOURCE_DATE_EPOCH, https://reproducible-builds.org/specs/source-date-epoch/;
ranlib -D and ld --out-implib, GNU binutils documentation; ninja restat,
https://ninja-build.org/manual.html#ref_rule.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A registered range larger than 64 MiB is imported one 64 MiB window at a
time for buffer and fill accesses instead of whole, and the live imports
stay under a budget: APS5_HOST_IMPORT_MIB, else the larger of 6 GiB and
half the physical memory, at most the largest host heap less 2 GiB
(imports and type-3 mirrors share it; #167's rule). An import past the budget
retires the least recently used imports not used in the last 256
lookups.
- An access across window edges imports the cells it touches, grown to
the union of the range's imports it overlaps, so uses across
neighbouring edges share one import instead of replacing each other.
- An address-based build (AcquireRegistered) imports the range whole; a
window over its first byte does not serve it.
- Every refusal (alignment, a failed run, a held import, the budget with
what may be evicted) is decided before anything is retired.
- An import made when a range is mapped never evicts: the map observer
holds no GPU lock.
The window size and the budget have setters for the tests.
A texture over a depth surface could only sample one 2D slice of the
plane's own format; 2D arrays, cubes and the R16_UINT view of a D16
plane were refused ("sampling the depth plane of depth surface ... is
not implemented"). A one-slice 2D array now views the depth image as an
array. Several slices, a cube, or an integer view of a D16 plane are
copied into a DepthPlaneCopy image at every use: each slice from the
depth surface bound at its address (base plus one 64 KiB-block depth
slice per layer), otherwise from guest memory, detiled. A depth-bits
R16_UINT view (a depth swizzle) keeps the depth copy, since the shader
converts the sampled depth to its bits itself (GTA V's compute
0x242d4a200 samples a D16 plane array that way, reported by
@tibzejoker). Such a texture is not kept in resource templates.
Storage-image access to depth surfaces is no longer part of this
change; #4500 implements it.
TechnicalDebt (Depth plane copies) and ARCHITECTURE describe the copy and
what still throws.
The memo stamp held TrackerGeneration, which every collect and driver store
advances, so it almost never matched. Driver stores into uncovered ranges
record their generation per 64 KiB block, and NewestStamp of the surface
replaces TrackerGeneration in the stamp.
The guest-access guard covers CB bytes times samples (the single-sample layout size alone missed the later sample planes' tile padding), with a test row in that padding. Remove two TechnicalDebt lines duplicated by the rebase, document the guard bound and the unevicted targets, and add the multisampled targets to ARCHITECTURE.md.
sceAgcFuseShaderHalves returns a fused hull as a copy of its back half
(type 7) that is not registered: type 3, no user data, and shader
registers copied to scratch memory with the checksum, RSRC1/RSRC2 and
the LS program address rewritten. sceAgcCreatePrimState accepts only that
type-3 header, and graphics ABI resolution refused it as a replaced
shader header.
SameHeader now also takes a type-3 header as the registered type-7
header for the same code when every other field is equal and its shader
registers differ only in the fused ones. Graphics ABI resolution then
applies the bound header's shader registers over the registered state.
PPSA26344 fuses its hull halves during its loading screen.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Yunus Shaikh