Read-only addresses are not part of the dispatch key, so the fresh build after a refused
move replaced the template that serves the other address, and the two addresses
rebuilt each other. A refused dispatch now uses or builds an entry under the exact key
(read-only bases included) and leaves the movable template alone. The [dispmove] line
counts the rebuilds after a refusal, the builds inserted beside the template and the
dispatches served by an exact entry. Restores the MarkGpuWrites comment.
A read-only V# the build clamps (size above the descriptor range limit cut to the
mapped end, stride overhang cut to the allocation end) was compared with the template
by its raw size, so an unchanged buffer counted as moved, was refused over the limit
and the dispatch rebuilt every time; a moved one with an overhang was snapshotted past
the allocation end. The build and the move proof now take the range from one function
(ShaderResources::guestBufferBytes), compare clamped with clamped and snapshot the
clamped range.
An overlay without a descriptor set is refused like one that could not be made, and
recording one throws instead of binding the template's set.
The content key of a compute stage leaves the base address and size of its
read-only guest buffers out, as a draw's key does. A template hit whose such
buffers moved is proved the way a draw's is (ShaderResources::MovedDispatchBuffers:
read-only, read in place when built, in reach, nothing pending over the new range)
and binds snapshots of the new ranges in a descriptor set of its own, kept with the
batch that records the dispatch. A refused proof builds the dispatch fresh, as
before. APS5_NO_DISPATCH_MOVE=1 turns the path off, APS5_DISPMOVE_MAX_KIB caps the
bytes snapshotted per dispatch.
A template whose storage image lies over memory that a depth surface covers is no
longer proved from stamps (fastRevalidate): cachedStorageTexture answers such a
binding with the plane's colour mirror, copied in from the depth image for every use,
and nothing stamps the appearance of a depth surface after the build. The full walk
decides, and a stale image can no longer be copied over the depth image.
Tests: Graphics.cpp (verdicts of the proof), execution DispatchMove (real device:
moved, exact, shortened, stored-over, own-output and written-range refusals) and
DispatchMoveDepth (a depth surface appearing over a template's plain storage image,
planes inside and outside the write watch, imports watched and unwatched).
agc_driver_rect_list_indirect compared the indirect draws' output with
the direct draws only, so it also passed when the records were read on
the CPU. Its records lived in a static array that is not guest memory,
so every indirect draw fell back to the CPU path as not imported.
The records now sit in a registered 64 KiB guest allocation, the test
turns on APS5_PROFILE_DRAW and reads the indirect-draw counters through
the new Graphics::IndirectDrawCounts, and it requires all four indirect
draws on the GPU path. It fails with APS5_NO_RECT_INDIRECT=1 and with
the IndirectDraw.cpp and Draw.cpp changes reverted (4 non-vertex path).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A program that stores to a 2D multisampled texture writes the samples of
its host multisampled color target through a storage view. Targets are
created without storage usage; the first storage binding that resolves
to a target moves it to an image with storage usage, copying its samples
once, and keeps the previous image until the target is released. Storage
usage needs the device's shaderStorageImageMultisample feature and its
format at the target's sample count. A surface no draw rendered to is
created from its uniform color memory, as for sampling; other color
memory still throws.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Shader validation accepts the extension only in fragment shaders and only when the device enabled shaderTerminateInvocation. The flag is part of the validation memo key.
Refs #2009
A graphics pipeline the store builds or a compute pipeline a dispatch builds is
appended to pipelines.journal next to the shader cache entries, as the
materialized result of each stage (the specialized SPIR-V, bindings and push
constants) plus the recipe the pipeline was built from: the stages'
identities, vertex input layout, descriptor set layout keys and the state
fields the Pipeline constructor reads. When a VulkanDevice is created with a
window (a headless device waits for its first draw or dispatch, since the
window's device usually replaces it first) the journal is loaded and its
pipelines are built on a pool of threads while the title loads, so the first
draw or dispatch of a configuration seen on an earlier run finds its pipeline
resident instead of compiling it on the queue worker.
The pipeline stores key on RecompileResult::Identity, a hash of the module
words, specialization constants, subgroup width and bindings that is the same
on every run (SharedSpirv::ContentHash, computed once per shared module),
rather than on the process-lifetime variant ids; a generated stage is keyed
like any other. CachedPipeline builds outside the store's mutex and the
library parts and link are built outside the library store's mutex, so replay
threads and queue workers build in parallel; a concurrent build of the same
key keeps the first inserted. A replayed pipeline enters the store at its
least recently used end and a full store takes none, and its optimized relink
waits for the first draw that binds it, so the title's own pipelines come
first in both. The compute pipeline creation moves into
VulkanDevice::createComputePipeline, shared by dispatches and the replay. The
store key also covers the color component mapping, the depth target's sample
count and a depth bias clamp, which the constructor reads.
The file names the device it was written for (the properties the driver's
pipeline cache file is named by) and starts over for another; a torn tail is
cut at a record boundary; two runs sharing the directory take turns on a lock
file and keep each other's records; a read failure never starts the file
over. A device cancels its replay before it is replaced or destroyed, and a
replaced device destroys its pipelines then rather than at the driver's stop.
APS5_NO_PIPELINE_JOURNAL=1 disables it; APS5_PIPELINE_JOURNAL_THREADS,
APS5_PIPELINE_JOURNAL_ENTRIES (the first recorded), APS5_PIPELINE_STORE_ENTRIES
and APS5_PIPELINE_JOURNAL_MAX_MIB bound it; APS5_PROFILE_DRAW prints a
[pipeline-journal] summary. APS5_VERIFY_SET_LAYOUTS=1 now also checks a
dispatch's set layout against the one described from its shader. The lavapipe
test suite, which shares one cache directory, runs with the journal off except
for the journal's own tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6hw3kRAuQdfcU9YCZgaoK
An in-place draw input flushed pending GPU writes over its range on the
CPU before the bind, a submit-and-wait of the open batch when a dispatch
in it wrote the buffer. The CPU never reads a vertex buffer's bytes, so a
vertex input under a pending write is bound in place behind a barrier from
shader and transfer writes to vertex attribute reads, recorded before the
draw's rendering begins; index inputs keep the flush, since the highest
index is read on the CPU. Minecraft generates a vertex buffer with a
compute dispatch every frame and drew from it one CPU wait later.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The headless device is replaced by the windowed one at window attach and then only
lingers for in-flight workers, but it parsed its own full copy of the cache file.
It now starts empty (it still merges what it compiles into the file). Load reuses
the file buffer as the initial data instead of copying it, and save() writes the
live data directly when the file still carries the header this object loaded or
wrote, skipping the read-back and the merge cache.
The kernel maps a dma-buf import's pages for the GPU at the first submission that uses it. On RADV
that took about 2 s for a 3 GiB import, in the second frame of Hades (PPSA03355), and its staging
pool ran out while the frame was held.
Linux now makes the settle queue from #6060 when the device imports dma-buf memory, and settles
every dma-buf import of 1 MiB or more right after it is made with settleImport's 4-byte copy. The
settle queue needs neither the GPU mutex nor the main queue's work to finish. A dma-buf import's
buffer is shared with the settle queue's family instead of aliased, since a sparse import is bound
to many memory objects. Where the sparse queue already takes that family, the settle queue is a
second queue of it.
8_8_8_8 and 16_16_16_16 targets with COMP_SWAP 2 or 3 keep the exported
alpha in their first channel and red or blue in their fourth, so a plain
Vulkan blend would run the alpha equation on a color channel. Such draws
threw "blending into a color target with a reversed component order is
not implemented" (Gran Turismo 7, #4782).
The AMD SI and CIK register references describe CB_TARGET_MASK in the
order of components after blending and before COMP_SWAP is applied, and
RADV programs CB_BLENDn_CONTROL without regard to the swap while it
renders R4G4B4A4, B4G4R4A4, R5G5B5A1 and B5G5R5A1 through SWAP_STD_REV
and SWAP_ALT_REV and passes the CTS blend tests on Navi10. The color
block is therefore taken to blend in export order: the alpha equation
and source-alpha factors use the exported alpha and the destination
alpha is the first channel.
TranslateReversedBlend keeps the factor pairs Vulkan applies per channel
in the pipeline. Other source factors are computed by the pixel shader
from its MRT0 export (clamped for UNORM targets, as BLEND_CLAMP says)
and handed to the blend as a dual-source second color, which the
fragment output post-pass adds at location 0 index 1; when both
positions need it, the export is premultiplied by its source factor.
The code is a pipeline specialization, serialized in request version
26, and stays out of the prepared shader key.
sRGB, SNORM and integer targets, constant and second-source factors,
destination alpha in the color equation, destination factors the two
equations do not share, different color and alpha operations, MIN or
MAX with factors other than ONE, other color exports when the shader
computes factors, and devices without dualSrcBlend still throw, now
with the blend control, mapping and format in the message.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Track successful releases cumulatively across the complete depth and HTILE ownership ranges. Preserve color retirement and share its single callback; keep retired image contents alive through sampled views and recorded work. Add registry, partial/failure/protection, remap and mixed teardown regressions. Depends on native depth sampling in PR5496 and its descriptor prerequisite PR4820.
With APS5_PIPELINED_DRAWS=1, queue 0's worker decodes, captures and recompiles a draw and hands it to
a committer thread that records it (Graphics::Draw under the GPU lock), so the two run in parallel
instead of one after the other. Work the committer has not reached yet is tracked by the guest ranges
it will write; a later packet that reads or writes them, or one the pipeline cannot order, drains it.
Ordered as commit items instead of drains:
- labels (WRITE_DATA, RELEASE_MEM) and same-queue waits on them; APS5_PIPELINE_DRAIN_LABELS=1 drains;
- the end of a queue-0 submission and a released REWIND tail; APS5_PIPELINE_DRAIN_SUBMISSIONS=1 drains;
- the title's buffer fill kernels; APS5_PIPELINE_DRAIN_FILLS=1 drains;
- DMA_DATA immediates and copies (APS5_PIPELINE_DMA=1).
A draw's write ranges include the storage images it writes and its exact color and depth target
ranges (APS5_EXACT_DRAW_WRITES=1), not estimates.
Draw-path costs the pipeline exposed:
- ShaderMemory moves, so a commit owns its capture's pages;
- a submission made before the worker's current collect epoch began keeps the epoch, so a worker
behind a queue of submissions walks a surface once per catch-up (APS5_SUBMISSION_EPOCH_EACH=1 bumps
at every submission); a thread remembers the ranges it walked for the whole epoch;
- the pipeline signals only threads that wait; the pending-image and unit-shadow registries take a
spinning slim lock instead of std::mutex (winpthreads contention was kernel events); ScanPending skips
the registry walk when no pending image can overlap;
- fewer heap allocations per recorded draw (reused descriptor sets and scratch vectors, buffer pool
size classes kept).
GTA V Performance mode, prologue bank interior, RTX 5080: 11.7 -> about 20 fps together with the cheaper
captures below it.
Off by default here; the follow-up turns it on.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Gjon Gojcevic <7807870+gjonn@users.noreply.github.com>
A depth surface lives only in its Vulkan image, so a view of its memory
with another layout could only throw, and a matching view kept reading
the image after other writes over its memory. Retire the surface when a
color target or a written storage image it does not serve overlaps its
planes, or when a view it cannot serve finds guest memory over the planes
written since the surface was last bound; lookups then skip it and the
view reads the memory like any texture. Binding the target again brings
the surface back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review on #6060: the Windows probe now settles a 4,096-page import with the same 4-byte read
production uses and requires no marks after it. A failed settle destroys the import instead of
leaking it. The settle's destination, command buffer and fence are created once per device. Where
the graphics family has a single queue (AMD on Windows), the settle queue comes from another family
that copies, through a buffer aliasing the import's memory. The recorder tests' device gets the
settle queue too, and agc_driver_layered_volume_submit_other_family runs that path
(APS5_SETTLE_OTHER_FAMILY=1).
On Windows a host import without a section alias left guest write tracking, and its changes were
found by comparison. Since resident direct memory became private MEM_WRITE_WATCH memory, its
imports take that path, and every storage-texture refresh over it compares its whole snapshot.
ProbeImportWriteProtection gets a Windows body on MEM_WRITE_WATCH scratch memory, and
decideImportWatch keeps direct imports of private read-write memory watched when GetWriteWatch
stays honest after an import's first GPU use. The driver marks every imported page written until
that use completes, so each such import is settled inside ImportWatched's window by a 4-byte read on
a second queue of the graphics family (the imports lock comes after GpuMutex, so the main queue
cannot be used there). APS5_NO_WATCHED_PRIVATE_IMPORTS=1 keeps the comparisons.
For an indirect draw, whose counts only the GPU knows, the driver copies
the whole range a vertex buffer descriptor covers, records * stride from
the descriptor's address. The last record's element ends sooner when the
attribute sits at an offset in its record, and the fetch can read at most
(records - 1) * stride plus the element.
Forspoken (PPSA03984) draws instances from a 48 MiB buffer of 96-byte
records with six attributes at offsets 0 to 0x50 in each record. The copy
for the attribute at 0x50 ran 0x50 bytes past the buffer into the
unmapped page after it, and the draw stopped with "guest memory is not
readable".
VertexBufferExtent now ends at the last record's element, as
VertexBufferReadSize does for the last record of a direct draw.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
materializeRegisterClear clears the resident image to CB_COLOR_CLEAR_WORD
when the target's DCC keys read as the register clear code. Most of the
time those keys come from a key store the driver recorded itself
(NoteKeysFillOnGpu) and the GPU has not run it yet, so FillClear refused
with "keys pending" and the fallback waited for that store (SyncThrough).
The wait buys nothing when the newest recorded writer of the whole key
range is the noted store: ProvedCurrentDccKeys already answered from the
same memo (PendingStoreKeys), so the keys the GPU will hold are known and
are the code this clear stands for. The image clear and the
"uncompressed" key store after it are recorded into the same queue behind
that store and land after it, the order the wait gave.
KnownPendingDccKeys exposes that proof, and the register clear passes it
as FillClear's allowRegisterClear, which skips only the "keys pending"
refusal; the key scan still runs. Any other pending writer (a title
kernel, a later partial or wider write, a shadowed range) keeps the wait.
Tests: the proof answers only for a pending noted store covering the
range, and a clear recorded behind a pending register clear key store
leaves the clear word in memory and uncompressed keys.
TextureDetiler reset its descriptor pools only when the recorder was idle, so a recorder that keeps
work in flight grew a 64-set pool for every 64 detiles until vkAllocateDescriptorSets failed with
VK_ERROR_OUT_OF_DEVICE_MEMORY. PPSA25344 with its title near 9 FPS stopped on that after 12 and 22
minutes. Each pool now keeps the serial of the newest batch that used one of its sets, and a pool
whose batches completed (Recorder::CompletedThrough) is reset and reused before a new one is
created. The texture, image and CMASK pools share that ring.
Decode the four PA_SC_AA_MASK fields at the rasterized sample count. Use the native Vulkan mask when all four pixels agree and apply varying masks in the fragment shader, preserving helpers and combining coverage with alpha-to-coverage and shader mask exports. Add the normalized masks to pipeline, draw and shader keys and the versioned replay request.
Refuse unsupported conservative-raster, centroid and early depth/stencil memory-write cases. Synthetic readbacks cover driver submission, depth/stencil suppression, fragment stores and one-sample and four-sample coverage.
With PA_SC_AA_MASK covering only some pixels of a quad, the uncovered pixels
run on the hardware as helper lanes, so derivatives of the covered pixels stay
defined. OpKill ends those invocations; OpDemoteToHelperInvocation keeps them
as helpers. The device enables VK_EXT_shader_demote_to_helper_invocation when
available and passes the DemoteToHelperInvocation capability and its SPIR-V
extension to the recompiler target and the graphics shader validation. A
demoted pixel also reads as a helper, so its initial EXEC bit is clear as on the
hardware. Without demote, OpKill stays for pixel programs that read no
neighbouring lanes; implicit-LOD sampling, LOD queries and cross-lane reads are
refused. The quad pixel mask execution test now also samples a mipmapped
texture with implicit LOD under the mask.
Reject native sampled views of the depth image attached by the same draw before pipeline creation. Keep unrelated views, other devices and the single-sample path unaffected. Add unrun native-view refusal and scope controls; full dependency-stack build and runtime validation remain pending.
Adapt our preserved native-depth sampling work from c0d9edf and sample-aware footprint work from 6e35a877, retaining creation-time coherence and initial HTILE proof. The descriptor interface in upstream PR 4820 remains a separate prerequisite. This source draft has not been built or run.
AlxcBTW