Shader disk cache FormatVersion 43: main took 42 with #7966, and this
change also changes the cached plan.
Co-authored-by: mazen <mazenshaar@hotmail.com>
Capture evaluated every flattened scalar slot even when a clean scalar
branch skipped it, so an inaccessible direct user-data root refused the
draw. Record flat-slot uses on resource control-flow blocks and evaluate
only the reachable ones; keep refusing taken and unresolved paths.
Fixes#8272.
Back 1D and 1D-array depth comparison textures with 2D Vulkan images and views of height 1. Drivers such as AMD RADV reject 1D depth images (VK_ERROR_FORMAT_NOT_SUPPORTED). The shader recompiler maps 1D depth comparison samplers to 2D image descriptors, and pins the normalized vertical coordinate to 0.5 to sample the center of row 0 without hitting borders.
A data-only hit (#494) reused a cached stage only when every user data word
of the stage was unchanged. A stage records the user words it depends on when
it is captured: the words its SRT walk reads as values or bases
(SrtUserDataUse), the words its binding layout forwards into shader data
(userDataRegisterReads), and the vertex fetch table words of its vertex stage
info. A cached stage is now a candidate when only words outside that set
changed; its captured memory is validated as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pixel shader that reads barycentrics on a device without
VK_KHR_fragment_shader_barycentric used to need a vertex shader so a
generated geometry stage could follow it, which a mesh pipeline cannot
have. The mesh program is now compiled a second time with the layout
its pixel shader needs: it writes its clip position as a varying and the
three corner positions of each primitive as per-primitive outputs, and
the pixel shader solves for the weights, for both smooth and linear
interpolation.
The layout is part of the mesh cache key, the stored artifact and the
captured request. Without VK_EXT_mesh_shader the draw still throws, now
saying so. Per-vertex parameters on the mesh path still throw.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Since 2688b7646 a read and written V# element is staged in device memory
up to 16 MiB when the direct dispatch's threads times the V# stride cover
at least half of it. A raw buffer has stride 0, so it never passed and
kept the 2 MiB written window: GL storage buffers compiled by Mesa stay
bound in place in the host import however hard a dispatch sweeps them.
The PS5 OpenGL Showcase's particle update (262,144 threads, each loading
and storing its own 32-byte record of an 8 MiB SSBO) took about 53 ms of
GPU per frame on an RTX 4060 bound in place; staged it takes about 0.1 ms
plus 2.2 ms of bulk copies.
The recompiler already records the furthest byte each buffer resource's
accesses reach past their base (maxByteExtent). The binding plan now
carries it per guest buffer element (DescriptorBinding::bufferExtent,
shader disk cache format 37), and a raw element uses it as its record
size in the sweep test. Strided elements are unchanged.
The data-hit verification keeps this branch's pending-span query on its ShaderMemory and takes #5958's CaptureInvocation, which retries a capture with converted samples.
Keeps this branch's tagVariantRuns declaration next to main's dumpRequest suffix parameter in Driver.hpp, and reads #6232's pointer-chain test words from its table's snapshot, where this branch keeps them.
#7527's selected-record limit test reads a record's descriptor words through the
table snapshot's shared words list; this branch keeps a snapshot's words per column,
so the test reads them from its column instead.
10_11_11_UNORM (format 30) images are read and written through an
R32_UINT view, so the hardware sampler cannot filter them. image_sample
of a 2D or 2D array view now fetches the one or four texels at LOD 0,
decodes them and blends them in the shader, with bilinear weights on
the 8-bit grid, for point or bilinear filtering with wrap or
clamp-to-edge addressing.
A shader gets the converted sample mode only after a converted
descriptor reaches one of its sampled images: that capture throws
ConvertedSampleRequired and the driver prepares the program again
with GuestContext::convertedSamples.
The R10_G11_B11 float format is sampled natively since #7271 and keeps
its refusal on the converted path.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The new samplerCoordinatesIgnored field adds 40 bytes to DescriptorBinding
under libstdc++ on Linux x86_64, where the size assert is compiled. The
encoder and decoder already carry the field.
AI-assisted: yes
An S# that forces unnormalized coordinates was refused when it samples a
T# the shader declares 3D, 2D-array, cube or multisampled, since Vulkan
allows unnormalized coordinates only with a single-level 1D or 2D view.
A null T# reads zeros at any coordinate, and a T# whose channels all
select constants is emitted as those constants without a sample, so when
the S# samples no other T# its coordinates cannot change a result. Such
an S# is now marked in its binding (samplerCoordinatesIgnored) and the
driver creates its sampler without FORCE_UNNORMALIZED. The flag is part
of the binding content key and of the shader disk cache, whose format
version is raised to 38. An S# that also samples a real T# while a null
T# of such a shape shares it is still refused. PPSA01317 pairs an
unnormalized clamped bilinear S# with a null 3D T# and a 3D T# whose
channels all select 0 for image_sample_lz in a compute shader during
boot.
AI-assisted: yes
Every IR value kept two lists of its users: uses (the users) and
operandUses (the users with their operand index). AddArgument,
ReplaceArgument and ReplaceAllUsesWith updated both, and each value
allocated and freed both. Only operandUses is kept now. Uses() is a view
of the users in operandUses, and HasUses, UseCount and Invalidate read
it. sizeof(IrValue) drops from 152 to 128 bytes; the plan value encoder
writes no use lists, so only its size check changes.
The two lists held the same users in the same order, except after
ReplaceArgument on a user that took the value in more than one
operand: uses dropped the user's first entry and operandUses the
replaced operand's entry. Uses() now follows operandUses there too.
shader_ir_use_rewrite counted the allocations of both lists: its
allocation failure check now fails the one reserve ReplaceAllUsesWith
makes, and its growth check allows the allocations of one list, which
main exceeds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The flag that adds the vertex stage to the push constant stages was not part
of the encoded result, so a journalled remapped vertex shader replayed
without it. Encode and decode it with the invocation and cover it in the
result round trip and the push stage test.
Main's #6402 stores to image tables: a table with a written entry takes
the fresh path, as it does on main, and so does one without a sampled
table binding, whose entries the fresh path rejects. The shared place
step uses the binding as a pointer. Main's #6614 frees descriptor sets
from shared pools one by one, while this branch keeps only dedicated
pools, so both stay. #7101's bulk key read counts its unmapped-page
skips as unmapped, and its key words sit next to this branch's write
stamp and compared runs.
Main's #7101 reads a key buffer's words in one call where this branch
names the reason a narrowing is skipped. Keep main's bulk read and
count both of its unmapped-page skips as unmapped.
main's 81e3db853c counts each device's active library builds
(activeBuilds), so ClearPipelineLibraries waits for them. Keep it
beside this branch's joinable worker and stopping flag, which replace
started.
APS5_SPANNED_WAVE is now read once by the driver, which sets
SpirvTarget::spannedWaves only on a device with 32-lane compute subgroup
control and computeFullSubgroups. The recompiler no longer reads the
environment. The field is part of the cache key and of the request
serialization (version 32), so a cached or replayed module keeps the mode
it was built in.
A spanned module numbers its lanes SubgroupId * 32 +
SubgroupLocalInvocationId, which needs two full 32-lane subgroups. The
artifact records this as fullSubgroups (disk cache and pipeline identity
included), and the driver creates those pipelines with
REQUIRE_FULL_SUBGROUPS. Compute pipeline captures accept the flag with a
required subgroup size.
The two-lane fallback now catches only SpannedWaveUnsupported, thrown for
a shuffle wider than 32 bits; other emit errors surface as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A graphics pipeline the store builds or a compute pipeline a dispatch builds is
appended to pipelines.journal next to the shader cache entries, as the
materialized result of each stage (the specialized SPIR-V, bindings and push
constants) plus the recipe the pipeline was built from: the stages'
identities, vertex input layout, descriptor set layout keys and the state
fields the Pipeline constructor reads. When a VulkanDevice is created with a
window (a headless device waits for its first draw or dispatch, since the
window's device usually replaces it first) the journal is loaded and its
pipelines are built on a pool of threads while the title loads, so the first
draw or dispatch of a configuration seen on an earlier run finds its pipeline
resident instead of compiling it on the queue worker.
The pipeline stores key on RecompileResult::Identity, a hash of the module
words, specialization constants, subgroup width and bindings that is the same
on every run (SharedSpirv::ContentHash, computed once per shared module),
rather than on the process-lifetime variant ids; a generated stage is keyed
like any other. CachedPipeline builds outside the store's mutex and the
library parts and link are built outside the library store's mutex, so replay
threads and queue workers build in parallel; a concurrent build of the same
key keeps the first inserted. A replayed pipeline enters the store at its
least recently used end and a full store takes none, and its optimized relink
waits for the first draw that binds it, so the title's own pipelines come
first in both. The compute pipeline creation moves into
VulkanDevice::createComputePipeline, shared by dispatches and the replay. The
store key also covers the color component mapping, the depth target's sample
count and a depth bias clamp, which the constructor reads.
The file names the device it was written for (the properties the driver's
pipeline cache file is named by) and starts over for another; a torn tail is
cut at a record boundary; two runs sharing the directory take turns on a lock
file and keep each other's records; a read failure never starts the file
over. A device cancels its replay before it is replaced or destroyed, and a
replaced device destroys its pipelines then rather than at the driver's stop.
APS5_NO_PIPELINE_JOURNAL=1 disables it; APS5_PIPELINE_JOURNAL_THREADS,
APS5_PIPELINE_JOURNAL_ENTRIES (the first recorded), APS5_PIPELINE_STORE_ENTRIES
and APS5_PIPELINE_JOURNAL_MAX_MIB bound it; APS5_PROFILE_DRAW prints a
[pipeline-journal] summary. APS5_VERIFY_SET_LAYOUTS=1 now also checks a
dispatch's set layout against the one described from its shader. The lavapipe
test suite, which shares one cache directory, runs with the journal off except
for the journal's own tests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6hw3kRAuQdfcU9YCZgaoK
Resolves#4943's draw relocation against this branch: tagVariantRuns, which now builds a draw variant's runs and words, also records the captured words it shares with a table's supplied blocks, per region as before. A variant whose runs relocation splits keeps sharing for the draw cache's word compare, and skips the vouched validation, which already requires one run per region.
The shader disk cache's writer, started with the cache at the first
shader lookup, was detached and never stopped. It stayed parked in a
timed wait on its queue until the process exit killed it, and entries
still queued at exit were never written. A process that exited right
after its first shaders could kill the writer inside its first wait,
which takes winpthreads' global condition-variable lock, as with the
driver's own threads.
The writer is now joinable. The driver's shutdown stops it after the
other driver threads: the writer writes the entries still queued and
leaves, and the shutdown joins it. A store after that is written on the
caller, and a cache first used after the shutdown starts no writer.
ShaderDiskCache::Shutdown joins the names the cache key check has
reviewed, since it changes nothing a recompile produces.
Brings the series onto main 4e9a2a871 (#1193, #1201, #1202, #6060,
#6333 and the descriptor word tracking of 682b07eb0).
Resolutions:
- ShaderMemory: the column memo's supplied and spilled blocks now sit
on #1193's page vector, page pool and word masks. Pages are found by
binary search, since a spilled block can add many pages, and a block
placed into a lazily read page fills the rest of the 16-word blocks it
touches, so a later lazy read never overwrites placed words.
- ResourceMaterializer and SharedValues: the selected-record path and
the words index carry availableDwordMask, as the snapshot hash does.
A selected record's dwords past the end of its buffer count as read
zeros, which is what a scalar buffer load returns there, so a key past
the heap still selects a null image without reading the heap.
- DescriptorCache: kept table sets live beside #1201's recycled sets.
- Tests and CMake: both sides' tests are kept.
The shader disk cache keys every plan and variant with the recompiler's APS5_* switches that are
set (ShaderDiskCache's switch key), except the ones listed as neutral. APS5_NO_TABLE_SNAPSHOT_REUSE
and APS5_NO_TABLE_RESOLUTION_REUSE were not listed, so turning the table reuse off for a comparison
run gave every shader a new key: a cache filled with the reuse on missed throughout the boot, and
the other way round, and the comparison measured recompiles instead of the reuse.
Both switches only choose between building a column or a resolution and taking the same contents
by reference, so the output does not depend on them: they are now neutral switches. The cache
format is unchanged and FormatVersion stays as it is; only a process that sets one of them now
finds the entries stored without it.
The plan loading test gains a process with both switches set, which must load every stored plan
and variant without running the front end and produce the same output as the first process.
Before this change it failed with every plan and variant missed (17 plan misses, 17 variant
misses); now it loads all 17 with the same output.
Dean