APS5_SPANNED_WAVE is now read once by the driver, which sets
SpirvTarget::spannedWaves only on a device with 32-lane compute subgroup
control and computeFullSubgroups. The recompiler no longer reads the
environment. The field is part of the cache key and of the request
serialization (version 32), so a cached or replayed module keeps the mode
it was built in.
A spanned module numbers its lanes SubgroupId * 32 +
SubgroupLocalInvocationId, which needs two full 32-lane subgroups. The
artifact records this as fullSubgroups (disk cache and pipeline identity
included), and the driver creates those pipelines with
REQUIRE_FULL_SUBGROUPS. Compute pipeline captures accept the flag with a
required subgroup size.
The two-lane fallback now catches only SpannedWaveUnsupported, thrown for
a shuffle wider than 32 bits; other emit errors surface as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SRT evaluator records every raw read with the user-data word its address is
based on, and marks a word used as a plain value. The result also names the
user-data word behind each shader-data dword.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
8_8_8_8 and 16_16_16_16 targets with COMP_SWAP 2 or 3 keep the exported
alpha in their first channel and red or blue in their fourth, so a plain
Vulkan blend would run the alpha equation on a color channel. Such draws
threw "blending into a color target with a reversed component order is
not implemented" (Gran Turismo 7, #4782).
The AMD SI and CIK register references describe CB_TARGET_MASK in the
order of components after blending and before COMP_SWAP is applied, and
RADV programs CB_BLENDn_CONTROL without regard to the swap while it
renders R4G4B4A4, B4G4R4A4, R5G5B5A1 and B5G5R5A1 through SWAP_STD_REV
and SWAP_ALT_REV and passes the CTS blend tests on Navi10. The color
block is therefore taken to blend in export order: the alpha equation
and source-alpha factors use the exported alpha and the destination
alpha is the first channel.
TranslateReversedBlend keeps the factor pairs Vulkan applies per channel
in the pipeline. Other source factors are computed by the pixel shader
from its MRT0 export (clamped for UNORM targets, as BLEND_CLAMP says)
and handed to the blend as a dual-source second color, which the
fragment output post-pass adds at location 0 index 1; when both
positions need it, the export is premultiplied by its source factor.
The code is a pipeline specialization, serialized in request version
26, and stays out of the prepared shader key.
sRGB, SNORM and integer targets, constant and second-source factors,
destination alpha in the color equation, destination factors the two
equations do not share, different color and alpha operations, MIN or
MAX with factors other than ONE, other color exports when the shader
computes factors, and devices without dualSrcBlend still throw, now
with the blend control, mapping and format in the message.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move bounded vertex folding and empty-image feedback into separate topics. Keep the replay coverage proof, stage contracts, finite-reference tests and mapped cache regressions; version the stencil layout separately as format 34.
Co-Authored-By: Codex <noreply@openai.com>
Preserve instance and triangle order with one dynamic-reference shader pipeline. Require bounded references, complete export coverage, replay-safe mask control and resources; refuse unsupported side effects and feedback.
Co-Authored-By: Codex <noreply@openai.com>
An s_swappc_b64 whose target is not resolved compiles to a faulting call
only at the fallback the table-call linker emits; the linker lists those
calls in the request (GuestContext::linkedCallFallbacks), and every other
computed call is refused at compile time again, as on main. The new
agc_driver_compute_table_calls test runs a linked two-record table on
Vulkan in wave32 and wave64, with SCC 0 and 1 reaching the callee, and a
record past the table reporting the UnlinkedCall fault.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With GE_CNTL.BREAK_WAVE_AT_EOI clear, the instances of a tessellated draw share
hull threadgroups: the relative patch index counts on through the instances,
while the host's PrimitiveID restarts with each one. The local stage now passes
its draw-relative instance (InstanceIndex - BaseInstance) to the hull, which
numbers its patches instance * patches-per-instance + PrimitiveID before taking
(mod PRIM_GRP_SIZE) mod NUM_PATCHES. Instances that start a threadgroup anyway
(BREAK_WAVE_AT_EOI, or patches per instance a multiple of the group) keep the
per-instance PrimitiveID.
This replaces #4201's refusal of a hull or domain that uses its relative index
as data while BREAK_WAVE_AT_EOI is clear, and builds on its decode of the bit
(breakAtInstance, request serialization and draw keys).
- Hull and domain cache keys hold the patches per instance instead of the bit;
an indirect draw recompiles its hull and domain for each record's instance
count.
- The device enables shaderDrawParameters when available. Without it, a hull
using its relative index as data in such a draw is refused, as is a domain
doing so (passing the instance on to the domain is not implemented).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plan runtime-indexed T# and S# columns read through a 64-bit pointer.
Preserve the console offset alignment, signed immediate and key bounds.
Track the descriptor dwords consumed by each table in its invocation.
Reject table reads when a writable buffer or storage image overlaps them.
Cover the overlapping-storage fault and isolated second-table behavior.
Materialize runtime-indexed image and sampler records into descriptor
tables while keeping descriptor selection in the shader. Report invalid
active entries through the existing deferred fault buffer.
Bind fragment tables in the fragment set when pipeline libraries split it,
and reserve a physical descriptor for empty tables. Reject unsupported
sampler, border, cube and anisotropic pairings per entry. Keep one column
per heap access shape, one T# shared by several image operations included
(milescoviello's #5550 case), and narrow it to the union of its keys'
domains when every key has one (SleglJan's key-domain changes). Plan
columns read at (key << n) & mask over their whole domain (tibzejoker's
fix). Read records in runs through SleglJan's bulk reader, and reject
entries whose surface has no mapped byte, as Tamalero proposed. Store
table columns in disk plans; update the runtime ABI and disk format. Bound
table-sized entries in the resource cache (65,536 table elements) and the
dispatch cache (256 MiB of captured words), evicting LRU.
A cached variant kept its SPIR-V in memory for the whole run. On Linux and
macOS the words are now a view into the mapped cache file, and a freshly
compiled variant is written and mapped back, so the kernel can drop them.
docs/dev/CONVENTIONS.md asks for C#-style template parameter names
(TKey, TValue). recordEmissionFailure, added with the split of variant
compilation, named its callable parameter Step; rename it to TStep and
write the template header the way the neighbouring recompiler sources do.
No behaviour change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PrepareRegistered prepares every registered pixel shader twice: with the
plan's layout, then with pushConstantSizeBytes = 0. The layout is not part
of the source key, so both requests share one SourceEntry, but the first
variant consumes the entry's front-end program. On a cold cache the second
layout therefore ran PrepareResourceProgram again (translation, SSA, the
fold and eliminator passes, SrtWalker, ResourceTracker) and the interface
passes after it, although none of them reads the layout. Only decode, CFG
and structurization were shared through the context (#2526).
A ShaderPreparationContext now keeps the program it last emitted from,
after the interface step, together with its source entry. Another layout of
the same source at the same code address resets the binding allocation and
emits from that program, as the registration word-limit fallback already
does within one variant, and copies the compiled information out instead of
moving it. A different source or code address builds a program as before.
Building another program through the context releases the kept one first,
and the program dies with the context, so no program outlives the
registration that built it (the rule of #471). ResourcePrograms() counts
the programs built through a context.
Disk-cache loads and stores, the emission-failure memo and the variant
order are unchanged. APS5_NO_PERF_LAYOUT_PAIR=1 restores the previous path;
it is a neutral switch, so it does not change disk-cache keys.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
compileVariant ran the layout-independent interface passes
(ApplyStaticInterface, identity and dead-code removal, ShaderInfoCollector)
and the layout-dependent emission (binding allocation, SPIR-V emission, the
registration word-limit fallback, validation) as one step on a program it
consumed. Split it into prepareInterface and emitVariant, which take the
program by reference, and keep compileVariant as their composition with the
same emission-failure recording, so a later change can emit another layout
from a program whose interface is already prepared.
No behaviour change: compileVariant still runs both steps once per variant
and still moves the compiled information out of the program.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The front end rebuilds operand flushes, NaN checks and bitcasts for every
use of a register, and nothing merged them when SPIRV-Tools is off. A
dominator-scoped value numbering pass before SPIR-V emission replaces a
pure value with an earlier identical one in the same or a dominating
block. Only operations that are functions of their operands are merged;
loads, stores, atomics, barriers, phis and cross-lane reads are left
alone. A block the entry does not reach throws.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
getSource built ResourceProgram (PrepareResourceProgram and ExtractPlan)
for the first request of every shader a process saw, before
findOrCompileVariant looked in the disk cache. With a warm cache every
launch still decoded, built the CFG, structurized, translated and ran
SSA, the fold passes, SrtWalker and ResourceTracker for every shader;
only SPIR-V emission was skipped.
getSource now builds the plan key and tries LoadPlan first. On a hit it
sets the plan and leaves the program null: findOrCompileVariant already
rebuilds the program when no stored variant fits the layout. The loaded
plan takes the request's code address as its shader hash, as a plan
built in this process would. On a miss it builds the plan as before and
stores it. A plan the front end refuses is not stored. The disk is not
used when the cache is off, under the debug probe, with APS5_DUMP_IR
(which prints the front-end passes), or with APS5_NO_PERF_DISK_PLANS=1,
a cache-neutral switch for comparisons.
FrontEndBuilds() counts PrepareResourceProgram calls. The new
cross-process test prepares, captures and materializes 14 shader
fixtures in one process and repeats it in a second process sharing the
cache directory: the second process must run the front end 0 times, hit
14 plans and 14 variants, and produce byte-identical SPIR-V, bindings,
push constants, snapshots, read traces and plans. A damaged plan entry
is rejected and rebuilt alone with the same output and stored again; a
changed code word and another switch value miss.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since #1463, every f16 operand read decodes sign, exponent and mantissa
with integer operations, 8 to 12 instructions more per read than
UnpackHalf2x16 on NVIDIA. The decode works around an AMD Windows driver
that folds the mask of a byte-selected half into UnpackHalf2x16.
A new target flag, float16DenormPreserve, is set when the device reports
shaderDenormPreserveFloat16 and denormBehaviorIndependence ALL, the same
rule as float64DenormPreserve. With it, EmitF16BitsToF32 uses
UnpackHalf2x16 and declares DenormPreserve for 16-bit floats, under
which SPV_KHR_float_controls keeps unpacked subnormal halves. NVIDIA
returns the default NaN for every NaN half, so the NaN bits are rebuilt
from the half: its sign at bit 31, its payload at bits 22:13. The
DenormPreserve 16 mode is declared once, shared with the native f32 to
f16 conversion. Without the flag (lavapipe; RADV reports 32_BIT_ONLY)
the integer decode stays. Halves of fields narrower than 16 bits, such
as SDWA byte selects, keep the integer decode on every device.
The flag is serialized (request version 17) and part of the cache key.
New test agc_driver_f16_to_f32_unpack converts all 65,536 halves, with
nonzero high bits, through v_cvt_f32_f16, and byte-selected halves
through v_add_f16 SDWA, with each decode the device supports, and
compares the bits with an exact decode. agc_driver_f16_to_f32_unpack_spirv
checks which decode the SPIR-V uses with and without the flag.
A packed ancillary word that still reaches an export, or any other read
the masked-select pass cannot drop, no longer stops the recompile. After
that pass every remaining use gets the composed word: the primitive class
at bits 0-1, SampleId at bits 8-11 and Layer at bits 16-28, with every
other bit zero. Those lanes do export the ancillary VGPR on the console,
so the word has to be right, not just present.
The primitive class (0 point, 1 line, 2 triangle, 3 rectangle) comes from
the draw. RasterPrimitiveOf maps rect lists to 3, line lists on the vertex
path to 1 and everything else the driver accepts to 2, because its
geometry and tessellation paths only output triangles. It reaches the
shader as pipeline specialization constant 8233, the way dual-source
blending does: a request field that is folded into the specialized
module and is part of the result memo, the draw key and the serialized
request (version 17).
Only shaders that read the class pay for it. ShaderInfo records the read
(disk cache format 28), and only those shaders put the class in their
module key, so a shader that reads just the sample or layer field keeps
one module for every primitive class. The draw sets the class only when
the pixel shader loads the ancillary VGPR, so other draws keep the draw
cache entries they have today.
The VRS rates (bits 2-5) and the eye id (bit 7) are 0 because the driver
supports neither VRS nor multiview. SampleId is 0 because the driver
rejects MSAA and per-sample shading, and Layer is 0 while it ignores
vertex layer exports. TechnicalDebt.md records the layout sources and
these assumptions.
Tests:
- pixel_ancillary builds an export to target 0 whose two channels select
between a packed value and a phi carrying the word. It checks the word
for each primitive class, sample and layer, checks that the three
shapes the masked-select pass keeps now compile with the composed word,
and that a program whose select the pass drops reads no class.
- agc_driver_pixel_ancillary_word exports the word from a draw and reads
back 2, 3 and 1 for triangle, rectangle and line. A shader that reads
only the sample field shares one module across the classes, and the
whole-word shader gets one per class. It also checks RasterPrimitiveOf.
- agc_shader_memory covers the version 17 round trip, and
agc_shader_disk_cache the new flag.
Both ancillary tests fail with the old refusal when the composition is
disabled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Commit 48dcaace7 introduced PrimedSharedMutex and converted every host std::shared_mutex that existed at the time, because winpthreads initializes a static rwlock on first use and a concurrent first lock returns EINVAL, which std::shared_mutex ignores (docs/dev/TechnicalDebt.md:10).
Two host mutexes were added afterward and missed the conversion:
- Recompiler.cpp registrationModule() static mutex (added by baf3437465)
- MultisampleTarget.cpp guardedMutex() static mutex (added by d109f3966f)
Apply the same priming. Recompiler.cpp reuses the PrimedSharedMutex already defined in that translation unit; MultisampleTarget.cpp gets the identical local helper to match the existing inline precedent.
Tested: Windows 11 x64, WinLibs GCC 15.2.0 (matches CI). Compiled Recompiler.cpp.obj and all three MultisampleTarget.cpp.obj targets with the project build flags; check_conventions.py passes.
lavapipe and MoltenVK leave the vertex stage out of subgroupSupportedStages,
so the driver's shader validation refuses every vertex program that declares
GroupNonUniform ("subgroup capability 61 is unsupported for shader stage 1").
Most do, since the entry EXEC is a ballot.
The device target now records whether the vertex stage has subgroups. When
it does not, vertex and local programs get a host subgroup width of 1: s3
reports one vertex and one primitive, the lane id is 0, a ballot is the
invocation's own bit, a shuffle returns the invocation's value, an elect is
true and a half-wave reduction read is the key itself. Reads of other lanes
keep the existing narrow-host rules and throw where they did. Devices with
vertex subgroups emit the same SPIR-V as before.
v_pk_add_f16 and v_pk_mul_f16 computed each half in f32 and rounded it
back to f16, then fixed the NaN, FP16_OVFL and clamp results on the
bits. On a device with native f16 that rounds to nearest even and keeps
f16 denormals (the Float16, RoundingModeRTE and DenormPreserve
capabilities that #1252 already checks), the arithmetic is now one
f16vec2 OpFAdd or OpFMul decorated NoContraction, and the same NaN,
FP16_OVFL and clamp rules run on its result bits.
The results don't change. f32 has at least 2 * 11 + 2 significand bits,
so rounding an f16 sum or product to f32 and then to f16 gives the
correctly rounded f16 result. Vulkan requires f16 OpFAdd and OpFMul to
be correctly rounded, the module keeps f16 denormals, and any NaN they
produce is replaced by the NaN the existing rules select.
v_pk_fma_f16 and v_pk_fmac_f16 keep the f32 path: Vulkan gives GLSL Fma
only the precision of OpFMul followed by OpFAdd, so a native f16 Fma
may round twice.
New test agc_driver_packed_float16_native compiles one shader with
packed add, mul, fma and fmac (op_sel, op_sel_hi, neg_lo, neg_hi and
clamp variants) with and without Float16, checks that only add and mul
use f16vec2 math, and compares every result bit for bit on random and
special inputs (zeros, denormals, infinities, NaNs, values near the f16
maximum) in nine float modes.
winpthreads initializes a statically initialized rwlock on its first
lock and fails a concurrent first lock with EINVAL, which
std::shared_mutex ignores, so two threads can hold the lock at once.
The source, binding plan and specialized module caches take their
first locks from several threads, so their mutexes now take and
release the lock in their constructor, as guest rwlocks do.
The GuestWriteWatch mutex the technical debt entry also named is in
the Linux-only watch, so it is not affected.
DecodeShaderStages accepts LS, HS and an NGG passthrough ES, reads the
3-bit VGT_TF_PARAM fields, refuses its other bits but DISTRIBUTION_MODE,
and needs HOS levels that keep [1, 63]; 0x286-0x287 join the draw key.
Local and domain programs are prepared with their stage group: its
headers are merged over one set of defaults, and a local program left
unbound by the hull header is prepared at its first draw.
The local program ends at its jump into the hull half.
TessellationLowering lowers LDS and ring addresses that are affine in
the control point and relative index to tessellation interfaces with
strides from the code, and refuses accesses outside the patch, per-patch
writes from other control points and a modified primitive export. Hull
lanes read their own saved exec bit, also of host-subgroup-masked
copies; other masks are refused, gl_InvocationID loads as a scalar and
factors clamp to 63.
Shaders with f64 instructions now declare DenormPreserve for 64-bit floats
when the device reports shaderDenormPreserveFloat64. Without it Mesa ANV
flushed the subnormal result of v_fma_f64 to zero.
(cherry picked from commit 7df6c709de994cea0019707d45b6e7b96698fad1)
Adrià