Main's #7325 adds verifyChainedTableOffset to the image table tests
where this branch adds its selected-record tests. Keep both, with
main's call after this branch's.
SplitImmediateAddend peeled a single immediate operand off a T# dword's
scalar offset. A title that loads a 32-byte record with two
s_load_dwordx4, the second at soffset (first soffset + 16) where the
first is (key << 5) + C, left IAdd32(key << 5, C) as the base of dwords
4-7, which never matched dword 0's base, and the image failed with
"GetImageResource dword 0 is not a valid runtime value".
Peel immediate IAdd32 operands until none is left and sum them modulo
2^32, as the scalar adds do. The chained offsets then reach the existing
folded-addend path. verifyChainedTableOffset is the pointer table offset
carry shader with the second offset chained off the first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's #7101 reads a key buffer's words in one call where this branch
names the reason a narrowing is skipped. Keep main's bulk read and
count both of its unmapped-page skips as unmapped.
An image_sample_*_a sampler whose dword 3 is built only from values
inside the reserved bits 12-29 was refused with "GetSamplerResource
dword 3 is not a valid runtime value" when the value was not an OR at the
top. The reserved bits of a sample-adjust S# are already dropped term by
term from an OR (canonicalizeSampleAdjustDword3); a dword 3 whose
possible bits all lie in that mask, such as a shifted field with nothing
else, is now dropped whole the same way. PPSA01317 builds such a dword 3
from a wave ballot shifted to bit 12 for image_sample_a in a pixel
shader. A plain image_sample with the same S# is still refused.
AI-assisted: yes
An IMAGE_STORE whose T# comes from a runtime-indexed image table used to
poison the whole table as an unsupported operation. Written, non-atomic
table images now get storage modes, a STORAGE_IMAGE table binding (260,
the sampled image table moves to 261 and keeps the variable count), and
storage views of tracked storage textures at the same element indices.
Every bound entry is marked written, so each dispatch writes back all of
the table's storage images. Atomic, packed, BY2/BY4 and dynamic-mip table
stores still fault. Tables with written entries are not reused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CheckTableStores deselects a selected column whose key the dispatch may
write: a key read through a pointer or a V# while the shader stores
through a pointer, or through a V# the shader writes. The column kept the
key domains narrowed from that same key buffer, which the domain rule
clears only when the domain's own V# is written, so a shader storing
through a pointer still narrowed its whole table on a key it may
overwrite. Deselection now clears them as well.
An image table cannot store, update atomically or read PCK and BY2/BY4
through its records (#757): those operations fault and read zeros. #6243
turns a heap T# whose words and record offset the SRT walker evaluates
into a selected one-record table, which kept that limitation although
such a T# is one known descriptor, as #5188's direct images were.
PlanTableColumns now binds such a T# directly, for every operation of the
shader on it, when any of them is a store, an atomic, a packed or a
BY2/BY4 access. A T# keyed per lane, or by a value the walker cannot
evaluate, keeps its table and its unsupported-operation fault. The disk
format moves to 37 so that plans cached by #6243's builds are not reused.
Since #757, a T# loaded from a buffer heap at a runtime offset is an
image table column, and every capture snapshots the whole heap, up to
65,536 records, even when user data or an SRT read fixes the offset.
When the T#'s eight dwords and its offset are runtime values the SRT
walker evaluates, the column is now selected: the offset is kept as a
descriptor source, the shader indexes entry 0, and the snapshot reads
only the record at the evaluated offset, with the heap base moved to
that record.
The column keeps a table's semantics. An invalid T#, an unmapped record
or an offset that carries past 32 bits is poison that faults only when a
lane selects it; an offset the walker cannot read faults the table
instead of throwing; the record's mode is picked per entry, so no
pipeline variant is added; and a dispatch that writes the record faults
it as written by its dispatch. Dwords past the heap read zeros, as the
scalar loads would, and a record past the key cap is read rather than
poisoned.
A column stays a whole table when its offset is chosen per lane, or when
the shader may write the memory the offset is read from at run time: the
buffer it is read from (by V#), or any memory when the read or a write
goes through a pointer or a GPU-only descriptor. Samplers are unchanged:
an S# the walker evaluates is already a direct sampler. The plan stores
the selection, so the disk cache format is 36.
Five agc_shader_memory and agc_shader_bindless checks asserted the old
planning and now assert the selected record:
- "image table SRT key": a key read through the SRT narrowed a 4-key
table to records 0 and 2. It now selects record 2 alone, captures no
other record, and a changed key moves the capture without changing the
module or its disk key.
- "image table key union" and "image table late key": two keys of one
heap shared one column with two value key domains. Each key now
selects its own record, as two one-record tables.
- "image table unkeyed access": a read keyed through the SRT shared a
whole column with a read keyed per lane. The first now selects its
record; the second keeps the whole, unnarrowed table.
- "uniform table image": the T# at a user data offset was one key of a
full table. It is now the one selected record, and the record the user
data does not select is not read.
A value key domain only arises for a key the walker evaluates, so it no
longer narrows an image column; key buffer domains (keys loaded per
lane) still do.
A 2D image_sample_l through a 1D T# now takes the 1D runtime mode, as
image_sample_lz already does, instead of failing the runtime image
interface. Measured on RDNA2 (gfx1036) with tools/hw-oracle against the
same sample with DIM 1D: u inside, on and past the edges, v of 0, 0.5, 1,
-0.25 and 3, LOD -1 to 6 on a 5-level R32_FLOAT texture, point, linear and
linear-mip filtering, wrap, clamp and border addressing, and HEIGHT or
DEPTH 3. All 4725 rows matched: the 1D T# decides the addressing and v is
ignored.
The execution test samples a 5-level 1D T# with distinct mip contents at
LODs -1 to 6, fractional ones included, with u inside, on and past the
edges, under point filtering with wrap and clamp and linear filtering with
linear mips and wrap, clamp and border addressing. Every result must match
the same image_sample_l with DIM 1D bit for bit; point-filtered results
must also be the expected texel of the expected level, and linear-mip
results the blend of the two levels the LOD selects.
A 3D image access bound to a 1D T# matched no runtime image mode and threw
"image descriptor is incompatible with the static runtime image interface"
(SONIC SUPERSTARS, PPSA06888, which reads a 1x1 1D T# as 3D with a plain
image_sample and an image_load). Add a 1D mode for 3D accesses whose only
uses are image_sample_lz, plain image_sample or mip-less image_load.
Measured on RDNA1 (RX 5700 XT, gfx1010) with the hardware oracle:
image_sample_lz reads the texel u selects and ignores v and r with every
addressing mode; a plain image_sample takes its texel and LOD from u alone;
image_load reads the x texel for any y when z is 0 and returns zero for any
other z, as the 3D-on-2D load measured in #2023 does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QQNVXyqeecpyneonkYL6H1
A descriptor whose dwords come from two scalar buffer loads of the same
record, where one load's offset carries an immediate addend, was refused
because the column wanted one offset expression and one buffer node for
every dword. The column now peels an immediate addend off each read's
offset and compares effective offsets (memory offset plus addend) and
the remaining base expressions, and accepts distinct buffer nodes that
are equivalent. Dword 0 keeps its offset expression, so columns that were
already planned are planned the same way.
Folding a second load into the first changes where its offset wraps:
at a dynamic offset of 0xfffffff0, a second load at +16 wraps to the
start of the buffer, while the folded read lands past 4 GiB. A column
folds a load only when its addend is no smaller than dword 0's and its
immediate is small, and it records dword 0's addend and the largest extra
addend. A record whose offset would carry past 32 bits between the two
loads is not read and becomes OffsetCarry poison.
PPSA26344 pixel shader 0x8000296000 reads T# dwords 0-3 with an
s_buffer_load_dwordx16 at key * 0x88 and dwords 4-7 with an
s_buffer_load_dwordx4 at 48 + key * 0x88.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A table column can now read its descriptor through a 64-bit pointer
stored in each record: the shader loads the record's pointer at
key * stride + addend + offset from a scalar buffer, then the T# (or S#)
through that pointer. The materializer reads each record's pointer when
it builds the table, reads the descriptor behind it and records one
read range per record. Pointer columns take their own path ahead of the
batched record reader, since a descriptor's address is known only once
its record's pointer is read. The column is planned only when the descriptor
dwords are read at an immediate offset from a GetAddressResource whose
two dwords are consecutive reads of one buffer at the same affine offset.
PPSA26344 compute shader 0x8000333c00 picks a record per lane in a
waterfall loop, reads the record's pointer from it and samples the
image behind it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plan runtime-indexed T# and S# columns read through a 64-bit pointer.
Preserve the console offset alignment, signed immediate and key bounds.
Track the descriptor dwords consumed by each table in its invocation.
Reject table reads when a writable buffer or storage image overlaps them.
Cover the overlapping-storage fault and isolated second-table behavior.
Materialize runtime-indexed image and sampler records into descriptor
tables while keeping descriptor selection in the shader. Report invalid
active entries through the existing deferred fault buffer.
Bind fragment tables in the fragment set when pipeline libraries split it,
and reserve a physical descriptor for empty tables. Reject unsupported
sampler, border, cube and anisotropic pairings per entry. Keep one column
per heap access shape, one T# shared by several image operations included
(milescoviello's #5550 case), and narrow it to the union of its keys'
domains when every key has one (SleglJan's key-domain changes). Plan
columns read at (key << n) & mask over their whole domain (tibzejoker's
fix). Read records in runs through SleglJan's bulk reader, and reject
entries whose surface has no mapped byte, as Tamalero proposed. Store
table columns in disk plans; update the runtime ABI and disk format. Bound
table-sized entries in the resource cache (65,536 table elements) and the
dispatch cache (256 MiB of captured words), evicting LRU.
agc_driver_waterfall_images fails on main since fd438f663: its compute shader
runs an LOD query with a local size of 8x1, and the module gets
DerivativeGroupQuadsKHR, which SPIR-V validation rejects unless both
dimensions are even (VUID-StandaloneSpirv-DerivativeGroupQuadsKHR-10151).
This reverts commit fd438f663, reversing changes made to 51031742e.
Co-Authored-By: Claude <noreply@anthropic.com>
A lane whose IDXEN load was skipped keeps the T# VGPRs it had before, and a
waterfall that reaches it uses that value as its T#. The fold refused only a
value that was itself a uniform descriptor or a table read, so a default T#
copied into the VGPRs under EXEC, or merged by a phi, was accepted and those
lanes read zeros. The skipped lanes' values are now followed through the masked
selects and phis that all eight dwords share, and the fold is refused when an
arm is a descriptor or a table read.
Every lane of a per-lane table runs the access with its own record, and the
runtime image mode picks the SPIR-V path per lane, so lanes of one quad whose
records need different modes take different paths around an implicit-derivative
sample or LOD query, where derivatives are undefined. Such a table now throws
at dispatch; accesses of one table share the check, so they keep one table.
The test also covers v_readfirstlane and v_cmpx_eq_u64 waterfalls and refuses
loops that compare seven T# dwords, run the image instruction after EXEC is
restored, or store.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A waterfall loop (s_ff1 or v_readfirstlane picks a lane, v_readlane copies
its T# to SGPRs, v_cmpx_eq keeps the lanes whose T# VGPRs match, the image
instruction runs, the matched lanes leave the loop) reads the T# from VGPRs,
which no descriptor source evaluates, so planning failed with "GetImageResource
dword 0 is not a valid runtime value".
When every T# dword of an image read, sample, gather or query comes from
v_readlane_b32 or v_readfirstlane_b32 and the EXEC its results are written
under implies that each of them equals the lane's own VGPR, the access takes
each lane's own T#. A T# that each lane loaded with buffer_load_dword* (IDXEN)
from one evaluable V# becomes a bindless image table keyed by the lane's
index, addressed by the V# stride, and its slot is decorated NonUniform.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TryMakeTableImage planned a T# read from a bindless image table only when
its eight scalar reads had no user other than that one GetImageResource.
Descriptor-phi splitting gives each image operation its own copy of the
T# over the same reads, so a table entry that is both queried and sampled
was refused, and GetHandle then rejected the scalar reads.
Accept reads whose every user is a GetImageResource built from exactly
those eight reads; each copy is planned as the same table.
Crimson Desert (PPSA25286) stops on such a shader at start-up:
image_get_resinfo and image_sample_lz on one s_buffer_load_dwordx8 T#.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GetHandle refused every T# whose dwords are scalar buffer loads (ReadConstBuffer) before ValidateSource looked at them. Tables with a 32-byte stride and a wave-uniform key are planned as bindless tables before GetHandle runs, so the refusal only hit the other cases: a T# read at an offset the runtime walk can evaluate, like Resident Evil Village's mesh programs (48-byte records with the T# at +16, the record index read through two more constant buffers keyed by user data). ValidateSource already decides whether such a chain is a uniform runtime value, so leave the decision to it; the walk then reads the T# at each capture.
A program with an atomic on a V# it specializes was compiled as
address-based (usesDma) so that the atomic's unaligned-address check could
call the fault function. The atomic itself goes through the element's
storage buffer and nothing in such a program reads the address tables. In
the driver an address-based build leases all registered guest memory and
binds every V# of the program in place inside a host import, so none of
its elements is staged in device memory, the atomic one included.
The atomic now asks for the fault buffer alone (usesFaultBuffer) and
records the fault through RecordBdaFaultWords, which has both forms. The
fault record, the stop of the invocation and the atomic are unchanged. The
64-bit fault address needs shaderInt64, as the scalar buffer load does. A
tessellation-control program with such an atomic is refused as before.
Test: agc_shader_memory compiles a buffer_atomic_inc on a specialized V#
and requires no address table binding, one fault buffer, one call of the
fault function, logical addressing and the atomic element flag; without
shaderInt64 the program is refused.
Several image instructions of different dimensions on the same T# address share
one descriptor source, so a source stays active as soon as one of them is taken
and the others are still checked against the T#. Record the images a block uses
as well, deactivate the images used only behind a branch the capture does not
take, and bind them as null descriptors.
PPSA15246 (Teardown) then passes the "incompatible with the static runtime
image interface" check of the three image_store variants that are not taken.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A compute kernel that picks a buffer or an image path from an SRT word reads
the same address as a V# in one path and as a T# in another. The resource
tracker now records which block uses each descriptor source and builds the
resource control flow from the structured block terminators, so a source whose
only uses sit behind a conditional branch with a clean, evaluable condition is
left inactive (bound as zero words, not validated) when the capture takes the
other path. A condition that reads inaccessible memory is not evaluable instead
of throwing.
PPSA15246 (Teardown) stopped on "runtime image descriptor is invalid" for the
untaken path of such a kernel.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A T# picked per lane with SelectU32 and made uniform with
v_readfirstlane, as in a waterfall loop, left a descriptor web without
Phis that the web walk could not split. CollectDescriptorPhis now steps
through ReadFirstLane, and a Phi-free tuple containing SelectU32 or
ReadFirstLane is split on the shared condition (SelectU32) or wrapped in
a matching ReadFirstLane. The new nodes are placed after the last
original node of the tuple in its block and inserted only when the web
is split; otherwise they are invalidated like the web's Phis.
PPSA26344 compute shader 0x800037f200 selects its T# per lane between
SRT reads or zero and reads the first lane of the result.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A plain image_sample that says DIM 2D on a 1D T# takes its texel and its LOD
from the first coordinate alone, measured on RDNA2 (gfx1036) with the
hardware oracle. Astro's Playroom draws with such a pair.
Since #2133, a 3D image_sample_lz or image_load bound to a 2D-array T#
decodes as 2D and matches the new 2D mode, so it reads layer 0 for any
r and returns zero for z != 0. adriadam10 measured on RDNA2 (gfx1035)
that dim 3D on a 2D-array T# behaves exactly like dim 2D-array, with r
or z as the layer. Decode a 2D-array T# bound to a 3D access as 2D-array
and add a 2D-array mode next to the 2D one.
The same measurement covers image_load_mip on both T# types and
image_get_resinfo, which returns (w, h, 1, levels) on a 2D T# and
(w, h, layers, levels) on a 2D-array T#. Allow those accesses and return
1 as the third resinfo component through the 2D mode. image_get_lod is
not measured and keeps throwing.
MemoryIndexBelongsTo walked every instruction of the program and asked
BufferAccessOf, AddressOpcodeInfoOf and ImageOpcodeInfoOf about each one.
TryMakeTableImage calls it for each of a table handle's eight descriptor
dwords and once more for a material key, and PlanIndirectImages calls
TryMakeTableImage for every image instruction whose handle is not planned
yet, so each such instruction could cost up to nine walks of the whole
program. PlanIndirectImages now indexes the instructions that access
memory by their memory index once, before it plans, and
MemoryIndexBelongsTo looks the index up while that pass runs. The
planning pass adds, removes and changes no instruction, so the index
stays valid until the pass drops it at its end. Outside the pass, where
the function has no caller, it walks the program as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fnrk5Dazjb2tLooVSpdRQq
A sample whose T# is chosen in one if/else and whose S# is chosen in a later one has descriptor Phis in two blocks. The web walk required every Phi of the tuple to sit in one block, so it gave up and the plan threw "GetImageResource dword 0 is not a valid runtime value". The walk now splits at the Phis of the latest block in program order, and Phis of earlier blocks pass through each edge unchanged until the walk reaches their block.
A compute shader in Astro Bot samples an image as 2D while the bound
T# is 1D, and the recompiler refused it. Offer a 1D runtime mode for
sampled images used only by image_sample_lz and mip-less image_load,
which reads the texel the first coordinate selects, as measured on
RDNA2 in #2237.
A 3D image access bound to a 2D T# matched no runtime image mode and
threw "image descriptor is incompatible with the static runtime image
interface" (Sonic Origins, #2023). CouchRiv measured RDNA2 (gfx1036)
with the hardware oracle: image_sample_lz with dim 3D ignores r and
reads the (u,v) texel, and image_load reads the (x,y) texel when z is
0 and returns zero otherwise. Add a 2D mode for 3D accesses whose only
uses are those two forms, and make the load return zero for z != 0.
Every other access keeps throwing.
Fixes#2023
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When every dst_sel of a sampled view selects constant 0 or 1, a sample
or gather returns that constant whatever the texel, filter, LOD or
coordinates. Such placeholder views can have a dimension the static
runtime image interface has no mode for (a 3D view under a 2D sample,
a 1D view under a 2D gather), and a 3D one paired with an unnormalized
sampler fails the unnormalized proof.
A direct sampled image whose accesses are all non-compare samples or
gathers gets three constant runtime modes (float, uint, sint). The mode
is selected when all four dst_sel are 0 or 1, unless the view is FMASK,
depth bits, converted or sRGB-decoded. In it the emitter returns each
channel's constant (the dmask channel's, in all four texels, for a
gather) from the image's swizzle specialization constants, and the
image is left unbound. ProveUnnormalized skips the mode after the
run-time-selected check, so table roots still throw.
A path whose T# or S# dwords hold a program-counter, EXEC or compare-mask
value is no longer folded onto the first descriptor: the web split gives up
and the read throws as on main. Cache the invariant-Phi resolution per web,
lower the state limit to 512 and test the program-counter, EXEC and
compare-mask rows, unwritten registers, and the arm and state limits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Yunus Shaikh