A GPU hang or a crash in a title leaves no record of what the queues
were last given: APS5_TRACE_GPU prints every submission (too much to
leave on), and the WAIT_REG_MEM timeout message shows only the packet
before the wait on the waiting queue.
Each queue now keeps a ring of its last 16 submissions
(APS5_PM4_RING=<n>, 0 to 4096, 0 disables): the TraceMs time in tenths
of a millisecond, serial, size, guest address and a copy of the first
512 dwords after indirect buffers are inlined, recorded under the
driver lock next to the serial, REWIND tails included. The ring of a
queue and the failure report registration are set up at the top of
Submit, before any output is reserved, so recording under the lock
does not allocate. Disabled, Submit only tests a static flag; enabled,
it costs one bounded copy per submission.
DumpSubmissionRings decodes the kept dwords with Pm4::PacketWords and
the new Pm4::KnownName, which returns the decoder's names as views of
static strings (empty for an unknown opcode, printed as UNKNOWN_0x%02x);
Pm4::Name now builds its std::string from it. Every line is formatted
with integer and string conversions only into a 512-byte stack buffer
and written with one WriteFile / write(2) on the stderr handle, so the
dump does not allocate from the C runtime heap, lines from other
threads do not interleave and the CRT stream lock is not taken (the
rule of #3871). The header gives the number of queues and the ring
size, and each queue line the submissions recorded and kept.
libc gets a small registry of failure reports
(LibcRegisterFailureReport_nid_no_patch, Unregister, Run); Run runs
the registered reports the first time it is called and returns at once
after that. The Windows crash report runs it after the thread dump of
an unhandled exception, after SIGABRT (which also covers the
APS5_HANG_DUMP_SECS timed dump, since it aborts) and after an invalid
CRT parameter, so an abort after a reported exception does not print
the rings twice. The driver registers its dump at the first submission
and unregisters at static destruction; the dump there waits a bounded
time for each ring lock. What the crash-path dump cannot do (a ring
held by the crashed thread, rings corrupted by the crash, Linux) is
recorded in technical debt.
APS5_STALL_WATCHDOG_SECS=<n> (0 to 86400, 0 disables) makes a waiter
that has not progressed for n seconds dump the rings once per wait: a
WAIT_REG_MEM (progress as for APS5_GPU_WAIT_TIMEOUT_MS: another packet
runs or completes), a REWIND waiting for the title, and a compute
submission held behind queue 0. The last one prints under the driver
lock, so submissions and the other queues wait for it (USAGE.md says
so). Both variables are read at the top of Submit, so an invalid value
of either throws at the first submission and never later inside a GPU
worker. The order wait behind queue 0 releases its order hold on scope
exit, so a throwing dump cannot leave it counted.
No packet semantics change; packet sizes and names are those of the
existing decoder (PM4 type-3 header layout as in Mesa
src/amd/common/sid.h). The queue bound of Submit and of the rings is
one constant, SubmissionQueueEnd.
Tests: agc_driver_pm4_tests gets ten modes, each its own ctest:
agc_driver_pm4_ring (APS5_PM4_RING=4, six submissions on queue 0 and
two on 0x20: the dump holds #3..#6 and #1..#2 in order, nothing
older), agc_driver_pm4_ring_empty (one line), _ring_disabled (one
line), _ring_invalid and agc_driver_pm4_stall_watchdog_invalid
(APS5_STALL_WATCHDOG_SECS=1x: the submission throws and is not
executed), _ring_report (LibcRunFailureReports_nid_no_patch dumps the
rings), _ring_report_heap (Windows: with ucrtbase.dll's HeapAlloc and
HeapReAlloc imports counting calls from the reporting thread, the
failure report of a submission holding INDEX_BUFFER_SIZE names it and
allocates nothing, while Pm4::Name of the same header is seen
allocating), and with APS5_STALL_WATCHDOG_SECS=1 a WAIT_REG_MEM
(agc_driver_pm4_stall_watchdog), a REWIND (_stall_watchdog_rewind) and
a compute submission held behind a queue 0 rendering wait
(_stall_watchdog_order), each held for about 2.5 s, dump once and
complete when released. The PM4 catalog test checks KnownName against
every opcode and custom name. windows_backtrace_failure_report checks
that abort() runs the registered reports once and not an unregistered
one; windows_backtrace_failure_report_once that abort() does not run
them again after an earlier run. With the dump decoding through
Pm4::Name, agc_driver_pm4_ring_report_heap fails with "the failure
report allocated from the C runtime heap"; with Run resetting its
flag, windows_backtrace_failure_report_once prints the report twice.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dispatch now leaves its initiator and, for DISPATCH_DIRECT, its group
counts in the shader registers, where the CP puts them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ds_ordered_count adds the first active lane's value to a GDS counter and
returns the previous value, like ds_append adds the lane count. Waves are
not serialized in launch order: each still gets its own range. Ordered
swap and multi-dword counts stay refused. DISPATCH_DIRECT and
DISPATCH_INDIRECT accept ORDERED_APPEND_ENBL (bit 3) of the initiator.
Compute shaders of MXGP 26 (PPSA30839) use both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The driver had no handler for the PFP INDEX_ATTRIBUTES_INDIRECT packet,
so a submission with it threw "opcode is not known in the reference".
The packet now reads the 16-byte record at attribute_base +
attribute_index * 16 when it executes and loads INDEX_BASE,
INDEX_BUFFER_SIZE and INDEX_TYPE from it as the direct packets do.
The packet encoding follows PAL's BuildIndexAttributesIndirect and the
record layout PAL's IndexAttribIndirect (gfx9Chip.h, c5e80007).
indirect_mode 1, reserved bits and record index types above 3 throw.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Track mutations against the current draw dependency mask and preserve device and shader registry identity in the draw key. Validate mutations, projected masks, PM4 packets and prepared graphics against full recomputation.
Written with AI assistance.
Killzone: Liberation (PPSA11142) writes its frame fence with
`sceAgcDcbWriteData(dst 6, cache policy 3, write confirm)`, which the
SPRX-confirmed encoding turns into WRITE_DATA with DST_SEL 3 and a label
address in the title's memory that RELEASE_MEM and WAIT_REG_MEM then
use. The validator rejected DST_SEL 3 as the GDS; it now writes guest
memory like DST_SEL 5, resolves as a label, and the title runs past its
first frame. TechnicalDebt records where the semantics come from.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 9c53b7f6e8ce2ca81d8f0fa551d12632acfeb60d)
QueueState held the 48 KiB constant RAM inline, so every QueueState
cleared 48 KiB when constructed and copied it when copied.
ResolveShaderAbi and ResolveGraphicsStagesAbi construct one on each call
(sceAgcLinkShaders, sceAgcCreatePrimState, interpolant mapping),
resolveGroupAhead copies the live state, and a queue reset builds a new
one, although few command buffers write the constant RAM.
The constant RAM is now a vector allocated when WRITE_CONST_RAM first
writes it. ConstantRam returns a shared all-zero view until then, so
DUMP_CONST_RAM of unwritten words still stores zeros, and the bounds use
QueueState::ConstantRamWords. The store view ResolveStore returns is
copied by the recorder within the packet, as before.
testMemory dumps unwritten constant RAM and checks for zeros, and checks
that a copied queue state keeps the words written before the copy.
A QueueState is 336 bytes instead of 49464; constructing one takes 0.3
to 0.8 us instead of 1.5 to 2.1 us, and copying one 0.4 to 0.6 us
instead of 1.9 to 3.8 us (GCC 13 -O3). On the title screen of PPSA21402
(Balatro) the clears from the two resolvers took 30 to 129 of 3459 to
7838 CPU samples per 10 s.
Iterating a Registers advances with nextPresent, which recomputes the
word, mask and next bit from the index on every step, about 3.5 ns per
register. BuildRegisteredAbiKey walks all of a state's registers, about
300 with the context, and ResolveGraphicsStagesAbi walks each stage's
registered state to merge it, on every sceAgcLinkShaders,
sceAgcCreatePrimState and interpolant mapping call.
Registers::ForEach visits the set registers in ascending order straight
from the presence words, and both paths use it. The key and the merged
state are unchanged.
testRegisterFile checks that ForEach visits what iteration visits, in
order, and skips an erased register.
Building the key for 297 registers takes 200 ns instead of 1030 ns (GCC
13 -O3). On the title screen of PPSA21402 (Balatro) the two functions
took 0.18 to 0.29 CPU samples per sample of the title's own code, and
0.06 to 0.12 with this change.
The buffer fill and copy fast paths returned before the partial last group
was applied, so a PARTIAL_TG_EN dispatch of the fill kernel filled whole
groups, and a zero last group did not throw. The partial sizes are now
decoded before the fast paths, which are skipped when the last group is
partial; the dispatch then runs the shader with the partial thread counts.
PartialGroupThreads validates all three axes before an empty dispatch
returns, and throws when (groups - 1) * full + last does not fit in 32 bits
instead of wrapping. Validate rejects PARTIAL_TG_EN combined with
USE_THREAD_DIMENSIONS (initiator 0x63).
agc_driver_pattern_buffer_fill now runs the fill kernel with a 16-of-64 last
group through the driver and checks that 80 dwords are written.
AI-assisted (Claude Code).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RADV sets the draw and dispatch indirect base with SET_BASE to the
argument buffer address plus the command's offset, which Vulkan requires
only to be a multiple of 4; vkQuake's base ends in 0x14. The validator
required 8-byte alignment. The records are read as 32-bit words, so a
4-byte-aligned base is now accepted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RADV emits 0 as the start-instance location of DRAW_INDIRECT and
DRAW_INDEX_INDIRECT when the vertex shader does not read the base instance,
and as the draw-index location of the multi forms when draw IDs are off.
The validator accepted only 0x280 as "no register". Location 0 is now
accepted and decoded as 0x280; vkQuake's draws use it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RADV clears ORDER_MODE in the dispatch initiator for a dispatch that
needs its waves launched in order, and the validator required it set.
In-order launch is the stricter guarantee and the driver does not reorder
waves either way, so a direct or indirect dispatch now needs only
COMPUTE_SHADER_EN; vkQuake's map load issues one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RADV writes every COPY_DATA memory destination with DST_SEL 5
(COPY_DATA_DST_MEM in Mesa's sid.h), the current form of the deprecated
GRBM-synchronized DST_SEL 1 that was already accepted. vkQuake's timestamp
writes use it. It is now validated, resolved and executed as memory like 1
and 2; GDS and performance-counter destinations are still rejected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Where the GPU has no LOAD_CONTEXT_REG_INDEX, RADV loads an image's clear
values into DB_STENCIL_CLEAR/DB_DEPTH_CLEAR and CB_COLORn_CLEAR_WORD0/1
with COPY_DATA from memory to a register (DST_SEL 0), one or two dwords.
The validator rejected every register destination.
A COPY_DATA from memory to one or two context registers is now accepted.
It is not a memory store, so the packet drains like other COPY_DATA the
driver cannot record on the GPU, and executing it reads the source then and
writes the registers. Register, GDS, immediate and clock sources, and
destinations outside the context registers, are still rejected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RADV sets PARTIAL_TG_EN in the dispatch initiator for an unaligned direct
dispatch, with each axis's last-group size in the upper half of
COMPUTE_NUM_THREAD_X/Y/Z, and TUNNEL_ENABLE on every GFX10 dispatch. The
validator rejected PARTIAL_TG_EN, and DecodeComputeStageInfo read the
whole register, so a partial size would have widened the workgroup.
A direct dispatch with PARTIAL_TG_EN now runs (groups - 1) * full + last
threads on each axis through the dispatch-size limit USE_THREAD_DIMENSIONS
already uses, or plain groups when every last group is complete; a zero
or oversized last group throws. The workgroup size is NUM_THREAD_FULL.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GuestMemory::WriteChanged stores only the bytes the GPU changed, so that CPU writes to the bytes
around them survive. It found them a byte at a time inside every changed 256-byte block and
stored each run with memcpy. Where changed and unchanged bytes interleave (the low bytes of
floats), every run is a few bytes long, and that loop took 43% of the GPU submission thread in
Disney Epic Mickey: Rebrushed's first level (0.9-1.0 ms per MiB).
Each 16-byte chunk is now compared with SSE2. A chunk with no change is skipped, a chunk that
changed entirely is stored whole, and a mixed one is stored with MASKMOVDQU, which writes only the
bytes selected by the mask. An SFENCE after the loop orders those non-temporal stores before the
stores that follow. Bytes the GPU did not change are still never written.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0134RWCyiyydJBbfKz8g9Bwe
- DISPATCH_DIRECT and DISPATCH_INDIRECT with the SHADER_TYPE header bit,
which RADV sets on every dispatch, as other packets already may.
- FORCE_START_AT_000 in the dispatch initiator. The driver never offsets
dispatches by COMPUTE_START_X/Y/Z, so it already starts at zero.
- LOAD_CONTEXT_REG_INDEX, LOAD_SH_REG_INDEX and LOAD_UCONFIG_REG_INDEX in
their consecutive-register form (a register offset and a count, DATA_FORMAT
0), which RADV uses to load DB_STENCIL_CLEAR and DB_DEPTH_CLEAR from an
image's clear values. The offset-and-data form keeps its old checks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RADV builds its own command streams. Its preamble and draws use packet
forms the validator refused:
- CONTEXT_CONTROL that only updates the load and shadow enables to zero,
which RADV emits when it does not shadow registers. It is a no-op.
- SET_UCONFIG_REG_INDEX of VGT_PRIMITIVE_TYPE with index 1, next to the
index-2 VGT_INDEX_TYPE write already accepted.
- WRITE_DATA to memory (DST_SEL 5) from the graphics queue.
- DMA_DATA with DST_SEL NOWHERE, RADV's L2 prefetch of shader code. It
writes nothing and resolves to an empty store.
- EVENT_WRITE PIPELINESTAT_START, PIPELINESTAT_STOP, SQ_NON_EVENT and
VGT_FLUSH. The statistics counters are not modelled, and the events that
would read them stay rejected.
- ACQUIRE_MEM over the full 48-bit range (COHER_SIZE_HI and COHER_BASE_HI).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0xffff1000 is a type-3 NOP whose count field is 0x3fff, which the command processor reads as a single dword. The driver read it as a 16385-dword packet and rejected every sceAgcDcbResetQueue envelope. Pm4::FillerPacket now also matches exactly this header, so every packet walker steps over it as one dword.
Refs #1581
Debug markers are NOP payloads the command processor skips, so a pop
with no marker pushed (a title that pops a marker it pushed in an
earlier command buffer) has no effect on the GPU; it no longer throws.
Uncharted: Legacy of Thieves Collection (PPSA05684) pops one marker more than it pushes and stopped with "marker stack underflow".
DMA_DATA with DST_SEL 2 (V_411_NOWHERE in Mesa's src/amd/common/sid.h)
reads its source into L2 and stores nothing; libSceAgc emits it as an
L2 prefetch with the source address repeated as the destination. It
threw as a register or prefetch destination. A prefetch of a memory
source is now accepted and writes nothing: ResolveStore returns it as an
empty store, so the packet stays ordered without a drain,
DecodeMemoryCopy keeps rejecting it and Execute skips it. A prefetch of
a register, GDS or immediate source and a register destination still
throw.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A suspend point flushes the labels queued since the last submission. It
recorded them only when a Vulkan device existed, and the device is created
lazily, so labels queued by a title that had not yet drawn were never
written. RADV ends each submission with a RELEASE_MEM to a marker it polls
from the CPU; when the end of that submission was folded into a following
suspend point, the marker stayed at 0 and RADV reported the device lost.
The labels are now recorded either way, which writes them from the CPU
without a device.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
COPY_DATA rejected source select 9 (GPU clock count) and any cache
policy bits. PPSA16064 (Minecraft Dungeons II, Unreal Engine)
calibrates its GPU timer at start-up with a submission of five
COPY_DATA packets, control 0x06016209: GPU clock source, memory
destination, 64-bit count, source and destination cache policy 3. The
submission was rejected and the title aborted.
The clock source now writes the same counter as a RELEASE_MEM
timestamp (the host steady clock in 10 ns ticks), read when the packet
runs, through the direct execute path and through ResolveStore for
stores recorded on the GPU. A 32-bit count writes its low half. The
source and destination cache policy fields are accepted and ignored,
as for WRITE_DATA and DMA_DATA. The reference clock (source select 10)
and the other unimplemented sources still throw.
agc_driver_pm4 runs the title's packet twice and checks a nonzero,
non-decreasing counter and an 8-byte resolved store, and that source
select 10 is still rejected.
The driver copies a submission's packets when it is submitted, but
SET_SH_REG_INDIRECT, SET_UCONFIG_REG_INDIRECT and SET_CONTEXT_REG_INDIRECT
read their (offset, value) lists from guest memory only when the queue
worker reaches the packet. A title that reuses its per-frame memory once
it expects the GPU to be done with it can rewrite a list in between, and
the worker then throws on the data it finds there ("extended register
semantics are not implemented (offset dword 0x3a888889)").
The lists are now read with the packets, after validation, in Submit and
for a rewind tail, and the worker writes the registers from that copy.
Pm4::Execute keeps reading the list itself for direct callers.
PPSA21402 (Balatro) keeps three frame slots and does not wait for the GPU
before reusing one: its slot labels start at 0xffffffff and the GPU only
ever writes 0xffffffff to them, so the poll before a slot is reused never
blocks. When the worker falls more than two frames behind, the title
writes per-draw constants over a SET_CONTEXT_REG_INDIRECT list the worker
has not read yet.
The console's PFP reads the list when it reaches the packet; the
difference is recorded in TechnicalDebt.md.
A stage's user data registers are read for the count in its RSRC2
(COMPUTE_PGM_RSRC2 for compute). A register the title never wrote no
longer refuses the draw or dispatch: it reads 0, as Kyty and SharpEmu
read it and as AMD PAL and Mesa initialize the user data registers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
COND_WRITE (opcode 0x45) with a memory poll and a memory destination is
validated and executed: the driver drains the queue like WRITE_DATA,
reads the poll dword through the range synchronization, compares it
masked with the reference like a 32-bit WAIT_REG_MEM, and writes the
data dword on a match. Register polls and register or scratch
destinations are rejected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PPSA14632 (Unreal Engine) writes sceAgcCbBranch(mode 1, compare
function 0, mask 0, reference 0) with a then buffer and a null else
buffer of 0 dwords. It leaves the compare address uninitialised, so it
is sometimes only 4-byte aligned. The writer rejected the null else
buffer and the misaligned compare address, and the driver accepted
opcode 0x3f only as the 4-dword INDIRECT_BUFFER, so the 14-dword branch
would have failed at submission.
sceAgcCbBranch no longer checks the compare address of an always-pass
branch, which reads nothing, or the address of a buffer of 0 dwords.
The driver runs the first buffer of an always-pass COND_INDIRECT_BUFFER
in modes 1 and 2 (if-then and if-then-else) in place of the packet. A
comparison, a predicated branch and other modes throw.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SET_PREDICATION threw, and packets with the predicate header bit were
rejected. The executor now keeps the queue's predication: operations 3
and 4 read a 64- or 32-bit value and bit 8 selects whether predicated
packets run when it is non-zero or zero; operation 0 clears it. Before
a predicated packet runs while predication is active, queued labels are
recorded and the value is read, as for COND_EXEC; the packet is skipped
when the predicate fails.
Indirect buffers are inlined when a submission is copied, so a
predicated INDIRECT_BUFFER call keeps its header in front of the
inlined commands and records their end like a COND_EXEC range; the
block runs or is skipped on one evaluation, as on the command
processor. Query predication, a predicated COND_EXEC, predicated chains
and flips inside predicated command buffers throw.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WRITE_DATA's control dword carries a two-bit cache policy in bits
26:25 (LRU, stream, no-allocate, bypass). Validate refused it, so a
submission with a WRITE_DATA built with a non-zero policy was rejected
as a whole.
The policy only steers how the GPU L2 keeps the written lines; the
stored words are the same, and Execute already writes them to guest
memory in queue order. The mask now accepts bits 26:25, as was done
for DMA_DATA's cache policy hints. The CE engine and the remaining
reserved bits still throw, and the message no longer names the cache.
agc_driver_pm4 stores both dwords of a WRITE_DATA with control
0x04100200 and checks that reserved bit 27 is still refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Validate refused every COND_EXEC, so a submission that uses one was
rejected as a whole. The packet now reads the dword at its address once
the earlier packets of its queue and their GPU writes have completed,
and skips the exec_count dwords after it when the dword is zero.
The command copy that inlines INDIRECT_BUFFERs records where each range
ends, so a range over an INDIRECT_BUFFER skips the whole buffer.
Unaligned addresses, address bits 48 and up, and reserved control and
count bits throw; the cache policy is accepted on compute queues only,
as the MEC defines it. A flip inside a range, a range that ends inside
a packet or past its command buffer, and a chained INDIRECT_BUFFER or
REWIND inside a range throw.
agc_driver_pm4 and agc_driver_flip cover the skips and the rejections.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WRITE_DATA's ENGINE_SEL (control bits 31:30) picks the CP engine that
performs the write: 0 the micro engine, 1 the prefetch parser, 2 the
constant engine. Validate refused every engine bit, so a submission
with a PFP WRITE_DATA failed as a whole. The prefetch parser handles
packets in queue order ahead of the micro engine, and the driver already
performs the write in queue order, so PFP is accepted like ME. The
constant engine stays refused.
Tests: agc_driver_pm4_tests runs a PFP WRITE_DATA and checks that a CE
one is still refused.
A DMA_DATA too large to resolve as a store drained the device and then
copied on the submitting worker's thread. For a large range that copy takes
milliseconds, and meanwhile another queue's worker records, submits and runs
work that reads the destination in place. PPSA21564 copies each 12 MiB
frame of its intro video into one of two textures with a single DMA_DATA on
one queue and samples it on queue 0 without waiting for that queue; the
GPU's own DMA finishes in microseconds, so the title never needed to. Here
queue 0's upload of the texture ran while the CPU copy was still writing
it, and a frame showed the new frame's luma over the chroma of the frame
two before.
Such a copy (both addresses memory and incrementing, ranges disjoint and
non-empty) now goes through VulkanDevice::CopyBuffer like the copy kernel:
a transfer between the host imports, recorded in order, so everything
recorded after the packet reads the whole copy. The drain and CPU copy
remain the fallback when either range is not host-imported.
Every SET_*_REG packet stored each register into a std::map, and every
draw read dozens of them back through map lookups (the decoders, the
draw key's range walks, the prechecks). PPSA21564 sends ~350k register
packets per 10 s at the title, and the map's node allocations and tree
walks were a visible share of the graphics queue worker's time.
Registers is now a flat value array with a presence bitmap per bank. It
keeps the map's interface and semantics: absent registers stay absent,
iteration and lower_bound walk the present registers in offset order,
emplace does not overwrite and at() throws for an unset register.
