On Windows the write watch is dropped in whole 64 KiB blocks
(WriteWatchCoverage::Exclude), but the unwatch hook was given the page
range of the import. An image with results pending in the same block as
an imported page, outside that page, lost the watch without its guest
bytes saved, and storing its results threw.
The hook now gets the span of blocks Exclude drops (ExcludedSpan, the same
block math), so every image with results pending in those blocks saves its
bytes first.
A pattern fill reads its count and pattern from guest memory on the queue worker. With pipelined
draws it read them while earlier commit items that write them (labels, DMA stores, draws) were
still queued, so it stored the stale value (agc_driver_pattern_buffer_fill under load). It now
drains the draw pipeline first when a queued item writes one of its input ranges.
The same fill held the GPU lock from the input read through DrawPipeline::Enqueue. With the
pipeline full, Enqueue waited for the committer while the committer's next item waited for the GPU
lock: a deadlock. The input lock is released before the fill is queued, and Enqueue throws when it
would wait for room while the calling thread holds the GPU lock.
(cherry picked from commit c653e9ea1d)
With APS5_PIPELINED_DRAWS=1, queue 0's worker decodes, captures and recompiles a draw and hands it to
a committer thread that records it (Graphics::Draw under the GPU lock), so the two run in parallel
instead of one after the other. Work the committer has not reached yet is tracked by the guest ranges
it will write; a later packet that reads or writes them, or one the pipeline cannot order, drains it.
Ordered as commit items instead of drains:
- labels (WRITE_DATA, RELEASE_MEM) and same-queue waits on them; APS5_PIPELINE_DRAIN_LABELS=1 drains;
- the end of a queue-0 submission and a released REWIND tail; APS5_PIPELINE_DRAIN_SUBMISSIONS=1 drains;
- the title's buffer fill kernels; APS5_PIPELINE_DRAIN_FILLS=1 drains;
- DMA_DATA immediates and copies (APS5_PIPELINE_DMA=1).
A draw's write ranges include the storage images it writes and its exact color and depth target
ranges (APS5_EXACT_DRAW_WRITES=1), not estimates.
Draw-path costs the pipeline exposed:
- ShaderMemory moves, so a commit owns its capture's pages;
- a submission made before the worker's current collect epoch began keeps the epoch, so a worker
behind a queue of submissions walks a surface once per catch-up (APS5_SUBMISSION_EPOCH_EACH=1 bumps
at every submission); a thread remembers the ranges it walked for the whole epoch;
- the pipeline signals only threads that wait; the pending-image and unit-shadow registries take a
spinning slim lock instead of std::mutex (winpthreads contention was kernel events); ScanPending skips
the registry walk when no pending image can overlap;
- fewer heap allocations per recorded draw (reused descriptor sets and scratch vectors, buffer pool
size classes kept).
GTA V Performance mode, prologue bank interior, RTX 5080: 11.7 -> about 20 fps together with the cheaper
captures below it.
Off by default here; the follow-up turns it on.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Gjon Gojcevic <7807870+gjonn@users.noreply.github.com>
The per-draw capture of a shader's memory (the SRT walk and the guest pages it reads) cost about
15 us per draw packet in GTA V Performance mode; it now costs about 6.6.
- The SRT evaluator memoizes per plan value in a thread's reused, stamped array instead of a hash
table allocated and grown on every walk (evaluate 5.3 -> 3.6 us per capture).
- ExtractPlan also stores the plan's values as 32-byte records with argument ids, and the walk
evaluates that compact copy: the IR walk spent most of its time in cache misses (3.6 -> 2.7 us).
The evaluator is a template over the node view, so both walks share every opcode's semantics, memo,
cycle and failure rules. A plan that cannot be expressed compactly keeps the IR walk.
APS5_VERIFY_SRT_COMPACT=1 walks both and throws on any difference.
- ShaderMemory's valid/read/recent marks are arrays of 64-bit words scanned with countr_zero instead
of bitsets tested bit by bit (about 1 us per draw packet).
- Captured pages come from a per-thread pool, are found through a short vector with a last-page
shortcut, and a page no recorded GPU work writes gets the flush hook a whole-page read would make,
then has each 64-byte block copied when a word of it is first read (about 0.3 us per packet).
Pages with pending GPU writes and unmapped pages are read as before.
- APS5_PROFILE_DRAW splits the capture time into SRT evaluation and materialization, and prints the
[capture] line once per 10 s instead of every 500 captures (it cost about 2 us per capture).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Gjon Gojcevic <7807870+gjonn@users.noreply.github.com>
Since 39fdc48b5 a graphics program outside every registered shader was
read raw only while the registry was empty, and any RegisterShader after
the first raw draw threw. A title that registers some shaders through
sceAgcCreateShader and also draws programs it never registers stops at
its first such draw with "graphics program does not belong to a
compatible registered shader": oops-sdk homebrew registers its display
shaders while radeonsi (oops-mesa) submits raw programs.
Graphics programs now resolve as compute dispatches already do: an
address inside a registered shader resolves to that registration (a
registration of another type still throws), any other address is read
raw. RegisterShader throws only when the new shader's code overlaps a
graphics program that was drawn raw, where the program would get a
second identity (the draw cache revalidates raw programs by reading them
raw), instead of after any raw draw.
The drawn range is recorded once per raw snapshot: a flag on the
snapshot keeps later decodes of the same program off the lock. Recorded
ranges are merged, so the overlap check looks only at the predecessor of
the queried end.
Tests: agc_driver_prepared_graphics (a draw whose programs are registered
as compute shaders is refused, where it used an address outside every
registration before), agc_driver (ProgramSnapshot reads raw beside registered shaders
and refuses a registered program of another type; nested, overlapping
and adjacent raw ranges), agc_driver_prepared_unregistered_graphics
(raw-drawn ranges recorded exactly, unrelated code not reported).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A draw stage whose captured words differ from a cached variant only at pure
data positions (read by the shader as data, never as an address, descriptor
or decode input) is now a hit: the live words are patched into the cached
compiled result, as the dispatch cache's EqualData hits do. The positions are
recorded at insert from the stage's resource capture, minus the words the
draw decode reads. Patched draws take no draw recipe.
APS5_NO_DRAW_STAGE_DATA_HITS=1 turns it off; APS5_VERIFY_DRAW_DATA_HITS=1
builds the stage anyway and throws on any difference. The draw-cache profile
line counts stage data hits and refreshed words.
Co-Authored-By: Claude <noreply@anthropic.com>
Since #5216 a key gets its entry at its second sight, so a shape whose
user words change on every draw never got an entry and its data-only hits
never started. A key whose shape was seen before but has no entry is now
admitted at its first sight. A shape that already has an entry serves
data-only hits from it, and its new keys are still admitted at their
second sight, so per-draw keys don't churn the cache. Shapes are
remembered in a second sightings table of the same size.
agc_driver_draw_cache_reuse draws three times with new pixel user words
over a cached shape (three data-only hits, the vertex stages reused, no
insert), then three keys of a new shape (the second one is admitted), and
runs again with APS5_VERIFY_DRAW_DATA_HITS=1.
Co-Authored-By: Claude <noreply@anthropic.com>
The draw key now has a shape: the key without the user words the stages read.
A draw whose key misses but whose shape has an entry copies that entry's decode,
re-reads only the user words (and merged-stage pointers), and reuses each
compiled stage whose captured reads still hold, compiling the rest.
APS5_NO_DRAW_DATA_HITS=1 disables it; APS5_VERIFY_DRAW_DATA_HITS=1 also decodes
and compiles in full and throws when the results differ.
A shape keeps its previous entries and only points at its newest key, so
draws of one shape that alternate between stable user words keep their own
key hits. The shape is passed only when data hits are on, so
APS5_NO_DRAW_DATA_HITS=1 keeps the previous behaviour exactly.
Rebased onto main a8b032b2 by CouchRiv (the conflicts with #639 resolved; built and tested on Linux with an RTX 4070 Ti and RADV, and on Windows with an RTX 5080), and onto main 6ac63bf0c by gjonn (main's free-function decoder with its static-ABI mode, its incremental register key as the shape key).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: CouchRiv <270190108+CouchRiv@users.noreply.github.com>
Co-Authored-By: Gjon Gojcevic <7807870+gjonn@users.noreply.github.com>
A dispatch now leaves its initiator and, for DISPATCH_DIRECT, its group
counts in the shader registers, where the CP puts them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PrepareDispatch now creates a missing compute pipeline once the
descriptor set layout is known, so the driver's compile no longer runs
under GuestMemory::GpuMutex. Other queues, presentation and label
writes go on during it, and compiles from different queues overlap.
Two queues that need the same new variant create it once.
The pipeline is still created in dispatch() when the layout is only
built under the lock, or with APS5_LOCKED_BUILD=1. The creation passes
the context's PipelineCacheSynchronization to CreateComputePipeline.
Since registered shaders are prepared in the background, a draw whose shader
was registered just before could wait seconds for its preparation inside the
queue worker. Under CPU load Goat Simulator 3's first submissions took ~3.7 s;
the title treated the GPU as hung, stopped submitting, and its 10 s garbage
collection timeout crashed it on every launch. Driver::Submit now waits for
the preparations queued before it, on the submitting thread, before the
commands reach the GPU.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The trace summary line keeps `trace_dropped` as before: true when any
thread lost a slice, any thread's totals table was full, or the gpu
track was full. When it is true, the line now also carries
`trace_dropped_threads`, one entry per affected thread name, so the reader
can tell which thread lost data and how much. Threads that share a
name, such as the shader prepare threads, share one entry: losses are
summed, the earliest fill is kept and totals_full is true if any is.
Each entry has `events_full_after_s`, the seconds from the first
present of the window to the start of the first lost slice. It is
absent for `gpu` and in every summary line (windowed frames/seconds
and APS5_TRACE_SUMMARY_EVERY) whenever the first lost slice starts
outside the line's span of presents, including a slice whose scope
began before the window opened or one lost after the last present.
`events_lost` is the number of slices not recorded and is absent when
0; in interval lines it counts the buffer's losses since it started.
`totals_full` is true when the thread's totals table overflowed, which
makes the summary numbers incomplete; a thread with only `events_lost`
has complete totals.
The Chrome trace gets a `trace buffer full` instant event on a thread
at the point where its buffer filled.
BUILD.md documents the new fields and the event.
`trace_dropped` is unchanged, and when nothing drops the default output
is byte-identical to before.
Refs #5786
Depends on #5808
Materialize runtime-indexed image and sampler records into descriptor
tables while keeping descriptor selection in the shader. Report invalid
active entries through the existing deferred fault buffer.
Bind fragment tables in the fragment set when pipeline libraries split it,
and reserve a physical descriptor for empty tables. Reject unsupported
sampler, border, cube and anisotropic pairings per entry. Keep one column
per heap access shape, one T# shared by several image operations included
(milescoviello's #5550 case), and narrow it to the union of its keys'
domains when every key has one (SleglJan's key-domain changes). Plan
columns read at (key << n) & mask over their whole domain (tibzejoker's
fix). Read records in runs through SleglJan's bulk reader, and reject
entries whose surface has no mapped byte, as Tamalero proposed. Store
table columns in disk plans; update the runtime ABI and disk format. Bound
table-sized entries in the resource cache (65,536 table elements) and the
dispatch cache (256 MiB of captured words), evicting LRU.
The stream copy kernel is matched by its 16 code words instead of a hash, and the
match and its range checks become free functions (MatchesStreamCopyKernel,
StreamCopyRange) so a test can reach them. Its loads and stores are format
instructions, so the match also requires the descriptor the pattern fill takes:
32_32_32_32_UINT, XYZW selects, no add-tid, out-of-bounds mode 0, type 0. Copy
paths are counted in copyPathCounts whether or not APS5_PROFILE_DRAW is set.
agc_driver_stream_copy checks the match against every word with a bit flipped,
the descriptor and dispatch refusals, the range clamp and refusals, and runs nine
dispatches through the driver: four must take the HLE path and copy elements
g*128+t and g*128+64+t, five must be refused for their shape and leave the
shader's result.
A device that supports VK_EXT_device_fault reports, on the first
VK_ERROR_DEVICE_LOST a check sees, the fault description, the faulting
addresses (invalid reads and writes, or the instruction pointers of waves
still running when the device hung) and the vendor fault codes.
APS5_GPU_CHECKPOINTS=1 records a VK_NV_device_diagnostic_checkpoints
marker with the program address before each guest dispatch and prints the
last markers the GPU reached, which names the dispatch that hung. Without
either extension nothing changes.
Test: agc_device_fault drives the report through mock
vkGetDeviceFaultInfoEXT and vkGetQueueCheckpointDataNV entry points.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: overlordxrz-source <overlordxrz-source@users.noreply.github.com>
A 16-word compute kernel copies a buffer in 16-byte elements: 64 threads a
group, thread t of group g loads elements g*128+t and g*128+64+t from the
index-addressed buffer in s[0:3] and stores them at the same indices of
s[4:7]. Run as a shader against host imports it costs 11.8 ms of GPU per
present in PPSA25344's main menu on an RTX 4060.
Matched like the existing copy kernel, by its code hash and its descriptors
(16-byte stride and no swizzle on both, the same word 3), it becomes one copy
through the existing copy HLE path: the elements the kernel stores, min of
groups * 128 and the destination's record count, when the source's record
count covers them all. APS5_NO_COPY_HLE turns it off with the other kernel.
A recorder batch takes a trace id when it opens while a trace records. The
thread that opened it stamps the id on every slice it records afterwards,
and the batch's GPU ranges (the whole batch and its command classes) carry
the same id, written as "args":{"batch":N}. A GPU batch in the trace can
then be tied to the CPU slices that recorded it.
Adrià