The second queue that settles imports is always created on Windows, and a device without one keeps
the comparisons. The other-family settle path is tested through the recorder test device built
with its settle queue on another family (agc_import_watch_other_family) instead of
APS5_SETTLE_OTHER_FAMILY.
The settle read keeps a private import's window open for one round trip on the settle queue, so a
store from another thread can land inside it. CheckImportWindowOtherThread: a store of new bytes from
another thread within the window counts as written, and a same-value one keeps its possible CPU
touch. The debt entry notes the longer window.
An import that leaves its range watched copies the whole range first, so
that each page the import resolves can be compared with its bytes when
the window closes (#3874). Ghost of Tsushima (PPSA02225) maps and
imports 4.9 GB of direct memory at start-up, and that copy is the
largest allocation the title makes.
Pages that are all zero are no longer copied. Each page of the range
gets a slot, the index of its copy, or none for a page that was all
zero, and such a page is compared with a zero page. Every page is
compared with the same bytes as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three ways to break the untracked hint passed every test:
- hinted() without its coverage check walks, and resets, the marks of
blocks that fresh backing put back under the watch;
- walkHints stamping the walk's first block instead of the marked page's
block hides a store from an image over another block;
- compareUntracked without NewestStamp in a hinted key misses a GPU write
into an import, which sets no mark and is only recorded by MarkWritten.
CheckUntrackedHint now walks the whole range after a store into its
second block and reads each block's hint on its own. It also recommits
fresh backing over the hinted range: the range must have no hint, and a
CPU store must still reach its collect. RunHintedImport hides a store's
mark and records it with MarkWritten, and the image must still be
uploaded at the next epoch. Each of the three changes above now fails
agc_write_tracking or agc_driver_dcc_keys_first_write.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Windows every direct host import leaves the write watch, and a storage
image over it compares its whole surface with its guest-byte copy at every
collect epoch. #4432 measured 7-8.6 s of every 10 s in those compares on a
GTA V scene, with 98% of the compared surfaces unchanged.
Tracker blocks that leave the watch now keep their write-watch marks as a
hint. Leaving the watch gives a block a fresh hint stamp, and
UntrackedHint walks a range of such blocks, at most once per collect epoch
and thread, stamping the blocks whose pages a CPU store marked; it
returns the newest stamp of the range. compareUntracked puts that stamp in
its memo key in place of the collect epoch, so an image is compared again
after a mark in its range, a driver store, a mapping change or a recorded
GPU write. The byte compare still decides whether anything changed: a mark
the driver makes without a CPU store costs one compare and drops nothing.
Nothing else reads the hint. Collects, Watched, UnchangedSince,
ChangedBlocks and NewestStamp still treat those blocks as untracked, and
Linux is unchanged.
agc_driver_dcc_keys_first_write refreshes an imported storage image over
50 collect epochs with no store: it compared the image 50 times before and
0 times now. A CPU store is still uploaded on the next refresh, and a store
of the bytes already there costs one compare and keeps the pending
results. agc_write_tracking checks the hint itself.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A sampled texture over a partially resident surface (a texture array whose
heap the guest commits on demand) never got a write generation: CollectWrites
fails on a range with reserved pages, so every cache hit compared the whole
committed part with its snapshot. CollectCommittedWrites collects the
committed runs one by one and returns the runs with the generation; the cache
entry records them, and a hit skips the compare only when the committed runs
are the same and UnchangedCommittedSince holds over them. A commit, decommit,
view mapping or unmapping inside the surface changes the runs or stamps the
new pages, so it is compared as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The second import-window case of agc_write_tracking makes one page
unreadable with VirtualProtect or mprotect and expects an import that
stays watched to throw before it copies its range. On Windows CheckRange
answers from its table of committed arena pages, which is kept current
by the registry's invalidator. The first case had checked the same
allocation, so the table still said the page was readable, the check
passed and the copy faulted on the page.
ProtectPage now reports the change through GuestAllocationsInvalidate,
as the registry does for a guest protection change and as
tests/BdaResources.cpp does after VirtualProtect.
A host import that leaves its range watched can resolve the write
protection of the imported pages without storing anything, as NVIDIA's
Linux driver does for every page. ImportWatched stamped those pages as a
possible CPU store (BlockMaybeWritten), and only a storage image tracked
by 64 KiB unit could clear the stamp, by comparing with a copy of the
guest bytes it was uploaded from. Since #3372 a fast-cleared upload
therefore reads its whole surface from guest memory wherever such an
import is possible, although nothing but that compare uses the copy.
ImportWatched now takes whether the import leaves its range watched. If
it does, the range is copied before the stores made so far are
collected, and after the import each page whose write protection was
resolved is compared with that copy. A page with other bytes is stamped
as a CPU store. A page with the same bytes keeps only the CPU-side stamp
the label rules read, so UnchangedSince holds for it for every consumer:
storage images tracked by layer, snapshots and caches as well.
If it does not (a host-pointer import on a device whose imports are
compared, a direct import on Windows), nothing is copied or compared:
the pages resolved in the window get the CPU-side stamp and the range
leaves the watch before the tracker is unlocked. Where host-pointer and
dma-buf imports differ in this, the dma-buf import that follows a failed
host-pointer import is a window of its own, so each window knows its
kind before it starts.
BlockMaybeWritten, the written-stamp table, WatchedImportsStampPages and
the unit compare in StorageTexture::Refresh are gone. The fast-clear
upload skips the guest-byte copy whenever its range is watched, as
before #3372.
On Windows the guest arena is reserved while libc loads, around whatever
the process has already mapped in its interval. About one start in forty
has nothing mapped there, so the arena has no host hole:
agc_driver_graphics then fails with "write-watched arena contained no
host exclusion to test" and agc_write_tracking prints that its host-hole
checks were skipped.
agc_write_tracking now starts itself suspended, reserves one allocation
granule in the middle of the child's arena interval before the child's
loader runs, and the child runs the host-hole checks on that range. The
range is a host hole on every start, so the checks no longer depend on
the address layout. The four queries agc_driver_graphics made on an
enumerated hole are among them and are removed there.
The memo stamp held TrackerGeneration, which every collect and driver store
advances, so it almost never matched. Driver stores into uncovered ranges
record their generation per 64 KiB block, and NewestStamp of the surface
replaces TrackerGeneration in the stamp.
On Windows agc_write_tracking has failed since #3594 with
write tracking test failed: an unwatched range reads as not stored over
from the check in CheckSharedBlock:
if (!Watched(outside, unwatched.size())) Require(StoredOver(outside, unwatched.size(), TrackerGeneration()), "an unwatched range reads as not stored over");
The host stack array lies in a host hole of the guest arena. #3594 made
Watched, collectWrites, UnchangedSince, UnchangedSinceAll and storeOwn
reject ranges overlapping such holes, but StoredOver, MarkWritten,
WrittenSince, ChangedBlocks and ImportWatched still accepted them through
WriteTracker::covers(), so StoredOver proved a range not stored over while
Watched called it unwatched (#4087).
The exclusion now lives in WriteTracker::covers() on Windows, which every
query already asks, and the per-function checks are removed. Host holes
are padded to the allocation granularity and the tracker blocks are 64 KiB
from the arena base, so no block is shared between a hole and arena
memory, and ranges outside the holes get the same answers as before.
A new case enumerates the arena for a host hole and checks every query
against it; on the previous code StoredOver, MarkWritten, StoreOwnBytes,
WrittenSince and ChangedBlocks fail it.
Fixes#4087
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Scan version stamps within contiguous leaves. On Linux, collect nearby
pages within queue epochs only after confirming full watched coverage.
Keep the requested range on Windows. Select the range before collection
and do not retry a failed walk.
Test sparse-leaf boundaries, write histories, partial unwatches and epochs.
Fedora 44 Release tests pass with default, disabled and shared caching.
Isolated upstream comparison reduces the saved collect replay by 11-12%.
Windows was not tested.
AI-assisted: yes.
StorageTexture::writeBackLayers keeps the CPU's bytes for an edge unit when StoredOver
reports that the CPU wrote it since the unit's generation. That check read one generation
per 64 KiB block, so a store to another allocation sharing the block counted as a store
over this surface and the whole block was kept, discarding GPU results written into it.
The driver path in the same check already compares range pieces; the CPU path now does
the same. Each collect records the generation of every page it found dirty, and StoredOver
tests only the pages its range overlaps. A block with no page record keeps the previous
whole-block answer.
Measured on an RTX 3080 Ti, main at df16c4c2:
- agc_write_tracking stores at base + Block + 8, outside the queried range but inside the
shared block. That case fails on main ("a CPU store outside the shared range counts for
it") and passes here.
- Full suite 408 of 408 (Linux, CachyOS, GCC, Release, BUILD_TESTING=ON).
This does not cover the other half of #865: the report's finding 2, where the stamps fall
on the surface's own pages and the bytes under them are unchanged, still keeps the block.
No title was run here, so the fix is offered for the report's own repro.
A store the driver makes with the CPU into watched guest memory (a
GuestMemory::Write, a changed-bytes write-back, a completion action's label
store, the copy HLE's CPU path) faults its pages dirty in the write watch and
stamps the range with MarkWritten. The faults stayed behind: collects are
memoized per packet, so the next walk over those pages came later and stamped
them again, as a CPU write made after every check in between.
PPSA21564 fast-clears its normal G-buffer every frame with a register DCC key
fill on the GPU. While that fill is still pending, the first draw into the
target materializes the clear with a CPU store of the clear texels and a
refresh of the resident image. A dozen draws later a walk found the store's
faults, the target read as changed, and the image was uploaded again from
guest memory, dropping the draws recorded in between. Lighting then sampled
cleared normals and the level's moon and rocks went unlit in about half the
frames.
The driver's stores now consume those faults themselves: a walk before the
store stamps earlier CPU writes as such, a walk after it takes the store's
own faults as a driver store, and the range is stamped as before.
On Windows the arena's range also holds host regions that were already in
use when it was reserved (heaps, thread stacks). They are not write-watched,
so a store into one is stamped as before, without a walk.
Write stamps are per 64 KiB block, so a write-back of one surface stamps the
whole block it shares with the surface packed right after it. A pending
image of that neighbour read the stamp as a store over its own bytes: its
write-back kept the guest bytes of the shared block in place of its results
and its next refresh re-uploaded them, stale. PPSA21564's half-resolution
fog texture (960x540 RGBA16F, linear, starting 0xb800 bytes before a block
end right after another 960x540 surface) lost its first 6 rows that way,
which the fog composite upsampled into an orange band across the top 12
rows of the scene in one of its cinematics.
The tracker now remembers the byte ranges of the driver's own stores
(MarkWritten) in the blocks they cover only in part, and StoredOver says
whether a store reached a given range since a generation (a CPU store still
counts for its whole block). The write-back keep rule and the refresh drop
rule ask it for the blocks a surface covers only in part.
Romans Pokrovskis