4 Commits

Include renames

Author	SHA1	Message	Date
fablevoid	195682e190	fix(libc): watch direct memory mapped over its reservation on macOS ... Register copied the protection it had recorded for pages it already watched over the protection the kernel had just reported. A guest that reserves address space, which registers it as PROT_NONE, then maps direct memory over it read-write and registers it again kept the stale PROT_NONE. Collect only write-protects pages with PROT_WRITE, so writes to that memory were reported once and never again. With PPSA12544 on the macOS port, the title menu text blinked and glyphs went missing. Register now keeps a recorded protection only where its own write protection explains what the kernel reports, and takes the kernel's protection otherwise. A protected page that was mapped again also loses its protected mark, so erase no longer restores the old protection over the new mapping. The kernel is read under the lock, so a concurrent Collect cannot protect a page between the read and the merge. Test write_watch_mapped: a reservation, a mapping over it and a second registration keep reporting writes, and a protected page reserved again stays PROT_NONE. AI-assisted: yes	2026-10-11 11:04:02 +00:00
raghav314andClaude Opus 5.5	c7f4690cd3	fix(agc): host imports write-protect only pages a collect already protects ... Importing guest memory collected the write watch over the whole import, which write-protected every page of it, so pages nothing had collected yet took a write fault on their first CPU write. The import now collects only the parts a collect already protects (GuestWriteWatchCollectArmed). The rest stays unprotected until something collects it, which reports it as written; a GPU write recorded into such memory (MarkWritten) first collects the never-collected parts of its 64 KiB blocks (GuestWriteWatchCollectFresh), so that report is older than the GPU's results and does not read as a CPU write over them. Those pages are stamped as written and their CPU pages are noted, but the block's CPU stamp is left alone, so a GPU label record or a pending storage image elsewhere in the block does not see a CPU write. StoredOver and UnchangedSinceCollected read the noted pages whenever a block has them, so a store to such a page is still seen by a check over that page. The extra generation step happens only when something was collected; a block whose write protection fails is unwatched and the write returns no generation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 11:38:28 +05:30
Elijah De GuzmanandClaude Opus 5.5	80b0bab279	feat(libc): write watch on macOS hosts ... macOS has no counterpart to userfaultfd write-protect or MEM_WRITE_WATCH, so GuestWriteWatch had no backend there and the driver re-read every guest buffer on each use. The macOS backend write-protects watched pages. The first write to a page faults, the SIGBUS/SIGSEGV handler records the page and restores its protection, and Collect reports the recorded pages and protects them again. A page is protected before its record is cleared, under the lock the handler takes, so a write that lands in between is recorded. The lock hands out turns in order: an unfair one starved writers under Rosetta. The protection is real, which adds three entry points: - GuestWriteWatchBeginHostWrite/EndHostWrite lift it around a system call that writes guest memory, which would otherwise fail with EFAULT. GuestArena::HostWrite calls them, as it calls the Windows equivalent. - GuestWriteWatchProtectionChanged tells the watch the protection given to watched pages by mprotect or a new mapping. - GuestWriteWatchProtection reports that protection, since a watched page is read-only between a collection and the next write. Register keeps the recorded protection of a page it already watches: read from the kernel it would be the watch's own read-only protection, and the page would never be watched again. The tests build without the prx libraries and run natively and through Rosetta. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-09 21:11:40 -04:00
Dean Galvin	d8c7c0ccdf	feat(agc): track guest memory writes on Linux with async userfaultfd write protection ... GuestMemory::CollectWrites was built on the Windows arena's MEM_WRITE_WATCH and answered 0 on Linux, so every cached copy of guest memory (textures, resident render targets, buffers, DCC key proofs) read as changed at every check: resident targets were stored and re-uploaded at every refresh and unit shadows stayed off. Astro Bot's intro ran at ~3 fps. libc gains GuestWriteWatch, the Linux counterpart of GetWriteWatch. Every guest mapping libkernel makes (MapAligned: fixed, hinted and aligned) is registered with one userfaultfd in asynchronous write-protect mode (UFFD_FEATURE_WP_ASYNC: the kernel lifts a page's protection on its first write without a handler thread, for user and kernel-mode writes alike), and PAGEMAP_SCAN with PM_SCAN_WP_MATCHING reports the pages written since the last scan of a range and protects them again in one call. Unmap forgets the range. Direct memory is mapped from shared backings over its reservation and a fixed remap replaces the mapping, so both register the new mapping afresh. The watch is per mapping: a write through one mapping of shared physical pages is not seen at another, so physical pages mapped at more than one address leave the watch in every mapping of them. A registered mapping reads as written until a collect covers it. The watch keeps that itself instead of relying on how the kernel classifies pages never faulted in: Linux 6.17 reports a range without page tables as not written and leaves it unprotected, 7.2 reports it as written. The first collect of a fresh range reports it written and write-protects it with UFFDIO_WRITEPROTECT, which puts markers on its unpopulated pages (UFFD_FEATURE_WP_UNPOPULATED), so from then on only real writes report. Fallback: the userfaultfd is opened with UFFD_USER_MODE_ONLY where vm.unprivileged_userfaultfd restricts it, and a probe at startup checks the mechanism on scratch pages (a populated and an unpopulated page written after protection report exactly, untouched ones do not). Without it (kernel older than 6.7, no userfaultfd permission, headers without PAGEMAP_SCAN, or APS5_NO_WRITE_WATCH=1) the watch reports unavailable and the tracker stays unwatched exactly as before: CollectWrites answers 0 and callers compare bytes. A range with a page outside the registered mappings also answers 0, and a protection or scan that fails reports its remaining pages as written and answers 0. The driver's tracker keeps its contract and generations. On Linux its block stamps live in 4 GiB leaves over the user address space, made on first stamp, instead of the arena's flat arrays; the collect memo now works on both platforms. GuestMemory::WriteWatched and Watched replace the arena queries in UnitShadowEnabled and ShadowDestinationFor, so unit shadows run on Linux over watched memory and stay off everywhere else. The Windows arena path is unchanged apart from going through the same stamp helpers. Host imports: a driver may lift the write protection of imported pages itself. amdgpu validates the userptr BO behind a VK_EXT_external_memory_host import with a writable HMM fault on every submission after the pages were protected again, so PAGEMAP_SCAN reports them as written with no CPU store, and a resident display image read as written at every present (found by Pacuka on an RX 6650 XT with RADV: the presented image went black). NVIDIA pins the pages once at the import and stays exact from then on. A probe at device setup decides per device: it maps and watches scratch pages, collects them clean, imports them, collects, submits a GPU copy that reads the import, waits, and collects again. Pages written after the submission mean imports resolve write protection: every import from then on takes its range off the watch (GuestMemory::Unwatch) before it is made, so Watched is false for it, CollectWrites answers 0 and callers compare bytes; GPU writes to watched memory keep their MarkWritten stamps. The result is logged once: "[write-watch] host imports resolve write protection: yes/no". A probe that cannot run takes the unwatch side. APS5_WRITE_WATCH_IMPORTS=watch|unwatch overrides the probe (default probe; anything else throws), and APS5_NO_WRITE_WATCH=1 still turns the watch off. On the RTX 4090 the probe reads 0 of 4 pages written after the GPU read (4 after the import itself), so imports stay watched there. The import itself lifts the protection once where imports stay watched: on the RTX 4090 the pinning at vkAllocateMemory reports every imported page as written (4 of 4 in the probe), with no CPU store. Such a report would read as a CPU write over any GPU results pending in the range and drop them. So every import of a watched range is made inside an import window (GuestMemory::ImportWatched), under the tracker lock: the range is collected right before the import, so earlier CPU writes are stamped as usual, and again right after it, and what the window reports is stamped "maybe written". Readers without pending results see it as a change (UnchangedSince fails, ChangedBlocks reports it), so caches re-upload or compare. Decisions about pending GPU results read it as untracked: ChangedBlocks reports BlockMaybeWritten apart from BlockWritten, and the drop rule, the write-back's keep blocks, the alias checks, the unit shadow's stale drop and the whole-image stale checks (through the new WrittenSince, which also answers false for a range off the watch) store the results instead of dropping them. A real CPU store racing an import also races the pending GPU writes to the same memory, which has no defined order on hardware either, so the GPU results winning is acceptable. From the second collect on the range is protected again and watched normally. A range the watch does not cover whole, or a collect of the window that fails, is taken off the watch instead. An unwatched import stays off the watch for the rest of its mapping: a remap registers the range afresh, the import kept over it takes it off again at the next reconcile, and a re-import after an unmap follows the decision. Unwatch invalidates the collect memo, so a collect memoized before the import answers 0. A range that leaves the watch while a storage image holds a generation for it no longer reads as CPU-written: ChangedBlocks now tells whether the range is watched, and the drop rule, the write-back's keep blocks and the alias borrow treat an unwatched range as untracked (stored, never dropped), as a generation of 0 is. Tests: guest_memory checks the watch through libkernel mappings (a CPU store, a read, a kernel write through read(), another thread, a memcpy across pages, an mprotect cycle, a partial munmap and a fixed remap) and over a fresh 2 MiB-aligned mapping without page tables, checks direct memory (watched, an alias takes the shared pages off the watch in both mappings, a fixed direct remap over flexible memory is watched afresh), and names the failing line and the reported pages. The recorder tests allocate watched memory on both platforms, so the UnchangedSince, key proof, close race and unit shadow tests run on Linux, and the storage refresh test gains a watched variant: results kept on the GPU reach a CPU reader, and a plain CPU store into one unit with newer results pending is found by the watch alone. The recorder tests now also run the real import probe, check the decision with the result injected (unwatch: an import leaves the watch, a memoized collect, a partly imported range and the unimported neighbour answer as they should, MarkWritten stamps only watched memory, a remap and a re-import after an unmap follow the decision; watch: the import stays watched and stamped), and check that results pending when their memory leaves the watch are stored rather than dropped. A further test imports the memory of a storage image with results pending in watch mode: the results survive the import window and reach guest memory, a cache generation taken before the import sees a change, the window reads as maybe written and not as a CPU store, and a CPU store after it reads as one again. Unit shadows and the watched storage refresh report themselves as not run when imports are compared.	2026-10-01 00:08:08 +13:00