fix(agc): keep half the host heap out of the host import budget

Imports, type-3 mirrors and the buffers and textures placed in host memory share the driver's system memory heap. On Windows that heap is half the physical memory, so on a 16 GiB machine the old budget (the heap less 2 GiB) let imports hold about 6 of its 8 GiB, and host buffer allocations then overflowed into the 256 MiB device-local host-visible heap and failed. The budget now reserves the larger of 2 GiB and half the largest host heap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
AhirandClaude Opus 5.5 authored and Adrià Franch committed 2026-10-19 04:11:15 +02:00
1 parent 40393b0f95
commit f970620d95
4 files changed
+44 -11

No files matched your search

@@ -77,6 +77,9 @@ void SetHostImportWindow(std::uint64_t bytes);
void SetHostImportRangeWindow(std::uint64_t bytes);
void SetHostImportBudget(std::uint64_t bytes);
void SetHostImportDeferredSettles(bool enabled);
// The larger of 6 GiB and half the physical memory, at most the largest host heap less the larger of 2 GiB and half
// that heap: the other half stays for the buffers, mirrors and textures that share it.
std::uint64_t HostImportBudget(const VkPhysicalDeviceMemoryProperties& memory, std::uint64_t physicalBytes);
std::uint64_t HostImportLiveBytes();
struct HostImportSettleCounts {
std::uint64_t submitted = 0;
@@ -1304,9 +1304,9 @@ std::uint64_t physicalMemoryBytes() {
std::atomic<std::uint64_t> budgetOverride{0};
// Pinned imports count against the driver's system memory heap, which imports and type-3 mirrors share; past it
// ordinary host allocations fail. The budget is APS5_HOST_IMPORT_MIB, else the larger of 6 GiB and half the physical
// memory, at most the largest host heap less 2 GiB.
// Pinned imports count against the driver's system memory heap, which imports, type-3 mirrors, buffers and textures
// placed in host memory share; past it ordinary host allocations fail. The budget is APS5_HOST_IMPORT_MIB, else
// HostImportBudget.
std::uint64_t importBudget(const Context& context) {
if (const auto set = budgetOverride.load(std::memory_order_relaxed); set != 0) return set;
static const std::uint64_t configured = []() -> std::uint64_t {
@@ -1314,13 +1314,7 @@ std::uint64_t importBudget(const Context& context) {
return value != nullptr ? std::strtoull(value, nullptr, 10) << 20u : 0u;
}();
if (configured != 0) return configured;
std::uint64_t hostHeap = 0;
for (std::uint32_t index = 0; index < context.memory.memoryHeapCount; ++index) {
if ((context.memory.memoryHeaps[index].flags & VK_MEMORY_HEAP_DEVICE_LOCAL_BIT) == 0) hostHeap = std::max<std::uint64_t>(hostHeap, context.memory.memoryHeaps[index].size);
}
const auto fromMemory = std::max<std::uint64_t>(6144ull << 20u, physicalMemoryBytes() / 2);
constexpr std::uint64_t reserve = 2048ull << 20u;
return hostHeap > reserve ? std::min(fromMemory, hostHeap - reserve) : fromMemory;
return HostImportBudget(context.memory, physicalMemoryBytes());
}
std::atomic<std::uint64_t> rangeImportWindow{64ull << 20u};
@@ -2982,6 +2976,16 @@ void SetHostImportDeferredSettles(bool enabled) {
#endif
}
std::uint64_t HostImportBudget(const VkPhysicalDeviceMemoryProperties& memory, const std::uint64_t physicalBytes) {
std::uint64_t hostHeap = 0;
for (std::uint32_t index = 0; index < std::min<std::uint32_t>(memory.memoryHeapCount, VK_MAX_MEMORY_HEAPS); ++index) {
if ((memory.memoryHeaps[index].flags & VK_MEMORY_HEAP_DEVICE_LOCAL_BIT) == 0) hostHeap = std::max<std::uint64_t>(hostHeap, memory.memoryHeaps[index].size);
}
const auto fromMemory = std::max<std::uint64_t>(6144ull << 20u, physicalBytes / 2);
const auto reserve = std::max<std::uint64_t>(2048ull << 20u, hostHeap / 2);
return hostHeap > reserve ? std::min(fromMemory, hostHeap - reserve) : fromMemory;
}
std::uint64_t HostImportLiveBytes() {
auto& state = Imports();
std::lock_guard lock(state.mutex);
@@ -5829,6 +5829,31 @@ void samplerTableDeviceLimitTests(std::optional<std::uint32_t> selected = std::n
}
}
void hostImportBudgetTests() {
using AgcDriver::Graphics::HostImportBudget;
constexpr std::uint64_t MiB = 1ull << 20u;
constexpr std::uint64_t GiB = 1ull << 30u;
VkPhysicalDeviceMemoryProperties memory{};
Require(HostImportBudget(memory, 0) == 6 * GiB, "a device without memory heaps does not keep the 6 GiB import budget");
Require(HostImportBudget(memory, 32 * GiB) == 16 * GiB, "a device without a host heap does not take half the physical memory");
memory.memoryHeapCount = 3;
memory.memoryHeaps[0] = {7920 * MiB, VK_MEMORY_HEAP_DEVICE_LOCAL_BIT};
memory.memoryHeaps[1] = {8128 * MiB, 0};
memory.memoryHeaps[2] = {256 * MiB, VK_MEMORY_HEAP_DEVICE_LOCAL_BIT};
Require(HostImportBudget(memory, 16 * GiB) == 4064 * MiB, "imports on a 16 GiB machine take more than half its 8128 MiB host heap");
memory.memoryHeaps[1].size = 16 * GiB;
Require(HostImportBudget(memory, 32 * GiB) == 8 * GiB, "imports on a 32 GiB machine take more than half its 16 GiB host heap");
memory.memoryHeaps[1].size = 32 * GiB;
Require(HostImportBudget(memory, 64 * GiB) == 16 * GiB, "imports on a 64 GiB machine take more than half its 32 GiB host heap");
memory.memoryHeaps[1].size = 3 * GiB;
Require(HostImportBudget(memory, 16 * GiB) == 1 * GiB, "a 3 GiB host heap does not keep its 2 GiB reserve");
memory.memoryHeaps[1].size = 8128 * MiB;
memory.memoryHeaps[0].size = 64 * GiB;
Require(HostImportBudget(memory, 16 * GiB) == 4064 * MiB, "a device-local heap counted as the host heap");
memory.memoryHeapCount = 1;
Require(HostImportBudget(memory, 16 * GiB) == 8 * GiB, "a host heap past memoryHeapCount limits the import budget");
}
void textureCacheBudgetTests() {
using AgcDriver::Graphics::TextureCacheBudget;
constexpr std::uint64_t GiB = 1ull << 30u;
@@ -9420,6 +9445,7 @@ int main(int argc, char** argv) {
samplerTableDeviceLimitTests();
misalignedShaderDataTests();
debugBranchTests();
hostImportBudgetTests();
textureCacheBudgetTests();
sampledTextureBudgetTests();
storageTextureBudgetTests();
+1 -1
View File
@@ -178,7 +178,7 @@
- [16-bit depth with stencil](../../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp) (libSceAgcDriver) is stored as `D24_UNORM_S8_UINT`, or `D32_SFLOAT_S8_UINT`, on a device without `D16_UNORM_S8_UINT` as an attachment. Depth comparisons, clears and the depth image see the wider format's precision, so results at the 16-bit quantization boundary can differ from the console; the constant depth bias is scaled by 256 for D24, and a non-zero constant on the D32 float fallback throws. Not measured on console hardware.
- [Depth surfaces created over storage writes](../../../core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp) (libSceAgcDriver): a depth surface created without HTILE copies its depth plane from the storage image at the plane's address, a pending one first, when that image has the plane's extent, one level, a 2D view or a 2D array view (its layer 0 is copied) and a 16-bit format for a 16-bit plane or a 32-bit format for a 32-bit plane. A multisampled surface, a surface the device stores in another format, a surface with HTILE and a surface without such an image start from `DB_DEPTH_CLEAR`, and the stencil plane from `DB_STENCIL_CLEAR`; CPU writes to the plane's memory are not read. Not measured on the console, where the plane is that memory.
- [Flip packets raise the graphics end-of-pipe event](../../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Packets/PacketExecution.cpp) (libSceAgcDriver) - a flip delivers the queue's end-of-pipe event (`sceAgcDriverAddEqEvent`) once the work recorded before it completes, as a `RELEASE_MEM` with `INT_SEL` does, but only on a queue that has never delivered an interrupt from such a release: the queue is latched the first time a release interrupt reaches the event queue, and flips on that queue deliver nothing while its own release interrupts keep arriving. The latch is cleared when the queue has flipped five times without another interrupt from its own releases: the fifth flip delivers the event and unlatched the queue, so a title that waits on the flip event after latching recovers only if it submits further flips. The bound cannot be by time: the next release interrupt can arrive seconds later than a suppressed flip across loading stalls, and delivering an unexpected flip event on a queue that pairs flips with releases can crash; no fixed window short enough to unblock a waiting title keeps such a title alive. Whether the console raises the event from the flip itself or from an interrupt its flip sequence contains is not measured; nor is how the console behaves for a title that waits on the flip event and also raises release interrupts on the same queue.
- [Host import windows and budget](../../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp) (libSceAgcDriver): a registered range larger than 64 MiB is imported one 64 MiB window at a time for buffer and fill accesses; an access across window edges imports the cells it touches, grown to the union of the range's imports it overlaps. An address-based build (`AcquireRegistered`) imports the range whole and replaces its windows. Live imports stay under a budget: `APS5_HOST_IMPORT_MIB`, else the larger of 6 GiB and half the physical memory, at most the largest host heap less 2 GiB (imports and type-3 mirrors share it; #167's rule). A new import beyond it retires the least recently used imports not used in the last 256 lookups; one that would not fit even then is refused before anything is retired, and imports made when a range is mapped never evict. A refused import is tried again at its next use. The 64 MiB window, the 256-lookup protection and the 2 GiB reserve are choices, not measured.
- [Host import windows and budget](../../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp) (libSceAgcDriver): a registered range larger than 64 MiB is imported one 64 MiB window at a time for buffer and fill accesses; an access across window edges imports the cells it touches, grown to the union of the range's imports it overlaps. An address-based build (`AcquireRegistered`) imports the range whole and replaces its windows. Live imports stay under a budget: `APS5_HOST_IMPORT_MIB`, else the larger of 6 GiB and half the physical memory, at most the largest host heap less the larger of 2 GiB and half that heap (imports, type-3 mirrors and buffers and textures placed in host memory share it; on a 16 GiB machine whose host heap is half the memory, a 2 GiB reserve left too little for the rest). A new import beyond it retires the least recently used imports not used in the last 256 lookups; one that would not fit even then is refused before anything is retired, and imports made when a range is mapped never evict. A refused import is tried again at its next use. The 64 MiB window, the 256-lookup protection and the 2 GiB reserve are choices, not measured.
- [Registration preparation at submit](../../../core/libs/prx/libSceAgcDriver/Execution/src/Driver/Queues/Submission.cpp) (libSceAgcDriver) - `Driver::Submit` waits for every registration-time shader preparation queued before the call, not only those of the shaders the commands use, so a submission behind a long queue of unrelated preparations waits for all of them on the submitting thread. Waiting there instead of in the queue worker keeps the GPU side of a submission short: a title that sees a submission take seconds treats the GPU as hung (Goat Simulator 3, PPSA04159, then stops submitting and its garbage collector fails a 10 s timeout). On the console shaders are compiled ahead of time, so neither wait exists; how long a title tolerates a slow submit call is not known.
- [Resource cache entries holding released textures](../../../core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp) (libSceAgcDriver) - a sampled texture the texture cache drops while something else holds it, or makes without caching (`admitTexture` refusing a first use), is marked released, and `ResourceCache::Insert` refuses descriptor sets that hold one and removes the entries that do. The removal waits for the next insert: until then `Find` still returns those entries and their textures stay alive. One process-wide flag starts the sweep, so while the sampled cache is over its budget every insert scans all entries (1,024 at start, up to 16,384 after growth, or `APS5_RESOURCE_CACHE_ENTRIES`, each with its textures); that cost is not measured. When `ReleaseTextureCaches` runs on device memory exhaustion, the sampled textures these entries hold are only marked released and are freed at the next insert, so the immediate allocation retry gets no memory back from them. A draw or dispatch whose textures the cache refused builds its descriptor sets again on each use for as long as the cache stays full; the count bounds (`APS5_RESOURCE_CACHE_ENTRIES`, `APS5_RESOURCE_CACHE_TABLE_ELEMENTS`) still bound nothing by bytes, and sampled textures made outside the texture cache (the disabled cache, depth-surface copies) are never marked. Measured on one title on one RTX 4090: with the default bounds, sampled textures held outside the cache peaked at 2,836 MiB (7,830 MiB before) and device-local heap 0 stayed under its 22.9 GB budget for the whole run (26,357 MiB before, which aborted with `VK_ERROR_OUT_OF_DEVICE_MEMORY` at the wolf scene). Whether the extra descriptor-set builds cost frame time elsewhere is not measured.