Merge pull request #6317 from SleglJan/pr/clip-distance-full

fix(shader): leave out the all-zero position distance when guest distances fill all eight

core/libs/prx/libSceAgcDriver/tests/PositionInvariance.cpp

@@ -58,6 +58,45 @@ void verifyVertex() {
     requireInvariantPositions(Recompile(baseRequest(ShaderStage::Vertex)), "vertex shader");
 }
 
+std::uint32_t clipDistanceLength(const ShaderRecompiler::RecompileResult& result) {
+    const auto& words = result.spirv.Words();
+    std::uint32_t variable = 0;
+    for (std::size_t at = 5; at < words.size() && (words[at] >> 16u) != 0; at += words[at] >> 16u) {
+        const auto count = words[at] >> 16u;
+        if (static_cast<spv::Op>(words[at] & 0xffffu) == spv::OpDecorate && count == 4 && words[at + 2] == spv::DecorationBuiltIn && words[at + 3] == spv::BuiltInClipDistance) variable = words[at + 1];
+    }
+    if (variable == 0) return 0;
+    std::uint32_t pointer = 0;
+    std::uint32_t array = 0;
+    std::uint32_t length = 0;
+    for (std::size_t at = 5; at < words.size() && (words[at] >> 16u) != 0; at += words[at] >> 16u) {
+        if (static_cast<spv::Op>(words[at] & 0xffffu) == spv::OpVariable && words[at + 2] == variable) pointer = words[at + 1];
+    }
+    for (std::size_t at = 5; at < words.size() && (words[at] >> 16u) != 0; at += words[at] >> 16u) {
+        if (static_cast<spv::Op>(words[at] & 0xffffu) == spv::OpTypePointer && words[at + 1] == pointer) array = words[at + 3];
+    }
+    for (std::size_t at = 5; at < words.size() && (words[at] >> 16u) != 0; at += words[at] >> 16u) {
+        if (static_cast<spv::Op>(words[at] & 0xffffu) == spv::OpTypeArray && words[at + 1] == array) length = words[at + 3];
+    }
+    for (std::size_t at = 5; at < words.size() && (words[at] >> 16u) != 0; at += words[at] >> 16u) {
+        if (static_cast<spv::Op>(words[at] & 0xffffu) == spv::OpConstant && words[at + 2] == length) return words[at + 3];
+    }
+    throw std::runtime_error("the clip distance array has no constant length");
+}
+
+constexpr std::array<std::uint32_t, 7> DistanceExports{0xf80000dfu, 0u, 0xf80000efu, 0u, 0xf80008cfu, 0u, 0xbf810000u};
+
+void verifyGuestClipDistances() {
+    using namespace ShaderRecompiler;
+    for (const auto [enabled, expected] : {std::pair{0x007fu, 8u}, std::pair{0x00ffu, 8u}, std::pair{0xf007u, 4u}, std::pair{0xf00fu, 4u}}) {
+        auto request = baseRequest(ShaderStage::Vertex);
+        request.shader.code = DistanceExports;
+        request.context.vertex->paClVsOutCntl = 0x00c00000u | enabled;
+        const auto length = clipDistanceLength(Recompile(request));
+        if (length != expected) throw std::runtime_error("clip and cull distance enables " + std::to_string(enabled) + " gave a clip distance array of " + std::to_string(length));
+    }
+}
+
 void verifyMesh() {
     using namespace ShaderRecompiler;
     static constexpr std::array<std::uint32_t, 1> capabilities{spv::CapabilityMeshShadingEXT};
@@ -81,6 +120,7 @@ void verifyMesh() {
 int main() {
     try {
         verifyVertex();
+        verifyGuestClipDistances();
         verifyMesh();
         std::cout << "Vertex and mesh position outputs are Invariant\n";
         return 0;

core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp

@@ -329,15 +329,12 @@ void DefineOutputs(SpirvEmitterState& state) {
     }
     const auto stage = state.program.Resources().stage;
     const bool exportsPosition = std::ranges::any_of(state.outputs, [](const SpirvOutputBinding& output) { return output.kind == StageOutputKind::Position; });
-    if (exportsPosition && (stage == IrShaderStage::Vertex || stage == IrShaderStage::TessellationEvaluation)) {
+    if (exportsPosition && (stage == IrShaderStage::Vertex || stage == IrShaderStage::TessellationEvaluation) && clipDistanceCount + cullDistanceCount < MaxClipDistances) {
         if (std::ranges::find(state.supportedCapabilities, spv::CapabilityClipDistance) == state.supportedCapabilities.end()) {
             FailEmit("clipping a primitive with an all-zero clip position needs shaderClipDistance on the target device");
         }
         state.zeroPositionClipIndex = clipDistanceCount;
         clipDistanceCount += 1u;
-        if (clipDistanceCount > MaxClipDistances) {
-            FailEmit("guest clip distances plus the all-zero clip position distance exceed " + std::to_string(MaxClipDistances) + " clip distances");
-        }
     }
     const auto BuiltIn = [&](std::uint32_t& variable, std::uint32_t type, const char* name, std::uint32_t builtin) {
         if (variable == 0u) {

docs/dev/debt/shader.md

@@ -17,7 +17,7 @@
 - [s_subvector_loop_begin and s_subvector_loop_end](../../../core/shader/recompiler/Translation/src/ControlFlowInstructions.cpp) (core/shader/recompiler) - follow AMD's RDNA 2 instruction set reference and a gfx1036 (wave64) measurement, pinned in [SubvectorLoop.cpp](../../../core/libs/prx/libSceAgcDriver/tests/execution/SubvectorLoop.cpp); `S0` in the end instruction's `elif(S0 == 0)` is read as its SGPR operand. A wave32 shader and a destination other than an SGPR throw because neither is measured. Measured on PC RDNA2 (Radeon 610M), not on a PS5. The [specialization pass](../../../core/shader/recompiler/SpirvBackend/src/SpirvSpecialization.cpp) keeps a constant-condition branch whose untaken target is a loop header, since folding it removes the loop's back edge, which SPIR-V requires exactly once per loop header (SPIRV-Tools rejects the module and the NVIDIA driver 615.78.08 crashes compiling it); a loop whose back-edge block becomes unreachable in another way is not handled
 - [Vertex entry merged wave info](../../../core/shader/recompiler/Translation/src/InstructionTranslator.cpp) (recompiler) - `s3` of every vertex-path shader is set to `waveSize | waveSize << 8` (vertex and primitive count of a full wave, wave index 0), or to `32 | 32 << 8` for a wave64 program on a 32-lane host subgroup, which runs it at one lane per invocation and holds no lanes 32-63. On a device whose `subgroupSupportedStages` lacks the vertex stage (lavapipe, MoltenVK), vertex and local programs run as one-lane waves: `s3` is `1 | 1 << 8`, the lane id is 0, a ballot is the invocation's own bit, a shuffle or broadcast returns the invocation's value, an elect is true and a subgroup reduction is the value itself, so no `GroupNonUniform` capability is declared. Layout taken from the prologue Mesa's ACO emits for NGG vertex shaders (`s_bfe_u64 exec` from `s3`), not measured on hardware. The driver gives every graphics program user data from `s8`, so the condition does not tell NGG from legacy vertex shaders: they just never read `s3`. A draw whose last wave is partial still reports a full wave, and so does a host wave with idle invocations (an RTX 5070 ran every vertex warp of a non-indexed triangle list with 30 of its 32 invocations).
 - [s_barrier in vertex-path programs](../../../core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp) (recompiler) - emits no instruction when the program has no LDS or GDS access and no memory write or atomic, so no wave of the subgroup can observe another; the `GS_ALLOC_REQ` such a barrier orders on NGG passthrough programs is already dropped on the vertex path. The Vertex execution model has no workgroup barrier. Programs with LDS, GDS or memory writes throw. Not measured on hardware
-- [Vertex clip and cull distances](../../../core/shader/recompiler/Translation/src/ShaderInputInfoBuilder.cpp) (libSceAgcDriver) - `PA_CL_VS_OUT_CNTL` enable bit `s` of `CLIP_DIST_ENA`/`CULL_DIST_ENA` is taken to select component `s` of the `pos1`/`pos2` exports, with the Vulkan array index counted by the enabled bits below it, and `VS_OUT_CCDIST0_VEC_ENA` as bit 22; this follows the recompiler's existing position-export mapping, not hardware or documentation. The guest clip distances share one `gl_ClipDistance` array with the all-zero clip position distance, which takes the index after them, so a shader with eight guest clip and cull distances exceeds the eight the device holds and fails to compile. A vertex program whose own code has no `pos1`..`pos3` export is compiled and prepared without these bits (`PositionExportLayout`), so registration and draws agree on its artifact; an export in a subroutine it calls is not seen by that scan
+- [Vertex clip and cull distances](../../../core/shader/recompiler/Translation/src/ShaderInputInfoBuilder.cpp) (libSceAgcDriver) - `PA_CL_VS_OUT_CNTL` enable bit `s` of `CLIP_DIST_ENA`/`CULL_DIST_ENA` is taken to select component `s` of the `pos1`/`pos2` exports, with the Vulkan array index counted by the enabled bits below it, and `VS_OUT_CCDIST0_VEC_ENA` as bit 22; this follows the recompiler's existing position-export mapping, not hardware or documentation. The guest clip distances share one `gl_ClipDistance` array with the all-zero clip position distance, which takes the index after them. A vertex program whose own code has no `pos1`..`pos3` export is compiled and prepared without these bits (`PositionExportLayout`), so registration and draws agree on its artifact; an export in a subroutine it calls is not seen by that scan
 - [32_AR color exports](../../../core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp) (shader recompiler) - with `SPI_SHADER_COL_FORMAT` 3 the red value is export channel 0 and the alpha is export channel 1, the GFX10 layout of Mesa's [`emit_ps_color_export`](https://gitlab.freedesktop.org/mesa/mesa/-/blob/3e32b0e1f132231b7a972c1b0b5b06c7e16cf8b9/src/amd/common/nir/ac_nir_lower_ps_late.c#L400-L411), ACO and radeonsi in Mesa 22.0 ([aco_instruction_selection.cpp](https://gitlab.freedesktop.org/mesa/mesa/-/blob/a82920af6efa4a7a66919885c6477fbd8fdc7b9d/src/amd/compiler/aco_instruction_selection.cpp#L11087-L11096), [si_shader_llvm_ps.c](https://gitlab.freedesktop.org/mesa/mesa/-/blob/a82920af6efa4a7a66919885c6477fbd8fdc7b9d/src/gallium/drivers/radeonsi/si_shader_llvm_ps.c#L323-L333)) and AMD's [LLPC](https://github.com/GPUOpen-Drivers/llpc/blob/40cb8d95ad8d6f7f1652e3fd47d39667594cce08/lgc/lowering/FragmentColorExport.cpp#L157-L168); before GFX10 the alpha was channel 3. Channels 2 and 3, which none of these compilers enables, are ignored; green and blue are 0 and the alpha of an export without channel 1 is 1.0, the defaults of the other formats, which no source gives. Not measured on hardware, neither on gfx1013 nor on a PC RDNA 2 GPU; pinned in [Pixel32ArExport.cpp](../../../core/libs/prx/libSceAgcDriver/tests/execution/Pixel32ArExport.cpp)
 
 ### Functional
@@ -145,7 +145,7 @@
 - [FORCE_DEGAMMA on RGBA8_UINT](../../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) materializes a direct image sampled only by plain biased pixel accesses as RGBA8_SRGB when every paired sampler forces degamma. Measured on AMD integrated RDNA2 gfx1036 (driver 32.0.21036.18, Windows 11, wave32, IEEE off, DX10 clamp on, denormals preserved, round to nearest even, FP16 overflow off): a native `image_sample_b` DIM 2D on a 16x16x1 RGBA8_UINT 3D T# returns integer channels without the bit, but the same floating-point decoded values as RGBA8_UNORM with it, including point min/max reduction and the title's exact sampler. The metadata and image view must both become floating-point sRGB; ignoring the sampler bit or retaining a UINT heap is incorrect. The accepted case requires one slice, mip zero, normalized point sampling and uniform FORCE_DEGAMMA across its samplers. Other image accesses, mixed sampler states, indirect descriptors and other integer formats remain refused. Wave64, other filtering modes and the PS5's gfx1013 are not measured.
 - The [s_swappc_b64 fetch-shader call at vertex/local/tessellation/mesh entry](../../../core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp) assumes the called fetch-shader blob ends in `s_setpc_b64` to the link pair (the GNM fetch-shader shape shadPS4's `EmitFetch` copies); the blob is not part of the shader program, so its return closes the pairing by assumption and any in-program reader of the link pair throws. Attribute fetches are inlined from the request's parsed fetch-shader plan through the same `GetAttribute` path as the embedded fetch analyzer, honoring the buffer resource `dst_sel`; an attribute writing more than four registers, or an entry call without a parsed plan, throws. The vertex/instance offset accumulator inside the fetch blob is not replayed for external blobs.
 - The scalar aperture sources [src_shared_base, src_shared_limit, src_private_base, src_private_limit](../../../core/shader/recompiler/RdnaDecoder/src/RdnaInstructionDecoder.cpp) (core/shader/recompiler, scalar source codes 235 to 238) read as the 64-bit apertures the flat routing uses: a base is the aperture's high dword (`SharedApertureHigh`, `PrivateApertureHigh`) over a low dword of 0, a limit is the same high dword over 0xffffffff. A 64-bit read (`s_mov_b64`, a 64-bit VALU source, a wave64 lane mask) gets the whole value and a 32-bit read gets the low dword. This follows LLVM's `SIRegisterInfo.td` (aperture registers are 64-bit, their low 32 bits are always 0 for a base and -1 for a limit) and `getSegmentAperture` (`AMDGPULegalizerInfo.cpp`, `SIISelLowering.cpp`: a 64-bit read whose high half is the aperture), the KFD's `MAKE_LDS_APP_LIMIT` (limit = base | 0xffffffff) and the gfx1030 and gfx1035 readouts on [#1190](https://github.com/boykopovar/AnyPS5/pull/1190) and [#3803](https://github.com/boykopovar/AnyPS5/issues/3803). Not measured on a console: the PS5's high dwords are the inferred values of the flat aperture bases entry, and the wave is taken to be in 64-bit address mode (PTR32 clear)
-- [EmitSetAttribute](../../../core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp) (shader recompiler) - a primitive with a vertex whose clip position is (0,0,0,0) is clipped away through an extra `gl_ClipDistance` element on vertex and tessellation evaluation shaders, matching what a Radeon 610M (RDNA2, RADV) and llvmpipe rasterise; NVIDIA 615.78.08 without it draws the vertex at the viewport centre. Measured for triangles (strip and list) and for a point at (0,0,0,0), which the Radeon 610M does not draw, NVIDIA draws at the viewport centre and llvmpipe at the top-left corner; lines with such a vertex are clipped by the same element and were not measured. Mesh, geometry and the generated barycentric geometry stage do not write it, the generated rect-list and barycentric tessellation stages forward the vertex shader's array, a device without `shaderClipDistance` cannot compile vertex shaders (the compile throws), and guest clip distances are still refused by the pipeline validator. The PS5 rasteriser was not measured.
+- [EmitSetAttribute](../../../core/shader/recompiler/SpirvBackend/src/SpirvModuleEmitter.cpp) (shader recompiler) - a primitive with a vertex whose clip position is (0,0,0,0) is clipped away through an extra `gl_ClipDistance` element on vertex and tessellation evaluation shaders, matching what a Radeon 610M (RDNA2, RADV) and llvmpipe rasterise; NVIDIA 615.78.08 without it draws the vertex at the viewport centre. Measured for triangles (strip and list) and for a point at (0,0,0,0), which the Radeon 610M does not draw, NVIDIA draws at the viewport centre and llvmpipe at the top-left corner; lines with such a vertex are clipped by the same element and were not measured. Mesh, geometry and the generated barycentric geometry stage do not write it, the generated rect-list and barycentric tessellation stages forward the vertex shader's array, and a vertex or tessellation evaluation shader whose guest clip and cull distances already take all eight does not write it either, so NVIDIA 615.78.08 draws such a vertex at the viewport centre; a device without `shaderClipDistance` cannot compile vertex shaders that write it (the compile throws). The PS5 rasteriser was not measured.
 - [PERSP_SAMPLE and LINEAR_SAMPLE](../../../core/shader/recompiler/Translation/src/InstructionTranslator.cpp) (recompiler) hold the pixel-centre barycentrics, like the centroid pairs. With one sample this is the sample position ([Vulkan standard sample locations](https://docs.vulkan.org/spec/latest/chapters/primsrast.html#primsrast-multisampling): (0.5, 0.5)); with more samples the console's per-sample weights are not computed and were not measured.
 - [LINE_STIPPLE_TEX](../../../core/shader/recompiler/Translation/src/InstructionTranslator.cpp) (recompiler) reads as 0: line stipple is not emulated. RADV enables the input only as the required pixel input of a shader that reads no interpolants, because it initializes the fewest VGPRs ([Mesa](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/vulkan/radv_shader.c) `radv_compute_spi_ps_input`), and never reads it
 - [Static scalar calls](../../../core/shader/recompiler/ControlFlow/src/GraphBuilder.cpp) (`s_call_b64` and `s_swappc_b64`) require an ordinary even-aligned SGPR link pair (s0:s1 through s104:s105) and a later `s_setpc_b64` return reading that exact pair. A pair serves one call, or several calls whose regions are disjoint, each closed by the first return reading the pair after its target. Outside every region of its calls the pair is free to use wherever no return address can reach: a dataflow over the decoded instructions follows each dword of the pair from the calls that write it until a scalar load, SALU or VALU result is known to overwrite it (a VALU mask counts as one SGPR unless the request says wave64), and an instruction there that may read a dword still holding a return address throws; SOPC compares are exempt, as they only set SCC. Regions must nest or be disjoint; recursion, other shared links, escaping link values (including overlapping tuples and resource descriptors), invalid targets, interleaved regions, cross-region branches or fall-through, relative SGPR reads and dynamic `s_setpc_b64` in a program with calls throw. The analysis checks all decoded instructions, including unreachable code. Arbitrary indirect calls and general call frames are not implemented.