AnyPS5/docs/dev/HW_ORACLE.md

Hardware oracle

tools/hw-oracle/ runs a few lines of RDNA assembly on a local AMD GPU, one input row per lane, and returns what the hardware computed. Use it to settle instruction semantics that the ISA documentation, LLVM or Mesa leave open, then pin the measured rows as expected values in tests/execution/.

The PS5 GPU is gfx1013. Other RDNA2 GPUs (gfx103x) share the VALU, SALU, DS, MUBUF and MIMG encodings, apart from ops that exist only on gfx10.1 (v_mad_legacy_f32, v_mac_legacy_f32, v_mul_lo_i32, s_dcache_discard, ...). RDNA3 also runs the tool, but differs from gfx1013 in more places. Say which GPU you measured on in the PR.

Requirements

Linux

• Linux with the amdgpu driver and read/write access to /dev/kfd (usually the render group). Root is not needed.
• The ROCm HSA runtime with its headers: libhsa-runtime-dev (Debian, Ubuntu), rocm-runtime-devel (Fedora), hsa-rocr (Arch), or AMD's ROCm packages. A runtime under ROCM_PATH or /opt/rocm is used first. The rest of ROCm is not needed.
• A C compiler, and clang + ld.lld with the AMDGPU target (llc --version lists amdgcn). If they aren't in PATH, they are taken from $ROCM_PATH/llvm/bin.

The first run builds oracle.c into ~/.cache/anyps5-hw-oracle/ (HW_ORACLE_CACHE overrides it). The target is read from the GPU (HW_ORACLE_TARGET overrides it).

Windows

• AMD's Windows driver, which installs the HIP runtime (amdhip64_6.dll) and the code object manager (amd_comgr_2.dll) in System32. ROCm, a C compiler, clang and ld.lld are not needed.
• Python 3 from python.org or the Microsoft Store.

On Windows oracle_hip.py replaces oracle.c: comgr assembles and links the kernel, and HIP runs it. The CLI, the Python interface and the results are the same as on Linux. HW_ORACLE_RUNTIME picks the runtime (hip on Windows, hsa elsewhere). HW_ORACLE_HIP and HW_ORACLE_COMGR name other DLLs. ROCR_VISIBLE_DEVICES selects the GPU, through HIP_VISIBLE_DEVICES, unless that variable is already set.

Kernel

template.s wraps the body under test. Per lane:

• v4-v7 hold the row's 4 input dwords. v10-v25 start at 0 and are stored as the result (16 dwords).
• Leave v0 (lane id), v1 (input offset), v2 (output offset) and s[4:7] (input and output addresses) unchanged.
• 4 KiB of LDS, or the group segment size --lds (Python lds) gives in bytes, up to 64 KiB. Rows run in workgroups of up to 1024 lanes, padded to whole waves with zero rows.
• Bytes passed as extra follow the rows of each dispatch, at s[4:5] + 16 * rows: buffer contents, texels, etc. Build buffer and image descriptors in SGPRs from that address. A kernel that needs more than 4 input dwords per lane takes the rest from there too, for example v_lshlrev_b32 v40, 3, v0 / v_add_nc_u32 v40, 16 * rows, v40 / global_load_dwordx2 v[8:9], v40, s[4:5] for two more dwords per lane, with rows the padded count.

Every run must explicitly set all floating-point controls, including runs of integer instructions. The CLI flags use hyphens; Python keywords use underscores. Record these settings alongside the GPU and measured results.

Python keyword	Values
ieee	0 or 1: IEEE mode disabled or enabled
dx10_clamp	0 or 1: DX10 clamp disabled or enabled
denorm32, denorm16	0: flush input/output; 1: preserve input, flush output; 2: flush input, preserve output; 3: preserve both
round32, round16	0: nearest even; 1: toward +infinity; 2: toward -infinity; 3: toward zero
fp16_overflow	0: overflow to infinity; 1: clamp computed overflow to the largest finite value (an infinite source operand still produces infinity, as does v_div_fixup_f16 for a zero denominator or an infinite numerator, but not for an infinite quotient; v_rcp_f16, v_rsq_f16 and v_log_f16 of zero are clamped)

denorm16 and round16 control both f16 and f64. These values follow the LLVM AMDGPU kernel descriptor documentation. Float atomics need coarse: on fine-grained system memory they do nothing.

Use

cat > body.s <<'EOF'
  v_add_f16 v10, v4, v5
EOF
printf '0x3c00 0x3c00 0 0\n0x7e01 0x3c00 0 0\n' > rows.txt
python3 tools/hw-oracle/hw_oracle.py body.s rows.txt --outs 1 \
  --ieee 0 --dx10-clamp 1 --denorm32 0 --denorm16 3 \
  --round32 0 --round16 0 --fp16-overflow 0

prints one line per row with the output dwords in hex. From Python:

import sys
sys.path.insert(0, "tools/hw-oracle")
from hw_oracle import run

rows = [(a, b, 0, 0) for a in values for b in values]
results = run(body, rows, wave64=True, ieee=0, dx10_clamp=1,
              denorm32=0, denorm16=3, round32=0, round16=0, fp16_overflow=0)

Usual workflow: run the instruction on rows that separate the candidate models (edge values, NaN payloads, rounding ties), write a model that matches every row, implement it, and put measured rows in the execution test.

Limits

Only compute kernels. Pixel-shader ops that read their inputs from LDS also work: v_interp_* reads the attribute parameters from LDS at M0, so writing them there by hand is enough. Behaviour that depends on the graphics pipeline (parameter cache, exports, rasterization, SPI_* state) can't be measured with it.

Argument handling can be tested without a GPU:

python3 tools/hw-oracle/test_hw_oracle.py