From 05fde211c01ac3d4587a0b971f24fb2ded42d658 Mon Sep 17 00:00:00 2001 From: TheTom Date: Wed, 1 Apr 2026 12:17:43 -0500 Subject: [PATCH] Add head_dim=64 limitation warning to getting-started guide Models with head_dim=64 (e.g., GPT-OSS-120B) may crash or produce degraded output with turbo compression. WHT requires sufficient dimensionality for CLT convergence. Reported by NigelTufnel12345. Co-Authored-By: Claude Opus 4.6 (1M context) Co-Authored-By: tturney@psyguard.ai --- docs/getting-started.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/getting-started.md b/docs/getting-started.md index 58e318f..77fce5e 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -54,6 +54,8 @@ Run a model with TurboQuant+ KV cache compression: Asymmetric: full precision K, compressed V. Safe on all tested models including sensitive ones like Qwen2.5. +> **Known limitation:** Models with `head_dim=64` (e.g., GPT-OSS-120B) may crash or produce degraded output with turbo V compression. The WHT kernel requires sufficient dimensionality for CLT convergence, and d=64 is at the lower boundary. If you hit issues on a head_dim=64 model, fall back to `-ctk q8_0 -ctv q8_0` (no turbo compression). The paper validated at head_dim=128 and 256. + ### More compression (works on most models) ```bash