mirror of
https://github.com/TheTom/turboquant_plus.git
synced 2026-10-22 02:53:13 -05:00
TheTomandClaude Opus 4.7
f71ec25f5c
Adds two independent third-party confirmations to the appendices: asymmetric-kv-compression.md: @jagmarques tested asymmetric K/V on Mistral-7B with E8 lattice VQ (a different quantization family from TurboQuant) on Cerebrium A10/A100. Confirms asymmetric advantage scales with context length: at 1742-tok prefix K3V2 +0.82% vs K2V2 +0.91%; at 3544-tok prefix K3V2 +0.35% vs K2V2 +2.26% (6.4× improvement). Clean GQA-amplification explanation: 8 query heads per K head means K quantization error is effectively 8× worse than V error in output impact. Cross-method validation that the asymmetric conclusion is not TurboQuant-specific. turbo4-resurrection.md: @unamedkr / quantumaikr (quant.cpp) ported the TurboQuant paper end-to-end in single-header C and found the QJL stage contributed byte-identical zero to attention scores regardless of input. Disabling QJL produced bit-for-bit identical perplexity. Theoretical explanation: the √(π/2)/m constant in the unbiased estimator is derived for Gaussian projection rows; on Rademacher (±1) rows the per-key contribution gets dwarfed by residual-norm scaling. Independently took the freed bits and reinvested in 4-bit codebook (8 → 16 centroids), same path as turbo4 resurrection. Llama 3.2 3B: turbo_kv_4b PPL 14.28 (+5.3% vs FP32), beating llama.cpp q4_0 KV (~14.99, +10.6%). Targets embedded/edge (iOS, Android, WASM, microcontrollers). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>