mirror of
https://github.com/TheTom/turboquant_plus.git
synced 2026-10-23 11:32:33 -05:00
Remove nihilistau reference from asymmetric K/V paper
Co-Authored-By: tturney@psyguard.ai Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
aefb66ce86
commit
7c716e0156
1 file changed
-6
@@ -344,12 +344,6 @@ These findings have been independently confirmed by multiple researchers:
|
||||
- Originally hit corrupt multilingual output on Madreag's fork with symmetric turbo3. Madreag's fork needs to pick up the same fix separately
|
||||
- First end-user confirmation of the head_dim=256 fix on Blackwell (SM 120) consumer hardware
|
||||
|
||||
**@nihilistau** — [Vilenkin basis spectral analysis, Phi-2 + turbo3](https://github.com/ggml-org/llama.cpp/discussions/20969#discussioncomment-16442465) (2026-04-03):
|
||||
- Independent harmonic analysis of K and V cache vectors in the Vilenkin (p-adic/mixed-radix) basis suggests a spectral explanation for why asymmetric compression works
|
||||
- K vectors appear to concentrate energy in a small number of universal spectral indices (6 indices at 100% universality across all positions). V vectors appear to spread energy diffusely across the entire spectrum (max 20% universality, no dominant modes)
|
||||
- K and V appear to occupy disjoint spectral bands with zero overlap, suggesting they compress independently without interference
|
||||
- If confirmed across architectures, this would provide a mathematical basis for the empirical observation that V compression is free while K precision is critical
|
||||
- **Caveat:** tested on Phi-2 only. Needs validation on Qwen, Llama, and Gemma architectures before drawing general conclusions. The spectral structure may be model-dependent
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in new issue
Block a user