mirror of
https://github.com/TheTom/turboquant_plus.git
synced 2026-10-22 11:02:42 -05:00
TheTomandClaude Opus 4.7
0ca1439b3c
Adds the empirical study covering the broader 35-hypothesis investigation spawned by the EDEN/Mitzenmacher/Portnoy critique (#87, #89). Companion to eden-optimal-s-revisit.md. Headline result: a K-cache centroid table that improves per-vector reconstruction MSE by 1–13% across five model families (Qwen3-0.6B, Llama-3.2-1B, Mistral-7B, Phi-3-mini, Gemma-2-2b) causes 70–90% mean KL@D regressions and 50–60% catastrophic-rate failures at the model output. This is a sign inversion, not a small mismatch: methods that are strictly better under MSE are systematically worse under KL@D in deployment. The paper is structured in three acts: - Act 1: K cache is sub-Gaussian post-WHT (H8); V is Gaussian - Act 2: per-prompt fitting fails (PDS), ensemble (ens_4way) reaches an irreducible ~8% catastrophic floor at scale (N=100) - Act 3: the universal-K cross-model counterexample (F3) demonstrates the MSE→KL inversion at the cleanest possible level Includes a 2-D toy attention illustration of the softmax bucket-flip mechanism, an explicit decision boundary for when MSE proxies quality (linear consumers) vs when it doesn't (attention softmax), an ablation summary distinguishing first-order from second-order factors, operational guidance for practitioners, and a "why this wasn't caught earlier" diagnosis of the methodology blind spots. Companion paper handles the algorithmic claim head-on. This paper is the broader evaluation-methodology critique. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>