vLLM TurboQuant integration explicitly cites turboquant_plus for
boundary layers and asymmetric KV. Community ablations on A100/H100
confirm our findings.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add compression reference table and config warning to getting-started.md
- Add independent validation from sztlink, Madreag, and mudler (APEX/LocalAI) across asymmetric, layer-aware, MoE, and sparse V papers
- Update sparse-v-dequant.md with Madreag 4-GPU cross-validation
- Minor README wording fix (branch -> repository)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: tturney@psyguard.ai
New paper: q8_0/turbo2 with Boundary V on Qwen3.5-35B MoE achieves
7.53x V compression with PPL within 1% of q8_0 at 512c/8K/32K.
Quality matches or beats q8_0/turbo3 at every tested length. 32K
decode ~2-3% faster than turbo3-V (n=2, M5 Max). Scoped to one
tested MoE model. Includes decode investigation negative results
(20+ approaches exhausted) and dense-vs-MoE bottleneck analysis.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: tturney@psyguard.ai
Corrections from verification review:
- Fix HyperionMS2040 dates (03-30 not 03-31) and numbers (Qwen2.5
symmetric was catastrophic PPL 3472, not +3.6%)
- Fix Aaryan-Kapoor attribution (was TheTom's words, not his)
- Remove seanrasch block_size=128 claim (he reported a crash, not
confirmation). Replace with HyperionMS2040 PR #32 fix
- Fix Arclabs001 framing (QJL works with independent rotations,
not "QJL hurts")
- Reword elusznik (scoped out QJL for PR size, not as validation)
- Fix Madreag date (03-26), AmesianX dates (03-30), dusterbloom
dates (03-30)
- Mark AMD HIP as author's own testing, not independent validation
- Mark treblewoe and Corianas_ as X collaborators
- Mark spiritbuun prefill claim as X collaborator with approximate date
- Add jesusmb1995 Vulkan mixed K/V data to asymmetric paper
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Each paper now documents who confirmed the findings, on what hardware,
with what data, and when. Validations span Metal (M5 Max), CUDA (RTX
3090, 4090, 5090, Blackwell), and AMD HIP (RX 9070 XT).
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Tested f16 vs q8_0 boundary layers at 2 and 4 layer widths.
Finding: width matters more than precision. Wide q8_0 (first4+last4)
beats narrow f16 (first2+last2) on both phi-4 and Qwen3.5-27B.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New: docs/papers/layer-aware-v-compression.md — full research writeup
- Updated: docs/turboquant-recommendations.md — Boundary V as advanced experimental option
- New: proof/niah/niah_boundary_v/ — NIAH retrieval evidence
Boundary V protects first 2 + last 2 layers with q8_0-V while compressing
remaining layers with turbo2-V. Quality between turbo3 and turbo2, effective
compression between the two. Validated across 4 models on Metal.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: tturney@psyguard.ai
