mirror of
https://github.com/TheTom/turboquant_plus.git
synced 2026-10-19 01:30:50 -05:00
docs: add vLLM PR #38479 as independent validation (asymmetric + boundary papers)
vLLM TurboQuant integration explicitly cites turboquant_plus for boundary layers and asymmetric KV. Community ablations on A100/H100 confirm our findings. Co-Authored-By: tturney@psyguard.ai Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
7001bde828
commit
77b2e1b0c2
2 files changed
+8
No files matched your search
@@ -265,6 +265,13 @@ These findings have been independently confirmed by multiple researchers:
|
||||
- APEX Mini (12.2 GB) + TurboQuant = 35B MoE at 8K context on a 16GB consumer GPU
|
||||
- Independently confirms boundary layer sensitivity for weights: "first and last 5 transformer layers are 10x more sensitive than the middle," consistent with our Boundary V finding for KV cache
|
||||
|
||||
**vLLM TurboQuant PR** — [vllm-project/vllm#38479](https://github.com/vllm-project/vllm/pull/38479) (2026-04-03):
|
||||
- TurboQuant KV cache compression being integrated into vLLM (major production inference framework)
|
||||
- PR author (@vibhavagarwal5) explicitly cites turboquant_plus for boundary layer implementation and asymmetric KV cache
|
||||
- @Alberto-Codes independently diagnosed the 0% gsm8k quality failure as the symmetric K/V compression problem documented in our asymmetric paper
|
||||
- @varjoranta validated turbo4-resurrection findings via ablation on A100 with Qwen3-8B, confirming our paper's predictions about compression quality tradeoffs
|
||||
- Community testing across H100, A100, A4000, DGX Spark (GB10) confirms our findings that symmetric turbo3 on sensitive models produces garbage, asymmetric fixes it
|
||||
|
||||
---
|
||||
|
||||
## 8. Limitations
|
||||
|
||||
@@ -371,6 +371,7 @@ The boundary layer sensitivity finding has been independently confirmed:
|
||||
- **@HyperionMS2040** — RTX 3090 (2026-03-30): Qwen3.5-4B asymmetric data shows clean quality ladder with no catastrophic failure when K precision is maintained, consistent with boundary layer sensitivity.
|
||||
- **@Corianas_** — Independent validation of Boundary V on NanoGPT (X collaborator, 2026-03-29).
|
||||
- **@mudler (Ettore Di Giacinto)** — [APEX](https://github.com/mudler/apex-quant) MoE weight quantization, [LocalAI](https://github.com/mudler/LocalAI) (44.7k stars) (2026-04-01): Independently discovered that "first and last 5 transformer layers are 10x more sensitive than the middle" for weight quantization on Qwen3.5-35B-A3B across 25+ experiments. Same boundary sensitivity principle as Boundary V, applied to weights instead of KV cache. APEX assigns higher precision to edge layers, lower to middle layers. Directly validates the boundary layer hypothesis from the weight quantization direction.
|
||||
- **vLLM TurboQuant PR** — [vllm-project/vllm#38479](https://github.com/vllm-project/vllm/pull/38479) (2026-04-03): PR author (@vibhavagarwal5) implementing TQ_boundary_layers in vLLM, explicitly "inspired from https://github.com/TheTom/turboquant_plus". Boundary layer protection being adopted by a major production inference framework.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in new issue
Block a user