mirror of
https://github.com/TheTom/turboquant_plus.git
synced 2026-10-22 02:53:13 -05:00
c819477692
Block_size=128 achieves 5.12x compression (vs 4.57x at 32) with
identical PPL across all tested Metal paths: asymmetric q8_0-K +
turbo{2,3}-V, symmetric turbo3/turbo3, and Boundary V (LA-V7).
Validated on 3 architectures (dense, Qwen, hybrid MoE), 3 context
lengths (512, 8K, 32K), and 2 hardware platforms (M5 Max, M2 Pro).
M2 Pro showed +3-7% decode improvement on tested 1.5B model.
No speed regression on M5 Max.
Also fixes pre-existing flaky d=192 turbo4 test threshold (0.6 -> 0.7).
Credit to @AmesianX whose block_size=256 implementation prompted
the investigation.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>