mirror of
https://github.com/TheTom/turboquant_plus.git
synced 2026-10-19 01:30:50 -05:00
661bc7fb47
Complete turbo4 4-bit PolarQuant benchmark suite: - PPL: +0.23% vs q8_0 (best after q8_0, better than q4_0) - Prefill: 101-105% of q8_0 across 2K-32K - Decode: 79.87 tok/s (93% q8_0, faster than turbo3) - NIAH: 7/9 (matches q8_0) - KLD: 0.0096 (40% lower than turbo3, matches q4_0 same-top-p) - PDF 24K: 63.7 tok/s decode (20% faster than turbo3) Also includes turbo4-resurrection.md paper documenting the full journey from PPL 679 to beating q4_0 quality. Co-Authored-By: tturney@psyguard.ai Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
37 lines
1.3 KiB
Plaintext
37 lines
1.3 KiB
Plaintext
Turbo4 SET_ROWS Fix Results
|
|
===========================
|
|
|
|
Date: 2026-03-28
|
|
Branch: experiment/turbo4-quality-investigation
|
|
Hardware: Apple M5 Max 128GB
|
|
Model: Qwen3.5-35B-A3B Q8_0
|
|
Corpus: wikitext-2 wiki.test.raw, c=512, 8 chunks
|
|
|
|
Fix: Dedicated kernel_set_rows_turbo4 with correct 3-bit packing + QJL signs.
|
|
Replaces shared kernel_set_rows_turbo which used turbo3's 2+1 bit scheme.
|
|
|
|
BEFORE FIX (shared SET_ROWS, turbo3 packing applied to turbo4):
|
|
turbo4 PPL = 679.27 (completely broken)
|
|
|
|
AFTER FIX (dedicated SET_ROWS with correct 3-bit packing + QJL):
|
|
turbo4 PPL = 6.1894 +/- 0.331
|
|
|
|
COMPARISON:
|
|
q8_0: 6.1109 (baseline)
|
|
turbo3: 6.1756 (+1.06% vs q8_0)
|
|
turbo4: 6.1894 (+1.28% vs q8_0, +0.22% vs turbo3)
|
|
|
|
OBSERVATION:
|
|
turbo4 (4.25 bits) is WORSE than turbo3 (3.5 bits) by 0.22%.
|
|
This is unexpected — more bits should mean better quality.
|
|
Possible cause: QJL correction adding noise instead of signal.
|
|
Consistent with scos-lab finding (issue #45): QJL variance
|
|
amplified by softmax, MSE beats Prod for attention.
|
|
|
|
NEXT STEPS:
|
|
1. Test turbo4 with QJL disabled (zero out signs) — isolate QJL effect
|
|
2. If QJL hurts, turbo4 becomes "turbo3 in 128-block format"
|
|
3. Run decode speed comparison
|
|
|
|
turbo3 sanity: 6.1756 (unchanged, no regression from turbo4 fix)
|