These belong in our repo, not in a fork of someone else's project.
Added:
- docs/quality-benchmarks.md — perplexity validation, bisection log, top-of-tree table
- docs/turbo-speed-investigation.md — Metal gotchas, fp16 WHT, optimization history
- docs/pre-rotate-queries-investigation.md — WHT/RoPE commutativity issue
- docs/pre-rotate-queries-plan.md — original optimization plan
- scripts/turbo-quality-gate.sh — pre-push perplexity check
README links updated to point locally.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: tturney@psyguard.ai
Previous speed claims (51-77 tok/s, 91-97% of q8_0) were INVALID.
Pre-rotate-queries never executed due to GQA head layout mismatch.
Those benchmarks measured speed of producing garbage output.
Correct numbers:
- Quality: PPL 6.194 (+1.4% of q8_0) ✅
- Speed: ~10.7 tok/s (8× gap from inverse rotation in dequant)
- Compression: 4.6×
Speed optimization path: pre-rotate-queries with per-head GQA reshape.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pre-rotate-queries optimization delivers:
- MoE 35B-A3B: 51.4 tok/s (60% of q8_0) at 4.9× compression
- Qwopus 27B: 14.6 tok/s (83% of q8_0) at 4.9× compression
The dense 27B model is now practical for long-context inference.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previous benchmarks were CPU due to #include bug in Metal shader.
Real Metal GPU numbers:
- MoE: 10.7 tok/s gen (8× slower than q8_0, was 35×)
- Qwopus: 5.3 tok/s gen (3.3× slower than q8_0, was 13.5×)
- Prompt: 60-67 tok/s MoE, 25-30 tok/s Qwopus
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Single coherent flow: results → getting started → architecture → roadmap
- Removed duplicate sections and local paths
- Added Contributing section
- Added proper paper links (arXiv URLs)
- Copyright notice in License section
- Changed "we" to "I" throughout
- Added benchmark_results.md and run_benchmark.py to project structure
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Added full benchmark tables for both models (MoE + Dense) with all
4 cache types. Added compression quality table from Python prototype.
Added key validation section (rotation Gaussianization proof).
Updated roadmap with current status and issue links.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Removed CLAUDE.md from tracking (private dev config)
- Removed .claude/ agents from tracking
- Added CLAUDE.md and .claude/ to .gitignore
- Removed development workflow section from README
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Critical experiment: real Qwen3-1.7B K cache has kurtosis=900.
After norm extraction + random rotation: kurtosis=2.9 (Gaussian=3.0).
Std matches 1/sqrt(d) to 6 decimal places. Ratio = 1.000 exactly.
Paper's core mathematical claim is validated on real data.
Gaussian codebook is correct — no per-layer calibration needed (#20 closed).
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fixed HuggingFace DynamicCache extraction (transformers 5.x API).
First real tensor validation on Qwen3-1.7B complete.
Key findings:
- Real KV tensors have kurtosis=918 (K) vs Gaussian=3.0
- Norm extraction critical: norms range 6.8-603.8
- 3.5-bit outlier: cosine 0.945, ratio 8.0× on real tensors
- 4-bit uniform: cosine 0.975 on real tensors
- Attention cosines 0.64-0.82 (lower than synthetic due to heavy tails)
Results posted to issue #18.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add input validation to rotation.py: d>=1, pow2 checks on hadamard
and FWHT, with ValueError messages (#12)
- Add README.md with quick start, results, architecture, roadmap
- Add benchmarks/validate_real_model.py for Phase A validation (#18)
- Close#9, #16 (superseded by #18)
5 new tests for validation guards. 137 total tests passing.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Splits channels into outlier (higher bits) and normal (lower bits):
- 2.5-bit: 64/128 at 3b + 64/128 at 2b
- 3.5-bit: 64/128 at 4b + 64/128 at 3b
This matches Prince Canuma's MLX approach for achieving 4.9× compression.
8 tests, 91% coverage on outlier.py.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CRITICAL BUG (found by roast review): PolarQuant codebook was calibrated
for unit-norm vectors only. Real KV cache tensors have norms ~10-50,
causing every coordinate to saturate to extreme centroids.
Fix (per paper page 5): extract L2 norms before quantization, normalize
to unit vectors, quantize, then rescale on dequantization. Norms stored
as float32 metadata alongside indices.
API change: quantize() now returns (indices, norms), dequantize() takes
(indices, norms). All downstream code updated (TurboQuant, KVCache, tests).
Added test_non_unit_norm_vectors at scales [0.01, 1.0, 10.0, 50.0, 100.0].
Zero vector now correctly reconstructs to zero (norm=0).
128 tests, 98% coverage, all green.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TurboQuant (Algorithm 2): PolarQuant(b-1) + QJL(1) = b bits total.
19 tests, 93% coverage.
KVCacheCompressor: TurboQuant for K cache (IP), TurboQuantMSE for V cache (MSE).
5 tests, 100% coverage.
Utils: bit packing, memory footprint calculation.
9 tests, 98% coverage.
Distortion validation suite: MSE bounds, IP bounds, monotonicity.
20 tests verifying against paper Table 2.
Full suite: 127 tests, 98% total coverage.
Codex review on TurboQuant:
- Noted that double-side IP estimation is biased (documented, tested)
- True bit rate is b + 32/d (norm overhead documented)
- QJL correction factor is inside QJL.dequantize (verified)
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements dense Haar-distributed rotation (QR decomposition) and fast
structured rotation (Walsh-Hadamard + random signs).
Dense rotation is the primary path for PolarQuant — verified orthogonal,
det=+1, norm-preserving, IP-preserving, and post-rotation distribution
matches paper's Beta(d/2, d/2) → Gaussian(0, 1/d).
29 tests, 100% coverage on rotation.py.
Reviews completed:
- Codex review on tests: fixed Haar distribution test methodology,
switched FWHT reference to scipy, clarified non-pow2 behavior
- Codex review on implementation: documented slogdet cost, float64
requirement, and non-pow2 limitations
- Roast review: found non-pow2 transpose bug (#11), missing input
validation (#12), single D-H-D round vs paper's 3-round (#13)
Known limitations (tracked in issues):
- Fast rotation broken for non-pow2 dims (#11)
- No input validation (#12)
- Single D-H-D round (#13)
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Set up bare project infrastructure:
- .gitignore, pyproject.toml, setup.py
- GitHub Actions CI with Python 3.10/3.11/3.12 matrix
- CI enforces >95% coverage when tests exist, skips gracefully when empty
- CLAUDE.md with full development workflow rules
- PLAN.md with phased implementation plan
- Empty turboquant/, tests/, benchmarks/ packages
No algorithm code — scaffold only.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
