7fa354b7e8
docs: Mario PDF test confirms decode scaling — 0.66x q8_0 at 48K context
...
Real-world test with 70-page PDF (48K tokens):
- Prefill: 0.99x (parity)
- Decode: 0.66x (34% slower)
- KV cache: 280 vs 680 MiB (2.4x savings)
Decode ratio degrades ~4% per doubling:
12 tok: 0.88x → 8K: 0.76x → 48K: 0.66x
Root cause: centroid LUT constant cache thrashing (verified).
Experiment branch is net negative — main branch is optimal.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-Authored-By: tturney@psyguard.ai
2026-03-25 22:08:23 -05:00
8c9e7a1514
docs: decode speed status report — 0.84x q8_0, experiment branch was net negative
...
Main branch at 65.8 decode tok/s (0.84x q8_0). Experiment branch
with custom GGML_OP_TURBO_WHT dropped to 61.5 (-6.5%, net negative).
Root cause: centroid LUT is 100% of the gap (verified with speed ceiling test).
All attempted LUT optimizations either marginal or regressive.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-Authored-By: tturney@psyguard.ai
2026-03-25 21:34:05 -05:00
b4fa8b6810
docs: TurboQuant+ experiments master document — status, findings, dead ends
...
Covers all 9 experiments: 2 merged, 2 active, 1 blocked, 1 dead, 3 not started.
Key learnings: layer sensitivity, dequant bottleneck, byte-level GPU optimization.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-Authored-By: tturney@psyguard.ai
2026-03-25 20:49:49 -05:00
9be9ca18c1
docs: extended context scaling verified — 0.99x q8_0 through 32K context
...
Tested 2K/4K/8K/16K/32K: ratio stays 0.987x-0.995x (flat).
PPL: 5.471 (32-chunk, +1.1% of q8_0).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-Authored-By: tturney@psyguard.ai
2026-03-25 20:21:22 -05:00
034d4c911e
docs: context scaling FIXED — optimized dequant gives flat 98% of q8_0
...
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-Authored-By: tturney@psyguard.ai
2026-03-25 20:13:40 -05:00
e5fcbe34db
docs: group-32 rotation failed (PPL 7.06)
...
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-Authored-By: tturney@psyguard.ai
2026-03-25 19:31:11 -05:00
0255ba2d7f
docs: context scaling fix investigation — group-32 rotation approach
...
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-Authored-By: tturney@psyguard.ai
2026-03-25 19:22:06 -05:00
6406922ace
docs: update all docs with q8_0 parity results (2747 tok/s, 4.6x compression)
...
- README: top-of-tree table, speed journey, updated roadmap and contributing
- quality-benchmarks.md: verified PPL numbers with error bars (5.460 ± 0.141)
- turbo-speed-investigation.md: full 4-step optimization breakdown
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-Authored-By: tturney@psyguard.ai
2026-03-25 18:01:14 -05:00
d50ef98f5b
docs: block-32 breakthrough — q8_0 parity at 4.6x compression
...
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-Authored-By: tturney@psyguard.ai
2026-03-25 17:34:27 -05:00
1bd4fcb181
docs: experiment 3 results — graph WHT gives 2095 tok/s (0.78x q8_0)
...
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-Authored-By: tturney@psyguard.ai
2026-03-25 17:23:25 -05:00
d06ff09f1c
docs: experiment 1 results — half4 WHT gives 1411 tok/s (+31%)
...
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-Authored-By: tturney@psyguard.ai
2026-03-25 17:00:58 -05:00
445a3dc4c7
docs: move engineering docs here from llama.cpp fork
...
These belong in our repo, not in a fork of someone else's project.
Added:
- docs/quality-benchmarks.md — perplexity validation, bisection log, top-of-tree table
- docs/turbo-speed-investigation.md — Metal gotchas, fp16 WHT, optimization history
- docs/pre-rotate-queries-investigation.md — WHT/RoPE commutativity issue
- docs/pre-rotate-queries-plan.md — original optimization plan
- scripts/turbo-quality-gate.sh — pre-push perplexity check
README links updated to point locally.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-Authored-By: tturney@psyguard.ai
2026-03-25 16:16:58 -05:00
