3 people 7fa354b7e8 docs: Mario PDF test confirms decode scaling — 0.66x q8_0 at 48K context ...

Real-world test with 70-page PDF (48K tokens):
- Prefill: 0.99x (parity)
- Decode: 0.66x (34% slower)
- KV cache: 280 vs 680 MiB (2.4x savings)

Decode ratio degrades ~4% per doubling:
  12 tok: 0.88x → 8K: 0.76x → 48K: 0.66x

Root cause: centroid LUT constant cache thrashing (verified).
Experiment branch is net negative — main branch is optimal.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: tturney@psyguard.ai

2026-03-25 22:08:23 -05:00

..

context-scaling-deep-dive.md

docs: extended context scaling verified — 0.99x q8_0 through 32K context

2026-03-25 20:21:22 -05:00

decode-speed-status.md

docs: Mario PDF test confirms decode scaling — 0.66x q8_0 at 48K context

2026-03-25 22:08:23 -05:00

experiment-context-scaling-fix.md

docs: group-32 rotation failed (PPL 7.06)

2026-03-25 19:31:11 -05:00

pre-rotate-queries-investigation.md

docs: move engineering docs here from llama.cpp fork

2026-03-25 16:16:58 -05:00

pre-rotate-queries-plan.md

docs: move engineering docs here from llama.cpp fork

2026-03-25 16:16:58 -05:00

quality-benchmarks.md

docs: update all docs with q8_0 parity results (2747 tok/s, 4.6x compression)

2026-03-25 18:01:14 -05:00

speed-experiments.md

docs: block-32 breakthrough — q8_0 parity at 4.6x compression

2026-03-25 17:34:27 -05:00

turbo-speed-investigation.md

docs: update all docs with q8_0 parity results (2747 tok/s, 4.6x compression)

2026-03-25 18:01:14 -05:00

turboquant-plus-experiments.md

docs: TurboQuant+ experiments master document — status, findings, dead ends

2026-03-25 20:49:49 -05:00