turboquant_plus

starred/turboquant_plus

0 Fork 0

mirror of https://github.com/TheTom/turboquant_plus.git synced 2026-10-22 02:53:13 -05:00

Files

T

History

TheTomandClaude Opus 4.7 d231c4c469 docs: cross-engine REFRACT bench paper on MI300X (Qwen3.6-35B-A3B)

Documents the 3-engine apples-to-apples cross-engine benchmark:
- BF16 baseline (load, PPL, prefill, decode, KV size) across vLLM, SGLang, llama.cpp
- REFRACT --full 4-axis fidelity scoring (gtm, kld, rniah, plad) on each engine's native 8-bit KV
- Headline finding: same fp8_e4m3 dtype string produces 1.8x different KL drift between vLLM and SGLang on the same model
- 8 engine-side bugs encountered and worked around (vLLM max_num_seqs, prompt_logprobs cap, GPU mem carryover; SGLang aiter.dtypes, AITER fp8 prefill broken, KV dtype fixed at server launch)
- Full reproducibility scripts and REFRACT framework changes documented

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: tturney1@gmail.com <tturney1@gmail.com>

2026-05-02 07:19:14 -05:00

asymmetric-kv-compression.md

Remove nihilistau reference from asymmetric K/V paper

2026-04-06 22:23:31 -05:00

attn-rotation-and-ppl-artifact.md

Add attn-rotation + PPL artifact paper

2026-04-29 18:15:39 -05:00

block-size-experiment.md

Fix addendum accuracy: dates, attributions, numbers

2026-03-31 11:49:10 -05:00

cross-engine-mi300x.md

docs: cross-engine REFRACT bench paper on MI300X (Qwen3.6-35B-A3B)