d38742e3f9
docs: add TODO section to cross-model validation for deferred tests
...
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
Co-Authored-By: tturney@psyguard.ai
2026-03-28 20:54:00 -05:00
af7bdf3d0c
docs: complete cross-model rerun with ISWA fix + Gemma outlier log
...
Gemma 2: 3/3 NIAH all types (ISWA fix works), turbo4 slightly worse
than turbo3 (outlier, logged for investigation).
Llama 70B: confirmed 3/3 NIAH, PPL matches prior run.
Qwen MoE: sanity passed, known 8K/100% NIAH weakness confirmed.
Corrected Gemma PPL from 1-chunk estimate to proper 8-chunk values.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-03-28 19:15:10 -05:00
5eade98036
docs: full ISWA investigation story in cross-model-validation.md
...
Added complete investigation narrative: wrong initial hypothesis (hd256),
GGUF parsing, q4_0 comparison, FA elimination, 5 build_attn overloads,
root cause (missing WHT in ISWA overload), 16-line fix, PPL 13.7T → 3.80.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-03-28 18:57:26 -05:00
5bdd8b3737
docs: move ISWA discovery to cross-model-validation.md where it belongs
...
Resurrection doc is about turbo4 journey. ISWA discovery is a cross-model
testing finding. Added "Why Cross-Model Testing Matters" section to
cross-model-validation.md.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-03-28 18:53:51 -05:00
3ce8c388e7
docs: cross-model complete — 7 models, Gemma FIXED, Llama 70B added
...
Gemma 2 ISWA fix: PPL 13.7T → 3.80 (missing WHT rotation in ISWA build_attn)
Llama 70B: turbo4 +8.3% (Q4_K_M weight+KV quant stacking)
All 7 models across 5 families tested. turbo4 consistently beats turbo3.
ISWA fix pushed to llama-cpp-turboquant fork.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-03-28 18:40:54 -05:00
5ec5939750
docs: Mixtral skipped (bad GGUF), Llama 70B running
...
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-03-28 18:04:33 -05:00
e0aca7f521
docs: cross-model validation — 5 models tested, hd256 broken
...
hd128 models all work: Qwen MoE/Dense, Phi-4, Mistral Small.
turbo4 consistently closer to q8_0 than turbo3.
Gemma 2 27B (hd256): PPL catastrophe (13 trillion). Known limitation.
Mixtral 8x7B and Llama 70B pending.
Quick bench parsing fixes committed.
Co-Authored-By: tturney@psyguard.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-03-28 17:59:58 -05:00
