3 people c46f6b9c43 docs(papers): add block-selector sparse attention WIP log ...

Sprint journal documenting the in-progress block-selector sparse
attention work on Apple Silicon: companion to sparse-v-dequant.md
(post-softmax) covering the pre-attention K-skip lever.

Includes the preliminary local M5 Max bench grid (1.16-1.73× decode
across B=4-8 × ctx=16K-48K on Qwen2.5-14B-1M-4bit), the Apple-Silicon-
specific findings (MLX_SDPA_BLOCKS heuristic mis-tune for the float-mask
kernel variant, pass-2 reducer modulo bug, MetalAllocator non-determinism
cliff), the failed-paths catalogue (custom Metal kernel 14× slower at B=8,
compose-gather 13.7× slower, bool mask 25% regression), and the
per-family port pattern across 10 model families.

Framed as a sprint journal — the work is not yet merged to upstream
alpha (PRs ekryski/mlx#34, ekryski/mlx-swift#36, ekryski/mlx-swift-lm#224,
ekryski/mlx-swift-lm#225 all open). Numbers are local M5 Max only;
recommendations are current best guess rather than settled defaults.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: tturney@psyguard.ai

2026-05-18 10:26:56 -05:00

..

papers

docs(papers): add block-selector sparse attention WIP log

2026-05-18 10:26:56 -05:00

threshold-ablation-logs

docs: complete cross-model rerun with ISWA fix + Gemma outlier log

2026-03-28 19:15:10 -05:00

asymmetric-kv-discovery.md

docs: fix asymmetric-kv-discovery accuracy

2026-03-29 14:43:02 -05:00

attention-gated-optimizations.md

docs: f16 sparse V experiment — not worth it, reverted

2026-03-27 13:13:35 -05:00

context-scaling-deep-dive.md

docs: extended context scaling verified — 0.99x q8_0 through 32K context

2026-03-25 20:21:22 -05:00

cross-model-validation.md

docs: add TODO section to cross-model validation for deferred tests

2026-03-28 20:54:00 -05:00

decode-speed-hardware-analysis.md

Experiment #25 result: context-adaptive dispatch negative on 1.5B

2026-03-29 00:00:05 -05:00

decode-speed-status.md

docs: Mario PDF test confirms decode scaling — 0.66x q8_0 at 48K context

2026-03-25 22:08:23 -05:00

experiment-context-scaling-fix.md

docs: group-32 rotation failed (PPL 7.06)

2026-03-25 19:31:11 -05:00

experiment-layer-adaptive-extended.md

docs: layer-adaptive mode 2 verified 2K-32K — q8_0 quality at 3.5x compression

2026-03-26 00:17:35 -05:00

experiment-layer-adaptive.md

docs: layer-adaptive results — last 8 layers at q8_0 matches q8_0 quality at 3.5x compression

2026-03-25 18:44:37 -05:00

getting-started.md

docs: update all references — PR #45 merged to main

2026-04-03 11:27:00 -05:00

hardware-comparison-matrix.md

docs: update comparison matrix with 10 community testers from X post

2026-03-26 12:03:57 -05:00

kl-divergence-results.md

Add Sjoerd L40S validation, KLD context-length stability data

2026-04-01 15:09:08 -05:00

long-context-sparse-v-validation.md

docs: add q4_0 baseline to 50-chunk 32K validation

2026-03-27 21:35:48 -05:00

m2-ppl-investigation.md

docs: asymmetric K/V recommendations + investigation writeup

2026-03-29 14:36:45 -05:00

mlx-swift-pr14-gemma4-benchmark-brief.md

docs: incorporate Eric's feedback on turbo decode and Python parity

2026-04-10 14:33:40 -05:00

pre-rotate-queries-investigation.md

docs: move engineering docs here from llama.cpp fork

2026-03-25 16:16:58 -05:00

pre-rotate-queries-plan.md

docs: move engineering docs here from llama.cpp fork

2026-03-25 16:16:58 -05:00

quality-benchmarks.md

docs: add q4_0 baseline to 50-chunk 32K validation

2026-03-27 21:35:48 -05:00

smem-dequant-experiment.md

docs: add QC precompute result (#16) + ILP root cause analysis

2026-03-28 18:25:37 -05:00

sparse-v-upstream-validation.md

docs: turbo4 full characterization — PPL, prefill, decode, NIAH, KLD, PDF

2026-03-28 14:19:35 -05:00

speed-experiments.md

docs: block-32 breakthrough — q8_0 parity at 4.6x compression

2026-03-25 17:34:27 -05:00

test-suite-definition.md

docs: final polish + test suite formalized

2026-03-27 23:48:59 -05:00

threshold-ablation.md

docs: threshold ablation sweep (τ=1e-4 to 1e-8) for sparse V paper

2026-03-27 07:07:07 -05:00

turbo4-investigation.md

docs: turbo4 full characterization — PPL, prefill, decode, NIAH, KLD, PDF

2026-03-28 14:19:35 -05:00

turbo4-next-steps.md

docs: turbo4 full characterization — PPL, prefill, decode, NIAH, KLD, PDF

2026-03-28 14:19:35 -05:00

turbo4-profiling-plan.md

docs: turbo4 full characterization — PPL, prefill, decode, NIAH, KLD, PDF

2026-03-28 14:19:35 -05:00

turbo-speed-investigation.md

docs: update all docs with q8_0 parity results (2747 tok/s, 4.6x compression)

2026-03-25 18:01:14 -05:00

turboquant-plus-experiments.md

docs: comprehensive experiment status update — all 2026-03-26 results

2026-03-26 18:25:59 -05:00

turboquant-recommendations.md

docs: Sjoerd asymmetric correction + MTMD tip + community update

2026-04-02 10:07:30 -05:00

upstream-pr-plan.md

fix: address Codex + Gemini review findings

2026-03-26 09:34:29 -05:00

weight-compression-results.md

docs: add WaveboSF RTX 4090 + RTX 5090 benchmark data

2026-04-04 09:57:44 -05:00

windows-rdna4-setup.md

Add AMD RDNA 4 benchmarks and Windows HIP setup guide

2026-03-30 21:15:34 -05:00