mirror of
https://github.com/TheTom/turboquant_plus.git
synced 2026-10-22 02:53:13 -05:00
c46f6b9c43
Sprint journal documenting the in-progress block-selector sparse attention work on Apple Silicon: companion to sparse-v-dequant.md (post-softmax) covering the pre-attention K-skip lever. Includes the preliminary local M5 Max bench grid (1.16-1.73× decode across B=4-8 × ctx=16K-48K on Qwen2.5-14B-1M-4bit), the Apple-Silicon- specific findings (MLX_SDPA_BLOCKS heuristic mis-tune for the float-mask kernel variant, pass-2 reducer modulo bug, MetalAllocator non-determinism cliff), the failed-paths catalogue (custom Metal kernel 14× slower at B=8, compose-gather 13.7× slower, bool mask 25% regression), and the per-family port pattern across 10 model families. Framed as a sprint journal — the work is not yet merged to upstream alpha (PRs ekryski/mlx#34, ekryski/mlx-swift#36, ekryski/mlx-swift-lm#224, ekryski/mlx-swift-lm#225 all open). Numbers are local M5 Max only; recommendations are current best guess rather than settled defaults. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-Authored-By: tturney@psyguard.ai