mirror of
https://github.com/TheTom/turboquant_plus.git
synced 2026-10-21 10:40:06 -05:00
TheTomandClaude Opus 4.7
75ad729303
Three findings from the Mac mini PyPI install walkthrough (2026-05-03): 1. `--prompts` was required even though wheel ships them — defaulted to bundled refract/prompts/v0.1.jsonl via importlib.resources (v0.3.2.2) 2. macOS system python3 is 3.9, mlx-lm needs >=3.10 — documented in QUICKSTART Step 0 with brew install hint 3. MLX run_kld returned mean_kld=NaN on Qwen2.5-1.5B-Instruct-8bit + q8_0/q8_0 KV. Root cause: log(softmax + 1e-12) underflows on sharply-peaked logits from already-8bit weight models. Rewritten in terms of log-softmax (logits - logsumexp) which is stable. Non-finite per-position KL values now filtered before aggregation; mean computed across valid positions only. If every position is non-finite the backend raises BackendCapabilityError with an actionable hint instead of silently returning NaN. Test mock updated for the new mx symbols (logsumexp, exp, isfinite, where, zeros_like). 416 tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-Authored-By: tturney1@gmail.com <tturney1@gmail.com>