mirror of
https://github.com/TheTom/turboquant_plus.git
synced 2026-10-19 01:30:50 -05:00
TheTomandClaude Opus 4.7
75ad729303
Three findings from the Mac mini PyPI install walkthrough (2026-05-03): 1. `--prompts` was required even though wheel ships them — defaulted to bundled refract/prompts/v0.1.jsonl via importlib.resources (v0.3.2.2) 2. macOS system python3 is 3.9, mlx-lm needs >=3.10 — documented in QUICKSTART Step 0 with brew install hint 3. MLX run_kld returned mean_kld=NaN on Qwen2.5-1.5B-Instruct-8bit + q8_0/q8_0 KV. Root cause: log(softmax + 1e-12) underflows on sharply-peaked logits from already-8bit weight models. Rewritten in terms of log-softmax (logits - logsumexp) which is stable. Non-finite per-position KL values now filtered before aggregation; mean computed across valid positions only. If every position is non-finite the backend raises BackendCapabilityError with an actionable hint instead of silently returning NaN. Test mock updated for the new mx symbols (logsumexp, exp, isfinite, where, zeros_like). 416 tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-Authored-By: tturney1@gmail.com <tturney1@gmail.com>
20 lines
699 B
Python
20 lines
699 B
Python
"""REFRACT v0.1 — REFerence-anchored Robust Acid-test for Compressed Transformers.
|
|
|
|
A benchmaxx-resistant alternative to corpus PPL for evaluating KV-cache
|
|
quantization quality. Replaces "lower PPL = better" with a composite of:
|
|
|
|
- Axis A (GTM): Greedy Trajectory Match — do quantized + reference produce
|
|
the same generated tokens?
|
|
- Axis B (KLD@D): KL Divergence vs the fp16-KV reference distribution
|
|
(v0.1 uses corpus KLD via llama-perplexity as a proxy for trajectory KLD).
|
|
|
|
See:
|
|
- refract/README.md for usage.
|
|
- docs/papers/attn-rotation-and-ppl-artifact.md for the motivating paper.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
__version__ = "0.3.2.3"
|
|
__all__ = ["__version__"]
|