mirror of
https://github.com/TheTom/turboquant_plus.git
synced 2026-10-22 02:53:13 -05:00
TheTomandClaude Opus 4.7
c034e2f6af
Issues found by running the freshly-pushed v0.3.2 through a clean clone
on toms-mac-mini.local:
- Version banner showed v0.3.1; refract/__init__.py was missing the
bump. Now reports v0.3.2 correctly.
- pyproject.toml `requires-python` was >=3.10 but mini ships 3.9.6;
REFRACT works fine on 3.9 thanks to `from __future__ import
annotations`. Lowered the floor.
- pyproject.toml lacked optional deps for the MLX and vLLM backends.
Added `refract-mlx` and `refract-vllm` extras with the right
minimum versions (mlx-lm >= 0.31 since 0.29 doesn't recognise the
qwen3_5 model type).
- pyproject.toml had no `[project.scripts]` entry point. Added
`refract = "refract.cli:main"` so installs get a proper command
instead of `python3 -m refract.cli` only.
- selftest's thinking-mode probe printed "✓ no markers detected"
even when generation failed. Gated the probe on gen_ok so the
happy-path output only appears when the model actually generated.
- selftest description leaned on the "30 minutes of a real run"
framing, scaring readers who don't realize default mode is
5-7 min. Reworded.
- QUICKSTART now leads with a runtime expectations table:
selftest: ~1s static, ~30s with --model
score (default): ~5-7 min (Trajectory + KLD)
score --full: ~25-30 min (+ R-NIAH + PLAD)
repeatability: 4× default
Default is quick. Most users should never wait 30 minutes.
- QUICKSTART preflight example now shows BOTH .gguf and MLX
directory paths so MLX users see they're supported.
- HTML report hardware card showed "137.4 GB" RAM on a 128 GB Mac
(used /1e9 SI gigabytes). Switched to /1024**3 binary GiB so
output matches Apple's marketing convention (128 GB exact).
Also fixed model file size to use binary GiB.
- Regenerated examples/*.html with the GiB fix and live hardware
metadata.
Tests: 82/82 pass. No semantic changes to scoring, only metadata
display + setup ergonomics.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
20 lines
697 B
Python
20 lines
697 B
Python
"""REFRACT v0.1 — REFerence-anchored Robust Acid-test for Compressed Transformers.
|
|
|
|
A benchmaxx-resistant alternative to corpus PPL for evaluating KV-cache
|
|
quantization quality. Replaces "lower PPL = better" with a composite of:
|
|
|
|
- Axis A (GTM): Greedy Trajectory Match — do quantized + reference produce
|
|
the same generated tokens?
|
|
- Axis B (KLD@D): KL Divergence vs the fp16-KV reference distribution
|
|
(v0.1 uses corpus KLD via llama-perplexity as a proxy for trajectory KLD).
|
|
|
|
See:
|
|
- refract/README.md for usage.
|
|
- docs/papers/attn-rotation-and-ppl-artifact.md for the motivating paper.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
__version__ = "0.3.2"
|
|
__all__ = ["__version__"]
|