mirror of
https://github.com/TheTom/turboquant_plus.git
synced 2026-10-23 11:32:33 -05:00
TheTomandClaude Opus 4.7
2d8a2472f7
v0.1.3 ships no new functional axes — only the test coverage, fail-loud guards, and corpus-identity machinery needed to stop the v0.1.x patch-round-after-matrix-run cycle. Code fixes: - axes/gtm.py: tokenizer-failure fallback now RAISES instead of silently using whitespace (was producing wrong-unit scores with no detection). - axes/gtm.py: GTM score now normalizes by mean_cand_length instead of n_predict (fixes gemma-31B 2.87× retokenize inflation that re-clipped the score to 100 in v0.1.2). - cli.py: --measure-floor additionally asserts mean_prefix == mean_cand on ref-vs-ref greedy (KLD ref-vs-ref is bit-exact zero on Metal so the composite floor passes even when GTM is broken). - runner.py: _TOPP_RE regex now tolerates 'Same top p' (space) form (test surfaced that v0.1.2 only matched the dashed form). - runner.py: new corpus_identity / sidecar machinery; KLDResult includes corpus path/size/SHA; refuses to score against a base built from a different corpus. New tests (30 across 6 files; all pass in <0.05s, no subprocess): - test_strip_noise.py: 5 — pins runner._strip_noise (banner / backspace / block-char regressions from v0.1.1). - test_command_construction.py: 4 — pins --single-turn presence, --no-conversation absence, --kl-divergence-base, errors='replace'. - test_tokenize_to_ids.py: 5 — parser + raise-on-error contract. - test_kld_regex.py: 6 — perplexity output regexes incl. gemma Same-top-p missing case (this test caught the regex bug). - test_report_json_layout.py: 5 — pins schema + composite-is-scalar. - test_corpus_identity.py: 5 — sidecar machinery. - test_validation.py: +4 GTM assertions (full_match_rate < 1.0, mean_prefix > 0, ≥3 distinct refs, ratio == 1.0 on ref-vs-ref). Total: 14 → 44 unit tests + 1 strengthened integration test. Docs: - LIMITATIONS.md: NEW — documents the limitations we ship with (detokenize→retokenize gap, corpus-anchored KLD, provisional bands, single-platform tested, GTM/EOS conflation, v0.1.3 fail-loud removal). - README.md: GTM scoring formula updated; new test files listed; points at LIMITATIONS.md. - axes/gtm.py module docstring: refreshed to reflect v0.1.2 + v0.1.3. - CHANGELOG.md: v0.1.3 entry at top with the v0.1.2 matrix table and the file-by-file change summary. - report.py: schema bumped refract.report.v0.1.1 → v0.1.3. - __init__.py: __version__ = "0.1.3". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-Authored-By: tturney <tturney1@gmail.com>