mirror of
https://github.com/TheTom/turboquant_plus.git
synced 2026-10-22 02:53:13 -05:00
TheTomandClaude Opus 4.7
b331b8ca83
The 4 vLLM "skeleton" tests in test_backends.py asserted that
VLLMBackend methods raise NotImplementedError; they break in v0.3.2.1
where the backend is a working implementation. Replace with:
vLLM (5 tests):
- _kv_str_to_vllm_dtype known mappings (f16 -> auto, q8_0 -> fp8_e4m3)
- TurboQuant preset round-trips (turbo3, turbo4, q8_0/turbo4)
- BackendCapabilityError on unmappable ctk/ctv
- run_completion signature has the kw-only contract args
- all 5 backend methods are present and callable
SGLang (5 tests, new):
- get_backend("sglang") returns SGLangBackend
- _validate_kv_str accepts f16/bf16/q8_0
- TurboQuant rejected loudly (no SGLang TurboQuant path)
- run_kld without REFRACT_SGLANG_REF/CAND_URL raises with actionable msg
- all 5 backend methods are present and callable
Also: MLX backend uses mlx_lm.generate.maybe_quantize_kv_cache when
mlx_lm.models.cache no longer exposes it (0.31+); fall back via
getattr(cache_mod, ...) so tests that mock the old surface still work.
Refract test count: 399 -> 404 (+10 new, -4 stale, 1 mlx fix).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: tturney1@gmail.com <tturney1@gmail.com>