TheTomandClaude Opus 4.7 19ec3f4164 refract: batch PLAD ref/cand by KV config for memory-pressured backends ...

PLAD axis previously interleaved ref/cand per (prompt, perturbation) cell.
On vLLM with hybrid Qwen3.6 (~120GB per LLM, two won't fit 192GB MI300X),
each ref/cand swap triggers eviction-and-reload of ~30-90s. With 30 prompts
× 4 cells (anchor + 3 perturbations) = 240 swaps, the axis runs ~3 hours
just for eviction overhead.

Now plans all cells upfront, runs all ref-side completions, then all
cand-side completions, then computes drift. Two LLM loads total.
No effect on backends that can hold both configs in cache cheaply
(llama.cpp, MLX).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: tturney1@gmail.com <tturney1@gmail.com>

2026-05-01 20:47:11 -05:00

..

__init__.py

REFRACT v0.3.2: alpha release — 4 axes, 4 subcommands, 3 backends, 9 docs

2026-04-30 15:13:04 -05:00

gtm.py

REFRACT v0.1.3: defensive patch round (tests + fail-loud + honest docs)

2026-04-29 20:36:25 -05:00

kld.py

REFRACT v0.3.2: alpha release — 4 axes, 4 subcommands, 3 backends, 9 docs

2026-04-30 15:13:04 -05:00

plad.py

refract: batch PLAD ref/cand by KV config for memory-pressured backends

2026-05-01 20:47:11 -05:00

rniah.py

REFRACT v0.3.2: alpha release — 4 axes, 4 subcommands, 3 backends, 9 docs

2026-04-30 15:13:04 -05:00

trajectory.py

refract: batch trajectory ref/cand by KV config to support evict-on-swap

2026-05-01 15:56:32 -05:00