3 people deb2d8409d experiment: temporal decay prototype — 3→2 requant VIABLE ...

Synthetic (d=128, 1000 vectors):
  turbo3 cos=0.983, decay cos=0.940 (>0.80 threshold)
  Decay within 5% of direct 2-bit — double-quant adds minimal error
  Inner product error 1.76x worse (acceptable for old tokens)

Real Qwen3-1.7B KV tensors:
  3-bit cos=0.985, decay cos=0.949 (>0.75 threshold)
  Layer 0 (early): 0.971 — most decay-tolerant
  Layer 2 (mid): 0.927 — worst case, still viable

Memory savings: 30-34% at 32K-256K context
  256K: 1120 MB → 742 MB (378 MB saved)

Approach: dequant turbo3 → find nearest 2-bit centroid → recompute
norm correction → repack. Codex confirmed zero-signs approach is
broken (biased remap), proper requant is required.

Next: Metal implementation in llama_kv_cache maintenance path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: tturney@psyguard.ai

2026-03-26 17:29:58 -05:00

..

__init__.py

infra: project scaffold — CI, pyproject, CLAUDE.md, PLAN.md #1

2026-03-24 20:43:16 -05:00

benchmark_llama.sh

feat: README overhaul, Apache 2.0 license, benchmark script #22

2026-03-25 00:09:59 -05:00

benchmark_results.md

feat: benchmark results — turbo3/turbo4 vs q8_0/q4_0 on M5 Max #22

2026-03-25 01:06:59 -05:00

demo.py

feat: add demo benchmark script for quick validation

2026-03-24 21:05:53 -05:00

run_benchmark.py

feat: benchmark results — turbo3/turbo4 vs q8_0/q4_0 on M5 Max #22

2026-03-25 01:06:59 -05:00

temporal_decay_prototype.py

experiment: temporal decay prototype — 3→2 requant VIABLE

2026-03-26 17:29:58 -05:00

test_outlier_comparison.py

feat: outlier comparison benchmark — matches Prince Canuma's ratios

2026-03-24 21:21:53 -05:00

test_with_llama.py

feat: integration test at real Qwen 3.5 KV cache dimensions #16

2026-03-24 21:15:14 -05:00

validate_real_model.py

feat: validate rotation Gaussianization on real KV tensors #18 #20

2026-03-24 21:36:34 -05:00