mirror of
https://github.com/storytold/effectcraft.git
synced 2026-10-19 02:09:58 -05:00
Runs every scenario N times (default 1, unchanged). Every run is still appended to results.jsonl; with N > 1 one "kind": "summary" row per key follows, whose metrics are the medians of the successful runs (user_s and sys_s included, byte/instruction counts rounded) and whose summary holds median, p90 (nearest rank, = max for N < 10), min and max. A failed run marks the summary failed so the gate refuses it. --compare uses the latest row per key; the table labels each side "median of N" or "1 run" and warns when they differ, and --fail-over refuses such keys. --repeat with --with is rejected. docs/testing.md documents the format.