From 767c36ad12699f70d60bd6762d0e9acc8cd29fe3 Mon Sep 17 00:00:00 2001 From: Joe Date: Fri, 4 Sep 2026 17:46:00 -0400 Subject: [PATCH 1/3] docs: measured comparison of three CSI frame-selection gates, and the harness How a node rate-limits CSI callbacks decides whether cross-node fusion works at all, because it decides whether two nodes keep the SAME frames or merely the same NUMBER of frames. We measured three gates on a nine-node ESP32-C6 fleet as a 2x2 factorial, three Latin-square blocks, every arm verified live on every node. Selecting frames by 802.11 rx_seq roughly doubled cross-node pairing over the elapsed-time gate: 0.5605 against 0.2434, +31.7 pp, reproduced in all three blocks at ~70x the measured drift floor. Two results we did not want and are reporting anyway: - Mesh-time alignment -- a gate we built and expected to be the answer -- bought +0.5 pp, about one standard deviation of noise. Indistinguishable from doing nothing. - We predicted rx_seq selection would win most on node pairs spanning a broken sync domain. Per-pair yield says the opposite: never-synced nodes gained LEAST. So it is not a workaround for broken time sync, and repairing our mesh split would probably not have closed the gap. The document leads with the method rather than the numbers, because the numbers are one house. It records the three mistakes that made an earlier attempt unreadable -- comparing different node subsets, not verifying the gate was actually live, and using frame rate as the check that a gate is working when a bucket gate makes frame rate nearly insensitive to it -- and the null run that establishes the noise floor without which none of the contrasts can be judged. run_arms.py is included so a reader can measure their own environment instead of trusting ours. That is the point of the contribution; our figures are the worked example. Stated as a trade, not a win: pairing is bought with raw frame rate. On-device edge processing was unaffected here (separately rate-limited), so the cost fell on the uplink -- but whether that is acceptable depends on what consumes the stream. Co-Authored-By: claude-flow --- docs/csi-frame-selection-measured.md | 141 +++++++++++++++++++++ firmware/esp32-csi-node/run_arms.py | 182 +++++++++++++++++++++++++++ 2 files changed, 323 insertions(+) create mode 100644 docs/csi-frame-selection-measured.md create mode 100644 firmware/esp32-csi-node/run_arms.py diff --git a/docs/csi-frame-selection-measured.md b/docs/csi-frame-selection-measured.md new file mode 100644 index 000000000..4444777b6 --- /dev/null +++ b/docs/csi-frame-selection-measured.md @@ -0,0 +1,141 @@ +# Which CSI frames a node keeps decides whether cross-node fusion works + +A measured comparison of three frame-selection gates on a nine-node ESP32-C6 +fleet, and a method for repeating it in your own environment. + +**This is one house.** Nine boards, one floor plan, one RF environment, one +transmitter population. The numbers below are what we measured here; the part +worth reusing is the method, not our figures. `run_arms.py` is included so you +can measure yours. + +--- + +## The problem + +Cross-node fusion pairs frames by transmission identity: two nodes are fusing +usefully only when they hold measurements of *the same packet in the air*. The +metric is `paired_fraction` — of the transmissions the fleet reported, what +share were captured by two or more nodes. + +Every node also rate-limits which CSI callbacks it keeps, because processing +every callback at 100–500 Hz destabilises the WiFi ISR. **How that limiting is +done turns out to matter far more than we expected**, because it decides +whether two nodes keep the *same* frames or merely the *same number* of frames. + +Three mechanisms, all valid rate limits: + +| gate | rule | +|---|---| +| `elapsed` | at least 20 ms since **this node's** last accepted frame | +| `mesh` | first frame heard in each shared 20 ms window of a mesh-aligned clock | +| `seq` | keep frames where the 802.11 `rx_seq` is divisible by N | + +`elapsed` gives every node an independent phase. `mesh` makes nodes agree on +the *window*. `seq` makes them agree on the *frame*, using a number the +transmitter assigns — so it needs no clock, no leader and no synchronisation. + +## Method + +Four arms — the three gates plus `mesh`+`seq` stacked — run as a 2x2 factorial. +Three deliberate choices, each of which we got wrong first: + +**Whole fleet per arm, never a subset.** An earlier attempt compared four gated +nodes against four ungated ones. Nodes differ enormously in how much traffic +they hear, so that comparison could not be read at all. + +**Verify the gate is actually running.** Each switch is read back from every +node before sampling. "Config was stored" is not "gate is live", and an earlier +run could not distinguish a null result from a change that never applied. + +**Frame rate is NOT a valid check that a gate is working.** A bucket gate needs +only one survivor per window, so at ~14 candidate frames per bucket even a +perfect 1-in-4 gate moves frame rate about 2%. Use instead + + R_window = (delta early_drop + delta accepted) / delta accepted + +computed from **differenced** counters. Never divide lifetime totals: they +carry a large post-boot burst. One node read R = 1126 moments after a reboot, +26.5 an hour later, and 19.2 windowed. + +**Establish the noise floor first.** A null run — three hours, no changes — +gave drift of sd **0.45 pp** across 20-minute windows, about 2x the ±0.2 pp +binomial counting error. Without that number none of the results below can be +called significant. + +Arms are 20 minutes in Latin-square blocks (`U M S MS / S MS M U / MS U S M`), +so slow drift lands on every arm equally rather than on whichever ran first. + +## Results + +Three blocks, twelve arms, every arm verified live on all nine nodes. + +| arm | block 0 | block 1 | block 2 | mean | +|---|---|---|---|---| +| `elapsed` | 0.2881 | 0.2409 | 0.2012 | **0.2434** | +| `mesh` | 0.2692 | 0.2580 | 0.2167 | **0.2480** | +| `seq` (period 8) | 0.5819 | 0.5573 | 0.5422 | **0.5605** | +| `mesh`+`seq` | 0.5639 | 0.5623 | 0.5093 | 0.5452 | + +**`seq` roughly doubles pairing: +31.7 pp, 2.30x.** That gap is ~70x the +measured drift floor and reproduces in all three blocks; within-block contrasts +sit at +0.153 to +0.175 every time. + +**Mesh-time alignment bought essentially nothing here: +0.5 pp**, about one +standard deviation of the noise floor. We built that gate and expected it to be +the answer; it is indistinguishable from doing nothing. Stacking it on top of +`seq` is marginally *worse* than `seq` alone. + +### It is a trade, not a free win + +At period 8 the seq arms carried ~75–95k transmissions per 20 minutes against +~120–160k ungated. You are buying pairing with raw frame rate. Absolute paired +transmissions per minute still went **up** (2629 vs 2524), because the frames +it discards are disproportionately ones only one node would have kept — but +whether that trade is right depends on what else consumes the stream. + +One measurement that made the trade easier here: on-device edge processing was +**unaffected**. Frames actually processed stayed ~1900 per node per 4 minutes +in both modes, because that pipeline is separately rate-limited. The cost falls +on the uplink, not on local sensing. + +### A prediction we got wrong + +Our mesh had split into two leader domains with three nodes never synchronising +at all. We predicted `seq` would win *most* on node pairs spanning that split. +Per-pair yield says otherwise: + +| pair group | pairs | seq / mesh | +|---|---|---| +| same sync domain | 7 | 5.82x | +| spans the two domains | 8 | 6.24x | +| involves a never-synced node | 21 | **5.16x** | + +Cross-domain is marginally best, but the never-synced nodes are the *lowest* — +the opposite of the prediction — and within-group spread (3.6x–8.1x) dwarfs +every between-group difference. + +So `seq` is **not** a workaround for broken time sync. It wins roughly +uniformly, including where sync is healthy. The useful consequence is negative: +repairing our mesh split would probably not have closed the gap, and we would +have spent that effort for nothing. + +## What we suggest + +Not "use `seq`". **Make it selectable and measure your own environment.** The +right answer depends on your transmitter population, your node placement and +what consumes your CSI stream — none of which we can see from here. + +`run_arms.py` runs the factorial unattended: it applies each arm fleet-wide, +verifies every node adopted it, samples `/api/v1/fusion`, and writes one row +per arm. Run the null pass first; without your own drift floor the contrasts +cannot be judged. + +## Limits + +- One environment, nine nodes. Treat the numbers as an existence proof, not a + constant. +- Only period 8 was characterised. 4 and 16 are unmeasured, so the + yield/pairing curve has a single point on it. +- `paired_fraction` measures **frame pairing**, which is a precondition for + cross-receiver work. Nothing here measures sensing accuracy, and none of it + should be read as an accuracy claim. diff --git a/firmware/esp32-csi-node/run_arms.py b/firmware/esp32-csi-node/run_arms.py new file mode 100644 index 000000000..91a2284ab --- /dev/null +++ b/firmware/esp32-csi-node/run_arms.py @@ -0,0 +1,182 @@ +#!/usr/bin/env python3 +"""Run the gate factorial: U / M / S / MS, Latin-square blocked. + +Arms (NVS gate_mode, applied live -- no reboot): + 0 U elapsed-time gate, as upstream ships it + 1 M mesh-aligned bucket gate (ours, 4b54ef8e) + 2 S rx_seq selection + elapsed underneath + 3 MS both stacked + +Why it is built this way, and what each guard is for: + +* Every arm is applied to the WHOLE fleet, never to a subset. The stopped + 2026-09-04 run compared four gated nodes against four different control + nodes; nodes differ, and that comparison could not be read. + +* After every switch the runner reads back `active.gate_mode` from EVERY node + and refuses to sample unless all nine agree. "Config was stored" is not + "gate is running" -- that gap is exactly what the previous run could not + distinguish from a null result. + +* Arms are short and blocked in a Latin square rather than run in long + sequential batches, so slow drift lands on all arms equally instead of being + aliased onto whichever arm ran first. Measured drift over 20 min windows is + sd 0.0045; counting noise is +/-0.002. + +* Metrics come from differenced cumulative counters, so the server is never + restarted and the fleet stays in production throughout. + +Output: arms_