fix: harden CSI timing and baseline reliability

This commit is contained in:
ruv committed 2026-09-06 11:38:30 -04:00
1 parent 48c6244d37
commit a2c662f9f7
17 files changed
+690 -48

No files matched your search

@@ -0,0 +1,151 @@
# ADR 356: Rate truthful CSI and reliability gated room adaptation
Status: Accepted
Date: 2026-09-05
## Context
RuView mixed three different notions of time. ESP32 firmware requested a packet
rate, the server observed a delivered rate, and several server paths still used
fixed 2 Hz, 10 Hz, or 20 Hz assumptions. One feature named
`dominant_freq_hz` was not a temporal frequency at all. It was the index of the
strongest subcarrier multiplied by 0.05. A second defect compared a newly
inserted frame with itself, forcing that motion delta to zero.
These errors can move a spectral peak into the breathing or heartbeat band and
produce plausible but false values. They also make models sensitive to WiFi
traffic and processor scheduling rather than physical motion.
ESP-IDF documents that `wifi_csi_info_t.first_word_invalid` marks the first four
CSI bytes invalid because of a hardware limitation. It also documents that the
CSI receive callback runs in the WiFi task and should hand work to a lower
priority task rather than perform lengthy processing. RuView serialized the
invalid bytes unchanged and fed the original buffer to edge processing.
Recent work suggests four useful directions with different maturity:
1. OpenCSI represents each link as a dimensionless residual against its own
quiet period temporal standard deviation and exposes baseline maturity. Its
reported transfer results are limited to binary presence.
2. MORIC derives motion centered delay and Doppler views to reduce dependence
on static environment specific CSI structure.
3. Parameter efficient CSI crowd counting combines self supervised pretraining,
small room adapters, and a stateful counting machine.
4. WiFi JEPA masks complete radio links and predicts their latent embeddings,
using real and ray traced CSI for pose representation learning.
The papers report promising external results. None is measured RuView evidence,
and none authorizes deployment without leakage free occupied and empty held out
evaluation.
## Decision
### Firmware ingestion
1. When ESP-IDF marks the first CSI word invalid, zero the first four serialized
CSI bytes while retaining the fixed packet shape.
2. Set ADR-018 byte 19 bit 5 to record that sanitation occurred.
3. Feed edge DSP from the sanitized serialized payload, never the original
invalid driver buffer.
4. Keep callback work bounded to sanitation, serialization, and the existing
queue handoff. Do not add transforms to the WiFi task.
### Server time model
1. Estimate delivered CSI rate from accepted monotonic frame timestamps after
five samples. Use 20 Hz only during startup.
2. Cap the physical input rate at the firmware contract of 50 Hz so a burst or
timestamp artifact cannot retune downstream filters above their supported
range.
3. Reconfigure the vital detector only after a 20 percent rate change. Clear
temporal and smoothing state whenever its clock changes. Samples calculated
with different clocks must never be blended.
4. Compute temporal dominant frequency from the recent mean amplitude series
with demeaning, a Hann window, and bounded direct Fourier bins from 0.1 Hz to
the smaller of 4 Hz or 45 percent of sample rate.
5. Return zero for short, flat, nonfinite, or spectrally ambiguous windows.
6. Compare the current frame with the prior frame, not itself, for motion delta.
7. Export zero sample rate until the estimate is mature. Never present a startup
fallback as a measured node property.
### Model compatibility and room reliability
1. Preserve the historical fifth input feature for version 1 adaptive models at
the model boundary. Version 2 and later models consume the true temporal
frequency. This prevents an existing model from silently receiving a
different input distribution after a server update.
2. A background model is unreliable until it contains at least ten finite,
nonnegative residual references. An unreliable model cannot suppress a
runtime change and exposes no conformance score.
3. Expose background maturity, reliability, and a robust residual Z score based
on the median and median absolute deviation.
4. Keep the persisted empty home prior negative only. It cannot authorize
calibrated presence, person count, pose, or numeric vitals.
### Model roadmap
The following work remains proposal only:
1. Evaluate an OpenCSI style per link quiet residual normalization for binary
presence without replacing absolute magnitude features used for motion.
2. Evaluate delay and Doppler views after phase sanitation for activity.
3. Pretrain a frozen multilink encoder and adapt small room specific modules for
counting.
4. Evaluate link masked self supervision for pose only when at least three
spatially distinct links and synchronized positive pose labels exist.
Every candidate uses immutable manifests and splits by room, subject,
recording, and time. It is compared with the frozen baseline under identical
seeds. Counting includes confusion matrices and mean absolute error. Pose
includes PCK at 20 centimetres, MPJPE, and the mean pose baseline. A shuffled
label control must fail.
Promotion requires at least 10 percent held out lift, less than 10 milliseconds
added p95 inference latency, no more than 20 percent peak memory overhead, zero
leakage or secret findings, frozen anchor retention, and no regression in empty
rejection or occupied recall. Promotion remains an explicit maintainer action.
## Consequences
The firmware no longer preserves a known invalid prefix as sensor evidence. The
server uses an observed physical clock for spectral features and vital filters.
Existing version 1 adaptive models remain compatible. Sparse background samples
fail open to observation and fail closed for suppression.
The direct Fourier estimator is intentionally small and bounded to 128 temporal
samples. It is portable and deterministic, but it is not a replacement for a
phase sanitized delay and Doppler pipeline.
The largest remaining uncertainty is human recall. Empty home evidence can
measure false presence but cannot show whether weak occupied signals are being
suppressed. The fix path is a consented, leakage separated occupied holdout with
multiple spatial links and synchronized reference labels.
## Research basis
1. Espressif ESP-IDF WiFi CSI programming guide:
https://docs.espressif.com/projects/esp-idf/en/v5.5.4/esp32/api-guides/wifi.html
2. Espressif ESP CSI reference implementation:
https://github.com/espressif/esp-csi
3. OpenCSI:
https://arxiv.org/abs/2607.26665
4. MORIC:
https://arxiv.org/abs/2506.12997
5. Parameter efficient CSI crowd counting:
https://arxiv.org/abs/2601.02203
6. WiFi JEPA:
https://arxiv.org/abs/2607.11064
## Acceptance test
Run the firmware serializer host tests and the focused hardware, signal, and
sensing server Rust suites. The deterministic serializer test must prove the
first four bytes are zero only when ESP-IDF marks them invalid and bit 5 must
round trip through the Rust parser. The temporal estimator must recover a
1.2 Hz synthetic signal within 0.21 Hz, abstain on short and flat inputs, and
remain below 10 milliseconds p95 on the target Mac. Existing version 1 model
input must retain the legacy proxy. A background model with nine references
must never report a match or suppress a runtime change. Physical firmware is
not qualified until an S3 and C6 boot and stream the sanitized flag without
watchdog, memory, or transport regressions during a five minute burn in.
+1
View File
@@ -97,6 +97,7 @@ Statuses: **Proposed** (under discussion), **Accepted** (approved and/or impleme
| [ADR-350](ADR-350-ruvector-predictive-memory-and-ruvllm-boundary.md) | RuVector predictive memory and RuVLLM authority boundary | Proposed |
| [ADR-354](ADR-354-forecast-governance-enforcement-in-ruforecast-submodule.md) | fal.ai forecast governance and spend enforcement now real, in the vendored RuForecast submodule | Accepted |
| [ADR-355](ADR-355-local-empty-room-bootstrap-prior.md) | Local empty room bootstrap prior | Accepted |
| [ADR-356](ADR-356-rate-truthful-csi-and-reliability-gated-adaptation.md) | Rate truthful CSI and reliability gated room adaptation | Accepted |
### Platform and UI
+8 -1
View File
@@ -257,10 +257,17 @@ Offset Size Field
12 4 Sequence number (LE u32)
16 1 RSSI (i8)
17 1 Noise floor (i8)
18 2 Reserved
18 1 PPDU type (ADR-110; zero when tagging is disabled)
19 1 Flags: bit0=40 MHz, bit2=STBC, bit4=sync valid,
bit5=invalid first CSI word zeroed by firmware
20 N*2 I/Q pairs (n_antennas * n_subcarriers * 2 bytes)
```
When ESP-IDF marks `first_word_invalid`, the firmware preserves packet geometry,
zeros the first four CSI bytes, and sets byte 19 bit 5. The same sanitized bytes
feed the edge DSP path. This prevents a documented hardware artifact from being
learned as motion while keeping older readers wire compatible.
### Vitals Packet (32 bytes)
```
+14 -3
View File
@@ -263,8 +263,16 @@ size_t csi_serialize_frame(const wifi_csi_info_t *info, uint8_t *buf, size_t buf
buf[19] = 0;
#endif
/* I/Q data */
/* I/Q data. ESP-IDF documents that the first four CSI bytes are invalid
* when first_word_invalid is set. Never let those hardware artifacts feed
* either the host or edge DSP path. Preserve the fixed ADR-018 geometry by
* zeroing rather than removing bytes, and mark the sanitation in bit 5. */
memcpy(&buf[CSI_HEADER_SIZE], info->buf, iq_len);
if (info->first_word_invalid && iq_len > 0) {
size_t invalid_len = iq_len < 4 ? iq_len : 4;
memset(&buf[CSI_HEADER_SIZE], 0, invalid_len);
buf[19] |= CSI_FLAG_FIRST_WORD_SANITIZED;
}
return frame_size;
}
@@ -330,13 +338,16 @@ static void wifi_csi_callback(void *ctx, wifi_csi_info_t *info)
* while the on-device Tier 1/2 pipeline receives a uniform, sustainable
* stream. Enqueuing every burst frame overloaded the unicore C6 DSP and
* turned 30-40 callback pps into an irregular approximately 8 Hz subset. */
if (info->buf && info->len > 0) {
if (frame_len > CSI_HEADER_SIZE) {
if (s_next_edge_enqueue_us == 0) {
s_next_edge_enqueue_us = now_us;
}
if (now_us >= s_next_edge_enqueue_us) {
(void)edge_enqueue_csi((const uint8_t *)info->buf, (uint16_t)info->len,
/* Reuse the sanitized ADR-018 payload. Feeding info->buf here
* would reintroduce first_word_invalid artifacts on device. */
(void)edge_enqueue_csi(&frame_buf[CSI_HEADER_SIZE],
(uint16_t)(frame_len - CSI_HEADER_SIZE),
(int8_t)info->rx_ctrl.rssi, info->rx_ctrl.channel);
/* Preserve the configured sample clock instead of resetting it to
@@ -17,6 +17,9 @@
/** ADR-018 header size in bytes. */
#define CSI_HEADER_SIZE 20
/** ADR-018 byte 19: ESP-IDF reported and firmware sanitized invalid CSI prefix. */
#define CSI_FLAG_FIRST_WORD_SANITIZED (1U << 5)
/** Maximum frame buffer size (header + 4 antennas * 256 subcarriers * 2 bytes). */
#define CSI_MAX_FRAME_SIZE (CSI_HEADER_SIZE + 4 * 256 * 2)
+16 -5
View File
@@ -45,10 +45,20 @@ FUZZ_JOBS ?= 1
.PHONY: all clean run_serialize run_edge run_nvs run_all test_adr110 run_adr110 \
test_vitals run_vitals test_mmwave_detect run_mmwave_detect host_tests \
test_thermal run_thermal
test_thermal run_thermal test_csi_sanitize run_csi_sanitize
all: fuzz_serialize fuzz_edge fuzz_nvs test_adr110 test_vitals test_mmwave_detect \
test_thermal
test_thermal test_csi_sanitize
# --- ESP-IDF invalid first-word sanitation ---
# Links the production serializer. Function sections and dead stripping keep
# unrelated device-only callback code out of this host executable.
test_csi_sanitize: test_csi_sanitize.c $(MAIN_DIR)/csi_collector.c $(STUBS_SRC)
cc -std=c99 -Wall -Wextra -Istubs -I$(MAIN_DIR) -ffunction-sections \
$^ -Wl,-dead_strip -o $@ -lm
run_csi_sanitize: test_csi_sanitize
./test_csi_sanitize
# --- ADR-110 encoding unit tests ---
# Host-side, no libFuzzer needed — plain C99 deterministic table tests
@@ -93,8 +103,8 @@ test_thermal: test_thermal_hysteresis.c $(MAIN_DIR)/thermal.h
run_thermal: test_thermal
./test_thermal
host_tests: run_adr110 run_vitals run_mmwave_detect run_thermal
@echo "Host tests passed (ADR-110 + vitals #998/#996 + mmwave detect #1135 + thermal)"
host_tests: run_adr110 run_vitals run_mmwave_detect run_thermal run_csi_sanitize
@echo "Host tests passed (ADR-110 + CSI sanitation + vitals + mmwave + thermal)"
# --- Serialize fuzzer ---
# Tests csi_serialize_frame() with random wifi_csi_info_t inputs.
@@ -130,5 +140,6 @@ run_nvs: fuzz_nvs
run_all: run_serialize run_edge run_nvs
clean:
rm -f fuzz_serialize fuzz_edge fuzz_nvs test_adr110 test_vitals
rm -f fuzz_serialize fuzz_edge fuzz_nvs test_adr110 test_vitals test_mmwave_detect \
test_thermal test_csi_sanitize
rm -rf corpus_serialize/ corpus_edge/ corpus_nvs/
@@ -90,6 +90,7 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
info.rx_ctrl.sig_mode = legacy_inputs & 0x03;
info.rx_ctrl.cwb = (legacy_inputs >> 2) & 0x01;
info.rx_ctrl.stbc = (legacy_inputs >> 3) & 0x01;
info.first_word_invalid = (test_case & 0x80) != 0;
/* Use remaining fuzz data as I/Q buffer content. */
uint16_t iq_len;
@@ -136,6 +137,17 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
if (result > out_len) {
__builtin_trap(); /* Buffer overflow detected. */
}
if (result >= CSI_HEADER_SIZE + iq_len && info.first_word_invalid && iq_len > 0) {
size_t invalid_len = iq_len < 4 ? iq_len : 4;
for (size_t i = 0; i < invalid_len; i++) {
if (out_buf[CSI_HEADER_SIZE + i] != 0) {
__builtin_trap(); /* Invalid hardware prefix escaped sanitation. */
}
}
if ((out_buf[19] & CSI_FLAG_FIRST_WORD_SANITIZED) == 0) {
__builtin_trap(); /* Sanitized payload must carry provenance. */
}
}
/* --- Test case 1: NULL info pointer --- */
if (test_case & 0x01) {
@@ -90,6 +90,7 @@ typedef struct {
typedef struct {
wifi_pkt_rx_ctrl_t rx_ctrl;
uint8_t mac[6];
bool first_word_invalid;
int16_t len; /**< Length of the I/Q buffer in bytes. */
int8_t *buf; /**< Pointer to I/Q data. */
} wifi_csi_info_t;
@@ -0,0 +1,56 @@
/** Deterministic host test for ESP-IDF first_word_invalid sanitation. */
#include "esp_stubs.h"
#include "nvs_config.h"
#include "csi_collector.h"
#include <assert.h>
#include <stdio.h>
#include <string.h>
nvs_config_t g_nvs_config;
static void make_info(wifi_csi_info_t *info, int8_t *iq, uint16_t len, bool invalid)
{
memset(info, 0, sizeof(*info));
info->rx_ctrl.channel = 6;
info->rx_ctrl.rssi = -42;
info->rx_ctrl.noise_floor = -91;
info->len = (int16_t)len;
info->buf = iq;
info->first_word_invalid = invalid;
}
int main(void)
{
int8_t iq[] = {1, 2, 3, 4, 5, 6, 7, 8};
uint8_t frame[CSI_MAX_FRAME_SIZE];
wifi_csi_info_t info;
make_info(&info, iq, sizeof(iq), true);
size_t len = csi_serialize_frame(&info, frame, sizeof(frame));
assert(len == CSI_HEADER_SIZE + sizeof(iq));
assert(frame[CSI_HEADER_SIZE] == 0);
assert(frame[CSI_HEADER_SIZE + 1] == 0);
assert(frame[CSI_HEADER_SIZE + 2] == 0);
assert(frame[CSI_HEADER_SIZE + 3] == 0);
assert(frame[CSI_HEADER_SIZE + 4] == 5);
assert((frame[19] & CSI_FLAG_FIRST_WORD_SANITIZED) != 0);
make_info(&info, iq, sizeof(iq), false);
len = csi_serialize_frame(&info, frame, sizeof(frame));
assert(len == CSI_HEADER_SIZE + sizeof(iq));
assert(memcmp(&frame[CSI_HEADER_SIZE], iq, sizeof(iq)) == 0);
assert((frame[19] & CSI_FLAG_FIRST_WORD_SANITIZED) == 0);
int8_t short_iq[] = {9, 10};
make_info(&info, short_iq, sizeof(short_iq), true);
len = csi_serialize_frame(&info, frame, sizeof(frame));
assert(len == CSI_HEADER_SIZE + sizeof(short_iq));
assert(frame[CSI_HEADER_SIZE] == 0);
assert(frame[CSI_HEADER_SIZE + 1] == 0);
assert((frame[19] & CSI_FLAG_FIRST_WORD_SANITIZED) != 0);
puts("CSI invalid-prefix sanitation: PASS");
return 0;
}
@@ -145,13 +145,16 @@ impl PpduType {
/// bit 2 : STBC
/// bit 3 : LDPC (reserved — not yet populated by firmware)
/// bit 4 : 802.15.4 time-sync valid (C6 only)
/// bit 5-7 : reserved
/// bit 5 : ESP-IDF invalid first CSI word was zeroed by firmware
/// bit 6-7 : reserved
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct Adr018Flags {
pub bw40: bool,
pub stbc: bool,
pub ldpc: bool,
pub ieee802154_sync_valid: bool,
#[serde(default)]
pub first_word_sanitized: bool,
}
impl Adr018Flags {
@@ -161,6 +164,7 @@ impl Adr018Flags {
stbc: (b & 0x04) != 0,
ldpc: (b & 0x08) != 0,
ieee802154_sync_valid: (b & 0x10) != 0,
first_word_sanitized: (b & 0x20) != 0,
}
}
pub fn to_byte(self) -> u8 {
@@ -169,6 +173,7 @@ impl Adr018Flags {
if self.stbc { b |= 0x04; }
if self.ldpc { b |= 0x08; }
if self.ieee802154_sync_valid { b |= 0x10; }
if self.first_word_sanitized { b |= 0x20; }
b
}
}
@@ -413,15 +413,16 @@ mod tests {
#[test]
fn adr110_flags_round_trip_all_bits() {
// All known flag bits set: bw40 (0x01) + STBC (0x04) + LDPC (0x08) + 15.4-sync (0x10) = 0x1D
let data = build_test_frame_with_he(6, 1, &[(0, 0); 56], 1, 0x1D);
// All known flag bits set: bw40 + STBC + LDPC + sync + sanitized prefix = 0x3D.
let data = build_test_frame_with_he(6, 1, &[(0, 0); 56], 1, 0x3D);
let (frame, _) = Esp32CsiParser::parse_frame(&data).unwrap();
assert!(frame.metadata.adr018_flags.bw40);
assert!(frame.metadata.adr018_flags.stbc);
assert!(frame.metadata.adr018_flags.ldpc);
assert!(frame.metadata.adr018_flags.ieee802154_sync_valid);
assert!(frame.metadata.adr018_flags.first_word_sanitized);
// Round-trip the encoder
assert_eq!(frame.metadata.adr018_flags.to_byte(), 0x1D);
assert_eq!(frame.metadata.adr018_flags.to_byte(), 0x3D);
}
#[test]
@@ -139,6 +139,39 @@ pub fn features_from_runtime(feat: &serde_json::Value, amps: &[f64]) -> [f64; N_
]
}
/// Preserve the fifth feature's historical meaning for models trained before
/// the server began measuring temporal frequency from a frame window.
///
/// Version 1 models learned a proxy equal to the strongest subcarrier index
/// multiplied by 0.05. Version 2 and later models consume the measured
/// temporal frequency in hertz. Keeping this translation at the model boundary
/// prevents a server upgrade from silently changing an existing model's input
/// distribution.
pub fn compatible_dominant_frequency(
model_version: u32,
temporal_frequency_hz: f64,
amplitudes: &[f64],
) -> f64 {
if model_version >= 2 {
return if temporal_frequency_hz.is_finite() {
temporal_frequency_hz.max(0.0)
} else {
0.0
};
}
amplitudes
.iter()
.enumerate()
.filter(|(_, amplitude)| amplitude.is_finite())
.max_by(|(_, left), (_, right)| {
left.partial_cmp(right)
.unwrap_or(std::cmp::Ordering::Equal)
})
.map(|(index, _)| index as f64 * 0.05)
.unwrap_or(0.0)
}
/// Compute statistical features from raw subcarrier amplitudes.
fn subcarrier_stats(amps: &[f64]) -> (f64, f64, f64, f64, f64, f64, f64, f64) {
if amps.is_empty() {
@@ -642,3 +675,21 @@ pub fn train_from_recordings(recordings_dir: &Path) -> Result<AdaptiveModel, Str
pub fn model_path() -> PathBuf {
PathBuf::from("data/adaptive_model.json")
}
#[cfg(test)]
mod compatibility_tests {
use super::compatible_dominant_frequency;
#[test]
fn version_one_keeps_legacy_subcarrier_proxy() {
let amplitudes = [1.0, 2.0, 8.0, 3.0];
assert_eq!(compatible_dominant_frequency(1, 1.25, &amplitudes), 0.10);
}
#[test]
fn version_two_uses_measured_temporal_frequency() {
let amplitudes = [1.0, 20.0, 2.0];
assert_eq!(compatible_dominant_frequency(2, 1.25, &amplitudes), 1.25);
assert_eq!(compatible_dominant_frequency(2, f64::NAN, &amplitudes), 0.0);
}
}
@@ -11,6 +11,141 @@ use crate::vital_signs::VitalSigns;
const EDGE_MAX_PERSONS: u8 = 4;
/// Estimate a real temporal motion frequency from recent CSI amplitude
/// history. Each frame is reduced to its mean amplitude, detrended, Hann
/// windowed, and evaluated on exact DFT bins.
pub fn estimate_temporal_dominant_frequency_hz(
frame_history: &VecDeque<Vec<f64>>,
sample_rate_hz: f64,
) -> f64 {
if !sample_rate_hz.is_finite() || sample_rate_hz <= 0.0 {
return 0.0;
}
let recent_frames: Vec<&Vec<f64>> = frame_history
.iter()
.rev()
.take(128)
.collect();
if recent_frames
.iter()
.any(|frame| frame.is_empty() || frame.iter().any(|value| !value.is_finite()))
{
return 0.0;
}
let samples: Vec<f64> = recent_frames
.into_iter()
.rev()
.map(|frame| frame.iter().sum::<f64>() / frame.len() as f64)
.collect();
let n = samples.len();
if n < 16 {
return 0.0;
}
let mean = samples.iter().sum::<f64>() / n as f64;
let upper_hz = 4.0_f64.min(sample_rate_hz * 0.45);
if upper_hz < 0.1 {
return 0.0;
}
let mut powers = Vec::new();
for bin in 1..=(n / 2) {
let frequency_hz = bin as f64 * sample_rate_hz / n as f64;
if !(0.1..=upper_hz).contains(&frequency_hz) {
continue;
}
let mut real = 0.0;
let mut imag = 0.0;
let angle_step = 2.0 * std::f64::consts::PI * bin as f64 / n as f64;
let (step_sin, step_cos) = angle_step.sin_cos();
let mut angle_cos = 1.0;
let mut angle_sin = 0.0;
for (index, sample) in samples.iter().enumerate() {
let window =
0.5 * (1.0 - (2.0 * std::f64::consts::PI * index as f64 / (n - 1) as f64).cos());
let value = (sample - mean) * window;
real += value * angle_cos;
imag -= value * angle_sin;
let next_cos = angle_cos * step_cos - angle_sin * step_sin;
angle_sin = angle_sin * step_cos + angle_cos * step_sin;
angle_cos = next_cos;
}
powers.push((frequency_hz, real * real + imag * imag));
}
if powers.is_empty() {
return 0.0;
}
let mean_power = powers.iter().map(|(_, power)| power).sum::<f64>() / powers.len() as f64;
let (peak_hz, peak_power) = powers
.into_iter()
.max_by(|left, right| {
left.1
.partial_cmp(&right.1)
.unwrap_or(std::cmp::Ordering::Equal)
})
.unwrap_or((0.0, 0.0));
if mean_power <= f64::EPSILON || peak_power / mean_power < 2.0 {
0.0
} else {
peak_hz
}
}
#[cfg(test)]
mod temporal_frequency_tests {
use super::*;
#[test]
fn recovers_temporal_frequency_in_hertz() {
let sample_rate_hz = 20.0;
let target_hz = 1.2;
let mut history = VecDeque::new();
for sample in 0..100 {
let t = sample as f64 / sample_rate_hz;
let value = 20.0 + 2.0 * (2.0 * std::f64::consts::PI * target_hz * t).sin();
history.push_back(vec![value; 56]);
}
let measured = estimate_temporal_dominant_frequency_hz(&history, sample_rate_hz);
assert!((measured - target_hz).abs() <= 0.21, "measured {measured} Hz");
}
#[test]
fn abstains_on_short_or_flat_history() {
let short = VecDeque::from(vec![vec![20.0; 56]; 15]);
assert_eq!(estimate_temporal_dominant_frequency_hz(&short, 20.0), 0.0);
let flat = VecDeque::from(vec![vec![20.0; 56]; 100]);
assert_eq!(estimate_temporal_dominant_frequency_hz(&flat, 20.0), 0.0);
let mut corrupt = flat;
corrupt[50][4] = f64::NAN;
assert_eq!(estimate_temporal_dominant_frequency_hz(&corrupt, 20.0), 0.0);
}
#[test]
fn temporal_frequency_p95_stays_below_server_budget() {
let sample_rate_hz = 50.0;
let mut history = VecDeque::new();
for sample in 0..128 {
let t = sample as f64 / sample_rate_hz;
let value = 20.0 + 2.0 * (2.0 * std::f64::consts::PI * 1.2 * t).sin();
history.push_back(vec![value; 256]);
}
let mut timings = Vec::with_capacity(1_000);
for _ in 0..1_000 {
let start = std::time::Instant::now();
std::hint::black_box(estimate_temporal_dominant_frequency_hz(
std::hint::black_box(&history),
sample_rate_hz,
));
timings.push(start.elapsed());
}
timings.sort_unstable();
let p95 = timings[949];
eprintln!("temporal dominant frequency p95: {p95:?}");
assert!(p95 < std::time::Duration::from_millis(10), "p95 {p95:?}");
}
}
/// Person count is supporting evidence, never an independent occupancy claim.
/// Return zero and mark invalid for contradictory or out-of-range firmware.
fn sanitize_edge_person_count(presence: bool, raw: u8) -> (u8, bool) {
@@ -469,14 +604,7 @@ pub fn extract_features_from_frame(
0.0
};
let peak_idx = frame
.amplitudes
.iter()
.enumerate()
.max_by(|a, b| a.1.partial_cmp(b.1).unwrap_or(std::cmp::Ordering::Equal))
.map(|(i, _)| i)
.unwrap_or(0);
let dominant_freq_hz = peak_idx as f64 * 0.05;
let dominant_freq_hz = estimate_temporal_dominant_frequency_hz(frame_history, sample_rate_hz);
let threshold = mean_amp * 1.2;
let change_points = frame
@@ -485,7 +613,9 @@ pub fn extract_features_from_frame(
.filter(|w| (w[0] < threshold) != (w[1] < threshold))
.count();
let temporal_motion_score = if let Some(prev_frame) = frame_history.back() {
// The caller appends the current frame before extraction, so the true
// predecessor is the second item from the back.
let temporal_motion_score = if let Some(prev_frame) = frame_history.iter().rev().nth(1) {
let n_cmp = n_sub.min(prev_frame.len());
if n_cmp > 0 {
let diff_energy: f64 = (0..n_cmp)
@@ -636,13 +766,18 @@ pub fn adaptive_override(
.back()
.map(|v| v.as_slice())
.unwrap_or(&[]);
let dominant_frequency = adaptive_classifier::compatible_dominant_frequency(
model.version,
features.dominant_freq_hz,
amps,
);
let feat_arr = adaptive_classifier::features_from_runtime(
&serde_json::json!({
"variance": features.variance,
"motion_band_power": features.motion_band_power,
"breathing_band_power": features.breathing_band_power,
"spectral_power": features.spectral_power,
"dominant_freq_hz": features.dominant_freq_hz,
"dominant_freq_hz": dominant_frequency,
"change_points": features.change_points,
"mean_rssi": features.mean_rssi,
}),
@@ -47,6 +47,9 @@ const MAX_SINGLE_LINK_OCCUPANCY: usize = 3;
pub struct BootstrapBackgroundMatch {
pub matches_empty: bool,
pub score: Option<f64>,
pub normalized_residual_z: Option<f64>,
pub maturity: f64,
pub reliable: bool,
pub residual_energy: f64,
pub residual_energy_threshold: f64,
pub window_size: usize,
@@ -227,6 +230,9 @@ pub fn bootstrap_background_match(
Some(BootstrapBackgroundMatch {
matches_empty: result.matches_empty,
score: result.score,
normalized_residual_z: result.normalized_residual_z,
maturity: result.maturity,
reliable: result.reliable,
residual_energy: result.residual_energy,
residual_energy_threshold: result.residual_energy_threshold,
window_size: result.window_size,
@@ -502,6 +508,9 @@ mod tests {
.expect("background score");
assert!(background.matches_empty);
assert_eq!(background.reference_window_count, 12);
assert!(background.reliable);
assert_eq!(background.maturity, 0.6);
assert!(background.normalized_residual_z.is_some());
assert!(background
.score
.is_some_and(|score| (0.5..=1.0).contains(&score)));
@@ -530,5 +539,6 @@ mod tests {
.expect("shifted background result");
assert!(!shifted_background.matches_empty);
assert_eq!(shifted_background.score, Some(0.0));
assert!(shifted_background.reliable);
}
}
@@ -915,10 +915,10 @@ const NOVELTY_SKETCH_VERSION: u16 = 1;
/// bursts whose intra-burst arrival deltas are tens of microseconds apart —
/// a 36 µs delta yields `1/dt ≈ 27 kHz`, which the old `< 1 s` guard let
/// straight into the EMA and inflated `csi_fps_ema` by 1–3 orders of
/// magnitude (issue #1180). We reject any delta implying more than 200 fps
/// (4× the physical ceiling, leaving slack for benign arrival jitter); such
/// deltas are burst artifacts, not distinct production intervals.
/// magnitude (issue #1180). We reject sub-5 ms deltas as burst artifacts and
/// cap accepted estimates to the firmware's 50 fps physical ceiling.
pub(crate) const MIN_PLAUSIBLE_CSI_DT_SEC: f64 = 0.005;
pub(crate) const MAX_PHYSICAL_CSI_FPS: f64 = 50.0;
/// ADR-110 iter 18 — EMA update for per-node CSI fps tracking.
///
@@ -934,7 +934,7 @@ pub(crate) fn update_csi_fps_ema(prev_fps: f64, dt_sec: f64) -> Option<f64> {
if !(dt_sec >= MIN_PLAUSIBLE_CSI_DT_SEC && dt_sec < 1.0) {
return None;
}
let instantaneous = 1.0 / dt_sec;
let instantaneous = (1.0 / dt_sec).min(MAX_PHYSICAL_CSI_FPS);
// y[n] = y[n-1] + (x - y[n-1]) / 8
Some(prev_fps + (instantaneous - prev_fps) / 8.0)
}
@@ -983,6 +983,15 @@ mod fps_ema_tests {
assert!(update_csi_fps_ema(40.0, 0.005).is_some());
}
#[test]
fn accepted_burst_edge_is_capped_to_firmware_ceiling() {
let mut fps = 50.0;
for _ in 0..32 {
fps = update_csi_fps_ema(fps, 0.005).unwrap();
}
assert!(fps <= 50.0, "reported {fps} Hz above firmware ceiling");
}
#[test]
fn burst_interleaved_with_nominal_stays_in_band() {
// A true ~40 fps node whose frames arrive in sub-ms bursts: feeding
@@ -1004,6 +1013,18 @@ mod fps_ema_tests {
}
impl NodeState {
/// Measured sample rate only after enough inter-frame deltas have warmed
/// the EMA. The physical ceiling is enforced again at the consumption
/// boundary so restored or malformed state cannot overclock DSP math.
fn measured_sample_rate_hz(&self) -> Option<f64> {
(self.csi_fps_samples >= 5 && self.csi_fps_ema.is_finite())
.then(|| self.csi_fps_ema.clamp(1.0, MAX_PHYSICAL_CSI_FPS))
}
fn effective_sample_rate_hz(&self) -> f64 {
self.measured_sample_rate_hz().unwrap_or(20.0)
}
/// ADR-110 §A0.12 timestamp recovery: given a CSI frame's node-local
/// `esp_timer_get_time()` snapshot, return the mesh-aligned epoch
/// computed from this node's most recent sync packet — or `None`
@@ -1051,7 +1072,7 @@ impl NodeState {
// samples; until then fall back to the 20 Hz firmware ceiling. The
// §A0.12 capture showed real bench fps ≈ 10, so the measured value
// is significantly more accurate than the constant fallback.
let fps = if self.csi_fps_samples >= 5 { self.csi_fps_ema } else { 20.0 };
let fps = self.effective_sample_rate_hz();
Some(sync.mesh_aligned_us_for_sequence(frame_sequence, fps))
}
@@ -1099,7 +1120,7 @@ impl NodeState {
is_valid: sync.flags.is_valid,
smoothed: sync.flags.smoothed_used,
sequence: sync.sequence,
csi_fps_ema: self.csi_fps_ema,
csi_fps_ema: self.effective_sample_rate_hz(),
csi_fps_samples: self.csi_fps_samples,
staleness_ms: self.latest_sync_at.map(|t| t.elapsed().as_millis() as u64),
})
@@ -1163,7 +1184,7 @@ impl NodeState {
hr_buffer: VecDeque::with_capacity(8),
br_buffer: VecDeque::with_capacity(8),
rssi_history: VecDeque::new(),
vital_detector: VitalSignDetector::new(10.0),
vital_detector: VitalSignDetector::new(20.0),
latest_vitals: VitalSigns::default(),
last_frame_time: None,
edge_vitals: None,
@@ -1349,7 +1370,7 @@ fn build_node_features(
},
rssi_dbm: ns.rssi_history.back().copied().unwrap_or(0.0),
last_seen_ms,
frame_rate_hz: 0.0, // Computed elsewhere; not yet plumbed here.
frame_rate_hz: ns.measured_sample_rate_hz().unwrap_or(0.0),
stale,
novelty_score: ns.last_novelty_score,
}
@@ -2768,15 +2789,9 @@ fn extract_features_from_frame(
0.0
};
// ── Dominant frequency via peak subcarrier index ──
let peak_idx = frame
.amplitudes
.iter()
.enumerate()
.max_by(|a, b| a.1.partial_cmp(b.1).unwrap_or(std::cmp::Ordering::Equal))
.map(|(i, _)| i)
.unwrap_or(0);
let dominant_freq_hz = peak_idx as f64 * 0.05;
// A subcarrier index is a spatial-frequency bin, not temporal hertz.
let dominant_freq_hz =
csi::estimate_temporal_dominant_frequency_hz(frame_history, sample_rate_hz);
// ── Change point detection (threshold-crossing count in current frame) ──
let threshold = mean_amp * 1.2;
@@ -2789,7 +2804,9 @@ fn extract_features_from_frame(
// ── Motion score: sliding-window temporal difference ──
// Compare current frame against the most recent historical frame.
// The difference is normalised by the mean amplitude to be scale-invariant.
let temporal_motion_score = if let Some(prev_frame) = frame_history.back() {
// Callers append the current frame first, making the true predecessor the
// second item from the back.
let temporal_motion_score = if let Some(prev_frame) = frame_history.iter().rev().nth(1) {
let n_cmp = n_sub.min(prev_frame.len());
if n_cmp > 0 {
let diff_energy: f64 = (0..n_cmp)
@@ -3022,13 +3039,18 @@ fn adaptive_override(
.back()
.map(|v| v.as_slice())
.unwrap_or(&[]);
let dominant_frequency = adaptive_classifier::compatible_dominant_frequency(
model.version,
features.dominant_freq_hz,
amps,
);
let feat_arr = adaptive_classifier::features_from_runtime(
&serde_json::json!({
"variance": features.variance,
"motion_band_power": features.motion_band_power,
"breathing_band_power": features.breathing_band_power,
"spectral_power": features.spectral_power,
"dominant_freq_hz": features.dominant_freq_hz,
"dominant_freq_hz": dominant_frequency,
"change_points": features.change_points,
"mean_rssi": features.mean_rssi,
}),
@@ -6452,6 +6474,9 @@ async fn calibration_status(State(state): State<SharedState>) -> Json<serde_json
"state": if result.matches_empty { "matched" } else { "changed" },
"matches_empty": result.matches_empty,
"score": result.score,
"normalized_residual_z": result.normalized_residual_z,
"maturity": result.maturity,
"reliable": result.reliable,
"residual_energy": result.residual_energy,
"residual_energy_threshold": result.residual_energy_threshold,
"window_size": result.window_size,
@@ -8044,7 +8069,7 @@ async fn udp_receiver_task(
ns.frame_history.pop_front();
}
let sample_rate_hz = 1000.0 / 500.0_f64;
let sample_rate_hz = ns.effective_sample_rate_hz();
let (
features,
mut classification,
@@ -8057,13 +8082,19 @@ async fn udp_receiver_task(
// Adaptive override using cloned model (safe, no raw pointers).
if let Some(ref model) = adaptive_model_clone {
let amps = ns.frame_history.back().map(|v| v.as_slice()).unwrap_or(&[]);
let dominant_frequency =
adaptive_classifier::compatible_dominant_frequency(
model.version,
features.dominant_freq_hz,
amps,
);
let feat_arr = adaptive_classifier::features_from_runtime(
&serde_json::json!({
"variance": features.variance,
"motion_band_power": features.motion_band_power,
"breathing_band_power": features.breathing_band_power,
"spectral_power": features.spectral_power,
"dominant_freq_hz": features.dominant_freq_hz,
"dominant_freq_hz": dominant_frequency,
"change_points": features.change_points,
"mean_rssi": features.mean_rssi,
}),
@@ -8081,6 +8112,18 @@ async fn udp_receiver_task(
ns.rssi_history.pop_front();
}
if ns.csi_fps_samples >= 5
&& ns.vital_detector.reconfigure_sample_rate(sample_rate_hz)
{
// Never smooth estimates computed against two clocks.
ns.smoothed_hr = 0.0;
ns.smoothed_br = 0.0;
ns.smoothed_hr_conf = 0.0;
ns.smoothed_br_conf = 0.0;
ns.hr_buffer.clear();
ns.br_buffer.clear();
}
let raw_vitals = ns
.vital_detector
.process_frame(&frame.amplitudes, &frame.phases);
@@ -10620,6 +10663,20 @@ mod sync_snapshot_helper_tests {
assert_eq!(snap.csi_fps_samples, 42);
}
#[test]
fn exported_node_rate_is_mature_and_physically_bounded() {
let mut ns = NodeState::new();
assert_eq!(ns.measured_sample_rate_hz(), None);
ns.csi_fps_ema = 74.0;
ns.csi_fps_samples = 5;
assert_eq!(ns.measured_sample_rate_hz(), Some(50.0));
ns.latest_sync = Some(populated_sync(9));
ns.latest_sync_at = Some(std::time::Instant::now());
assert_eq!(ns.sync_snapshot().unwrap().csi_fps_ema, 50.0);
}
#[test]
fn observe_csi_frame_arrival_ignores_subms_bursts() {
// Issue #1180 regression: a ~40 fps node whose frames are delivered
@@ -346,6 +346,31 @@ impl VitalSignDetector {
self.frame_count = 0;
}
/// Retune temporal windows to an observed CSI clock. Changes smaller than
/// twenty percent are ignored to avoid resetting on normal arrival jitter.
/// A real clock change clears history so samples from different clocks are
/// never interpreted in one spectrum.
pub fn reconfigure_sample_rate(&mut self, sample_rate: f64) -> bool {
if !sample_rate.is_finite() || sample_rate < 1.0 {
return false;
}
let relative_change = (sample_rate - self.sample_rate).abs() / self.sample_rate.max(1.0);
if relative_change < 0.20 {
return false;
}
self.sample_rate = sample_rate;
self.breathing_capacity =
((sample_rate * self.breathing_window_secs) as usize).max(1);
self.heartbeat_capacity =
((sample_rate * self.heartbeat_window_secs) as usize).max(1);
self.reset();
true
}
pub fn sample_rate_hz(&self) -> f64 {
self.sample_rate
}
/// Current buffer fill levels for diagnostics.
/// Returns (breathing_len, breathing_capacity, heartbeat_len, heartbeat_capacity).
pub fn buffer_status(&self) -> (usize, usize, usize, usize) {
@@ -842,6 +867,22 @@ mod tests {
assert_eq!(hb_len, 0);
}
#[test]
fn measured_clock_reconfigures_once_and_clears_mixed_history() {
let mut detector = VitalSignDetector::new(20.0);
let amp = vec![10.0; 56];
let phase = vec![0.0; 56];
for _ in 0..32 {
detector.process_frame(&amp, &phase);
}
assert!(!detector.reconfigure_sample_rate(18.0));
assert_eq!(detector.sample_rate_hz(), 20.0);
assert!(detector.reconfigure_sample_rate(12.0));
assert_eq!(detector.sample_rate_hz(), 12.0);
assert_eq!(detector.buffer_status(), (0, 360, 0, 180));
assert!(!detector.reconfigure_sample_rate(f64::NAN));
}
#[test]
fn test_vital_signs_default() {
let vs = VitalSigns::default();
@@ -380,6 +380,12 @@ pub struct FieldNormalMode {
pub struct EmptyRoomMatch {
pub matches_empty: bool,
pub score: Option<f64>,
/// Robust residual distance from the quiet reference median.
pub normalized_residual_z: Option<f64>,
/// Reference coverage relative to the normal calibration target.
pub maturity: f64,
/// Whether the reference set is large and valid enough to suppress change.
pub reliable: bool,
pub residual_energy: f64,
pub residual_energy_threshold: f64,
pub window_size: usize,
@@ -644,6 +650,27 @@ fn percentile_95(mut values: Vec<f64>) -> Option<f64> {
values.get(index).copied()
}
fn robust_residual_z(reference: &[f64], residual: f64) -> Option<f64> {
if !residual.is_finite() || reference.is_empty() {
return None;
}
let mut sorted: Vec<f64> = reference
.iter()
.copied()
.filter(|value| value.is_finite() && *value >= 0.0)
.collect();
if sorted.len() != reference.len() {
return None;
}
sorted.sort_by(|left, right| left.partial_cmp(right).unwrap_or(std::cmp::Ordering::Equal));
let median = sorted[sorted.len() / 2];
let mut deviations: Vec<f64> = sorted.iter().map(|value| (value - median).abs()).collect();
deviations.sort_by(|left, right| left.partial_cmp(right).unwrap_or(std::cmp::Ordering::Equal));
let mad = deviations[deviations.len() / 2];
let robust_scale = (1.4826 * mad).max((median.abs() * 0.01).max(f64::EPSILON));
Some((residual - median) / robust_scale)
}
#[cfg(feature = "eigenvalue")]
fn percentile_95_usize(mut values: Vec<usize>) -> Option<usize> {
values.sort_unstable();
@@ -1315,12 +1342,20 @@ impl FieldModel {
let residual_energy_threshold = modes
.empty_room_residual_energy_threshold?
.max(EMPTY_ROOM_RESIDUAL_ENERGY_MAX);
let matches_empty = residual_energy <= residual_energy_threshold;
let reference_window_count = modes.empty_room_residual_energy_reference.len();
let score = if !matches_empty {
Some(0.0)
} else if reference_window_count == 0 {
let maturity = (reference_window_count as f64
/ MIN_BACKGROUND_REFERENCE_WINDOWS as f64)
.clamp(0.0, 1.0);
let reliable = reference_window_count >= 10
&& modes
.empty_room_residual_energy_reference
.iter()
.all(|value| value.is_finite() && *value >= 0.0);
let matches_empty = reliable && residual_energy <= residual_energy_threshold;
let score = if !reliable {
None
} else if !matches_empty {
Some(0.0)
} else {
let at_or_below = modes
.empty_room_residual_energy_reference
@@ -1331,6 +1366,12 @@ impl FieldModel {
Some(EmptyRoomMatch {
matches_empty,
score,
normalized_residual_z: robust_residual_z(
&modes.empty_room_residual_energy_reference,
residual_energy,
),
maturity,
reliable,
residual_energy,
residual_energy_threshold,
window_size,
@@ -2149,6 +2190,9 @@ mod tests {
.expect("background match");
assert!(background.matches_empty);
assert_eq!(background.reference_window_count, 12);
assert!(background.reliable);
assert_eq!(background.maturity, 0.6);
assert!(background.normalized_residual_z.is_some());
assert!(background
.score
.is_some_and(|score| (0.5..=1.0).contains(&score)));
@@ -2237,6 +2281,51 @@ mod tests {
let background = model.empty_room_match(&shifted).expect("shifted match");
assert!(!background.matches_empty);
assert_eq!(background.score, Some(0.0));
assert!(background.reliable);
assert!(background.normalized_residual_z.is_some());
}
#[cfg(feature = "eigenvalue")]
#[test]
fn sparse_background_reference_cannot_suppress_runtime_change() {
let config = FieldModelConfig {
n_links: 1,
n_subcarriers: 56,
n_modes: 3,
min_calibration_frames: 500,
min_calibration_duration_s: 0.0,
baseline_expiry_s: 86_400.0,
};
let mut model = FieldModel::new(config).unwrap();
let mut calibration = Vec::new();
for frame_index in 0..600 {
let time = frame_index as f64 * 0.071;
let frame: Vec<f64> = (0..56)
.map(|subcarrier| {
let carrier = subcarrier as f64;
18.0 + carrier * 0.08
+ (time + carrier * 0.13).sin() * 0.8
+ (time * 0.37 + carrier * 0.031).cos() * 0.35
})
.collect();
model.feed_calibration(&[frame.clone()]).unwrap();
calibration.push(frame);
}
model.finalize_calibration(1_000_000, 0).unwrap();
model
.modes
.as_mut()
.unwrap()
.empty_room_residual_energy_reference
.truncate(9);
let result = model
.empty_room_match(&calibration[550..600])
.expect("background comparison");
assert!(!result.reliable);
assert!(!result.matches_empty);
assert_eq!(result.score, None);
assert_eq!(result.maturity, 0.45);
}
#[test]