starred/RuView
1
0 Fork 0
mirror of https://github.com/ruvnet/RuView.git synced 2026-10-18 21:00:22 -05:00
Code Issues Packages Projects Releases 10 Wiki Activity

feat(calibration): learn empty room startup confidence (#1840)

Browse Source
This commit is contained in:
rUv authored and GitHub committed 2026-09-04 23:02:06 -04:00
1 parent 2e9e87c65c
commit 2227b42841
6 files changed
+1218 -110
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/adr/ADR-355-local-empty-room-bootstrap-prior.md
+43 -12
View File
@@ -23,17 +23,26 @@ directory, which is unreliable for embedded application hosts.
RuView may persist a completed empty room field model as a local bootstrap
prior only after two server controlled stages:
1. Normal calibration reaches both 12,000 accepted frames and 600 seconds.
2. Promotion observes 12 new one second samples. At least 10 must resolve
empty, all ticks must be fresh, and none may publish numeric vital signs.
1. Normal calibration reaches both 1,000 accepted frames and 600 seconds.
The frame target represents twenty complete fifty frame runtime windows.
Collection binds to the first accepted ESP32 source and rejects frames from
other radios for that single link model.
2. Promotion observes 12 new samples spaced by at least one second. Each
sample has a four second bounded acquisition timeout. At least 10 must
resolve empty, all ticks must be fresh, and none may publish numeric vital
signs.
The stored image contains aggregate means, environmental modes, mode energies,
noise statistics, configuration, contributing node IDs, and a server generated
model ID. It excludes raw CSI, waveforms, device addresses, room names, pose,
identity, credentials, and positive human labels.
noise statistics, sorted scalar residual references, configuration, one source
node ID, and a server generated model ID. It excludes raw CSI, waveforms,
device addresses, room names, pose, identity, credentials, and positive human
labels. Runtime matching compares a fifty frame mean with the learned residual
distribution. Its score is empirical background conformance, not an absence
probability.
The image is bound to a SHA 256 digest of the stable installation ID, carries
an integrity digest, is capped at 1 MiB, and inherits the field model expiry.
an integrity digest over the exact stored payload bytes, is capped at 1 MiB,
and inherits the field model expiry.
Files with an invalid schema, dimensions, values, node set, installation
binding, digest, size, or lifetime fail closed.
@@ -43,6 +52,11 @@ evidence or numeric heart and breathing rates. Explicit calibration replaces
it with fresh statistics. Cancelling an unfinished capture restores a valid
prior. Administrator scoped reset removes it.
The status endpoint reports the learned runtime window, residual threshold,
reference count, current background conformance, and the negative only
authority boundary. A restored model never resumes collection and never
persists raw calibration frames.
The server accepts `--data-dir` or `RUVIEW_DATA_DIR`, allowing an application
host to provide a protected state directory.
@@ -71,12 +85,29 @@ occupied sequences. Empty data is never relabeled as human training.
denies calibrated evidence and numeric vital authority.
5. Numeric vitals fail closed unless an explicit fresh calibration resolves
exactly one occupant at sufficient signal quality and confidence.
6. The legacy `edge_vitals` WebSocket message applies the same publication
gate as `sensing_update`. When the bootstrap background matches, it emits
absent state, zero people, null numeric rates, and an explicit abstention
reason instead of forwarding raw edge candidates.
## Measured installation evidence
One physical empty home run on source Node 5 collected 1,762 accepted frames
over 655 seconds and produced 35 privacy reduced reference windows. Promotion
observed 12 of 12 fresh empty samples, zero stale samples, and zero numeric
vital samples. After process restart, 12 of 12 fresh samples matched the empty
background and contained no numeric vital signs. Background conformance scores
ranged from 50.0 to 64.3 percent with a median of 54.3 percent.
This is measured negative evidence for one installation. No occupied positive
holdout was recorded, so it does not prove person recall, person counting, or
cross home accuracy improvement.
## Acceptance test
With a physically empty room, complete calibration on stable nodes, promote
the model, and restart the server with the same installation ID and data
directory. Status must report `binding_mode=bootstrap_only`, zero collection
frames, and false authority for calibrated evidence and numeric vitals. Twelve
fresh empty samples must contain no numeric vital signs. Starting a new room
calibration must replace the prior and begin at frame zero.
the single source model, and restart the server with the same installation ID
and data directory. Status must report `binding_mode=bootstrap_only`, zero
collection frames, and false authority for calibrated evidence and numeric
vitals. Twelve fresh empty samples must contain no numeric vital signs.
Starting a new room calibration must replace the prior and begin at frame zero.
v2/crates/wifi-densepose-sensing-server/Cargo.toml
+1 -1
View File
@@ -30,7 +30,7 @@ mdns-sd = "0.11"
# Serialization
serde = { workspace = true }
serde_json.workspace = true
serde_json = { workspace = true, features = ["raw_value"] }
# Logging
tracing.workspace = true
v2/crates/wifi-densepose-sensing-server/src/bootstrap_baseline.rs
+51 -3
View File
@@ -6,6 +6,7 @@
use rand::{rngs::OsRng, RngCore};
use serde::{Deserialize, Serialize};
use serde_json::value::RawValue;
use sha2::{Digest, Sha256};
use std::fs::{self, OpenOptions};
use std::io::{Read, Write};
@@ -19,6 +20,8 @@ pub const BOOTSTRAP_BASELINE_SCHEMA: &str = "ruview.bootstrap-empty-field-model.
pub const BOOTSTRAP_BASELINE_AUTHORITY: &str = "bootstrap_only";
pub const BOOTSTRAP_VALIDATION_SAMPLES: usize = 12;
pub const BOOTSTRAP_VALIDATION_MIN_EMPTY: usize = 10;
pub const BOOTSTRAP_VALIDATION_MIN_SPACING_MS: u64 = 1_000;
pub const BOOTSTRAP_VALIDATION_SAMPLE_TIMEOUT_MS: u64 = 4_000;
const BOOTSTRAP_MAX_FILE_BYTES: u64 = 1_048_576;
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
@@ -41,6 +44,13 @@ struct BootstrapImageV1 {
content_sha256: String,
}
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct BootstrapImageRawV1 {
payload: Box<RawValue>,
content_sha256: String,
}
#[derive(Debug, Clone, Serialize, PartialEq, Eq)]
pub struct BootstrapBaselineMetadata {
pub authority: &'static str,
@@ -181,11 +191,19 @@ pub fn load(
if bytes.len() as u64 > BOOTSTRAP_MAX_FILE_BYTES {
return Err(BootstrapBaselineError::FileTooLarge);
}
let image: BootstrapImageV1 = serde_json::from_slice(&bytes)?;
validate_payload(&image.payload, installation_id, current_unix_ms)?;
if payload_digest(&image.payload)? != image.content_sha256 {
// Verify the exact payload bytes written by storage before decoding the
// floating point model. Equivalent JSON number spellings may serialize
// differently, so decode and reserialize is not an integrity contract.
let raw_image: BootstrapImageRawV1 = serde_json::from_slice(&bytes)?;
if hex_sha256(raw_image.payload.get().as_bytes()) != raw_image.content_sha256 {
return Err(BootstrapBaselineError::DigestMismatch);
}
let payload: BootstrapPayloadV1 = serde_json::from_str(raw_image.payload.get())?;
validate_payload(&payload, installation_id, current_unix_ms)?;
let image = BootstrapImageV1 {
payload,
content_sha256: raw_image.content_sha256,
};
let model = FieldModel::from_snapshot(
image.payload.field_model.clone(),
current_unix_ms.saturating_mul(1_000),
@@ -368,6 +386,11 @@ mod tests {
#[test]
fn promotion_gate_requires_twelve_fresh_samples_and_zero_vitals() {
assert_eq!(BOOTSTRAP_VALIDATION_MIN_SPACING_MS, 1_000);
assert_eq!(BOOTSTRAP_VALIDATION_SAMPLE_TIMEOUT_MS, 4_000);
assert!(
BOOTSTRAP_VALIDATION_SAMPLE_TIMEOUT_MS >= BOOTSTRAP_VALIDATION_MIN_SPACING_MS
);
let passing = vec![
BootstrapValidationSample {
fresh_tick: true,
@@ -385,4 +408,29 @@ mod tests {
vital[0].vital_signs_absent = false;
assert!(!evaluate_validation(&vital).passed);
}
#[test]
fn restart_verifies_the_exact_stored_payload_bytes() {
let dir = tempfile::tempdir().unwrap();
let path = path_in(dir.path());
let now_ms = 1_000_000;
let model = completed_model(now_ms * 1_000);
store(&path, "home-a", &[5], "model", now_ms, &model).unwrap();
let original = fs::read_to_string(&path).unwrap();
let image: BootstrapImageRawV1 = serde_json::from_str(&original).unwrap();
let rewritten_payload = image.payload.get().replacen(
"\"min_calibration_duration_s\":0.0",
"\"min_calibration_duration_s\":0.00",
1,
);
assert_ne!(rewritten_payload, image.payload.get());
let digest = hex_sha256(rewritten_payload.as_bytes());
let rewritten =
format!("{{\"payload\":{rewritten_payload},\"content_sha256\":\"{digest}\"}}");
fs::write(&path, rewritten).unwrap();
let (_, loaded) = load(&path, "home-a", now_ms + 1).unwrap();
assert_eq!(loaded.content_sha256, digest);
}
}
v2/crates/wifi-densepose-sensing-server/src/field_bridge.rs
+284 -16
View File
@@ -40,6 +40,103 @@ const ENERGY_THRESH_3: f64 = 25.0;
/// reliable — genuine higher counts come from the multistatic fusion path.
const MAX_SINGLE_LINK_OCCUPANCY: usize = 3;
/// Low-authority comparison of the current source window with a restored
/// empty-room bootstrap image. This is background conformance only and cannot
/// authorize presence, person count, pose, identity, or vital signs.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct BootstrapBackgroundMatch {
pub matches_empty: bool,
pub score: Option<f64>,
pub residual_energy: f64,
pub residual_energy_threshold: f64,
pub window_size: usize,
pub reference_window_count: usize,
}
fn mean_frame(frames: &[Vec<f64>]) -> Option<Vec<f64>> {
let width = frames.first()?.len();
if width == 0 {
return None;
}
let mut mean = vec![0.0_f64; width];
let mut count = 0_usize;
for frame in frames {
if frame.len() != width {
continue;
}
for (sum, value) in mean.iter_mut().zip(frame.iter()) {
*sum += value;
}
count += 1;
}
if count == 0 {
return None;
}
for value in &mut mean {
*value /= count as f64;
}
Some(mean)
}
fn perturbation_occupancy(field: &FieldModel, frames: &[Vec<f64>]) -> Option<usize> {
let adaptive_threshold = field.empty_room_residual_energy_threshold();
let frame = match adaptive_threshold {
Some(_) => mean_frame(frames)?,
None => frames.first()?.clone(),
};
let perturbation = field.extract_perturbation(&[frame]).ok()?;
let count = match adaptive_threshold {
Some(threshold) => {
let empty_threshold = threshold.max(1.0);
if perturbation.total_energy > empty_threshold * 25.0 {
3
} else if perturbation.total_energy > empty_threshold * 12.0 {
2
} else if perturbation.total_energy > empty_threshold {
1
} else {
0
}
}
None => {
if perturbation.total_energy > ENERGY_THRESH_3 {
3
} else if perturbation.total_energy > ENERGY_THRESH_2 {
2
} else if perturbation.total_energy > 1.0 {
1
} else {
0
}
}
};
Some(count)
}
/// Provenance for a count produced by the calibrated field model. Callers
/// that attach calibration evidence must never substitute the score heuristic
/// when the calibrated path cannot evaluate the observation.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum CalibratedOccupancyMethod {
Eigenvalue,
PerturbationEnergy,
}
impl CalibratedOccupancyMethod {
pub const fn wire_name(self) -> &'static str {
match self {
Self::Eigenvalue => "field_model_eigenvalue_v1",
Self::PerturbationEnergy => "field_model_perturbation_energy_v1",
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct CalibratedOccupancy {
pub person_count: usize,
pub method: CalibratedOccupancyMethod,
}
/// Create a FieldModelConfig for single-link mode (one ESP32 node = one link).
/// This avoids the DimensionMismatch error when feeding single-frame observations.
pub fn single_link_config() -> FieldModelConfig {
@@ -67,7 +164,7 @@ pub fn occupancy_or_fallback(
.iter()
.rev()
.take(OCCUPANCY_WINDOW)
.cloned()
.map(|frame| CALIB_NORMALIZER.resample_to_canonical(frame))
.collect();
if frames.is_empty() {
@@ -87,26 +184,86 @@ pub fn occupancy_or_fallback(
// Fallback: perturbation energy thresholds.
// FieldModel expects [n_links][n_subcarriers] — we use n_links=1.
let observation = vec![frames[0].clone()];
match field.extract_perturbation(&observation) {
Ok(perturbation) => {
if perturbation.total_energy > ENERGY_THRESH_3 {
3
} else if perturbation.total_energy > ENERGY_THRESH_2 {
2
} else if perturbation.total_energy > 1.0 {
1
} else {
0
}
}
Err(_) => score_to_person_count(smoothed_score, prev_count),
}
perturbation_occupancy(field, &frames)
.unwrap_or_else(|| score_to_person_count(smoothed_score, prev_count))
}
_ => score_to_person_count(smoothed_score, prev_count),
}
}
/// Return true only when a restored low-authority bootstrap model has a full
/// runtime window and that model resolves the window as empty. This helper is
/// deliberately one-way: a bootstrap prior may suppress an obvious background
/// false positive, but cannot authorize presence or vital signs.
pub fn bootstrap_empty_prior_applies(
field: &FieldModel,
frame_history: &VecDeque<Vec<f64>>,
observed_at_us: u64,
) -> bool {
bootstrap_background_match(field, frame_history, observed_at_us)
.is_some_and(|result| result.matches_empty)
}
/// Score the current canonical runtime window against the learned empty-room
/// residual references. Legacy models without scalar references preserve the
/// binary match but intentionally return no score.
pub fn bootstrap_background_match(
field: &FieldModel,
frame_history: &VecDeque<Vec<f64>>,
observed_at_us: u64,
) -> Option<BootstrapBackgroundMatch> {
if frame_history.len() < OCCUPANCY_WINDOW
|| field.check_freshness(observed_at_us) != CalibrationStatus::Fresh
{
return None;
}
let frames: Vec<Vec<f64>> = frame_history
.iter()
.rev()
.take(OCCUPANCY_WINDOW)
.map(|frame| CALIB_NORMALIZER.resample_to_canonical(frame))
.collect();
let result = field.empty_room_match(&frames)?;
Some(BootstrapBackgroundMatch {
matches_empty: result.matches_empty,
score: result.score,
residual_energy: result.residual_energy,
residual_energy_threshold: result.residual_energy_threshold,
window_size: result.window_size,
reference_window_count: result.reference_window_count,
})
}
/// Produce a fail-closed occupancy result from a fresh calibrated model.
pub fn calibrated_occupancy(
field: &FieldModel,
frame_history: &VecDeque<Vec<f64>>,
observed_at_us: u64,
) -> Option<CalibratedOccupancy> {
if field.check_freshness(observed_at_us) != CalibrationStatus::Fresh {
return None;
}
let frames: Vec<Vec<f64>> = frame_history
.iter()
.rev()
.take(OCCUPANCY_WINDOW)
.map(|frame| CALIB_NORMALIZER.resample_to_canonical(frame))
.collect();
if frames.is_empty() {
return None;
}
if let Ok(person_count) = field.estimate_occupancy(&frames) {
return Some(CalibratedOccupancy {
person_count: person_count.min(MAX_SINGLE_LINK_OCCUPANCY),
method: CalibratedOccupancyMethod::Eigenvalue,
});
}
Some(CalibratedOccupancy {
person_count: perturbation_occupancy(field, &frames)?,
method: CalibratedOccupancyMethod::PerturbationEnergy,
})
}
/// Feed the latest frame to the FieldModel during calibration collection.
///
/// Acts while the model is `Uncalibrated` or `Collecting`. The first fed frame
@@ -174,6 +331,33 @@ pub fn parse_node_positions(input: &str) -> Vec<[f32; 3]> {
mod tests {
use super::*;
fn fresh_test_model() -> FieldModel {
let config = FieldModelConfig {
n_links: 1,
n_subcarriers: 56,
n_modes: 3,
min_calibration_frames: 10,
min_calibration_duration_s: 0.0,
baseline_expiry_s: 2.0,
};
let mut field = FieldModel::new(config).expect("field model");
for index in 0..20 {
let phase = index as f64 * 0.1;
let observation = vec![(0..56)
.map(|subcarrier| {
1.0 + subcarrier as f64 * 0.01 + (phase + subcarrier as f64 * 0.03).sin() * 0.01
})
.collect()];
field
.feed_calibration(&observation)
.expect("calibration frame");
}
field
.finalize_calibration(1_000_000, 7)
.expect("finalize calibration");
field
}
#[test]
fn test_parse_node_positions() {
let positions = parse_node_positions("0,0,1.5;3,0,1.5;1.5,3,1.5");
@@ -263,4 +447,88 @@ mod tests {
"wide frame must accumulate, not be silently dropped"
);
}
#[test]
fn calibrated_occupancy_is_fresh_model_only_and_normalizes_runtime_frames() {
let field = fresh_test_model();
let mut history = VecDeque::new();
history.push_back(vec![1.02; 128]);
let result =
calibrated_occupancy(&field, &history, 1_500_000).expect("fresh calibrated inference");
assert!(result.person_count <= MAX_SINGLE_LINK_OCCUPANCY);
assert!(matches!(
result.method,
CalibratedOccupancyMethod::Eigenvalue | CalibratedOccupancyMethod::PerturbationEnergy
));
assert_eq!(calibrated_occupancy(&field, &history, 4_000_000), None);
assert_eq!(
calibrated_occupancy(&field, &VecDeque::new(), 1_500_000),
None
);
}
#[test]
fn adaptive_runtime_reference_scores_empty_and_rejects_shift() {
let config = FieldModelConfig {
n_links: 1,
n_subcarriers: 56,
n_modes: 3,
min_calibration_frames: 500,
min_calibration_duration_s: 0.0,
baseline_expiry_s: 60.0,
};
let mut field = FieldModel::new(config).expect("field model");
let mut calibration = Vec::new();
for frame_index in 0..600 {
let time = frame_index as f64 * 0.071;
let frame: Vec<f64> = (0..56)
.map(|subcarrier| {
let carrier = subcarrier as f64;
18.0 + carrier * 0.08
+ (time + carrier * 0.13).sin() * 0.8
+ (time * 0.37 + carrier * 0.031).cos() * 0.35
})
.collect();
field.feed_calibration(&[frame.clone()]).unwrap();
calibration.push(frame);
}
field.finalize_calibration(1_000_000, 7).unwrap();
let empty_history: VecDeque<Vec<f64>> = calibration[550..600].iter().cloned().collect();
let empty = calibrated_occupancy(&field, &empty_history, 1_500_000)
.expect("adaptive perturbation result");
assert_eq!(empty.person_count, 0);
let background = bootstrap_background_match(&field, &empty_history, 1_500_000)
.expect("background score");
assert!(background.matches_empty);
assert_eq!(background.reference_window_count, 12);
assert!(background
.score
.is_some_and(|score| (0.5..=1.0).contains(&score)));
let short_history: VecDeque<Vec<f64>> = empty_history.iter().take(49).cloned().collect();
assert!(!bootstrap_empty_prior_applies(
&field,
&short_history,
1_500_000
));
let shifted_history: VecDeque<Vec<f64>> = calibration[550..600]
.iter()
.map(|frame| {
frame
.iter()
.enumerate()
.map(|(index, value)| value + if index % 3 == 0 { 8.0 } else { -5.0 })
.collect()
})
.collect();
let shifted = calibrated_occupancy(&field, &shifted_history, 1_500_000)
.expect("shifted perturbation result");
assert!(shifted.person_count >= 1);
let shifted_background = bootstrap_background_match(&field, &shifted_history, 1_500_000)
.expect("shifted background result");
assert!(!shifted_background.matches_empty);
assert_eq!(shifted_background.score, Some(0.0));
}
}
v2/crates/wifi-densepose-sensing-server/src/main.rs
+284 -43
View File
@@ -40,7 +40,9 @@ use wifi_densepose_sensing_server::{
trainer,
};
use wifi_densepose_sensing_server::bootstrap_baseline::{
self, BootstrapBaselineMetadata, BootstrapValidationSample, BOOTSTRAP_VALIDATION_SAMPLES,
self, BootstrapBaselineMetadata, BootstrapValidationSample,
BOOTSTRAP_VALIDATION_MIN_SPACING_MS, BOOTSTRAP_VALIDATION_SAMPLES,
BOOTSTRAP_VALIDATION_SAMPLE_TIMEOUT_MS,
};
// ADR-295 / ADR-297: canonical provenance state + per-node/room inference.
use wifi_densepose_sensing_server::inference::{fuse_room, NodeInference, RoomInference};
@@ -1715,12 +1717,27 @@ impl AppStateInner {
fn person_count(&self) -> usize {
match self.field_model.as_ref() {
Some(fm) => {
// Prefer global frame_history (populated by wifi/simulate paths).
// Fall back to freshest per-node history (populated by ESP32 paths).
let history = if !self.frame_history.is_empty() {
// A single-link model may score only the source node it was
// calibrated against. Applying one node's baseline to a
// different radio creates deterministic false occupancy from
// hardware-specific amplitude offsets.
let bound_source_node_id = (self.calibration_source_node_ids.len() == 1)
.then(|| self.calibration_source_node_ids.iter().next().copied())
.flatten()
.or_else(|| {
self.bootstrap_baseline.as_ref().and_then(|metadata| {
(metadata.source_node_ids.len() == 1)
.then_some(metadata.source_node_ids[0])
})
});
let history = if let Some(node_id) = bound_source_node_id {
self.node_states
.get(&node_id)
.map(|state| &state.frame_history)
.unwrap_or(&self.frame_history)
} else if !self.frame_history.is_empty() {
&self.frame_history
} else {
// Find the node with the most recent frame
self.node_states
.values()
.filter(|ns| !ns.frame_history.is_empty())
@@ -1739,6 +1756,36 @@ impl AppStateInner {
}
}
/// A restored startup image has negative-only authority. It may suppress
/// an empty-room false positive after one complete runtime window, but it
/// cannot establish positive presence or publish vital signs.
fn bootstrap_empty_prior_applies(&self, observed_at_unix_ms: u64) -> bool {
self.bootstrap_background_match(observed_at_unix_ms)
.is_some_and(|result| result.matches_empty)
}
/// Return current background conformance for the exact source node bound
/// into the restored image. This remains diagnostic and negative-only.
fn bootstrap_background_match(
&self,
observed_at_unix_ms: u64,
) -> Option<field_bridge::BootstrapBackgroundMatch> {
if !self.bootstrap_baseline_active {
return None;
}
let field_model = self.field_model.as_ref()?;
let metadata = self.bootstrap_baseline.as_ref()?;
let [source_node_id] = metadata.source_node_ids.as_slice() else {
return None;
};
let history = &self.node_states.get(source_node_id)?.frame_history;
field_bridge::bootstrap_background_match(
field_model,
history,
observed_at_unix_ms.saturating_mul(1_000),
)
}
fn effective_source(&self) -> String {
if self.source == "esp32" {
if let Some(last) = self.last_esp32_frame {
@@ -6041,6 +6088,50 @@ fn vitals_for_publication(
})
}
fn edge_vitals_message_for_publication(
raw: &Esp32VitalsPacket,
published_vitals: Option<&VitalSigns>,
bootstrap_empty: bool,
explicit_calibration_fresh: bool,
person_count: usize,
) -> serde_json::Value {
let numeric_vitals_authorized = published_vitals.is_some();
let abstention_reason = if numeric_vitals_authorized {
None
} else if bootstrap_empty {
Some("bootstrap_empty_background")
} else if !explicit_calibration_fresh {
Some("explicit_calibration_required")
} else if person_count != 1 {
Some("exactly_one_occupant_required")
} else {
Some("vital_quality_gate_failed")
};
let effective_presence = raw.presence && !bootstrap_empty;
let effective_motion = raw.motion && !bootstrap_empty;
let effective_person_count = if bootstrap_empty { 0 } else { person_count };
serde_json::json!({
"type": "edge_vitals",
"node_id": raw.node_id,
"presence": effective_presence,
"fall_detected": raw.fall_detected && !bootstrap_empty,
"motion": effective_motion,
"breathing_rate_bpm": published_vitals
.and_then(|vitals| vitals.breathing_rate_bpm),
"heartrate_bpm": published_vitals
.and_then(|vitals| vitals.heart_rate_bpm),
"n_persons": effective_person_count,
"person_count_valid": raw.person_count_valid && !bootstrap_empty,
"motion_energy": if bootstrap_empty { 0.0 } else { raw.motion_energy },
"presence_score": if bootstrap_empty { 0.0 } else { raw.presence_score },
"rssi": raw.rssi,
"calibrated_evidence_authorized": explicit_calibration_fresh,
"numeric_vitals_authorized": numeric_vitals_authorized,
"abstention_reason": abstention_reason,
})
}
fn opaque_calibration_model_id() -> String {
let mut bytes = [0_u8; 16];
OsRng.fill_bytes(&mut bytes);
@@ -6048,10 +6139,26 @@ fn opaque_calibration_model_id() -> String {
format!("cal-model-{suffix}")
}
fn calibration_source_accepts(
source_node_ids: &std::collections::BTreeSet<u8>,
node_id: u8,
) -> bool {
source_node_ids.is_empty() || source_node_ids.contains(&node_id)
}
#[cfg(test)]
mod bootstrap_vital_publication_tests {
use super::*;
#[test]
fn single_link_calibration_binds_to_first_source() {
let mut sources = std::collections::BTreeSet::new();
assert!(calibration_source_accepts(&sources, 5));
sources.insert(5);
assert!(calibration_source_accepts(&sources, 5));
assert!(!calibration_source_accepts(&sources, 3));
}
fn strong_candidates() -> VitalSigns {
VitalSigns {
breathing_rate_bpm: Some(15.0),
@@ -6083,6 +6190,61 @@ mod bootstrap_vital_publication_tests {
candidates.signal_quality = 0.2;
assert!(vitals_for_publication(&candidates, true, 1).is_none());
}
fn raw_edge_vitals() -> Esp32VitalsPacket {
Esp32VitalsPacket {
node_id: 5,
presence: true,
fall_detected: true,
motion: true,
breathing_rate_bpm: 15.0,
heartrate_bpm: 72.0,
rssi: -42,
n_persons: 1,
person_count_valid: true,
motion_energy: 0.8,
presence_score: 0.9,
timestamp_ms: 42,
}
}
#[test]
fn bootstrap_empty_background_suppresses_legacy_edge_vitals() {
let message = edge_vitals_message_for_publication(
&raw_edge_vitals(),
None,
true,
false,
0,
);
assert_eq!(message["presence"], false);
assert_eq!(message["motion"], false);
assert_eq!(message["fall_detected"], false);
assert_eq!(message["n_persons"], 0);
assert!(message["breathing_rate_bpm"].is_null());
assert!(message["heartrate_bpm"].is_null());
assert_eq!(message["numeric_vitals_authorized"], false);
assert_eq!(message["abstention_reason"], "bootstrap_empty_background");
}
#[test]
fn explicit_single_occupant_may_publish_legacy_edge_vitals() {
let published = vitals_for_publication(&strong_candidates(), true, 1).unwrap();
let message = edge_vitals_message_for_publication(
&raw_edge_vitals(),
Some(&published),
false,
true,
1,
);
assert_eq!(message["presence"], true);
assert_eq!(message["fall_detected"], true);
assert_eq!(message["n_persons"], 1);
assert_eq!(message["breathing_rate_bpm"], 15.0);
assert_eq!(message["heartrate_bpm"], 72.0);
assert_eq!(message["numeric_vitals_authorized"], true);
assert!(message["abstention_reason"].is_null());
}
}
// ── Field model calibration endpoints (eigenvalue person counting) ──────────
@@ -6194,6 +6356,17 @@ async fn calibration_stop(State(state): State<SharedState>) -> Json<serde_json::
async fn calibration_status(State(state): State<SharedState>) -> Json<serde_json::Value> {
let s = state.read().await;
let observed_at_unix_ms = chrono::Utc::now().timestamp_millis().max(0) as u64;
let bootstrap_background_match = s.bootstrap_background_match(observed_at_unix_ms);
let runtime_reference = s.field_model.as_ref().and_then(|model| {
model.modes().map(|modes| serde_json::json!({
"window_size": modes.baseline_runtime_window_size,
"significant_eigenvalue_p95": modes.baseline_runtime_eigenvalue_count,
"residual_energy_threshold": modes.empty_room_residual_energy_threshold,
"residual_reference_window_count": modes.empty_room_residual_energy_reference.len(),
"raw_calibration_frames_persisted": false,
}))
});
let (active, status, frame_count, min_frames, elapsed_s, frames_per_second, min_duration_s) =
s.field_model.as_ref().map_or(
(false, "none".to_string(), 0, 0, 0.0, 0.0, 0.0),
@@ -6219,6 +6392,7 @@ async fn calibration_status(State(state): State<SharedState>) -> Json<serde_json
"min_duration_s": min_duration_s,
"model_id": s.calibration_model_id,
"source_node_ids": s.calibration_source_node_ids,
"runtime_reference": runtime_reference,
"binding_mode": if s.bootstrap_baseline_active { "bootstrap_only" } else if active { "runtime" } else { "none" },
"bootstrap_baseline": s.bootstrap_baseline.as_ref().map(|metadata| serde_json::json!({
"stored": true,
@@ -6229,6 +6403,15 @@ async fn calibration_status(State(state): State<SharedState>) -> Json<serde_json
"created_at_unix_ms": metadata.created_at_unix_ms,
"expires_at_unix_ms": metadata.expires_at_unix_ms,
"content_sha256": metadata.content_sha256,
"background_match": bootstrap_background_match.map(|result| serde_json::json!({
"state": if result.matches_empty { "matched" } else { "changed" },
"matches_empty": result.matches_empty,
"score": result.score,
"residual_energy": result.residual_energy,
"residual_energy_threshold": result.residual_energy_threshold,
"window_size": result.window_size,
"reference_window_count": result.reference_window_count,
})),
"calibrated_evidence_authorized": false,
"numeric_vitals_authorized": false,
})).unwrap_or_else(|| serde_json::json!({
@@ -6274,8 +6457,11 @@ async fn calibration_promote_bootstrap(
let mut samples = Vec::with_capacity(BOOTSTRAP_VALIDATION_SAMPLES);
let mut last_tick = None;
let mut next_sample_at = tokio::time::Instant::now();
for _ in 0..BOOTSTRAP_VALIDATION_SAMPLES {
let interval_deadline = tokio::time::Instant::now() + Duration::from_secs(1);
tokio::time::sleep_until(next_sample_at).await;
let interval_deadline = tokio::time::Instant::now()
+ Duration::from_millis(BOOTSTRAP_VALIDATION_SAMPLE_TIMEOUT_MS);
let mut sample = BootstrapValidationSample {
fresh_tick: false,
calibrated_empty: false,
@@ -6310,7 +6496,8 @@ async fn calibration_promote_bootstrap(
tokio::time::sleep(Duration::from_millis(50)).await;
}
samples.push(sample);
tokio::time::sleep_until(interval_deadline).await;
next_sample_at = tokio::time::Instant::now()
+ Duration::from_millis(BOOTSTRAP_VALIDATION_MIN_SPACING_MS);
}
let validation = bootstrap_baseline::evaluate_validation(&samples);
@@ -6341,11 +6528,11 @@ async fn calibration_promote_bootstrap(
}));
};
let source_node_ids: Vec<u8> = s.calibration_source_node_ids.iter().copied().collect();
if source_node_ids.is_empty() || source_node_ids.len() > 16 {
if source_node_ids.len() != 1 {
return Json(serde_json::json!({
"success": false,
"error_code": "calibration_source_nodes_missing",
"error": "The active calibration must contain between 1 and 16 contributing ESP32 nodes.",
"error": "The single-link startup baseline must contain exactly one contributing ESP32 node.",
}));
}
let Some(field_model) = s.field_model.as_ref() else {
@@ -6825,6 +7012,7 @@ async fn mesh_endpoint(State(state): State<SharedState>) -> Json<serde_json::Val
async fn nodes_endpoint(State(state): State<SharedState>) -> Json<serde_json::Value> {
let s = state.read().await;
let now = std::time::Instant::now();
let observed_at_unix_ms = chrono::Utc::now().timestamp_millis().max(0) as u64;
let nodes: Vec<serde_json::Value> = s
.node_states
.iter()
@@ -6836,13 +7024,24 @@ async fn nodes_endpoint(State(state): State<SharedState>) -> Json<serde_json::Va
let stale = elapsed_ms > 5000;
let status = if stale { "stale" } else { "active" };
let rssi = ns.rssi_history.back().copied().unwrap_or(-90.0);
let bootstrap_empty = s.bootstrap_baseline_active
&& s.bootstrap_baseline.as_ref().is_some_and(|metadata| {
metadata.source_node_ids.as_slice() == [id]
})
&& s.field_model.as_ref().is_some_and(|field_model| {
field_bridge::bootstrap_empty_prior_applies(
field_model,
&ns.frame_history,
observed_at_unix_ms.saturating_mul(1_000),
)
});
serde_json::json!({
"node_id": id,
"status": status,
"last_seen_ms": elapsed_ms,
"rssi_dbm": rssi,
"motion_level": &ns.current_motion_level,
"person_count": ns.prev_person_count,
"motion_level": if bootstrap_empty { "absent" } else { &ns.current_motion_level },
"person_count": if bootstrap_empty { 0 } else { ns.prev_person_count },
"person_count_valid": ns.edge_vitals
.as_ref()
.map(|vitals| vitals.person_count_valid),
@@ -7088,24 +7287,6 @@ async fn udp_receiver_task(
vitals.presence
);
let mut s = state.write().await;
// Broadcast vitals via WebSocket.
if let Ok(json) = serde_json::to_string(&serde_json::json!({
"type": "edge_vitals",
"node_id": vitals.node_id,
"presence": vitals.presence,
"fall_detected": vitals.fall_detected,
"motion": vitals.motion,
"breathing_rate_bpm": vitals.breathing_rate_bpm,
"heartrate_bpm": vitals.heartrate_bpm,
"n_persons": vitals.n_persons,
"person_count_valid": vitals.person_count_valid,
"motion_energy": vitals.motion_energy,
"presence_score": vitals.presence_score,
"rssi": vitals.rssi,
})) {
let _ = s.tx.send(json);
}
// Issue #323: Also emit a sensing_update so the UI renders
// detections for ESP32 nodes running the edge DSP pipeline
// (Tier 2+). Without this, vitals arrive but the UI shows
@@ -7153,9 +7334,18 @@ async fn udp_receiver_task(
0.05
};
// Aggregate person count: gate on presence first (matching WiFi path).
let observed_at_unix_ms =
chrono::Utc::now().timestamp_millis().max(0) as u64;
let bootstrap_empty =
s.bootstrap_empty_prior_applies(observed_at_unix_ms);
// A startup prior has negative-only authority. Once a full
// runtime window matches the stored background, raw edge
// presence cannot force the count back to one.
let now = std::time::Instant::now();
let total_persons = if vitals.presence {
let total_persons = if bootstrap_empty {
0
} else if vitals.presence {
let dedup = s.dedup_factor;
let (fused, fallback_count) = multistatic_bridge::fuse_or_fallback(
&s.multistatic_fuser,
@@ -7208,10 +7398,16 @@ async fn udp_receiver_task(
.get(&node_id)
.map(|ns| ns.frame_history.clone())
{
let accepted = if let Some(ref mut fm) = s.field_model {
let before = fm.calibration_frame_count();
field_bridge::maybe_feed_calibration(fm, &frame_history);
fm.calibration_frame_count() > before
let source_allowed =
calibration_source_accepts(&s.calibration_source_node_ids, node_id);
let accepted = if source_allowed {
if let Some(ref mut fm) = s.field_model {
let before = fm.calibration_frame_count();
field_bridge::maybe_feed_calibration(fm, &frame_history);
fm.calibration_frame_count() > before
} else {
false
}
} else {
false
};
@@ -7278,7 +7474,15 @@ async fn udp_receiver_task(
// node's own reading no longer overwrites the room's. The
// old ad-hoc "boost confidence by node count" is replaced by
// `room_inference`'s freshness-weighted multi-node confidence.
let classification = debounce_room_classification(&mut s, &room_inference);
let classification = if bootstrap_empty {
ClassificationInfo {
motion_level: "absent".to_string(),
presence: false,
confidence: 0.0,
}
} else {
debounce_room_classification(&mut s, &room_inference)
};
let signal_field = generate_signal_field(
fused_features.mean_rssi,
@@ -7306,6 +7510,21 @@ async fn udp_receiver_task(
total_persons,
);
// Keep the legacy edge_vitals stream aligned with the
// governed sensing envelope. Raw numeric candidates are
// never published before the explicit calibration and
// single-occupant gates have passed.
let edge_vitals_message = edge_vitals_message_for_publication(
&vitals,
published_vitals.as_ref(),
bootstrap_empty,
explicit_calibration_fresh,
total_persons,
);
if let Ok(json) = serde_json::to_string(&edge_vitals_message) {
let _ = s.tx.send(json);
}
let mut update = SensingUpdate {
msg_type: "sensing_update".to_string(),
timestamp: chrono::Utc::now().timestamp_millis() as f64 / 1000.0,
@@ -7645,9 +7864,17 @@ async fn udp_receiver_task(
0.05
};
// Aggregate person count: gate on presence first (matching WiFi path).
let observed_at_unix_ms =
chrono::Utc::now().timestamp_millis().max(0) as u64;
let bootstrap_empty =
s.bootstrap_empty_prior_applies(observed_at_unix_ms);
// A restored prior can suppress a background-only raw
// classification. It cannot authorize positive presence.
let now = std::time::Instant::now();
let total_persons = if classification.presence {
let total_persons = if bootstrap_empty {
0
} else if classification.presence {
let dedup = s.dedup_factor;
let (fused, fallback_count) = multistatic_bridge::fuse_or_fallback(
&s.multistatic_fuser,
@@ -7700,10 +7927,16 @@ async fn udp_receiver_task(
.get(&node_id)
.map(|ns| ns.frame_history.clone())
{
let accepted = if let Some(ref mut fm) = s.field_model {
let before = fm.calibration_frame_count();
field_bridge::maybe_feed_calibration(fm, &frame_history);
fm.calibration_frame_count() > before
let source_allowed =
calibration_source_accepts(&s.calibration_source_node_ids, node_id);
let accepted = if source_allowed {
if let Some(ref mut fm) = s.field_model {
let before = fm.calibration_frame_count();
field_bridge::maybe_feed_calibration(fm, &frame_history);
fm.calibration_frame_count() > before
} else {
false
}
} else {
false
};
@@ -7763,7 +7996,15 @@ async fn udp_receiver_task(
active_nodes.iter().filter_map(|ni| ni.node_inference.as_ref()),
NODE_STALE_AFTER_MS,
);
let room_classification = debounce_room_classification(&mut s, &room_inference);
let room_classification = if bootstrap_empty {
ClassificationInfo {
motion_level: "absent".to_string(),
presence: false,
confidence: 0.0,
}
} else {
debounce_room_classification(&mut s, &room_inference)
};
let explicit_calibration_fresh = !s.bootstrap_baseline_active
&& s.field_model.as_ref().is_some_and(|model| {
matches!(model.status(), CalibrationStatus::Fresh)
v2/crates/wifi-densepose-signal/src/ruvsense/field_model.rs
+555 -35
View File
@@ -7,8 +7,8 @@
//! (the residual).
//!
//! # Algorithm
//! 1. Collect CSI during empty-room calibration (>=10 min wall-clock; the
//! frame target is this duration at the assumed 20 Hz single-node rate)
//! 1. Collect CSI during empty-room calibration (>=10 min wall-clock and at
//! least twenty complete runtime-sized background windows)
//! 2. Compute per-link baseline mean (Welford online accumulator)
//! 3. Decompose covariance via SVD to extract environmental modes
//! 4. At runtime: observation - baseline, project out top-K modes, keep residual
@@ -24,6 +24,7 @@ use ndarray_linalg::Eigh;
#[cfg(feature = "eigenvalue")]
use ndarray_linalg::UPLO;
use serde::{Deserialize, Serialize};
use std::collections::VecDeque;
// ---------------------------------------------------------------------------
// Calibration window constants
@@ -33,21 +34,30 @@ use serde::{Deserialize, Serialize};
///
/// The Welford statistics exist to absorb slow environmental variation —
/// HVAC cycles, thermal drift — so the calibration gate is expressed in
/// wall-clock time. The historical frame target of 12,000 is this duration at
/// the assumed single-node rate. Every accepted node packet advances the same
/// field model, so N nodes reach any frame target N times faster; frames alone
/// therefore cannot gate calibration (#1756).
/// wall-clock time. Every accepted node packet advances the same field model,
/// so N nodes reach any frame target N times faster; frames alone therefore
/// cannot gate calibration (#1756).
pub const CALIBRATION_DURATION_S: f64 = 600.0;
/// Assumed per-node CSI rate (Hz) used to relate the frame target to
/// [`CALIBRATION_DURATION_S`] ("10 min at 20 Hz").
pub const ASSUMED_CALIBRATION_RATE_HZ: f64 = 20.0;
/// A recent window whose mean remains within this residual energy of the
/// empty room manifold is treated as background before eigenvalue counting.
#[cfg(feature = "eigenvalue")]
const EMPTY_ROOM_RESIDUAL_ENERGY_MAX: f64 = 1.0;
/// Runtime occupancy is evaluated over fifty frames. Keep ten raw windows
/// from the end of calibration for covariance rank learning, plus bounded
/// privacy reduced window means spanning the full capture for residual
/// conformance scoring.
const RUNTIME_OCCUPANCY_WINDOW: usize = 50;
const CALIBRATION_REFERENCE_FRAMES: usize = RUNTIME_OCCUPANCY_WINDOW * 10;
/// Minimum independent runtime sized means required to learn an empirical
/// empty room residual distribution. Wall clock duration remains the primary
/// coverage gate, so slow hardware is not forced to imitate a nominal rate.
pub const MIN_BACKGROUND_REFERENCE_WINDOWS: usize = 20;
pub const MIN_CALIBRATION_FRAMES: usize =
RUNTIME_OCCUPANCY_WINDOW * MIN_BACKGROUND_REFERENCE_WINDOWS;
const CALIBRATION_BACKGROUND_WINDOWS_MAX: usize = 512;
const EMPTY_ROOM_RESIDUAL_MARGIN: f64 = 1.5;
// ---------------------------------------------------------------------------
// Error types
// ---------------------------------------------------------------------------
@@ -276,12 +286,11 @@ pub struct FieldModelConfig {
pub n_subcarriers: usize,
/// Number of environmental modes to retain (K). Max 5.
pub n_modes: usize,
/// Minimum calibration frames before baseline is valid. Derived from the
/// intended wall-clock window at the assumed single-node rate
/// (`CALIBRATION_DURATION_S * ASSUMED_CALIBRATION_RATE_HZ` = 12,000).
/// A frame count alone is not sufficient: N nodes streaming in aggregate
/// reach it in 1/N of the window, so `min_calibration_duration_s` also
/// gates finalization (#1756).
/// Minimum calibration frames before baseline is valid. This is the
/// runtime window size times the minimum number of independent background
/// references. A frame count alone is not sufficient: N nodes streaming
/// in aggregate reach it in 1/N of the window, so
/// `min_calibration_duration_s` also gates finalization (#1756).
pub min_calibration_frames: usize,
/// Minimum wall-clock duration of the calibration window in seconds.
/// Slow environmental variation (HVAC cycles, thermal drift) can only be
@@ -298,8 +307,7 @@ impl Default for FieldModelConfig {
n_links: 6,
n_subcarriers: 56,
n_modes: 3,
min_calibration_frames: (CALIBRATION_DURATION_S * ASSUMED_CALIBRATION_RATE_HZ)
as usize,
min_calibration_frames: MIN_CALIBRATION_FRAMES,
min_calibration_duration_s: CALIBRATION_DURATION_S,
baseline_expiry_s: 86_400.0,
}
@@ -336,6 +344,37 @@ pub struct FieldNormalMode {
/// per-window sample size. Defaults to 0.0 in the diagonal-fallback path.
/// Issue #942.
pub baseline_noise_var: f64,
/// Empty room significant eigenvalue count learned from runtime sized
/// windows. The percentile is stored instead of raw CSI. `None` identifies
/// snapshots created before runtime window calibration was introduced.
#[serde(default)]
pub baseline_runtime_eigenvalue_count: Option<usize>,
/// Number of frames used for the runtime window reference above.
#[serde(default)]
pub baseline_runtime_window_size: Option<usize>,
/// Hardware adaptive upper bound for the residual energy of an empty room
/// window mean. Only this aggregate boundary is persisted.
#[serde(default)]
pub empty_room_residual_energy_threshold: Option<f64>,
/// Sorted residual energies of privacy reduced runtime window means sampled
/// across the full calibration. Raw CSI is never persisted. These scalar
/// references support an empirical background match score without turning
/// an empty room model into positive person evidence.
#[serde(default)]
pub empty_room_residual_energy_reference: Vec<f64>,
}
/// A bounded comparison between one runtime window and the learned empty room
/// residual distribution. `score` is an empirical conformance score, not a
/// probability that a person is absent.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct EmptyRoomMatch {
pub matches_empty: bool,
pub score: Option<f64>,
pub residual_energy: f64,
pub residual_energy_threshold: f64,
pub window_size: usize,
pub reference_window_count: usize,
}
/// Body perturbation extracted from a CSI observation.
@@ -405,6 +444,16 @@ pub struct FieldModel {
/// Monotonic timestamp of the first accepted calibration frame of the
/// current session; `None` until collection starts (#1756).
calibration_started: Option<std::time::Instant>,
/// Bounded, memory only tail used to learn inference sized empty room
/// covariance statistics. It is never included in a snapshot.
calibration_reference_tail: VecDeque<Vec<Vec<f64>>>,
/// Per link sum for the current privacy reduced runtime sized window.
calibration_window_sum: Vec<Vec<f64>>,
/// Number of accepted observations in `calibration_window_sum`.
calibration_window_count: usize,
/// Runtime sized per link means spanning the full calibration. This is
/// bounded, memory only, and reduced to scalar residuals at finalization.
calibration_window_means: VecDeque<Vec<Vec<f64>>>,
}
/// Diagonal variance fallback for when full covariance SVD is unavailable.
@@ -460,6 +509,133 @@ fn diagonal_fallback(
(mode_energies, environmental_modes, baseline_count)
}
#[cfg(feature = "eigenvalue")]
fn significant_eigenvalue_count(
frames: &[Vec<f64>],
n_subcarriers: usize,
baseline_noise_var: f64,
) -> Option<usize> {
if frames.len() < 10 {
return None;
}
let mut mean = vec![0.0_f64; n_subcarriers];
let mut count = 0_usize;
for frame in frames {
if frame.len() >= n_subcarriers {
for index in 0..n_subcarriers {
mean[index] += frame[index];
}
count += 1;
}
}
if count < 2 {
return None;
}
for value in &mut mean {
*value /= count as f64;
}
let mut covariance = Array2::<f64>::zeros((n_subcarriers, n_subcarriers));
for frame in frames {
if frame.len() >= n_subcarriers {
for row in 0..n_subcarriers {
let centered_row = frame[row] - mean[row];
for column in row..n_subcarriers {
let value = centered_row * (frame[column] - mean[column]);
covariance[[row, column]] += value;
if row != column {
covariance[[column, row]] += value;
}
}
}
}
}
covariance *= 1.0 / (count as f64 - 1.0);
let (eigenvalues, _) = covariance.eigh(UPLO::Upper).ok()?;
let mut positive: Vec<f64> = eigenvalues
.iter()
.copied()
.filter(|value| *value > 1e-10)
.collect();
positive.sort_by(|left, right| left.partial_cmp(right).unwrap_or(std::cmp::Ordering::Equal));
let local_noise_var = if positive.len() >= 4 {
let half = positive.len() / 2;
positive[..half].iter().sum::<f64>() / half as f64
} else {
positive.first().copied()?
};
let noise_var = if baseline_noise_var > 0.0 {
local_noise_var.max(baseline_noise_var)
} else {
local_noise_var
};
let ratio = n_subcarriers as f64 / count as f64;
let threshold = noise_var * (1.0 + ratio.sqrt()).powi(2);
Some(
eigenvalues
.iter()
.filter(|&&value| value > threshold)
.count(),
)
}
fn residual_energy_for_link(
modes: &FieldNormalMode,
link_index: usize,
observation: &[f64],
) -> Option<f64> {
let baseline = modes.baseline.get(link_index)?;
if observation.len() != baseline.len() {
return None;
}
let mut residual: Vec<f64> = observation
.iter()
.zip(baseline.iter())
.map(|(value, reference)| value - reference)
.collect();
for mode in &modes.environmental_modes {
let projection: f64 = residual
.iter()
.zip(mode.iter())
.map(|(value, basis)| value * basis)
.sum();
for (value, basis) in residual.iter_mut().zip(mode.iter()) {
*value -= projection * basis;
}
}
Some(
residual
.iter()
.map(|value| value * value)
.sum::<f64>()
.sqrt(),
)
}
fn percentile_95(mut values: Vec<f64>) -> Option<f64> {
values.retain(|value| value.is_finite());
values.sort_by(|left, right| left.partial_cmp(right).unwrap_or(std::cmp::Ordering::Equal));
let index = values
.len()
.saturating_mul(95)
.div_ceil(100)
.saturating_sub(1);
values.get(index).copied()
}
#[cfg(feature = "eigenvalue")]
fn percentile_95_usize(mut values: Vec<usize>) -> Option<usize> {
values.sort_unstable();
let index = values
.len()
.saturating_mul(95)
.div_ceil(100)
.saturating_sub(1);
values.get(index).copied()
}
impl FieldModel {
/// Create a new field model for the given configuration.
pub fn new(config: FieldModelConfig) -> Result<Self, FieldModelError> {
@@ -480,6 +656,7 @@ impl FieldModel {
let link_stats = (0..config.n_links)
.map(|_| LinkBaselineStats::new(config.n_subcarriers))
.collect();
let calibration_window_sum = vec![vec![0.0_f64; config.n_subcarriers]; config.n_links];
Ok(Self {
config,
@@ -490,6 +667,10 @@ impl FieldModel {
covariance_sum: None,
covariance_count: 0,
calibration_started: None,
calibration_reference_tail: VecDeque::with_capacity(CALIBRATION_REFERENCE_FRAMES),
calibration_window_sum,
calibration_window_count: 0,
calibration_window_means: VecDeque::with_capacity(CALIBRATION_BACKGROUND_WINDOWS_MAX),
})
}
@@ -503,6 +684,14 @@ impl FieldModel {
self.modes.as_ref()
}
/// Hardware adaptive empty room boundary learned at finalization. Returns
/// `None` for legacy snapshots that predate runtime window calibration.
pub fn empty_room_residual_energy_threshold(&self) -> Option<f64> {
self.modes
.as_ref()
.and_then(|modes| modes.empty_room_residual_energy_threshold)
}
/// Export aggregate model state suitable for local restart persistence.
/// Raw calibration observations and Welford accumulators are not exported.
pub fn export_snapshot(&self) -> Result<FieldModelSnapshotV1, FieldModelError> {
@@ -561,7 +750,26 @@ impl FieldModel {
&& (0.0..=10.0).contains(&modes.variance_explained)
&& modes.baseline_noise_var.is_finite()
&& modes.baseline_noise_var >= 0.0
&& modes.baseline_eigenvalue_count <= config.n_subcarriers;
&& modes.baseline_eigenvalue_count <= config.n_subcarriers
&& modes
.baseline_runtime_eigenvalue_count
.is_none_or(|count| count <= config.n_subcarriers)
&& modes
.baseline_runtime_window_size
.is_none_or(|size| (10..=CALIBRATION_REFERENCE_FRAMES).contains(&size))
&& modes
.empty_room_residual_energy_threshold
.is_none_or(|threshold| threshold.is_finite() && threshold >= 0.0)
&& modes.empty_room_residual_energy_reference.len()
<= CALIBRATION_BACKGROUND_WINDOWS_MAX
&& modes
.empty_room_residual_energy_reference
.iter()
.all(|value| value.is_finite() && *value >= 0.0)
&& modes
.empty_room_residual_energy_reference
.windows(2)
.all(|window| window[0] <= window[1]);
if !baseline_shape_valid || !mode_shape_valid || !values_valid {
return Err(FieldModelError::InvalidConfig(
"snapshot modes are malformed or non finite".into(),
@@ -678,6 +886,39 @@ impl FieldModel {
// Count once per frame (not per link) for correct MP ratio
self.covariance_count += 1;
if self.calibration_reference_tail.len() == CALIBRATION_REFERENCE_FRAMES {
self.calibration_reference_tail.pop_front();
}
self.calibration_reference_tail
.push_back(observations.to_vec());
for (link_sum, observation) in self
.calibration_window_sum
.iter_mut()
.zip(observations.iter())
{
for (sum, value) in link_sum.iter_mut().zip(observation.iter()) {
*sum += value;
}
}
self.calibration_window_count += 1;
if self.calibration_window_count == RUNTIME_OCCUPANCY_WINDOW {
let divisor = self.calibration_window_count as f64;
let means: Vec<Vec<f64>> = self
.calibration_window_sum
.iter()
.map(|link_sum| link_sum.iter().map(|value| value / divisor).collect())
.collect();
if self.calibration_window_means.len() == CALIBRATION_BACKGROUND_WINDOWS_MAX {
self.calibration_window_means.pop_front();
}
self.calibration_window_means.push_back(means);
for link_sum in &mut self.calibration_window_sum {
link_sum.fill(0.0);
}
self.calibration_window_count = 0;
}
Ok(())
}
@@ -812,8 +1053,7 @@ impl FieldModel {
}
Err(_) => {
// Fallback to diagonal approximation on SVD failure
let (e, m, b) =
diagonal_fallback(&self.link_stats, n_sc, n_modes);
let (e, m, b) = diagonal_fallback(&self.link_stats, n_sc, n_modes);
(e, m, b, 0.0_f64)
}
}
@@ -865,7 +1105,7 @@ impl FieldModel {
0.0
};
let field_mode = FieldNormalMode {
let mut field_mode = FieldNormalMode {
baseline,
environmental_modes,
mode_energies,
@@ -874,15 +1114,142 @@ impl FieldModel {
geometry_hash,
baseline_eigenvalue_count: baseline_eig_count,
baseline_noise_var,
baseline_runtime_eigenvalue_count: None,
baseline_runtime_window_size: None,
empty_room_residual_energy_threshold: None,
empty_room_residual_energy_reference: Vec::new(),
};
// The full calibration covariance and a fifty frame runtime covariance
// have different Marcenko Pastur aspect ratios. This first version is
// deliberately single link: combining link specific residual scales
// requires a separate multistatic calibration model.
if self.config.n_links == 1 {
let reference_frames: Vec<Vec<f64>> = self
.calibration_reference_tail
.iter()
.filter_map(|observation| observation.first().cloned())
.collect();
let complete_windows: Vec<&[Vec<f64>]> = reference_frames
.chunks(RUNTIME_OCCUPANCY_WINDOW)
.filter(|window| window.len() == RUNTIME_OCCUPANCY_WINDOW)
.collect();
#[cfg(feature = "eigenvalue")]
{
let counts: Vec<usize> = complete_windows
.iter()
.filter_map(|window| {
significant_eigenvalue_count(window, n_sc, baseline_noise_var)
})
.collect();
field_mode.baseline_runtime_eigenvalue_count = percentile_95_usize(counts);
}
let mut residual_energies: Vec<f64> = self
.calibration_window_means
.iter()
.filter_map(|means| means.first())
.filter_map(|mean| residual_energy_for_link(&field_mode, 0, mean))
.filter(|value| value.is_finite() && *value >= 0.0)
.collect();
if residual_energies.is_empty() {
residual_energies = complete_windows
.iter()
.filter_map(|window| {
let mut mean = vec![0.0_f64; n_sc];
for frame in *window {
for (sum, value) in mean.iter_mut().zip(frame.iter()) {
*sum += value;
}
}
for value in &mut mean {
*value /= window.len() as f64;
}
residual_energy_for_link(&field_mode, 0, &mean)
})
.collect();
}
residual_energies.sort_by(|left, right| {
left.partial_cmp(right).unwrap_or(std::cmp::Ordering::Equal)
});
field_mode.empty_room_residual_energy_threshold =
percentile_95(residual_energies.clone()).map(|value| {
(value * EMPTY_ROOM_RESIDUAL_MARGIN).max(EMPTY_ROOM_RESIDUAL_ENERGY_MAX)
});
field_mode.empty_room_residual_energy_reference = residual_energies;
if field_mode.baseline_runtime_eigenvalue_count.is_some()
|| field_mode.empty_room_residual_energy_threshold.is_some()
{
field_mode.baseline_runtime_window_size = Some(RUNTIME_OCCUPANCY_WINDOW);
}
}
self.modes = Some(field_mode);
self.status = CalibrationStatus::Fresh;
self.last_calibration_us = timestamp_us;
self.calibration_reference_tail.clear();
self.calibration_window_means.clear();
for link_sum in &mut self.calibration_window_sum {
link_sum.fill(0.0);
}
self.calibration_window_count = 0;
Ok(self.modes.as_ref().unwrap())
}
/// Compare a runtime sized single link window with the learned empty room
/// distribution. The score is empirical conformance while the learned
/// boundary holds. Crossing the boundary returns zero. Legacy snapshots
/// without residual references retain binary suppression but no score.
pub fn empty_room_match(&self, recent_frames: &[Vec<f64>]) -> Option<EmptyRoomMatch> {
let modes = self.modes.as_ref()?;
if self.config.n_links != 1 {
return None;
}
let window_size = modes.baseline_runtime_window_size?;
if recent_frames.len() < window_size {
return None;
}
let mut mean = vec![0.0_f64; self.config.n_subcarriers];
for frame in recent_frames.iter().take(window_size) {
if frame.len() < self.config.n_subcarriers {
return None;
}
for (sum, value) in mean.iter_mut().zip(frame.iter()) {
*sum += value;
}
}
for value in &mut mean {
*value /= window_size as f64;
}
let residual_energy = residual_energy_for_link(modes, 0, &mean)?;
let residual_energy_threshold = modes
.empty_room_residual_energy_threshold?
.max(EMPTY_ROOM_RESIDUAL_ENERGY_MAX);
let matches_empty = residual_energy <= residual_energy_threshold;
let reference_window_count = modes.empty_room_residual_energy_reference.len();
let score = if !matches_empty {
Some(0.0)
} else if reference_window_count == 0 {
None
} else {
let at_or_below = modes
.empty_room_residual_energy_reference
.partition_point(|value| *value <= residual_energy);
let empirical_quantile = at_or_below as f64 / reference_window_count as f64;
Some((1.0 - empirical_quantile * 0.5).clamp(0.5, 1.0))
};
Some(EmptyRoomMatch {
matches_empty,
score,
residual_energy,
residual_energy_threshold,
window_size,
reference_window_count,
})
}
/// Extract body perturbation from a runtime observation.
///
/// Subtracts baseline, projects out environmental modes, returns residual.
@@ -993,14 +1360,19 @@ impl FieldModel {
*m /= count as f64;
}
// Rank alone is not person evidence. A continuation window can expose
// more significant eigenvalues while its mean remains on the learned
// empty room manifold. Fail toward empty only inside the same
// conservative residual energy boundary used by the server fallback.
if self
// Rank alone is not person evidence. A short continuation window can
// expose more significant eigenvalues while its mean remains on the
// learned empty room manifold. Use the hardware adaptive residual
// boundary when available.
let empty_room_residual_energy_max = modes
.empty_room_residual_energy_threshold
.unwrap_or(EMPTY_ROOM_RESIDUAL_ENERGY_MAX)
.max(EMPTY_ROOM_RESIDUAL_ENERGY_MAX);
let mean_residual_energy = self
.extract_perturbation(&[mean.clone()])
.is_ok_and(|perturbation| perturbation.total_energy <= EMPTY_ROOM_RESIDUAL_ENERGY_MAX)
{
.ok()
.map(|perturbation| perturbation.total_energy);
if mean_residual_energy.is_some_and(|energy| energy <= empty_room_residual_energy_max) {
return Ok(0);
}
@@ -1064,7 +1436,23 @@ impl FieldModel {
let mp_threshold = noise_var * (1.0 + ratio.sqrt()).powi(2);
let significant = eigenvalues.iter().filter(|&&ev| ev > mp_threshold).count();
let occupancy = significant.saturating_sub(modes.baseline_eigenvalue_count);
let reference_count = if modes.baseline_runtime_window_size == Some(count) {
modes
.baseline_runtime_eigenvalue_count
.unwrap_or(modes.baseline_eigenvalue_count)
} else {
modes.baseline_eigenvalue_count
};
let rank_occupancy = significant.saturating_sub(reference_count);
// A stable person can shift the window mean without increasing its
// covariance rank. Preserve at least one occupant after the learned
// empty room residual boundary is exceeded.
let occupancy =
if mean_residual_energy.is_some_and(|energy| energy > empty_room_residual_energy_max) {
rank_occupancy.max(1)
} else {
rank_occupancy
};
Ok(occupancy.min(10)) // Cap at 10 persons
}
@@ -1103,6 +1491,12 @@ impl FieldModel {
self.covariance_sum = None;
self.covariance_count = 0;
self.calibration_started = None;
self.calibration_reference_tail.clear();
self.calibration_window_count = 0;
for link_sum in &mut self.calibration_window_sum {
link_sum.fill(0.0);
}
self.calibration_window_means.clear();
}
}
@@ -1623,6 +2017,131 @@ mod tests {
assert_eq!(occupancy, 0, "Noise-only frames should yield 0 occupancy");
}
#[cfg(feature = "eigenvalue")]
#[test]
fn runtime_sized_empty_reference_scores_held_out_background() {
let config = FieldModelConfig {
n_links: 1,
n_subcarriers: 56,
n_modes: 3,
min_calibration_frames: 500,
min_calibration_duration_s: 0.0,
baseline_expiry_s: 86_400.0,
};
let mut model = FieldModel::new(config).unwrap();
let mut calibration = Vec::new();
for frame_index in 0..600 {
let time = frame_index as f64 * 0.071;
let frame: Vec<f64> = (0..56)
.map(|subcarrier| {
let carrier = subcarrier as f64;
18.0 + carrier * 0.08
+ (time + carrier * 0.13).sin() * 0.8
+ (time * 0.37 + carrier * 0.031).cos() * 0.35
})
.collect();
model.feed_calibration(&[frame.clone()]).unwrap();
calibration.push(frame);
}
model.finalize_calibration(1_000_000, 0).unwrap();
let modes = model.modes().unwrap();
assert_eq!(
modes.baseline_runtime_window_size,
Some(RUNTIME_OCCUPANCY_WINDOW)
);
assert!(modes.baseline_runtime_eigenvalue_count.is_some());
assert!(modes.empty_room_residual_energy_threshold.is_some());
assert_eq!(modes.empty_room_residual_energy_reference.len(), 12);
let held_out_empty = calibration[550..600].to_vec();
assert_eq!(model.estimate_occupancy(&held_out_empty).unwrap(), 0);
let background = model
.empty_room_match(&held_out_empty)
.expect("background match");
assert!(background.matches_empty);
assert_eq!(background.reference_window_count, 12);
assert!(background
.score
.is_some_and(|score| (0.5..=1.0).contains(&score)));
}
#[cfg(feature = "eigenvalue")]
#[test]
fn runtime_reference_rejects_shifted_held_out_window() {
let config = FieldModelConfig {
n_links: 1,
n_subcarriers: 56,
n_modes: 3,
min_calibration_frames: 500,
min_calibration_duration_s: 0.0,
baseline_expiry_s: 86_400.0,
};
let mut model = FieldModel::new(config).unwrap();
for frame_index in 0..600 {
let time = frame_index as f64 * 0.071;
let frame: Vec<f64> = (0..56)
.map(|subcarrier| {
let carrier = subcarrier as f64;
18.0 + carrier * 0.08
+ (time + carrier * 0.13).sin() * 0.8
+ (time * 0.37 + carrier * 0.031).cos() * 0.35
})
.collect();
model.feed_calibration(&[frame]).unwrap();
}
model.finalize_calibration(1_000_000, 0).unwrap();
let shifted: Vec<Vec<f64>> = (0..RUNTIME_OCCUPANCY_WINDOW)
.map(|frame_index| {
let time = (600 + frame_index) as f64 * 0.071;
(0..56)
.map(|subcarrier| {
let carrier = subcarrier as f64;
let body_shift = if subcarrier % 3 == 0 { 8.0 } else { -5.0 };
18.0 + carrier * 0.08
+ (time + carrier * 0.13).sin() * 0.8
+ (time * 0.37 + carrier * 0.031).cos() * 0.35
+ body_shift
})
.collect()
})
.collect();
assert!(model.estimate_occupancy(&shifted).unwrap() >= 1);
let background = model.empty_room_match(&shifted).expect("shifted match");
assert!(!background.matches_empty);
assert_eq!(background.score, Some(0.0));
}
#[test]
fn runtime_background_reference_is_single_link_only() {
let config = FieldModelConfig {
n_links: 2,
n_subcarriers: 8,
n_modes: 2,
min_calibration_frames: 100,
min_calibration_duration_s: 0.0,
baseline_expiry_s: 86_400.0,
};
let mut model = FieldModel::new(config).unwrap();
for frame_index in 0..100 {
let phase = frame_index as f64 * 0.1;
model
.feed_calibration(&[
(0..8).map(|index| phase + index as f64).collect(),
(0..8).map(|index| phase - index as f64).collect(),
])
.unwrap();
}
model.finalize_calibration(1_000_000, 0).unwrap();
let modes = model.modes().unwrap();
assert_eq!(modes.baseline_runtime_window_size, None);
assert_eq!(modes.baseline_runtime_eigenvalue_count, None);
assert_eq!(modes.empty_room_residual_energy_threshold, None);
assert!(modes.empty_room_residual_energy_reference.is_empty());
}
#[test]
fn test_baseline_eigenvalue_count_stored() {
let config = FieldModelConfig {
@@ -1735,14 +2254,15 @@ mod tests {
// -----------------------------------------------------------------
#[test]
fn test_default_frame_target_derives_from_duration() {
fn test_default_frame_target_covers_background_reference_windows() {
let cfg = FieldModelConfig::default();
assert_eq!(cfg.min_calibration_duration_s, CALIBRATION_DURATION_S);
assert_eq!(
cfg.min_calibration_frames,
(CALIBRATION_DURATION_S * ASSUMED_CALIBRATION_RATE_HZ) as usize,
"frame target must be the intended window at the assumed rate"
cfg.min_calibration_frames, MIN_CALIBRATION_FRAMES,
"frame target must cover the minimum independent runtime windows"
);
assert_eq!(MIN_CALIBRATION_FRAMES, 1_000);
assert_eq!(MIN_BACKGROUND_REFERENCE_WINDOWS, 20);
}
#[test]