Compare commits

...
2 Commits
10 changed files with 2339 additions and 120 deletions

No files matched your search

@@ -0,0 +1,113 @@
# ADR 355: Local empty room bootstrap prior
Status: Accepted
Date: 2026-09-04
## Context
Most installations are occupied when RuView first starts. An uncalibrated
signal heuristic can interpret HVAC, clock error, neighboring motion, or a
short covariance window as a person. It can also surface plausible numeric
vitals before a room model establishes occupant attribution.
One empty home is not representative training data for all homes and contains
no positive human examples. It is useful only as a recent negative prior for
the same installation. It cannot define a universal human signal model.
The field model was memory only. The server also used a relative `data/`
directory, which is unreliable for embedded application hosts.
## Decision
RuView may persist a completed empty room field model as a local bootstrap
prior only after two server controlled stages:
1. Normal calibration reaches both 1,000 accepted frames and 600 seconds.
The frame target represents twenty complete fifty frame runtime windows.
Collection binds to the first accepted ESP32 source and rejects frames from
other radios for that single link model.
2. Promotion observes 12 new samples spaced by at least one second. Each
sample has a four second bounded acquisition timeout. At least 10 must
resolve empty, all ticks must be fresh, and none may publish numeric vital
signs.
The stored image contains aggregate means, environmental modes, mode energies,
noise statistics, sorted scalar residual references, configuration, one source
node ID, and a server generated model ID. It excludes raw CSI, waveforms,
device addresses, room names, pose, identity, credentials, and positive human
labels. Runtime matching compares a fifty frame mean with the learned residual
distribution. Its score is empirical background conformance, not an absence
probability.
The image is bound to a SHA 256 digest of the stable installation ID, carries
an integrity digest over the exact stored payload bytes, is capped at 1 MiB,
and inherits the field model expiry.
Files with an invalid schema, dimensions, values, node set, installation
binding, digest, size, or lifetime fail closed.
Restoring an image creates a field model in `bootstrap_only` mode. It may
reduce an uncalibrated false person count, but it cannot authorize calibrated
evidence or numeric heart and breathing rates. Explicit calibration replaces
it with fresh statistics. Cancelling an unfinished capture restores a valid
prior. Administrator scoped reset removes it.
The status endpoint reports the learned runtime window, residual threshold,
reference count, current background conformance, and the negative only
authority boundary. A restored model never resumes collection and never
persists raw calibration frames.
The server accepts `--data-dir` or `RUVIEW_DATA_DIR`, allowing an application
host to provide a protected state directory.
## Consequences
Startup can use recent measured background structure instead of only generic
heuristics. No accuracy lift is claimed until occupied and empty held out data
from multiple installations is recorded.
The largest failure mode is overfitting to a temporarily quiet home and
suppressing a weak occupant. Controls are low authority, 24 hour expiry,
installation binding, vital abstention, explicit recalibration, and reset.
A human signal model still requires positive, consented, leakage separated
occupied sequences. Empty data is never relabeled as human training.
## Implementation
1. `wifi-densepose-signal::ruvsense::field_model` exports and validates
`FieldModelSnapshotV1`.
2. `wifi-densepose-sensing-server::bootstrap_baseline` owns schema, integrity,
storage, lifetime, and the held out gate.
3. `POST /api/v1/calibration/bootstrap/promote` measures the validation window
and stores the image only on pass.
4. `GET /api/v1/calibration/status` reports bootstrap authority and explicitly
denies calibrated evidence and numeric vital authority.
5. Numeric vitals fail closed unless an explicit fresh calibration resolves
exactly one occupant at sufficient signal quality and confidence.
6. The legacy `edge_vitals` WebSocket message applies the same publication
gate as `sensing_update`. When the bootstrap background matches, it emits
absent state, zero people, null numeric rates, and an explicit abstention
reason instead of forwarding raw edge candidates.
## Measured installation evidence
One physical empty home run on source Node 5 collected 1,762 accepted frames
over 655 seconds and produced 35 privacy reduced reference windows. Promotion
observed 12 of 12 fresh empty samples, zero stale samples, and zero numeric
vital samples. After process restart, 12 of 12 fresh samples matched the empty
background and contained no numeric vital signs. Background conformance scores
ranged from 50.0 to 64.3 percent with a median of 54.3 percent.
This is measured negative evidence for one installation. No occupied positive
holdout was recorded, so it does not prove person recall, person counting, or
cross home accuracy improvement.
## Acceptance test
With a physically empty room, complete calibration on stable nodes, promote
the single source model, and restart the server with the same installation ID
and data directory. Status must report `binding_mode=bootstrap_only`, zero
collection frames, and false authority for calibrated evidence and numeric
vitals. Twelve fresh empty samples must contain no numeric vital signs.
Starting a new room calibration must replace the prior and begin at frame zero.
+1
View File
@@ -96,6 +96,7 @@ Statuses: **Proposed** (under discussion), **Accepted** (approved and/or impleme
| [ADR-349](ADR-349-governed-local-and-fal-forecast-training.md) | Governed local and fal.ai forecast training | Proposed |
| [ADR-350](ADR-350-ruvector-predictive-memory-and-ruvllm-boundary.md) | RuVector predictive memory and RuVLLM authority boundary | Proposed |
| [ADR-354](ADR-354-forecast-governance-enforcement-in-ruforecast-submodule.md) | fal.ai forecast governance and spend enforcement now real, in the vendored RuForecast submodule | Accepted |
| [ADR-355](ADR-355-local-empty-room-bootstrap-prior.md) | Local empty room bootstrap prior | Accepted |
### Platform and UI
@@ -30,7 +30,7 @@ mdns-sd = "0.11"
# Serialization
serde = { workspace = true }
serde_json.workspace = true
serde_json = { workspace = true, features = ["raw_value"] }
# Logging
tracing.workspace = true
@@ -43,6 +43,8 @@ per ADR-022 Phase 3.
- **WebSocket broadcast** -- Real-time sensing updates pushed to all connected clients at
`ws://localhost:8765/ws/sensing`.
- **Static file serving** -- Hosts the sensing UI on port 8080 with CORS headers.
- **Private startup baseline** -- Restores a recent, installation-bound empty-room field model
without storing raw CSI or authorizing numeric vital signs (ADR-355).
## Modules
@@ -74,6 +76,23 @@ cargo run -p wifi-densepose-sensing-server -- \
--static-dir ./ui
```
### Empty-room startup baseline
Provide a stable installation ID and an application-owned private state directory:
```bash
cargo run -p wifi-densepose-sensing-server -- \
--installation-id installation-01 \
--data-dir /path/to/private/application-state
```
Complete the normal empty-room calibration, then call
`POST /api/v1/calibration/bootstrap/promote`. The server measures a separate 12-sample holdout
before storing an aggregate field-model snapshot. On a later start with the same installation ID,
the snapshot is restored with `bootstrap_only` authority. It can reduce startup background false
positives, but cannot authorize calibrated evidence or numeric heart and breathing rates. Use
`POST /api/v1/calibration/reset` with administrator scope to remove it.
### Using as a library
```rust
@@ -491,6 +491,10 @@ const READ_SAFE_MUTATIONS: &[&str] = &[
// Capture and calibration: create data, never destroy it.
"/api/v1/calibration/start",
"/api/v1/calibration/stop",
// Promotion consumes only server measured held out evidence and stores a
// bounded privacy reduced prior. Cancel discards only an unfinished run.
"/api/v1/calibration/bootstrap/promote",
"/api/v1/calibration/cancel",
"/api/v1/pose/calibrate",
"/api/v1/recording/start",
"/api/v1/recording/stop",
@@ -1970,6 +1974,7 @@ mod scope_gate_polarity_tests {
(Method::DELETE, "/api/v1/models/m1"),
(Method::DELETE, "/api/v1/recording/r1"),
(Method::POST, "/api/v1/config/ground-truth"),
(Method::POST, "/api/v1/calibration/reset"),
] {
assert_eq!(
required_scope_for(&m, p),
@@ -0,0 +1,436 @@
//! Local, privacy reduced empty room bootstrap model persistence.
//!
//! A bootstrap image is deliberately lower authority than a completed room
//! calibration. It can suppress background perturbations while the server
//! starts, but restoring it never authorizes calibrated evidence or vitals.
use rand::{rngs::OsRng, RngCore};
use serde::{Deserialize, Serialize};
use serde_json::value::RawValue;
use sha2::{Digest, Sha256};
use std::fs::{self, OpenOptions};
use std::io::{Read, Write};
use std::path::{Path, PathBuf};
use thiserror::Error;
use wifi_densepose_signal::ruvsense::field_model::{
FieldModel, FieldModelError, FieldModelSnapshotV1,
};
pub const BOOTSTRAP_BASELINE_SCHEMA: &str = "ruview.bootstrap-empty-field-model.v1";
pub const BOOTSTRAP_BASELINE_AUTHORITY: &str = "bootstrap_only";
pub const BOOTSTRAP_VALIDATION_SAMPLES: usize = 12;
pub const BOOTSTRAP_VALIDATION_MIN_EMPTY: usize = 10;
pub const BOOTSTRAP_VALIDATION_MIN_SPACING_MS: u64 = 1_000;
pub const BOOTSTRAP_VALIDATION_SAMPLE_TIMEOUT_MS: u64 = 4_000;
const BOOTSTRAP_MAX_FILE_BYTES: u64 = 1_048_576;
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(deny_unknown_fields)]
struct BootstrapPayloadV1 {
schema: String,
authority: String,
installation_binding_sha256: String,
source_node_ids: Vec<u8>,
source_model_id: String,
created_at_unix_ms: u64,
expires_at_unix_ms: u64,
field_model: FieldModelSnapshotV1,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(deny_unknown_fields)]
struct BootstrapImageV1 {
payload: BootstrapPayloadV1,
content_sha256: String,
}
#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct BootstrapImageRawV1 {
payload: Box<RawValue>,
content_sha256: String,
}
#[derive(Debug, Clone, Serialize, PartialEq, Eq)]
pub struct BootstrapBaselineMetadata {
pub authority: &'static str,
pub source_node_ids: Vec<u8>,
pub source_model_id: String,
pub created_at_unix_ms: u64,
pub expires_at_unix_ms: u64,
pub content_sha256: String,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct BootstrapValidationSample {
pub fresh_tick: bool,
pub calibrated_empty: bool,
pub vital_signs_absent: bool,
}
#[derive(Debug, Clone, Copy, Serialize, PartialEq, Eq)]
pub struct BootstrapValidationResult {
pub sample_count: usize,
pub empty_sample_count: usize,
pub stale_sample_count: usize,
pub vital_sign_sample_count: usize,
pub passed: bool,
}
#[derive(Debug, Error)]
pub enum BootstrapBaselineError {
#[error("a stable installation_id is required for a local bootstrap baseline")]
MissingInstallationId,
#[error("bootstrap baseline path is invalid")]
InvalidPath,
#[error("bootstrap baseline file is too large")]
FileTooLarge,
#[error("bootstrap baseline image is malformed: {0}")]
Malformed(String),
#[error("bootstrap baseline belongs to another installation")]
InstallationMismatch,
#[error("bootstrap baseline has expired")]
Expired,
#[error("bootstrap baseline digest does not verify")]
DigestMismatch,
#[error("field model snapshot is invalid: {0}")]
FieldModel(#[from] FieldModelError),
#[error("bootstrap baseline storage failed: {0}")]
Io(#[from] std::io::Error),
#[error("bootstrap baseline serialization failed: {0}")]
Json(#[from] serde_json::Error),
}
pub fn path_in(data_dir: &Path) -> PathBuf {
data_dir.join("calibration").join("bootstrap-empty-v1.json")
}
pub fn installation_binding(installation_id: &str) -> Result<String, BootstrapBaselineError> {
let installation_id = installation_id.trim();
if installation_id.is_empty() || installation_id.len() > 128 {
return Err(BootstrapBaselineError::MissingInstallationId);
}
Ok(hex_sha256(installation_id.as_bytes()))
}
pub fn evaluate_validation(samples: &[BootstrapValidationSample]) -> BootstrapValidationResult {
let empty_sample_count = samples
.iter()
.filter(|sample| sample.fresh_tick && sample.calibrated_empty)
.count();
let stale_sample_count = samples.iter().filter(|sample| !sample.fresh_tick).count();
let vital_sign_sample_count = samples
.iter()
.filter(|sample| !sample.vital_signs_absent)
.count();
BootstrapValidationResult {
sample_count: samples.len(),
empty_sample_count,
stale_sample_count,
vital_sign_sample_count,
passed: samples.len() == BOOTSTRAP_VALIDATION_SAMPLES
&& empty_sample_count >= BOOTSTRAP_VALIDATION_MIN_EMPTY
&& stale_sample_count == 0
&& vital_sign_sample_count == 0,
}
}
pub fn store(
path: &Path,
installation_id: &str,
source_node_ids: &[u8],
source_model_id: &str,
created_at_unix_ms: u64,
field_model: &FieldModel,
) -> Result<BootstrapBaselineMetadata, BootstrapBaselineError> {
let snapshot = field_model.export_snapshot()?;
let calibrated_at_ms = snapshot.modes.calibrated_at_us / 1_000;
let expiry_ms = (snapshot.config.baseline_expiry_s * 1_000.0) as u64;
let payload = BootstrapPayloadV1 {
schema: BOOTSTRAP_BASELINE_SCHEMA.to_string(),
authority: BOOTSTRAP_BASELINE_AUTHORITY.to_string(),
installation_binding_sha256: installation_binding(installation_id)?,
source_node_ids: source_node_ids.to_vec(),
source_model_id: source_model_id.to_string(),
created_at_unix_ms,
expires_at_unix_ms: calibrated_at_ms.saturating_add(expiry_ms),
field_model: snapshot,
};
validate_payload(&payload, installation_id, created_at_unix_ms)?;
let content_sha256 = payload_digest(&payload)?;
let image = BootstrapImageV1 {
payload,
content_sha256,
};
let bytes = serde_json::to_vec(&image)?;
if bytes.len() as u64 > BOOTSTRAP_MAX_FILE_BYTES {
return Err(BootstrapBaselineError::FileTooLarge);
}
atomic_write_private(path, &bytes)?;
Ok(metadata(&image))
}
pub fn load(
path: &Path,
installation_id: &str,
current_unix_ms: u64,
) -> Result<(FieldModel, BootstrapBaselineMetadata), BootstrapBaselineError> {
let file_metadata = fs::symlink_metadata(path)?;
if file_metadata.file_type().is_symlink() || !file_metadata.is_file() {
return Err(BootstrapBaselineError::InvalidPath);
}
if file_metadata.len() > BOOTSTRAP_MAX_FILE_BYTES {
return Err(BootstrapBaselineError::FileTooLarge);
}
let mut bytes = Vec::with_capacity(file_metadata.len() as usize);
OpenOptions::new()
.read(true)
.open(path)?
.take(BOOTSTRAP_MAX_FILE_BYTES + 1)
.read_to_end(&mut bytes)?;
if bytes.len() as u64 > BOOTSTRAP_MAX_FILE_BYTES {
return Err(BootstrapBaselineError::FileTooLarge);
}
// Verify the exact payload bytes written by storage before decoding the
// floating point model. Equivalent JSON number spellings may serialize
// differently, so decode and reserialize is not an integrity contract.
let raw_image: BootstrapImageRawV1 = serde_json::from_slice(&bytes)?;
if hex_sha256(raw_image.payload.get().as_bytes()) != raw_image.content_sha256 {
return Err(BootstrapBaselineError::DigestMismatch);
}
let payload: BootstrapPayloadV1 = serde_json::from_str(raw_image.payload.get())?;
validate_payload(&payload, installation_id, current_unix_ms)?;
let image = BootstrapImageV1 {
payload,
content_sha256: raw_image.content_sha256,
};
let model = FieldModel::from_snapshot(
image.payload.field_model.clone(),
current_unix_ms.saturating_mul(1_000),
)?;
Ok((model, metadata(&image)))
}
pub fn remove(path: &Path) -> Result<bool, BootstrapBaselineError> {
match fs::symlink_metadata(path) {
Ok(metadata) => {
if metadata.file_type().is_symlink() || !metadata.is_file() {
return Err(BootstrapBaselineError::InvalidPath);
}
fs::remove_file(path)?;
Ok(true)
}
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false),
Err(error) => Err(error.into()),
}
}
fn validate_payload(
payload: &BootstrapPayloadV1,
installation_id: &str,
current_unix_ms: u64,
) -> Result<(), BootstrapBaselineError> {
if payload.schema != BOOTSTRAP_BASELINE_SCHEMA
|| payload.authority != BOOTSTRAP_BASELINE_AUTHORITY
|| payload.source_model_id.is_empty()
|| payload.source_model_id.len() > 128
|| payload.source_node_ids.is_empty()
|| payload.source_node_ids.len() > 16
|| payload
.source_node_ids
.windows(2)
.any(|pair| pair[0] >= pair[1])
|| payload.created_at_unix_ms == 0
|| payload.expires_at_unix_ms <= payload.created_at_unix_ms
{
return Err(BootstrapBaselineError::Malformed(
"invalid schema, authority, identity, node set, or lifetime".into(),
));
}
if payload.installation_binding_sha256 != installation_binding(installation_id)? {
return Err(BootstrapBaselineError::InstallationMismatch);
}
if current_unix_ms > payload.expires_at_unix_ms {
return Err(BootstrapBaselineError::Expired);
}
Ok(())
}
fn payload_digest(payload: &BootstrapPayloadV1) -> Result<String, serde_json::Error> {
Ok(hex_sha256(&serde_json::to_vec(payload)?))
}
fn metadata(image: &BootstrapImageV1) -> BootstrapBaselineMetadata {
BootstrapBaselineMetadata {
authority: BOOTSTRAP_BASELINE_AUTHORITY,
source_node_ids: image.payload.source_node_ids.clone(),
source_model_id: image.payload.source_model_id.clone(),
created_at_unix_ms: image.payload.created_at_unix_ms,
expires_at_unix_ms: image.payload.expires_at_unix_ms,
content_sha256: image.content_sha256.clone(),
}
}
fn hex_sha256(bytes: &[u8]) -> String {
let digest = Sha256::digest(bytes);
digest.iter().map(|byte| format!("{byte:02x}")).collect()
}
fn atomic_write_private(path: &Path, bytes: &[u8]) -> Result<(), BootstrapBaselineError> {
let parent = path.parent().ok_or(BootstrapBaselineError::InvalidPath)?;
fs::create_dir_all(parent)?;
let file_name = path
.file_name()
.and_then(|name| name.to_str())
.ok_or(BootstrapBaselineError::InvalidPath)?;
let mut nonce = [0_u8; 8];
OsRng.fill_bytes(&mut nonce);
let temporary = parent.join(format!(
".{file_name}.{:016x}.tmp",
u64::from_le_bytes(nonce)
));
let mut options = OpenOptions::new();
options.write(true).create_new(true);
#[cfg(unix)]
{
use std::os::unix::fs::OpenOptionsExt;
options.mode(0o600);
}
let write_result = (|| -> Result<(), std::io::Error> {
let mut file = options.open(&temporary)?;
file.write_all(bytes)?;
file.sync_all()?;
fs::rename(&temporary, path)?;
Ok(())
})();
if write_result.is_err() {
let _ = fs::remove_file(&temporary);
}
write_result.map_err(Into::into)
}
#[cfg(test)]
mod tests {
use super::*;
use wifi_densepose_signal::ruvsense::field_model::FieldModelConfig;
fn completed_model(now_us: u64) -> FieldModel {
let mut model = FieldModel::new(FieldModelConfig {
n_links: 1,
n_subcarriers: 4,
n_modes: 2,
min_calibration_frames: 3,
min_calibration_duration_s: 0.0,
baseline_expiry_s: 3_600.0,
})
.unwrap();
for index in 0..3 {
model
.feed_calibration(&[vec![1.0 + index as f64, 2.0, 3.0, 4.0]])
.unwrap();
}
model.finalize_calibration(now_us, 7).unwrap();
model
}
#[test]
fn round_trip_is_installation_bound_and_contains_no_raw_frames() {
let dir = tempfile::tempdir().unwrap();
let path = path_in(dir.path());
let now_ms = 1_000_000;
let model = completed_model(now_ms * 1_000);
let stored = store(&path, "home-a", &[5], "model", now_ms, &model).unwrap();
let text = String::from_utf8(fs::read(&path).unwrap()).unwrap();
assert!(!text.contains("home-a"));
assert!(!text.contains("raw_csi"));
let (restored, loaded) = load(&path, "home-a", now_ms + 1).unwrap();
assert_eq!(stored, loaded);
assert_eq!(
restored.export_snapshot().unwrap(),
model.export_snapshot().unwrap()
);
assert!(matches!(
load(&path, "home-b", now_ms + 1),
Err(BootstrapBaselineError::InstallationMismatch)
));
}
#[test]
fn tamper_and_expiry_fail_closed() {
let dir = tempfile::tempdir().unwrap();
let path = path_in(dir.path());
let now_ms = 1_000_000;
let model = completed_model(now_ms * 1_000);
store(&path, "home-a", &[5], "model", now_ms, &model).unwrap();
let mut image: serde_json::Value =
serde_json::from_slice(&fs::read(&path).unwrap()).unwrap();
image["payload"]["source_model_id"] = serde_json::json!("forged");
fs::write(&path, serde_json::to_vec(&image).unwrap()).unwrap();
assert!(matches!(
load(&path, "home-a", now_ms + 1),
Err(BootstrapBaselineError::DigestMismatch)
));
store(&path, "home-a", &[5], "model", now_ms, &model).unwrap();
assert!(matches!(
load(&path, "home-a", now_ms + 3_600_001),
Err(BootstrapBaselineError::Expired)
| Err(BootstrapBaselineError::FieldModel(
FieldModelError::BaselineExpired { .. }
))
));
}
#[test]
fn promotion_gate_requires_twelve_fresh_samples_and_zero_vitals() {
assert_eq!(BOOTSTRAP_VALIDATION_MIN_SPACING_MS, 1_000);
assert_eq!(BOOTSTRAP_VALIDATION_SAMPLE_TIMEOUT_MS, 4_000);
assert!(
BOOTSTRAP_VALIDATION_SAMPLE_TIMEOUT_MS >= BOOTSTRAP_VALIDATION_MIN_SPACING_MS
);
let passing = vec![
BootstrapValidationSample {
fresh_tick: true,
calibrated_empty: true,
vital_signs_absent: true,
};
BOOTSTRAP_VALIDATION_SAMPLES
];
assert!(evaluate_validation(&passing).passed);
let mut stale = passing.clone();
stale[0].fresh_tick = false;
assert!(!evaluate_validation(&stale).passed);
let mut vital = passing;
vital[0].vital_signs_absent = false;
assert!(!evaluate_validation(&vital).passed);
}
#[test]
fn restart_verifies_the_exact_stored_payload_bytes() {
let dir = tempfile::tempdir().unwrap();
let path = path_in(dir.path());
let now_ms = 1_000_000;
let model = completed_model(now_ms * 1_000);
store(&path, "home-a", &[5], "model", now_ms, &model).unwrap();
let original = fs::read_to_string(&path).unwrap();
let image: BootstrapImageRawV1 = serde_json::from_str(&original).unwrap();
let rewritten_payload = image.payload.get().replacen(
"\"min_calibration_duration_s\":0.0",
"\"min_calibration_duration_s\":0.00",
1,
);
assert_ne!(rewritten_payload, image.payload.get());
let digest = hex_sha256(rewritten_payload.as_bytes());
let rewritten =
format!("{{\"payload\":{rewritten_payload},\"content_sha256\":\"{digest}\"}}");
fs::write(&path, rewritten).unwrap();
let (_, loaded) = load(&path, "home-a", now_ms + 1).unwrap();
assert_eq!(loaded.content_sha256, digest);
}
}
@@ -40,6 +40,103 @@ const ENERGY_THRESH_3: f64 = 25.0;
/// reliable — genuine higher counts come from the multistatic fusion path.
const MAX_SINGLE_LINK_OCCUPANCY: usize = 3;
/// Low-authority comparison of the current source window with a restored
/// empty-room bootstrap image. This is background conformance only and cannot
/// authorize presence, person count, pose, identity, or vital signs.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct BootstrapBackgroundMatch {
pub matches_empty: bool,
pub score: Option<f64>,
pub residual_energy: f64,
pub residual_energy_threshold: f64,
pub window_size: usize,
pub reference_window_count: usize,
}
fn mean_frame(frames: &[Vec<f64>]) -> Option<Vec<f64>> {
let width = frames.first()?.len();
if width == 0 {
return None;
}
let mut mean = vec![0.0_f64; width];
let mut count = 0_usize;
for frame in frames {
if frame.len() != width {
continue;
}
for (sum, value) in mean.iter_mut().zip(frame.iter()) {
*sum += value;
}
count += 1;
}
if count == 0 {
return None;
}
for value in &mut mean {
*value /= count as f64;
}
Some(mean)
}
fn perturbation_occupancy(field: &FieldModel, frames: &[Vec<f64>]) -> Option<usize> {
let adaptive_threshold = field.empty_room_residual_energy_threshold();
let frame = match adaptive_threshold {
Some(_) => mean_frame(frames)?,
None => frames.first()?.clone(),
};
let perturbation = field.extract_perturbation(&[frame]).ok()?;
let count = match adaptive_threshold {
Some(threshold) => {
let empty_threshold = threshold.max(1.0);
if perturbation.total_energy > empty_threshold * 25.0 {
3
} else if perturbation.total_energy > empty_threshold * 12.0 {
2
} else if perturbation.total_energy > empty_threshold {
1
} else {
0
}
}
None => {
if perturbation.total_energy > ENERGY_THRESH_3 {
3
} else if perturbation.total_energy > ENERGY_THRESH_2 {
2
} else if perturbation.total_energy > 1.0 {
1
} else {
0
}
}
};
Some(count)
}
/// Provenance for a count produced by the calibrated field model. Callers
/// that attach calibration evidence must never substitute the score heuristic
/// when the calibrated path cannot evaluate the observation.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum CalibratedOccupancyMethod {
Eigenvalue,
PerturbationEnergy,
}
impl CalibratedOccupancyMethod {
pub const fn wire_name(self) -> &'static str {
match self {
Self::Eigenvalue => "field_model_eigenvalue_v1",
Self::PerturbationEnergy => "field_model_perturbation_energy_v1",
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct CalibratedOccupancy {
pub person_count: usize,
pub method: CalibratedOccupancyMethod,
}
/// Create a FieldModelConfig for single-link mode (one ESP32 node = one link).
/// This avoids the DimensionMismatch error when feeding single-frame observations.
pub fn single_link_config() -> FieldModelConfig {
@@ -67,7 +164,7 @@ pub fn occupancy_or_fallback(
.iter()
.rev()
.take(OCCUPANCY_WINDOW)
.cloned()
.map(|frame| CALIB_NORMALIZER.resample_to_canonical(frame))
.collect();
if frames.is_empty() {
@@ -87,26 +184,86 @@ pub fn occupancy_or_fallback(
// Fallback: perturbation energy thresholds.
// FieldModel expects [n_links][n_subcarriers] — we use n_links=1.
let observation = vec![frames[0].clone()];
match field.extract_perturbation(&observation) {
Ok(perturbation) => {
if perturbation.total_energy > ENERGY_THRESH_3 {
3
} else if perturbation.total_energy > ENERGY_THRESH_2 {
2
} else if perturbation.total_energy > 1.0 {
1
} else {
0
}
}
Err(_) => score_to_person_count(smoothed_score, prev_count),
}
perturbation_occupancy(field, &frames)
.unwrap_or_else(|| score_to_person_count(smoothed_score, prev_count))
}
_ => score_to_person_count(smoothed_score, prev_count),
}
}
/// Return true only when a restored low-authority bootstrap model has a full
/// runtime window and that model resolves the window as empty. This helper is
/// deliberately one-way: a bootstrap prior may suppress an obvious background
/// false positive, but cannot authorize presence or vital signs.
pub fn bootstrap_empty_prior_applies(
field: &FieldModel,
frame_history: &VecDeque<Vec<f64>>,
observed_at_us: u64,
) -> bool {
bootstrap_background_match(field, frame_history, observed_at_us)
.is_some_and(|result| result.matches_empty)
}
/// Score the current canonical runtime window against the learned empty-room
/// residual references. Legacy models without scalar references preserve the
/// binary match but intentionally return no score.
pub fn bootstrap_background_match(
field: &FieldModel,
frame_history: &VecDeque<Vec<f64>>,
observed_at_us: u64,
) -> Option<BootstrapBackgroundMatch> {
if frame_history.len() < OCCUPANCY_WINDOW
|| field.check_freshness(observed_at_us) != CalibrationStatus::Fresh
{
return None;
}
let frames: Vec<Vec<f64>> = frame_history
.iter()
.rev()
.take(OCCUPANCY_WINDOW)
.map(|frame| CALIB_NORMALIZER.resample_to_canonical(frame))
.collect();
let result = field.empty_room_match(&frames)?;
Some(BootstrapBackgroundMatch {
matches_empty: result.matches_empty,
score: result.score,
residual_energy: result.residual_energy,
residual_energy_threshold: result.residual_energy_threshold,
window_size: result.window_size,
reference_window_count: result.reference_window_count,
})
}
/// Produce a fail-closed occupancy result from a fresh calibrated model.
pub fn calibrated_occupancy(
field: &FieldModel,
frame_history: &VecDeque<Vec<f64>>,
observed_at_us: u64,
) -> Option<CalibratedOccupancy> {
if field.check_freshness(observed_at_us) != CalibrationStatus::Fresh {
return None;
}
let frames: Vec<Vec<f64>> = frame_history
.iter()
.rev()
.take(OCCUPANCY_WINDOW)
.map(|frame| CALIB_NORMALIZER.resample_to_canonical(frame))
.collect();
if frames.is_empty() {
return None;
}
if let Ok(person_count) = field.estimate_occupancy(&frames) {
return Some(CalibratedOccupancy {
person_count: person_count.min(MAX_SINGLE_LINK_OCCUPANCY),
method: CalibratedOccupancyMethod::Eigenvalue,
});
}
Some(CalibratedOccupancy {
person_count: perturbation_occupancy(field, &frames)?,
method: CalibratedOccupancyMethod::PerturbationEnergy,
})
}
/// Feed the latest frame to the FieldModel during calibration collection.
///
/// Acts while the model is `Uncalibrated` or `Collecting`. The first fed frame
@@ -174,6 +331,33 @@ pub fn parse_node_positions(input: &str) -> Vec<[f32; 3]> {
mod tests {
use super::*;
fn fresh_test_model() -> FieldModel {
let config = FieldModelConfig {
n_links: 1,
n_subcarriers: 56,
n_modes: 3,
min_calibration_frames: 10,
min_calibration_duration_s: 0.0,
baseline_expiry_s: 2.0,
};
let mut field = FieldModel::new(config).expect("field model");
for index in 0..20 {
let phase = index as f64 * 0.1;
let observation = vec![(0..56)
.map(|subcarrier| {
1.0 + subcarrier as f64 * 0.01 + (phase + subcarrier as f64 * 0.03).sin() * 0.01
})
.collect()];
field
.feed_calibration(&observation)
.expect("calibration frame");
}
field
.finalize_calibration(1_000_000, 7)
.expect("finalize calibration");
field
}
#[test]
fn test_parse_node_positions() {
let positions = parse_node_positions("0,0,1.5;3,0,1.5;1.5,3,1.5");
@@ -263,4 +447,88 @@ mod tests {
"wide frame must accumulate, not be silently dropped"
);
}
#[test]
fn calibrated_occupancy_is_fresh_model_only_and_normalizes_runtime_frames() {
let field = fresh_test_model();
let mut history = VecDeque::new();
history.push_back(vec![1.02; 128]);
let result =
calibrated_occupancy(&field, &history, 1_500_000).expect("fresh calibrated inference");
assert!(result.person_count <= MAX_SINGLE_LINK_OCCUPANCY);
assert!(matches!(
result.method,
CalibratedOccupancyMethod::Eigenvalue | CalibratedOccupancyMethod::PerturbationEnergy
));
assert_eq!(calibrated_occupancy(&field, &history, 4_000_000), None);
assert_eq!(
calibrated_occupancy(&field, &VecDeque::new(), 1_500_000),
None
);
}
#[test]
fn adaptive_runtime_reference_scores_empty_and_rejects_shift() {
let config = FieldModelConfig {
n_links: 1,
n_subcarriers: 56,
n_modes: 3,
min_calibration_frames: 500,
min_calibration_duration_s: 0.0,
baseline_expiry_s: 60.0,
};
let mut field = FieldModel::new(config).expect("field model");
let mut calibration = Vec::new();
for frame_index in 0..600 {
let time = frame_index as f64 * 0.071;
let frame: Vec<f64> = (0..56)
.map(|subcarrier| {
let carrier = subcarrier as f64;
18.0 + carrier * 0.08
+ (time + carrier * 0.13).sin() * 0.8
+ (time * 0.37 + carrier * 0.031).cos() * 0.35
})
.collect();
field.feed_calibration(&[frame.clone()]).unwrap();
calibration.push(frame);
}
field.finalize_calibration(1_000_000, 7).unwrap();
let empty_history: VecDeque<Vec<f64>> = calibration[550..600].iter().cloned().collect();
let empty = calibrated_occupancy(&field, &empty_history, 1_500_000)
.expect("adaptive perturbation result");
assert_eq!(empty.person_count, 0);
let background = bootstrap_background_match(&field, &empty_history, 1_500_000)
.expect("background score");
assert!(background.matches_empty);
assert_eq!(background.reference_window_count, 12);
assert!(background
.score
.is_some_and(|score| (0.5..=1.0).contains(&score)));
let short_history: VecDeque<Vec<f64>> = empty_history.iter().take(49).cloned().collect();
assert!(!bootstrap_empty_prior_applies(
&field,
&short_history,
1_500_000
));
let shifted_history: VecDeque<Vec<f64>> = calibration[550..600]
.iter()
.map(|frame| {
frame
.iter()
.enumerate()
.map(|(index, value)| value + if index % 3 == 0 { 8.0 } else { -5.0 })
.collect()
})
.collect();
let shifted = calibrated_occupancy(&field, &shifted_history, 1_500_000)
.expect("shifted perturbation result");
assert!(shifted.person_count >= 1);
let shifted_background = bootstrap_background_match(&field, &shifted_history, 1_500_000)
.expect("shifted background result");
assert!(!shifted_background.matches_empty);
assert_eq!(shifted_background.score, Some(0.0));
}
}
@@ -9,6 +9,7 @@
//! - Real-time CSI introspection / low-latency tap (`introspection`, ADR-099)
pub mod bearer_auth;
pub mod bootstrap_baseline;
pub mod browser_session;
pub mod ws_ticket;
pub mod cli;
File diff suppressed because it is too large. Load diff
@@ -7,8 +7,8 @@
//! (the residual).
//!
//! # Algorithm
//! 1. Collect CSI during empty-room calibration (>=10 min wall-clock; the
//! frame target is this duration at the assumed 20 Hz single-node rate)
//! 1. Collect CSI during empty-room calibration (>=10 min wall-clock and at
//! least twenty complete runtime-sized background windows)
//! 2. Compute per-link baseline mean (Welford online accumulator)
//! 3. Decompose covariance via SVD to extract environmental modes
//! 4. At runtime: observation - baseline, project out top-K modes, keep residual
@@ -23,6 +23,8 @@ use ndarray::Array2;
use ndarray_linalg::Eigh;
#[cfg(feature = "eigenvalue")]
use ndarray_linalg::UPLO;
use serde::{Deserialize, Serialize};
use std::collections::VecDeque;
// ---------------------------------------------------------------------------
// Calibration window constants
@@ -32,15 +34,29 @@ use ndarray_linalg::UPLO;
///
/// The Welford statistics exist to absorb slow environmental variation —
/// HVAC cycles, thermal drift — so the calibration gate is expressed in
/// wall-clock time. The historical frame target of 12,000 is this duration at
/// the assumed single-node rate. Every accepted node packet advances the same
/// field model, so N nodes reach any frame target N times faster; frames alone
/// therefore cannot gate calibration (#1756).
/// wall-clock time. Every accepted node packet advances the same field model,
/// so N nodes reach any frame target N times faster; frames alone therefore
/// cannot gate calibration (#1756).
pub const CALIBRATION_DURATION_S: f64 = 600.0;
/// Assumed per-node CSI rate (Hz) used to relate the frame target to
/// [`CALIBRATION_DURATION_S`] ("10 min at 20 Hz").
pub const ASSUMED_CALIBRATION_RATE_HZ: f64 = 20.0;
/// A recent window whose mean remains within this residual energy of the
/// empty room manifold is treated as background before eigenvalue counting.
const EMPTY_ROOM_RESIDUAL_ENERGY_MAX: f64 = 1.0;
/// Runtime occupancy is evaluated over fifty frames. Keep ten raw windows
/// from the end of calibration for covariance rank learning, plus bounded
/// privacy reduced window means spanning the full capture for residual
/// conformance scoring.
const RUNTIME_OCCUPANCY_WINDOW: usize = 50;
const CALIBRATION_REFERENCE_FRAMES: usize = RUNTIME_OCCUPANCY_WINDOW * 10;
/// Minimum independent runtime sized means required to learn an empirical
/// empty room residual distribution. Wall clock duration remains the primary
/// coverage gate, so slow hardware is not forced to imitate a nominal rate.
pub const MIN_BACKGROUND_REFERENCE_WINDOWS: usize = 20;
pub const MIN_CALIBRATION_FRAMES: usize =
RUNTIME_OCCUPANCY_WINDOW * MIN_BACKGROUND_REFERENCE_WINDOWS;
const CALIBRATION_BACKGROUND_WINDOWS_MAX: usize = 512;
const EMPTY_ROOM_RESIDUAL_MARGIN: f64 = 1.5;
// ---------------------------------------------------------------------------
// Error types
@@ -261,7 +277,8 @@ impl LinkBaselineStats {
// ---------------------------------------------------------------------------
/// Configuration for field model calibration and runtime.
#[derive(Debug, Clone)]
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(deny_unknown_fields)]
pub struct FieldModelConfig {
/// Number of links in the mesh.
pub n_links: usize,
@@ -269,12 +286,11 @@ pub struct FieldModelConfig {
pub n_subcarriers: usize,
/// Number of environmental modes to retain (K). Max 5.
pub n_modes: usize,
/// Minimum calibration frames before baseline is valid. Derived from the
/// intended wall-clock window at the assumed single-node rate
/// (`CALIBRATION_DURATION_S * ASSUMED_CALIBRATION_RATE_HZ` = 12,000).
/// A frame count alone is not sufficient: N nodes streaming in aggregate
/// reach it in 1/N of the window, so `min_calibration_duration_s` also
/// gates finalization (#1756).
/// Minimum calibration frames before baseline is valid. This is the
/// runtime window size times the minimum number of independent background
/// references. A frame count alone is not sufficient: N nodes streaming
/// in aggregate reach it in 1/N of the window, so
/// `min_calibration_duration_s` also gates finalization (#1756).
pub min_calibration_frames: usize,
/// Minimum wall-clock duration of the calibration window in seconds.
/// Slow environmental variation (HVAC cycles, thermal drift) can only be
@@ -291,8 +307,7 @@ impl Default for FieldModelConfig {
n_links: 6,
n_subcarriers: 56,
n_modes: 3,
min_calibration_frames: (CALIBRATION_DURATION_S * ASSUMED_CALIBRATION_RATE_HZ)
as usize,
min_calibration_frames: MIN_CALIBRATION_FRAMES,
min_calibration_duration_s: CALIBRATION_DURATION_S,
baseline_expiry_s: 86_400.0,
}
@@ -304,7 +319,8 @@ impl Default for FieldModelConfig {
/// Learned from SVD on the covariance of CSI amplitudes during
/// empty-room calibration. The top-K modes capture environmental
/// variation (temperature, humidity, time-of-day effects).
#[derive(Debug, Clone)]
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(deny_unknown_fields)]
pub struct FieldNormalMode {
/// Per-link baseline mean: `[n_links][n_subcarriers]`.
pub baseline: Vec<Vec<f64>>,
@@ -328,6 +344,37 @@ pub struct FieldNormalMode {
/// per-window sample size. Defaults to 0.0 in the diagonal-fallback path.
/// Issue #942.
pub baseline_noise_var: f64,
/// Empty room significant eigenvalue count learned from runtime sized
/// windows. The percentile is stored instead of raw CSI. `None` identifies
/// snapshots created before runtime window calibration was introduced.
#[serde(default)]
pub baseline_runtime_eigenvalue_count: Option<usize>,
/// Number of frames used for the runtime window reference above.
#[serde(default)]
pub baseline_runtime_window_size: Option<usize>,
/// Hardware adaptive upper bound for the residual energy of an empty room
/// window mean. Only this aggregate boundary is persisted.
#[serde(default)]
pub empty_room_residual_energy_threshold: Option<f64>,
/// Sorted residual energies of privacy reduced runtime window means sampled
/// across the full calibration. Raw CSI is never persisted. These scalar
/// references support an empirical background match score without turning
/// an empty room model into positive person evidence.
#[serde(default)]
pub empty_room_residual_energy_reference: Vec<f64>,
}
/// A bounded comparison between one runtime window and the learned empty room
/// residual distribution. `score` is an empirical conformance score, not a
/// probability that a person is absent.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct EmptyRoomMatch {
pub matches_empty: bool,
pub score: Option<f64>,
pub residual_energy: f64,
pub residual_energy_threshold: f64,
pub window_size: usize,
pub reference_window_count: usize,
}
/// Body perturbation extracted from a CSI observation.
@@ -362,6 +409,18 @@ pub enum CalibrationStatus {
Expired,
}
/// Privacy reduced restart image for a completed field model.
///
/// This contains only aggregate baseline statistics and environmental modes.
/// It excludes raw CSI frames, device addresses, room names, credentials, and
/// calibration authority. A restored image is only a bootstrap prior.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(deny_unknown_fields)]
pub struct FieldModelSnapshotV1 {
pub config: FieldModelConfig,
pub modes: FieldNormalMode,
}
/// The persistent field model for a single room.
///
/// Maintains per-link Welford statistics during calibration, then
@@ -385,6 +444,16 @@ pub struct FieldModel {
/// Monotonic timestamp of the first accepted calibration frame of the
/// current session; `None` until collection starts (#1756).
calibration_started: Option<std::time::Instant>,
/// Bounded, memory only tail used to learn inference sized empty room
/// covariance statistics. It is never included in a snapshot.
calibration_reference_tail: VecDeque<Vec<Vec<f64>>>,
/// Per link sum for the current privacy reduced runtime sized window.
calibration_window_sum: Vec<Vec<f64>>,
/// Number of accepted observations in `calibration_window_sum`.
calibration_window_count: usize,
/// Runtime sized per link means spanning the full calibration. This is
/// bounded, memory only, and reduced to scalar residuals at finalization.
calibration_window_means: VecDeque<Vec<Vec<f64>>>,
}
/// Diagonal variance fallback for when full covariance SVD is unavailable.
@@ -440,6 +509,133 @@ fn diagonal_fallback(
(mode_energies, environmental_modes, baseline_count)
}
#[cfg(feature = "eigenvalue")]
fn significant_eigenvalue_count(
frames: &[Vec<f64>],
n_subcarriers: usize,
baseline_noise_var: f64,
) -> Option<usize> {
if frames.len() < 10 {
return None;
}
let mut mean = vec![0.0_f64; n_subcarriers];
let mut count = 0_usize;
for frame in frames {
if frame.len() >= n_subcarriers {
for index in 0..n_subcarriers {
mean[index] += frame[index];
}
count += 1;
}
}
if count < 2 {
return None;
}
for value in &mut mean {
*value /= count as f64;
}
let mut covariance = Array2::<f64>::zeros((n_subcarriers, n_subcarriers));
for frame in frames {
if frame.len() >= n_subcarriers {
for row in 0..n_subcarriers {
let centered_row = frame[row] - mean[row];
for column in row..n_subcarriers {
let value = centered_row * (frame[column] - mean[column]);
covariance[[row, column]] += value;
if row != column {
covariance[[column, row]] += value;
}
}
}
}
}
covariance *= 1.0 / (count as f64 - 1.0);
let (eigenvalues, _) = covariance.eigh(UPLO::Upper).ok()?;
let mut positive: Vec<f64> = eigenvalues
.iter()
.copied()
.filter(|value| *value > 1e-10)
.collect();
positive.sort_by(|left, right| left.partial_cmp(right).unwrap_or(std::cmp::Ordering::Equal));
let local_noise_var = if positive.len() >= 4 {
let half = positive.len() / 2;
positive[..half].iter().sum::<f64>() / half as f64
} else {
positive.first().copied()?
};
let noise_var = if baseline_noise_var > 0.0 {
local_noise_var.max(baseline_noise_var)
} else {
local_noise_var
};
let ratio = n_subcarriers as f64 / count as f64;
let threshold = noise_var * (1.0 + ratio.sqrt()).powi(2);
Some(
eigenvalues
.iter()
.filter(|&&value| value > threshold)
.count(),
)
}
fn residual_energy_for_link(
modes: &FieldNormalMode,
link_index: usize,
observation: &[f64],
) -> Option<f64> {
let baseline = modes.baseline.get(link_index)?;
if observation.len() != baseline.len() {
return None;
}
let mut residual: Vec<f64> = observation
.iter()
.zip(baseline.iter())
.map(|(value, reference)| value - reference)
.collect();
for mode in &modes.environmental_modes {
let projection: f64 = residual
.iter()
.zip(mode.iter())
.map(|(value, basis)| value * basis)
.sum();
for (value, basis) in residual.iter_mut().zip(mode.iter()) {
*value -= projection * basis;
}
}
Some(
residual
.iter()
.map(|value| value * value)
.sum::<f64>()
.sqrt(),
)
}
fn percentile_95(mut values: Vec<f64>) -> Option<f64> {
values.retain(|value| value.is_finite());
values.sort_by(|left, right| left.partial_cmp(right).unwrap_or(std::cmp::Ordering::Equal));
let index = values
.len()
.saturating_mul(95)
.div_ceil(100)
.saturating_sub(1);
values.get(index).copied()
}
#[cfg(feature = "eigenvalue")]
fn percentile_95_usize(mut values: Vec<usize>) -> Option<usize> {
values.sort_unstable();
let index = values
.len()
.saturating_mul(95)
.div_ceil(100)
.saturating_sub(1);
values.get(index).copied()
}
impl FieldModel {
/// Create a new field model for the given configuration.
pub fn new(config: FieldModelConfig) -> Result<Self, FieldModelError> {
@@ -460,6 +656,7 @@ impl FieldModel {
let link_stats = (0..config.n_links)
.map(|_| LinkBaselineStats::new(config.n_subcarriers))
.collect();
let calibration_window_sum = vec![vec![0.0_f64; config.n_subcarriers]; config.n_links];
Ok(Self {
config,
@@ -470,6 +667,10 @@ impl FieldModel {
covariance_sum: None,
covariance_count: 0,
calibration_started: None,
calibration_reference_tail: VecDeque::with_capacity(CALIBRATION_REFERENCE_FRAMES),
calibration_window_sum,
calibration_window_count: 0,
calibration_window_means: VecDeque::with_capacity(CALIBRATION_BACKGROUND_WINDOWS_MAX),
})
}
@@ -483,6 +684,123 @@ impl FieldModel {
self.modes.as_ref()
}
/// Hardware adaptive empty room boundary learned at finalization. Returns
/// `None` for legacy snapshots that predate runtime window calibration.
pub fn empty_room_residual_energy_threshold(&self) -> Option<f64> {
self.modes
.as_ref()
.and_then(|modes| modes.empty_room_residual_energy_threshold)
}
/// Export aggregate model state suitable for local restart persistence.
/// Raw calibration observations and Welford accumulators are not exported.
pub fn export_snapshot(&self) -> Result<FieldModelSnapshotV1, FieldModelError> {
let modes = self.modes.clone().ok_or(FieldModelError::NotCalibrated)?;
Ok(FieldModelSnapshotV1 {
config: self.config.clone(),
modes,
})
}
/// Restore a bounded, validated aggregate snapshot.
pub fn from_snapshot(
snapshot: FieldModelSnapshotV1,
current_us: u64,
) -> Result<Self, FieldModelError> {
let config = &snapshot.config;
if config.n_links > 16 || config.n_subcarriers > 2_048 {
return Err(FieldModelError::InvalidConfig(
"snapshot dimensions exceed bounded limits".into(),
));
}
if !config.min_calibration_duration_s.is_finite()
|| config.min_calibration_duration_s < 0.0
|| !config.baseline_expiry_s.is_finite()
|| config.baseline_expiry_s <= 0.0
|| config.baseline_expiry_s > 604_800.0
{
return Err(FieldModelError::InvalidConfig(
"snapshot timing values are invalid".into(),
));
}
let modes = &snapshot.modes;
let baseline_shape_valid = modes.baseline.len() == config.n_links
&& modes
.baseline
.iter()
.all(|link| link.len() == config.n_subcarriers);
let mode_shape_valid = modes.environmental_modes.len() == modes.mode_energies.len()
&& modes.environmental_modes.len() <= config.n_modes
&& modes
.environmental_modes
.iter()
.all(|mode| mode.len() == config.n_subcarriers);
let values_valid = modes
.baseline
.iter()
.flatten()
.chain(modes.environmental_modes.iter().flatten())
.all(|value| value.is_finite())
&& modes
.mode_energies
.iter()
.all(|value| value.is_finite() && *value >= 0.0)
&& modes.variance_explained.is_finite()
&& (0.0..=10.0).contains(&modes.variance_explained)
&& modes.baseline_noise_var.is_finite()
&& modes.baseline_noise_var >= 0.0
&& modes.baseline_eigenvalue_count <= config.n_subcarriers
&& modes
.baseline_runtime_eigenvalue_count
.is_none_or(|count| count <= config.n_subcarriers)
&& modes
.baseline_runtime_window_size
.is_none_or(|size| (10..=CALIBRATION_REFERENCE_FRAMES).contains(&size))
&& modes
.empty_room_residual_energy_threshold
.is_none_or(|threshold| threshold.is_finite() && threshold >= 0.0)
&& modes.empty_room_residual_energy_reference.len()
<= CALIBRATION_BACKGROUND_WINDOWS_MAX
&& modes
.empty_room_residual_energy_reference
.iter()
.all(|value| value.is_finite() && *value >= 0.0)
&& modes
.empty_room_residual_energy_reference
.windows(2)
.all(|window| window[0] <= window[1]);
if !baseline_shape_valid || !mode_shape_valid || !values_valid {
return Err(FieldModelError::InvalidConfig(
"snapshot modes are malformed or non finite".into(),
));
}
if modes.calibrated_at_us == 0
|| modes.calibrated_at_us > current_us.saturating_add(300_000_000)
{
return Err(FieldModelError::InvalidConfig(
"snapshot calibration time is invalid".into(),
));
}
let elapsed_s = current_us.saturating_sub(modes.calibrated_at_us) as f64 / 1_000_000.0;
if elapsed_s > config.baseline_expiry_s {
return Err(FieldModelError::BaselineExpired {
elapsed_s,
max_s: config.baseline_expiry_s,
});
}
let mut model = Self::new(snapshot.config)?;
model.status = if elapsed_s > model.config.baseline_expiry_s * 0.5 {
CalibrationStatus::Stale
} else {
CalibrationStatus::Fresh
};
model.last_calibration_us = modes.calibrated_at_us;
model.modes = Some(snapshot.modes);
Ok(model)
}
/// Number of calibration frames collected so far.
pub fn calibration_frame_count(&self) -> u64 {
self.link_stats
@@ -568,6 +886,39 @@ impl FieldModel {
// Count once per frame (not per link) for correct MP ratio
self.covariance_count += 1;
if self.calibration_reference_tail.len() == CALIBRATION_REFERENCE_FRAMES {
self.calibration_reference_tail.pop_front();
}
self.calibration_reference_tail
.push_back(observations.to_vec());
for (link_sum, observation) in self
.calibration_window_sum
.iter_mut()
.zip(observations.iter())
{
for (sum, value) in link_sum.iter_mut().zip(observation.iter()) {
*sum += value;
}
}
self.calibration_window_count += 1;
if self.calibration_window_count == RUNTIME_OCCUPANCY_WINDOW {
let divisor = self.calibration_window_count as f64;
let means: Vec<Vec<f64>> = self
.calibration_window_sum
.iter()
.map(|link_sum| link_sum.iter().map(|value| value / divisor).collect())
.collect();
if self.calibration_window_means.len() == CALIBRATION_BACKGROUND_WINDOWS_MAX {
self.calibration_window_means.pop_front();
}
self.calibration_window_means.push_back(means);
for link_sum in &mut self.calibration_window_sum {
link_sum.fill(0.0);
}
self.calibration_window_count = 0;
}
Ok(())
}
@@ -702,8 +1053,7 @@ impl FieldModel {
}
Err(_) => {
// Fallback to diagonal approximation on SVD failure
let (e, m, b) =
diagonal_fallback(&self.link_stats, n_sc, n_modes);
let (e, m, b) = diagonal_fallback(&self.link_stats, n_sc, n_modes);
(e, m, b, 0.0_f64)
}
}
@@ -755,7 +1105,7 @@ impl FieldModel {
0.0
};
let field_mode = FieldNormalMode {
let mut field_mode = FieldNormalMode {
baseline,
environmental_modes,
mode_energies,
@@ -764,15 +1114,142 @@ impl FieldModel {
geometry_hash,
baseline_eigenvalue_count: baseline_eig_count,
baseline_noise_var,
baseline_runtime_eigenvalue_count: None,
baseline_runtime_window_size: None,
empty_room_residual_energy_threshold: None,
empty_room_residual_energy_reference: Vec::new(),
};
// The full calibration covariance and a fifty frame runtime covariance
// have different Marcenko Pastur aspect ratios. This first version is
// deliberately single link: combining link specific residual scales
// requires a separate multistatic calibration model.
if self.config.n_links == 1 {
let reference_frames: Vec<Vec<f64>> = self
.calibration_reference_tail
.iter()
.filter_map(|observation| observation.first().cloned())
.collect();
let complete_windows: Vec<&[Vec<f64>]> = reference_frames
.chunks(RUNTIME_OCCUPANCY_WINDOW)
.filter(|window| window.len() == RUNTIME_OCCUPANCY_WINDOW)
.collect();
#[cfg(feature = "eigenvalue")]
{
let counts: Vec<usize> = complete_windows
.iter()
.filter_map(|window| {
significant_eigenvalue_count(window, n_sc, baseline_noise_var)
})
.collect();
field_mode.baseline_runtime_eigenvalue_count = percentile_95_usize(counts);
}
let mut residual_energies: Vec<f64> = self
.calibration_window_means
.iter()
.filter_map(|means| means.first())
.filter_map(|mean| residual_energy_for_link(&field_mode, 0, mean))
.filter(|value| value.is_finite() && *value >= 0.0)
.collect();
if residual_energies.is_empty() {
residual_energies = complete_windows
.iter()
.filter_map(|window| {
let mut mean = vec![0.0_f64; n_sc];
for frame in *window {
for (sum, value) in mean.iter_mut().zip(frame.iter()) {
*sum += value;
}
}
for value in &mut mean {
*value /= window.len() as f64;
}
residual_energy_for_link(&field_mode, 0, &mean)
})
.collect();
}
residual_energies.sort_by(|left, right| {
left.partial_cmp(right).unwrap_or(std::cmp::Ordering::Equal)
});
field_mode.empty_room_residual_energy_threshold =
percentile_95(residual_energies.clone()).map(|value| {
(value * EMPTY_ROOM_RESIDUAL_MARGIN).max(EMPTY_ROOM_RESIDUAL_ENERGY_MAX)
});
field_mode.empty_room_residual_energy_reference = residual_energies;
if field_mode.baseline_runtime_eigenvalue_count.is_some()
|| field_mode.empty_room_residual_energy_threshold.is_some()
{
field_mode.baseline_runtime_window_size = Some(RUNTIME_OCCUPANCY_WINDOW);
}
}
self.modes = Some(field_mode);
self.status = CalibrationStatus::Fresh;
self.last_calibration_us = timestamp_us;
self.calibration_reference_tail.clear();
self.calibration_window_means.clear();
for link_sum in &mut self.calibration_window_sum {
link_sum.fill(0.0);
}
self.calibration_window_count = 0;
Ok(self.modes.as_ref().unwrap())
}
/// Compare a runtime sized single link window with the learned empty room
/// distribution. The score is empirical conformance while the learned
/// boundary holds. Crossing the boundary returns zero. Legacy snapshots
/// without residual references retain binary suppression but no score.
pub fn empty_room_match(&self, recent_frames: &[Vec<f64>]) -> Option<EmptyRoomMatch> {
let modes = self.modes.as_ref()?;
if self.config.n_links != 1 {
return None;
}
let window_size = modes.baseline_runtime_window_size?;
if recent_frames.len() < window_size {
return None;
}
let mut mean = vec![0.0_f64; self.config.n_subcarriers];
for frame in recent_frames.iter().take(window_size) {
if frame.len() < self.config.n_subcarriers {
return None;
}
for (sum, value) in mean.iter_mut().zip(frame.iter()) {
*sum += value;
}
}
for value in &mut mean {
*value /= window_size as f64;
}
let residual_energy = residual_energy_for_link(modes, 0, &mean)?;
let residual_energy_threshold = modes
.empty_room_residual_energy_threshold?
.max(EMPTY_ROOM_RESIDUAL_ENERGY_MAX);
let matches_empty = residual_energy <= residual_energy_threshold;
let reference_window_count = modes.empty_room_residual_energy_reference.len();
let score = if !matches_empty {
Some(0.0)
} else if reference_window_count == 0 {
None
} else {
let at_or_below = modes
.empty_room_residual_energy_reference
.partition_point(|value| *value <= residual_energy);
let empirical_quantile = at_or_below as f64 / reference_window_count as f64;
Some((1.0 - empirical_quantile * 0.5).clamp(0.5, 1.0))
};
Some(EmptyRoomMatch {
matches_empty,
score,
residual_energy,
residual_energy_threshold,
window_size,
reference_window_count,
})
}
/// Extract body perturbation from a runtime observation.
///
/// Subtracts baseline, projects out environmental modes, returns residual.
@@ -883,6 +1360,22 @@ impl FieldModel {
*m /= count as f64;
}
// Rank alone is not person evidence. A short continuation window can
// expose more significant eigenvalues while its mean remains on the
// learned empty room manifold. Use the hardware adaptive residual
// boundary when available.
let empty_room_residual_energy_max = modes
.empty_room_residual_energy_threshold
.unwrap_or(EMPTY_ROOM_RESIDUAL_ENERGY_MAX)
.max(EMPTY_ROOM_RESIDUAL_ENERGY_MAX);
let mean_residual_energy = self
.extract_perturbation(&[mean.clone()])
.ok()
.map(|perturbation| perturbation.total_energy);
if mean_residual_energy.is_some_and(|energy| energy <= empty_room_residual_energy_max) {
return Ok(0);
}
let mut cov = Array2::<f64>::zeros((n, n));
for frame in recent_frames {
if frame.len() >= n {
@@ -943,7 +1436,23 @@ impl FieldModel {
let mp_threshold = noise_var * (1.0 + ratio.sqrt()).powi(2);
let significant = eigenvalues.iter().filter(|&&ev| ev > mp_threshold).count();
let occupancy = significant.saturating_sub(modes.baseline_eigenvalue_count);
let reference_count = if modes.baseline_runtime_window_size == Some(count) {
modes
.baseline_runtime_eigenvalue_count
.unwrap_or(modes.baseline_eigenvalue_count)
} else {
modes.baseline_eigenvalue_count
};
let rank_occupancy = significant.saturating_sub(reference_count);
// A stable person can shift the window mean without increasing its
// covariance rank. Preserve at least one occupant after the learned
// empty room residual boundary is exceeded.
let occupancy =
if mean_residual_energy.is_some_and(|energy| energy > empty_room_residual_energy_max) {
rank_occupancy.max(1)
} else {
rank_occupancy
};
Ok(occupancy.min(10)) // Cap at 10 persons
}
@@ -982,6 +1491,12 @@ impl FieldModel {
self.covariance_sum = None;
self.covariance_count = 0;
self.calibration_started = None;
self.calibration_reference_tail.clear();
self.calibration_window_count = 0;
for link_sum in &mut self.calibration_window_sum {
link_sum.fill(0.0);
}
self.calibration_window_means.clear();
}
}
@@ -1502,6 +2017,131 @@ mod tests {
assert_eq!(occupancy, 0, "Noise-only frames should yield 0 occupancy");
}
#[cfg(feature = "eigenvalue")]
#[test]
fn runtime_sized_empty_reference_scores_held_out_background() {
let config = FieldModelConfig {
n_links: 1,
n_subcarriers: 56,
n_modes: 3,
min_calibration_frames: 500,
min_calibration_duration_s: 0.0,
baseline_expiry_s: 86_400.0,
};
let mut model = FieldModel::new(config).unwrap();
let mut calibration = Vec::new();
for frame_index in 0..600 {
let time = frame_index as f64 * 0.071;
let frame: Vec<f64> = (0..56)
.map(|subcarrier| {
let carrier = subcarrier as f64;
18.0 + carrier * 0.08
+ (time + carrier * 0.13).sin() * 0.8
+ (time * 0.37 + carrier * 0.031).cos() * 0.35
})
.collect();
model.feed_calibration(&[frame.clone()]).unwrap();
calibration.push(frame);
}
model.finalize_calibration(1_000_000, 0).unwrap();
let modes = model.modes().unwrap();
assert_eq!(
modes.baseline_runtime_window_size,
Some(RUNTIME_OCCUPANCY_WINDOW)
);
assert!(modes.baseline_runtime_eigenvalue_count.is_some());
assert!(modes.empty_room_residual_energy_threshold.is_some());
assert_eq!(modes.empty_room_residual_energy_reference.len(), 12);
let held_out_empty = calibration[550..600].to_vec();
assert_eq!(model.estimate_occupancy(&held_out_empty).unwrap(), 0);
let background = model
.empty_room_match(&held_out_empty)
.expect("background match");
assert!(background.matches_empty);
assert_eq!(background.reference_window_count, 12);
assert!(background
.score
.is_some_and(|score| (0.5..=1.0).contains(&score)));
}
#[cfg(feature = "eigenvalue")]
#[test]
fn runtime_reference_rejects_shifted_held_out_window() {
let config = FieldModelConfig {
n_links: 1,
n_subcarriers: 56,
n_modes: 3,
min_calibration_frames: 500,
min_calibration_duration_s: 0.0,
baseline_expiry_s: 86_400.0,
};
let mut model = FieldModel::new(config).unwrap();
for frame_index in 0..600 {
let time = frame_index as f64 * 0.071;
let frame: Vec<f64> = (0..56)
.map(|subcarrier| {
let carrier = subcarrier as f64;
18.0 + carrier * 0.08
+ (time + carrier * 0.13).sin() * 0.8
+ (time * 0.37 + carrier * 0.031).cos() * 0.35
})
.collect();
model.feed_calibration(&[frame]).unwrap();
}
model.finalize_calibration(1_000_000, 0).unwrap();
let shifted: Vec<Vec<f64>> = (0..RUNTIME_OCCUPANCY_WINDOW)
.map(|frame_index| {
let time = (600 + frame_index) as f64 * 0.071;
(0..56)
.map(|subcarrier| {
let carrier = subcarrier as f64;
let body_shift = if subcarrier % 3 == 0 { 8.0 } else { -5.0 };
18.0 + carrier * 0.08
+ (time + carrier * 0.13).sin() * 0.8
+ (time * 0.37 + carrier * 0.031).cos() * 0.35
+ body_shift
})
.collect()
})
.collect();
assert!(model.estimate_occupancy(&shifted).unwrap() >= 1);
let background = model.empty_room_match(&shifted).expect("shifted match");
assert!(!background.matches_empty);
assert_eq!(background.score, Some(0.0));
}
#[test]
fn runtime_background_reference_is_single_link_only() {
let config = FieldModelConfig {
n_links: 2,
n_subcarriers: 8,
n_modes: 2,
min_calibration_frames: 100,
min_calibration_duration_s: 0.0,
baseline_expiry_s: 86_400.0,
};
let mut model = FieldModel::new(config).unwrap();
for frame_index in 0..100 {
let phase = frame_index as f64 * 0.1;
model
.feed_calibration(&[
(0..8).map(|index| phase + index as f64).collect(),
(0..8).map(|index| phase - index as f64).collect(),
])
.unwrap();
}
model.finalize_calibration(1_000_000, 0).unwrap();
let modes = model.modes().unwrap();
assert_eq!(modes.baseline_runtime_window_size, None);
assert_eq!(modes.baseline_runtime_eigenvalue_count, None);
assert_eq!(modes.empty_room_residual_energy_threshold, None);
assert!(modes.empty_room_residual_energy_reference.is_empty());
}
#[test]
fn test_baseline_eigenvalue_count_stored() {
let config = FieldModelConfig {
@@ -1538,6 +2178,48 @@ mod tests {
);
}
#[test]
fn snapshot_round_trip_preserves_aggregate_modes_without_raw_frames() {
let config = make_config(1, 4, 10);
let mut model = FieldModel::new(config).unwrap();
for i in 0..10 {
model
.feed_calibration(&[vec![1.0 + i as f64 * 0.01, 2.0, 3.0, 4.0]])
.unwrap();
}
model.finalize_calibration(1_000_000, 0xCA1).unwrap();
let snapshot = model.export_snapshot().unwrap();
let restored = FieldModel::from_snapshot(snapshot.clone(), 2_000_000).unwrap();
assert_eq!(restored.status(), CalibrationStatus::Fresh);
assert_eq!(restored.calibration_frame_count(), 0);
assert_eq!(restored.export_snapshot().unwrap(), snapshot);
}
#[test]
fn snapshot_restore_rejects_expired_and_malformed_images() {
let config = make_config(1, 4, 10);
let mut model = FieldModel::new(config).unwrap();
for _ in 0..10 {
model.feed_calibration(&[vec![1.0, 2.0, 3.0, 4.0]]).unwrap();
}
model.finalize_calibration(1_000_000, 0).unwrap();
let snapshot = model.export_snapshot().unwrap();
assert!(matches!(
FieldModel::from_snapshot(snapshot.clone(), 90_000_000_000),
Err(FieldModelError::BaselineExpired { .. })
));
let mut malformed = snapshot;
malformed.modes.baseline[0].pop();
assert!(matches!(
FieldModel::from_snapshot(malformed, 2_000_000),
Err(FieldModelError::InvalidConfig(_))
));
}
#[test]
fn test_environmental_projection_removes_drift() {
let config = make_config(1, 4, 10);
@@ -1572,14 +2254,15 @@ mod tests {
// -----------------------------------------------------------------
#[test]
fn test_default_frame_target_derives_from_duration() {
fn test_default_frame_target_covers_background_reference_windows() {
let cfg = FieldModelConfig::default();
assert_eq!(cfg.min_calibration_duration_s, CALIBRATION_DURATION_S);
assert_eq!(
cfg.min_calibration_frames,
(CALIBRATION_DURATION_S * ASSUMED_CALIBRATION_RATE_HZ) as usize,
"frame target must be the intended window at the assumed rate"
cfg.min_calibration_frames, MIN_CALIBRATION_FRAMES,
"frame target must cover the minimum independent runtime windows"
);
assert_eq!(MIN_CALIBRATION_FRAMES, 1_000);
assert_eq!(MIN_BACKGROUND_REFERENCE_WINDOWS, 20);
}
#[test]