From 8cd8742d2bd639af33ca84342fde7ef13963ccf5 Mon Sep 17 00:00:00 2001 From: ruv Date: Tue, 15 Sep 2026 13:57:01 -0400 Subject: [PATCH 1/4] Add bounded RTL8721Dx RAC1 CSI ingest to current RuView main --- ...R-323-rtl8721dx-ameba-csi-wire-protocol.md | 218 +++++++ scripts/benchmark-rtl8721dx-csi.py | 201 +++++++ .../benches/realtek_csi_bench.rs | 69 +++ .../src/bin/realtek-csi-sim.rs | 107 ++++ v2/crates/wifi-densepose-hardware/src/lib.rs | 3 + .../src/realtek_csi.rs | 563 ++++++++++++++++++ .../wifi-densepose-sensing-server/src/main.rs | 94 ++- .../src/realtek_csi.rs | 124 ++++ 8 files changed, 1377 insertions(+), 2 deletions(-) create mode 100644 docs/adr/ADR-323-rtl8721dx-ameba-csi-wire-protocol.md create mode 100644 scripts/benchmark-rtl8721dx-csi.py create mode 100644 v2/crates/wifi-densepose-hardware/benches/realtek_csi_bench.rs create mode 100644 v2/crates/wifi-densepose-hardware/src/bin/realtek-csi-sim.rs create mode 100644 v2/crates/wifi-densepose-hardware/src/realtek_csi.rs create mode 100644 v2/crates/wifi-densepose-sensing-server/src/realtek_csi.rs diff --git a/docs/adr/ADR-323-rtl8721dx-ameba-csi-wire-protocol.md b/docs/adr/ADR-323-rtl8721dx-ameba-csi-wire-protocol.md new file mode 100644 index 000000000..228e99a29 --- /dev/null +++ b/docs/adr/ADR-323-rtl8721dx-ameba-csi-wire-protocol.md @@ -0,0 +1,218 @@ +# ADR-323: RTL8721Dx (Realtek AmebaDplus) CSI Wire Protocol + +- **Status**: accepted +- **Date**: 2026-09-15 +- **Deciders**: RuView maintainers +- **Tags**: realtek, ameba, amebadplus, rtl8721dx, csi, protocol, rust, udp + +## Context + +Unlike the MediaTek/Qualcomm CSI paths (ADR-266..269), which are simulator-first +because no supported public vendor interface exports complex CSI, Realtek's +official Ameba RTOS SDK (`Ameba-AIoT/ameba-rtos`, Apache-2.0) documents and +exposes a real, callback-driven CSI API for the RTL8721Dx ("AmebaDplus") +family: `wifi_csi_config()` plus the `RTW_EVENT_CSI_DONE` event delivering a +`struct rtw_event_csi_report_info` (per-tone signed I/Q, 8-bit or 16-bit +precision, selectable tone decimation, passive or active/triggered capture). +This has been confirmed directly against SDK source +(`component/wifi/api/wifi_api_event.h`) and against the official +`example/wifi/wifi_csi` example, and validated on real PKM8721DAF-C13-F10 +hardware (build+flash+boot confirmed on branch `feat/rtl8721dx-csi-bringup`). + +The RTL8721Dx is a 1x1 WiFi 4 part — single TX/RX chain, not MIMO — so the +MediaTek/Qualcomm wire format (Tx/Rx dimensioned matrices, chipset chain +counts) does not fit. The vendor SDK's raw struct is also not a safe host +wire format as-is: it has no CRC, a C flexible-array-member payload, and +fields sized for firmware convenience (e.g. a wrapping `u32` microsecond +timestamp) rather than network robustness. + +Per the existing repo convention (ESP32/ADR-018, MediaTek/ADR-266-267, +Qualcomm/ADR-268-269, RTL8720F radar/ADR-263-264), each vendor CSI/radar +source gets its own self-delimiting envelope and its own Rust module pair +(`wifi-densepose-hardware` for framing+codec+simulator, +`wifi-densepose-sensing-server` for the bounded JSON-safe snapshot), never a +shared "generic vendor CSI" struct that would either lose vendor-specific +fields or invite magic-space collisions. `SourceKind::Realtek` (value 2) is +already assigned to the RTL8720F FMCW radar path (ADR-263/264) and must not +be reused for CSI. + +## Decision + +Define `RAC1` ("Realtek Ameba CSI", version 1) as a little-endian, +self-delimiting envelope, magic `0x3143_4152` (distinct from `MTC1` +`0x3143544d`, `QCS1` 0x31535143, `RTR1` 0x31525452, and all ADR-018 +`0xC511000x` siblings): + +Fixed 49-byte header (explicit field-by-field serialization, no `#[repr(C)]` +padding, matching the MediaTek/Qualcomm modules' style): + +| Field | Type | Notes | +|---|---|---| +| magic | u32 LE | `0x3143_4152` | +| version | u8 | 1 | +| header_len | u16 LE | 49 | +| frame_len | u32 LE | header + payload + 4-byte CRC | +| node_id | u8 | disambiguates multi-board illuminator/receiver deployments; 0 = unset | +| csi_mode | u8 | 0=passive, 1=active, 2=board-to-board (`CsiMode`) | +| sequence | u32 LE | from firmware `csi_sequence` | +| timestamp_us | u32 LE | from firmware `hw_assigned_timestamp` (hardware-clock microseconds, wraps ~71 min — consumers must not assume monotonic across a capture longer than that) | +| peer_mac | [u8; 6] | firmware `mac_addr` | +| trig_mac | [u8; 6] | firmware `trig_addr` | +| channel | u8 | operating channel | +| bandwidth | u8 | 0=20MHz, 1=40MHz | +| rx_rate | u8 | firmware `rx_rate` | +| protocol_mode | u8 | 0=OFDM,1=HT,2=VHT,3=HE (`ProtocolMode`) | +| num_sub_carrier | u16 LE | tone count in payload | +| num_bit_per_tone | u8 | 16 (8-bit I + 8-bit Q) or 32 (16-bit I + 16-bit Q) | +| decimation | u8 | tone group (1/2/4/8 — every Nth tone) | +| rssi_dbm | i8 | single RF chain (1x1 hardware) | +| rxsc | u8 | sub-20 MHz channel used | +| csi_valid | u8 | 0/1 from firmware | +| flags | u8 | bit 0 = SYNTHETIC (simulator provenance, never clearable downstream) | +| payload_len | u32 LE | bytes, == `num_sub_carrier * (num_bit_per_tone / 8)` | + +Payload: `num_sub_carrier` consecutive (I, Q) tone pairs, each +`num_bit_per_tone / 16` bytes per component (1 byte/component when +`num_bit_per_tone == 16`, 2 bytes/component little-endian signed when `== 32`) +— i.e. exactly the byte layout the firmware already produces in +`csi_data[]`, just framed. CRC-32/IEEE over header+payload, final 4 +little-endian bytes. One envelope per UDP datagram. + +Parsers reject: unknown version, wrong `header_len`, `frame_len` exceeding +the IPv4 UDP payload cap (65,507 bytes), `num_bit_per_tone` outside +`{16, 32}`, `payload_len` inconsistent with `num_sub_carrier * +(num_bit_per_tone/8)`, trailing bytes, and bad CRC — mirroring the +MediaTek/Qualcomm parser contract exactly. + +Host side: add `realtek_csi.rs` to both +`wifi-densepose-hardware` (frame + codec + simulator, mirroring +`mediatek_csi.rs`/`qualcomm_csi.rs`) and `wifi-densepose-sensing-server` +(bounded `RealtekCsiSnapshot`, mirroring `QualcommCsiSnapshot`), wired into +the same UDP listener loop and `/api/v1/csi/realtek/latest` route as the +existing vendor sources. The initial Mac compatibility port keeps Realtek CSI +as a separate bounded diagnostic stream. Fresh ESP32 frames remain the primary +sensing source. This change does not add `SourceKind::RealtekCsi` to a signed +ingest contract; that requires a separate reviewed identity-bound hardware +proof. RAC1 CRC checks corruption, not origin. + +Firmware side (`example/wifi/wifi_csi` derivative, built against the +official Ameba RTOS SDK, tracked outside this repository per the existing +`CLAUDE.local.md` convention — the vendor SDK itself is never vendored into +this repo): join the board to the deployment's WiFi AP in STA mode +(passive CSI against the real AP as peer, since only one physical board is +available in this bring-up), read the AP's BSSID at runtime after +association (never hardcode a MAC into firmware source), pack each +`RTW_EVENT_CSI_DONE` report into one `RAC1` datagram, and `sendto()` it over +UDP to the host sensing-server's existing CSI port. + +## Consequences + +### Positive + +- CSI is real, MEASURED hardware output from day one — no simulator-first + bridge phase is needed the way MediaTek/Qualcomm required, because the + vendor SDK genuinely exposes the capability and we have the board in hand. +- One board is sufficient for a first useful signal (passive CSI against the + home AP); active and board-to-board modes are additive later without a + wire-format break (`csi_mode` already reserves the values). +- Reuses the bounded UDP parser and status-route pattern without changing + existing ESP32 sensing. Unsigned RAC1 diagnostics are not authorized as + calibrated presence, person count, pose, or vital evidence. + +### Negative + +- 1x1 hardware means no spatial (multi-antenna) diversity from a single + board; multi-board deployment is required for the differential/localization + use cases described in the original RTL8721Dx capability analysis. +- The 71-minute microsecond timestamp wraparound (inherited from the + firmware's `u32` field) means long unattended captures need host-side + sequence-based reordering, not raw timestamp arithmetic, across a wrap. + +### Neutral + +- The RAC1 envelope intentionally does not carry the SDK's raw + `csi_signature` (0xABCD) marker — our own `magic`/CRC supersede it as the + framing integrity check. + +## Hardware gates + +- [x] Confirm SDK source exposes `RTW_EVENT_CSI_DONE` / `rtw_event_csi_report_info` (done — `component/wifi/api/wifi_api_event.h`). +- [x] Build and flash a baseline image on real PKM8721DAF-C13-F10 hardware, confirm boot over serial (done — `feat/rtl8721dx-csi-bringup`). +- [x] Flash the CSI-streaming firmware and confirm `RAC1` datagrams parse on the host from the real board (done, 2026-09-15 — see below). +- [x] Record MEASURED throughput/latency/jitter from the real running firmware (done, 2026-09-15 — see below). +- [ ] A human walk-test across the link (amplitude perturbation vs. static baseline) is deferred — requires the operator physically present; not claimed until performed. + +### MEASURED: real RAC1 capture and sustained throughput (2026-09-15) + +Passive CSI (`RTW_CSI_MODE_NORMAL`, `RTW_CSI_TRIG_BEACON | RTW_CSI_TRIG_DATA | RTW_CSI_TRIG_QOS_DATA`) +against the deployment AP. A single `RAC1` datagram was first +cross-checked field-by-field against the firmware's own serial log of the +same report — every byte matched: magic `0x31434152`, RSSI byte exact match, +channel/bandwidth/protocol/subcarrier-count exact match, 157-byte total frame +(49-byte header + 104-byte payload + 4-byte CRC, exactly as specified above). + +`RTW_CSI_MODE_RX_RESP` (estimate CSI from ACKs to the STA's own transmissions) +never fired `RTW_EVENT_CSI_DONE` across two full test runs with confirmed +real traffic (not a hang) — this mode does not work for this hardware/AP +combination and `RTW_CSI_MODE_NORMAL` with beacon+data triggers is the +working configuration. + +**An earlier note here claimed `NORMAL` mode fires only once per CSI enable +— that was wrong**, caused by a debug log line gated to print only every +50th frame (`frame_count % 50 == 1`), which was misread as "only one report +total." A follow-up capture with a UDP listener running concurrently with a +board reset measured **277 real RAC1 datagrams in 40 seconds (43,489 bytes)**, +with `csi_sequence` climbing continuously and monotonically in the firmware's +own serial log across six separate 50-frame log checkpoints in one 45s +session — CSI reporting is continuous, roughly 7-12 fps, not sparse. + +### MEASURED: throughput optimization pass (2026-09-15) + +Formalized as a repeatable tool: `scripts/benchmark-rtl8721dx-csi.py` (listens +on the sensing-server's CSI UDP port, reports fps/throughput/RSSI/inter-frame +interval/sequence-gap stats from real received `RAC1` datagrams). + +**Baseline** (`trig_period=200` [64ms], `trig_frame_ctrl=0`, +`CSI_REPORT_BUF_NUM=1`), 30s capture: + +| Metric | Value | +|---|---| +| RAC1 fps | 14.67 | +| Throughput | 2,302.7 bytes/s | +| Mean inter-frame interval | 68.3 ms | +| RSSI | mean -34.4 dBm | + +`trig_period` (units of 320µs) was hypothesized as the rate-limiting +throttle — the observed 68.3ms mean interval tracks the 64ms configured +period almost exactly. Confirmed by tuning: + +1. `trig_period`: 200 → 15 (the SDK doc's stated minimum recommended value, + ~4.8ms) +2. `trig_frame_ctrl`: 0 → `RTW_CSI_TRIG_ACK` (second trigger source: every + link-layer ACK the AP sends for the heartbeat's own outgoing traffic, on + top of the existing beacon trigger) +3. `CSI_REPORT_BUF_NUM`: 1 → 4 (a "lack of csi buf!" drop was observed even + at the slower baseline rate; more headroom needed at a higher rate) + +**Tuned**, 30s capture: + +| Metric | Value | vs. baseline | +|---|---|---| +| RAC1 fps | 66.40 | **4.5x** | +| Throughput | 10,424.8 bytes/s | **4.5x** | +| Mean inter-frame interval | 14.9 ms | 4.6x lower | +| Sequence gaps | 78 / ~2036 (~3.8%) | new (0 observed at the slower baseline rate) | +| RSSI | mean -31.1 dBm | (different session/position, not a controlled variable here) | + +The `trig_period` hypothesis is confirmed as the primary lever. The ~3.8% +frame-loss rate at 66 fps is a reasonable place to stop for now — it is +UDP-over-WiFi loss, not a firmware defect, and is small enough not to block +downstream sensing use; revisit only if a use case needs stronger delivery +guarantees (sequence-gap-triggered re-request, TCP fallback, etc.). + +## Links + +- [ADR-266](ADR-266-mediatek-filogic-csi-platform.md) / [ADR-267](ADR-267-mediatek-mimo-csi-wire-protocol.md) — sibling vendor CSI ADR pair (simulator-first precedent) +- [ADR-268](ADR-268-qualcomm-atheros-csi-platform.md) / [ADR-269](ADR-269-qualcomm-csi-wire-protocol.md) — sibling vendor CSI ADR pair +- [ADR-263](ADR-263-rtl8720f-2-4ghz-fmcw-radar-platform.md) / [ADR-264](ADR-264-rtl8720f-radar-wire-protocol.md) — existing Realtek (RTL8720F radar) path; `SourceKind::Realtek` owner +- `CLAUDE.local.md` — RTL8721Dx toolchain setup, board/COM port, first bring-up log diff --git a/scripts/benchmark-rtl8721dx-csi.py b/scripts/benchmark-rtl8721dx-csi.py new file mode 100644 index 000000000..0c975155e --- /dev/null +++ b/scripts/benchmark-rtl8721dx-csi.py @@ -0,0 +1,201 @@ +#!/usr/bin/env python3 +""" +RTL8721Dx (Realtek AmebaDplus) live CSI throughput/latency benchmark. + +Listens on the wifi-densepose-sensing-server's CSI UDP port for real RAC1 +frames (ADR-323, docs/adr/ADR-323-rtl8721dx-ameba-csi-wire-protocol.md) sent +by the ruview_csi_node firmware (branch feat/rtl8721dx-csi-bringup) running +on real RTL8721Dx hardware, and reports MEASURED — not simulated — throughput +and frame-rate statistics. + +This does not itself trigger a board reset; run it while the firmware is +already connected and streaming (see CLAUDE.local.md for the reset+capture +pattern used to bring a fresh boot up during firmware iteration). + +Usage: + python scripts/benchmark-rtl8721dx-csi.py --duration 40 + python scripts/benchmark-rtl8721dx-csi.py --duration 40 --port 5005 --output results.json +""" + +from __future__ import annotations + +import argparse +import json +import socket +import struct +import sys +import time +from dataclasses import dataclass, asdict, field +from pathlib import Path + +RAC1_MAGIC = bytes.fromhex("52414331") # "RAC1" little-endian, see ADR-323 +RHB1_MAGIC = bytes.fromhex("52484231") # heartbeat marker (not a CSI frame) +RAC1_HEADER_LEN = 49 + + +@dataclass +class Rac1Frame: + recv_time: float + length: int + sequence: int + timestamp_us: int + channel: int + bandwidth: int + num_sub_carrier: int + rssi_dbm: int + csi_valid: int + + +def parse_rac1_header(data: bytes, recv_time: float) -> Rac1Frame | None: + if len(data) < RAC1_HEADER_LEN or data[:4] != RAC1_MAGIC: + return None + sequence = struct.unpack_from(" BenchmarkResult: + sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + sock.bind((bind_ip, port)) + sock.settimeout(0.5) + + frames: list[Rac1Frame] = [] + total = 0 + heartbeats = 0 + other = 0 + + deadline = time.time() + duration + print(f"Listening on {bind_ip}:{port} for {duration}s ...", file=sys.stderr) + while time.time() < deadline: + try: + data, _addr = sock.recvfrom(4096) + except socket.timeout: + continue + recv_time = time.time() + total += 1 + if data[:4] == RHB1_MAGIC: + heartbeats += 1 + continue + frame = parse_rac1_header(data, recv_time) + if frame is not None: + frames.append(frame) + else: + other += 1 + sock.close() + + rac1_bytes = sum(f.length for f in frames) + seqs = [f.sequence for f in frames] + rssis = [f.rssi_dbm for f in frames] + channels = sorted({f.channel for f in frames}) + + gaps = 0 + for a, b in zip(seqs, seqs[1:]): + if b != a + 1: + gaps += 1 + + intervals_ms = [ + (b.recv_time - a.recv_time) * 1000.0 for a, b in zip(frames, frames[1:]) + ] + intervals_ms.sort() + + def percentile(values: list[float], p: float) -> float | None: + if not values: + return None + idx = min(len(values) - 1, int(len(values) * p)) + return values[idx] + + return BenchmarkResult( + duration_s=duration, + total_datagrams=total, + rac1_frames=len(frames), + heartbeat_packets=heartbeats, + other_packets=other, + rac1_bytes=rac1_bytes, + rac1_fps=len(frames) / duration if duration > 0 else 0.0, + rac1_bytes_per_sec=rac1_bytes / duration if duration > 0 else 0.0, + sequence_gaps=gaps, + first_sequence=seqs[0] if seqs else None, + last_sequence=seqs[-1] if seqs else None, + rssi_min_dbm=min(rssis) if rssis else None, + rssi_max_dbm=max(rssis) if rssis else None, + rssi_mean_dbm=(sum(rssis) / len(rssis)) if rssis else None, + inter_frame_interval_ms_mean=( + sum(intervals_ms) / len(intervals_ms) if intervals_ms else None + ), + inter_frame_interval_ms_p95=percentile(intervals_ms, 0.95), + channels_seen=channels, + ) + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument("--port", type=int, default=5005, help="UDP port to listen on (default: 5005, matches wifi-densepose-sensing-server)") + parser.add_argument("--bind-ip", type=str, default="0.0.0.0") + parser.add_argument("--duration", type=float, default=40.0, help="capture window in seconds") + parser.add_argument("--output", type=Path, default=None, help="write results as JSON to this path") + args = parser.parse_args() + + result = run_benchmark(args.port, args.duration, args.bind_ip) + + print(f"\n=== RTL8721Dx RAC1 CSI benchmark ({result.evidence}) ===") + print(f"duration: {result.duration_s:.1f}s") + print(f"total UDP datagrams: {result.total_datagrams}") + print(f"RAC1 CSI frames: {result.rac1_frames}") + print(f"heartbeat packets: {result.heartbeat_packets}") + print(f"other/unrecognized: {result.other_packets}") + print(f"RAC1 throughput: {result.rac1_fps:.2f} fps, {result.rac1_bytes_per_sec:.1f} bytes/s") + print(f"sequence gaps: {result.sequence_gaps} (dropped/reordered frames)") + if result.first_sequence is not None: + print(f"sequence range: {result.first_sequence} .. {result.last_sequence}") + if result.rssi_mean_dbm is not None: + print(f"RSSI: min={result.rssi_min_dbm} max={result.rssi_max_dbm} mean={result.rssi_mean_dbm:.1f} dBm") + if result.inter_frame_interval_ms_mean is not None: + print(f"inter-frame interval: mean={result.inter_frame_interval_ms_mean:.1f}ms p95={result.inter_frame_interval_ms_p95:.1f}ms") + print(f"channels seen: {result.channels_seen}") + + if args.output: + args.output.write_text(json.dumps(asdict(result), indent=2)) + print(f"\nwrote {args.output}", file=sys.stderr) + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/v2/crates/wifi-densepose-hardware/benches/realtek_csi_bench.rs b/v2/crates/wifi-densepose-hardware/benches/realtek_csi_bench.rs new file mode 100644 index 000000000..c0990b2e1 --- /dev/null +++ b/v2/crates/wifi-densepose-hardware/benches/realtek_csi_bench.rs @@ -0,0 +1,69 @@ +//! Benchmarks for ADR-323 Realtek RTL8721Dx (AmebaDplus) CSI encode/decode +//! throughput, at the real hardware's own reported dimensions (52 +//! subcarriers, 16-bit tone precision — see `example/wifi/wifi_csi/README.md` +//! in the vendor SDK). + +use criterion::{black_box, criterion_group, criterion_main, Criterion}; +use wifi_densepose_hardware::realtek_csi::simulator::{RealtekCsiSimulator, SimulatorConfig}; +use wifi_densepose_hardware::realtek_csi::CsiFrame; + +fn bench_to_bytes_8bit(c: &mut Criterion) { + let mut sim = RealtekCsiSimulator::new(SimulatorConfig::default()).unwrap(); + let frame = sim.next_frame(); + c.bench_function("realtek_csi_to_bytes_52sc_8bit", |b| { + b.iter(|| black_box(frame.to_bytes().unwrap())); + }); +} + +fn bench_from_bytes_8bit(c: &mut Criterion) { + let mut sim = RealtekCsiSimulator::new(SimulatorConfig::default()).unwrap(); + let wire = sim.next_frame().to_bytes().unwrap(); + c.bench_function("realtek_csi_from_bytes_52sc_8bit", |b| { + b.iter(|| black_box(CsiFrame::from_bytes(black_box(&wire)).unwrap())); + }); +} + +fn bench_to_bytes_16bit(c: &mut Criterion) { + let cfg = SimulatorConfig { + num_bit_per_tone: 32, + ..Default::default() + }; + let mut sim = RealtekCsiSimulator::new(cfg).unwrap(); + let frame = sim.next_frame(); + c.bench_function("realtek_csi_to_bytes_52sc_16bit", |b| { + b.iter(|| black_box(frame.to_bytes().unwrap())); + }); +} + +fn bench_from_bytes_16bit(c: &mut Criterion) { + let cfg = SimulatorConfig { + num_bit_per_tone: 32, + ..Default::default() + }; + let mut sim = RealtekCsiSimulator::new(cfg).unwrap(); + let wire = sim.next_frame().to_bytes().unwrap(); + c.bench_function("realtek_csi_from_bytes_52sc_16bit", |b| { + b.iter(|| black_box(CsiFrame::from_bytes(black_box(&wire)).unwrap())); + }); +} + +fn bench_round_trip_full_datagram(c: &mut Criterion) { + let mut sim = RealtekCsiSimulator::new(SimulatorConfig::default()).unwrap(); + c.bench_function("realtek_csi_round_trip_52sc_8bit", |b| { + b.iter(|| { + let frame = sim.next_frame(); + let wire = black_box(frame.to_bytes().unwrap()); + black_box(CsiFrame::from_bytes(&wire).unwrap()); + }); + }); +} + +criterion_group!( + benches, + bench_to_bytes_8bit, + bench_from_bytes_8bit, + bench_to_bytes_16bit, + bench_from_bytes_16bit, + bench_round_trip_full_datagram, +); +criterion_main!(benches); diff --git a/v2/crates/wifi-densepose-hardware/src/bin/realtek-csi-sim.rs b/v2/crates/wifi-densepose-hardware/src/bin/realtek-csi-sim.rs new file mode 100644 index 000000000..21a9f0cae --- /dev/null +++ b/v2/crates/wifi-densepose-hardware/src/bin/realtek-csi-sim.rs @@ -0,0 +1,107 @@ +//! Deterministic Realtek RTL8721Dx (AmebaDplus) 1x1 CSI simulator (ADR-323). + +use clap::Parser; +use std::{ + fs::File, + io::{self, Write}, + net::{SocketAddr, UdpSocket}, + path::PathBuf, + thread, + time::Duration, +}; +use wifi_densepose_hardware::realtek_csi::{ + simulator::{RealtekCsiSimulator, SimulatorConfig}, + CsiFrame, CsiMode, +}; + +#[derive(Debug, Parser)] +#[command( + name = "realtek-csi-sim", + about = "Emit synthetic ADR-323 Realtek RTL8721Dx CSI frames" +)] +struct Args { + #[arg(long, default_value_t = 100)] + frames: u32, + #[arg(long, default_value = "0x5241433152544c31", value_parser=parse_u64)] + seed: u64, + #[arg(long, default_value_t = 6)] + channel: u8, + #[arg(long)] + bandwidth_40mhz: bool, + #[arg(long, default_value_t = 52)] + subcarriers: u16, + #[arg(long, default_value_t = 16)] + tone_bits: u8, + #[arg(long, default_value_t = 1)] + decimation: u8, + #[arg(long, default_value_t = 64)] + interval_ms: u64, + #[arg(long)] + udp: Option, + /// Replay: little-endian u32 length followed by one ADR-323 envelope. + #[arg(long)] + output: Option, + #[arg(long)] + realtime: bool, +} + +fn parse_u64(v: &str) -> Result { + if let Some(h) = v.strip_prefix("0x").or_else(|| v.strip_prefix("0X")) { + u64::from_str_radix(h, 16).map_err(|e| e.to_string()) + } else { + v.parse() + .map_err(|e: std::num::ParseIntError| e.to_string()) + } +} + +fn emit( + frame: CsiFrame, + socket: Option<&UdpSocket>, + destination: Option, + output: &mut Option, +) -> Result> { + let wire = frame.to_bytes()?; + if let (Some(s), Some(d)) = (socket, destination) { + if s.send_to(&wire, d)? != wire.len() { + return Err(io::Error::new(io::ErrorKind::WriteZero, "partial UDP datagram").into()); + } + } + if let Some(f) = output { + f.write_all(&(wire.len() as u32).to_le_bytes())?; + f.write_all(&wire)?; + } + Ok(wire.len()) +} + +fn main() -> Result<(), Box> { + let a = Args::parse(); + if a.udp.is_none() && a.output.is_none() { + return Err("select at least one sink with --udp or --output".into()); + } + let cfg = SimulatorConfig { + seed: a.seed, + csi_mode: CsiMode::Passive, + channel: a.channel, + bandwidth: if a.bandwidth_40mhz { 1 } else { 0 }, + num_sub_carrier: a.subcarriers, + num_bit_per_tone: a.tone_bits, + decimation: a.decimation, + frame_period_us: (a.interval_ms * 1000) as u32, + ..Default::default() + }; + let mut sim = RealtekCsiSimulator::new(cfg)?; + let socket = a.udp.map(|_| UdpSocket::bind("0.0.0.0:0")).transpose()?; + let mut output = a.output.as_ref().map(File::create).transpose()?; + let mut bytes = 0usize; + for _ in 0..a.frames { + bytes += emit(sim.next_frame(), socket.as_ref(), a.udp, &mut output)?; + if a.realtime { + thread::sleep(Duration::from_millis(a.interval_ms)); + } + } + eprintln!( + "emitted {} synthetic Realtek RTL8721Dx CSI frames ({} bytes, channel={}, subcarriers={}, tone_bits={}, seed={:#x})", + a.frames, bytes, a.channel, a.subcarriers, a.tone_bits, a.seed + ); + Ok(()) +} diff --git a/v2/crates/wifi-densepose-hardware/src/lib.rs b/v2/crates/wifi-densepose-hardware/src/lib.rs index bca235850..08980f2da 100644 --- a/v2/crates/wifi-densepose-hardware/src/lib.rs +++ b/v2/crates/wifi-densepose-hardware/src/lib.rs @@ -62,6 +62,9 @@ pub mod qualcomm_csi; /// ADR-264 host-side framing for Realtek RTL8720F CFR and FMCW radar reports. /// This module has no dependency on the vendor SDK. pub mod rtl8720f; +/// ADR-323 vendor-neutral Realtek RTL8721Dx (AmebaDplus) 1x1 CSI framing and +/// simulator. This module has no dependency on the vendor SDK. +pub mod realtek_csi; pub use bridge::CsiData; pub use csi_frame::{ diff --git a/v2/crates/wifi-densepose-hardware/src/realtek_csi.rs b/v2/crates/wifi-densepose-hardware/src/realtek_csi.rs new file mode 100644 index 000000000..73f2f094d --- /dev/null +++ b/v2/crates/wifi-densepose-hardware/src/realtek_csi.rs @@ -0,0 +1,563 @@ +//! Vendor-neutral Realtek RTL8721Dx (AmebaDplus) 1x1 CSI transport and +//! deterministic simulator. This is not the vendor SDK's raw +//! `rtw_event_csi_report_info` ABI (no CRC, C flexible-array payload); see +//! ADR-323 for the field-by-field mapping and rationale. + +use serde::{Deserialize, Serialize}; +use thiserror::Error; + +pub const RAC1_MAGIC: u32 = 0x3143_4152; // "RAC1" little endian +pub const RAC1_VERSION: u8 = 1; +pub const RAC1_HEADER_LEN: usize = 49; +pub const RAC1_CRC_LEN: usize = 4; +pub const RAC1_MAX_FRAME_LEN: usize = 65_507; +/// Upper bound on subcarriers a 1x1 20/40 MHz OFDM/HT/VHT/HE capture can +/// plausibly report; guards `to_bytes`/`from_bytes` against runaway sizes. +pub const RAC1_MAX_SUBCARRIERS: usize = 4_096; + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[repr(u8)] +pub enum CsiMode { + Passive = 0, + Active = 1, + BoardToBoard = 2, +} + +impl TryFrom for CsiMode { + type Error = CsiParseError; + fn try_from(value: u8) -> Result { + match value { + 0 => Ok(Self::Passive), + 1 => Ok(Self::Active), + 2 => Ok(Self::BoardToBoard), + _ => Err(CsiParseError::UnknownCsiMode(value)), + } + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[repr(u8)] +pub enum ProtocolMode { + Ofdm = 0, + Ht = 1, + Vht = 2, + He = 3, +} + +impl TryFrom for ProtocolMode { + type Error = CsiParseError; + fn try_from(value: u8) -> Result { + match value { + 0 => Ok(Self::Ofdm), + 1 => Ok(Self::Ht), + 2 => Ok(Self::Vht), + 3 => Ok(Self::He), + _ => Err(CsiParseError::UnknownProtocolMode(value)), + } + } +} + +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)] +pub struct CsiFlags(pub u8); + +impl CsiFlags { + pub const SYNTHETIC: u8 = 1 << 0; + pub fn contains(self, flag: u8) -> bool { + self.0 & flag != 0 + } +} + +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct CsiFrame { + /// Disambiguates multi-board illuminator/receiver deployments; 0 = unset. + pub node_id: u8, + pub csi_mode: CsiMode, + pub sequence: u32, + /// Hardware-clock microseconds; wraps roughly every 71 minutes. Consumers + /// must reorder by `sequence`, not raw timestamp arithmetic, across a wrap. + pub timestamp_us: u32, + pub peer_mac: [u8; 6], + pub trig_mac: [u8; 6], + pub channel: u8, + /// 0 = 20 MHz, 1 = 40 MHz. + pub bandwidth: u8, + pub rx_rate: u8, + pub protocol_mode: ProtocolMode, + pub num_sub_carrier: u16, + /// 16 (8-bit I + 8-bit Q) or 32 (16-bit I + 16-bit Q). + pub num_bit_per_tone: u8, + /// Tone group: 1/2/4/8 (every Nth tone reported). + pub decimation: u8, + /// Single RF chain (1x1 hardware). + pub rssi_dbm: i8, + pub rxsc: u8, + pub csi_valid: bool, + pub flags: CsiFlags, + /// One (I, Q) pair per reported subcarrier, widened to i16 regardless of + /// wire precision (an 8-bit wire value fits losslessly). + pub payload: Vec<(i16, i16)>, +} + +impl CsiFrame { + fn component_bytes(&self) -> Result { + match self.num_bit_per_tone { + 16 => Ok(1), + 32 => Ok(2), + other => Err(CsiParseError::InvalidToneBits(other)), + } + } + + pub fn to_bytes(&self) -> Result, CsiParseError> { + self.validate()?; + let comp_bytes = self.component_bytes()?; + let payload_len = self + .payload + .len() + .checked_mul(comp_bytes * 2) + .ok_or(CsiParseError::LengthOverflow)?; + let frame_len = RAC1_HEADER_LEN + .checked_add(payload_len) + .and_then(|n| n.checked_add(RAC1_CRC_LEN)) + .ok_or(CsiParseError::LengthOverflow)?; + if frame_len > RAC1_MAX_FRAME_LEN { + return Err(CsiParseError::FrameTooLarge(frame_len)); + } + let mut out = Vec::with_capacity(frame_len); + out.extend_from_slice(&RAC1_MAGIC.to_le_bytes()); + out.push(RAC1_VERSION); + out.extend_from_slice(&(RAC1_HEADER_LEN as u16).to_le_bytes()); + out.extend_from_slice(&(frame_len as u32).to_le_bytes()); + out.push(self.node_id); + out.push(self.csi_mode as u8); + out.extend_from_slice(&self.sequence.to_le_bytes()); + out.extend_from_slice(&self.timestamp_us.to_le_bytes()); + out.extend_from_slice(&self.peer_mac); + out.extend_from_slice(&self.trig_mac); + out.push(self.channel); + out.push(self.bandwidth); + out.push(self.rx_rate); + out.push(self.protocol_mode as u8); + out.extend_from_slice(&self.num_sub_carrier.to_le_bytes()); + out.push(self.num_bit_per_tone); + out.push(self.decimation); + out.push(self.rssi_dbm as u8); + out.push(self.rxsc); + out.push(self.csi_valid as u8); + out.push(self.flags.0); + out.extend_from_slice(&(payload_len as u32).to_le_bytes()); + debug_assert_eq!(out.len(), RAC1_HEADER_LEN); + for (i, q) in &self.payload { + if comp_bytes == 1 { + out.push(*i as i8 as u8); + out.push(*q as i8 as u8); + } else { + out.extend_from_slice(&i.to_le_bytes()); + out.extend_from_slice(&q.to_le_bytes()); + } + } + out.extend_from_slice(&crc32_ieee(&out).to_le_bytes()); + Ok(out) + } + + pub fn from_bytes(input: &[u8]) -> Result<(Self, usize), CsiParseError> { + if input.len() < RAC1_HEADER_LEN { + return Err(CsiParseError::InsufficientData { + needed: RAC1_HEADER_LEN, + got: input.len(), + }); + } + let magic = u32_at(input, 0); + if magic != RAC1_MAGIC { + return Err(CsiParseError::InvalidMagic(magic)); + } + if input[4] != RAC1_VERSION { + return Err(CsiParseError::UnsupportedVersion(input[4])); + } + let header_len = u16_at(input, 5) as usize; + if header_len != RAC1_HEADER_LEN { + return Err(CsiParseError::InvalidHeaderLength(header_len)); + } + let frame_len = u32_at(input, 7) as usize; + if frame_len > RAC1_MAX_FRAME_LEN { + return Err(CsiParseError::FrameTooLarge(frame_len)); + } + if frame_len < header_len + RAC1_CRC_LEN { + return Err(CsiParseError::InvalidFrameLength(frame_len)); + } + if input.len() < frame_len { + return Err(CsiParseError::InsufficientData { + needed: frame_len, + got: input.len(), + }); + } + let expected_crc = u32_at(input, frame_len - 4); + let actual_crc = crc32_ieee(&input[..frame_len - 4]); + if expected_crc != actual_crc { + return Err(CsiParseError::CrcMismatch { + expected: expected_crc, + actual: actual_crc, + }); + } + + let node_id = input[11]; + let csi_mode = CsiMode::try_from(input[12])?; + let sequence = u32_at(input, 13); + let timestamp_us = u32_at(input, 17); + let mut peer_mac = [0u8; 6]; + peer_mac.copy_from_slice(&input[21..27]); + let mut trig_mac = [0u8; 6]; + trig_mac.copy_from_slice(&input[27..33]); + let channel = input[33]; + let bandwidth = input[34]; + let rx_rate = input[35]; + let protocol_mode = ProtocolMode::try_from(input[36])?; + let num_sub_carrier = u16_at(input, 37); + let num_bit_per_tone = input[39]; + let decimation = input[40]; + let rssi_dbm = input[41] as i8; + let rxsc = input[42]; + let csi_valid = input[43] != 0; + let flags = CsiFlags(input[44]); + let payload_len = u32_at(input, 45) as usize; + + if header_len + payload_len + RAC1_CRC_LEN != frame_len { + return Err(CsiParseError::PayloadLengthMismatch); + } + let comp_bytes = match num_bit_per_tone { + 16 => 1usize, + 32 => 2usize, + other => return Err(CsiParseError::InvalidToneBits(other)), + }; + let expected_payload_len = (num_sub_carrier as usize) + .checked_mul(comp_bytes * 2) + .ok_or(CsiParseError::LengthOverflow)?; + if payload_len != expected_payload_len { + return Err(CsiParseError::PayloadLengthMismatch); + } + let payload_bytes = &input[header_len..header_len + payload_len]; + let payload = if comp_bytes == 1 { + payload_bytes + .chunks_exact(2) + .map(|b| (b[0] as i8 as i16, b[1] as i8 as i16)) + .collect() + } else { + payload_bytes + .chunks_exact(4) + .map(|b| { + ( + i16::from_le_bytes([b[0], b[1]]), + i16::from_le_bytes([b[2], b[3]]), + ) + }) + .collect() + }; + + let frame = Self { + node_id, + csi_mode, + sequence, + timestamp_us, + peer_mac, + trig_mac, + channel, + bandwidth, + rx_rate, + protocol_mode, + num_sub_carrier, + num_bit_per_tone, + decimation, + rssi_dbm, + rxsc, + csi_valid, + flags, + payload, + }; + frame.validate()?; + Ok((frame, frame_len)) + } + + fn validate(&self) -> Result<(), CsiParseError> { + if !matches!(self.bandwidth, 0 | 1) { + return Err(CsiParseError::InvalidBandwidth(self.bandwidth)); + } + let comp_bytes = self.component_bytes()?; + if self.num_sub_carrier == 0 || self.num_sub_carrier as usize > RAC1_MAX_SUBCARRIERS { + return Err(CsiParseError::InvalidSubcarrierCount(self.num_sub_carrier)); + } + if self.payload.len() != self.num_sub_carrier as usize { + return Err(CsiParseError::PayloadLengthMismatch); + } + if comp_bytes == 1 { + let in_range = |v: i16| (i8::MIN as i16..=i8::MAX as i16).contains(&v); + if self.payload.iter().any(|(i, q)| !in_range(*i) || !in_range(*q)) { + return Err(CsiParseError::ToneValueOutOfRange); + } + } + Ok(()) + } +} + +#[derive(Debug, Error, PartialEq)] +pub enum CsiParseError { + #[error("insufficient data: needed {needed}, got {got}")] + InsufficientData { needed: usize, got: usize }, + #[error("invalid magic {0:#010x}")] + InvalidMagic(u32), + #[error("unsupported version {0}")] + UnsupportedVersion(u8), + #[error("unknown CSI mode {0}")] + UnknownCsiMode(u8), + #[error("unknown protocol mode {0}")] + UnknownProtocolMode(u8), + #[error("invalid header length {0}")] + InvalidHeaderLength(usize), + #[error("invalid frame length {0}")] + InvalidFrameLength(usize), + #[error("frame too large: {0}")] + FrameTooLarge(usize), + #[error("length arithmetic overflow")] + LengthOverflow, + #[error("payload length mismatch")] + PayloadLengthMismatch, + #[error("invalid num_bit_per_tone {0} (must be 16 or 32)")] + InvalidToneBits(u8), + #[error("invalid subcarrier count {0}")] + InvalidSubcarrierCount(u16), + #[error("invalid bandwidth code {0} (must be 0 or 1)")] + InvalidBandwidth(u8), + #[error("tone I/Q value out of range for 8-bit precision")] + ToneValueOutOfRange, + #[error("CRC mismatch: expected {expected:#010x}, actual {actual:#010x}")] + CrcMismatch { expected: u32, actual: u32 }, +} + +pub mod simulator { + use super::*; + + #[derive(Debug, Clone)] + pub struct SimulatorConfig { + pub seed: u64, + pub node_id: u8, + pub csi_mode: CsiMode, + pub peer_mac: [u8; 6], + pub trig_mac: [u8; 6], + pub channel: u8, + pub bandwidth: u8, + pub rx_rate: u8, + pub protocol_mode: ProtocolMode, + pub num_sub_carrier: u16, + pub num_bit_per_tone: u8, + pub decimation: u8, + pub rssi_dbm: i8, + pub frame_period_us: u32, + } + + impl Default for SimulatorConfig { + fn default() -> Self { + Self { + seed: 0x5241_4331_5254_4c31, // "RAC1RTL1"-ish + node_id: 1, + csi_mode: CsiMode::Passive, + peer_mac: [0xa4, 0x39, 0xb3, 0xa4, 0xbe, 0x2d], + trig_mac: [0x00, 0xe0, 0x4c, 0x00, 0x0a, 0xa3], + channel: 6, + bandwidth: 0, + rx_rate: 12, // RTW_RATE_6M + protocol_mode: ProtocolMode::Ofdm, + num_sub_carrier: 52, + num_bit_per_tone: 16, + decimation: 1, + rssi_dbm: -45, + frame_period_us: 64_000, + } + } + } + + pub struct RealtekCsiSimulator { + config: SimulatorConfig, + rng: u64, + sequence: u32, + timestamp_us: u32, + motion_phase: f32, + } + + impl RealtekCsiSimulator { + pub fn new(config: SimulatorConfig) -> Result { + let s = Self { + rng: config.seed, + config, + sequence: 0, + timestamp_us: 0, + motion_phase: 0.0, + }; + s.csi_frame().validate()?; + Ok(s) + } + + pub fn next_frame(&mut self) -> CsiFrame { + let frame = self.csi_frame(); + self.sequence = self.sequence.wrapping_add(1); + self.timestamp_us = self.timestamp_us.wrapping_add(self.config.frame_period_us); + self.motion_phase += 0.041; + frame + } + + fn csi_frame(&self) -> CsiFrame { + let mut rng = self.rng ^ self.sequence as u64; + let comp_max = if self.config.num_bit_per_tone == 16 { + 110.0 + } else { + 1800.0 + }; + let payload = (0..self.config.num_sub_carrier as usize) + .map(|tone| { + rng ^= rng << 13; + rng ^= rng >> 7; + rng ^= rng << 17; + let noise = ((rng >> 48) as i16 % 8) as f32; + let phase = tone as f32 * 0.11 + self.motion_phase; + ( + ((phase.cos() * comp_max) + noise) as i16, + ((phase.sin() * comp_max) - noise) as i16, + ) + }) + .collect(); + CsiFrame { + node_id: self.config.node_id, + csi_mode: self.config.csi_mode, + sequence: self.sequence, + timestamp_us: self.timestamp_us, + peer_mac: self.config.peer_mac, + trig_mac: self.config.trig_mac, + channel: self.config.channel, + bandwidth: self.config.bandwidth, + rx_rate: self.config.rx_rate, + protocol_mode: self.config.protocol_mode, + num_sub_carrier: self.config.num_sub_carrier, + num_bit_per_tone: self.config.num_bit_per_tone, + decimation: self.config.decimation, + rssi_dbm: self.config.rssi_dbm, + rxsc: 0, + csi_valid: true, + flags: CsiFlags(CsiFlags::SYNTHETIC), + payload, + } + } + } +} + +fn u16_at(b: &[u8], o: usize) -> u16 { + u16::from_le_bytes([b[o], b[o + 1]]) +} +fn u32_at(b: &[u8], o: usize) -> u32 { + u32::from_le_bytes(b[o..o + 4].try_into().unwrap()) +} +fn crc32_ieee(data: &[u8]) -> u32 { + let mut crc = 0xffff_ffffu32; + for &byte in data { + crc ^= byte as u32; + for _ in 0..8 { + crc = (crc >> 1) ^ ((0u32.wrapping_sub(crc & 1)) & 0xedb8_8320); + } + } + !crc +} + +#[cfg(test)] +mod tests { + use super::*; + use simulator::*; + + #[test] + fn simulator_round_trip_is_deterministic() { + let cfg = SimulatorConfig::default(); + let mut a = RealtekCsiSimulator::new(cfg.clone()).unwrap(); + let mut b = RealtekCsiSimulator::new(cfg).unwrap(); + let wa = a.next_frame().to_bytes().unwrap(); + assert_eq!(wa, b.next_frame().to_bytes().unwrap()); + let (decoded, n) = CsiFrame::from_bytes(&wa).unwrap(); + assert_eq!(n, wa.len()); + assert!(decoded.flags.contains(CsiFlags::SYNTHETIC)); + assert_eq!(decoded.payload.len(), 52); + } + + #[test] + fn sixteen_bit_precision_round_trip() { + let cfg = SimulatorConfig { + num_bit_per_tone: 32, + ..Default::default() + }; + let mut sim = RealtekCsiSimulator::new(cfg).unwrap(); + let frame = sim.next_frame(); + let wire = frame.to_bytes().unwrap(); + let (decoded, n) = CsiFrame::from_bytes(&wire).unwrap(); + assert_eq!(n, wire.len()); + assert_eq!(decoded, frame); + } + + #[test] + fn crc_corruption_is_rejected() { + let mut s = RealtekCsiSimulator::new(SimulatorConfig::default()).unwrap(); + let mut w = s.next_frame().to_bytes().unwrap(); + let last = w.len() - 1; + w[last] ^= 1; + assert!(matches!( + CsiFrame::from_bytes(&w), + Err(CsiParseError::CrcMismatch { .. }) + )); + } + + #[test] + fn truncation_is_rejected() { + let mut s = RealtekCsiSimulator::new(SimulatorConfig::default()).unwrap(); + let w = s.next_frame().to_bytes().unwrap(); + assert!(matches!( + CsiFrame::from_bytes(&w[..w.len() - 1]), + Err(CsiParseError::InsufficientData { .. }) + )); + } + + #[test] + fn zero_subcarriers_is_rejected() { + let cfg = SimulatorConfig { + num_sub_carrier: 0, + ..Default::default() + }; + assert!(matches!( + RealtekCsiSimulator::new(cfg), + Err(CsiParseError::InvalidSubcarrierCount(0)) + )); + } + + #[test] + fn invalid_tone_bits_is_rejected() { + let cfg = SimulatorConfig { + num_bit_per_tone: 24, + ..Default::default() + }; + assert!(matches!( + RealtekCsiSimulator::new(cfg), + Err(CsiParseError::InvalidToneBits(24)) + )); + } + + #[test] + fn parser_never_panics_on_prefixes() { + let mut s = RealtekCsiSimulator::new(SimulatorConfig::default()).unwrap(); + let w = s.next_frame().to_bytes().unwrap(); + for end in 0..w.len() { + let _ = CsiFrame::from_bytes(&w[..end]); + } + } + + #[test] + fn magic_mismatch_is_rejected() { + let mut s = RealtekCsiSimulator::new(SimulatorConfig::default()).unwrap(); + let mut w = s.next_frame().to_bytes().unwrap(); + w[0] ^= 0xff; + assert!(matches!( + CsiFrame::from_bytes(&w), + Err(CsiParseError::InvalidMagic(_)) + )); + } +} diff --git a/v2/crates/wifi-densepose-sensing-server/src/main.rs b/v2/crates/wifi-densepose-sensing-server/src/main.rs index f8216bea5..89cbb2f24 100644 --- a/v2/crates/wifi-densepose-sensing-server/src/main.rs +++ b/v2/crates/wifi-densepose-sensing-server/src/main.rs @@ -19,6 +19,7 @@ mod model_format; mod multistatic_bridge; mod mediatek_csi; mod qualcomm_csi; +mod realtek_csi; mod realtek_radar; mod path_safety; pub mod pose; @@ -1768,6 +1769,10 @@ struct AppStateInner { latest_qualcomm_csi: Option, /// Instant of the last validated Qualcomm CSI UDP frame. last_qualcomm_frame: Option, + /// Latest validated RTL8721Dx CSI summary; distinct from RTL8720F radar. + latest_realtek_csi: Option, + /// Instant of the last validated RTL8721Dx CSI UDP frame. + last_realtek_csi_frame: Option, /// Latest bounded ADR-270 event per vendor. Complex CSI uses dedicated transports. latest_vendor_rf: BTreeMap, tx: broadcast::Sender, @@ -2312,7 +2317,13 @@ impl AppStateInner { } } } - if self.source.starts_with("realtek") { + if self.source.starts_with("realtek_csi") { + if let Some(last) = self.last_realtek_csi_frame { + if last.elapsed() > ESP32_OFFLINE_TIMEOUT { + return format!("{}:offline", self.source); + } + } + } else if self.source.starts_with("realtek") { if let Some(last) = self.last_realtek_frame { if last.elapsed() > ESP32_OFFLINE_TIMEOUT { return format!("{}:offline", self.source); @@ -2382,6 +2393,8 @@ impl AppStateInner { last_mediatek_frame: None, latest_qualcomm_csi: None, last_qualcomm_frame: None, + latest_realtek_csi: None, + last_realtek_csi_frame: None, latest_vendor_rf: BTreeMap::new(), tx: broadcast::channel::(16).0, intro: wifi_densepose_sensing_server::introspection::IntrospectionState::new(), @@ -4869,6 +4882,53 @@ async fn latest_qualcomm_csi(State(state): State) -> Json) -> Json { + let s = state.read().await; + match &s.latest_realtek_csi { + Some(snapshot) => Json(serde_json::to_value(snapshot).unwrap_or_default()), + None => Json(serde_json::json!({"status": "no Realtek RTL8721Dx CSI data yet"})), + } +} + +fn primary_source_with_realtek( + last_esp32_frame: Option, + realtek_source: &str, +) -> String { + if last_esp32_frame.is_some_and(|seen| seen.elapsed() < ESP32_OFFLINE_TIMEOUT) { + "esp32".to_string() + } else { + realtek_source.to_string() + } +} + +#[cfg(test)] +mod realtek_ingest_tests { + use super::*; + use wifi_densepose_hardware::realtek_csi::simulator::{RealtekCsiSimulator, SimulatorConfig}; + + #[test] + fn secondary_realtek_does_not_displace_fresh_esp32_room_source() { + assert_eq!( + primary_source_with_realtek(Some(std::time::Instant::now()), "realtek_csi"), + "esp32" + ); + assert_eq!(primary_source_with_realtek(None, "realtek_csi"), "realtek_csi"); + } + + #[tokio::test] + async fn latest_route_keeps_synthetic_provenance_and_node_identity() { + let state: SharedState = Arc::new(RwLock::new(AppStateInner::minimal())); + let mut simulator = RealtekCsiSimulator::new(SimulatorConfig::default()).unwrap(); + let snapshot = realtek_csi::RealtekCsiSnapshot::from_frame(&simulator.next_frame()); + let expected_node_id = snapshot.node_id; + state.write().await.latest_realtek_csi = Some(snapshot); + let Json(value) = latest_realtek_csi(State(state)).await; + assert_eq!(value["node_id"], expected_node_id); + assert_eq!(value["source"], "realtek_csi:simulated"); + assert_eq!(value["synthetic"], true); + } +} + async fn vendor_descriptors() -> Json { Json( serde_json::to_value(wifi_densepose_sensing_server::vendor_rf::descriptors()) @@ -8826,7 +8886,7 @@ async fn udp_receiver_task( let addr = format!("{bind_ip}:{udp_port}"); let socket = match UdpSocket::bind(&addr).await { Ok(s) => { - info!("UDP listening on {addr} for ESP32, MediaTek, Qualcomm CSI, and RTL8720F radar frames"); + info!("UDP listening on {addr} for ESP32, MediaTek, Qualcomm, RTL8721Dx CSI, and RTL8720F radar frames"); s } Err(e) => { @@ -8928,6 +8988,33 @@ async fn udp_receiver_task( } continue; } + if len >= 4 + && u32::from_le_bytes(buf[..4].try_into().expect("four-byte slice")) + == wifi_densepose_hardware::realtek_csi::RAC1_MAGIC + { + match wifi_densepose_hardware::realtek_csi::CsiFrame::from_bytes(&buf[..len]) { + Ok((frame, consumed)) if consumed == len => { + let snapshot = realtek_csi::RealtekCsiSnapshot::from_frame(&frame); + debug!("RTL8721Dx CSI from {src}: node={} seq={} subcarriers={}", snapshot.node_id, snapshot.sequence, snapshot.num_sub_carrier); + let json = serde_json::to_string(&snapshot).ok(); + let mut s = state.write().await; + // A secondary Realtek link must not replace a fresh ESP32 + // room source and make its vitals disappear from the UI. + s.source = primary_source_with_realtek( + s.last_esp32_frame, + snapshot.source, + ); + s.last_realtek_csi_frame = Some(std::time::Instant::now()); + s.latest_realtek_csi = Some(snapshot); + if let Some(json) = json { + let _ = s.tx.send(json); + } + } + Ok((_, consumed)) => warn!("RTL8721Dx CSI datagram from {src} has trailing bytes: consumed={consumed} received={len}"), + Err(error) => warn!("Rejected RTL8721Dx CSI datagram from {src}: {error}"), + } + continue; + } // ADR-039: Try edge vitals packet first (magic 0xC511_0002). if let Some(vitals) = parse_esp32_vitals(&buf[..len]) { debug!( @@ -11271,6 +11358,8 @@ async fn main() { last_mediatek_frame: None, latest_qualcomm_csi: None, last_qualcomm_frame: None, + latest_realtek_csi: None, + last_realtek_csi_frame: None, latest_vendor_rf: BTreeMap::new(), tx, intro: wifi_densepose_sensing_server::introspection::IntrospectionState::new(), @@ -11616,6 +11705,7 @@ async fn main() { .route("/api/v1/radar/latest", get(latest_realtek_radar)) .route("/api/v1/csi/mediatek/latest", get(latest_mediatek_csi)) .route("/api/v1/csi/qualcomm/latest", get(latest_qualcomm_csi)) + .route("/api/v1/csi/realtek/latest", get(latest_realtek_csi)) .route("/api/v1/rf/vendors", get(vendor_descriptors)) .route("/api/v1/rf/vendors/latest", get(latest_vendor_events)) .route("/api/v1/rf/vendors/:vendor/latest", get(latest_vendor_event)) diff --git a/v2/crates/wifi-densepose-sensing-server/src/realtek_csi.rs b/v2/crates/wifi-densepose-sensing-server/src/realtek_csi.rs new file mode 100644 index 000000000..189570de3 --- /dev/null +++ b/v2/crates/wifi-densepose-sensing-server/src/realtek_csi.rs @@ -0,0 +1,124 @@ +//! Bounded summaries for ADR-323 Realtek RTL8721Dx (AmebaDplus) 1x1 CSI frames. + +use serde::Serialize; +use wifi_densepose_hardware::realtek_csi::{CsiFlags, CsiFrame}; + +#[derive(Debug, Clone, PartialEq, Serialize)] +pub(crate) struct RealtekCsiSnapshot { + pub event_type: &'static str, + pub source: &'static str, + pub csi_mode: &'static str, + pub node_id: u8, + pub sequence: u32, + pub timestamp_us: u32, + pub peer_mac: String, + pub trig_mac: String, + pub channel: u8, + pub bandwidth_mhz: u16, + pub rx_rate: u8, + pub protocol_mode: String, + pub num_sub_carrier: u16, + pub num_bit_per_tone: u8, + pub decimation: u8, + pub rssi_dbm: i8, + pub rxsc: u8, + pub csi_valid: bool, + pub synthetic: bool, + pub mean_amplitude: Option, + pub peak_amplitude: Option, +} + +impl RealtekCsiSnapshot { + pub(crate) fn from_frame(frame: &CsiFrame) -> Self { + let synthetic = frame.flags.contains(CsiFlags::SYNTHETIC); + let (mean_amplitude, peak_amplitude) = amplitude_summary(frame); + Self { + event_type: "realtek_csi", + source: if synthetic { + "realtek_csi:simulated" + } else { + "realtek_csi" + }, + csi_mode: match frame.csi_mode { + wifi_densepose_hardware::realtek_csi::CsiMode::Passive => "passive", + wifi_densepose_hardware::realtek_csi::CsiMode::Active => "active", + wifi_densepose_hardware::realtek_csi::CsiMode::BoardToBoard => "board_to_board", + }, + node_id: frame.node_id, + sequence: frame.sequence, + timestamp_us: frame.timestamp_us, + peer_mac: mac_string(&frame.peer_mac), + trig_mac: mac_string(&frame.trig_mac), + channel: frame.channel, + bandwidth_mhz: if frame.bandwidth == 1 { 40 } else { 20 }, + rx_rate: frame.rx_rate, + protocol_mode: format!("{:?}", frame.protocol_mode).to_ascii_lowercase(), + num_sub_carrier: frame.num_sub_carrier, + num_bit_per_tone: frame.num_bit_per_tone, + decimation: frame.decimation, + rssi_dbm: frame.rssi_dbm, + rxsc: frame.rxsc, + csi_valid: frame.csi_valid, + synthetic, + mean_amplitude, + peak_amplitude, + } + } +} + +fn mac_string(mac: &[u8; 6]) -> String { + mac.iter() + .map(|b| format!("{b:02x}")) + .collect::>() + .join(":") +} + +fn amplitude_summary(frame: &CsiFrame) -> (Option, Option) { + if frame.payload.is_empty() { + return (None, None); + } + let amplitudes: Vec = frame + .payload + .iter() + .map(|(i, q)| (*i as f32).hypot(*q as f32)) + .collect(); + let mean = amplitudes.iter().sum::() / amplitudes.len() as f32; + let peak = amplitudes.into_iter().max_by(f32::total_cmp); + (Some(mean), peak) +} + +#[cfg(test)] +mod tests { + use super::*; + use wifi_densepose_hardware::realtek_csi::simulator::{RealtekCsiSimulator, SimulatorConfig}; + + #[test] + fn simulator_summary_preserves_dimensions_and_provenance() { + let mut sim = RealtekCsiSimulator::new(SimulatorConfig::default()).unwrap(); + let snapshot = RealtekCsiSnapshot::from_frame(&sim.next_frame()); + assert_eq!(snapshot.source, "realtek_csi:simulated"); + assert_eq!(snapshot.num_sub_carrier, 52); + assert_eq!(snapshot.csi_mode, "passive"); + assert!(snapshot.mean_amplitude.unwrap() > 0.0); + assert!(snapshot.peak_amplitude.unwrap() >= snapshot.mean_amplitude.unwrap()); + } + + #[test] + fn source_string_is_distinct_from_realtek_radar() { + let mut sim = RealtekCsiSimulator::new(SimulatorConfig::default()).unwrap(); + let snapshot = RealtekCsiSnapshot::from_frame(&sim.next_frame()); + // Must never collapse to the bare "realtek"/"realtek:simulated" source + // strings already owned by the RTL8720F radar path (realtek_radar.rs) — + // effective_source()'s prefix matching in main.rs depends on this. + assert_ne!(snapshot.source, "realtek"); + assert_ne!(snapshot.source, "realtek:simulated"); + assert!(snapshot.source.starts_with("realtek_csi")); + } + + #[test] + fn mac_addresses_are_formatted_colon_separated() { + let mut sim = RealtekCsiSimulator::new(SimulatorConfig::default()).unwrap(); + let snapshot = RealtekCsiSnapshot::from_frame(&sim.next_frame()); + assert_eq!(snapshot.peer_mac, "a4:39:b3:a4:be:2d"); + } +} From 815384dcc4be9e31db8b9b9e1084dd0fd5420986 Mon Sep 17 00:00:00 2001 From: ruv Date: Tue, 15 Sep 2026 14:02:38 -0400 Subject: [PATCH 2/4] Update rustls lockfile for RUSTSEC-2026-0285 --- v2/Cargo.lock | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/v2/Cargo.lock b/v2/Cargo.lock index 72660aa40..5e3102f90 100644 --- a/v2/Cargo.lock +++ b/v2/Cargo.lock @@ -9109,9 +9109,9 @@ dependencies = [ [[package]] name = "rustls" -version = "0.23.37" +version = "0.23.45" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "758025cb5fccfd3bc2fd74708fd4682be41d99e5dff73c377c0646c6012c73a4" +checksum = "0d41d731c7d2f962d1ccc364cec258de3c0e93b38c2fb3ba97ac74513048d634" dependencies = [ "log", "once_cell", @@ -9173,9 +9173,9 @@ checksum = "f87165f0995f63a9fbeea62b64d10b4d9d8e78ec6d7d51fb2125fda7bb36788f" [[package]] name = "rustls-webpki" -version = "0.103.13" +version = "0.103.15" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "61c429a8649f110dddef65e2a5ad240f747e85f7758a6bccc7e5777bd33f756e" +checksum = "f3c3cf1d8b1e7d4927e2d154c3fcb02979afb9939629c62cd9048d4f07b60ac2" dependencies = [ "ring", "rustls-pki-types", From a04fbc4694b7b5987d6ede426f224a648d9c00a2 Mon Sep 17 00:00:00 2001 From: ruv Date: Tue, 15 Sep 2026 14:06:31 -0400 Subject: [PATCH 3/4] Resolve ADR 348 collision in upstream forecast CI --- ...nboarding.md => ADR-363-native-macos-usb-node-onboarding.md} | 2 +- docs/adr/README.md | 2 +- firmware/esp32-csi-node/README.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) rename docs/adr/{ADR-348-native-macos-usb-node-onboarding.md => ADR-363-native-macos-usb-node-onboarding.md} (99%) diff --git a/docs/adr/ADR-348-native-macos-usb-node-onboarding.md b/docs/adr/ADR-363-native-macos-usb-node-onboarding.md similarity index 99% rename from docs/adr/ADR-348-native-macos-usb-node-onboarding.md rename to docs/adr/ADR-363-native-macos-usb-node-onboarding.md index f1bb9652c..50ff63007 100644 --- a/docs/adr/ADR-348-native-macos-usb-node-onboarding.md +++ b/docs/adr/ADR-363-native-macos-usb-node-onboarding.md @@ -1,4 +1,4 @@ -# ADR 348: Native macOS USB node onboarding +# ADR 363: Native macOS USB node onboarding ## Status diff --git a/docs/adr/README.md b/docs/adr/README.md index f3eeef3a3..c6e382052 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -115,7 +115,7 @@ Statuses: **Proposed** (under discussion), **Accepted** (approved and/or impleme | [ADR-344](ADR-344-adaptive-local-installation-discovery.md) | Adaptive Local Installation Discovery | Accepted (local software path) | | [ADR-346](ADR-346-fail-closed-edge-occupancy-evidence.md) | Fail closed ESP32 occupancy evidence | Accepted (C6 occupancy integrity qualified) | | [ADR-347](ADR-347-rate-aware-esp32-temporal-sensing.md) | Rate aware ESP32 temporal sensing | Accepted (C6 timing and transport qualified) | -| [ADR-348](ADR-348-native-macos-usb-node-onboarding.md) | Native macOS USB node onboarding with nonce bound discovery and fresh server verification | Accepted and physically qualified on one ESP32 S3; C6 and signed distribution pending | +| [ADR-363](ADR-363-native-macos-usb-node-onboarding.md) | Native macOS USB node onboarding with nonce bound discovery and fresh server verification | Accepted and physically qualified on one ESP32 S3; C6 and signed distribution pending | | [ADR-115](ADR-115-home-assistant-integration.md) | Home Assistant integration via MQTT auto-discovery + Matter bridge (HA-DISCO + HA-FABRIC + HA-MIND) | Accepted (MQTT track) / Proposed (Matter SDK P8b) | | [ADR-169](ADR-169-adam-mode-light-theme.md) | adam-mode — light theme toggle for the three.js realtime demo | Proposed | | [ADR-170](ADR-170-yoga-mode-pose-system.md) | yoga-mode — yoga pose detection, classification, and scoring for the three.js realtime demo | Proposed | diff --git a/firmware/esp32-csi-node/README.md b/firmware/esp32-csi-node/README.md index 1e5e70813..5f7505aae 100644 --- a/firmware/esp32-csi-node/README.md +++ b/firmware/esp32-csi-node/README.md @@ -32,7 +32,7 @@ matching chip build through the documented backup and flash process once. The USB protocol rejects public target addresses and invalid identifiers. New WiFi credentials are bounded, committed only in NVS, and never included in the receipt or diagnostic output. See -[ADR 348](../../docs/adr/ADR-348-native-macos-usb-node-onboarding.md). +[ADR 363](../../docs/adr/ADR-363-native-macos-usb-node-onboarding.md). On ESP32 C6, the default optional mmWave UART uses GPIO 4 and GPIO 5. Firmware refuses any mmWave configuration that overlaps the active console pins. This From af2d6981a31ae9f9143337a221ce2932d86bc34d Mon Sep 17 00:00:00 2001 From: ruv Date: Tue, 15 Sep 2026 14:16:05 -0400 Subject: [PATCH 4/4] ci: bound S3 8 MB image to 1200 KiB --- .github/workflows/firmware-ci.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/firmware-ci.yml b/.github/workflows/firmware-ci.yml index defda2b01..0a8e8fe38 100644 --- a/.github/workflows/firmware-ci.yml +++ b/.github/workflows/firmware-ci.yml @@ -53,7 +53,9 @@ jobs: sdkconfig: sdkconfig.defaults partition_table_name: partitions_display.csv size_warn_kb: 1100 - size_limit_kb: 1152 + # Current 8 MB image is 1163 KiB after USB onboarding. The OTA app + # partition is 2048 KiB; retain a 1200 KiB hard gate and warning. + size_limit_kb: 1200 artifact_app: esp32-csi-node.bin artifact_pt: partition-table.bin - variant: 4mb