* feat(forecast): add independent Rust training stack
Signed-off-by: Codex <codex@openai.com>
* docs(forecast): record informal HPO exploration note (unaccepted)
Records an exploratory Darwin Mode numeric-genome hyperparameter search
against the tiny_ci synthetic accuracy protocol: WQL improved 0.257 -> 0.161
-> 0.153 over two search rounds, staying ahead of both baselines throughout.
Explicitly scoped as informal/exploratory (not a frozen leakage-free report,
not eligible for the evidence ledger, not validated for large_linux or any
real dataset) per this doc's own evidence-tagging discipline.
* feat(forecast-core): add per-horizon weighted quantile loss
weighted_quantile_loss collapses the whole horizon into one aggregate
number, hiding whether error grows with lead time. Add a sibling
function using the identical per-cell pinball formula and domain
checks, just reduced per horizon step instead of globally. A unit
test reconstructs the aggregate from the per-step numerators/
denominators to prove the two functions agree exactly, not just
approximately.
* feat(forecast-model): add evaluation-only local activation
Adds activate_for_evaluation and build_eval_input, gated behind the
cpu feature. activate_for_evaluation self-signs an unsigned candidate
with a fixed, publicly-known, non-secret Ed25519 key so a CLI operator
can run inference against their own just-trained candidate without a
real release signature -- explicitly never a production trust path,
documented as such in the module doc comment. The schema-digest check
in ArtifactActivationPolicy still applies in full; only signing is
relaxed. build_eval_input constructs a single-window ModelInput for
CPU inference from raw context-major values/mask arrays, matching the
training batch builders exact time-feature/descriptor encoding.
Six unit tests cover: malformed-candidate rejection, digest-mismatch
rejection (proving the schema check has teeth even on this relaxed
path), getting past the policy gate with a matching digest, and
eval-input shape validation.
* feat(forecast-train): add evaluate and prepare-synthetic-dataset CLI
Turns this sessions throwaway HPO scripts into real, tested CLI
surfaces, replacing two example binaries that bypassed the CLI/TOML
training path entirely.
prepare-synthetic-dataset generates a larger, configurable synthetic
training shard (default 24 windows) plus a matching train-local.toml
with the given OptimizerSpec hyperparameters baked in, and a separate
held-out test.jsonl -- the same synthetic-only, local-validation-only
posture as prepare-local-example, just bigger and configurable.
evaluate scores a trained (unsigned) candidate against held-out
windows and the LastValue/SeasonalNaive baselines: overall weighted
quantile loss, a per-horizon breakdown, 80% interval coverage, and
missingness, for the model and both baselines. Explicitly lists what
docs/benchmarks/ruforecast.mds full accuracy protocol additionally
asks for that this does not cover (abstention coverage, selective
risk, site/device slices, interference regime, RuVector-retrieval
ablation) rather than silently omitting them -- each needs
infrastructure a single-entity synthetic fixture does not have.
Together the three real commands (prepare-synthetic-dataset,
train-local, evaluate) reproduce the exact WQL numbers the old
library-bypassing example scripts produced for the same
hyperparameters, confirmed by re-running the harness/ruview HPO
dry-run end to end against the new CLI path.
New integration test trains a real candidate via smoke, evaluates it
against held-out synthetic windows, and asserts the full report shape
and an empty-input rejection.
* feat(forecast-train): add prepare-synthetic-dataset --seed + Autogenous bridge
--seed (default 0, backward compatible) makes prepare-synthetic-dataset's
generator deterministic per offset: same seed -> byte-identical corpus
across runs (needed to train two genomes on the SAME corpus for a fair
comparison); different seeds -> genuinely independent corpora (needed for
honest multi-seed/multi-judge evaluation upstream, both in
harness/ruview/flywheel/ruforecast/gate.mjs's fitness function and in the
new v2/crates/ruforecast-autogenous-bridge crate added here).
ruforecast-autogenous-bridge is a LOCAL-DEV-ONLY (excluded from the v2
workspace, publish = false) bridge from this crate's real evaluate CLI
into ruvnet/autogenous's new regression-candidate promotion path (separate,
unpushed branch feat/regression-candidate-kind): runs N independent
train+evaluate judges on genuinely distinct synthetic corpora, signs
receipts, and gets a real cryptographically-verified PROMOTE/REJECT
decision -- defense in depth on top of, not a replacement for, Darwin's own
promotion gate. Path-depends on a sibling autogenous checkout that does not
exist in CI; build/run directly via `cargo build --manifest-path
crates/ruforecast-autogenous-bridge/Cargo.toml`.
cargo test -p ruview-forecast-train --no-default-features --features
cpu,cli: all green (unaffected existing tests + this backward-compatible
addition).
* docs(forecast): retract round-2 HPO result, record honest multi-seed finding
The earlier "Informal HPO exploration note" claimed a real improvement
(WQL 0.257 -> 0.153) from a 3-round hyperparameter search. That search
evaluated every candidate against one fixed synthetic corpus (seed 0) for
every round -- textbook overfitting. Independent verification against two
fresh corpora (via the new ruforecast-autogenous-bridge crate) showed the
"winner" losing to the baseline on both.
Fixed the root cause in harness/ruview/flywheel/ruforecast/gate.mjs:
candidates are now scored against three independent corpora, worst-case
across them, not one fixed corpus.
Re-ran the search under the fix. It found a new winner that genuinely beat
baseline on all three of its own search seeds -- and that winner ALSO lost
independent verification on fresh seeds. Two independent search rounds,
pre- and post-fix, both produced an illusory "winner." The honest reading:
at this dataset scale (24 synthetic windows) held-out WQL varies enormously
by which corpus is drawn, regardless of hyperparameters -- confirmed
directly by the baseline genome's own primary swinging from 0.83 to a full
regression across the three fixed search seeds with unchanged
hyperparameters. No RuForecast hyperparameter configuration has been shown
to reliably beat the trivial baselines out-of-sample at this scale.
Also fixes a real design inconsistency surfaced by this exploration: the
Autogenous regression-candidate promotion verifier
(envelope::regression::verify_regression_promotion, separate unpushed
ruvnet/autogenous branch feat/regression-candidate-kind) required all
judges to share one corpus_id, which conflicts with this kind's
intentional cross-corpus judge design. Corrected there (commit bfa4c48);
did not change either REJECT verdict, which were already driven by the
real NotBetterThanParent signal on their own.
Append-only: the original round-2 row is kept, read together with this
new amendment section, per this doc's own evidence-ledger discipline.
* refactor(forecast): extract RuForecast to ruvnet/RuForecast submodule
Mirrors the v2/crates/worldgraph pattern: RuForecast becomes its own
independent public repo/workspace (ruforecast-core/model/train), mounted
at v2/crates/ruforecast as a git submodule, with v2/Cargo.toml path-depping
into its sub-crates and excluding the submodule from the v2 workspace.
Full v2 --workspace check passes; the ruforecast CLI builds and runs
correctly from its new location. Real git history for the extracted
crates (7 commits) was preserved via git-filter-repo, not squashed.
* feat(autogenous-bridge): real-data independent verification tool
real_data_verify.rs: signs two real, temporally-independent judge
measurements (different train/test split boundaries on the same real
household vitals corpus, not synthetic seeds) through the real
Autogenous regression-candidate promotion path, and gets a genuine
cryptographically-checked PROMOTE/REJECT verdict.
Result recorded (see docs/benchmarks/ruforecast.md): REJECT. One judge
nominally beat the trivial baseline but by less than the 0.01
non-inferiority margin; the other judge lost outright. Consistent with
every synthetic-data search this session -- no configuration has yet
been shown to reliably beat trivial baselines out-of-sample, now
including a real 6390-sample household corpus.
Bumps the ruforecast submodule to pick up the evaluate real-gap fix.
* docs(forecast): record real-household-data result (REJECT, signed)
6390 real vital-signs samples, two independent temporal splits,
independently verified via the real Autogenous regression-candidate
promotion path. Signed verdict: REJECT -- neither split cleared the
non-inferiority margin. Same conclusion as every synthetic search this
session, now confirmed with real data too: more real data is the
credible next lever, not further search on this scale of fixture.
* chore(forecast): point ruforecast submodule at published main (post gap-tolerance fix)
* ci(forecast): fix ruforecast-ci.yml for the new v2/crates/ruforecast submodule layout
The RuForecast crates were extracted into a standalone submodule (v2/crates/ruforecast, mirroring the existing v2/crates/worldgraph pattern) and are no longer members of the v2 workspace. The CI workflow still referenced the old in-tree package names/paths (ruview-forecast-core/model/train under v2/), which broke every forecast job with "cannot specify features for packages outside of workspace".
Fixed every job to target the submodule: working-directory -> v2/crates/ruforecast, -p ruview-forecast-* -> -p ruforecast-*, all hardcoded Cargo.toml/source paths in the Python assertion and clean-room-scan blocks, the Swatinem/rust-cache workspaces input, the artifact-tree output paths, and the two levels of relative path (../ -> ../../) in the informational-benchmark steps whose working directory moved one level deeper.
Also collapsed the paths: trigger filters internal-crate globs (v2/crates/ruview-forecast-*/**) down to the single v2/crates/ruforecast path, since GitHub Actions path filters only ever see a submodule gitlink change as one entry in the parent tree, never its internal file paths -- the old globs could never have matched anything.
Includes a real cargo fmt fix (submodule commit d9902f9) for formatting drift left over from the earlier real-data gap-tolerance fix, which the contract job's fmt --check step would otherwise have failed on.
All four forecast CI jobs' real commands verified locally before push: cargo test/check for CPU, CUDA/WGPU compile-check, the contract job's Python assertion block, the hosted-boundaries clean-room scan, and cargo fmt --check.
* docs(forecast): record real BIDMC cross-entity holdout result (REJECT)
Two independent real-patient partitions (34/16 and 24/26 splits, 53 total
ICU patients from PhysioNet BIDMC, Open Data Commons Attribution License
v1.0) both show the model losing to trivial baselines by a decisive margin
(+69% and +601% worse WQL). First genuine cross-entity real-data test this
project has run; same conclusion as every prior synthetic and single-household
test. Full Autogenous signed verification was not run for this entry -
flagged explicitly in the note.
* chore(forecast): point ruforecast submodule at published main (example feature-gate fix)
* chore(forecast): point ruforecast submodule at published main (clippy fix)
---------
Signed-off-by: Codex <codex@openai.com>
ADR-262 §8 question 5 left the second modality open, asking whether it should
be rvcsi. This answers ultrasonic instead, because the cost collapsed:
rufield-adapters now ships UltrasonicReplayAdapter, the first adapter for
Modality::Ultrasonic (registry code 7, empty since v0.1), which parses,
validates and signs BatVu range profiles upstream. RuView only has to decide
what it will put on a wire.
Bumps vendor/rufield 43b1df3 -> 9955672. Two struct literals in bridge.rs gain
fields added upstream (Observation: track_id, attributes, identity_evidence,
channel_sounding_provenance; SensorDescriptor: coordinate_frame, position_m,
orientation_xyzw). All left empty, each for a stated reason rather than a
convenient default — the pose fields in particular, because a CSI link has no
boresight and §6 makes no validated room-coordinate claim. The nine existing
P1 gates pass unchanged.
The decision this module makes is structural rather than a runtime refusal.
The adapter's full per-bin frame is P0 and would be dropped by the egress gate
after all the work of parsing and signing it; its 32-bin coarse reduction is
P1 and egress-safe. So the module does not offer the choice — it configures
the coarse mode, because a consumer cannot un-coarsen a coarse profile whereas
a check can be reordered. The gate still runs and is asserted to drop nothing.
12 gates in tests/ultrasonic_gates.rs, including the honest negative result: an
ultrasonic scan produces no fused inferences at all, and both independent
reasons are pinned. The adapter declines to populate `presence` — one
transducer pair cannot distinguish a person from a coat over the back of a
chair — and the engine's feature vocabulary is entirely statements about a
body, so range_m has nothing to drive.
The fixture is BatVu's own emitter output, byte-identical to the one in
ruvnet/rufield, so schema drift fails a build in one of three repositories
rather than an ingest in a deployment.
Not wired into the running server; P1 shipped as a library before P3 wired it
in, and this follows the same staging.
Use mesh-aligned capture timestamps, remediate Rust advisories, harden the audit gate, and correct deployment claims. Includes the live MQTT subscriber lifetime fix verified against Mosquitto.
Dragan Spiridonov