feat(wiflow-dynamics): target-peeking bound at rung 3 resolves the open question

Runs the discriminating experiment: does an oracle that selects probes
using the TRUE y_t+1 beat not probing at all, at rung 3?

Answer: YES, decisively, and against the intuition.
  K=3  oracle 0.9620 vs no-probe 0.9416  (+0.0204, t=+19.5)
  K=9  oracle 0.9705 vs no-probe 0.9416  (+0.0290, t=+22.3)

So the marginal information in z_t+1 bands is NOT absent. Persistence
R2 0.871 suggested it might be; that inference was wrong. Probing can be
worth ~+2.9 pts at K=9. EIG-driven selection cannot find it -- it loses
0.4 pts instead of gaining 2.9, a ~3.3 pt gap to the bound.

Mechanism, and it is a design choice made here: the variance head is
deliberately blind to revealed VALUES (it sees only z_t and the mask),
which is what kept EIG closed-form and peek-free. The bound shows probe
value is value-dependent, not merely mask-dependent -- the oracle
exploits which values actually came back, and a mask-only variance head
structurally cannot represent that.

The fix is the observation head the original design constraint called
for and which was traded away for tractability: sample candidate
z_t+1[S] from p(z_t+1[S] | z_t, revealed) and average the resulting pose
entropies (Monte-Carlo EIG). That is the next experiment.

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01JJnDvpYYk2rTvjkudQUoCk

benchmarks/wiflow-dynamics/STATE.md

@@ -33,13 +33,28 @@ All numbers below are **MEASURED by this work**; the 0.9609 baseline is **CITED*
    speculated: diagonalising rung 2's covariance gives NLL −81.92 vs rung 3's ≈−98, so
    heteroscedasticity does pay on marginals; rung 2 won on joint NLL via its full
    30×30 covariance. Next: low-rank + diagonal head, heavier-tailed predictive.
-8. **The open question this leaves** (do not resolve by assertion): is the missing
-   information absent, or merely unreachable by this architecture? Persistence R²
-   0.871 says 87% of `z_{t+1}` is already implied by `z_t`, which argues *absent*.
-   Rung 2's target-peeking bound argued *some exists*. The discriminating experiment
-   is a target-peeking bound **at rung 3**: if even an oracle cannot beat no-probe,
-   the marginal information is not there and the primitive's value must come from a
-   different observation definition than time-averaged bands.
+8. **The open question is RESOLVED, and against the intuition.** I ran the
+   discriminating experiment — a target-peeking bound **at rung 3** (n=5,000 pairs):
+
+   | K | oracle | no-probe | delta | SE | t |
+   |---|---|---|---|---|---|
+   | 3 | 0.9620 | 0.9416 | **+0.0204** | 0.00104 | **+19.5** |
+   | 9 | 0.9705 | 0.9416 | **+0.0290** | 0.00130 | **+22.3** |
+
+   **The information EXISTS — abundantly.** Persistence R² 0.871 suggested it might be
+   absent; that inference was wrong, and measuring beat asserting. Probing *can* be
+   worth ~+2.9 pts at K=9. **EIG-driven selection simply cannot find it** (it loses
+   0.4 pts instead of gaining 2.9 — a ~3.3 pt gap to the bound).
+
+9. **Why EIG cannot reach it — the mechanism, and it is a design choice I made.** My
+   variance head is deliberately blind to revealed *values* (only `z_t` and the mask),
+   which is what kept EIG closed-form and peek-free. But this bound shows the value of
+   a probe here is **value-dependent**, not merely mask-dependent: the oracle exploits
+   *which values actually came back*, and a mask-only variance head structurally cannot
+   represent that. **The fix is the observation head the original design constraint
+   called for and I traded away for tractability:** sample candidate `z_{t+1}[S]` from
+   `p(z_{t+1}[S] | z_t, revealed)` and average the resulting pose entropies — proper
+   Monte-Carlo EIG. That is the single highest-value next experiment.
 
 **Nothing here is SOTA-beating, and nothing is comparable to the 96.09% full-CSI
 number** — different input abstraction (540 raw channels x 20 frames → 270 band means)

benchmarks/wiflow-dynamics/harness/src/bin/rung3.rs

@@ -77,6 +77,57 @@ fn random_groups(rng: &mut Rng, k: usize) -> Vec<usize> {
     pool[..k].to_vec()
 }
 
+/// Per-sample greedy probe selection using the TRUE y_{t+1} -- a target-peeking upper
+/// bound on every policy, achievable by none.
+fn oracle_probe_sets(
+    model: &HeteroModel<BI>,
+    z: &[f32],
+    gt: &[f32],
+    pairs: &[usize],
+    k: usize,
+    device: &NdArrayDevice,
+) -> Vec<Vec<usize>> {
+    let mut out = vec![Vec::new(); pairs.len()];
+    for _step in 0..k {
+        let mut best: Vec<(f64, usize)> = vec![(f64::INFINITY, usize::MAX); pairs.len()];
+        for cand in 0..NUM_PROBE_GROUPS {
+            let gs: Vec<Vec<usize>> = (0..pairs.len())
+                .map(|i| {
+                    let mut set = out[i].clone();
+                    if !set.contains(&cand) {
+                        set.push(cand);
+                    }
+                    set
+                })
+                .collect();
+            let (zt, mn, mk, yy) = make_batch::<BI>(z, gt, pairs, &gs, device);
+            let (mu, _) = model.forward(zt, mn, mk);
+            let mud: Vec<f32> = mu.into_data().to_vec().unwrap();
+            let yd: Vec<f32> = yy.into_data().to_vec().unwrap();
+            for i in 0..pairs.len() {
+                if out[i].contains(&cand) {
+                    continue;
+                }
+                let err: f64 = (0..POSE_DIM)
+                    .map(|d| {
+                        let j = i * POSE_DIM + d;
+                        ((mud[j] - yd[j]) as f64).powi(2)
+                    })
+                    .sum();
+                if err < best[i].0 {
+                    best[i] = (err, cand);
+                }
+            }
+        }
+        for i in 0..pairs.len() {
+            if best[i].1 != usize::MAX {
+                out[i].push(best[i].1);
+            }
+        }
+    }
+    out
+}
+
 /// Assemble one training/eval batch.
 fn make_batch<Bk: burn::tensor::backend::Backend>(
     z: &[f32],
@@ -248,6 +299,30 @@ fn main() {
         }));
     }
 
+    // ---- discriminating experiment: target-peeking bound AT RUNG 3 ----------
+    // Open question this resolves: is the marginal information ABSENT, or merely
+    // unreachable by the EIG criterion / this architecture? An oracle that selects
+    // probes using the TRUE y_{t+1} upper-bounds every policy. If even IT cannot beat
+    // not probing, the information is not there for this observation definition.
+    // Subsampled to 5,000 pairs for cost; no-probe is rescored on the SAME pairs.
+    let oracle_pairs = subsample(&test_pairs, 5_000, 999);
+    let oracle_noprobe = evaluate(&inf, z, gt, &oracle_pairs, &vec![vec![]; oracle_pairs.len()], &device);
+    let mut oracle_rows = Vec::new();
+    for &k in &[3usize, 9] {
+        let sets = oracle_probe_sets(&inf, z, gt, &oracle_pairs, k, &device);
+        let o = evaluate(&inf, z, gt, &oracle_pairs, &sets, &device);
+        let (d, se, t) = paired(&o.per_sample, &oracle_noprobe.per_sample);
+        oracle_rows.push(serde_json::json!({
+            "budget_k": k,
+            "oracle_pck@20": o.pck20,
+            "no_probe_pck@20": oracle_noprobe.pck20,
+            "oracle_minus_noprobe": { "delta": d, "se": se, "t": t, "beats_by_2se": t > 2.0 }
+        }));
+    }
+    let oracle_beats_noprobe = oracle_rows
+        .iter()
+        .any(|r| r["oracle_minus_noprobe"]["beats_by_2se"].as_bool().unwrap_or(false));
+
     let cov_ok = no_probe
         .coverage
         .as_array()
@@ -286,6 +361,18 @@ fn main() {
             "3_eig_beats_random_by_2se": eig_ok,
             "rung2_best_pck_for_reference": RUNG2_BEST_PCK
         },
+        "target_peeking_bound_at_rung3": {
+            "pairs": oracle_pairs.len(),
+            "rows": oracle_rows,
+            "oracle_beats_no_probe_by_2se": oracle_beats_noprobe,
+            "interpretation": if oracle_beats_noprobe {
+                "Information EXISTS but the EIG criterion / architecture cannot reach it."
+            } else {
+                "Even a target-peeking oracle cannot beat not probing: the marginal \
+                 information in time-averaged z_{t+1} bands is ABSENT, not merely \
+                 unreachable. The primitive needs a different observation definition."
+            }
+        },
         "does_probing_help_at_all": {
             "any_budget_where_eig_greedy_beats_no_probe_by_2se": any_probe_beats_noprobe,
             "no_probe_pck@20": no_probe.pck20,

benchmarks/wiflow-dynamics/results/rung3.json

@@ -15,6 +15,7 @@
     "mpjpe": 0.02603940523099254,
     "pck@20": 0.75565
   },
+  "init_seed": 2026,
   "milestone": "Rung 3 -- neural heteroscedastic head. MEASURED by harness/src/bin/rung3.rs, this run.",
   "no_probe": {
     "coverage": [
@@ -243,6 +244,35 @@
     "3_eig_beats_random_by_2se": false,
     "rung2_best_pck_for_reference": 0.8208
   },
+  "target_peeking_bound_at_rung3": {
+    "interpretation": "Information EXISTS but the EIG criterion / architecture cannot reach it.",
+    "oracle_beats_no_probe_by_2se": true,
+    "pairs": 5000,
+    "rows": [
+      {
+        "budget_k": 3,
+        "no_probe_pck@20": 0.9416,
+        "oracle_minus_noprobe": {
+          "beats_by_2se": true,
+          "delta": 0.02036000000000004,
+          "se": 0.0010449355175440225,
+          "t": 19.484455890496886
+        },
+        "oracle_pck@20": 0.96196
+      },
+      {
+        "budget_k": 9,
+        "no_probe_pck@20": 0.9416,
+        "oracle_minus_noprobe": {
+          "beats_by_2se": true,
+          "delta": 0.028946666666666586,
+          "se": 0.0012961647253593282,
+          "t": 22.33255241430974
+        },
+        "oracle_pck@20": 0.9705466666666667
+      }
+    ]
+  },
   "training": {
     "batch": 512,
     "epochs": 400,