starred/RuView
1
0 Fork 0
mirror of https://github.com/ruvnet/RuView.git synced 2026-10-21 22:22:33 -05:00
Code Issues Packages Projects Releases 10 Wiki Activity

feat(training): compact retained ACF model and record spatial ablation

Browse Source
This commit is contained in:
rUv committed 2026-10-12 17:22:34 -07:00
1 parent 4ec8a86f5d
commit b39a0b2a2c
11 files changed
+1292
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/adr/ADR-420-public-amplitude-model-optimization.md
+20
View File
@@ -118,3 +118,23 @@ See [protocol and results](../espfi-resolution-experiment.md) and the
[semantic label audit](../espfi-label-audit-20261013.md). Numeric label renaming
alone cannot improve accuracy or macro F1. Deployment requires independent
confirmation and target device measurement even if a research gate passes.
## Cross-carrier and compact runtime boundary, 2026-10-13
The cross-carrier experiment keeps the original 190-sample inputs and training
folds. Deterministic centered log-amplitude correlations add no physical
coordinates or field history. Its frozen 18-configuration search selects the
unchanged control; reject both new representations and retain the negative
result in [the experiment report](../espfi-spatial-experiment.md).
Compact storage is a separate numerical transformation of the retained ACF
classifier. Reconstruct its exact seeded random map and preserve the original
float64 feature extraction. Require exact unfused logits and class predictions,
bounded folded error, provenance and artifact integrity before retaining it.
Storage savings do not establish runtime memory savings or speed gains. See
[the compact protocol](../espfi-motion-compact.md).
The [independent data audit](../espfi-independent-data-20261013.md) distinguishes
dataset terms from code licenses, anonymous metadata access from signal intake,
and compatible labels from similar activity names. No external dataset is
silently admitted as independent confirmation for this seven-class model.
docs/espfi-independent-data-20261013.md
+44
View File
@@ -0,0 +1,44 @@
# Independent CSI evaluation data audit
Checked 2026-10-13. **No signal recordings were downloaded, ingested or evaluated.** Anonymous requests retrieved repository source, dataset catalog metadata, a README, a label map and the initial metadata CSV header only. No credentials, upstream messages or paid resources were used.
The old ESP-Fi test was opened during the first pilot and the development partition has been reused adaptively. See [label and confirmation audit](espfi-label-audit-20261013.md). Another public dataset can supply independent evidence only after permissions, label compatibility and a frozen evaluation protocol are established.
## Decision matrix
| Candidate | Version and anonymous access | Data permission | Immediate decision |
| --- | --- | --- | --- |
| CSI-Bench | Kaggle version 12, updated 2026-06-16T23:36:11.083Z. Catalog, file listing and individual README/label-map downloads returned HTTP 200 without login | Dataset explicitly CC BY-NC-ND 4.0 and research only | Technically accessible metadata, but not admitted to the commercial model pipeline |
| WiFiTAD | Code revision `0bf3098bbc990cc5713415811ca08c05e10f6522`, committed 2025-08-26. Dataset linked as one Drive ZIP, without an explicit dataset version | Repository code Apache 2.0; a distinct dataset permission statement was not verified | Useful continuous evaluation candidate, pending dataset terms and verified archive access |
## CSI-Bench: strongest grouping, restrictive data terms
[Author repository](https://github.com/guozhen-jenn-zhu/CSI-Bench-Real-WiFi-Sensing-Benchmark/blob/b1beac7b0fa46e01748901195ecfbe309dfc0180/README.md), inspected revision `b1beac7b0fa46e01748901195ecfbe309dfc0180`, explicitly identifies Kaggle version 12 as the corrected benchmark release. Its last commit timestamp was 2026-06-21T22:45:59Z. Use this release rather than earlier reshaped arrays or third party mirrors.
The [author dataset](https://www.kaggle.com/datasets/guozhenjennzhu/csi-bench/versions/12) catalog reports 79,258,851,301 bytes. Anonymous [catalog](https://www.kaggle.com/api/v1/datasets/list?search=csi-bench) and [versioned file listing](https://www.kaggle.com/api/v1/datasets/list/guozhenjennzhu/csi-bench?datasetVersionNumber=12) were accessible. This is evidence for metadata access, not verification of every large data download.
The [versioned dataset README](https://www.kaggle.com/api/v1/datasets/download/guozhenjennzhu/csi-bench/README.md?datasetVersionNumber=12) was retrieved successfully: 6,033 bytes. It identifies processed HDF5, raw MATLAB files, and a `RawContinuousRecording` directory containing amplitude and phase. It explicitly restricts the dataset to research purposes and CC BY-NC-ND 4.0. These terms must not be replaced by the repository's MIT badge. Moreover, the repository README itself declares CC BY-NC-ND while its root LICENSE is MIT, so even the code permission presentation is inconsistent.
The [HAR label map](https://www.kaggle.com/api/v1/datasets/download/guozhenjennzhu/csi-bench/Multitask%2FHumanActivityRecognition%2Fmetadata%2Flabel_mapping.json?datasetVersionNumber=12), retrieved successfully at 284 bytes, defines five classes: jumping, running, seated-breathing, walking and wavinghand. They do not match all seven ESP-Fi classes. Semantic intersections cannot yet be frozen while the ESP-Fi naming conflict remains unresolved.
The versioned HAR `sample_metadata.csv` returned HTTP 200 with Content-Length 11,501,693 bytes; only its initial 4,096 bytes were read, to verify the header: `id,file_path,subject,user,activity,distance,num_sub,c_freq,environment,device,session,is_multi_subject,label`. No complete CSV counts or signal files were inspected. The repository describes user, activity, environment, device and session grouping, metadata CSVs, explicit label maps, and separate cross user/environment/device tests. Shared Multitask HDF5 recordings back HAR, identity and proximity tasks; those tasks cannot be treated as independent datasets. The author [project page](https://ai-iot-sensing.github.io/projects/project.html) reports 35 users, 26 environments and 16 device types across the whole collection; these are not verified counts for a proposed HAR slice.
Retrieved metadata SHA-256: dataset README `5835d7bdcddd94c7c3e4ac839dcca35d6928488950aaa090a9249eba1fa65067`; HAR label map `b5f649bbfc3940e056d51d1905504bdc8a7afcf603dd84a34f96951793de16e7`. These hashes attest metadata, not the uninspected signal archive.
Smallest next admissible operation: metadata inspection only. Read the complete versioned HAR metadata and official split lists, then determine whole session groups and class coverage without reading signal payloads. Do not choose sessions using candidate performance. If the intended use is permitted, predeclare a bounded pilot using whole sessions and a supported label contract before fetching just its referenced HDF5 files. Exact pilot bytes and the number of independently grouped sessions remain unknown. An isolated parser smoke test would not constitute confirmation accuracy.
## WiFiTAD: continuous annotations, limited environmental coverage
The [author paper](https://arxiv.org/html/2412.16233v2), revision dated 2025-01-26, and [AAAI publication](https://ojs.aaai.org/index.php/AAAI/article/view/32035), published 2025-04-11, describe 553 continuous recordings and 2,114 annotated activity instances. Collection used Intel 5300 hardware, 30 subcarriers at 100 Hz, three participants and one office. Labels include walking, running, jumping, waving, falling, sitting and standing with temporal boundaries. The published 70:30 partition is not a demonstrated unseen room split.
The [pinned loader](https://github.com/AVC2-UESTC/WiFiTAD/blob/0bf3098bbc990cc5713415811ca08c05e10f6522/TAD/dataset/load_csi.py) expects per recording NPY arrays, CSV recording and annotation tables, and a class index text file. The public code is Apache 2.0. That does not establish permission for the separately hosted dataset.
The official [Drive archive](https://drive.google.com/file/d/1gy0ppFtypVTtgBfrFzdMJUbXTb1MbPSK/view) rendered only a sign in link through the available web reader. This result does not establish that the file requires login for every download method, but anonymous signal access was not verified. No archive checksum, byte size, distinct dataset license or participant identifier schema was verified. No alternative mirror was substituted.
After these gaps are resolved, the smallest useful slice would be a complete annotated recording for parser and boundary alignment tests, followed by an independently grouped set with all required classes for scoring. One recording or one room cannot establish generalization across rooms. Do not concatenate ESP-Fi trials to imitate this continuous dataset.
## Recommendation and acceptance
Prioritize CSI-Bench version 12 for metadata planning because its group and split contracts are explicit. Do not train or evaluate the commercial candidate on its recordings under an assumption of permissive rights. WiFiTAD is the stronger match for continuous event timing, but first needs verified dataset permission and access. Neither is currently an admitted fresh confirmation set. Restricted WiFi-Wild is not proposed as an accessible fallback.
Acceptance: a frozen intake manifest records exact version, permitted purpose, source hashes, labels and whole session group membership; its adapter passes schema and timing checks; baseline and candidate are frozen before a previously unseen confirmation split is scored once. Any schema tuning data must be excluded from that confirmation split.
docs/espfi-model-optimization.md
+20
View File
@@ -492,3 +492,23 @@ naming conflicts and a separate benchmark output-index mapping. Consistent
semantic renaming cannot improve numeric class accuracy or macro F1. The
original final test was opened in the first pilot and is not reopened here;
repeated development scores remain adaptive research evidence.
## Cross-carrier ablation and compact ACF runtime, 2026-10-13
The [cross-carrier experiment](espfi-spatial-experiment.md) adds normalized
carrier correlations alone and alongside motion features. Both lose to the
exact retained control under the frozen training-fold selection. No accuracy
improvement or replacement model is claimed.
The [compact runtime protocol](espfi-motion-compact.md) separately removes
reconstructable random-map storage and folds classifier affine operations while
retaining float64 feature extraction. It requires all admitted-record prediction
parity before accepting a storage or execution improvement.
The retained compact weights occupy 16,088 bytes versus 496,488 bytes, with
identical predictions on all 907 admitted records. The small measured latency
difference misses the 10% speed threshold, so only the storage gain is claimed.
The [independent dataset audit](espfi-independent-data-20261013.md) verifies
corrected CSI-Bench metadata and distinguishes restrictive data terms from
repository licenses. Neither CSI-Bench nor WiFiTAD has been ingested or scored
as new confirmation evidence in this round.
docs/espfi-motion-compact.md
+102
View File
@@ -0,0 +1,102 @@
# Compact retained ACF model
Protocol frozen before execution, 2026-10-13. This changes the representation
and execution of the retained classifier, not its learned task or training data.
The exact original artifact remains the rollback model.
## Specification and architecture
The retained classifier stores a reproducible random Fourier projection and
phase alongside fitted weights. Reconstruct the projection from its exact
recipe instead of storing it: NumPy `default_rng(42)`, a float64 normal draw
of shape 468 by 128 with standard deviation sqrt(2 times 0.1 / 468), followed
by 128 uniform phases between zero and two pi. Export is allowed only when
both arrays exactly match the original model.
Store the 1,967 fitted float64 scalars: feature mean and scale, Fourier output
mean, classifier weights and bias. This removes reconstructable storage; it
does not reduce the runtime projection matrix or feature extraction memory.
Keep the existing float64 motion feature transform and its floors unchanged.
An unfused reconstructed runtime must reproduce original logits exactly.
The folded runtime computes the following once at load time:
* Projection divided by feature scale.
* Phase minus feature mean multiplied by the folded projection.
* Classifier bias minus output mean multiplied by classifier weights.
Folding changes floating point operation order, so it is judged separately
from exact projection reconstruction. The complete batch-one inference path
includes the original motion transform, validation and classification.
## Acceptance
Check all 837 admitted training and 70 development recordings against the
original artifact. Require exact predicted classes, exact unfused logits, and
folded maximum absolute logit error at most 1e-10. This is numerical equivalence
validation, not another accuracy search or independent confirmation result.
Do not deserialize the final test.
Run 1,000 rotating, alternating paired calls after 20 warmups. Both paths
exclude loading, random map generation, acquisition, pooling and I/O. Retain
the compact research artifact if its model file is below 10% of the original,
all parity checks pass, and measured p95 is at most 1.05 times the incumbent.
Call it a speed improvement only if median and p95 both improve by at least
10%. Storage improvement alone does not establish runtime memory or latency
improvement. No retraining, threshold search or development-based tuning occurs.
Verify bounded status, metadata and model bytes, model hashes, exact tensor
shape and dtype contracts, positive scales and known recipe. Reject failed
or corrupt artifacts and preserve failure status if export or verification
fails. Promotion remains unauthorized. Independent annotated data and target
hardware remain necessary for deployment claims.
## Measured result
The compact research artifact passes the declared retention gate. Its weights
file is 16,088 bytes versus 496,488 bytes, a 96.76% reduction. The three inference
artifact files total 17,245 bytes: weights 16,088, metadata 1,098 and status 59.
This total excludes code, dependencies and the optional measurement receipt.
The runtime reconstructs the large projection; no proportional RAM reduction
is claimed.
| Parity partition | Records | Unfused maximum logit error | Folded maximum logit error | Predictions |
| --- | ---: | ---: | ---: | --- |
| Training | 837 | 0 | 2.984e-15 | Identical |
| Reused development | 70 | 0 | 2.665e-15 | Identical |
Every admitted prediction matches. Development accuracy therefore remains
39/70, 55.71%, with macro F1 0.499871. This is a storage and numerical equivalence
result, not an accuracy improvement or fresh generalization evidence.
Paired median/p95 is 0.422395/0.594396 ms for compact folded inference versus
0.430370/0.623401 ms for the original. The candidate meets the allowed p95 budget
but misses the declared 10% gain threshold on both metrics. The single shared
CPU measurement does not establish a repeatable speed improvement. No further
timing search is used to change this decision.
Compact weights SHA256:
`87765a4da415465e3a4c5415b199da4593ef173e8fc0a2398dbe67faa454374d`.
The [sanitized receipt](validation/espfi-motion-compact-20261013.json) records
the original model, generated-map, implementation and full measurement digests.
Exact map regeneration is verified at load time; an incompatible RNG result
fails instead of silently changing the model. This run used NumPy 2.4.4.
All 59 public pipeline tests and 28 original encoder tests pass on ruOS.
The three new compact tests cover exact reconstruction, folded edge cases,
preserved original weights, artifact corruption, rehashed invalid state,
wrong seed recipe and failed export rejection. Independent source review found
no blocking issue.
```sh
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 PYTHONPATH=scripts \
python -m csi_training.espfi_motion_compact \
--manifest /private/espfi/prepared/manifest.json \
--baseline /private/espfi/motion-002-worst \
--output /private/espfi/motion-compact-001
```
Decision: retain the compact research bundle alongside the exact original.
Next acceptance is integration and latency measurement on the intended target,
with the same parity gate. Accuracy still needs independently annotated data;
this storage optimization does not address the existing zero-F1 class.
docs/espfi-spatial-experiment.md
+120
View File
@@ -0,0 +1,120 @@
# Cross-carrier correlation experiment
Protocol frozen on 2026-10-13 before execution. Research only, seven numeric
activity classes. This is a new representation experiment at the retained
190-sample temporal resolution, not a location or pose estimator.
## Hypothesis and specification
The retained model summarizes temporal autocorrelation and amplitude changes
independently for each of 52 subcarriers. Their joint fluctuations might add
activity information. This is a hypothesis; correlation can also encode room
conditions and worsen transfer. The experiment therefore retains the original
joint room and participant training folds and exact incumbent control.
Inputs are the existing admitted ESP-Fi training and development arrays and
retained motion artifact. No new capture intake or final test decoding is
needed. Outputs are a frozen selection receipt, bounded loadable research
artifact, source and model digests, paired timing and a retain or reject decision.
## Frozen architecture
| Family | Features | Definition |
| --- | ---: | --- |
| motion_acf | 468 | Exact retained autocorrelation and differences |
| cross_corr | 1,326 | Off-diagonal upper triangle of carrier correlations |
| motion_cross | 1,794 | Concatenation of both feature families |
For correlations, take log amplitude with floor 1e-6 and center each carrier
over the recording. Divide each centered carrier by the square root of its
energy, floored at 190e-12. Compute carrier dot products and retain the upper
triangle excluding the diagonal. Constant carriers have zero correlations.
This is a deterministic per-record transform with no fitted state or labels.
Process at most 32 records per correlation chunk. Carrier order follows the
source; it is not calibrated geometry, distance, position or field history.
Every family uses random Fourier features with 128 dimensions, seed 42,
gamma multipliers 0.1 and 1, and ridge penalties 0.01, 0.1 and 1. These 18
configurations are fixed before results. Fit scalers and classifier weights on
the fitting side of each original training fold only. Select by maximum minimum
fold macro F1, then mean F1, fewer stored scalars and declaration order.
The actual kernel gamma is the multiplier divided by the family's feature width,
after training-only feature standardization.
## Execution and integrity
1. Load only admitted training and development arrays through the existing
provenance and quarantine checks. Require the incumbent manifest binding.
2. Compute training representations and the frozen fold search.
3. Persist selection before extracting development features. Refit the exact
incumbent control and require identical learned tensors.
4. Fit the selected model, export bounded safetensors, and evaluate only this
selection on reused development. Require exact reload predictions.
5. Run 1,000 rotating, alternating paired batch-one calls after 20 warmups.
Measure both complete feature and classifier paths on the same inputs.
6. Complete the artifact only on success; errors mark it failed, output paths
cannot be reused, and incomplete or corrupt artifacts cannot be loaded.
Correlation models have larger projections, so their bounded artifact reader
allows up to 4 MiB. It verifies the digest of the same bytes deserialized and
checks tensor names, shapes, float64 dtype, finiteness and positive scales.
No signal arrays, full capture manifests or weights belong in Git.
## Acceptance and limits
Retain a research candidate only with development macro F1 gain at least 0.03,
nonregressing accuracy, no newly zero class F1 and candidate p95 below 1 ms.
Promotion remains unauthorized. Shared CPU timings exclude acquisition,
decoding, temporal pooling and I/O, and do not establish target device latency.
The original final test was opened once in the first pilot and is not reopened
here. Development has been reused across rounds; within-round selection does
not make it fresh independent evidence. Label semantics remain unresolved.
This experiment cannot establish SOTA, named activity performance, pose accuracy,
field generalization or benefit from calibrated spatial context.
```sh
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 PYTHONPATH=scripts \
python -m csi_training.espfi_spatial \
--manifest /private/espfi/prepared/manifest.json \
--baseline /private/espfi/motion-002-worst \
--output /private/espfi/spatial-001
```
The acceptance test for deployment remains a frozen comparison on independent
annotated sessions with the same accuracy gates and measured target latency.
## Measured result
The complete 18-configuration experiment finished successfully on existing
ruOS CPU compute. The unchanged control reproduced its learned tensors exactly.
| Best configuration per family | Weaker fold macro F1 | Mean fold macro F1 | Gamma multiplier | Ridge |
| --- | ---: | ---: | ---: | ---: |
| motion_acf | **0.332115** | **0.334030** | 0.1 | 0.1 |
| cross_corr | 0.115515 | 0.200194 | 1 | 0.01 |
| motion_cross | 0.239524 | 0.315203 | 0.1 | 0.01 |
The frozen selection returns the exact retained model. Development therefore
remains 39/70 correct, accuracy 55.71%, macro F1 0.499871. No development score
is computed for the rejected correlation configurations. No new model is
retained. These results reject this specific correlation representation and
grid; they do not prove that all joint-carrier methods are ineffective.
The selected artifact is 496,488 bytes with the same SHA256 as the incumbent:
`8d3c8b144663808159159f7af9d14b09f54ee78dc1f852c643fbd001a23d2fc5`.
Internal experiment duration is 3.210 seconds. Selected-wrapper median/p95 is
0.449270/0.623414 ms versus incumbent-wrapper 0.435755/0.605297 ms. Identical
weights and small wrapper differences do not support a model speed improvement.
The [sanitized receipt](validation/espfi-spatial-20261013.json) binds the full
measurement digest and source hashes. Four focused tests pass, covering the
mathematical reference, constant and anticorrelated carriers, batch independence,
development isolation, exact control, corrupt artifacts and failure lifecycle.
Independent source review found no blocking issue.
The integrated regression run passes all 59 public pipeline and 28 original
encoder tests on ruOS with verified exit code zero.
Decision: retain the existing ACF model and preserve this negative result.
The [independent data audit](espfi-independent-data-20261013.md) records the
access, label and license requirements for the next external benchmark.
docs/validation/espfi-motion-compact-20261013.json
+85
View File
@@ -0,0 +1,85 @@
{
"parity": {
"train": {
"samples": 837,
"runtimes": {
"unfused": {
"max_abs_logit_error": 0,
"identical_predictions": true
},
"folded": {
"max_abs_logit_error": 2.9837243786801082e-15,
"identical_predictions": true
}
}
},
"validation": {
"samples": 70,
"runtimes": {
"unfused": {
"max_abs_logit_error": 0,
"identical_predictions": true
},
"folded": {
"max_abs_logit_error": 2.6645352591003757e-15,
"identical_predictions": true
}
}
}
},
"latency": {
"candidate": {
"p50_ms": 0.42239499999999996,
"p95_ms": 0.5943955
},
"incumbent": {
"p50_ms": 0.43037000000000003,
"p95_ms": 0.6234009999999999
}
},
"model_bytes": 16088,
"baseline_model_bytes": 496488,
"stored_scalars": 1967,
"retention_checks": {
"model_below_10percent": true,
"parity_all_admitted": true,
"p95_within_5percent": true
},
"retained_as_research_candidate": true,
"measured_10percent_speed_gain": false,
"model_sha256": "87765a4da415465e3a4c5415b199da4593ef173e8fc0a2398dbe67faa454374d",
"baseline_model_sha256": "8d3c8b144663808159159f7af9d14b09f54ee78dc1f852c643fbd001a23d2fc5",
"random_map_sha256": {
"projection": "0384cc3a32f32185e45ebaf1f782430159178e784a2caee07f0782c4673321b5",
"phase": "03e0b646010f9adb25d6845c4de9c6f938a455fc0cda856fbdd04d1409f2b74e"
},
"source_sha256": {
"espfi_motion_compact.py": "74b13890afd33fbdf40efce3e91f44ddeb505d0f6d84f2d9a0b03adcd92c6f52",
"espfi_motion.py": "c6a8529ccf9d9f834383a2bdf5b14626764ec7ca2ade7ad5d4c1c14797d76054",
"espfi_kernel.py": "ae837cb2e3fac7f052b4bd371cd496e9636e5d5dbaf3a341526a4fc5aef23478",
"espfi_train.py": "f3692319b383c67d092776a7690b72963061efeaa3516f2d5e41b5dac1cfbd1e",
"espfi.py": "1ea4f72b3bbb0fe68e9903e6f32663a887bd9c8eac3a659932ff97b7e3d87124"
},
"elapsed_seconds": 1.7623121050000066,
"numpy_version": "2.4.4",
"full_receipt_sha256": "fadd6665fb97fa97f7fc858e8991b47a286d5503d07b433adb102bcb8ff05129",
"inference_files_bytes": {
"model.safetensors": 16088,
"metadata.json": 1098,
"status.json": 59
},
"schema_version": 1,
"protocol_date": "2026-10-13",
"status": "complete",
"evidence": "MEASURED",
"promotionAuthorized": false,
"manifest_sha256": "74e0fe039acbed1b287bd11ee6b779becbb8f9b66c3c249d260ed66f5583f02a",
"test_access": "Original final test opened once in pilot; not reopened. Train and reused development used only for numerical parity.",
"validation": {
"public_pipeline_tests": 59,
"original_encoder_tests": 28,
"verified_exit_code": 0,
"independent_source_review": "NO BLOCK"
},
"model_bytes_reduction_percent": 96.75963970931825
}
docs/validation/espfi-spatial-20261013.json
+249
View File
@@ -0,0 +1,249 @@
{
"development": {
"accuracy": 0.5571428571428572,
"macro_f1": 0.4998706443528175,
"per_class_f1": [
0.16666666666666666,
0.6451612903225806,
0,
0.8421052631578947,
0.5333333333333333,
0.6451612903225806,
0.6666666666666666
],
"confusion_matrix": [
[
1,
9,
0,
0,
0,
0,
0
],
[
0,
10,
0,
0,
0,
0,
0
],
[
0,
1,
0,
0,
1,
6,
2
],
[
0,
1,
1,
8,
0,
0,
0
],
[
0,
0,
2,
0,
4,
4,
0
],
[
0,
0,
0,
0,
0,
10,
0
],
[
1,
0,
1,
1,
0,
1,
6
]
],
"samples": 70
},
"incumbent_development": {
"accuracy": 0.5571428571428572,
"macro_f1": 0.4998706443528175,
"per_class_f1": [
0.16666666666666666,
0.6451612903225806,
0,
0.8421052631578947,
0.5333333333333333,
0.6451612903225806,
0.6666666666666666
],
"confusion_matrix": [
[
1,
9,
0,
0,
0,
0,
0
],
[
0,
10,
0,
0,
0,
0,
0
],
[
0,
1,
0,
0,
1,
6,
2
],
[
0,
1,
1,
8,
0,
0,
0
],
[
0,
0,
2,
0,
4,
4,
0
],
[
0,
0,
0,
0,
0,
10,
0
],
[
1,
0,
1,
1,
0,
1,
6
]
],
"samples": 70
},
"retention_checks": {
"macro_f1_gain": false,
"accuracy_nonregression": true,
"protect_each_previously_nonzero_class": true,
"p95_under_1ms": true
},
"retained_as_research_candidate": false,
"latency": {
"candidate": {
"p50_ms": 0.44926999999999995,
"p95_ms": 0.623414
},
"incumbent": {
"p50_ms": 0.435755,
"p95_ms": 0.605297
}
},
"model_bytes": 496488,
"model_sha256": "8d3c8b144663808159159f7af9d14b09f54ee78dc1f852c643fbd001a23d2fc5",
"baseline_model_sha256": "8d3c8b144663808159159f7af9d14b09f54ee78dc1f852c643fbd001a23d2fc5",
"source_sha256": {
"espfi_spatial.py": "fe690e72be77180b11efa80bccbf833c0f6154e994a12ab48d97aceafc1580b7",
"espfi.py": "1ea4f72b3bbb0fe68e9903e6f32663a887bd9c8eac3a659932ff97b7e3d87124",
"espfi_motion.py": "c6a8529ccf9d9f834383a2bdf5b14626764ec7ca2ade7ad5d4c1c14797d76054",
"espfi_generalize.py": "956b3a5e918f7c263a2b9183b4a6a70c8f2157604262a4ec6cee2c324a69224f",
"espfi_kernel.py": "ae837cb2e3fac7f052b4bd371cd496e9636e5d5dbaf3a341526a4fc5aef23478",
"espfi_train.py": "f3692319b383c67d092776a7690b72963061efeaa3516f2d5e41b5dac1cfbd1e"
},
"baseline_state_exact": true,
"elapsed_seconds": 3.20996624899999,
"manifest_sha256": "74e0fe039acbed1b287bd11ee6b779becbb8f9b66c3c249d260ed66f5583f02a",
"full_receipt_sha256": "6c54ce00b6d702bf90f5cb2767e1e78f8d79d5a9f2d30b689cfb9368daa3e410",
"selected": {
"family": "motion_acf",
"kind": "rff",
"dimensions": 128,
"gamma_multiplier": 0.1,
"alpha": 0.1
},
"families": [
{
"candidate": {
"family": "motion_acf",
"kind": "rff",
"dimensions": 128,
"gamma_multiplier": 0.1,
"alpha": 0.1
},
"minimum_cv_macro_f1": 0.33211530016861246,
"mean_cv_macro_f1": 0.33402959890262357
},
{
"candidate": {
"family": "cross_corr",
"kind": "rff",
"dimensions": 128,
"gamma_multiplier": 1,
"alpha": 0.01
},
"minimum_cv_macro_f1": 0.11551543694400837,
"mean_cv_macro_f1": 0.20019397158937186
},
{
"candidate": {
"family": "motion_cross",
"kind": "rff",
"dimensions": 128,
"gamma_multiplier": 0.1,
"alpha": 0.01
},
"minimum_cv_macro_f1": 0.2395240258000975,
"mean_cv_macro_f1": 0.31520255814042963
}
],
"schema_version": 1,
"evidence": "MEASURED",
"status": "complete",
"promotionAuthorized": false,
"protocol_date": "2026-10-13",
"test_access": "Original final test opened once in pilot; not reopened in this round. Development is reused adaptive evidence.",
"validation": {
"focused_tests": 4,
"focused_exit_code": 0,
"full_suite": "passed",
"independent_source_review": "NO BLOCK",
"public_pipeline_tests": 59,
"original_encoder_tests": 28,
"verified_exit_code": 0
}
}
scripts/csi_training/espfi_motion_compact.py
+190
View File
@@ -0,0 +1,190 @@
"""Lossless seed storage and measured float64 folding for the retained ACF model."""
import argparse
import hashlib
import json
from pathlib import Path
import time
import numpy as np
from safetensors.numpy import load, save_file
from .espfi_motion import MotionBundle, _read, _validate, motion_features
from .espfi_kernel import predict_features
from .espfi_train import digest, write_json
BASELINE = dict(family='motion_acf', kind='rff', dimensions=128, gamma_multiplier=.1, alpha=.1)
SHAPES = {'feature_mean': (468,), 'feature_scale': (468,), 'output_mean': (128,),
'weights': (128, 7), 'bias': (7,)}
SPEC = {'candidate': BASELINE, 'seed': 42, 'rng': 'numpy.default_rng.PCG64',
'recipe': 'normal(0,sqrt(2*.1/468),(468,128)), then uniform(0,2*pi,128)',
'precision': 'float64', 'features': 'unchanged motion_features motion_acf',
'max_abs_logit_error': 1e-10, 'promotionAuthorized': False}
def random_map():
rng = np.random.default_rng(42)
return {'projection': rng.normal(0, np.sqrt(2 * (.1 / 468)), (468, 128)),
'phase': rng.uniform(0, 2*np.pi, 128)}
def tensor_hash(value):
return hashlib.sha256(np.ascontiguousarray(value).tobytes()).hexdigest()
def validate(state):
if (set(state) != set(SHAPES) or any(state[k].shape != shape or state[k].dtype != np.float64
or not np.isfinite(state[k]).all() for k, shape in SHAPES.items())
or not (state['feature_scale'] > 0).all()):
raise ValueError('Invalid compact float64 ACF state')
class CompactMotionBundle:
def __init__(self, directory, folded=True):
root = Path(directory)
status = json.loads(_read(root/'status.json', 65536))
if not isinstance(status, dict) or status.get('status') != 'complete' or status.get('promotionAuthorized') is not False:
raise ValueError('Incomplete compact ACF artifact')
metadata = json.loads(_read(root/'metadata.json', 65536))
if (not isinstance(metadata, dict) or metadata.get('spec') != SPEC or metadata.get('status') != 'complete'
or metadata.get('promotionAuthorized') is not False):
raise ValueError('Invalid compact ACF metadata')
data = _read(root/'model.safetensors', 65536)
if hashlib.sha256(data).hexdigest() != metadata.get('model_sha256'):
raise ValueError('Compact model hash mismatch')
compact = load(data)
validate(compact)
generated = random_map()
if {k: tensor_hash(v) for k,v in generated.items()} != metadata.get('random_map_sha256'):
raise ValueError('Regenerated map differs from export')
self._initialize(compact, generated, folded)
def _initialize(self, compact, generated, folded):
self.folded = folded
if folded:
projection = generated['projection'] / compact['feature_scale'][:, None]
self.projection = np.ascontiguousarray(projection)
self.phase = generated['phase'] - compact['feature_mean'] @ projection
self.bias = compact['bias'] - compact['output_mean'] @ compact['weights']
self.weights = compact['weights']
if not all(np.isfinite(v).all() for v in (self.projection,self.phase,self.bias,self.weights)):
raise ValueError('Nonfinite folded state')
else:
self.state = dict(compact, **generated)
def predict(self, values):
features = motion_features(values, 'motion_acf')
if not self.folded:
return predict_features(self.state, features)
z = np.sqrt(2./128) * np.cos(features @ self.projection + self.phase)
result = z @ self.weights + self.bias
if not np.isfinite(result).all():
raise ValueError('Nonfinite compact inference')
return result
def export(parts, provenance, baseline_directory, output, max_seconds=600):
if not 1 <= max_seconds <= 3600:
raise ValueError('Invalid time budget')
_validate(parts)
baseline = MotionBundle(baseline_directory)
prior_bytes = _read(Path(baseline_directory)/'metrics.json',16*1024**2)
prior = json.loads(prior_bytes)
if (baseline.candidate != BASELINE or not provenance.get('manifest_sha256')
or prior.get('provenance',{}).get('manifest_sha256') != provenance['manifest_sha256']):
raise ValueError('Baseline candidate or provenance mismatch')
root = Path(output)
root.mkdir(parents=True, exist_ok=False)
write_json(root/'status.json',{'status':'running','promotionAuthorized':False})
start = time.monotonic()
try:
generated = random_map()
if any(not np.array_equal(baseline.state[k],generated[k]) for k in generated):
raise ValueError('Baseline projection differs from frozen seed recipe')
compact = {k:np.ascontiguousarray(baseline.state[k]) for k in SHAPES}
validate(compact)
save_file(compact,str(root/'model.safetensors'))
# Validate the exact serialized bytes before constructing either runtime.
replay = load(_read(root/'model.safetensors',65536))
validate(replay)
if any(not np.array_equal(replay[k],compact[k]) for k in SHAPES):
raise ValueError('Compact export changed state')
runtimes = {}
for folded in (False,True):
bundle = CompactMotionBundle.__new__(CompactMotionBundle)
bundle._initialize(replay,generated,folded)
runtimes['folded' if folded else 'unfused'] = bundle
parity = {}
for name,part in parts.items():
checks = {mode:{'max_abs_logit_error':0.,'identical_predictions':True} for mode in runtimes}
for offset in range(0,len(part['x']),32):
if time.monotonic()-start > max_seconds:
raise TimeoutError('Compact export budget exceeded')
x = part['x'][offset:offset+32]
expected = baseline.predict(x)
for mode,bundle in runtimes.items():
actual = bundle.predict(x)
error = float(np.max(np.abs(actual-expected)))
checks[mode]['max_abs_logit_error'] = max(checks[mode]['max_abs_logit_error'],error)
checks[mode]['identical_predictions'] &= bool(np.array_equal(actual.argmax(1),expected.argmax(1)))
if (mode == 'unfused' and not np.array_equal(actual,expected)) or error > 1e-10 or not checks[mode]['identical_predictions']:
raise ValueError('Compact runtime parity gate failed')
parity[name] = dict(samples=len(part['x']),runtimes=checks)
dev = parts['validation']['x']
selected = runtimes['folded']
for i in range(20):
x = dev[i%len(dev):i%len(dev)+1]
selected.predict(x); baseline.predict(x)
timings = {'candidate':[],'incumbent':[]}
for i in range(1000):
if time.monotonic()-start > max_seconds:
raise TimeoutError('Compact export budget exceeded')
x = dev[i%len(dev):i%len(dev)+1]
calls = [('candidate',selected.predict),('incumbent',baseline.predict)]
if i%2: calls.reverse()
for name,fn in calls:
before = time.perf_counter_ns(); fn(x)
timings[name].append((time.perf_counter_ns()-before)/1e6)
latency = {k:{'p50_ms':float(np.percentile(v,50)),'p95_ms':float(np.percentile(v,95))} for k,v in timings.items()}
size = (root/'model.safetensors').stat().st_size
old_size = (Path(baseline_directory)/'model.safetensors').stat().st_size
checks = {'model_below_10percent':size<.1*old_size,
'parity_all_admitted':True,
'p95_within_5percent':latency['candidate']['p95_ms']<=1.05*latency['incumbent']['p95_ms']}
metadata = {'status':'complete','promotionAuthorized':False,'spec':SPEC,
'model_sha256':digest(root/'model.safetensors'),
'random_map_sha256':{k:tensor_hash(v) for k,v in generated.items()},
'baseline_model_sha256':prior['model_sha256'],
'baseline_receipt_sha256':hashlib.sha256(prior_bytes).hexdigest(),
'provenance':{'manifest_sha256':provenance['manifest_sha256']}}
report = dict(metadata, provenance=provenance, evidence='SYNTHETIC' if provenance.get('evidence')=='SYNTHETIC' else 'MEASURED',
parity=parity,latency=latency,model_bytes=size,baseline_model_bytes=old_size,
stored_scalars=sum(v.size for v in compact.values()),runtime_mode='folded_float64',
retention_checks=checks,retained_as_research_candidate=all(checks.values()),
measured_10percent_speed_gain=all(latency['candidate'][q]<=.9*latency['incumbent'][q] for q in ('p50_ms','p95_ms')),
numpy_version=np.__version__,elapsed_seconds=time.monotonic()-start,
source_sha256={n:digest(Path(__file__).with_name(n)) for n in
('espfi_motion_compact.py','espfi_motion.py','espfi_kernel.py','espfi_train.py','espfi.py')},
scope='Seed storage and float64 algebra only; unchanged features, no training or accuracy search. '
'Train/development parity only, no final test. 1000 rotating alternating batch1 calls include features/classifier/checks, exclude pooling/acquisition/IO. '
'Storage reduction is not a speed claim; promotion remains unauthorized.')
if time.monotonic()-start > max_seconds:
raise TimeoutError('Compact export budget exceeded')
write_json(root/'metadata.json',metadata)
write_json(root/'metrics.json',report)
write_json(root/'status.json',{'status':'complete','promotionAuthorized':False})
return report
except Exception:
write_json(root/'status.json',{'status':'failed','promotionAuthorized':False})
raise
def main():
parser = argparse.ArgumentParser(description=__doc__)
for name in ('manifest','baseline','output'):
parser.add_argument('--'+name,required=True)
parser.add_argument('--max-seconds',type=int,default=600)
args=parser.parse_args()
from .espfi import load_development
parts,provenance=load_development(args.manifest)
print(json.dumps(export(parts,provenance,args.baseline,args.output,args.max_seconds),indent=2))
if __name__=='__main__':
main()
scripts/csi_training/espfi_spatial.py
+225
View File
@@ -0,0 +1,225 @@
"""Frozen cross-carrier relationship ablation on development-only public amplitudes."""
import argparse
import hashlib
import json
from pathlib import Path
import time
import numpy as np
from safetensors.numpy import load, save_file
from .espfi_kernel import fit, predict_features
from .espfi_motion import MotionBundle, _read, _validate, motion_features
from .espfi_train import digest, scores, write_json
WIDTHS = {'motion_acf': 468, 'cross_corr': 1326, 'motion_cross': 1794}
GRID = [dict(family=family, kind='rff', dimensions=128, gamma_multiplier=gamma, alpha=alpha)
for family in WIDTHS for gamma in (.1, 1.) for alpha in (.01, .1, 1.)]
BASELINE = dict(family='motion_acf', kind='rff', dimensions=128, gamma_multiplier=.1, alpha=.1)
CONTROL = BASELINE
CONTRACT = {
'seed': 42, 'grid': GRID, 'input': 'nonnegative_amplitude_N52T190',
'selection': 'minimum training-fold macro F1, then mean, then fewer stored scalars, then declaration order',
'log_floor': 1e-6, 'channel_energy_floor': 190e-12, 'transform_batch_size': 32,
'correlation': 'center log amplitude per channel; divide by sqrt(max(sum(centered**2),190e-12)); dot normalized channels; upper triangle excluding diagonal, row-major carrier indices',
'coordinates': 'Fixed dataset carrier order only; no physical spacing or calibrated frequency inferred',
'retention_gate': {'development_macro_f1_absolute_gain': .03,
'accuracy_nonregression': True,
'protect_each_previously_nonzero_class': True,
'paired_p95_limit_ms': 1.0},
'promotionAuthorized': False,
}
def spatial_features(values, family):
if family not in WIDTHS:
raise ValueError('Unknown spatial feature family')
x = np.asarray(values)
if (x.dtype.kind not in 'fiu' or x.ndim != 3 or x.shape[1:] != (52, 190)
or not 0 < len(x) <= 4096 or not np.isfinite(x).all() or (x < 0).any()):
raise ValueError('Invalid finite nonnegative spatial input shape')
if family == 'motion_acf':
return motion_features(x, 'motion_acf')
rows, upper = [], np.triu_indices(52, k=1)
for start in range(0, len(x), 32):
block = x[start:start+32].astype(np.float64, copy=False)
logged = np.log(np.maximum(block, 1e-6))
centered = logged - logged.mean(-1, keepdims=True)
norm = np.sqrt(np.maximum(np.sum(centered**2, axis=-1, keepdims=True), 190e-12))
normalized = centered / norm
matrix = normalized @ normalized.transpose(0, 2, 1)
rows.append(matrix[:, upper[0], upper[1]])
corr = np.concatenate(rows)
result = corr if family == 'cross_corr' else np.concatenate((motion_features(x, 'motion_acf'), corr), axis=1)
if result.shape != (len(x), WIDTHS[family]) or not np.isfinite(result).all():
raise ValueError('Nonfinite spatial features')
return np.ascontiguousarray(result)
def validate_parts(parts):
fold_ids = _validate(parts)
for part in parts.values():
x = np.asarray(part['x'])
if (not 0 < len(x) <= 4096 or x.shape != (len(part['y']), 52, 190)
or x.dtype.kind not in 'fiu' or not np.isfinite(x).all() or (x < 0).any()):
raise ValueError('Invalid bounded spatial partition')
return fold_ids
class SpatialBundle:
def __init__(self, directory):
root = Path(directory)
status = json.loads(_read(root / 'status.json', 65536))
if not isinstance(status, dict) or status.get('status') != 'complete':
raise ValueError('Invalid completed spatial artifact')
report = json.loads(_read(root / 'metrics.json', 16 * 1024**2))
if (not isinstance(status, dict) or not isinstance(report, dict)
or status.get('status') != 'complete' or report.get('status') != 'complete'
or status.get('promotionAuthorized') is not False
or report.get('promotionAuthorized') is not False
or report.get('contract') != CONTRACT
or report.get('task') != 'public_spatial_ablation'):
raise ValueError('Invalid completed spatial artifact')
self.candidate = report['selected']['candidate']
if self.candidate not in GRID:
raise ValueError('Unknown spatial candidate')
data = _read(root / 'model.safetensors', 4 * 1024**2)
if hashlib.sha256(data).hexdigest() != report.get('model_sha256'):
raise ValueError('Model hash mismatch')
self.state = load(data)
width = WIDTHS[self.candidate['family']]
shapes = {'feature_mean': (width,), 'feature_scale': (width,),
'projection': (width, 128), 'phase': (128,), 'output_mean': (128,),
'weights': (128, 7), 'bias': (7,)}
if (set(self.state) != set(shapes) or any(self.state[k].shape != shape
or self.state[k].dtype != np.float64 or not np.isfinite(self.state[k]).all()
for k, shape in shapes.items()) or not (self.state['feature_scale'] > 0).all()):
raise ValueError('Invalid spatial model state')
def predict(self, values):
return predict_features(self.state, spatial_features(values, self.candidate['family']))
def search(parts, provenance, baseline_directory, output, max_seconds=600):
if not 1 <= max_seconds <= 3600:
raise ValueError('Invalid time budget')
fold_ids = validate_parts(parts)
baseline = MotionBundle(baseline_directory)
prior_bytes = _read(Path(baseline_directory) / 'metrics.json', 16 * 1024**2)
prior = json.loads(prior_bytes)
if (baseline.candidate != BASELINE or not provenance.get('manifest_sha256')
or prior.get('provenance', {}).get('manifest_sha256') != provenance.get('manifest_sha256')):
raise ValueError('Baseline model or data provenance mismatch')
root = Path(output)
root.mkdir(exist_ok=False, parents=True)
write_json(root / 'status.json', {'status': 'running', 'promotionAuthorized': False})
write_json(root / 'search_contract.json', CONTRACT)
start = time.monotonic()
try:
train, dev = parts['train'], parts['validation']
cache = {family: spatial_features(train['x'], family) for family in WIDTHS}
results, selected, best = [], None, None
for candidate in GRID:
evaluations = []
for fit_ids, eval_ids in fold_ids:
if time.monotonic() - start > max_seconds:
raise TimeoutError('Spatial search budget exceeded')
state = fit(cache[candidate['family']][fit_ids], train['y'][fit_ids], candidate)
result = predict_features(state, cache[candidate['family']][eval_ids])
evaluations.append(scores(train['y'][eval_ids], result.argmax(1), 7))
row = {'candidate': candidate, 'folds': evaluations,
'minimum_cv_macro_f1': min(v['macro_f1'] for v in evaluations),
'mean_cv_macro_f1': float(np.mean([v['macro_f1'] for v in evaluations])),
'scalar_count': sum(v.size for v in state.values())}
results.append(row)
rank = (row['minimum_cv_macro_f1'], row['mean_cv_macro_f1'], -row['scalar_count'])
if best is None or rank > best:
best, selected = rank, row
# Persist the complete train-only choice before any development feature extraction.
write_json(root / 'selection.json', {'selected': selected, 'candidates': results, 'contract': CONTRACT})
chosen = selected['candidate']
state = fit(cache[chosen['family']], train['y'], chosen)
# Require the unchanged control to reproduce the exact existing learned state.
control = fit(cache['motion_acf'], train['y'], BASELINE)
if any(not np.array_equal(control[k], baseline.state[k]) for k in control):
raise ValueError('Unchanged baseline state did not reproduce')
save_file(state, str(root / 'model.safetensors'))
actual = predict_features(state, spatial_features(dev['x'], chosen['family']))
incumbent = scores(dev['y'], baseline.predict(dev['x']).argmax(1), 7)
measured = scores(dev['y'], actual.argmax(1), 7)
replay = load(_read(root / 'model.safetensors', 4 * 1024**2))
if not np.array_equal(actual, predict_features(replay, spatial_features(dev['x'], chosen['family']))):
raise ValueError('Reload changes logits')
def candidate_predict(x):
return predict_features(state, spatial_features(x, chosen['family']))
for i in range(20):
j = i % len(dev['y'])
candidate_predict(dev['x'][j:j+1])
baseline.predict(dev['x'][j:j+1])
timings = {'candidate': [], 'incumbent': []}
for i in range(1000):
if time.monotonic() - start > max_seconds:
raise TimeoutError('Spatial search budget exceeded')
j = i % len(dev['y'])
calls = [('candidate', candidate_predict, dev['x'][j:j+1]),
('incumbent', baseline.predict, dev['x'][j:j+1])]
if i % 2:
calls.reverse()
for name, fn, sample in calls:
before = time.perf_counter_ns()
fn(sample)
timings[name].append((time.perf_counter_ns() - before) / 1e6)
timing = {k: {'p50_ms': float(np.percentile(v, 50)), 'p95_ms': float(np.percentile(v, 95))}
for k, v in timings.items()}
gate = CONTRACT['retention_gate']
checks = {
'macro_f1_gain': measured['macro_f1'] >= incumbent['macro_f1'] + gate['development_macro_f1_absolute_gain'],
'accuracy_nonregression': measured['accuracy'] >= incumbent['accuracy'],
'protect_each_previously_nonzero_class': all(b == 0 or a > 0 for a, b in
zip(measured['per_class_f1'], incumbent['per_class_f1'])),
'p95_under_1ms': timing['candidate']['p95_ms'] < gate['paired_p95_limit_ms'],
}
report = {
'status': 'complete', 'promotionAuthorized': False, 'task': 'public_spatial_ablation',
'contract': CONTRACT, 'selected': selected, 'candidates': results,
'baseline_control': next(r for r in results if r['candidate'] == CONTROL),
'baseline_state_exact': True, 'reload_max_abs_logit_error': 0.0,
'evidence': 'SYNTHETIC' if provenance.get('evidence') == 'SYNTHETIC' else 'MEASURED',
'provenance': provenance, 'development': measured, 'incumbent_development': incumbent,
'retention_checks': checks, 'retained_as_research_candidate': all(checks.values()),
'latency': timing, 'model_bytes': (root / 'model.safetensors').stat().st_size,
'model_sha256': digest(root / 'model.safetensors'),
'baseline_model_sha256': prior['model_sha256'],
'baseline_receipt_sha256': hashlib.sha256(prior_bytes).hexdigest(),
'source_sha256': {name: digest(Path(__file__).with_name(name)) for name in
('espfi_spatial.py', 'espfi.py', 'espfi_motion.py', 'espfi_generalize.py', 'espfi_kernel.py', 'espfi_train.py')},
'elapsed_seconds': time.monotonic() - start,
'scope': '18 configurations selected by original training folds; adaptive research on reused development data. '
'1000 paired batch1 calls include feature extraction/checks/classifier, exclude acquisition/pooling/IO. '
'Same pooledN52T190 inputs; no acquisition/pooling timing or end-to-end speed claim. No physical geometry, final-test or hardware accuracy claim.',
}
if time.monotonic() - start > max_seconds:
raise TimeoutError('Spatial search budget exceeded')
write_json(root / 'metrics.json', report)
write_json(root / 'status.json', {'status': 'complete', 'promotionAuthorized': False})
return report
except Exception:
write_json(root / 'status.json', {'status': 'failed', 'promotionAuthorized': False})
raise
def main():
parser = argparse.ArgumentParser(description=__doc__)
for name in ('manifest', 'baseline', 'output'):
parser.add_argument('--' + name, required=True)
parser.add_argument('--max-seconds', type=int, default=600)
args = parser.parse_args()
from .espfi import load_development
parts, provenance = load_development(args.manifest)
result = search(parts, provenance, args.baseline, args.output, args.max_seconds)
print(json.dumps({k: result[k] for k in ('selected', 'development', 'incumbent_development',
'retention_checks', 'retained_as_research_candidate', 'latency', 'model_bytes')}, indent=2))
if __name__ == '__main__':
main()
scripts/tests/test_espfi_motion_compact.py
+94
View File
@@ -0,0 +1,94 @@
"""Synthetic storage and numerical-parity contracts, not sensing evidence."""
import json
from pathlib import Path
import sys
import tempfile
import unittest
import numpy as np
from safetensors.numpy import load, save_file
sys.path.insert(0,str(Path(__file__).resolve().parents[1]))
from csi_training.espfi_motion_compact import CompactMotionBundle, SHAPES, random_map, export
from csi_training.espfi_motion import MotionBundle
from csi_training.espfi_train import digest, write_json
from test_espfi_context import make_baseline
from test_espfi_generalize import fixture
class CompactMotionTests(unittest.TestCase):
def test_export_regeneration_folded_edges_storage_and_corruption(self):
parts,provenance=fixture()
with tempfile.TemporaryDirectory() as tmp:
root=Path(tmp)
make_baseline(parts,provenance,root/'baseline')
original=(root/'baseline'/'model.safetensors').read_bytes()
report=export(parts,provenance,root/'baseline',root/'compact')
self.assertEqual(report['stored_scalars'],1967)
self.assertLess(report['model_bytes'],.1*report['baseline_model_bytes'])
self.assertFalse(report['promotionAuthorized'])
self.assertEqual((root/'baseline'/'model.safetensors').read_bytes(),original)
saved=load((root/'compact'/'model.safetensors').read_bytes())
self.assertEqual(set(saved),set(SHAPES))
old=MotionBundle(root/'baseline')
unfused=CompactMotionBundle(root/'compact',folded=False)
folded=CompactMotionBundle(root/'compact')
self.assertFalse(hasattr(folded,'state'))
rng=np.random.default_rng(41)
for x in (parts['validation']['x'],np.zeros((2,52,190)),np.ones((2,52,190)),
np.full((2,52,190),1e100),
1e50*np.exp(rng.normal(0,1e-9,(2,52,190))),
parts['validation']['x'].astype(np.float64)*7):
expected=old.predict(x)
np.testing.assert_array_equal(unfused.predict(x),expected)
np.testing.assert_allclose(folded.predict(x),expected,rtol=0,atol=1e-10)
np.testing.assert_array_equal(folded.predict(x).argmax(1),expected.argmax(1))
self.assertTrue(report['parity']['train']['runtimes']['folded']['identical_predictions'])
path=root/'compact'/'model.safetensors'
path.write_bytes(path.read_bytes()+b'corruption')
with self.assertRaisesRegex(ValueError,'hash mismatch'):
CompactMotionBundle(root/'compact')
def test_recipe_mismatch_fails_lifecycle_and_provenance(self):
parts,provenance=fixture()
with tempfile.TemporaryDirectory() as tmp:
root=Path(tmp)
make_baseline(parts,provenance,root/'baseline')
with self.assertRaisesRegex(ValueError,'provenance'):
export(parts,dict(provenance,manifest_sha256='wrong'),root/'baseline',root/'wrong')
path=root/'baseline'/'model.safetensors'
state=load(path.read_bytes())
for name,value in random_map().items():
np.testing.assert_array_equal(state[name],value)
state['projection'][0,0]+=1e-9
save_file(state,str(path))
report=json.loads((root/'baseline'/'metrics.json').read_text())
report['model_sha256']=digest(path)
write_json(root/'baseline'/'metrics.json',report)
with self.assertRaisesRegex(ValueError,'seed recipe'):
export(parts,provenance,root/'baseline',root/'failed')
self.assertEqual(json.loads((root/'failed'/'status.json').read_text())['status'],'failed')
with self.assertRaises(ValueError):
CompactMotionBundle(root/'failed')
with self.assertRaises(FileExistsError):
export(parts,provenance,root/'baseline',root/'failed')
def test_rehashed_invalid_state_and_map_metadata_rejected(self):
parts,provenance=fixture()
with tempfile.TemporaryDirectory() as tmp:
root=Path(tmp)
make_baseline(parts,provenance,root/'baseline')
export(parts,provenance,root/'baseline',root/'compact')
meta_path=root/'compact'/'metadata.json'
original=json.loads(meta_path.read_text())
changed=dict(original,random_map_sha256={})
write_json(meta_path,changed)
with self.assertRaisesRegex(ValueError,'Regenerated map'):
CompactMotionBundle(root/'compact')
state_path=root/'compact'/'model.safetensors'
state=load(state_path.read_bytes());state['feature_scale'][0]=0.
save_file(state,str(state_path))
write_json(meta_path,dict(original,model_sha256=digest(state_path)))
with self.assertRaisesRegex(ValueError,'Invalid compact'):
CompactMotionBundle(root/'compact')
if __name__=='__main__':
unittest.main()
scripts/tests/test_espfi_spatial.py
+143
View File
@@ -0,0 +1,143 @@
"""Synthetic feature and experiment contracts, not real sensing accuracy."""
import copy
import json
from pathlib import Path
import sys
import tempfile
import unittest
from unittest.mock import patch
from safetensors.numpy import load
import numpy as np
from safetensors.numpy import save_file
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from csi_training.espfi_spatial import BASELINE, CONTRACT, GRID, WIDTHS, SpatialBundle, spatial_features, search
from csi_training.espfi_kernel import fit
from csi_training.espfi_motion import ROBUST_CONTRACT, motion_features
from csi_training.espfi_train import digest, write_json
from test_espfi_generalize import fixture
def make_baseline(parts, provenance, root):
root.mkdir()
state = fit(motion_features(parts['train']['x'], 'motion_acf'), parts['train']['y'], BASELINE)
save_file(state, str(root / 'model.safetensors'))
write_json(root / 'status.json', {'status': 'complete', 'promotionAuthorized': False})
write_json(root / 'metrics.json', {
'status': 'complete', 'promotionAuthorized': False, 'task': 'public_motion_ablation',
'contract': ROBUST_CONTRACT, 'selected': {'candidate': BASELINE},
'model_sha256': digest(root / 'model.safetensors'), 'provenance': provenance,
})
class SpatialTests(unittest.TestCase):
def test_correlation_reference_gain_reversal_control(self):
rng = np.random.default_rng(43)
x = np.exp(rng.normal(size=(2, 52, 190)))
actual = spatial_features(x, 'motion_cross')
np.testing.assert_array_equal(actual[:, :468], motion_features(x, 'motion_acf'))
np.testing.assert_array_equal(actual[:, 468:], spatial_features(x, 'cross_corr'))
np.testing.assert_allclose(actual, spatial_features(x[..., ::-1], 'motion_cross'), atol=1e-12)
gain = np.exp(rng.normal(size=(2, 52, 1)))
np.testing.assert_allclose(actual, spatial_features(x * gain, 'motion_cross'), atol=1e-12)
expected = np.stack([np.corrcoef(np.log(row))[np.triu_indices(52, 1)] for row in x])
np.testing.assert_allclose(actual[:, 468:], expected, atol=1e-12)
signal = np.linspace(-1, 1, 190)
logged = np.zeros((1, 52, 190))
logged[0, 0] = signal
logged[0, 1] = signal
logged[0, 2] = -signal
corr = spatial_features(np.exp(logged), 'cross_corr')
self.assertAlmostEqual(corr[0, 0], 1.)
self.assertAlmostEqual(corr[0, 1], -1.)
self.assertEqual(corr[0, 2], 0.)
self.assertLessEqual(np.abs(corr).max(), 1.+1e-12)
def test_chunk_independence_and_input_bounds(self):
x = np.exp(np.random.default_rng(51).normal(size=(35, 52, 190)))
for family in WIDTHS:
np.testing.assert_array_equal(spatial_features(x, family),
np.concatenate([spatial_features(row[None], family) for row in x]))
self.assertTrue(np.isfinite(spatial_features(np.zeros((1,52,190)),family)).all())
for invalid in (np.ones((0,52,190)),np.ones((1,52,950)),
-np.ones((1,52,190)),np.full((1,52,190),np.nan)):
with self.assertRaises(ValueError):
spatial_features(invalid,'cross_corr')
with self.assertRaises(ValueError):
spatial_features(x,'unknown')
def test_selection_isolation_control_reload_and_tamper(self):
parts, provenance = fixture()
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
make_baseline(parts, provenance, root / 'baseline')
original_features = spatial_features
def guarded(values, family):
if len(values) == len(parts['validation']['y']):
self.assertTrue((root/'a'/'selection.json').is_file())
return original_features(values, family)
with patch('csi_training.espfi_spatial.spatial_features', side_effect=guarded):
report = search(parts, provenance, root / 'baseline', root / 'a')
changed = copy.deepcopy(parts)
changed['validation']['x'] = changed['validation']['x'][..., ::-1] * 1.2
changed['validation']['y'] = (changed['validation']['y'] + 1) % 7
changed['validation']['identity'][:, 2] = changed['validation']['y'] + 1
other = search(changed, provenance, root / 'baseline', root / 'b')
self.assertEqual(len(GRID), 18)
self.assertEqual(report['selected'], other['selected'])
self.assertEqual(report['model_sha256'], other['model_sha256'])
self.assertTrue(report['baseline_state_exact'])
self.assertEqual(report['baseline_control']['candidate'], BASELINE)
winner = max(report['candidates'], key=lambda r: (r['minimum_cv_macro_f1'],
r['mean_cv_macro_f1'], -r['scalar_count']))
self.assertEqual(report['selected'], winner)
self.assertEqual(report['contract'], CONTRACT)
self.assertEqual(report['evidence'], 'SYNTHETIC')
self.assertFalse(report['promotionAuthorized'])
self.assertEqual(report['reload_max_abs_logit_error'], 0)
self.assertEqual(SpatialBundle(root / 'a').predict(parts['validation']['x']).shape, (7, 7))
# Changing the scored development labels cannot alter the persisted choice.
selection = json.loads((root / 'a' / 'selection.json').read_text())
self.assertEqual(selection['selected'], report['selected'])
self.assertNotIn('development', selection)
original = (root / 'a' / 'model.safetensors').read_bytes()
(root / 'a' / 'model.safetensors').write_bytes(original + b'corruption')
with self.assertRaisesRegex(ValueError, 'hash mismatch'):
SpatialBundle(root / 'a')
(root / 'a' / 'model.safetensors').write_bytes(original)
state = load(original)
state['feature_scale'][0] = 0.
save_file(state, str(root/'a'/'model.safetensors'))
forged = dict(report, model_sha256=digest(root/'a'/'model.safetensors'))
write_json(root/'a'/'metrics.json', forged)
with self.assertRaisesRegex(ValueError,'Invalid spatial model state'):
SpatialBundle(root/'a')
(root/'a'/'model.safetensors').write_bytes(original)
write_json(root/'a'/'metrics.json', report)
write_json(root / 'a' / 'status.json', {'status': 'failed', 'promotionAuthorized': False})
with self.assertRaises(ValueError):
SpatialBundle(root / 'a')
def test_partition_provenance_and_failed_control(self):
parts, provenance = fixture()
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
make_baseline(parts, provenance, root / 'baseline')
invalid = dict(parts, test=parts['validation'])
with self.assertRaises(ValueError):
search(invalid, provenance, root / 'baseline', root / 'bad_split')
with self.assertRaisesRegex(ValueError, 'provenance mismatch'):
search(parts, dict(provenance, manifest_sha256='different'), root / 'baseline', root / 'bad_hash')
# Wrong learned control cannot silently support a comparison.
changed = copy.deepcopy(parts)
changed['train']['x'][:, :, 10] *= 2.
with self.assertRaisesRegex(ValueError, 'did not reproduce'):
search(changed, provenance, root / 'baseline', root / 'bad_control')
self.assertEqual(json.loads((root / 'bad_control' / 'status.json').read_text())['status'], 'failed')
with self.assertRaises(FileExistsError):
search(parts, provenance, root / 'baseline', root / 'bad_control')
if __name__ == '__main__':
unittest.main()