mirror of
https://github.com/ruvnet/RuView.git
synced 2026-10-21 22:22:33 -05:00
Audit activity labels and evaluate native CSI resolution
Preserve native source precision and exact pooled controls under the original training and development split. Add an 18-configuration resolution ablation, input provenance and artifact tests, and measured results retaining the existing ACF model. Document unresolved semantic labels and confirmation limits.
This commit is contained in:
rUv committed
1 parent
56e5ae1a6d
commit
4ec8a86f5d
9 files changed
+1379
No files matched your search
@@ -98,3 +98,23 @@ The frozen temporal extension improved internal training folds but regressed
|
||||
development accuracy and macro F1. Reject it and preserve the retained motion
|
||||
ACF model. Evidence, executable commands and future field acceptance are in
|
||||
[`espfi-field-context-research.md`](../espfi-field-context-research.md).
|
||||
## Native resolution research boundary, 2026-10-13
|
||||
|
||||
The standalone resolution intake preserves the original manifest's admitted
|
||||
training and development identities and duplicate quarantine. It skips final
|
||||
test and excluded MATLAB payloads before decoding. Native amplitudes retain
|
||||
lossless float64 values; pooled control values preserve original source dtype,
|
||||
averaging order and float32 output. Every control tensor must match the parent
|
||||
dataset exactly, and the refitted control state must match the retained ACF
|
||||
artifact exactly.
|
||||
|
||||
The frozen 18-configuration experiment compares pooled190 with raw950 features
|
||||
at matched sample-index spans and shorter spans. Selection uses only the two
|
||||
original training folds. The original test was opened in the first pilot and
|
||||
is not reopened here. The reused development partition remains adaptive
|
||||
research evidence. No partition is relabeled as fresh confirmation.
|
||||
|
||||
See [protocol and results](../espfi-resolution-experiment.md) and the
|
||||
[semantic label audit](../espfi-label-audit-20261013.md). Numeric label renaming
|
||||
alone cannot improve accuracy or macro F1. Deployment requires independent
|
||||
confirmation and target device measurement even if a research gate passes.
|
||||
@@ -0,0 +1,57 @@
|
||||
# ESP-Fi label and confirmation audit
|
||||
|
||||
Audit date: 2026-10-13. Status: semantic mapping unresolved. No labels, recordings or model parameters changed.
|
||||
|
||||
The pinned source is `AutoSmartGroup/ESP-Fi-HAR` revision `c8bf6aa0d680fa02695c85a82b05e79b3ebb10c4`. A direct GitHub API check of `commits/main` on the audit date returned this same revision, dated 2026-03-04T04:49:57Z. There is no newer main revision resolving this conflict in the inspected source.
|
||||
|
||||
## Observed conflict
|
||||
|
||||
Both source README files describe the third filename component as the activity ID. Their semantic mapping disagrees with the supplied benchmark directories:
|
||||
|
||||
| Filename activity ID | README meaning | Benchmark directory meaning |
|
||||
| --- | --- | --- |
|
||||
| 1 | run | run |
|
||||
| 2 | fall | walk |
|
||||
| 3 | walk | jump |
|
||||
| 4 | turn | squat |
|
||||
| 5 | jump | arm wave |
|
||||
| 6 | squat | turn |
|
||||
| 7 | arm wave | fall |
|
||||
|
||||
The pinned recursive Git tree contains 560 benchmark MAT paths, with 80 filenames per activity ID. Each ID appears in exactly one semantic directory. For example, `Model Code/Data/test_amp/jump/3-1-3-1.mat` assigns filename ID 3 to the jump directory. This is direct path metadata evidence, not an interpretation of signal shape. The consistency of this subset does not establish which naming convention is correct for all 2,240 archive recordings.
|
||||
|
||||
The source `dataset.py` assigns output indices by enumerating directories in this order: `arm_wave`, `fall`, `jump`, `run`, `squat`, `turn`, `walk`. It does not parse activity numbers from filenames. Its classifier indices therefore differ from RuView's numeric IDs even independently of the README conflict. A comparison of score columns must use an explicit mapping, never positional coincidence.
|
||||
|
||||
Primary pinned sources and digests:
|
||||
|
||||
| Source | Git blob SHA1 | Content SHA256 |
|
||||
| --- | --- | --- |
|
||||
| [Root README](https://github.com/AutoSmartGroup/ESP-Fi-HAR/blob/c8bf6aa0d680fa02695c85a82b05e79b3ebb10c4/README.md) | `fe6e213a5f849b4ac62049f3e5a1d75a24890d27` | `b2b3f1f9296c848c9c4bf73bf760a719a907cd5a4acd2dd9325ca7a5b7792099` |
|
||||
| [Benchmark README](https://github.com/AutoSmartGroup/ESP-Fi-HAR/blob/c8bf6aa0d680fa02695c85a82b05e79b3ebb10c4/Model%20Code/README.md) | `74cd498c9a19859ec583ba4f1832f9b4038bb4dd` | `31f5b3ffc29ea1b0fd665f63f985620295eb780e716473070f4ee17cae333a2f` |
|
||||
| [Benchmark loader](https://github.com/AutoSmartGroup/ESP-Fi-HAR/blob/c8bf6aa0d680fa02695c85a82b05e79b3ebb10c4/Model%20Code/dataset.py) | `7379ded7c503b757c4490ee0fe88dfbec802d7a1` | `859cdf3150555002bc096258eb1473cef18f39d35dce559f78467ab6cccac977` |
|
||||
|
||||
[Recursive tree metadata](https://api.github.com/repos/AutoSmartGroup/ESP-Fi-HAR/git/trees/c8bf6aa0d680fa02695c85a82b05e79b3ebb10c4?recursive=1) provides the directory membership evidence. The current all states issue listing returned only [issue 1](https://github.com/AutoSmartGroup/ESP-Fi-HAR/issues/1), asking about sampling rate, with zero comments. No author correction of label semantics was found there. The publication remains [Wen et al., 2026](https://doi.org/10.1016/j.adhoc.2026.104192); no independently inspected authoritative annotation table resolves the discrepancy. Absence of a located correction is not proof that no correction exists elsewhere.
|
||||
|
||||
## What renaming can and cannot fix
|
||||
|
||||
RuView retains `activity_1` through `activity_7`, with target indices 0 through 6. Explicitly distinguish filename activity 3 from zero based classifier column 3, which represents filename activity 4.
|
||||
|
||||
A single bijection applied consistently to predictions and ground truth leaves every equality comparison unchanged, hence accuracy is invariant. It permutes the confusion matrix rows and columns together, so the seven per class F1 values are reordered and their macro average is unchanged. Thus semantic renaming cannot repair poor numeric class 3 recall or improve macro F1. It can correct descriptions and names only.
|
||||
|
||||
This invariance does not cover inconsistent relabeling, incorrect score column alignment, or annotation mistakes affecting individual recordings. The inspected benchmark subset supports a consistent naming permutation, but does not prove that every archive recording has a correct activity annotation. RuView must not promote a named fall detector from this evidence.
|
||||
|
||||
## Confirmation status
|
||||
|
||||
The first pilot opened the original 70 recording test partition once and reported 22/70 correct, macro F1 0.288679. Subsequent model development rounds did not reopen those arrays, according to the recorded experiment history and development loader boundary. This audit performed no dataset or test tensor access and is not an independent access log attestation.
|
||||
|
||||
Keeping the original test closed in later runs avoids additional direct test queries; it does not make previously disclosed test results fresh confirmation evidence again. The repeatedly evaluated 70 recording development partition is adaptive research data. Frozen candidate selection within individual rounds does not erase adaptation between rounds. Its gains remain development results.
|
||||
|
||||
The current split tests one particular room and participant combination. Neither a rename nor another evaluation on the same development set establishes performance across new homes, devices or users.
|
||||
|
||||
## Decision and acceptance
|
||||
|
||||
Keep numeric labels until an author supplied correction or independently verified annotation record identifies the mapping and its scope across the full archive. Preserve the old source revision and explicit mapping version when that happens. Do not contact upstream or alter labels as part of this audit.
|
||||
|
||||
Fresh confirmation requires independently labeled, consented captures or a genuinely untouched authorized dataset under a protocol frozen before inspecting candidate results. Include new sessions, multiple participants and environments, calibrated empty periods when evaluating presence, and hashes binding annotations to recordings.
|
||||
|
||||
Acceptance test: consistent permutation of a saved prediction vector and its targets must leave accuracy and macro F1 unchanged. Separately, a fixed model and baseline must pass the declared comparison on fresh confirmation data. Neither condition substitutes for the other.
|
||||
@@ -479,3 +479,16 @@ rejection policies. Public unlabeled replay is inspected for format only, with
|
||||
no assumed empty room, activity labels or fabricated positions. Software
|
||||
integrity and synthetic timing are not predictive accuracy evidence. The report
|
||||
and aggregate receipts preserve that distinction and the negative model result.
|
||||
## Native resolution and label audit, 2026-10-13
|
||||
|
||||
The [frozen resolution experiment](espfi-resolution-experiment.md) compares
|
||||
950 native samples against the existing 190 pooled means. Both native ACF
|
||||
families lose to the pooled control on minimum and mean training fold macro
|
||||
F1. Selection returns the exact existing model and weight hash. Development
|
||||
remains 39/70 correct with macro F1 0.499871; no new candidate is retained.
|
||||
|
||||
The [label audit](espfi-label-audit-20261013.md) confirms unresolved source
|
||||
naming conflicts and a separate benchmark output-index mapping. Consistent
|
||||
semantic renaming cannot improve numeric class accuracy or macro F1. The
|
||||
original final test was opened in the first pilot and is not reopened here;
|
||||
repeated development scores remain adaptive research evidence.
|
||||
@@ -0,0 +1,167 @@
|
||||
# Native resolution experiment
|
||||
|
||||
Protocol date: 2026-10-13. Research only. The existing ACF model remains the
|
||||
control. This experiment measures seven numeric activity classes, not pose.
|
||||
|
||||
## Specification
|
||||
|
||||
Question: does retaining native sample resolution improve the activity
|
||||
representation under the existing joint room and participant transfer protocol?
|
||||
The source has 950 samples per carrier, while the retained model receives 190
|
||||
means of five consecutive samples. Averaging could remove useful variation;
|
||||
that is a hypothesis, not an established explanation for any class error.
|
||||
|
||||
The [label audit](espfi-label-audit-20261013.md) treats semantic naming as a
|
||||
separate provenance problem. A consistent renaming cannot change accuracy or
|
||||
macro F1. No named activity or fall detection claim follows from numeric scores.
|
||||
|
||||
Inputs are the exact pinned ESP-Fi archive, the original prepared manifest and
|
||||
the retained ACF artifact. Outputs are a reproducible native intake, an explicit
|
||||
model contract, source and model hashes, training fold diagnostics, one adaptive
|
||||
development comparison, and a retain or reject research decision.
|
||||
|
||||
## Frozen architecture and selection
|
||||
|
||||
The complete grid is fixed before reading candidate development results:
|
||||
|
||||
| Family | Input samples | Autocorrelation lags | Absolute difference lags |
|
||||
| --- | ---: | --- | --- |
|
||||
| Pooled control | 190 | 1, 2, 4, 8, 16, 32 | 1, 4, 16 |
|
||||
| Native, matched spans | 950 | 5, 10, 20, 40, 80, 160 | 5, 20, 80 |
|
||||
| Native, short spans | 950 | 1, 2, 4, 8, 16, 32 | 1, 4, 16 |
|
||||
|
||||
All families have 468 features. Each uses gamma multipliers 0.1 and 1, ridge
|
||||
penalties 0.01, 0.1 and 1, 128 random Fourier dimensions and seed 42. These 18
|
||||
configurations use the original two training folds. Select highest minimum fold
|
||||
macro F1, then mean fold F1, then declaration order. No new grid or followup
|
||||
selection rule is introduced after results are observed.
|
||||
|
||||
The matched lags preserve spans in original sample indices. Source timestamps
|
||||
are unavailable, so neither lag family implies calibrated seconds or Hz.
|
||||
Log amplitude centering and global per-record RMS remain float64, with log and
|
||||
RMS floors of 1e-6. The energy floor is sample count times 1e-12.
|
||||
|
||||
## Intake and trust boundary
|
||||
|
||||
Verify the complete archive size and digest before processing. Validate the
|
||||
original manifest population, quarantine and split policy. Decode only its
|
||||
admitted train and development MAT records; skip test and excluded payloads
|
||||
before requesting their blocks or calling the MATLAB decoder. Require every
|
||||
selected member digest to match the original manifest.
|
||||
|
||||
Retain native values losslessly in float64. Produce the control directly from
|
||||
the original source dtype and averaging order, with its original final float32
|
||||
cast. Verify exact pooled tensor equality against the existing prepared data.
|
||||
Preserving raw float64 input uses ten times the tensor bytes of the float32
|
||||
190-sample control; the temporal sample count itself grows fivefold.
|
||||
|
||||
The new bundle stores train and development only, plus the original parent
|
||||
manifest for provenance. The loader verifies completion status, the parent
|
||||
manifest hash and quarantine, fixed array headers and sizes, bounded hashed
|
||||
bundle bytes, raw and pooled signal digests and exact raw-to-pooled conversion.
|
||||
Raw samples, individual manifests and model weights remain outside Git.
|
||||
|
||||
## Execution and failure paths
|
||||
|
||||
1. Prepare the native bundle without changing original split membership.
|
||||
2. Compute training features and fit each fold on its training rows only.
|
||||
3. Write the selected configuration and all fold results before computing any
|
||||
development features.
|
||||
4. Refit the pooled control and require exact equality with the retained model
|
||||
state. Intake separately verifies the pooled tensors against the parent data.
|
||||
5. Fit the selected configuration on all admitted training rows, export it,
|
||||
evaluate development once, and require exact reload predictions.
|
||||
6. Run 1,000 alternating paired batch-one calls after 20 warmups, recording
|
||||
median and p95. Each path receives its own already prepared input shape.
|
||||
7. Mark the artifact complete only after all checks finish. Exceptions mark it
|
||||
failed; failed artifacts cannot be loaded and output directories cannot be
|
||||
reused silently.
|
||||
|
||||
The timings include validation, feature extraction and classification. They
|
||||
exclude acquisition, source decoding, pooling and I/O. They are shared x86 CPU
|
||||
measurements, not target device or sensor-to-decision guarantees. No overlapping
|
||||
build or separate benchmark is launched by this experiment.
|
||||
|
||||
```sh
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 PYTHONPATH=scripts \
|
||||
python -m csi_training.espfi_resolution_data \
|
||||
--archive /private/espfi/source.rar \
|
||||
--parent-manifest /private/espfi/prepared/manifest.json \
|
||||
--output /private/espfi/native-prepared
|
||||
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 PYTHONPATH=scripts \
|
||||
python -m csi_training.espfi_resolution \
|
||||
--manifest /private/espfi/native-prepared/manifest.json \
|
||||
--baseline /private/espfi/motion-002-worst \
|
||||
--output /private/espfi/resolution-001
|
||||
```
|
||||
|
||||
## Acceptance and confirmation limits
|
||||
|
||||
The research retention gate is development macro F1 at least 0.03 above the
|
||||
retained ACF model, no accuracy regression, no newly zero class F1, and measured
|
||||
p95 below 1 ms. A passed gate would retain a research candidate only.
|
||||
`promotionAuthorized` remains false in every outcome.
|
||||
|
||||
The original final test was opened once in the first pilot. This round does
|
||||
not reopen it or call it fresh. The 70 development recordings have been reused
|
||||
throughout optimization; training-only selection within a round does not erase
|
||||
adaptation between rounds. No available partition is silently reassigned as a
|
||||
new confirmation set. Independent deployment acceptance still needs fresh
|
||||
annotated recordings and target hardware latency. Those inputs are not supplied
|
||||
by a source resolution change.
|
||||
|
||||
## Measured result
|
||||
|
||||
The pinned intake admits exactly 837 training and 70 development recordings.
|
||||
All admitted source matrices are float64. Original pooled samples and the
|
||||
refitted control state reproduce exactly. Intake took 31.518 seconds; the
|
||||
experiment process took 12.019 seconds including startup and loading, with
|
||||
6.865 seconds reported inside the model experiment.
|
||||
|
||||
| Best configuration per family | Weaker fold macro F1 | Mean fold macro F1 | Gamma | Ridge |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| Pooled190 control | **0.332115** | **0.334030** | 0.1 | 0.1 |
|
||||
| Raw950 matched spans | 0.300862 | 0.321984 | 0.1 | 0.1 |
|
||||
| Raw950 short spans | 0.299106 | 0.304703 | 1 | 0.1 |
|
||||
|
||||
The declared selection chooses the pooled control. Both raw families lose on
|
||||
the weaker fold and the mean fold score. Only the selected control is evaluated
|
||||
on development; this round does not report development accuracy for rejected
|
||||
raw configurations. No additional search follows the result.
|
||||
|
||||
Development remains 39/70 correct, accuracy 55.71%, macro F1 0.499871. Exported
|
||||
weights are byte identical to the retained artifact, SHA256
|
||||
`8d3c8b144663808159159f7af9d14b09f54ee78dc1f852c643fbd001a23d2fc5`,
|
||||
with 496,488 bytes. The required 0.03 macro F1 gain is absent, so no new candidate
|
||||
is retained. This result rejects these specific raw ACF representations; it
|
||||
does not establish that native resolution cannot help another architecture.
|
||||
|
||||
Numeric activity 3 is already weak in the pooled control's two training folds:
|
||||
F1 0.08 and 0.20, with 1/20 and 3/20 correct. The matched native family improves
|
||||
those values to 0.090909 and 0.307692, but loses on overall transfer. This is a
|
||||
diagnostic tradeoff, not a reason to override the frozen selection. The reused
|
||||
development class still has zero F1; semantic renaming cannot change that.
|
||||
|
||||
Paired selected-wrapper median/p95 is 0.492675/0.701795 ms, versus the incumbent
|
||||
wrapper's 0.480525/0.686287 ms. Both execute the same pooled model state. The
|
||||
small difference reflects wrapper overhead and shared-host timing variation;
|
||||
it is not a new model speed improvement. The raw candidates were not timed as
|
||||
retained deployment candidates.
|
||||
|
||||
The [sanitized aggregate receipt](validation/espfi-resolution-20261013.json)
|
||||
contains source, parent manifest, native manifest, model and full receipt
|
||||
digests, class diagnostics and explicit test-access limits. Full capture
|
||||
manifests and model artifacts remain outside the repository.
|
||||
|
||||
Validation on ruOS passes all 52 public pipeline tests and 28 original encoder
|
||||
tests. The eight new focused tests cover exact pooling and control state,
|
||||
source precision, admission before payload decoding, failure lifecycle,
|
||||
artifact tampering, development isolation and exact chunked arithmetic.
|
||||
Independent source review found no blocking issue.
|
||||
|
||||
Decision: keep the existing pooled ACF model. The immediate proposed acceptance
|
||||
test is a frozen comparison on independently annotated new sessions, with at
|
||||
least 0.03 absolute macro F1 gain, no newly zero F1 class and target device p95
|
||||
within the agreed budget. This round supplies neither fresh confirmation nor
|
||||
target hardware evidence and authorizes no deployment.
|
||||
@@ -0,0 +1,356 @@
|
||||
{
|
||||
"date_utc": "2026-10-13",
|
||||
"status": "complete",
|
||||
"promotionAuthorized": false,
|
||||
"task": "public_resolution_ablation",
|
||||
"evidence": "MEASURED",
|
||||
"development": {
|
||||
"accuracy": 0.5571428571428572,
|
||||
"macro_f1": 0.4998706443528175,
|
||||
"per_class_f1": [
|
||||
0.16666666666666666,
|
||||
0.6451612903225806,
|
||||
0,
|
||||
0.8421052631578947,
|
||||
0.5333333333333333,
|
||||
0.6451612903225806,
|
||||
0.6666666666666666
|
||||
],
|
||||
"confusion_matrix": [
|
||||
[
|
||||
1,
|
||||
9,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0
|
||||
],
|
||||
[
|
||||
0,
|
||||
10,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0
|
||||
],
|
||||
[
|
||||
0,
|
||||
1,
|
||||
0,
|
||||
0,
|
||||
1,
|
||||
6,
|
||||
2
|
||||
],
|
||||
[
|
||||
0,
|
||||
1,
|
||||
1,
|
||||
8,
|
||||
0,
|
||||
0,
|
||||
0
|
||||
],
|
||||
[
|
||||
0,
|
||||
0,
|
||||
2,
|
||||
0,
|
||||
4,
|
||||
4,
|
||||
0
|
||||
],
|
||||
[
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
10,
|
||||
0
|
||||
],
|
||||
[
|
||||
1,
|
||||
0,
|
||||
1,
|
||||
1,
|
||||
0,
|
||||
1,
|
||||
6
|
||||
]
|
||||
],
|
||||
"samples": 70
|
||||
},
|
||||
"incumbent_development": {
|
||||
"accuracy": 0.5571428571428572,
|
||||
"macro_f1": 0.4998706443528175,
|
||||
"per_class_f1": [
|
||||
0.16666666666666666,
|
||||
0.6451612903225806,
|
||||
0,
|
||||
0.8421052631578947,
|
||||
0.5333333333333333,
|
||||
0.6451612903225806,
|
||||
0.6666666666666666
|
||||
],
|
||||
"confusion_matrix": [
|
||||
[
|
||||
1,
|
||||
9,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0
|
||||
],
|
||||
[
|
||||
0,
|
||||
10,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0
|
||||
],
|
||||
[
|
||||
0,
|
||||
1,
|
||||
0,
|
||||
0,
|
||||
1,
|
||||
6,
|
||||
2
|
||||
],
|
||||
[
|
||||
0,
|
||||
1,
|
||||
1,
|
||||
8,
|
||||
0,
|
||||
0,
|
||||
0
|
||||
],
|
||||
[
|
||||
0,
|
||||
0,
|
||||
2,
|
||||
0,
|
||||
4,
|
||||
4,
|
||||
0
|
||||
],
|
||||
[
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
10,
|
||||
0
|
||||
],
|
||||
[
|
||||
1,
|
||||
0,
|
||||
1,
|
||||
1,
|
||||
0,
|
||||
1,
|
||||
6
|
||||
]
|
||||
],
|
||||
"samples": 70
|
||||
},
|
||||
"baseline_state_exact": true,
|
||||
"reload_max_abs_logit_error": 0,
|
||||
"retention_checks": {
|
||||
"macro_f1_gain": false,
|
||||
"accuracy_nonregression": true,
|
||||
"protect_each_previously_nonzero_class": true,
|
||||
"p95_under_1ms": true
|
||||
},
|
||||
"retained_as_research_candidate": false,
|
||||
"latency": {
|
||||
"candidate": {
|
||||
"p50_ms": 0.492675,
|
||||
"p95_ms": 0.7017945
|
||||
},
|
||||
"incumbent": {
|
||||
"p50_ms": 0.480525,
|
||||
"p95_ms": 0.6862874999999998
|
||||
}
|
||||
},
|
||||
"model_bytes": 496488,
|
||||
"model_sha256": "8d3c8b144663808159159f7af9d14b09f54ee78dc1f852c643fbd001a23d2fc5",
|
||||
"baseline_model_sha256": "8d3c8b144663808159159f7af9d14b09f54ee78dc1f852c643fbd001a23d2fc5",
|
||||
"baseline_receipt_sha256": "f8a565cb031d3ce9b488432dfb97408a58cee8f511c0ddc89aa74edb90388b88",
|
||||
"elapsed_seconds": 6.864779120000094,
|
||||
"selected": {
|
||||
"family": "pooled190",
|
||||
"kind": "rff",
|
||||
"dimensions": 128,
|
||||
"gamma_multiplier": 0.1,
|
||||
"alpha": 0.1
|
||||
},
|
||||
"full_receipt_sha256": "7bf24b959fdcc7ac760b34464991c9eecde706acb6c0bd5b8fb11da8b2e6ccc3",
|
||||
"source_sha256": {
|
||||
"espfi_resolution.py": "6203c6eb4a0a1f1e8e3b06b2af6daa84a4e917967426628a53b634c0bb6bc6a7",
|
||||
"espfi_resolution_data.py": "021ee037e7bb1c41a01fa590d50aa93a77fa3fdcc7f14562099a062145518274",
|
||||
"espfi_motion.py": "c6a8529ccf9d9f834383a2bdf5b14626764ec7ca2ade7ad5d4c1c14797d76054",
|
||||
"espfi_generalize.py": "956b3a5e918f7c263a2b9183b4a6a70c8f2157604262a4ec6cee2c324a69224f",
|
||||
"espfi_kernel.py": "ae837cb2e3fac7f052b4bd371cd496e9636e5d5dbaf3a341526a4fc5aef23478",
|
||||
"espfi_train.py": "f3692319b383c67d092776a7690b72963061efeaa3516f2d5e41b5dac1cfbd1e"
|
||||
},
|
||||
"manifest_sha256": "a7aaea19b9a30f8e278f5cb8d482d7b624d93f05c593998094d8dbffcab0f003",
|
||||
"parent_manifest_sha256": "74e0fe039acbed1b287bd11ee6b779becbb8f9b66c3c249d260ed66f5583f02a",
|
||||
"counts": {
|
||||
"train": 837,
|
||||
"validation": 70
|
||||
},
|
||||
"source_dtypes": [
|
||||
"<f8"
|
||||
],
|
||||
"best_by_family": [
|
||||
{
|
||||
"candidate": {
|
||||
"family": "pooled190",
|
||||
"kind": "rff",
|
||||
"dimensions": 128,
|
||||
"gamma_multiplier": 0.1,
|
||||
"alpha": 0.1
|
||||
},
|
||||
"minimum_cv_macro_f1": 0.33211530016861246,
|
||||
"mean_cv_macro_f1": 0.33402959890262357,
|
||||
"fold_f1": [
|
||||
0.33211530016861246,
|
||||
0.3359438976366347
|
||||
],
|
||||
"fold_class3_f1": [
|
||||
0.08,
|
||||
0.2
|
||||
],
|
||||
"fold_class3_confusion": [
|
||||
[
|
||||
0,
|
||||
0,
|
||||
1,
|
||||
2,
|
||||
6,
|
||||
2,
|
||||
9
|
||||
],
|
||||
[
|
||||
0,
|
||||
5,
|
||||
3,
|
||||
2,
|
||||
9,
|
||||
1,
|
||||
0
|
||||
]
|
||||
]
|
||||
},
|
||||
{
|
||||
"candidate": {
|
||||
"family": "raw950_matched",
|
||||
"kind": "rff",
|
||||
"dimensions": 128,
|
||||
"gamma_multiplier": 0.1,
|
||||
"alpha": 0.1
|
||||
},
|
||||
"minimum_cv_macro_f1": 0.3008616373852123,
|
||||
"mean_cv_macro_f1": 0.32198371114003,
|
||||
"fold_f1": [
|
||||
0.3008616373852123,
|
||||
0.34310578489484767
|
||||
],
|
||||
"fold_class3_f1": [
|
||||
0.09090909090909091,
|
||||
0.3076923076923077
|
||||
],
|
||||
"fold_class3_confusion": [
|
||||
[
|
||||
0,
|
||||
0,
|
||||
1,
|
||||
4,
|
||||
4,
|
||||
2,
|
||||
9
|
||||
],
|
||||
[
|
||||
0,
|
||||
1,
|
||||
10,
|
||||
1,
|
||||
8,
|
||||
0,
|
||||
0
|
||||
]
|
||||
]
|
||||
},
|
||||
{
|
||||
"candidate": {
|
||||
"family": "raw950_native",
|
||||
"kind": "rff",
|
||||
"dimensions": 128,
|
||||
"gamma_multiplier": 1,
|
||||
"alpha": 0.1
|
||||
},
|
||||
"minimum_cv_macro_f1": 0.2991058894120357,
|
||||
"mean_cv_macro_f1": 0.304702988594541,
|
||||
"fold_f1": [
|
||||
0.31030008777704626,
|
||||
0.2991058894120357
|
||||
],
|
||||
"fold_class3_f1": [
|
||||
0,
|
||||
0.11594202898550725
|
||||
],
|
||||
"fold_class3_confusion": [
|
||||
[
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
9,
|
||||
0,
|
||||
0,
|
||||
11
|
||||
],
|
||||
[
|
||||
0,
|
||||
0,
|
||||
4,
|
||||
8,
|
||||
6,
|
||||
1,
|
||||
1
|
||||
]
|
||||
]
|
||||
}
|
||||
],
|
||||
"scope": "MEASURED adaptive development research. Frozen 18 configurations selected on original training folds; only selected pooled control evaluated on reused development. Original test opened in first pilot, not reopened in this round. No native-family development accuracy or hardware claim.",
|
||||
"intake": {
|
||||
"test_payload_decoded": false,
|
||||
"excluded_payload_decoded": false,
|
||||
"pooled_exact": true,
|
||||
"source_dtype": "float64",
|
||||
"archive_sha256": "85f3df633c5b1a48ca4b1423467fce22821b21eeb3eed085a8d9f983dc156660",
|
||||
"decoded_recordings": 907,
|
||||
"intake_exit_code": 0,
|
||||
"intake_seconds": 31.518044787999997,
|
||||
"experiment_exit_code": 0,
|
||||
"experiment_process_seconds": 12.019457520999822
|
||||
},
|
||||
"validation": {
|
||||
"focused_tests": 8,
|
||||
"public_tests": 52,
|
||||
"original_tests": 28,
|
||||
"completionVerified": true,
|
||||
"exit_code": 0,
|
||||
"independent_source_review": "NO BLOCK",
|
||||
"final_test_reopened": false
|
||||
},
|
||||
"latency_scope": "1000 alternating batch1 pairs,20warmups,sharedx86CPU; both selected and incumbent models are the same pooled190 weights. Includes validation,feature extraction,classifier; excludes acquisition,pooling,IO. Small timing difference is wrapper overhead/noise, not an accuracy or fullpipeline speed gain."
|
||||
}
|
||||
@@ -0,0 +1,243 @@
|
||||
"""Frozen native-resolution ablation on development-only public amplitudes."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import time
|
||||
|
||||
import numpy as np
|
||||
from safetensors.numpy import load, save_file
|
||||
|
||||
from .espfi_kernel import fit, predict_features
|
||||
from .espfi_motion import MotionBundle, _read, _validate, motion_features
|
||||
from .espfi_train import digest, scores, write_json
|
||||
|
||||
WIDTHS = {'pooled190': 468, 'raw950_matched': 468, 'raw950_native': 468}
|
||||
LAGS = {'pooled190': ([1, 2, 4, 8, 16, 32], [1, 4, 16]),
|
||||
'raw950_matched': ([5, 10, 20, 40, 80, 160], [5, 20, 80]),
|
||||
'raw950_native': ([1, 2, 4, 8, 16, 32], [1, 4, 16])}
|
||||
GRID = [dict(family=family, kind='rff', dimensions=128, gamma_multiplier=gamma, alpha=alpha)
|
||||
for family in WIDTHS for gamma in (.1, 1.) for alpha in (.01, .1, 1.)]
|
||||
BASELINE = dict(family='motion_acf', kind='rff', dimensions=128, gamma_multiplier=.1, alpha=.1)
|
||||
CONTROL = dict(BASELINE, family='pooled190')
|
||||
CONTRACT = {
|
||||
'seed': 42, 'grid': GRID, 'input': {'pooled190': [52, 190], 'raw950_matched': [52, 950], 'raw950_native': [52, 950]},
|
||||
'selection': 'minimum training-fold macro F1, then mean, then declaration order',
|
||||
'lags': {name: {'acf': acf, 'difference': diff} for name, (acf, diff) in LAGS.items()},
|
||||
'log_floor': 1e-6, 'rms_floor': 1e-6, 'energy_floor': 'T * 1e-12',
|
||||
'raw_transform_batch_size': 32,
|
||||
'units': 'Original sample-index spans only; no measured sampling rate or physical time inferred',
|
||||
'retention_gate': {'development_macro_f1_absolute_gain': .03,
|
||||
'accuracy_nonregression': True,
|
||||
'protect_each_previously_nonzero_class': True,
|
||||
'paired_p95_limit_ms': 1.0},
|
||||
'promotionAuthorized': False,
|
||||
}
|
||||
|
||||
|
||||
def resolution_features(values, family):
|
||||
if family not in WIDTHS:
|
||||
raise ValueError('Unknown resolution feature family')
|
||||
x = np.asarray(values)
|
||||
shape = tuple(CONTRACT['input'][family])
|
||||
if (x.dtype.kind not in 'fiu' or x.ndim != 3 or x.shape[1:] != shape
|
||||
or not 0 < len(x) <= 4096 or not np.isfinite(x).all() or (x < 0).any()):
|
||||
raise ValueError('Invalid finite nonnegative resolution input shape')
|
||||
if family == 'pooled190':
|
||||
return motion_features(x, 'motion_acf')
|
||||
# All reductions are within one recording; bounded chunks preserve arithmetic.
|
||||
return np.ascontiguousarray(np.concatenate([_raw_features(x[start:start+32], family)
|
||||
for start in range(0, len(x), 32)]))
|
||||
|
||||
|
||||
def _raw_features(values, family):
|
||||
x = values.astype(np.float64, copy=False)
|
||||
logged = np.log(np.maximum(x, 1e-6))
|
||||
centered = logged - logged.mean(-1, keepdims=True)
|
||||
scale = np.maximum(np.sqrt(np.mean(centered**2, axis=(1, 2), keepdims=True)), 1e-6)
|
||||
relative = centered / scale
|
||||
energy = np.maximum(np.sum(centered**2, axis=-1), x.shape[-1] * 1e-12)
|
||||
acf_lags, diff_lags = LAGS[family]
|
||||
acf = [np.sum(centered[..., lag:] * centered[..., :-lag], axis=-1) / energy for lag in acf_lags]
|
||||
changes = [np.mean(np.abs(relative[..., lag:] - relative[..., :-lag]), axis=-1) for lag in diff_lags]
|
||||
result = np.ascontiguousarray(np.stack(acf + changes, axis=-1).reshape(len(x), 468))
|
||||
if not np.isfinite(result).all():
|
||||
raise ValueError('Nonfinite resolution features')
|
||||
return result
|
||||
|
||||
|
||||
def model_input(part, family):
|
||||
return part['pooled'] if family == 'pooled190' else part['x']
|
||||
|
||||
|
||||
def validate_parts(parts):
|
||||
fold_ids = _validate(parts)
|
||||
for part in parts.values():
|
||||
n = len(part['y'])
|
||||
if not 0 < n <= 4096:
|
||||
raise ValueError('Resolution partition exceeds recording bound')
|
||||
for name, shape, dtype in [('x', (n, 52, 950), np.float64), ('pooled', (n, 52, 190), np.float32)]:
|
||||
value = np.asarray(part[name])
|
||||
if (value.shape != shape or value.dtype != dtype or not np.isfinite(value).all() or (value < 0).any()):
|
||||
raise ValueError('Invalid resolution partition precision or shape')
|
||||
return fold_ids
|
||||
|
||||
|
||||
class ResolutionBundle:
|
||||
def __init__(self, directory):
|
||||
root = Path(directory)
|
||||
status = json.loads(_read(root / 'status.json', 65536))
|
||||
if not isinstance(status, dict) or status.get('status') != 'complete':
|
||||
raise ValueError('Invalid completed resolution artifact')
|
||||
report = json.loads(_read(root / 'metrics.json', 16 * 1024**2))
|
||||
if (not isinstance(status, dict) or not isinstance(report, dict)
|
||||
or status.get('status') != 'complete' or report.get('status') != 'complete'
|
||||
or status.get('promotionAuthorized') is not False
|
||||
or report.get('promotionAuthorized') is not False
|
||||
or report.get('contract') != CONTRACT
|
||||
or report.get('task') != 'public_resolution_ablation'):
|
||||
raise ValueError('Invalid completed resolution artifact')
|
||||
self.candidate = report['selected']['candidate']
|
||||
if self.candidate not in GRID:
|
||||
raise ValueError('Unknown resolution candidate')
|
||||
data = _read(root / 'model.safetensors', 1024**2)
|
||||
if hashlib.sha256(data).hexdigest() != report.get('model_sha256'):
|
||||
raise ValueError('Model hash mismatch')
|
||||
self.state = load(data)
|
||||
width = WIDTHS[self.candidate['family']]
|
||||
shapes = {'feature_mean': (width,), 'feature_scale': (width,),
|
||||
'projection': (width, 128), 'phase': (128,), 'output_mean': (128,),
|
||||
'weights': (128, 7), 'bias': (7,)}
|
||||
if (set(self.state) != set(shapes) or any(self.state[k].shape != shape
|
||||
or self.state[k].dtype != np.float64 or not np.isfinite(self.state[k]).all()
|
||||
for k, shape in shapes.items()) or not (self.state['feature_scale'] > 0).all()):
|
||||
raise ValueError('Invalid resolution model state')
|
||||
|
||||
def predict(self, values):
|
||||
return predict_features(self.state, resolution_features(values, self.candidate['family']))
|
||||
|
||||
|
||||
def search(parts, provenance, baseline_directory, output, max_seconds=600):
|
||||
if not 1 <= max_seconds <= 3600:
|
||||
raise ValueError('Invalid time budget')
|
||||
fold_ids = validate_parts(parts)
|
||||
baseline = MotionBundle(baseline_directory)
|
||||
prior_bytes = _read(Path(baseline_directory) / 'metrics.json', 16 * 1024**2)
|
||||
prior = json.loads(prior_bytes)
|
||||
if (baseline.candidate != BASELINE or not provenance.get('manifest_sha256') or provenance.get('pooled_exact') is not True
|
||||
or provenance.get('test_payload_decoded') is not False
|
||||
or prior.get('provenance', {}).get('manifest_sha256') != provenance.get('parent_manifest_sha256')):
|
||||
raise ValueError('Baseline model or data provenance mismatch')
|
||||
root = Path(output)
|
||||
root.mkdir(exist_ok=False, parents=True)
|
||||
write_json(root / 'status.json', {'status': 'running', 'promotionAuthorized': False})
|
||||
write_json(root / 'search_contract.json', CONTRACT)
|
||||
start = time.monotonic()
|
||||
try:
|
||||
train, dev = parts['train'], parts['validation']
|
||||
cache = {family: resolution_features(model_input(train, family), family) for family in WIDTHS}
|
||||
results, selected, best = [], None, None
|
||||
for candidate in GRID:
|
||||
evaluations = []
|
||||
for fit_ids, eval_ids in fold_ids:
|
||||
if time.monotonic() - start > max_seconds:
|
||||
raise TimeoutError('Resolution search budget exceeded')
|
||||
state = fit(cache[candidate['family']][fit_ids], train['y'][fit_ids], candidate)
|
||||
result = predict_features(state, cache[candidate['family']][eval_ids])
|
||||
evaluations.append(scores(train['y'][eval_ids], result.argmax(1), 7))
|
||||
row = {'candidate': candidate, 'folds': evaluations,
|
||||
'minimum_cv_macro_f1': min(v['macro_f1'] for v in evaluations),
|
||||
'mean_cv_macro_f1': float(np.mean([v['macro_f1'] for v in evaluations])),
|
||||
'scalar_count': sum(v.size for v in state.values())}
|
||||
results.append(row)
|
||||
rank = (row['minimum_cv_macro_f1'], row['mean_cv_macro_f1'])
|
||||
if best is None or rank > best:
|
||||
best, selected = rank, row
|
||||
# Persist the complete train-only choice before any development feature extraction.
|
||||
write_json(root / 'selection.json', {'selected': selected, 'candidates': results, 'contract': CONTRACT})
|
||||
chosen = selected['candidate']
|
||||
state = fit(cache[chosen['family']], train['y'], chosen)
|
||||
# Require the unchanged control to reproduce the exact existing learned state.
|
||||
control = fit(cache['pooled190'], train['y'], BASELINE)
|
||||
if any(not np.array_equal(control[k], baseline.state[k]) for k in control):
|
||||
raise ValueError('Unchanged baseline state did not reproduce')
|
||||
save_file(state, str(root / 'model.safetensors'))
|
||||
actual = predict_features(state, resolution_features(model_input(dev, chosen['family']), chosen['family']))
|
||||
incumbent = scores(dev['y'], baseline.predict(dev['pooled']).argmax(1), 7)
|
||||
measured = scores(dev['y'], actual.argmax(1), 7)
|
||||
replay = load(_read(root / 'model.safetensors', 1024**2))
|
||||
if not np.array_equal(actual, predict_features(replay, resolution_features(model_input(dev, chosen['family']), chosen['family']))):
|
||||
raise ValueError('Reload changes logits')
|
||||
def candidate_predict(x):
|
||||
return predict_features(state, resolution_features(x, chosen['family']))
|
||||
for i in range(20):
|
||||
j = i % len(dev['y'])
|
||||
candidate_predict(model_input(dev, chosen['family'])[j:j+1])
|
||||
baseline.predict(dev['pooled'][j:j+1])
|
||||
timings = {'candidate': [], 'incumbent': []}
|
||||
for i in range(1000):
|
||||
if time.monotonic() - start > max_seconds:
|
||||
raise TimeoutError('Resolution search budget exceeded')
|
||||
j = i % len(dev['y'])
|
||||
calls = [('candidate', candidate_predict, model_input(dev, chosen['family'])[j:j+1]),
|
||||
('incumbent', baseline.predict, dev['pooled'][j:j+1])]
|
||||
if i % 2:
|
||||
calls.reverse()
|
||||
for name, fn, sample in calls:
|
||||
before = time.perf_counter_ns()
|
||||
fn(sample)
|
||||
timings[name].append((time.perf_counter_ns() - before) / 1e6)
|
||||
timing = {k: {'p50_ms': float(np.percentile(v, 50)), 'p95_ms': float(np.percentile(v, 95))}
|
||||
for k, v in timings.items()}
|
||||
gate = CONTRACT['retention_gate']
|
||||
checks = {
|
||||
'macro_f1_gain': measured['macro_f1'] >= incumbent['macro_f1'] + gate['development_macro_f1_absolute_gain'],
|
||||
'accuracy_nonregression': measured['accuracy'] >= incumbent['accuracy'],
|
||||
'protect_each_previously_nonzero_class': all(b == 0 or a > 0 for a, b in
|
||||
zip(measured['per_class_f1'], incumbent['per_class_f1'])),
|
||||
'p95_under_1ms': timing['candidate']['p95_ms'] < gate['paired_p95_limit_ms'],
|
||||
}
|
||||
report = {
|
||||
'status': 'complete', 'promotionAuthorized': False, 'task': 'public_resolution_ablation',
|
||||
'contract': CONTRACT, 'selected': selected, 'candidates': results,
|
||||
'baseline_control': next(r for r in results if r['candidate'] == CONTROL),
|
||||
'baseline_state_exact': True, 'reload_max_abs_logit_error': 0.0,
|
||||
'evidence': 'SYNTHETIC' if provenance.get('evidence') == 'SYNTHETIC' else 'MEASURED',
|
||||
'provenance': provenance, 'development': measured, 'incumbent_development': incumbent,
|
||||
'retention_checks': checks, 'retained_as_research_candidate': all(checks.values()),
|
||||
'latency': timing, 'model_bytes': (root / 'model.safetensors').stat().st_size,
|
||||
'model_sha256': digest(root / 'model.safetensors'),
|
||||
'baseline_model_sha256': prior['model_sha256'],
|
||||
'baseline_receipt_sha256': hashlib.sha256(prior_bytes).hexdigest(),
|
||||
'source_sha256': {name: digest(Path(__file__).with_name(name)) for name in
|
||||
('espfi_resolution.py', 'espfi_resolution_data.py', 'espfi_motion.py', 'espfi_generalize.py', 'espfi_kernel.py', 'espfi_train.py')},
|
||||
'elapsed_seconds': time.monotonic() - start,
|
||||
'scope': '18 configurations selected by original training folds; adaptive research on reused development data. '
|
||||
'1000 paired batch1 calls include feature extraction/checks/classifier, exclude acquisition/pooling/IO. '
|
||||
'Respective input arrays: selected family190/950, baseline190; no acquisition/pooling timing or end-to-end speed claim. No physical-time, final-test or hardware accuracy claim.',
|
||||
}
|
||||
if time.monotonic() - start > max_seconds:
|
||||
raise TimeoutError('Resolution search budget exceeded')
|
||||
write_json(root / 'metrics.json', report)
|
||||
write_json(root / 'status.json', {'status': 'complete', 'promotionAuthorized': False})
|
||||
return report
|
||||
except Exception:
|
||||
write_json(root / 'status.json', {'status': 'failed', 'promotionAuthorized': False})
|
||||
raise
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
for name in ('manifest', 'baseline', 'output'):
|
||||
parser.add_argument('--' + name, required=True)
|
||||
parser.add_argument('--max-seconds', type=int, default=600)
|
||||
args = parser.parse_args()
|
||||
from .espfi_resolution_data import load_resolution
|
||||
parts, provenance = load_resolution(args.manifest)
|
||||
result = search(parts, provenance, args.baseline, args.output, args.max_seconds)
|
||||
print(json.dumps({k: result[k] for k in ('selected', 'development', 'incumbent_development',
|
||||
'retention_checks', 'retained_as_research_candidate', 'latency', 'model_bytes')}, indent=2))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,254 @@
|
||||
"""Pinned native-resolution intake; preserve the original development population.
|
||||
|
||||
Only previously admitted train/validation MAT payloads are decoded. The original
|
||||
manifest retains quarantine authority. No final test or excluded capture is
|
||||
reassigned. Raw arrays and manifests remain outside Git.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import argparse
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
from pathlib import Path
|
||||
import zipfile
|
||||
|
||||
import numpy as np
|
||||
from . import espfi
|
||||
from .espfi_motion import _read
|
||||
|
||||
SPLITS = ('train', 'validation')
|
||||
DTYPES = {np.dtype(t).str for t in ('float32', 'float64', 'int8', 'uint8',
|
||||
'int16', 'uint16', 'int32', 'uint32', 'int64', 'uint64')}
|
||||
|
||||
|
||||
def sha(data):
|
||||
return hashlib.sha256(data).hexdigest()
|
||||
|
||||
|
||||
def _parent_records(meta):
|
||||
if (meta.get('schema_version') != 1 or meta.get('classes') != espfi.CLASSES
|
||||
or meta.get('source_revision') != espfi.REVISION
|
||||
or meta.get('archive_sha256') != espfi.ARCHIVE_SHA256
|
||||
or meta.get('source_url') != espfi.SOURCE_URL
|
||||
or meta.get('license') != 'CC-BY-4.0'
|
||||
or meta.get('split_policy') != 'joint_room_subject_intersection_v1'
|
||||
or meta.get('temporal_mean_pool') != 5 or meta.get('window_samples') != 190):
|
||||
raise ValueError('Invalid pooled parent provenance')
|
||||
records = meta.get('records', [])
|
||||
expected = {(r, p, a, t) for r in range(1, 5) for p in range(1, 9)
|
||||
for a in range(1, 8) for t in range(1, 11)}
|
||||
lookup = {tuple(r['identity']): r for r in records}
|
||||
if len(records) != 2240 or set(lookup) != expected:
|
||||
raise ValueError('Incomplete parent population')
|
||||
counts = {}
|
||||
for record in records:
|
||||
for key in ('signal_sha256', 'sha256'):
|
||||
if not isinstance(record.get(key), str) or not espfi.re.fullmatch('[0-9a-f]{64}', record[key]):
|
||||
raise ValueError('Invalid parent digest')
|
||||
h = record['signal_sha256']
|
||||
counts[h] = counts.get(h, 0) + 1
|
||||
for ident, record in lookup.items():
|
||||
if (espfi.parse_identity(record['path']) != ident
|
||||
or record['split'] != (None if counts[record['signal_sha256']] > 1
|
||||
else espfi.split_identity(ident))):
|
||||
raise ValueError('Invalid parent quarantine or split')
|
||||
if meta.get('counts') != {s: sum(r['split'] == s for r in records)
|
||||
for s in (*SPLITS, 'test')}:
|
||||
raise ValueError('Invalid parent counts')
|
||||
return lookup
|
||||
|
||||
|
||||
def pool_native(raw, source_dtype):
|
||||
"""Reproduce original MAT dtype, orientation, averaging and final cast."""
|
||||
if source_dtype not in DTYPES or raw.shape != (52, 950):
|
||||
raise ValueError('Unsupported native schema')
|
||||
source = raw.T.astype(np.dtype(source_dtype))
|
||||
return np.ascontiguousarray(source.reshape(190, 5, 52).mean(1).T, dtype=np.float32)
|
||||
|
||||
|
||||
def decode_native(content):
|
||||
from scipy.io import loadmat
|
||||
if not 0 < len(content) <= espfi.MAX_MEMBER:
|
||||
raise ValueError('Invalid MAT size')
|
||||
x = loadmat(io.BytesIO(content), variable_names=['CSIamp']).get('CSIamp')
|
||||
if (x is None or x.dtype.str not in DTYPES or x.shape not in ((950, 52), (52, 950))
|
||||
or not np.isfinite(x).all() or (x < 0).any() or (x > 1e12).any()):
|
||||
raise ValueError('Invalid native amplitude')
|
||||
dtype = x.dtype.str
|
||||
if x.shape == (950, 52):
|
||||
x = x.T
|
||||
raw = np.ascontiguousarray(x, dtype=np.float64)
|
||||
# Require lossless conversion, including supported integer source types.
|
||||
if not np.array_equal(raw.astype(x.dtype), x):
|
||||
raise ValueError('Native conversion loses information')
|
||||
pooled = espfi.decode_amplitude(content, pool=5)
|
||||
if not np.array_equal(pool_native(raw, dtype), pooled):
|
||||
raise ValueError('Original pooling does not reproduce exactly')
|
||||
return raw, pooled, dtype
|
||||
|
||||
|
||||
def prepare(archive, parent_manifest, output):
|
||||
import libarchive
|
||||
archive, output = Path(archive), Path(output)
|
||||
if (archive.is_symlink() or not archive.is_file()
|
||||
or archive.stat().st_size != espfi.ARCHIVE_BYTES
|
||||
or espfi.digest(archive) != espfi.ARCHIVE_SHA256):
|
||||
raise ValueError('Archive differs from pinned source')
|
||||
parent_bytes = _read(parent_manifest, 3_000_000)
|
||||
parent = json.loads(parent_bytes)
|
||||
lookup = _parent_records(parent)
|
||||
prior, checked = espfi.load_development(parent_manifest)
|
||||
if checked['manifest_sha256'] != sha(parent_bytes):
|
||||
raise ValueError('Parent changed during intake')
|
||||
admitted = {i: r for i, r in lookup.items() if r['split'] in SPLITS}
|
||||
expected_pooled = {tuple(i): x for s in SPLITS
|
||||
for i, x in zip(prior[s]['identity'], prior[s]['x'])}
|
||||
output.mkdir(parents=True, exist_ok=False)
|
||||
def status(value):
|
||||
(output / 'status.json').write_text(json.dumps({'status': value, 'promotionAuthorized': False}))
|
||||
status('running')
|
||||
try:
|
||||
seen, found, rows, records = set(), set(), {s: [] for s in SPLITS}, []
|
||||
with libarchive.file_reader(str(archive)) as entries:
|
||||
for entry in entries:
|
||||
if not entry.pathname.endswith('.mat'):
|
||||
continue
|
||||
ident = espfi.parse_identity(entry.pathname)
|
||||
if (ident in seen or not entry.isfile or not 0 < entry.size <= espfi.MAX_MEMBER
|
||||
or entry.pathname != lookup[ident]['path']):
|
||||
raise ValueError('Unexpected source member')
|
||||
seen.add(ident)
|
||||
if ident not in admitted:
|
||||
continue # No get_blocks/loadmat for test or excluded records.
|
||||
content = bytearray()
|
||||
for block in entry.get_blocks():
|
||||
content.extend(block)
|
||||
if len(content) > espfi.MAX_MEMBER:
|
||||
raise ValueError('Expanded member exceeds bound')
|
||||
record = admitted[ident]
|
||||
if sha(content) != record['sha256']:
|
||||
raise ValueError('Source MAT digest mismatch')
|
||||
raw, pooled, dtype = decode_native(content)
|
||||
if (sha(pooled.tobytes()) != record['signal_sha256']
|
||||
or not np.array_equal(pooled, expected_pooled[ident])):
|
||||
raise ValueError('Pooled control differs from parent')
|
||||
found.add(ident)
|
||||
rows[record['split']].append((ident, raw, pooled))
|
||||
records.append({'identity': list(ident), 'split': record['split'],
|
||||
'source_dtype': dtype, 'source_sha256': record['sha256'],
|
||||
'raw_sha256': sha(raw.tobytes()),
|
||||
'pooled_sha256': record['signal_sha256']})
|
||||
if set(lookup) != seen or set(admitted) != found:
|
||||
raise ValueError('Incomplete source population')
|
||||
arrays = {}
|
||||
for split in SPLITS:
|
||||
samples = sorted(rows[split], key=lambda row: row[0])
|
||||
arrays[split + '_x'] = np.stack([x for _, x, _ in samples])
|
||||
arrays[split + '_pooled'] = np.stack([x for _, _, x in samples])
|
||||
arrays[split + '_identity'] = np.array([i for i, _, _ in samples], dtype=np.int64)
|
||||
arrays[split + '_y'] = arrays[split + '_identity'][:, 2] - 1
|
||||
target = output / 'amplitudes.npz'
|
||||
np.savez_compressed(target, **arrays)
|
||||
(output / 'parent_manifest.json').write_bytes(parent_bytes)
|
||||
meta = {'schema_version': 1, 'task': 'espfi_native_resolution',
|
||||
'source_revision': espfi.REVISION, 'archive_sha256': espfi.ARCHIVE_SHA256,
|
||||
'classes': espfi.CLASSES, 'evidence': 'real', 'license': 'CC-BY-4.0',
|
||||
'parent_manifest_sha256': sha(parent_bytes), 'pooled_exact': True,
|
||||
'raw_dtype': 'float64', 'test_payload_decoded': False,
|
||||
'npz': target.name, 'npz_sha256': espfi.digest(target),
|
||||
'counts': {s: len(rows[s]) for s in SPLITS},
|
||||
'records': sorted(records, key=lambda row: row['identity'])}
|
||||
(output / 'manifest.json').write_text(json.dumps(meta, indent=2) + '\n')
|
||||
status('complete')
|
||||
return output / 'manifest.json'
|
||||
except Exception:
|
||||
status('failed')
|
||||
raise
|
||||
|
||||
|
||||
def load_resolution(manifest):
|
||||
path = Path(manifest)
|
||||
data = _read(path, 3_000_000)
|
||||
meta = json.loads(data)
|
||||
lifecycle = json.loads(_read(path.parent / 'status.json', 65536))
|
||||
if (lifecycle != {'status': 'complete', 'promotionAuthorized': False}
|
||||
or meta.get('schema_version') != 1 or meta.get('task') != 'espfi_native_resolution'
|
||||
or meta.get('source_revision') != espfi.REVISION
|
||||
or meta.get('archive_sha256') != espfi.ARCHIVE_SHA256
|
||||
or meta.get('classes') != espfi.CLASSES or meta.get('license') != 'CC-BY-4.0'
|
||||
or meta.get('evidence') != 'real' or meta.get('raw_dtype') != 'float64'
|
||||
or meta.get('pooled_exact') is not True or meta.get('test_payload_decoded') is not False
|
||||
or meta.get('npz') != 'amplitudes.npz'):
|
||||
raise ValueError('Invalid completed resolution intake')
|
||||
parent = _read(path.parent / 'parent_manifest.json', 3_000_000)
|
||||
if sha(parent) != meta.get('parent_manifest_sha256'):
|
||||
raise ValueError('Parent manifest digest mismatch')
|
||||
lookup = _parent_records(json.loads(parent))
|
||||
admitted = {i: r for i, r in lookup.items() if r['split'] in SPLITS}
|
||||
records = meta.get('records', [])
|
||||
rows = {tuple(r['identity']): r for r in records}
|
||||
if len(records) != len(admitted) or set(rows) != set(admitted):
|
||||
raise ValueError('Resolution population differs from parent')
|
||||
counts = {s: sum(r['split'] == s for r in admitted.values()) for s in SPLITS}
|
||||
if meta.get('counts') != counts:
|
||||
raise ValueError('Resolution counts differ')
|
||||
for ident, r in rows.items():
|
||||
original = admitted[ident]
|
||||
if (r['split'] != original['split'] or r['source_sha256'] != original['sha256']
|
||||
or r['pooled_sha256'] != original['signal_sha256'] or r['source_dtype'] not in DTYPES):
|
||||
raise ValueError('Resolution record binding mismatch')
|
||||
payload = _read(path.parent / 'amplitudes.npz', espfi.MAX_NPZ)
|
||||
if sha(payload) != meta.get('npz_sha256'):
|
||||
raise ValueError('Resolution bundle digest mismatch')
|
||||
with zipfile.ZipFile(io.BytesIO(payload)) as z:
|
||||
expected = {s + '_' + k + '.npy': (shape, dtype) for s in SPLITS for k, shape, dtype in
|
||||
[('x', (counts[s], 52, 950), np.dtype('float64')),
|
||||
('pooled', (counts[s], 52, 190), np.dtype('float32')),
|
||||
('y', (counts[s],), np.dtype('int64')),
|
||||
('identity', (counts[s], 4), np.dtype('int64'))]}
|
||||
infos = z.infolist()
|
||||
if (len(infos) != 8 or {i.filename for i in infos} != set(expected)
|
||||
or sum(i.file_size for i in infos) > espfi.MAX_NPZ):
|
||||
raise ValueError('Unexpected resolution array population')
|
||||
for info in infos:
|
||||
with z.open(info) as f:
|
||||
version = np.lib.format.read_magic(f)
|
||||
if version not in ((1, 0), (2, 0)):
|
||||
raise ValueError('Unsupported numpy version')
|
||||
reader = (np.lib.format.read_array_header_1_0 if version == (1, 0)
|
||||
else np.lib.format.read_array_header_2_0)
|
||||
shape, fortran, dtype = reader(f)
|
||||
if ((shape, dtype) != expected[info.filename] or fortran
|
||||
or f.tell() + int(np.prod(shape)) * dtype.itemsize != info.file_size):
|
||||
raise ValueError('Unexpected resolution array header')
|
||||
parts, seen = {}, set()
|
||||
with np.load(io.BytesIO(payload), allow_pickle=False) as npz:
|
||||
for split in SPLITS:
|
||||
part = {key: npz[split + '_' + key] for key in ('x', 'pooled', 'y', 'identity')}
|
||||
for raw, pooled, label, identity in zip(part['x'], part['pooled'], part['y'], part['identity']):
|
||||
ident = tuple(map(int, identity))
|
||||
r = rows.get(ident)
|
||||
if (r is None or ident in seen or r['split'] != split or label != ident[2] - 1
|
||||
or not np.isfinite(raw).all() or (raw < 0).any() or (raw > 1e12).any()
|
||||
or sha(raw.tobytes()) != r['raw_sha256']
|
||||
or sha(pooled.tobytes()) != r['pooled_sha256']
|
||||
or not np.array_equal(pool_native(raw, r['source_dtype']), pooled)):
|
||||
raise ValueError('Resolution array provenance mismatch')
|
||||
seen.add(ident)
|
||||
parts[split] = part
|
||||
if seen != set(admitted):
|
||||
raise ValueError('Incomplete loaded resolution population')
|
||||
meta['manifest_sha256'] = sha(data)
|
||||
return parts, meta
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
for arg in ('archive', 'parent-manifest', 'output'):
|
||||
parser.add_argument('--' + arg, required=True)
|
||||
args = parser.parse_args()
|
||||
print(prepare(args.archive, args.parent_manifest, args.output))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,126 @@
|
||||
"""Synthetic contracts for frozen resolution ablation; not sensing evidence."""
|
||||
import copy
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
import numpy as np
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
from csi_training.espfi_resolution import (BASELINE, CONTROL, GRID, CONTRACT, ResolutionBundle,
|
||||
resolution_features, model_input, search)
|
||||
from csi_training.espfi_motion import motion_features
|
||||
from test_espfi_context import make_baseline
|
||||
from test_espfi_generalize import fixture as pooled_fixture
|
||||
|
||||
|
||||
def fixture():
|
||||
parts, prior = pooled_fixture()
|
||||
for part in parts.values():
|
||||
part['pooled'] = part['x']
|
||||
part['x'] = np.repeat(part['pooled'].astype(np.float64), 5, axis=-1)
|
||||
provenance = dict(prior, parent_manifest_sha256=prior['manifest_sha256'],
|
||||
manifest_sha256='synthetic_resolution', pooled_exact=True,
|
||||
raw_dtype='float64', test_payload_decoded=False)
|
||||
return parts, provenance, prior
|
||||
|
||||
|
||||
def baseline(parts, prior, path):
|
||||
pooled = {k: dict(v, x=v['pooled']) for k, v in parts.items()}
|
||||
make_baseline(pooled, prior, path)
|
||||
|
||||
|
||||
class ResolutionTests(unittest.TestCase):
|
||||
def test_control_parity_raw_reversal_gain_and_shape(self):
|
||||
rng = np.random.default_rng(29)
|
||||
pooled = np.exp(rng.normal(size=(2, 52, 190))).astype(np.float32)
|
||||
np.testing.assert_array_equal(resolution_features(pooled, 'pooled190'), motion_features(pooled, 'motion_acf'))
|
||||
raw = np.exp(rng.normal(size=(2, 52, 950)))
|
||||
for family in ('raw950_matched', 'raw950_native'):
|
||||
actual = resolution_features(raw, family)
|
||||
self.assertEqual(actual.shape, (2, 468))
|
||||
np.testing.assert_allclose(actual, resolution_features(raw[..., ::-1], family), atol=1e-12)
|
||||
np.testing.assert_allclose(actual, resolution_features(raw*2, family), atol=1e-12)
|
||||
np.testing.assert_array_equal(resolution_features(np.ones_like(raw), family), np.zeros_like(actual))
|
||||
with self.assertRaises(ValueError):
|
||||
resolution_features(pooled, family)
|
||||
self.assertGreater(np.max(np.abs(resolution_features(raw, 'raw950_matched') - resolution_features(raw, 'raw950_native'))), .001)
|
||||
for invalid in (np.ones((1, 52, 949)), np.ones((0, 52, 950)),
|
||||
np.full((1, 52, 950), np.inf), -np.ones((1, 52, 950))):
|
||||
with self.assertRaises(ValueError):
|
||||
resolution_features(invalid, 'raw950_native')
|
||||
with self.assertRaises(ValueError):
|
||||
resolution_features(raw, 'pooled190')
|
||||
|
||||
def test_raw_chunk_bound_preserves_per_record_arithmetic(self):
|
||||
x = np.exp(np.random.default_rng(81).normal(size=(35, 52, 950)))
|
||||
for family in ('raw950_matched', 'raw950_native'):
|
||||
actual = resolution_features(x, family)
|
||||
expected = np.concatenate([resolution_features(x[i:i+1], family) for i in range(len(x))])
|
||||
np.testing.assert_array_equal(actual, expected)
|
||||
|
||||
def test_search_control_isolation_reload_tamper(self):
|
||||
parts, provenance, prior = fixture()
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
baseline(parts, prior, root/'baseline')
|
||||
original_features = resolution_features
|
||||
def guarded(values, family):
|
||||
if len(values) == len(parts['validation']['y']):
|
||||
self.assertTrue((root/'first'/'selection.json').is_file())
|
||||
return original_features(values, family)
|
||||
with patch('csi_training.espfi_resolution.resolution_features', side_effect=guarded):
|
||||
result = search(parts, provenance, root/'baseline', root/'first')
|
||||
changed = copy.deepcopy(parts)
|
||||
changed['validation']['x'] *= 1.3
|
||||
changed['validation']['y'] = (changed['validation']['y']+1)%7
|
||||
changed['validation']['identity'][:, 2] = changed['validation']['y']+1
|
||||
other = search(changed, provenance, root/'baseline', root/'second')
|
||||
self.assertEqual(len(GRID), 18)
|
||||
self.assertEqual(result['selected'], other['selected'])
|
||||
self.assertEqual(result['model_sha256'], other['model_sha256'])
|
||||
self.assertTrue(result['baseline_state_exact'])
|
||||
self.assertEqual(result['baseline_control']['candidate'], CONTROL)
|
||||
winner = max(result['candidates'], key=lambda r:(r['minimum_cv_macro_f1'],r['mean_cv_macro_f1']))
|
||||
self.assertEqual(result['selected'],winner)
|
||||
self.assertEqual(result['contract'], CONTRACT)
|
||||
self.assertFalse(result['promotionAuthorized'])
|
||||
bundle = ResolutionBundle(root/'first')
|
||||
self.assertEqual(bundle.predict(model_input(parts['validation'], bundle.candidate['family'])).shape,(7,7))
|
||||
selected = json.loads((root/'first'/'selection.json').read_text())
|
||||
self.assertNotIn('development',selected)
|
||||
with (root/'first'/'model.safetensors').open('ab') as handle:
|
||||
handle.write(b'tampered')
|
||||
with self.assertRaisesRegex(ValueError,'hash mismatch'):
|
||||
ResolutionBundle(root/'first')
|
||||
|
||||
def test_failed_lifecycle_precision_provenance(self):
|
||||
parts, provenance, prior = fixture()
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
baseline(parts, prior, root/'baseline')
|
||||
invalid = copy.deepcopy(parts)
|
||||
invalid['train']['x'] = invalid['train']['x'].astype(np.float32)
|
||||
with self.assertRaises(ValueError):
|
||||
search(invalid,provenance,root/'baseline',root/'precision')
|
||||
with self.assertRaises(ValueError):
|
||||
search(dict(parts,test=parts['validation']),provenance,root/'baseline',root/'test')
|
||||
with self.assertRaises(ValueError):
|
||||
search(parts,dict(provenance,parent_manifest_sha256='wrong'),root/'baseline',root/'provenance')
|
||||
mismatched = copy.deepcopy(parts)
|
||||
mismatched['train']['pooled'][:, :, 10] *= 2
|
||||
with self.assertRaisesRegex(ValueError, 'did not reproduce'):
|
||||
search(mismatched,provenance,root/'baseline',root/'bad_control')
|
||||
self.assertEqual(json.loads((root/'bad_control'/'status.json').read_text())['status'],'failed')
|
||||
with patch('csi_training.espfi_resolution.fit',side_effect=RuntimeError('injected')):
|
||||
with self.assertRaises(RuntimeError):
|
||||
search(parts,provenance,root/'baseline',root/'failed')
|
||||
self.assertEqual(json.loads((root/'failed'/'status.json').read_text())['status'],'failed')
|
||||
with self.assertRaises(ValueError):
|
||||
ResolutionBundle(root/'failed')
|
||||
with self.assertRaises(FileExistsError):
|
||||
search(parts,provenance,root/'baseline',root/'failed')
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,143 @@
|
||||
"""Synthetic native intake tests; no captured signal fixture is committed."""
|
||||
import contextlib
|
||||
import io
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
import numpy as np
|
||||
from scipy.io import savemat
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
||||
from csi_training import espfi
|
||||
from csi_training import espfi_resolution_data as native
|
||||
|
||||
|
||||
class ResolutionIntakeTests(unittest.TestCase):
|
||||
@staticmethod
|
||||
def mat(values):
|
||||
out = io.BytesIO()
|
||||
savemat(out, {'CSIamp': values})
|
||||
return out.getvalue()
|
||||
|
||||
def test_source_precision_and_orientation(self):
|
||||
rng = np.random.default_rng(73)
|
||||
for dtype in (np.float32, np.float64, np.int32):
|
||||
x = (rng.random((950, 52)) * 1000).astype(dtype)
|
||||
for values in (x, x.T):
|
||||
content = self.mat(values)
|
||||
raw, pooled, source_dtype = native.decode_native(content)
|
||||
self.assertEqual(raw.dtype, np.float64)
|
||||
np.testing.assert_array_equal(raw, x.T)
|
||||
np.testing.assert_array_equal(pooled, espfi.decode_amplitude(content))
|
||||
np.testing.assert_array_equal(native.pool_native(raw, source_dtype), pooled)
|
||||
# Premature float32 rounding is detectably different on sensitive input.
|
||||
x = 1 + rng.uniform(0, 3e-7, size=(950, 52))
|
||||
raw, pooled, _ = native.decode_native(self.mat(x))
|
||||
self.assertFalse(np.array_equal(native.pool_native(raw, np.dtype('float32').str), pooled))
|
||||
|
||||
def test_native_rejects_bad_values(self):
|
||||
for x in (np.full((950, 52), np.nan), np.full((950, 52), np.inf),
|
||||
-np.ones((950, 52)), np.ones((950, 52)) * 1e13,
|
||||
np.ones((950, 52), dtype=complex), np.ones((10, 52))):
|
||||
with self.assertRaises(ValueError):
|
||||
native.decode_native(self.mat(x))
|
||||
with self.assertRaises(ValueError):
|
||||
native.decode_native(b'x' * (espfi.MAX_MEMBER + 1))
|
||||
|
||||
def parent(self):
|
||||
records = []
|
||||
for r in range(1, 5):
|
||||
for p in range(1, 9):
|
||||
for a in range(1, 8):
|
||||
for t in range(1, 11):
|
||||
i = (r, p, a, t)
|
||||
h = native.sha(str(i).encode())
|
||||
records.append(dict(identity=i, path='-'.join(map(str, i))+'.mat',
|
||||
split=espfi.split_identity(i), sha256=h, signal_sha256=h))
|
||||
return dict(schema_version=1, classes=espfi.CLASSES, source_revision=espfi.REVISION,
|
||||
archive_sha256=espfi.ARCHIVE_SHA256, source_url=espfi.SOURCE_URL,
|
||||
license='CC-BY-4.0', split_policy='joint_room_subject_intersection_v1',
|
||||
temporal_mean_pool=5, window_samples=190, records=records,
|
||||
counts={s: sum(r['split'] == s for r in records) for s in ('train', 'validation', 'test')})
|
||||
|
||||
def test_parent_population_and_quarantine(self):
|
||||
parent = self.parent()
|
||||
self.assertEqual(len(native._parent_records(parent)), 2240)
|
||||
parent['records'][1]['signal_sha256'] = parent['records'][0]['signal_sha256']
|
||||
with self.assertRaisesRegex(ValueError, 'quarantine'):
|
||||
native._parent_records(parent)
|
||||
for r in parent['records'][:2]:
|
||||
r['split'] = None
|
||||
parent['counts']['train'] -= 2
|
||||
self.assertEqual(len(native._parent_records(parent)), 2240)
|
||||
parent['records'].pop()
|
||||
with self.assertRaisesRegex(ValueError, 'population'):
|
||||
native._parent_records(parent)
|
||||
|
||||
def test_prepare_skip_payloads_reload_tamper_and_failure(self):
|
||||
# Small synthetic parent isolates streaming behavior; full population
|
||||
# validation is exercised independently above and never disabled in CLI.
|
||||
identities = [(1, 1, 1, 1), (3, 7, 2, 1), (4, 8, 3, 1), (1, 7, 4, 1)]
|
||||
contents, lookup, prior, accessed = {}, {}, {}, []
|
||||
for offset, ident in enumerate(identities):
|
||||
content = self.mat(np.full((950, 52), 1.25 + offset, dtype=np.float64))
|
||||
name = '-'.join(map(str, ident)) + '.mat'
|
||||
contents[ident] = content
|
||||
pooled = espfi.decode_amplitude(content)
|
||||
split = espfi.split_identity(ident)
|
||||
lookup[ident] = dict(identity=ident, path=name, split=split,
|
||||
sha256=native.sha(content), signal_sha256=native.sha(pooled.tobytes()))
|
||||
if split in native.SPLITS:
|
||||
prior[split] = dict(x=pooled[None], y=np.array([ident[2]-1]), identity=np.array([ident]))
|
||||
class Entry:
|
||||
isfile = True
|
||||
def __init__(self, ident):
|
||||
self.ident, self.pathname = ident, lookup[ident]['path']
|
||||
self.size = len(contents[ident])
|
||||
def get_blocks(self):
|
||||
accessed.append(self.ident)
|
||||
if self.ident not in identities[:2]:
|
||||
raise AssertionError('Test or excluded payload opened')
|
||||
yield contents[self.ident]
|
||||
@contextlib.contextmanager
|
||||
def archive_reader(_):
|
||||
yield iter(Entry(i) for i in identities)
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
root = Path(tmp)
|
||||
archive = root/'source.rar'
|
||||
archive.write_bytes(b'rar')
|
||||
parent = root/'original.json'
|
||||
parent.write_text('{}')
|
||||
provenance = {'manifest_sha256': native.sha(parent.read_bytes())}
|
||||
with patch.object(espfi, 'ARCHIVE_BYTES', 3), patch.object(espfi, 'ARCHIVE_SHA256', native.sha(b'rar')), \
|
||||
patch.object(native, '_parent_records', return_value=lookup), \
|
||||
patch.object(espfi, 'load_development', return_value=(prior, provenance)), \
|
||||
patch.dict(sys.modules, {'libarchive': types.SimpleNamespace(file_reader=archive_reader)}):
|
||||
target = native.prepare(archive, parent, root/'prepared')
|
||||
self.assertEqual(accessed, identities[:2])
|
||||
parts, meta = native.load_resolution(target)
|
||||
self.assertFalse(meta['test_payload_decoded'])
|
||||
self.assertEqual(set(parts), {'train', 'validation'})
|
||||
np.testing.assert_array_equal(parts['train']['pooled'], prior['train']['x'])
|
||||
bundle = target.parent/'amplitudes.npz'
|
||||
original = bundle.read_bytes()
|
||||
bundle.write_bytes(original + b'bad')
|
||||
with self.assertRaisesRegex(ValueError, 'digest'):
|
||||
native.load_resolution(target)
|
||||
bundle.write_bytes(original)
|
||||
status = target.parent/'status.json'
|
||||
status.write_text('{"status":"failed","promotionAuthorized":false}')
|
||||
with self.assertRaisesRegex(ValueError, 'completed'):
|
||||
native.load_resolution(target)
|
||||
with patch.object(native, 'decode_native', side_effect=ValueError('decoder failure')):
|
||||
with self.assertRaisesRegex(ValueError, 'decoder failure'):
|
||||
native.prepare(archive, parent, root/'failed')
|
||||
self.assertEqual(json.loads((root/'failed/status.json').read_text())['status'], 'failed')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user