Files
rea/tests/boundary/il2cpp/il2cppMetadataProvider.test.ts
T
JANG BYUNGKUNandgensokyo-dev 833136507d feat(il2cpp): inventory Unity IL2CPP global-metadata images, types and methods (#3807)
* feat(il2cpp): inventory IL2CPP global-metadata images, types and methods

Add inspect_il2cpp_metadata (CLI inspect-il2cpp-metadata), a read-only
in-process decoder for unencrypted Unity global-metadata.dat version 39.
It reports header sections, serialized index widths, images, type
definitions (nesting from the nestedTypes section) and method names.

Unrecognized magic and other versions are unsupported; damaged headers,
overlapping sections, inconsistent record sizes, out-of-range indices and
invalid strings fail with a reason code and no partial success. MCP
detail summary retains the complete Evidence and pages rows through
inspect_analysis_view kind il2cpp-metadata.

Adds an opt-in verify:il2cpp:metadata -- --metadata PATH check.

* feat(il2cpp): deliver metadata summaries by default over MCP

A 112,232-method file produced a 10.5 MB complete response, above the
10 MiB stdio budget. Default MCP delivery is now the retained summary;
detail complete stays available, and the one-shot CLI requests it.

* fix(il2cpp): reject shifted strings, nesting cycles and method owner drift

Review fuzzing showed that moving the strings section offset by one byte
silently renamed every image, type and method. Name indices must now start
a string, which also bounds total decoding by the section size.

Nesting cycles and methods whose declaring type disagrees with the owning
type range are rejected instead of returned. Adds service, shutdown and
index-width coverage, and moves environment-specific measurements out of
the public guide and tool description.

* feat(il2cpp): decode global-metadata versions 29 and 31

Unity 2022.3 writes version 29 or 31 metadata: a 256-byte header of
(offset, size) pairs, fixed 4-byte indices, type records with
elementTypeIndex, and method records that gained returnParameterToken in
version 31. Record counts come from section sizes and must be whole.

Sections report a null count for these versions and index widths report
fixed-width-version. Image, type and method rows keep the same shape, and
the string, nesting and method-ownership checks apply to every version.

* fix(il2cpp): bound report size before serializing and tighten coverage

Rows share decoded names, so a 130 KB file could expand into gigabytes of
JSON before the 64 MiB budget check, ending in an internal RangeError. The
provider now refuses from a lower bound on the serialized rows first.

The decoder also rejects non-empty sections inside the header, methods
outside their declaring type's range and overlapping image type ranges, and
keeps a leading byte order mark in names. Layouts now carry one indexing
mode instead of two coupled flags, and the tool description explains
nested_type_start.

* fix(il2cpp): refuse oversized metadata before decoding rows

Address review findings on the IL2CPP metadata inventory:

- check a lower bound from header row counts before any row object is
  allocated, so a large rejected file no longer builds every row first;
- validate sealed parent Evidence and count its rows once per report
  instead of on every inspect_analysis_view page;
- decode version 39 fixtures with 2- and 4-byte type-definition,
  generic-container and parameter indices.

* docs(il2cpp): attach the decoder comment to readIl2cppMetadata

* fix(il2cpp): deliver complete rows by default like the other inspection tools

inspect_il2cpp_metadata defaulted to detail summary, the only inspection
tool to do so. Omitted detail now returns every row inline, as
inspect_artifact, decode_interface_builder and the capture tools do;
summary stays one argument away and the guide, CLI reference and skill
reference say to select it first for large games, whose complete MCP
delivery is refused at the stdio budget with retained Evidence.

* test(il2cpp): cover narrow-width -1 sentinels and fix the skill's default

An empty image stores its type start as the all-ones value of a one- or
two-byte index; the decoder reads it as -1, but no fixture exercised it,
so removing either conversion passed the suite. The fixture can now append
an empty image, and both widths decode it. The skill still described a
summary default; it now says to select summary first, as the guide does.

* docs(il2cpp): request summary explicitly in the summary example

* chore(il2cpp): hash the verifier metadata through owned file resources

* fix(il2cpp): count escaped name length without building the escaped string

* fix(il2cpp): keep the verifier's check failure and small-table last page

* feat(domain): count JSON string length without building the escaped string

* fix(il2cpp): keep descriptor cleanup owners and share the JSON length count

* test(domain): give the backslash case real backslashes

* fix(il2cpp): select the CLI path and finish the command like the other tools

Follow main's absoluteCallerPath (#4067) and pass the command signal to finishAnalysisCommand.

* fix(il2cpp): select the verifier input path like the CLI

Use absoluteCallerPath (#4067) so a path with a symlink followed by .. names the same file the CLI and MCP inspect.

* feat(il2cpp): add compact composable metadata queries

Retain source-indexed Evidence for filtered batch and field queries. Bound representation growth before serialization, yield decoding for cancellation, and share CLI Evidence publication with desktop inspection. Verify public Unity metadata against independent byte and managed-DLL oracles and exercise actual stdio SDK workflows.

* fix(il2cpp): retain the validated metadata header magic

* test(il2cpp): remove obsolete execution wording exception

* fix(artifacts): preserve export selection and completed source facts

---------

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-20 07:20:47 +08:00

307 lines
11 KiB
TypeScript

import { setImmediate } from "node:timers/promises";
import { createHash } from "node:crypto";
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { afterEach, expect, it, vi } from "vitest";
import {
AnalysisCancelledError,
AnalysisResourceConstraintError,
} from "../../../src/domain/analysisErrorCore.js";
import { Il2cppMetadataProvider } from "../../../src/il2cpp/Il2cppMetadataProvider.js";
import { createIl2cppMetadataFixture } from "../../fixtures/il2cpp/metadata.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
afterEach(() => {
vi.restoreAllMocks();
});
/** Version 29 file whose methods all share one long name. */
const sharedNameMetadata = (
nameBytes: number | Buffer,
methods: number,
fill = 0x41,
) => {
const encoded = Buffer.isBuffer(nameBytes)
? nameBytes
: Buffer.alloc(nameBytes, fill);
const name = Buffer.concat([encoded, Buffer.from([0])]);
const records = Buffer.alloc(methods * 32);
for (let index = 0; index < methods; index += 1)
records.writeInt32LE(-1, index * 32 + 4);
const header = Buffer.alloc(256);
header.writeUInt32LE(0xfab11baf, 0);
header.writeInt32LE(29, 4);
let cursor = 256;
for (let section = 0; section < 31; section += 1) {
// strings is section 2 and methods is section 5.
const content = section === 2 ? name : section === 5 ? records : undefined;
header.writeInt32LE(cursor, 8 + section * 8);
header.writeInt32LE(content?.length ?? 0, 12 + section * 8);
cursor += content?.length ?? 0;
}
return Buffer.concat([header, name, records]);
};
it("refuses an oversized report from its row lower bound before serializing it", async () => {
const root = await createTestTempDirectory("rea-il2cpp-provider-budget-");
const path = join(root, "global-metadata.dat");
// Shared names make about 100 KiB of input expand beyond 64 MiB of JSON.
await writeFile(path, sharedNameMetadata(65_536, 1_100));
const stringify = vi.spyOn(JSON, "stringify");
const provider = new Il2cppMetadataProvider();
try {
const result = await provider.inspect({ path });
expect(result.ok).toBe(false);
if (!result.ok)
expect(result.error).toBeInstanceOf(AnalysisResourceConstraintError);
expect(
stringify.mock.calls.some(
([value]) =>
typeof value === "object" && value !== null && "methods" in value,
),
).toBe(false);
} finally {
await provider.close();
}
});
it("counts escaped name characters toward the budget without escaping the name", async () => {
const root = await createTestTempDirectory("rea-il2cpp-provider-escape-");
const path = join(root, "global-metadata.dat");
// 4 KiB of U+0001 is 4 KiB raw but 24 KiB once escaped as .
await writeFile(path, sharedNameMetadata(4096, 1, 0x01));
const stringify = vi.spyOn(JSON, "stringify");
const provider = new Il2cppMetadataProvider(8192);
try {
const result = await provider.inspect({ path });
expect(result.ok).toBe(false);
if (!result.ok)
expect(result.error).toBeInstanceOf(AnalysisResourceConstraintError);
expect(
stringify.mock.calls.some(
([value]) => typeof value === "string" && value.length >= 4096,
),
).toBe(false);
} finally {
await provider.close();
}
});
it("admits a report whose escaped names fit the budget exactly", async () => {
const root = await createTestTempDirectory("rea-il2cpp-provider-fit-");
const path = join(root, "global-metadata.dat");
await writeFile(path, sharedNameMetadata(4096, 1, 0x01));
const unbounded = new Il2cppMetadataProvider();
let bytes: number;
try {
const result = await unbounded.inspect({ path });
if (!result.ok) throw result.error;
expect(result.value.methods[0]?.name).toBe("".repeat(4096));
bytes = Buffer.byteLength(JSON.stringify(result.value));
} finally {
await unbounded.close();
}
const exact = new Il2cppMetadataProvider(bytes);
try {
expect((await exact.inspect({ path })).ok).toBe(true);
} finally {
await exact.close();
}
});
it("refuses from row counts before decoding any row", async () => {
const root = await createTestTempDirectory("rea-il2cpp-provider-counts-");
const path = join(root, "global-metadata.dat");
// A broken method name would fail as invalid-string if rows were decoded.
const fixture = createIl2cppMetadataFixture();
const bytes = Buffer.from(fixture.bytes);
bytes.writeInt32LE(0x7fff, fixture.record("methods", 0));
await writeFile(path, bytes);
const provider = new Il2cppMetadataProvider(256);
try {
const result = await provider.inspect({ path });
expect(result.ok).toBe(false);
if (!result.ok)
expect(result.error).toBeInstanceOf(AnalysisResourceConstraintError);
} finally {
await provider.close();
}
});
it.each([
{ requested: "cancel during metadata decoding" },
new AnalysisResourceConstraintError(
"caller-selected-reason",
"memory",
"Caller cancellation reason",
{},
),
])(
"accepts cancellation during decoded batches with exact reason %j",
async (reason) => {
const root = await createTestTempDirectory("rea-il2cpp-provider-cancel-");
const path = join(root, "global-metadata.dat");
const bytes = sharedNameMetadata(1, 100_000);
await writeFile(path, bytes);
const controller = new AbortController();
let batches = 0;
const provider = new Il2cppMetadataProvider(undefined, async () => {
await setImmediate();
if (++batches === 3) controller.abort(reason);
});
try {
const result = await provider.inspect(
{ path },
{ signal: controller.signal },
);
expect(result.ok).toBe(false);
if (result.ok) throw new Error("Expected cancellation");
expect(result.error).toBeInstanceOf(AnalysisCancelledError);
expect(result.error.cause).toBe(reason);
expect(batches).toBe(3);
expect(result.error.partialObservation).toMatchObject({
operation: "observe_il2cpp_metadata_source",
normalized_result: {
artifact: {
path,
sha256: createHash("sha256").update(bytes).digest("hex"),
bytes: bytes.length,
},
inventory_status: "not-returned-as-complete",
validated_section_row_counts: {
images: 0,
types: 0,
methods: 100_000,
},
},
});
const replacement = createIl2cppMetadataFixture();
await writeFile(path, replacement.bytes);
const next = await provider.inspect({ path });
if (!next.ok) throw next.error;
expect(next.value.artifact.sha256).toBe(
createHash("sha256").update(replacement.bytes).digest("hex"),
);
expect(next.value.methods).toHaveLength(5);
} finally {
await provider.close();
}
},
);
it("bounds concurrent snapshots and retains active decoding until close completes", async () => {
const root = await createTestTempDirectory("rea-il2cpp-provider-close-");
const path = join(root, "global-metadata.dat");
await writeFile(path, createIl2cppMetadataFixture().bytes);
let markEntered!: () => void;
const entered = new Promise<void>((resolve) => {
markEntered = resolve;
});
let resume!: () => void;
const release = new Promise<void>((resolve) => {
resume = resolve;
});
const provider = new Il2cppMetadataProvider(undefined, () => {
markEntered();
return release;
});
const active = provider.inspect({ path });
await entered;
let closed = false;
const closing = provider.close().then(() => {
closed = true;
});
try {
const refused = await provider.inspect({ path });
expect(refused.ok).toBe(false);
if (!refused.ok)
expect(refused.error).toBeInstanceOf(AnalysisResourceConstraintError);
await setImmediate();
expect(closed).toBe(false);
} finally {
resume();
await closing;
}
expect((await active).ok).toBe(true);
expect(closed).toBe(true);
expect((await provider.inspect({ path })).ok).toBe(false);
});
it("refuses UTF-8 expansion before serializing the complete report", async () => {
const root = await createTestTempDirectory("rea-il2cpp-provider-unicode-");
const path = join(root, "global-metadata.dat");
await writeFile(path, sharedNameMetadata(Buffer.from("雪".repeat(4096)), 1));
const stringify = vi.spyOn(JSON, "stringify");
const provider = new Il2cppMetadataProvider(8192);
try {
const result = await provider.inspect({ path });
expect(result.ok).toBe(false);
if (!result.ok)
expect(result.error).toBeInstanceOf(AnalysisResourceConstraintError);
expect(
stringify.mock.calls.some(
([value]) =>
typeof value === "object" && value !== null && "methods" in value,
),
).toBe(false);
} finally {
await provider.close();
}
});
it.each([false, true])(
"keeps an independent failure with shared cancellation reason: %s",
async (sharedReason) => {
const root = await createTestTempDirectory("rea-il2cpp-failure-cancel-");
const path = join(root, "global-metadata.dat");
await writeFile(path, createIl2cppMetadataFixture().bytes);
const controller = new AbortController();
const failure = Object.assign(
new Error("independent scheduler I/O failure"),
{ code: "EIO" },
);
const provider = new Il2cppMetadataProvider(undefined, () => {
controller.abort(sharedReason ? failure : "separate caller cancellation");
return Promise.reject(failure);
});
try {
const result = await provider.inspect(
{ path },
{ signal: controller.signal },
);
expect(result.ok).toBe(false);
if (result.ok) throw new Error("Expected the independent failure");
expect(result.error).not.toBeInstanceOf(AnalysisCancelledError);
expect(result.error.cause).toBe(failure);
} finally {
await provider.close();
}
},
);
it("retains the exact empty source observation without claiming a decoded inventory", async () => {
const root = await createTestTempDirectory("rea-il2cpp-empty-");
const path = join(root, "empty.dat");
await writeFile(path, Buffer.alloc(0));
const provider = new Il2cppMetadataProvider();
try {
const result = await provider.inspect({ path });
expect(result.ok).toBe(false);
if (result.ok) throw new Error("Expected unsupported empty metadata");
expect(result.error.message).toContain("unrecognized-magic");
expect(result.error.partialObservation).toMatchObject({
operation: "observe_il2cpp_metadata_source",
normalized_result: {
artifact: {
path,
bytes: 0,
sha256: createHash("sha256").digest("hex"),
},
inventory_status: "not-returned-as-complete",
},
});
} finally {
await provider.close();
}
});