mirror of
https://github.com/morluto/rea.git
synced 2026-10-22 22:02:27 -05:00
* feat(il2cpp): inventory IL2CPP global-metadata images, types and methods Add inspect_il2cpp_metadata (CLI inspect-il2cpp-metadata), a read-only in-process decoder for unencrypted Unity global-metadata.dat version 39. It reports header sections, serialized index widths, images, type definitions (nesting from the nestedTypes section) and method names. Unrecognized magic and other versions are unsupported; damaged headers, overlapping sections, inconsistent record sizes, out-of-range indices and invalid strings fail with a reason code and no partial success. MCP detail summary retains the complete Evidence and pages rows through inspect_analysis_view kind il2cpp-metadata. Adds an opt-in verify:il2cpp:metadata -- --metadata PATH check. * feat(il2cpp): deliver metadata summaries by default over MCP A 112,232-method file produced a 10.5 MB complete response, above the 10 MiB stdio budget. Default MCP delivery is now the retained summary; detail complete stays available, and the one-shot CLI requests it. * fix(il2cpp): reject shifted strings, nesting cycles and method owner drift Review fuzzing showed that moving the strings section offset by one byte silently renamed every image, type and method. Name indices must now start a string, which also bounds total decoding by the section size. Nesting cycles and methods whose declaring type disagrees with the owning type range are rejected instead of returned. Adds service, shutdown and index-width coverage, and moves environment-specific measurements out of the public guide and tool description. * feat(il2cpp): decode global-metadata versions 29 and 31 Unity 2022.3 writes version 29 or 31 metadata: a 256-byte header of (offset, size) pairs, fixed 4-byte indices, type records with elementTypeIndex, and method records that gained returnParameterToken in version 31. Record counts come from section sizes and must be whole. Sections report a null count for these versions and index widths report fixed-width-version. Image, type and method rows keep the same shape, and the string, nesting and method-ownership checks apply to every version. * fix(il2cpp): bound report size before serializing and tighten coverage Rows share decoded names, so a 130 KB file could expand into gigabytes of JSON before the 64 MiB budget check, ending in an internal RangeError. The provider now refuses from a lower bound on the serialized rows first. The decoder also rejects non-empty sections inside the header, methods outside their declaring type's range and overlapping image type ranges, and keeps a leading byte order mark in names. Layouts now carry one indexing mode instead of two coupled flags, and the tool description explains nested_type_start. * fix(il2cpp): refuse oversized metadata before decoding rows Address review findings on the IL2CPP metadata inventory: - check a lower bound from header row counts before any row object is allocated, so a large rejected file no longer builds every row first; - validate sealed parent Evidence and count its rows once per report instead of on every inspect_analysis_view page; - decode version 39 fixtures with 2- and 4-byte type-definition, generic-container and parameter indices. * docs(il2cpp): attach the decoder comment to readIl2cppMetadata * fix(il2cpp): deliver complete rows by default like the other inspection tools inspect_il2cpp_metadata defaulted to detail summary, the only inspection tool to do so. Omitted detail now returns every row inline, as inspect_artifact, decode_interface_builder and the capture tools do; summary stays one argument away and the guide, CLI reference and skill reference say to select it first for large games, whose complete MCP delivery is refused at the stdio budget with retained Evidence. * test(il2cpp): cover narrow-width -1 sentinels and fix the skill's default An empty image stores its type start as the all-ones value of a one- or two-byte index; the decoder reads it as -1, but no fixture exercised it, so removing either conversion passed the suite. The fixture can now append an empty image, and both widths decode it. The skill still described a summary default; it now says to select summary first, as the guide does. * docs(il2cpp): request summary explicitly in the summary example * chore(il2cpp): hash the verifier metadata through owned file resources * fix(il2cpp): count escaped name length without building the escaped string * fix(il2cpp): keep the verifier's check failure and small-table last page * feat(domain): count JSON string length without building the escaped string * fix(il2cpp): keep descriptor cleanup owners and share the JSON length count * test(domain): give the backslash case real backslashes * fix(il2cpp): select the CLI path and finish the command like the other tools Follow main's absoluteCallerPath (#4067) and pass the command signal to finishAnalysisCommand. * fix(il2cpp): select the verifier input path like the CLI Use absoluteCallerPath (#4067) so a path with a symlink followed by .. names the same file the CLI and MCP inspect. * feat(il2cpp): add compact composable metadata queries Retain source-indexed Evidence for filtered batch and field queries. Bound representation growth before serialization, yield decoding for cancellation, and share CLI Evidence publication with desktop inspection. Verify public Unity metadata against independent byte and managed-DLL oracles and exercise actual stdio SDK workflows. * fix(il2cpp): retain the validated metadata header magic * test(il2cpp): remove obsolete execution wording exception * fix(artifacts): preserve export selection and completed source facts --------- Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
307 lines
11 KiB
TypeScript
307 lines
11 KiB
TypeScript
import { setImmediate } from "node:timers/promises";
|
|
import { createHash } from "node:crypto";
|
|
import { writeFile } from "node:fs/promises";
|
|
import { join } from "node:path";
|
|
import { afterEach, expect, it, vi } from "vitest";
|
|
import {
|
|
AnalysisCancelledError,
|
|
AnalysisResourceConstraintError,
|
|
} from "../../../src/domain/analysisErrorCore.js";
|
|
import { Il2cppMetadataProvider } from "../../../src/il2cpp/Il2cppMetadataProvider.js";
|
|
import { createIl2cppMetadataFixture } from "../../fixtures/il2cpp/metadata.js";
|
|
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
|
|
|
afterEach(() => {
|
|
vi.restoreAllMocks();
|
|
});
|
|
|
|
/** Version 29 file whose methods all share one long name. */
|
|
const sharedNameMetadata = (
|
|
nameBytes: number | Buffer,
|
|
methods: number,
|
|
fill = 0x41,
|
|
) => {
|
|
const encoded = Buffer.isBuffer(nameBytes)
|
|
? nameBytes
|
|
: Buffer.alloc(nameBytes, fill);
|
|
const name = Buffer.concat([encoded, Buffer.from([0])]);
|
|
const records = Buffer.alloc(methods * 32);
|
|
for (let index = 0; index < methods; index += 1)
|
|
records.writeInt32LE(-1, index * 32 + 4);
|
|
const header = Buffer.alloc(256);
|
|
header.writeUInt32LE(0xfab11baf, 0);
|
|
header.writeInt32LE(29, 4);
|
|
let cursor = 256;
|
|
for (let section = 0; section < 31; section += 1) {
|
|
// strings is section 2 and methods is section 5.
|
|
const content = section === 2 ? name : section === 5 ? records : undefined;
|
|
header.writeInt32LE(cursor, 8 + section * 8);
|
|
header.writeInt32LE(content?.length ?? 0, 12 + section * 8);
|
|
cursor += content?.length ?? 0;
|
|
}
|
|
return Buffer.concat([header, name, records]);
|
|
};
|
|
|
|
it("refuses an oversized report from its row lower bound before serializing it", async () => {
|
|
const root = await createTestTempDirectory("rea-il2cpp-provider-budget-");
|
|
const path = join(root, "global-metadata.dat");
|
|
// Shared names make about 100 KiB of input expand beyond 64 MiB of JSON.
|
|
await writeFile(path, sharedNameMetadata(65_536, 1_100));
|
|
const stringify = vi.spyOn(JSON, "stringify");
|
|
const provider = new Il2cppMetadataProvider();
|
|
try {
|
|
const result = await provider.inspect({ path });
|
|
expect(result.ok).toBe(false);
|
|
if (!result.ok)
|
|
expect(result.error).toBeInstanceOf(AnalysisResourceConstraintError);
|
|
expect(
|
|
stringify.mock.calls.some(
|
|
([value]) =>
|
|
typeof value === "object" && value !== null && "methods" in value,
|
|
),
|
|
).toBe(false);
|
|
} finally {
|
|
await provider.close();
|
|
}
|
|
});
|
|
|
|
it("counts escaped name characters toward the budget without escaping the name", async () => {
|
|
const root = await createTestTempDirectory("rea-il2cpp-provider-escape-");
|
|
const path = join(root, "global-metadata.dat");
|
|
// 4 KiB of U+0001 is 4 KiB raw but 24 KiB once escaped as .
|
|
await writeFile(path, sharedNameMetadata(4096, 1, 0x01));
|
|
const stringify = vi.spyOn(JSON, "stringify");
|
|
const provider = new Il2cppMetadataProvider(8192);
|
|
try {
|
|
const result = await provider.inspect({ path });
|
|
expect(result.ok).toBe(false);
|
|
if (!result.ok)
|
|
expect(result.error).toBeInstanceOf(AnalysisResourceConstraintError);
|
|
expect(
|
|
stringify.mock.calls.some(
|
|
([value]) => typeof value === "string" && value.length >= 4096,
|
|
),
|
|
).toBe(false);
|
|
} finally {
|
|
await provider.close();
|
|
}
|
|
});
|
|
|
|
it("admits a report whose escaped names fit the budget exactly", async () => {
|
|
const root = await createTestTempDirectory("rea-il2cpp-provider-fit-");
|
|
const path = join(root, "global-metadata.dat");
|
|
await writeFile(path, sharedNameMetadata(4096, 1, 0x01));
|
|
const unbounded = new Il2cppMetadataProvider();
|
|
let bytes: number;
|
|
try {
|
|
const result = await unbounded.inspect({ path });
|
|
if (!result.ok) throw result.error;
|
|
expect(result.value.methods[0]?.name).toBe("".repeat(4096));
|
|
bytes = Buffer.byteLength(JSON.stringify(result.value));
|
|
} finally {
|
|
await unbounded.close();
|
|
}
|
|
const exact = new Il2cppMetadataProvider(bytes);
|
|
try {
|
|
expect((await exact.inspect({ path })).ok).toBe(true);
|
|
} finally {
|
|
await exact.close();
|
|
}
|
|
});
|
|
|
|
it("refuses from row counts before decoding any row", async () => {
|
|
const root = await createTestTempDirectory("rea-il2cpp-provider-counts-");
|
|
const path = join(root, "global-metadata.dat");
|
|
// A broken method name would fail as invalid-string if rows were decoded.
|
|
const fixture = createIl2cppMetadataFixture();
|
|
const bytes = Buffer.from(fixture.bytes);
|
|
bytes.writeInt32LE(0x7fff, fixture.record("methods", 0));
|
|
await writeFile(path, bytes);
|
|
const provider = new Il2cppMetadataProvider(256);
|
|
try {
|
|
const result = await provider.inspect({ path });
|
|
expect(result.ok).toBe(false);
|
|
if (!result.ok)
|
|
expect(result.error).toBeInstanceOf(AnalysisResourceConstraintError);
|
|
} finally {
|
|
await provider.close();
|
|
}
|
|
});
|
|
|
|
it.each([
|
|
{ requested: "cancel during metadata decoding" },
|
|
new AnalysisResourceConstraintError(
|
|
"caller-selected-reason",
|
|
"memory",
|
|
"Caller cancellation reason",
|
|
{},
|
|
),
|
|
])(
|
|
"accepts cancellation during decoded batches with exact reason %j",
|
|
async (reason) => {
|
|
const root = await createTestTempDirectory("rea-il2cpp-provider-cancel-");
|
|
const path = join(root, "global-metadata.dat");
|
|
const bytes = sharedNameMetadata(1, 100_000);
|
|
await writeFile(path, bytes);
|
|
const controller = new AbortController();
|
|
let batches = 0;
|
|
const provider = new Il2cppMetadataProvider(undefined, async () => {
|
|
await setImmediate();
|
|
if (++batches === 3) controller.abort(reason);
|
|
});
|
|
try {
|
|
const result = await provider.inspect(
|
|
{ path },
|
|
{ signal: controller.signal },
|
|
);
|
|
expect(result.ok).toBe(false);
|
|
if (result.ok) throw new Error("Expected cancellation");
|
|
expect(result.error).toBeInstanceOf(AnalysisCancelledError);
|
|
expect(result.error.cause).toBe(reason);
|
|
expect(batches).toBe(3);
|
|
expect(result.error.partialObservation).toMatchObject({
|
|
operation: "observe_il2cpp_metadata_source",
|
|
normalized_result: {
|
|
artifact: {
|
|
path,
|
|
sha256: createHash("sha256").update(bytes).digest("hex"),
|
|
bytes: bytes.length,
|
|
},
|
|
inventory_status: "not-returned-as-complete",
|
|
validated_section_row_counts: {
|
|
images: 0,
|
|
types: 0,
|
|
methods: 100_000,
|
|
},
|
|
},
|
|
});
|
|
const replacement = createIl2cppMetadataFixture();
|
|
await writeFile(path, replacement.bytes);
|
|
const next = await provider.inspect({ path });
|
|
if (!next.ok) throw next.error;
|
|
expect(next.value.artifact.sha256).toBe(
|
|
createHash("sha256").update(replacement.bytes).digest("hex"),
|
|
);
|
|
expect(next.value.methods).toHaveLength(5);
|
|
} finally {
|
|
await provider.close();
|
|
}
|
|
},
|
|
);
|
|
|
|
it("bounds concurrent snapshots and retains active decoding until close completes", async () => {
|
|
const root = await createTestTempDirectory("rea-il2cpp-provider-close-");
|
|
const path = join(root, "global-metadata.dat");
|
|
await writeFile(path, createIl2cppMetadataFixture().bytes);
|
|
let markEntered!: () => void;
|
|
const entered = new Promise<void>((resolve) => {
|
|
markEntered = resolve;
|
|
});
|
|
let resume!: () => void;
|
|
const release = new Promise<void>((resolve) => {
|
|
resume = resolve;
|
|
});
|
|
const provider = new Il2cppMetadataProvider(undefined, () => {
|
|
markEntered();
|
|
return release;
|
|
});
|
|
const active = provider.inspect({ path });
|
|
await entered;
|
|
let closed = false;
|
|
const closing = provider.close().then(() => {
|
|
closed = true;
|
|
});
|
|
try {
|
|
const refused = await provider.inspect({ path });
|
|
expect(refused.ok).toBe(false);
|
|
if (!refused.ok)
|
|
expect(refused.error).toBeInstanceOf(AnalysisResourceConstraintError);
|
|
await setImmediate();
|
|
expect(closed).toBe(false);
|
|
} finally {
|
|
resume();
|
|
await closing;
|
|
}
|
|
expect((await active).ok).toBe(true);
|
|
expect(closed).toBe(true);
|
|
expect((await provider.inspect({ path })).ok).toBe(false);
|
|
});
|
|
|
|
it("refuses UTF-8 expansion before serializing the complete report", async () => {
|
|
const root = await createTestTempDirectory("rea-il2cpp-provider-unicode-");
|
|
const path = join(root, "global-metadata.dat");
|
|
await writeFile(path, sharedNameMetadata(Buffer.from("雪".repeat(4096)), 1));
|
|
const stringify = vi.spyOn(JSON, "stringify");
|
|
const provider = new Il2cppMetadataProvider(8192);
|
|
try {
|
|
const result = await provider.inspect({ path });
|
|
expect(result.ok).toBe(false);
|
|
if (!result.ok)
|
|
expect(result.error).toBeInstanceOf(AnalysisResourceConstraintError);
|
|
expect(
|
|
stringify.mock.calls.some(
|
|
([value]) =>
|
|
typeof value === "object" && value !== null && "methods" in value,
|
|
),
|
|
).toBe(false);
|
|
} finally {
|
|
await provider.close();
|
|
}
|
|
});
|
|
|
|
it.each([false, true])(
|
|
"keeps an independent failure with shared cancellation reason: %s",
|
|
async (sharedReason) => {
|
|
const root = await createTestTempDirectory("rea-il2cpp-failure-cancel-");
|
|
const path = join(root, "global-metadata.dat");
|
|
await writeFile(path, createIl2cppMetadataFixture().bytes);
|
|
const controller = new AbortController();
|
|
const failure = Object.assign(
|
|
new Error("independent scheduler I/O failure"),
|
|
{ code: "EIO" },
|
|
);
|
|
const provider = new Il2cppMetadataProvider(undefined, () => {
|
|
controller.abort(sharedReason ? failure : "separate caller cancellation");
|
|
return Promise.reject(failure);
|
|
});
|
|
try {
|
|
const result = await provider.inspect(
|
|
{ path },
|
|
{ signal: controller.signal },
|
|
);
|
|
expect(result.ok).toBe(false);
|
|
if (result.ok) throw new Error("Expected the independent failure");
|
|
expect(result.error).not.toBeInstanceOf(AnalysisCancelledError);
|
|
expect(result.error.cause).toBe(failure);
|
|
} finally {
|
|
await provider.close();
|
|
}
|
|
},
|
|
);
|
|
|
|
it("retains the exact empty source observation without claiming a decoded inventory", async () => {
|
|
const root = await createTestTempDirectory("rea-il2cpp-empty-");
|
|
const path = join(root, "empty.dat");
|
|
await writeFile(path, Buffer.alloc(0));
|
|
const provider = new Il2cppMetadataProvider();
|
|
try {
|
|
const result = await provider.inspect({ path });
|
|
expect(result.ok).toBe(false);
|
|
if (result.ok) throw new Error("Expected unsupported empty metadata");
|
|
expect(result.error.message).toContain("unrecognized-magic");
|
|
expect(result.error.partialObservation).toMatchObject({
|
|
operation: "observe_il2cpp_metadata_source",
|
|
normalized_result: {
|
|
artifact: {
|
|
path,
|
|
bytes: 0,
|
|
sha256: createHash("sha256").digest("hex"),
|
|
},
|
|
inventory_status: "not-returned-as-complete",
|
|
},
|
|
});
|
|
} finally {
|
|
await provider.close();
|
|
}
|
|
});
|