feat(mcp): derive an API surface view from retained web network captures (#1891)

* feat(mcp): derive an API surface view from retained web network captures

* fix(mcp): fail closed on sensitive parents and budget api-surface path expansion

* fix(mcp): keep recorded api-surface paths and budget only templated segments

* fix(mcp): read api-surface paths only from canonical URLs and charge new values once

* fix(mcp): reject authority-less api-surface URLs and key groups by rule positions

* fix(mcp): label api-surface confidence from all groups and cap transient path memory

* fix(mcp): budget api-surface dedup sets and narrow detail outputs

Charge retained deduplication sets before allocation, advertise only
each analysis operation's summary from its detail output, and document
that any templated group makes every API-surface page inferred.

* fix(mcp): bound api-surface path text and record limitations

Charge retained literal path copies before allocation, publish only
valid media-type essences, and deduplicate examined record limitations
onto the view.

* fix(mcp): keep api-surface refusals and path splits inside the reply budget

Sensitive-parent refusals now use the token sensitive_values_unretained so declared literals are not echoed. A later path split is charged against bytes already retained, and the published example includes the HAR record limitation with its matching digest.

* fix(mcp): budget api-surface method text before group keys

Charge method and origin text against the reply budget before the group key is serialized. Document that samples merge only when method, origin, and exact segment rules match.

* fix(mcp): bound api-surface copies and control-character authorities

Reject authorities containing a tab or newline so WHATWG cannot move userinfo into the recorded path. Charge four path copies, two method and origin copies, and each distinct content type against bytes already retained.

* fix(mcp): preflight combined api-surface group charges

Total path, method, origin, and segment charges before retaining a new group. Track content types in a set so distinct MIME essences are not compared quadratically.

* fix(mcp): charge api-surface content-type sets on first use

Create the per-group content-type set when the first essence is retained, and charge it before allocation. Groups with no content type no longer keep an empty set.

* refactor(analysis): centralize API-surface ownership and admission

Own grouping, deduplication, and combined retained/temporary resource admission in one accumulator. Charge fixed group storage before allocation and materialize only the requested page.

Share recorded URL parsing between capture redaction and projection, preserving source paths and excluding ambiguous credential fields. Cover resource admission invariants and the compiled HAR-to-view workflow.

* fix(analysis): admit capture schema clone before parsing

* fix(javascript): share Electron mutation proof with wrapped calls

* test(mcp): align graph and catalog checks with current contracts

* fix(analysis): admit and validate portable capture views

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>

docs/web-network-captures.md

@@ -100,6 +100,11 @@ resources follow the same explicit declarations without changing error types;
 sensitive issue pointers identify a real ancestor instead of a fabricated
 coordinate. Header/cookie authentication values and known
 transport URL userinfo are excluded structurally, including native backups.
+HAR redaction and API-surface projection share the same recorded-URL parser.
+Canonical userinfo spans are removed without rewriting the remaining source
+text. Credential-bearing spellings without a safe raw span, such as slashless
+or extra-slash HTTP URLs, are excluded as a whole transport field. Ordinary
+extension text and noncredential path/query values remain unchanged.
 
 Generated free-text limitations follow the same literal declarations, including
 global, per-record and Evidence explanations added during projection. Contract
@@ -112,6 +117,74 @@ wire fragments. Native dictionary duplicate keys have the unchanged upstream
 decoder's last-value semantics; this adapter does not claim to recover discarded
 duplicates or the original numeric spelling of parsed native floats.
 
+## Derived API surface
+
+Pass retained HAR Evidence to `inspect_analysis_view` with
+`view: {"kind":"api-surface","offset":0,"limit":50}`. Omitted offset and limit
+are 0 and 50. The view groups request lines by method, WHATWG URL origin and
+path template, sorted by origin, template and method. Query strings, fragments,
+credentials, headers and bodies are not copied into groups.
+
+| Rule       | Exact segment format                                 | Placeholder  |
+| ---------- | ---------------------------------------------------- | ------------ |
+| `numeric`  | ASCII digits only                                    | `{numeric}`  |
+| `uuid`     | RFC 9562 textual UUID, any version, case-insensitive | `{uuid}`     |
+| `hex`      | At least 32 hexadecimal digits                       | `{hex}`      |
+| `iso-date` | Valid calendar date `YYYY-MM-DD`                     | `{iso-date}` |
+
+Rules apply in table order to the recorded, still-encoded request path, read
+before WHATWG dot-segment removal, so `/a/%2e%2e/admin` and `/admin` stay distinct. Segment indexes start
+at zero after the leading slash. Each matched segment lists its observed literals
+and `basis: "inferred"`. `templated_groups` counts templated groups across every
+page. Because paging follows the full grouping, every page is inferred Evidence
+when that count is nonzero, and derived Evidence only when no group was templated. Version prefixes, short opaque IDs
+and digit-bearing route names stay literal. Samples merge only when method,
+origin, and exact segment rules match. The view does not infer further
+cross-sample patterns.
+
+Groups cite parent record ordinals and distinct statuses and MIME types (lowercase
+type/subtype without parameters). A malformed media type is unknown, not a content
+type. Flags report single samples, mixed statuses or mixed content types.
+`coverage` pages groups; `records` separately reports examined, total and excluded
+parent records. Ordinal subsets carry a limitation. Null URLs are excluded as
+`url-redacted`; unparseable URLs, URLs without an origin, spellings without
+`scheme://authority` (such as `https:host/path`), and authorities containing a
+tab or newline as `url-unparseable`.
+Unavailable request objects or methods are explicitly excluded. Unavailable
+response fields, including HAR's `status: 0` and an empty `mimeType`, remain
+unknown, not invented statuses or MIME types.
+
+The view makes no handler-identity, schema or causality claim. It performs no
+network activity. Native mitmproxy captures return a typed unsupported error.
+Parents inspected with `sensitive_values`, or whose Evidence does not record
+`sensitive_value_count`, return a typed unsupported error whose reason is the
+canonical token `sensitive_values_unretained`. The view would write path
+templates and limitation text that it cannot check against declarations the
+parent does not retain, so the refusal does not repeat declaration text.
+
+One accumulator owns group identity, observed-value/status/MIME sets, ordinals,
+exclusions and deduplicated limitations. It scans path spans without splitting
+literal paths into arrays. Before schema parsing, it reserves the capture clone
+and validation scratch by walking the parent without cloning it. Inline requests
+borrow the Evidence value through CLI/MCP envelope validation; the application
+admits the entire portable envelope before authenticated parsing or hashing.
+The envelope clone, hash scratch and capture clone stay charged during grouping.
+Temporary URL/path/MIME work and retained state must
+fit together within a 96 MiB projection representation budget. A whole group or
+merge is admitted before allocating its retained sets, map entries or array
+values. Fixed group storage, distinct text, exclusions and limitations count as
+well as matched segments. Per-record reservations are released after each
+record; repeated values do not accumulate another retained charge. Only the
+requested page is materialized as output groups.
+
+The representation budget uses conservative storage and worst-case JSON-escaping
+allowances; it is not an enforced V8 heap or process-memory limit. It is separate
+from the decoder's actual reply-byte limit and MCP transport admission. Exceeding
+it returns `resource_constraint` without a partial view. Limitations on the
+parent envelope, capture and examined records are deduplicated onto the view.
+Portable captures must retain unique ordinals within their declared total;
+caller-selected subsets can preserve any requested ordinal order.
+
 ## Resource and lifecycle boundaries
 
 Complete evidence has a 32 MiB input, 96 MiB decoder reply and 64-level nesting