Systematic documentation review: 12 new docs pages (coordinator, streaming, budgets and limits, errors, hooks and callbacks, glossary, production checklist, Run Viewer, MCP, sandbox and shell, evaluation in CI, routing evaluation) plus a docs index; the three longest guides split with stub sections that keep existing anchors; observability release records moved to docs/internal; statements re-verified against the code and corrected; READMEs, AGENTS.md, CONTRIBUTING.md, and issue templates synced. Documentation only, no behavior change.
Follow-up to #548, which introduced VideoBlock. Two gaps were missed when that
PR was reviewed.
The four adapters that cannot map video threw a bare Error. isRetryableError
returns true for any Error without a status, so executeWithRetry treated a
deterministic capability gap as transient: it ran the full backoff ladder and
rewrote a checkpoint per attempt before failing the same way, and the failure
was recorded as retryable. UnsupportedContentBlockError makes that terminal,
names the adapter that rejected the block, and carries a code alongside the
existing UnsupportedToolResultContentError.
VideoBlock is the first top-level content block with a URL source, so it
inherited no validation. That source is dereferenced by the provider rather
than by OMA, and a file: path or a data URL pasted into the field was forwarded
verbatim; an unvalidated media_type could also alter the data URL built for
inline bytes. Public entry points now run video sources through the same
copyMediaSource rules already applied to tool-result media, so the documented
HTTP(S) constraint is enforced rather than aspirational. Provider-private
upload handles such as MiniMax's mm_file:// are rejected: supporting them needs
a source variant of its own, not an unchecked string.
Also from the same review:
- estimateTokens charged an inline video the 64-character fixed cost images
get, hiding a multi-megabyte payload from every context strategy keyed off
the estimate. It is now sized like rich tool-result media, so the attachment
that needs compacting is the one that triggers compaction.
- stripImageBlocksForSummary handles video too, so it is now
stripMediaBlocksForSummary.
- The two double casts in openai-common are narrowed to a single seam, and the
video_url extension type now records that it is not an OpenAI part.
- docs/structured-input.md states the rule the adapters actually follow:
OpenAI-shaped adapters forward and let the provider decide, adapters with a
typed content union reject locally. Bedrock and Gemini support video
upstream, so the gap there is OMA's mapping rather than theirs.
Adds a public VideoBlock content block with inline base64 and URL sources, serialized as an OpenAI-compatible video_url content part so MiniMax-M3 can receive video input. Video attachments are replaced with placeholders when preparing context summaries, keeping binary payloads out of the summarisation prompt. Adapters with a typed content union reject the block explicitly.
Jack Chen