mirror of
https://github.com/JackChen-me/open-multi-agent.git
synced 2026-10-19 01:29:57 -05:00
Jack Chen
5033842802
Follow-up to #548, which introduced VideoBlock. Two gaps were missed when that PR was reviewed. The four adapters that cannot map video threw a bare Error. isRetryableError returns true for any Error without a status, so executeWithRetry treated a deterministic capability gap as transient: it ran the full backoff ladder and rewrote a checkpoint per attempt before failing the same way, and the failure was recorded as retryable. UnsupportedContentBlockError makes that terminal, names the adapter that rejected the block, and carries a code alongside the existing UnsupportedToolResultContentError. VideoBlock is the first top-level content block with a URL source, so it inherited no validation. That source is dereferenced by the provider rather than by OMA, and a file: path or a data URL pasted into the field was forwarded verbatim; an unvalidated media_type could also alter the data URL built for inline bytes. Public entry points now run video sources through the same copyMediaSource rules already applied to tool-result media, so the documented HTTP(S) constraint is enforced rather than aspirational. Provider-private upload handles such as MiniMax's mm_file:// are rejected: supporting them needs a source variant of its own, not an unchecked string. Also from the same review: - estimateTokens charged an inline video the 64-character fixed cost images get, hiding a multi-megabyte payload from every context strategy keyed off the estimate. It is now sized like rich tool-result media, so the attachment that needs compacting is the one that triggers compaction. - stripImageBlocksForSummary handles video too, so it is now stripMediaBlocksForSummary. - The two double casts in openai-common are narrowed to a single seam, and the video_url extension type now records that it is not an OpenAI part. - docs/structured-input.md states the rule the adapters actually follow: OpenAI-shaped adapters forward and let the provider decide, adapters with a typed content union reject locally. Bedrock and Gemini support video upstream, so the gap there is OMA's mapping rather than theirs.