Follow-up to #548, which introduced VideoBlock. Two gaps were missed when that
PR was reviewed.
The four adapters that cannot map video threw a bare Error. isRetryableError
returns true for any Error without a status, so executeWithRetry treated a
deterministic capability gap as transient: it ran the full backoff ladder and
rewrote a checkpoint per attempt before failing the same way, and the failure
was recorded as retryable. UnsupportedContentBlockError makes that terminal,
names the adapter that rejected the block, and carries a code alongside the
existing UnsupportedToolResultContentError.
VideoBlock is the first top-level content block with a URL source, so it
inherited no validation. That source is dereferenced by the provider rather
than by OMA, and a file: path or a data URL pasted into the field was forwarded
verbatim; an unvalidated media_type could also alter the data URL built for
inline bytes. Public entry points now run video sources through the same
copyMediaSource rules already applied to tool-result media, so the documented
HTTP(S) constraint is enforced rather than aspirational. Provider-private
upload handles such as MiniMax's mm_file:// are rejected: supporting them needs
a source variant of its own, not an unchecked string.
Also from the same review:
- estimateTokens charged an inline video the 64-character fixed cost images
get, hiding a multi-megabyte payload from every context strategy keyed off
the estimate. It is now sized like rich tool-result media, so the attachment
that needs compacting is the one that triggers compaction.
- stripImageBlocksForSummary handles video too, so it is now
stripMediaBlocksForSummary.
- The two double casts in openai-common are narrowed to a single seam, and the
video_url extension type now records that it is not an OpenAI part.
- docs/structured-input.md states the rule the adapters actually follow:
OpenAI-shaped adapters forward and let the provider decide, adapters with a
typed content union reject locally. Bedrock and Gemini support video
upstream, so the gap there is OMA's mapping rather than theirs.
- separate application-owned data from model-visible text, image, and file content
- map rich results across built-in adapters and preserve them through recovery
- retain backward compatibility for string tools and text-only errors
Closes#466
* chore(deps)!: upgrade OpenAI SDK to v7
Require Node.js 22 across packages, starters, and CI. Add real SDK boundary coverage and reject unsupported custom tool calls explicitly.
Refs #391
* fix(core): classify OpenAI user aborts as cancellation
Use the SDK error constructor name because OpenAI v7 APIUserAbortError retains the default Error name. Cover retry and observability classification with the real SDK class.
* chore(deps)!: target OpenAI v6 and retain Node 20
* docs: simplify Node runtime guidance
Error-aware task retry (opt-in via maxRetries > 0):
- Skip retries on provably-terminal failures (4xx client errors except
408/409/429, token-budget, invalid-message, aborted calls).
- Equal jitter on backoff to avoid lockstep retries against a rate-limited
provider.
- Honor abortSignal between attempts (no full-backoff sleep or extra attempt
after cancel).
- Non-streaming path preserves the structured error on AgentRunResult.error so
classification works off the default runTeam path; test-covered.
Off by default; behavior changes only when retry is enabled.
Closes#337
Jack Chen