Files
open-multi-agent/docs/observability.md
T
Jack Chen 0fab4bce60 feat(memory): redact secrets on the persistence path via RedactingStore (#339) (#359)
Redaction stopped at the observability layer, so secrets an agent emitted into its answer were persisted verbatim to shared memory and on-disk checkpoints. Add RedactingStore, a MemoryStore decorator that scrubs values on write, plus optional custom patterns on the redaction utilities. One wrap of the durable store closes both persistence sinks.

Closes #339.
2026-07-09 22:25:53 +08:00

3.0 KiB

Observability

open-multi-agent exposes three telemetry layers: live progress events, structured trace spans, and a static post-run dashboard.

Progress Events

Use onProgress when you need lightweight lifecycle events for logs, terminal output, or a live UI.

const orchestrator = new OpenMultiAgent({
  onProgress: (event) => {
    console.log(event.type, event.task ?? event.agent ?? '')
  },
})

Common event types include task_start, task_complete, task_retry, task_skipped, agent_start, agent_complete, budget_exceeded, and error. A task_retry event's data.nextDelayMs is the actual, post-jitter delay before the next attempt, not the nominal backoff schedule.

Trace Spans

Use onTrace when you need structured spans for LLM calls, tool executions, and tasks. Each span carries a runId, its own spanId, an optional parentId, durations, token counts, and best-effort-redacted tool I/O.

const orchestrator = new OpenMultiAgent({
  onTrace: async (span) => {
    await traceSink.write(span)
  },
})

Forward trace spans to OpenTelemetry, Datadog, Honeycomb, Langfuse, or your own run database only after deciding what data is safe for that sink. See integrations/trace-observability for a runnable example.

Span parentage is best-effort and uses the causal structure known to the runtime. In team runs, worker agent spans point to their task span, and LLM/tool/stream spans point to the agent span. Root spans such as top-level agent runs omit parentId.

Post-Run Dashboard

renderTeamRunDashboard(result) returns a static HTML page that visualizes the executed task DAG with timing, token usage, per-task status, and task details.

import { writeFileSync } from 'node:fs'
import { renderTeamRunDashboard } from '@open-multi-agent/core'

const result = await orchestrator.runTeam(team, goal)
writeFileSync('run.html', renderTeamRunDashboard(result))

The library does not write files by itself. The CLI can write dashboard HTML for you with oma run --dashboard; see docs/cli.md.

The generated dashboard is self-contained and does not load remote scripts, stylesheets, or fonts. Sensitive-looking values in the embedded run payload are redacted before rendering.

What to Persist

For production runs, persist enough data to reconstruct a failure without replaying the entire job:

  • TeamRunResult.tasks for the executed DAG and task states.
  • TeamRunResult.totalTokenUsage for cost attribution.
  • onTrace spans for LLM calls and tool executions, keyed by runId + spanId.
  • The rendered dashboard HTML when you need a shareable post-mortem artifact.

Redaction scope. The redaction noted above applies to telemetry — trace spans and the dashboard payload. It does not cover persisted run state: shared-memory writes and checkpoint saves store agent output verbatim. To scrub secrets there, wrap the durable store with RedactingStore.