feat(orchestrator): add declared governance and post-execution floor (#420)

This commit is contained in:
Jack Chen authored and GitHub committed 2026-07-23 10:28:48 +08:00
1 parent 7dcde7d8fc
commit 26be338ff7
9 files changed
+816 -2

No files matched your search

+22
View File
@@ -114,6 +114,28 @@ and run duration because those facts are not fields on that result type. When
the available result or trace cannot establish a fact, the builder returns an
empty array or `null` for that field and sets `partial: true`.
### Post-execution governance conclusion
Every OMA-produced `TeamRunResult` carries `governanceConclusion`:
- `satisfied` — a `governanceIntent: 'required'` run executed every declared
role, every adjacent `requiredOrder` pair appears in the observed execution
order and is connected by a dependency path, and declarations with two or
more roles produced `independentReviewOccurred: true`;
- `unsatisfied` — at least one of those required execution facts is absent;
- `not-applicable` — governance was omitted, `none`, or `preferred`.
The conclusion is computed by passing the result through
`buildExecutionReceipt()` and then `evaluateGovernance()`. It is therefore a
post-execution topology verdict, not a judgment about answer wording or quality.
Review labels or approval markers in model output cannot satisfy the gate.
`success` is intentionally independent: it continues to mean that the run
finished without runtime errors. A run can be `success: true` and
`governanceConclusion: 'unsatisfied'`; applications that require governance
must gate on the latter field. In particular, multiple required roles without
a dependency-backed review chain do not count as independent review.
## TraceRecord schema v2
The core package exports the `TraceRecord` schema used by the internal
+36
View File
@@ -2,6 +2,42 @@
Agents can be configured with fine-grained tool access control using presets, allowlists, and denylists.
## Declared governance roles in `runTeam()`
Tool and credential boundaries are often tied to named agents. When a goal must actually pass through specific roster roles, declare the topology structurally on the `runTeam()` call:
```typescript
const result = await orchestrator.runTeam(team, 'Review this change before release.', {
governanceIntent: 'required',
requiredRoles: ['reviewer', 'security'],
requiredOrder: ['reviewer', 'security'],
})
if (result.governanceConclusion !== 'satisfied') {
throw new Error('Required governance was not satisfied by the executed topology.')
}
```
For both `required` and `preferred`, OMA skips coordinator decomposition and the simple-goal single-agent short circuit. It creates one task for every `requiredRoles` entry, keeps each task assigned to that roster agent, and uses `requiredOrder` to create dependency edges. Each task receives the original goal unchanged; the agent's `systemPrompt`, tools, and credentials define the role. Downstream tasks receive prerequisite outputs through dependency-scoped memory.
The goal text is not inspected to choose this topology. The same declaration therefore produces the same roles and dependency order for English, Chinese, or any other language. Every `requiredRoles` name must exist in the team roster, and `requiredOrder`, when present, must be a permutation of those roles. Invalid declarations throw before any agent runs.
Use `governanceIntent: 'none'` to opt into the existing automatic `runTeam()` route explicitly. Omitting `governanceIntent` also preserves the existing behavior, including simple-goal short circuit and coordinator-generated task planning.
After execution, required declarations are checked against the execution
receipt. The `governanceConclusion` value is `satisfied`, `unsatisfied`, or
`not-applicable`. `required` is the only enforced intent;
`preferred`, `none`, and an omitted intent return `not-applicable`. An
`unsatisfied` conclusion means that a required role, dependency path/order, or
independent review fact was not observed. It does not rewrite `result.success`,
which keeps its existing runtime-error meaning, so governance-sensitive callers
must check `governanceConclusion` explicitly.
The gate reads only the structured execution topology produced by
`buildExecutionReceipt()`. Agent answer text cannot prove that another role ran
or that an independent review occurred, even if it contains reviewer names,
approval labels, or audit markers.
## Built-in tools are opt-in (default-deny)
Built-in tools — `bash` and the filesystem tools (`file_read`, `file_write`, `file_edit`, `grep`, `glob`) — are **default-deny**. An agent receives a built-in tool only when it is granted explicitly via `tools` (an allowlist of names) or `toolPreset`. An agent that sets **neither** resolves to **zero** built-in tools:
+12
View File
@@ -109,6 +109,18 @@ Set `OPENAI_API_KEY` for this example. For other hosted or local models, see [Pr
Use `planOnly` to inspect a generated task graph before execution, then `createPlanArtifact()` and `runFromPlan()` to replay it. `runConsensus()` adds a proposer→judge verification loop when one answer needs extra scrutiny.
When an application must enforce named independent roles, declare that governance intent instead of relying on wording in the goal:
```typescript
const governed = await orchestrator.runTeam(team, 'Review the evidence and assess the risk.', {
governanceIntent: 'required',
requiredRoles: ['researcher', 'analyst'],
requiredOrder: ['researcher', 'analyst'],
})
```
`required` and `preferred` both bypass automatic decomposition and the simple-goal short circuit. OMA creates one task per declared roster name, assigns it to that agent, and chains tasks in `requiredOrder`; dependency outputs are passed to downstream roles. The topology comes only from these structured fields, so equivalent goals in different languages use the same roles and order. `none` or an omitted `governanceIntent` preserves the existing automatic `runTeam()` behavior.
## Capabilities
| Capability | What you get |
+3
View File
@@ -55,6 +55,8 @@
// ---------------------------------------------------------------------------
export { OpenMultiAgent, executeWithRetry, computeRetryDelay } from './orchestrator/orchestrator.js'
export { evaluateGovernance } from './orchestrator/governance.js'
export type { GovernanceDeclaration } from './orchestrator/governance.js'
export { Scheduler } from './orchestrator/scheduler.js'
export type { SchedulingStrategy } from './orchestrator/scheduler.js'
@@ -282,6 +284,7 @@ export type {
// Team
TeamConfig,
TeamRunResult,
GovernanceConclusion,
RunMetrics,
RunTeamOptions,
RunTasksOptions,
@@ -0,0 +1,195 @@
import type {
AgentConfig,
GovernanceConclusion,
RunTaskSpec,
RunTeamOptions,
} from '../types.js'
import type { ExecutionReceipt } from '../observability/execution-receipt.js'
/** The structured governance fields accepted by {@link RunTeamOptions}. */
export type GovernanceDeclaration = Pick<
RunTeamOptions,
'governanceIntent' | 'requiredRoles' | 'requiredOrder'
>
/**
* Compare required governance with an execution receipt.
*
* The evaluator deliberately has no access to agent output text. Required
* order is established by both observed start order and a dependency path
* between every adjacent declared role.
*/
export function evaluateGovernance(
declaration: GovernanceDeclaration | undefined,
receipt: ExecutionReceipt,
): GovernanceConclusion {
if (declaration?.governanceIntent !== 'required') return 'not-applicable'
const requiredRoles = declaration.requiredRoles ?? []
if (requiredRoles.length === 0 || new Set(requiredRoles).size !== requiredRoles.length) {
return 'unsatisfied'
}
const executedRoles = new Set(receipt.rolesExecuted)
if (requiredRoles.some((role) => !executedRoles.has(role))) return 'unsatisfied'
const requiredOrder = declaration.requiredOrder
if (requiredOrder !== undefined) {
if (!isPermutation(requiredOrder, requiredRoles)) return 'unsatisfied'
if (!matchesRequiredOrder(requiredOrder, receipt, executedRoles)) return 'unsatisfied'
}
if (requiredRoles.length >= 2 && !receipt.independentReviewOccurred) {
return 'unsatisfied'
}
return 'satisfied'
}
/**
* Build the explicit role topology requested by a structured governance
* declaration. `undefined` means the caller selected the legacy automatic
* `runTeam()` route.
*/
export function buildGovernanceTaskSpecs(
goal: string,
agentConfigs: readonly AgentConfig[],
options?: RunTeamOptions,
): readonly RunTaskSpec[] | undefined {
const intent = options?.governanceIntent
if (intent !== 'required' && intent !== 'preferred') return undefined
const requiredRoles = options?.requiredRoles ?? []
if (requiredRoles.length === 0) {
throw new Error(
`runTeam governanceIntent '${intent}' requires at least one role in requiredRoles.`,
)
}
const duplicateRoles = findDuplicates(requiredRoles)
if (duplicateRoles.length > 0) {
throw new Error(
`runTeam requiredRoles must contain unique team agent names; duplicate role(s): ${duplicateRoles.join(', ')}.`,
)
}
const rosterNames = new Set(agentConfigs.map((agent) => agent.name))
const unknownRoles = requiredRoles.filter((role) => !rosterNames.has(role))
if (unknownRoles.length > 0) {
throw new Error(
`runTeam requiredRoles must exist in the team roster; unknown role(s): ${unknownRoles.join(', ')}.`,
)
}
const requiredOrder = options?.requiredOrder
if (requiredOrder !== undefined) {
const declaredRoles = new Set(requiredRoles)
const unknownOrderRoles = requiredOrder.filter((role) => !declaredRoles.has(role))
if (unknownOrderRoles.length > 0) {
throw new Error(
`runTeam requiredOrder may reference only requiredRoles; invalid role(s): ${unknownOrderRoles.join(', ')}.`,
)
}
const duplicateOrderRoles = findDuplicates(requiredOrder)
if (duplicateOrderRoles.length > 0) {
throw new Error(
`runTeam requiredOrder must contain each required role once; duplicate role(s): ${duplicateOrderRoles.join(', ')}.`,
)
}
if (requiredOrder.length !== requiredRoles.length) {
const orderedRoles = new Set(requiredOrder)
const missingRoles = requiredRoles.filter((role) => !orderedRoles.has(role))
throw new Error(
`runTeam requiredOrder must be a permutation of requiredRoles; missing role(s): ${missingRoles.join(', ')}.`,
)
}
}
const executionOrder = requiredOrder ?? requiredRoles
// Keep task text role-neutral: assignment selects the roster agent, whose
// systemPrompt owns the role semantics. Titles exist only for dependency IDs.
const titleByRole = new Map(
executionOrder.map((role, index) => [role, `Governance task ${index + 1}`]),
)
return executionOrder.map((role, index) => ({
title: titleByRole.get(role)!,
description: goal,
assignee: role,
...(index > 0 && requiredOrder !== undefined
? { dependsOn: [titleByRole.get(executionOrder[index - 1]!)!] }
: {}),
memoryScope: 'dependencies',
}))
}
function findDuplicates(values: readonly string[]): string[] {
const seen = new Set<string>()
const duplicates = new Set<string>()
for (const value of values) {
if (seen.has(value)) duplicates.add(value)
seen.add(value)
}
return [...duplicates]
}
function isPermutation(values: readonly string[], expected: readonly string[]): boolean {
if (values.length !== expected.length || new Set(values).size !== values.length) return false
const expectedValues = new Set(expected)
return values.every((value) => expectedValues.has(value))
}
function matchesRequiredOrder(
requiredOrder: readonly string[],
receipt: ExecutionReceipt,
executedRoles: ReadonlySet<string>,
): boolean {
const positionByRole = new Map(receipt.executionOrder.map((role, index) => [role, index]))
const adjacency = new Map<string, Set<string>>()
for (const edge of receipt.dependencyEdges) {
if (!executedRoles.has(edge.from) || !executedRoles.has(edge.to)) continue
const targets = adjacency.get(edge.from) ?? new Set<string>()
targets.add(edge.to)
adjacency.set(edge.from, targets)
}
for (let index = 1; index < requiredOrder.length; index++) {
const predecessor = requiredOrder[index - 1]!
const successor = requiredOrder[index]!
const predecessorPosition = positionByRole.get(predecessor)
const successorPosition = positionByRole.get(successor)
if (
predecessorPosition === undefined
|| successorPosition === undefined
|| predecessorPosition >= successorPosition
|| !hasDependencyPath(predecessor, successor, adjacency)
) {
return false
}
}
return true
}
function hasDependencyPath(
from: string,
to: string,
adjacency: ReadonlyMap<string, ReadonlySet<string>>,
): boolean {
const pending = [from]
const visited = new Set([from])
while (pending.length > 0) {
const current = pending.shift()!
for (const next of adjacency.get(current) ?? []) {
if (next === to) return true
if (visited.has(next)) continue
visited.add(next)
pending.push(next)
}
}
return false
}
+41 -2
View File
@@ -88,6 +88,7 @@ import {
type RestoreMetadataResolution,
} from '../observability/identity.js'
import { classifyRunFailure, statusOnly } from '../observability/status.js'
import { buildExecutionReceipt } from '../observability/execution-receipt.js'
import {
createTraceRuntime,
LEGACY_TRACE_METADATA_ONLY,
@@ -129,6 +130,11 @@ import {
} from './agent-config.js'
import { isSimpleGoal, selectBestAgent } from './short-circuit.js'
import { executeQueue, saveRunCheckpoint } from './task-execution.js'
import {
buildGovernanceTaskSpecs,
evaluateGovernance,
type GovernanceDeclaration,
} from './governance.js'
import { runConsensusCore, applyConsensusDefaults, type ConsensusAgentDefaults } from './consensus.js'
import {
createOnlineEvaluator,
@@ -453,6 +459,29 @@ export class OpenMultiAgent {
options?: RunTeamOptions,
): Promise<TeamRunResult> {
const pendingEvaluation = this.beginOnlineEvaluation(goal)
const agentConfigs = team.getAgents()
const governanceTaskSpecs = buildGovernanceTaskSpecs(goal, agentConfigs, options)
if (options?.governanceIntent === 'required' && governanceTaskSpecs === undefined) {
throw new Error('Invariant violation: required runTeam governance must use an explicit task topology.')
}
if (governanceTaskSpecs !== undefined) {
const queue = new TaskQueue()
loadSpecsIntoQueue(governanceTaskSpecs, agentConfigs, queue)
return this.executeExplicitTaskQueue(
team,
queue,
options,
goal,
undefined,
undefined,
undefined,
undefined,
undefined,
pendingEvaluation,
options,
)
}
const { identity, metadata } = createRunFacts(identityOptionsForRun(options))
const traceRuntime = this.startTrace(identity, metadata)
const finish = (result: TeamRunResult): TeamRunResult => {
@@ -467,7 +496,6 @@ export class OpenMultiAgent {
this.completeOnlineEvaluation(pendingEvaluation, completedResult)
return completedResult
}
const agentConfigs = team.getAgents()
const coordinatorOverrides = options?.coordinator
// ------------------------------------------------------------------
@@ -480,7 +508,12 @@ export class OpenMultiAgent {
// The best-matching agent is selected via keyword affinity scoring
// (same algorithm as the `capability-match` scheduler strategy).
// ------------------------------------------------------------------
if (!options?.planOnly && agentConfigs.length > 0 && isSimpleGoal(goal)) {
if (
options?.governanceIntent !== 'required'
&& !options?.planOnly
&& agentConfigs.length > 0
&& isSimpleGoal(goal)
) {
const bestAgent = selectBestAgent(goal, agentConfigs)
// Use buildAgent() + agent.run() directly instead of this.runAgent()
@@ -1456,6 +1489,7 @@ export class OpenMultiAgent {
identity?: RunIdentity,
restoreMetadata?: RestoreMetadataResolution,
pendingEvaluation?: PendingOnlineEvaluation,
governanceDeclaration?: GovernanceDeclaration,
): Promise<TeamRunResult> {
const newRunFacts = identity === undefined
? createRunFacts(identityOptionsForRun(options))
@@ -1586,6 +1620,10 @@ export class OpenMultiAgent {
)
const completedResult: TeamRunResult = {
...result,
governanceConclusion: evaluateGovernance(
governanceDeclaration,
buildExecutionReceipt(result),
),
...(metadata !== undefined ? { metadata } : {}),
}
traceRuntime?.close({
@@ -1747,6 +1785,7 @@ export class OpenMultiAgent {
return {
success: status.code === 'ok',
governanceConclusion: 'not-applicable',
identity,
status,
...(errorInfo !== undefined ? { errorInfo } : {}),
+33
View File
@@ -1101,6 +1101,28 @@ export interface RunTasksOptions extends RunIdentityOptions {
*/
export interface RunTeamOptions extends RunTasksOptions {
readonly coordinator?: CoordinatorConfig
/**
* Optional structured governance signal for this goal.
*
* `'required'` and `'preferred'` both bypass coordinator decomposition and
* the simple-goal short circuit. Instead, `runTeam()` executes one task per
* {@link requiredRoles} entry using the declared roster assignments.
* `'none'` and an omitted value preserve the existing automatic routing
* behavior.
*/
readonly governanceIntent?: 'required' | 'preferred' | 'none'
/**
* Agent names from the team roster that must execute independent tasks when
* {@link governanceIntent} is `'required'` or `'preferred'`.
*/
readonly requiredRoles?: readonly string[]
/**
* Optional execution order for {@link requiredRoles}. When provided, it must
* be a permutation of `requiredRoles`; each role depends on the previous one.
* When omitted, the declared role tasks have no dependencies and may run in
* parallel.
*/
readonly requiredOrder?: readonly string[]
/**
* When true, the coordinator decomposes the goal but no task agents run.
* The returned {@link TeamRunResult} has `planOnly: true`, `success: true`,
@@ -1148,6 +1170,9 @@ export interface RunTeamOptions extends RunTasksOptions {
readonly verifyJudges?: readonly AgentConfig[]
}
/** Post-execution result of comparing required governance with execution facts. */
export type GovernanceConclusion = 'satisfied' | 'unsatisfied' | 'not-applicable'
/** Run-level aggregated metrics summary. */
export interface RunMetrics {
readonly totalTokens: TokenUsage
@@ -1164,6 +1189,14 @@ export interface RunMetrics {
/** Aggregated result for a full team run. */
export interface TeamRunResult extends RunOutcomeFields {
readonly success: boolean
/**
* Post-execution governance verdict for this run.
*
* OMA-produced results always set this field. It remains optional on the
* interface so older serialized results and caller-created fixtures continue
* to type-check. `success` retains its existing runtime-error semantics.
*/
readonly governanceConclusion?: GovernanceConclusion
readonly goal?: string
readonly tasks?: readonly TaskExecutionRecord[]
/**
@@ -0,0 +1,293 @@
import { describe, expect, it } from 'vitest'
import {
buildExecutionReceipt,
evaluateGovernance,
OpenMultiAgent,
} from '../src/index.js'
import type {
AgentConfig,
AgentRunResult,
AgentTrace,
ExecutionReceipt,
GovernanceDeclaration,
LLMAdapter,
LLMResponse,
RunTeamOptions,
TaskExecutionRecord,
} from '../src/index.js'
const requiredChain = {
governanceIntent: 'required',
requiredRoles: ['reviewer', 'security'],
requiredOrder: ['reviewer', 'security'],
} as const satisfies GovernanceDeclaration
function receipt(overrides: Partial<ExecutionReceipt> = {}): ExecutionReceipt {
return {
mode: 'single',
rolesExecuted: [],
executionOrder: [],
dependencyEdges: [],
independentRolesCount: 0,
independentReviewOccurred: false,
totalTokens: { input: 0, output: 0 },
durationMs: 0,
partial: false,
...overrides,
}
}
function textAdapter(output: string): LLMAdapter {
return {
name: 'governance-floor-test',
async chat(): Promise<LLMResponse> {
return {
id: `response-${output}`,
content: [{ type: 'text', text: output }],
model: 'mock-model',
stop_reason: 'end_turn',
usage: { input_tokens: 1, output_tokens: 1 },
}
},
async *stream() {
yield { type: 'done' as const, data: {} }
},
}
}
function errorAdapter(message: string): LLMAdapter {
return {
name: 'governance-floor-error-test',
async chat(): Promise<LLMResponse> {
throw new Error(message)
},
async *stream() {
yield { type: 'done' as const, data: {} }
},
}
}
function agent(name: string, adapter: LLMAdapter): AgentConfig {
return {
name,
model: 'mock-model',
systemPrompt: `You are the ${name}.`,
adapter,
}
}
async function runTeam(
options?: RunTeamOptions,
reviewerAdapter: LLMAdapter = textAdapter('reviewer output'),
securityAdapter: LLMAdapter = textAdapter('security output'),
) {
const orchestrator = new OpenMultiAgent({ defaultModel: 'mock-model' })
const team = orchestrator.createTeam('governance-floor-team', {
name: 'governance-floor-team',
agents: [
agent('reviewer', reviewerAdapter),
agent('security', securityAdapter),
],
sharedMemory: true,
})
return orchestrator.runTeam(team, 'Say hello.', options)
}
function topology(tasks: readonly TaskExecutionRecord[] | undefined) {
const records = tasks ?? []
const assigneeById = new Map(records.map((task) => [task.id, task.assignee]))
return records.map((task) => ({
assignee: task.assignee,
status: task.status,
dependsOn: task.dependsOn.map((dependency) => assigneeById.get(dependency)),
executed: task.metrics !== undefined,
}))
}
describe('evaluateGovernance', () => {
it('is satisfied when all required roles execute in the declared review chain', () => {
expect(evaluateGovernance(requiredChain, receipt({
mode: 'multi-agent',
rolesExecuted: ['reviewer', 'security'],
executionOrder: ['reviewer', 'security'],
dependencyEdges: [{ from: 'reviewer', to: 'security' }],
independentRolesCount: 2,
independentReviewOccurred: true,
}))).toBe('satisfied')
})
it('is unsatisfied when one required role did not execute', () => {
const declaration = {
governanceIntent: 'required',
requiredRoles: ['reviewer', 'security', 'operator'],
requiredOrder: ['reviewer', 'security', 'operator'],
} as const satisfies GovernanceDeclaration
expect(evaluateGovernance(declaration, receipt({
mode: 'multi-agent',
rolesExecuted: ['reviewer', 'security'],
executionOrder: ['reviewer', 'security'],
dependencyEdges: [{ from: 'reviewer', to: 'security' }],
independentRolesCount: 2,
independentReviewOccurred: true,
}))).toBe('unsatisfied')
})
it.each([
{
name: 'parallel roles with no dependency edge',
executionOrder: ['reviewer', 'security'],
dependencyEdges: [],
},
{
name: 'roles executing in the opposite dependency order',
executionOrder: ['security', 'reviewer'],
dependencyEdges: [{ from: 'security', to: 'reviewer' }],
},
{
name: 'the declared start order with a reversed dependency edge',
executionOrder: ['reviewer', 'security'],
dependencyEdges: [{ from: 'security', to: 'reviewer' }],
},
])('is unsatisfied for $name', ({ executionOrder, dependencyEdges }) => {
expect(evaluateGovernance(requiredChain, receipt({
mode: 'multi-agent',
rolesExecuted: ['reviewer', 'security'],
executionOrder,
dependencyEdges,
independentRolesCount: 2,
independentReviewOccurred: dependencyEdges.length > 0,
}))).toBe('unsatisfied')
})
it('accepts a dependency path between adjacent required roles', () => {
expect(evaluateGovernance(requiredChain, receipt({
mode: 'multi-agent',
rolesExecuted: ['reviewer', 'analyst', 'security'],
executionOrder: ['reviewer', 'analyst', 'security'],
dependencyEdges: [
{ from: 'reviewer', to: 'analyst' },
{ from: 'analyst', to: 'security' },
],
independentRolesCount: 3,
independentReviewOccurred: true,
}))).toBe('satisfied')
})
it('is unsatisfied when two roles are required but only one agent executed', () => {
expect(evaluateGovernance(requiredChain, receipt({
rolesExecuted: ['reviewer'],
executionOrder: ['reviewer'],
independentRolesCount: 1,
independentReviewOccurred: false,
}))).toBe('unsatisfied')
})
it('ignores review markers in answer text and judges only the I1 receipt', () => {
const answer: AgentRunResult = {
success: true,
output: 'reviewer approved; security approved; INDEPENDENT_REVIEW_COMPLETE',
messages: [],
tokenUsage: { input_tokens: 1, output_tokens: 1 },
toolCalls: [],
}
const trace: AgentTrace = {
type: 'agent',
runId: 'run-1',
spanId: 'span-1',
agent: 'reviewer',
startMs: 0,
endMs: 1,
durationMs: 1,
turns: 1,
tokens: { input_tokens: 1, output_tokens: 1 },
toolCalls: 0,
}
expect(evaluateGovernance(requiredChain, buildExecutionReceipt(answer, [trace])))
.toBe('unsatisfied')
})
it.each([
{ name: 'omitted intent', declaration: {} },
{
name: 'none intent',
declaration: { governanceIntent: 'none', requiredRoles: ['reviewer', 'security'] },
},
{
name: 'preferred intent',
declaration: { governanceIntent: 'preferred', requiredRoles: ['reviewer', 'security'] },
},
] as const)('is not applicable for $name', ({ declaration }) => {
expect(evaluateGovernance(declaration, receipt())).toBe('not-applicable')
})
})
describe('runTeam governance floor', () => {
it('returns satisfied for a required run with the declared executed topology', async () => {
const result = await runTeam(requiredChain)
const execution = buildExecutionReceipt(result)
expect(result.success).toBe(true)
expect(result.governanceConclusion).toBe('satisfied')
expect(result.tasks?.some((task) => task.id === 'short-circuit')).toBe(false)
expect(topology(result.tasks)).toEqual([
{ assignee: 'reviewer', status: 'completed', dependsOn: [], executed: true },
{ assignee: 'security', status: 'completed', dependsOn: ['reviewer'], executed: true },
])
expect(execution).toMatchObject({
rolesExecuted: ['reviewer', 'security'],
executionOrder: ['reviewer', 'security'],
dependencyEdges: [{ from: 'reviewer', to: 'security' }],
independentReviewOccurred: true,
})
})
it('returns unsatisfied when an upstream required task fails and a downstream role never runs', async () => {
const result = await runTeam(
requiredChain,
errorAdapter('reviewer failed'),
textAdapter('security output'),
)
const execution = buildExecutionReceipt(result)
expect(result.success).toBe(false)
expect(result.governanceConclusion).toBe('unsatisfied')
expect(topology(result.tasks)).toEqual([
{ assignee: 'reviewer', status: 'failed', dependsOn: [], executed: true },
{ assignee: 'security', status: 'failed', dependsOn: ['reviewer'], executed: false },
])
expect(execution.executionOrder).not.toEqual(['reviewer', 'security'])
})
it('keeps runtime success separate from an unsatisfied governance conclusion', async () => {
const result = await runTeam({
governanceIntent: 'required',
requiredRoles: ['reviewer', 'security'],
})
const execution = buildExecutionReceipt(result)
expect(result.success).toBe(true)
expect(result.governanceConclusion).toBe('unsatisfied')
expect(execution.dependencyEdges).toEqual([])
expect(execution.independentReviewOccurred).toBe(false)
})
it('returns not-applicable and preserves the existing simple-goal route when undeclared', async () => {
const result = await runTeam()
expect(result.success).toBe(true)
expect(result.governanceConclusion).toBe('not-applicable')
expect(result.tasks?.map((task) => task.id)).toEqual(['short-circuit'])
})
it('does not enforce the floor for preferred governance', async () => {
const result = await runTeam({
...requiredChain,
governanceIntent: 'preferred',
})
expect(result.success).toBe(true)
expect(result.governanceConclusion).toBe('not-applicable')
})
})
@@ -0,0 +1,181 @@
import { describe, expect, it } from 'vitest'
import { OpenMultiAgent } from '../src/orchestrator/orchestrator.js'
import type {
AgentConfig,
LLMAdapter,
LLMMessage,
LLMResponse,
RunTeamOptions,
TaskExecutionRecord,
} from '../src/types.js'
function userPrompt(messages: LLMMessage[]): string {
const message = [...messages].reverse().find((candidate) => candidate.role === 'user')
return (message?.content ?? [])
.filter((block): block is { type: 'text'; text: string } => block.type === 'text')
.map((block) => block.text)
.join('\n')
}
function responseAdapter(reply: string, captures: string[] = []): LLMAdapter {
return {
name: 'governance-test',
async chat(messages): Promise<LLMResponse> {
captures.push(userPrompt(messages))
return {
id: `response-${captures.length}`,
content: [{ type: 'text', text: reply }],
model: 'mock-model',
stop_reason: 'end_turn',
usage: { input_tokens: 1, output_tokens: 1 },
}
},
async *stream() {
yield { type: 'done' as const, data: {} }
},
}
}
function agent(name: string, reply: string, captures?: string[]): AgentConfig {
return {
name,
model: 'mock-model',
systemPrompt: `You are the ${name}.`,
adapter: responseAdapter(reply, captures),
}
}
function normalizeTopology(tasks: readonly TaskExecutionRecord[] | undefined) {
const records = tasks ?? []
const assigneeById = new Map(records.map((task) => [task.id, task.assignee]))
return records.map((task) => ({
assignee: task.assignee,
dependsOn: task.dependsOn.map((dependencyId) => assigneeById.get(dependencyId)),
}))
}
async function runGovernedGoal(goal: string, options: RunTeamOptions) {
const orchestrator = new OpenMultiAgent({ defaultModel: 'mock-model' })
const team = orchestrator.createTeam('governance-team', {
name: 'governance-team',
agents: [
agent('reviewer', 'reviewer output'),
agent('security', 'security output'),
],
sharedMemory: true,
})
return orchestrator.runTeam(team, goal, options)
}
describe('runTeam declared governance intent', () => {
const requiredGovernance = {
governanceIntent: 'required',
requiredRoles: ['reviewer', 'security'],
requiredOrder: ['reviewer', 'security'],
} as const satisfies RunTeamOptions
it('executes every required role in the declared dependency order', async () => {
const reviewerPrompts: string[] = []
const securityPrompts: string[] = []
const orchestrator = new OpenMultiAgent({ defaultModel: 'mock-model' })
const team = orchestrator.createTeam('governance-team', {
name: 'governance-team',
agents: [
agent('reviewer', 'reviewer output', reviewerPrompts),
agent('security', 'security output', securityPrompts),
],
sharedMemory: true,
})
const goal = 'Review this transfer request.'
const result = await orchestrator.runTeam(team, goal, requiredGovernance)
expect(result.success).toBe(true)
expect(result.agentResults.has('coordinator')).toBe(false)
expect(result.agentResults.has('reviewer')).toBe(true)
expect(result.agentResults.has('security')).toBe(true)
expect(normalizeTopology(result.tasks)).toEqual([
{ assignee: 'reviewer', dependsOn: [] },
{ assignee: 'security', dependsOn: ['reviewer'] },
])
expect(result.tasks?.every((task) => task.description === goal)).toBe(true)
expect(result.tasks?.every((task) => task.memoryScope === 'dependencies')).toBe(true)
expect(reviewerPrompts).toHaveLength(1)
expect(securityPrompts[0]).toContain('## Context from prerequisite tasks')
expect(securityPrompts[0]).toContain('reviewer output')
})
it('builds the same role topology for equivalent English and Chinese goals', async () => {
const english = await runGovernedGoal(
'Review the key rotation and verify that it is safe.',
requiredGovernance,
)
const chinese = await runGovernedGoal(
'审核密钥轮换并确认其安全性。',
requiredGovernance,
)
expect(normalizeTopology(english.tasks)).toEqual(normalizeTopology(chinese.tasks))
expect(normalizeTopology(chinese.tasks)).toEqual([
{ assignee: 'reviewer', dependsOn: [] },
{ assignee: 'security', dependsOn: ['reviewer'] },
])
})
it('treats preferred as the same satisfying topology in this release', async () => {
const result = await runGovernedGoal('Check this request.', {
...requiredGovernance,
governanceIntent: 'preferred',
})
expect(normalizeTopology(result.tasks)).toEqual([
{ assignee: 'reviewer', dependsOn: [] },
{ assignee: 'security', dependsOn: ['reviewer'] },
])
})
it('leaves declared roles unordered when requiredOrder is omitted', async () => {
const result = await runGovernedGoal('Check this request.', {
governanceIntent: 'required',
requiredRoles: ['reviewer', 'security'],
})
expect(normalizeTopology(result.tasks)).toEqual([
{ assignee: 'reviewer', dependsOn: [] },
{ assignee: 'security', dependsOn: [] },
])
})
it('preserves the existing simple-goal route for none and an omitted intent', async () => {
const withoutIntent = await runGovernedGoal('Say hello.', {
requiredRoles: ['reviewer', 'security'],
requiredOrder: ['reviewer', 'security'],
})
const withNone = await runGovernedGoal('Say hello.', { governanceIntent: 'none' })
expect(normalizeTopology(withNone.tasks)).toEqual(normalizeTopology(withoutIntent.tasks))
expect(withNone.tasks?.map((task) => task.id)).toEqual(['short-circuit'])
expect(withNone.agentResults.has('coordinator')).toBe(false)
expect(withoutIntent.tasks?.map((task) => task.id)).toEqual(['short-circuit'])
expect(withoutIntent.agentResults.has('coordinator')).toBe(false)
})
it('rejects required roles that are absent from the team roster', async () => {
await expect(runGovernedGoal('Review this request.', {
governanceIntent: 'required',
requiredRoles: ['reviewer', 'auditor'],
})).rejects.toThrow(
'runTeam requiredRoles must exist in the team roster; unknown role(s): auditor.',
)
})
it('rejects requiredOrder entries that are not declared in requiredRoles', async () => {
await expect(runGovernedGoal('Review this request.', {
governanceIntent: 'required',
requiredRoles: ['reviewer'],
requiredOrder: ['security'],
})).rejects.toThrow(
'runTeam requiredOrder may reference only requiredRoles; invalid role(s): security.',
)
})
})