Section III, cross-cutting discipline

Assurance

A control is only as good as the evidence that it holds. Assurance makes sure every control objective has evidence that someone outside the team can check: internal audit, an outside auditor, or a partner whose agents are about to trust the organization's agents.

What makes evidence count

Every control objective on this site names its evidence and who may check it. For that evidence to prove anything, it has to meet three conditions.

Nobody can alter it unnoticed

Evidence comes from records that are only ever added to and are signed as they are written. Otherwise whoever can alter them is vouching for themselves.

It is current

A review from last spring says nothing about the version deployed this morning. Evidence is tied to the version it covers, and the checks repeat.

Others can read it

A log only the vendor can open, or a limit only the agent knows about, proves nothing to an auditor. Evidence is readable by the people entitled to check it.

Because every control has its own evidence, being under control is a list of checks rather than a single yes or no. If an agent passes every objective but one, the useful answer names the one it fails.

Who may check the evidence

Checker	What they check	What they may see
Internal audit	Whether controls are designed well and working, across every tier, on the audit plan's schedule.	All of the organization's evidence.
An outside auditor	Whether controls meet a regulation, a standard or a contract, and whether the organization's claims about them are true.	The evidence within the scope and period of the engagement.
A partner	The facts it relies on about an agent it deals with, such as its identity, owner, version, mandate and stop.	Only the evidence that concerns its own relationships.

Checker

What they check

What they may see

Internal audit

Whether controls are designed well and working, across every tier, on the audit plan's schedule.

All of the organization's evidence.

An outside auditor

Whether controls meet a regulation, a standard or a contract, and whether the organization's claims about them are true.

The evidence within the scope and period of the engagement.

A partner

The facts it relies on about an agent it deals with, such as its identity, owner, version, mandate and stop.

Only the evidence that concerns its own relationships.

Audits and attestations

Internal audits are planned by risk tier, so that Tier 3 and Tier 4 agents and relationships are audited more often than others. An outside auditor reviews high-tier agents when obligations require it, and attests each year to the controls of Tier 4 agents. An attestation is a signed statement that others can rely on, and its scope and date are part of it.

Findings from every audit are tracked to closure, with an owner and a date, and someone other than the owner confirms the fix.

Verification across company lines

When an organization's agent deals with another company's agent, it cannot audit the other company. It can still verify the facts it relies on, if those facts come as signed records: who the agent is, who owns it, which version is running, what mandate the current work carries, and whether the agent has been stopped. A partner checks these facts without needing access to the other company's systems.

The same applies to agents the organization relies on but does not host. Where an agent runs does not settle whether it is under control. An agent hosted by someone else, with no named owner and no spending limit, is not under control. An agent on its owner's own servers can be, if the owner keeps the records and lets others read them. A contract with the owner of an outside agent says which evidence the owner will provide.

The shared rules that apply where two organizations' agents meet are covered under Network.

Control objectives

There are 9 control objectives here. Each one states what it asks for, the risk it addresses and how it is measured. It then says what each risk tier requires, what evidence proves it, and who may check that evidence. The Operating Model page explains the tiers.

ASR-01

Evidence for every objective

Name, for every control objective that applies to an agent, the evidence that proves it and where that evidence is kept.

Risk addressed: Controls that cannot be shown to work
Key metric: Share of applicable objectives with named evidence

What each tier requires

Tier 1 Exploratory
The builder can show the sandbox limits.
Tier 2 Operational
The evidence for each applicable objective is listed in the agent's record.
Tier 3 Critical
Evidence is produced by the systems that enforce the control, without anyone writing it by hand.
Tier 4 Strategic
The evidence list is reviewed by internal audit each year.

Evidence: The agent's evidence list, mapping each objective to its records.
Who may check it: Internal audit and an outside auditor.

ASR-02

Immutable evidence

Draw evidence from records that are only ever added to and are signed as they are written, so that nobody, whether the agent, its builders, its owner or the platform, can alter one without it showing.

Risk addressed: Self-reported evidence
Key metric: Share of evidence held in signed, append-only records

What each tier requires

Tier 1 Exploratory
No requirement applies to an experiment.
Tier 2 Operational
Evidence is stored outside the agent.
Tier 3 Critical
Evidence is append-only and signed when it is written, so neither the agent nor its operators can change it unnoticed.
Tier 4 Strategic
Anyone outside the organization can verify the signatures and the chain of records without the team's help.

Evidence: The evidence store's settings and its own access records.
Who may check it: Internal audit, an outside auditor and a partner.

ASR-03

Current evidence

Keep evidence current. A review from last year says nothing about the version deployed this morning, so checks repeat with every change and on a schedule.

Risk addressed: Old evidence for a new version
Key metric: Share of evidence that matches the running version

What each tier requires

Tier 1 Exploratory
No requirement applies to an experiment.
Tier 2 Operational
Evidence is refreshed at each release.
Tier 3 Critical
Evidence is tied to the fingerprint of the version it covers, and stale evidence is flagged.
Tier 4 Strategic
Evidence is refreshed continuously where the control allows it.

Evidence: Evidence records with the version fingerprint and date of each.
Who may check it: Internal audit and an outside auditor.

ASR-04

Readable by someone outside the team

Make evidence readable by people who did not build the agent: internal audit, an outside auditor or a partner, each with the access the operating model allows.

Risk addressed: Evidence only the builders can read
Key metric: Requests for evidence answered without the team's help

What each tier requires

Tier 1 Exploratory
No requirement applies to an experiment.
Tier 2 Operational
Internal audit can read the evidence.
Tier 3 Critical
An outside auditor can be given read access to the evidence for a defined scope and period.
Tier 4 Strategic
Partners can verify the evidence that concerns them without seeing the rest.

Evidence: The access grants for evidence readers and the log of their reads.
Who may check it: Internal audit, an outside auditor and a partner.

ASR-05

Internal audit plan

Plan internal audits of agents by tier, so that higher-risk agents and relationships are audited more often.

Risk addressed: High-risk agents that go unaudited
Key metric: Share of the audit plan completed

What each tier requires

Tier 1 Exploratory
Experiments are sampled when the sandbox is audited.
Tier 2 Operational
Tier 2 agents are sampled in the yearly audit.
Tier 3 Critical
Tier 3 agents and relationships are audited each year.
Tier 4 Strategic
Tier 4 agents and relationships are audited each half year.

Evidence: The audit plan, the audit reports and the tracking of findings.
Who may check it: Internal audit and an outside auditor.

ASR-06

Outside audit and attestation

Have high-tier agents audited by an independent party, and issue attestations that others can rely on.

Risk addressed: Claims of control that nobody independent has checked
Key metric: Share of Tier 4 agents with a current outside attestation

What each tier requires

Tier 1 Exploratory
No outside audit is required.
Tier 2 Operational
No outside audit is required, though the agent is in scope of the organization's general audits.
Tier 3 Critical
An outside auditor reviews the agent's controls when obligations require it.
Tier 4 Strategic
An outside auditor attests to the agent's controls each year.

Evidence: The auditor's report or attestation, with its scope and date.
Who may check it: An outside auditor and a partner.

ASR-07

Verification across company lines

Let a partner verify the facts about an agent that it relies on, such as its identity, owner, version, mandate and stop, from signed records, without needing access to the other organization's systems.

Risk addressed: Trusting another company's agent on its word
Key metric: Share of cross-company relationships with verified evidence

What each tier requires

Tier 1 Exploratory
Sandbox agents do not deal with partners.
Tier 2 Operational
Partners can verify the agent's identity and owner.
Tier 3 Critical
Partners can verify identity, owner, version and the mandate for each piece of work.
Tier 4 Strategic
Partners can verify the agent's attestations and its stop status at any time.

Evidence: The signed facts offered to partners, and the record of partner verifications.
Who may check it: A partner and an outside auditor.

ASR-08

Findings tracked to closure

Track every audit and assurance finding to closure, with an owner and a date, and confirm the fix.

Risk addressed: Findings that are never fixed
Key metric: Findings open past their due date

What each tier requires

Tier 1 Exploratory
Sandbox findings are noted.
Tier 2 Operational
Findings have an owner and a due date.
Tier 3 Critical
Fixes are confirmed by someone other than the owner.
Tier 4 Strategic
Overdue findings are reported to the governance council.

Evidence: The findings log with owners, dates and confirmations.
Who may check it: Internal audit and an outside auditor.

ASR-09

Agents that run elsewhere

Apply the same assurance to agents the organization relies on but does not host, by requiring their owners to keep the records and let the organization read them.

Risk addressed: A hosted agent assumed to be under control because someone else runs it
Key metric: Share of outside agents with readable evidence

What each tier requires

Tier 1 Exploratory
Sandbox agents rely on no outside agents.
Tier 2 Operational
Outside agents have a named owner and a contract.
Tier 3 Critical
Outside agents' owners provide the evidence named in the contract.
Tier 4 Strategic
Outside agents' evidence is verified as ASR-07 describes, and attested where the tier requires.

Evidence: The contract terms on evidence and the evidence received.
Who may check it: Internal audit, an outside auditor and a partner.

Quick reference

ID	Objective	Key metric
ASR-01	Evidence for every objective	Share of applicable objectives with named evidence
ASR-02	Immutable evidence	Share of evidence held in signed, append-only records
ASR-03	Current evidence	Share of evidence that matches the running version
ASR-04	Readable by someone outside the team	Requests for evidence answered without the team's help
ASR-05	Internal audit plan	Share of the audit plan completed
ASR-06	Outside audit and attestation	Share of Tier 4 agents with a current outside attestation
ASR-07	Verification across company lines	Share of cross-company relationships with verified evidence
ASR-08	Findings tracked to closure	Findings open past their due date
ASR-09	Agents that run elsewhere	Share of outside agents with readable evidence