There are 8 control objectives here. Each one states what it asks for, the risk it addresses and how it is measured. It then says what each risk tier requires, what evidence proves it, and who may check that evidence. The Operating Model page explains the tiers.
Give every release a version, keep the agent's identity across versions, and keep the old versions on record so that anyone can see what ran when.
- Risk addressed
- Nobody can say which version did what
- Key metric
- Share of running agents whose version is on record
What each tier requires
- Tier 1 Exploratory
The experiment notes its version when it changes.
- Tier 2 Operational
Every release has a version number and a fingerprint.
- Tier 3 Critical
Every past version stays retrievable for the retention period, with its build record.
- Tier 4 Strategic
The version history is part of the evidence an outside auditor can read.
- Evidence
- The version history with fingerprints, dates and build records.
- Who may check it
- Internal audit, an outside auditor and a partner.
CHG-02Impact on dependents
Before a change, name everything that depends on the agent and answer for each one. A change that is safe for its owner can still break the agents and people that rely on it.
- Risk addressed
- A change that breaks other teams' work
- Key metric
- Share of changes with a completed impact analysis
What each tier requires
- Tier 1 Exploratory
The builder checks that no one else uses the experiment.
- Tier 2 Operational
The change lists the dependents recorded in the catalog.
- Tier 3 Critical
Each dependent's owner is told before release and may object, and the analysis is recorded.
- Tier 4 Strategic
Dependents outside the organization are told under the terms of their contracts, with the notice those contracts require.
- Evidence
- The impact analysis listing each dependent, the notice given and any objection.
- Who may check it
- Internal audit and a partner.
CHG-03Approved with a reason
Give each change a written reason and an approver, so that every change to a running agent can be explained later.
- Risk addressed
- Changes that nobody can explain
- Key metric
- Share of changes with a reason and an approver
What each tier requires
- Tier 1 Exploratory
The builder writes a one-line reason.
- Tier 2 Operational
Each change has a reason and is approved by the agent's owner.
- Tier 3 Critical
Changes are approved by someone other than the person who made them.
- Tier 4 Strategic
Changes that alter what the agent may do or reach are approved by the governance council.
- Evidence
- The change record with its reason, approver and date.
- Who may check it
- Internal audit and an outside auditor.
CHG-04Updates proven before promotion
Release a new version as a candidate first, prove it against its tests and evaluations, and only then promote it to replace the running version.
- Risk addressed
- A new version that is worse than the old one
- Key metric
- Share of updates promoted after a passed candidate run
What each tier requires
- Tier 1 Exploratory
The experiment may be replaced directly.
- Tier 2 Operational
A new version passes its tests before it replaces the running one.
- Tier 3 Critical
The candidate runs beside the current version, and evaluation results are compared before promotion.
- Tier 4 Strategic
Promotion is staged, and an automatic return to the previous version is ready if quality falls.
- Evidence
- The candidate's test and evaluation results, compared with the current version's, and the promotion record.
- Who may check it
- Internal audit and an outside auditor.
CHG-05Reconfiguration as a change
Treat new settings, credentials, ceilings and permissions as changes with the same record as a new version, even when the code stays the same.
- Risk addressed
- Settings changed without anyone knowing
- Key metric
- Share of configuration changes with a change record
What each tier requires
- Tier 1 Exploratory
Configuration changes are noted by the builder.
- Tier 2 Operational
Each configuration change is recorded with who made it.
- Tier 3 Critical
Changes to limits, credentials and permissions need approval before they take effect.
- Tier 4 Strategic
Configuration is held as code, and changes go through the same pipeline as releases.
- Evidence
- The configuration history with who changed what, when and with whose approval.
- Who may check it
- Internal audit and an outside auditor.
CHG-06Rollback on purpose
Make going back a version a recorded step that has been practised, and include a plan for undoing what the bad version did, since moving the code back does not undo its effects.
- Risk addressed
- A failed change with no way back
- Key metric
- Rollback test success rate
What each tier requires
- Tier 1 Exploratory
The experiment can be replaced by its previous copy.
- Tier 2 Operational
Every release can be rolled back, and rollbacks are recorded.
- Tier 3 Critical
Rollback is tested before release, and the release names how its effects would be undone.
- Tier 4 Strategic
Rollback and the undoing of effects are rehearsed each quarter.
- Evidence
- Rollback records and test results, and the plan for undoing effects in each release.
- Who may check it
- Internal audit.
CHG-07Changes that go back to design
Send any change that alters what an agent does, refuses, reaches or may spend back through Design and Build, rather than letting it through as a routine update. This is how the lifecycle becomes a loop.
- Risk addressed
- A new agent arriving under the name of an old one
- Key metric
- Share of scope changes that went back through design review
What each tier requires
- Tier 1 Exploratory
An experiment that changes its purpose is registered again.
- Tier 2 Operational
A change to the agent's declaration needs a design review.
- Tier 3 Critical
A change to the agent's declaration, reach or limits goes back through Design and Build with a new security review.
- Tier 4 Strategic
Such a change also returns to Plan, so that its objective and obligations are confirmed again.
- Evidence
- The comparison of the old and new declarations, and the design review it triggered.
- Who may check it
- Internal audit and an outside auditor.
Allow an urgent change to skip steps only through a named emergency path, and review it within a set time afterwards.
- Risk addressed
- Emergencies used to avoid review
- Key metric
- Emergency changes reviewed within the set time
What each tier requires
- Tier 1 Exploratory
No emergency path is needed in a sandbox.
- Tier 2 Operational
Emergency changes are marked as such and reviewed within a week.
- Tier 3 Critical
Emergency changes need a named approver at the time and a full review within two working days.
- Tier 4 Strategic
Emergency changes are reported to the governance council, and a pattern of them triggers a review of the agent.
- Evidence
- The emergency change log, with the approver at the time and the later review.
- Who may check it
- Internal audit and an outside auditor.