Home > Insights

Insights

How do you prove what an autonomous system actually did?

When a system acts on its own, describing your governance is not the same as proving what it did. The case for decision provenance: a record of what happened, on what basis, and under whose authority.

The case for decision provenance.

An autonomous system makes a decision. It approves something, declines something, escalates something, or acts on your behalf. Weeks or months later, someone asks you to account for it. A customer’s procurement team sends a questionnaire. An underwriter asks what sat around the model at renewal. A board wants to understand a decision that went wrong. A regulator asks what happened, and when.

You can describe your governance. You have policies, a control framework, perhaps an attestation or two. But the question underneath all of those requests is narrower and harder: show me what this system actually did, on what basis, under whose authority, and how you know.

That is a different question, and most organisations cannot answer it from the records they keep.

Describing governance is not the same as proving a decision

The shift the market is going through is often summarised as moving from “tell us” to “show us”. For years the ask was descriptive: tell us you have oversight, tell us you manage model change, tell us a human is in the loop. Control frameworks and attestations answer that ask. They describe intent and posture. They are about the system in general.

What is being asked now is specific. When a system acts thousands or millions of times, a good posture in general does not tell anyone what happened in the one case that matters. Control posture is not the same as decision accountability. The first describes how the system is meant to behave. The second reconstructs what it did, in a particular instance, and shows who was accountable for it.

Answer the specific question and four things have to be present at once: what was done, on what basis it was done, under whose authority, and how each of those is known. Leave any one of them out and the record does not hold up to a serious reviewer. The last one, how each fact is known, is the part ordinary logs and audit trails leave out, and it is usually the one that decides whether a record stands up later.

Decision provenance is the record that answers it

Decision provenance is a record of what an autonomous system did, on what basis, under whose authority, and how each item was captured, kept so the decision can be reconstructed after the fact. It is the object Custara prepares and preserves.

An honest record does not pretend every fact is equally solid. Some items are generated by the infrastructure that executed the decision. Some are captured close to the event. Some are reconstructed afterwards, and some are simply stated by a person. A record that holds up shows which is which, rather than flattening them into one confident-looking log. Provenance is not a claim that everything is airtight. It is an honest account of how each fact is known, so the person reading it can weigh it themselves.

Custara prepares, structures and preserves that record. It is the reliable record of what happened. It is not a judgement about whether what happened was good enough.

Like audit working papers, not like the audit

There is a familiar version of this. In finance, an auditor does not simply assert that the accounts are sound. Behind the opinion sits a body of working papers: the underlying evidence, organised so that someone else could follow how a conclusion was reached. The working papers are not the audit. They are what makes an audit possible.

Decision provenance is the working papers for an autonomous system’s decisions. The record itself does not decide whether the governance was adequate. That is a separate step, done by someone else. A standard can define what the record should contain and how evidence quality is described, which is the work of the Decision Standards Institute. An independent assessor can review the record separately, which is the role of Attestra. Custara prepares the record. It does not assess it, and it does not mark its own work.

That separation is not a limitation. It is the reason the record is worth anything. One record that the company, its board, its insurer and its regulator can each work from is only possible if the party that prepared it is not also the party grading it.

Who is asking, right now

The same underlying need shows up in different rooms.

An AI company selling into a regulated buyer hits an AI governance questionnaire, and finds itself rebuilding the answer by hand from policies, tickets and screenshots every time. The record it needs is the same one, over and over.

An underwriter is watching AI risk move into exclusions, sublimits and tighter evidence requests. Increasingly, what a submission can evidence is what the renewal turns on. The underwriter is not asking to be reassured. They are asking to see.

A board or a regulator arrives after something has gone wrong. At that point, governance you can describe is worth far less than a decision you can reconstruct. The question is no longer what your policy said. It is what the system did, and whether you can show it.

None of these readers wants a score from you. They want to see the underlying facts and reach their own conclusion. That is exactly what a provenance record is for.

The boundary is the point

Decision provenance is preparing and preserving the record. It is not deciding what the record means. Custara does not assess, score, classify, certify, rate, advise or recommend. The reviewer, the assessor, the underwriter and the regulator decide what the record means. Custara’s job is to make sure that when they look, the record is there, structured, and honest about how each fact is known.

The market is moving from describing governance to proving decisions. Decision provenance is how you hold that evidence before the question is asked, rather than reconstructing it under pressure once it has been.

See how a single provenance record answers different reviewers in the worked examples, or read how the Governance Evidence Record is built.


The framework behind this approach, and the book that sets it out, are forthcoming. This article is general information, not legal, insurance or investment advice.

Register interest

If this is the workflow your team needs, register interest.