Your AI Made a Decision. Now Reconstruct It.

Category: Risk & Compliance

Somebody is going to ask.

Maybe it's a customer whose refund got split the wrong way. Maybe it's an enterprise buyer's security team, working through a vendor questionnaire before they'll sign anything. Maybe it's your auditor, or a board member who read something alarming over the weekend. The question always arrives in roughly the same shape: why did your system do that?

For a lot of agentic systems running in production right now, nobody actually knows. Not because the team was sloppy. They logged plenty. They logged the wrong shape of thing.

The standard here is evidentiary

A compliance checklist asks whether a control exists. Do you have human oversight? Yes. Do you log AI decisions? Yes. Do you have a model governance policy? Yes, it's a PDF, here it is. You can clear all of that in an afternoon and still be unable to explain a single transaction.

An evidentiary standard asks something much harder. Take one specific decision your system made four months ago. Hand it to a person who wasn't there, doesn't trust you, and has all day. Can they rebuild it? Can they get to the point where they say "okay, I see why it did that," with nobody from your team narrating over their shoulder?

That's the real bar, and it comes from how disputes actually play out. When money moves wrong, the argument is never about your governance policy. It's about one transaction, one timestamp, one decision, and whether you can account for it.

Four things have to survive.

1. What it knew

The information available at the moment of the decision. Not the information available now.

This is the one teams assume they've handled. Your agent read a contract, checked a merchant's standing, pulled a balance, looked at a few prior transactions. Come back six months later, re-query any of it, and you get today's answer. The merchant's status changed. The contract got amended in March. The balance is a completely different number and means nothing.

Reconstruction needs point-in-time capture: the actual values, or versioned references that still resolve to what existed then. A record saying "read merchant 4471" points at a moving target. A record saying "read merchant 4471, version 12, status verified, tier standard" is evidence.

The absences matter too. A surprising share of post-incident analysis lands on something the system simply couldn't see. If your record only shows what the agent read, you can't distinguish a reasoning failure from a data gap, and those get fixed in very different places.

2. What it accessed

The reach. Which tools it called, which systems, which documents, which records, under whose credentials.

Two different people care about this for opposite reasons. Whoever is investigating a bad outcome wants to know whether the agent had what it needed. Whoever is running your security review wants to know whether it touched something it shouldn't have. Same log, and most teams only design for the first reader.

The second one is what stalls deals. An agent with broad read access and no per-call record is an unbounded data exposure claim that you have no way to argue against. When a prospect's security lead asks whether your AI read their customer records, "almost certainly not" is not a sentence that gets a contract signed.

Worth capturing what it was permitted to reach and didn't. Scope of authority is part of the story, and it's the part that shows the boundaries were real.

3. What it did

Actions and their effects. Which call, which parameters, what changed, what came back, what got reversed and when.

Most engineering teams are already competent here. This is transaction logging and it predates all of this by decades. Two things still get dropped.

The first is the line between proposing and executing. In a system with an approval gate, the agent proposes and something else carries it out. If your log flattens those into one event, you have destroyed your own ability to demonstrate that the gate did anything at all.

The second is everything that didn't happen. Rejected proposals, modified proposals, retries, timeouts. Teams keep the successful path and let the rest age out, which is exactly backwards. A gate that only ever produced approvals is indistinguishable from no gate. The rejections are the proof that the control has teeth.

4. Why

Here's where it falls apart.

Ask a team for this and what usually comes back is the prompt template, the model version, the temperature setting, and the output. That's a recipe. It describes how the decision got produced. It says nothing about what the decision rested on.

You can't recover the reasoning by re-running it, either. You'll get different tokens on the second pass, and the world moved anyway. Asking the model to explain itself after the fact is worse, because you get a plausible story generated in hindsight that reads exactly like evidence and isn't.

The reason this one gets skipped is structural. The first three fall out of ordinary instrumentation. You log inputs because you log inputs. You log calls because your APM does it for you. Reasoning has no such byproduct. It exists for a moment inside a model's forward pass and then it's gone unless something deliberately catches it, gives it a shape, and writes it down before anything executes.

Making the reasoning a real object

Treat it as an artifact the system produces, with a schema, stored next to the proposal, created before the action.

Dumping raw chain-of-thought is not the answer. It produces volume without clarity, and it hands a reviewer several thousand words to hunt through for the one sentence that sounds terrible out of context.

What actually holds up is short and structured:

• The claim. This party receives 12%.
• The basis. Clause 4.2 of agreement A-2231, quoted, pinned to a specific version.
• The path. How it got from that clause to that number.
• The confidence, broken out per element, with the uncertainty named rather than averaged away.
• What it considered and set aside, with the reason.

That last item earns its keep. "Considered treating this as a chargeback under clause 9, ruled it out because the timestamp falls outside the dispute window" is a sentence a reviewer can go check. Someone can prove it wrong. That's precisely what makes it evidence instead of decoration.

We learned this building payout infrastructure with 47 distinct split scenarios. When a disbursement is questioned, the useful artifact was never the log line showing the transfer. It was the record of which contract language drove the number.

The human's reasoning counts as reasoning

If a person approved the action, their basis belongs in the record too.

An approval with no rationale attached is a signature on a blank page. And the thing they approved was a screen, which decays. The UI shipped three redesigns since. The data behind it updated. Six months later you cannot show what your operator was looking at when they clicked the button.

So capture the approval as its own object: what was presented, who acted, what they chose, what they wrote. If they overrode the agent, the reason. If they changed an amount, what they knew that the system didn't.

Those override reasons are some of the most valuable data your organization will ever generate. Every one is a labeled example of the model being wrong in a specific, correctable way.

A test you can run this week

Pick a decision your system made ninety days ago. Something with money attached.

Give the ID to an engineer who didn't build the thing. One day, no help from the team that did.

Ask them to produce a single page: what the system knew, what it accessed, what it did, and why it concluded what it concluded. Then check that page against what actually happened.

Most teams get three quarters of the way and stall in the same place. If that's where you land, at least you now know what you're building.

The timing on this

Vendor security questionnaires got here first, which is why this tends to surface as a sales problem before it surfaces as a legal one. Regulation is arriving behind it. The EU AI Act's high-risk obligations have slipped toward late 2027, Colorado's law lands in January 2027, and Texas already has one in force. The specifics vary and the deadlines keep moving, but they converge on the same handful of ideas: attributable logging, real human override points, and gating on consequential decisions.

None of that is unfamiliar territory for anyone who has built financial systems. Reconstructing a past state from an immutable record is what reconciliation has always been. What changed is that one participant in the decision is now a model, and models leave nothing behind on their own.

The part that doesn't get cheaper later

All four of these are inexpensive if you designed for them and painful if you didn't. Inputs you never snapshotted are gone. Reasoning you never captured was never anywhere to begin with. You can add the plumbing next quarter, and it will start working from the day you turn it on, which does nothing for the decision somebody is asking about today.

Build it in at the start. It costs less than people expect, and the first time someone asks, you get to hand them an answer instead of opening an investigation.