Can We Prove What the Agent Actually Did?

I ask a coding agent to upgrade an authentication dependency.

> upgrade the authentication dependency

The agent reads the repository, updates package.json, runs:

npm install
npm test

and gives me a diff.

Maybe it pushes a branch and opens a pull request.

From the transcript, the story looks simple:

Developer request
      ↓
Agent session
      ↓
npm install
      ↓
tests
      ↓
Git diff
      ↓
Pull request

But how much of that story can I actually prove?

npm install might execute a package lifecycle script. That script might start another process. That process could modify files, perform a DNS lookup, connect to an external service, or use a credential already available in the environment.

The agent transcript might still only tell me:

tool: bash
command: npm install
result: success

So what do I really know?

Did the agent make the change?

Did the package manager?

Did a child process?

Did GitHub actually accept the branch?

And how much of that story am I filling in because the events happened near each other?

The more execution we hand to agents, the more I think this distinction matters:

Seeing a collection of events is not the same thing as proving how those events relate.

The investigation starts before the agent

Most of my coding-agent workflows begin with a person at a terminal.

That makes it easy to think in terms of:

Human
  ↓
Agent
  ↓
Action

But an agent could just as easily start because of a scheduled job, webhook, queue message, CI event, API request, or another agent.

So the first investigation question isn't necessarily:

Who started the agent?

It is:

What caused this run to exist?

Imagine an agent that investigates new GitHub issues:

GitHub issue opened
       |
       v
Webhook receiver
       |
       v
Agent run
       |
       v
Repository analysis
       |
       v
Comment / pull request

If I'm investigating that run later, I want more than:

agent_run = 123

I want something closer to:

trigger: webhook
provider: GitHub
event: issues.opened
delivery_id: 72d316...
repository: payments-api
signature: verified
received_at: 14:32:10

agent_run: run-123

GitHub webhook deliveries already give us useful primitives for this. A delivery includes an event type and globally unique delivery identifier. If the webhook is configured with a secret, GitHub also sends an HMAC-SHA256 signature that the receiver can verify before trusting the payload.

A scheduled run needs different evidence:

trigger: schedule
schedule_id: iam-review-daily
scheduled_for: 03:00
scheduler_execution: 9821

agent_run: run-441

And delegation between agents needs another shape:

trigger: agent-delegation
parent_agent: security-orchestrator
parent_run: run-710
delegation_id: task-22
child_agent: iam-investigator

agent_run: run-711

The fields change, but the idea doesn't.

The agent itself shouldn't be the authority on what caused it to start.

The system receiving the event, validating it, and creating the run should record that relationship.

There are useful event formats we can borrow from here. CloudEvents, for example, gives events common context such as an ID, source, type, subject, and time. I care less about the format itself than making sure we preserve enough context to explain why the run existed.

The transcript is not the execution

Once the agent starts, the transcript becomes useful.

It can tell us what request the agent received, which tool it selected, what arguments it passed, what the tool returned, and whether an approval happened.

But that's the orchestration view.

Take:

npm install

The harness might see:

tool call
   |
   v
npm install

The runtime might see:

npm
 |
 v
node
 |
 v
package postinstall
 |
 +--> child process
 |
 +--> filesystem write
 |
 +--> DNS lookup
 |
 +--> outbound connection

Both records can be correct.

They just answer different questions.

The transcript tells me what the agent asked the harness to do.

The runtime tells me what it observed executing.

The network layer tells me where that execution tried to communicate.

The identity system tells me what credential was issued.

GitHub tells me whether the pull request actually exists.

No single one gives me the whole story.

Start with the evidence, then ask what it proves

Suppose I have:

12:00:01   npm test started
12:00:02   src/auth.ts modified

Did npm test modify src/auth.ts?

Maybe.

The timestamps don't prove it.

Another process could have touched the file. A background task might still have been running. A child process might have detached. Something outside the agent run could have modified the workspace.

I've started finding a simple vocabulary useful here:

Observed
Strongly linked
Correlated
Inferred
Unknown

For example:

OBSERVED

Process 4821 opened a connection
to 203.0.113.4.

A collector saw that event.

Then:

STRONGLY LINKED

Process 4821 was recorded as a child
of the shell process created by
execution 17.

Now we have mechanical lineage.

Then:

CORRELATED

Execution 17 occurred during
agent run ABC.

Useful, but different again.

And:

UNKNOWN

We cannot prove which exact model
decision caused that connection.

That last category matters.

A useful investigation system should be comfortable saying:

I don't know.

Otherwise it is very easy to turn a set of related events into a causal story that the evidence never established.

This is where I start using the word provenance.

Observability gives us records of what different systems saw.

Provenance is the work of establishing what those records allow us to say about the relationships between them.

Every layer knows something different

The investigation chain gives us a set of questions rather than one perfect log.

I don't think there will be one perfect "agent audit log."

An investigation joins evidence from systems that each know a different part of the truth.

And we need to be able to follow that evidence in both directions.

If I'm reviewing a known agent run, I can work forward:

What triggered the run?
       ↓
What did the agent decide?
       ↓
What did the harness dispatch?
       ↓
What actually executed?
       ↓
What did it touch?
       ↓
Which credential did it use?
       ↓
What changed downstream?

But security investigations often begin with the effect.

Maybe an IAM policy changed.

Maybe a repository received an unexpected commit.

Maybe a network control blocked an unfamiliar destination.

Now I work backwards:

Resource changed
       ↑
Which authenticated request?
       ↑
Which credential?
       ↑
Which workload?
       ↑
Which execution?
       ↑
Which tool call?
       ↑
Which agent run?
       ↑
What triggered it?

That's the version of agent auditability I find most useful.

Not only:

What did this session do?

But:

I found this change. Can I trace it back to the run that produced it?

Correlation isn't causation

A process tree gives us much stronger evidence than timestamps.

Imagine a trusted runtime collector sees:

execution 17
    |
    v
shell PID 4100
    |
    v
npm PID 4200
    |
    v
node PID 4821
    |
    +--> wrote package-lock.json
    |
    +--> connected to registry.npmjs.org

Now I can say something fairly strong:

Process 4821 was observed as a descendant of the process created for execution 17, and it was observed writing package-lock.json.

What I still shouldn't automatically say is:

The model caused package-lock.json to change.

There are several layers between those statements:

Model
  ↓
Tool selection
  ↓
Harness
  ↓
Shell
  ↓
Package manager
  ↓
Package script
  ↓
File write

Some of that code may have been written by people completely outside the agent system.

This is one of the things that became clearer to me while working through sandboxing too. Once we allow general-purpose execution, there can be a lot of distance between agent intent and system effect.

The evidence should preserve that distance rather than hide it.

A trace ID doesn't make the evidence trustworthy

The obvious way to join all of these systems is with IDs:

trigger_event_id
agent_run_id
tool_call_id
execution_id
sandbox_id
trace_id
credential_grant_id
request_id
change_id

Then propagate them:

Webhook 77
    |
    v
Agent run ABC
    |
    +--> tool call 17
    |
    +--> execution E91
    |
    +--> sandbox VM42
    |
    +--> credential G17
    |
    +--> request R72

That's valuable.

But I still need to ask:

Who created the ID?

If the process we're investigating reports:

agent_run_id = ABC

and that process controls the value, it may also be able to report:

agent_run_id = someone-elses-run

Everything might correlate perfectly.

The correlation just wouldn't necessarily be authentic.

So I think we need to separate self-reported context from context asserted by a trusted control point.

For example:

Webhook receiver

I verified delivery 77
and created run ABC.
        |
        v
Tool gateway

I accepted tool call 17
for run ABC
and created execution E91.
        |
        v
Sandbox manager

Execution E91 belongs to VM42.
        |
        v
Credential broker

VM42 received credential grant G17.
        |
        v
GitHub

The authenticated request created PR #123.

Now important relationships are being recorded by systems outside the workload we're investigating.

This is where distributed tracing helps without solving the entire problem.

W3C Trace Context gives us traceparent and tracestate for propagating trace information across distributed systems. That is useful for correlation, but the specification also calls out trust and abuse concerns around externally supplied trace information.

So the way I think about it is:

Context propagation solves correlation. It doesn't automatically solve authenticity.

Identity is part of the investigation

Imagine ten agent runs all use:

coding-agent@example.com

The downstream audit record says:

principal = coding-agent@example.com
action = create_pull_request
result = success

Good.

Which run?

Which tool call?

Which trigger?

Was it a developer, webhook, scheduled job, or another agent?

The service identity can't answer that by itself.

I think we need to preserve two related lineages.

The first explains why the work exists:

Originating event
       |
       v
Invocation
       |
       v
Agent run

The second explains what authority performed it:

Sandbox
   |
   v
Workload identity
   |
   v
Credential grant
   |
   v
Downstream principal

Then we join them:

GitHub webhook
       |
       v
Agent run ABC
       |
       v
Execution E17
       |
       v
Workload VM42
       |
       v
Credential G91
       |
       v
GitHub
       |
       v
PR #123

This is another reason I like short-lived, task-scoped credentials for agents.

It isn't only a least-privilege control.

Credential issuance can become part of the investigation trail.

Workload-identity systems already give us useful primitives here. SPIFFE, for example, gives workloads SPIFFE IDs and supports short-lived X.509 or JWT SVIDs delivered through the Workload API.

That can give us a strong answer to which workload authenticated, while still leaving us responsible for preserving the relationship back to the agent run and its trigger.

Instead of:

service account X did something

I want to get closer to:

grant G91
   |
   +--> issued to workload VM42
   |
   +--> for execution E17
   |
   +--> during agent run ABC
   |
   +--> originating from webhook 77

That's a much better attribution story.

Pi makes the gap easy to see

Pi is useful here because its execution model is relatively inspectable.

Pi persists sessions as JSONL with a tree structure, and shell commands invoked through its LLM-callable Bash or PowerShell tools receive session metadata including PI_SESSION_ID.

That gives us useful orchestration evidence:

Pi session ABC
      |
      v
tool call 17
      |
      v
npm install

But once npm install begins, Pi doesn't automatically become an operating-system provenance system.

The process can create children. Those children can modify files. They can make network requests. They can use credentials in the environment or hand work somewhere else.

Pi's own security documentation is explicit that its tools and extensions execute with the permissions of the Pi process unless an external containment boundary is used.

Whole-process sandboxing can give us a cleaner unit to observe:

Agent run
   |
   v
Sandbox VM42
   |
   +--> Pi
   +--> Bash
   +--> npm
   +--> child processes

Now runtime events can at least be associated with a defined execution environment.

But sandbox membership still tells us where execution happened.

It doesn't tell us why every effect happened.

Building BashGuard made this less theoretical

I ran into this directly while building BashGuard.

It started with a simple question:

Can I show what happened during a Pi session?

BashGuard now records supported Pi lifecycle events, prompts, tool and shell activity, file-tool activity, Git snapshots, authorization decisions, and explicit capture-gap evidence. It can attach to live sessions, inspect individual records, and produce evidence-based debriefs.

That gives me much more to investigate than a transcript alone.

But collecting more events exposed a harder problem.

Suppose BashGuard records:

tool call
edit src/auth.ts

and the Git snapshot later shows:

src/auth.ts modified

Those records are clearly interesting together.

But did that tool call cause the exact Git diff?

Not necessarily.

BashGuard deliberately treats file and Git relationships as observed overlap or correlation rather than claiming causality it cannot establish. It also reports missing, partial, redacted, and capture-gap evidence instead of pretending the session was completely observed.

That changed what I think the hard part is.

Putting events on a timeline is relatively easy.

The harder question is:

What relationship between those events can I actually defend?

I increasingly want the investigation model to be:

observe
   |
   v
correlate
   |
   v
establish confidence
   |
   v
expose gaps

Not:

collect events
   |
   v
generate confident story

What would I want during an investigation?

Probably not one enormous log entry.

I'd rather have an evidence bundle.

Start with why the run exists:

TRIGGER

GitHub webhook
event: issues.opened
delivery: 72d316
signature: verified

AGENT RUN

run-ABC

Then what the harness dispatched:

TOOL CALL

bash-17

command:
npm install

Then execution:

EXECUTION

exec-91

SANDBOX

vm-42

PROCESS TREE

npm
 └── node
      └── postinstall

Then effects:

FILESYSTEM

package-lock.json modified

NETWORK

attacker.example
decision: DENY
reason: destination not permitted

Then authority and downstream state:

CREDENTIAL

grant G-17
workload: vm-42
scope: repo:payments-api

GITHUB

PR #123 created
principal: coding-agent

And then the piece I'd care about most during an investigation:

EVIDENCE QUALITY

OBSERVED
The proxy denied attacker.example.

STRONGLY LINKED
postinstall was a descendant of exec-91.

CORRELATED
exec-91 belongs to agent run ABC.

UNKNOWN
Whether another process outside VM42
modified the workspace.

UNKNOWN
The exact model reasoning that resulted
in this dependency being selected.

That feels more useful to me than a polished paragraph explaining what "the agent did."

It keeps the evidence and our interpretation separate.

OpenTelemetry helps, but it isn't proof

OpenTelemetry is useful for this architecture.

Its GenAI semantic conventions include concepts for agent, conversation, and tool-call telemetry. The conventions are still evolving, which is another reason I treat them as an interoperability layer rather than a finished security evidence model.

That gives us a shared vocabulary for something like:

agent span
    |
    v
tool span
    |
    v
downstream span

I'd want that in an enterprise agent system.

But I wouldn't automatically treat workload-generated spans as trusted security evidence.

If the workload emits the telemetry, I have to assume it can influence what gets reported. A compromised process may omit events, change attributes, or propagate misleading context.

So I see OpenTelemetry as:

shared vocabulary
+
distributed correlation

not:

trusted provenance

It becomes much more useful when we join it with records from trusted invocation points, runtime controls, credential systems, network enforcement, and downstream services.

The evidence should be allowed to disagree

Once we collect evidence from several layers, sometimes it won't line up.

The transcript might say:

npm install completed successfully

while the runtime says:

child process still active

The agent might report:

deployment complete

while the downstream API says:

request rejected

A scheduled job might say:

scheduled for 03:00

while the scheduler's own record says:

dispatched at 03:12

I don't want the investigation system smoothing those differences into a cleaner story.

The disagreement is evidence too.

It may be one of the more useful signals we have.

Where I've landed

I don't think agents create an entirely new observability problem.

We already know how to collect process telemetry, network logs, workload identity, credential issuance, cloud audit events, source-control history, and downstream application logs.

What agents add is another lineage we increasingly need to preserve:

originating event
       |
       v
agent run
       |
       v
model / tool decision
       |
       v
execution
       |
       v
effect

And the originating event isn't always a person.

It might be a human, cron job, webhook, queue message, CI event, API request, or another agent.

That matters because the investigation starts there.

I don't think the goal is to construct a perfect recording of everything an agent ever does.

The goal is to be precise about what we know.

The invocation layer can tell us why the run exists.

The transcript can tell us what the agent attempted.

The harness can tell us what it dispatched.

The runtime can tell us what it observed executing.

The filesystem and network layers can tell us what that execution touched.

The identity system can tell us which credential was issued.

The downstream service can tell us what state it accepted or changed.

Provenance is the work of joining those pieces without pretending the evidence says more than it does.

If we can't prove an edge in that story, the system should be able to say so.

Sources

• GitHub webhook events and payloads
• GitHub validating webhook deliveries
• CloudEvents specification
• W3C Trace Context
• OpenTelemetry
• OpenTelemetry GenAI semantic conventions
• SPIFFE overview
• SPIFFE SVIDs
• Pi sessions
• Pi security
• BashGuard

How I used AI

I used AI as a research assistant, technical sounding board, and editor while developing this article. The ideas and conclusions came from working through agent execution, sandboxing, identity, and observability problems in my own tooling and architecture work, including what building BashGuard exposed about correlation and incomplete evidence. I used AI to challenge the distinctions between observability, correlation, and provenance, research the underlying telemetry and identity primitives, and help organize the argument into a clearer technical narrative.