Skip to content

Point of view

Observing an AI Agent Is Not the Same as Governing It

To claim a control prevented an action, show how its decision affected the request before the protected operation occurred.

Seeing a policy denial in an agent trace does not establish that the requested action was stopped. The missing link is whether the executor received that decision in time and applied it to the same request before the protected operation occurred.

Observability contributes to governance by making activity and decisions reviewable. A claim of preventive enforcement requires evidence of an additional fact: the decision constrained execution. That distinction matters when reviewing a control, investigating an incident, or reporting what an agent was prevented from doing.

Observability is necessary

Teams need observations to understand which operations were requested, what policy evaluation reported, and which outcomes were recorded. Without that context, a failed action could be mistaken for a successful control, or a permitted request for a completed operation.

OpenTelemetry's documentation on traces describes a span as a unit of work or operation, with start and end timestamps. Span events annotate meaningful points during that operation. Those structures can help relate a policy evaluation to a tool request, provided the application records the relevant information.

Observations can also supply context to a policy decision while a workflow is running. They do not have to arrive after the action, and an observability component can participate in a control architecture. The question is what the execution path does with the information it receives.

Observability is not proof of enforcement

Hypothetical example: in an isolated test, an agent requests that a tool send a synthetic document to a destination prohibited by policy. The tool records the request, a policy service reports a denial, and the event appears in an operational view.

That record shows a denial was reported. It does not, by itself, show whether the tool withheld the destination call. The call might have been stopped, already dispatched, or rejected by the destination for a separate reason. More evidence is needed to distinguish those outcomes.

Even the word “denied” needs a defined meaning. An application might emit it when the policy engine returns a verdict, when the executor stops the request, or when the destination refuses it. These are different points in the process. A destination refusal may protect the operation while leaving the effectiveness of an upstream control unproven.

The AI agent governance pillar describes the broader execution chain. A control review should go further than finding its components in a trace: it should establish the dependency between the decision and the attempted operation.

A decision point must affect what the executor can do

A decision point participates in preventive enforcement when execution of the protected operation depends on its result. Receiving a copy of a request and producing a verdict is insufficient if the executor can proceed without applying it.

That dependency can take different forms. A tool wrapper can wait for a decision before calling the destination. An asynchronous workflow can hold a request pending a decision and release it only under the permitted conditions. Both require an effective connection between the verdict and the component capable of carrying out the action.

The review should establish what happens when the result is missing, late, or unusable. If the executor proceeds on a policy error, a later denial record cannot demonstrate that the action was blocked. If it remains pending, the evidence should identify that state. The intended behavior and the implemented behavior need to agree for the integration being assessed.

Verify control before the protected operation

For the hypothetical document request, the relevant order is that the executor receives the proposed operation, holds it while policy is evaluated, and applies the denial before any call sends the document. Evidence collection may continue afterward. The export time of the evidence does not establish when the control acted.

A useful validation would use synthetic input and an instrumented simulated destination. Exercise a permitted request to confirm that the test can observe a destination call. Then exercise the denied case and check both the executor's decision handling and whether a destination call occurred. Merely finding no destination event is weaker evidence if the observation mechanism has not been checked.

The validation also needs to cover relevant retries, background workers, and alternate routes in the reviewed workflow. A wrapper may stop the first attempt while another path dispatches the operation. State which paths were exercised and which remain outside the conclusion; one successful test does not establish universal coverage.

The boundary should be specific about the effect being prevented. Stopping a later step does not reverse data already sent by an earlier one. Review the operation that crosses the protected boundary, rather than assuming that stopping a workflow somewhere is sufficient.

Approval must remain connected to the reviewed request

An approval event records a decision to permit something. To function as a control, the executor must wait when approval is required and apply a valid approval to the request it actually executes.

In the same hypothetical workflow, suppose policy routes a particular destination to human review instead of denying it. The approval must refer to the document, destination, and operation presented for review, with whatever scope and validity the policy requires. If those details change, the executor needs to determine whether the approval still applies. A historical approval elsewhere in the session cannot establish that connection.

Evidence should therefore connect the pending request, the approval, and the executor's handling of that approval. Correlation identifiers help locate the records, but the implementation must also check that the authorization applies. An identifier in a log does not perform that check.

The organizational side of that decision is covered in Why AI Governance Must Move From Models to Actions. For verification, the additional question is whether the approved scope governed the operation that followed.

Keep decision evidence and execution evidence distinct

After the decision, a reviewer should be able to connect the requested operation, the applicable policy, the returned verdict, the executor's response, and the available destination outcome. Preserve the source and meaning of each record so a report does not treat different observations as interchangeable.

OpenTelemetry's Logs Data Model distinguishes Timestamp, measured at the event's source, from ObservedTimestamp, measured when the collection system observes it. The values can coincide, and the source timestamp can be absent. Sorting records by a displayed time alone does not establish that one component waited for another.

Clock differences, collection delays, and incomplete correlation can leave the order uncertain. Inspect the implementation's execution dependency and the evidence of how it behaved. If records are missing because a path was not instrumented or collection was incomplete, retain an unknown outcome instead of converting missing evidence into a claim of prevention.

A review conclusion should identify the integration and configuration examined, the operation whose execution was tested, and the limits of the observation. Distinguish a decision that was reported, a request confirmed as withheld, and an operation whose outcome remains unknown. This gives the next reviewer a claim that can be checked.

KonaSense's position is that claims about preventive controls should meet this standard of evidence. When evaluating Kona for Agents, examine the specific integration's decision handling, execution behavior, and available records. Telemetry reception and policy evaluation should not be presented as proof that a protected action was stopped.