Enterprise AI Governance: From Policy to Implementation

Enterprise AI governance is the set of controls that keep AI systems accountable in production, and for autonomous agents it means governing the actions an agent takes, not just the text it produces. The controls that hold up are the ones enforced where the action happens: known identity, a durable action log, and hard boundaries on what an agent can do. A policy document that is not enforced at the point of action is theater.

Key takeaways

• Governance for AI agents is about actions, not outputs: who authorized the action, what it touched, and whether it could have done more than it should.
• The three concerns any agent program must answer are identity and authorization, action logging and auditability, and containment boundaries.
• NIST, the EU AI Act, and OWASP converge on the same short list: know who is acting, record it in a way you cannot quietly edit, and bound what is allowed.
• A control has to live where the action happens. Enforcement at the query layer beats a permission that is only written down in a policy.
• Query-layer enforcement covers identity, logging, and containment. It does not replace model-level guardrails, data governance, or the human decisions about which actions need approval.

What is enterprise AI governance?

Enterprise AI governance is the set of controls that keep AI systems accountable in production. For chatbots that answer and forget, the stakes were low. Autonomous agents change that, because an agent plans and carries out multi-step work across your systems. It writes to the CRM, moves money, files the ticket, revokes the access. Governance stops being a review of outputs and becomes a question about actions: who authorized this one, what did it touch, and could it have done more than it should. Understanding how AI agents operate is the starting point, because the agent's autonomy is exactly what the governance program now has to bound.

What a CIO actually has to decide

Ask most AI governance programs one question and the room goes quiet. At the moment an agent took an action last Tuesday, whose permissions did it use, and where is the record? If the honest answer is a service account and a model trace, you have a policy, not a control.

The gap in most programs is not ambition. It is the distance between a governance document and the place where an agent actually acts. A maturity model on a slide does not stop an over-permissioned agent from deleting the wrong records. The control has to live where the action happens. That is the argument of this piece, and it is a position you can disagree with: policy without enforcement at the point of action is theater.

There is a practical reason this matters more for agents than it did for earlier AI. A single agent run can touch five systems, make a dozen decisions, and leave side effects in each. If the governance evidence lives in the model's context window, it disappears when the run ends. Three months later, when a customer disputes a change or an auditor asks who approved it, the reasoning is gone and you are left reconstructing intent from logs that were never designed to be an audit trail. The programs that hold up are the ones where the evidence is a durable artifact, produced as a byproduct of the action, not a transcript you hope you can still retrieve.

The frameworks, briefly, and what they agree on

You already know the frameworks exist, so this stays short. The NIST AI Risk Management Framework (AI RMF 1.0, document NIST.AI.100-1, released January 2023 and currently being revised) organizes AI risk work into four functions: Govern, Map, Measure, and Manage. It tells you what to achieve. It is deliberately technology-neutral about how.

The EU AI Act is more prescriptive where it counts. Article 12, Record-keeping, requires that high-risk AI systems technically allow for the automatic recording of events, logs, over the lifetime of the system. The OWASP Top 10 for LLM Applications (2025) ranks Excessive Agency as LLM06:2025, the risk that an agent can take actions beyond what the task required because its permissions, tools, or autonomy were too broad.

Read across the three and they converge on the same short list. Know who is acting. Record what they did in a way you cannot quietly edit later. Bound what they are allowed to do. Every serious framework says some version of this. Almost none of them shows you how to enforce it inside an agent stack.

Governing AI agents specifically

In February 2026, NIST's Center for AI Standards and Innovation launched an AI Agent Standards Initiative, including a Request for Information on securing AI agent systems and an NCCoE concept paper on agent identity and authorization. The named focus areas and RFI topics cluster around three concerns that any team governing agents will recognize. I will treat them as three themes, not a fixed standard, because the work is still in the comment stage.

Identity and authorization

An agent that acts through a shared service account is unattributable by construction. You can see that the account did something. You cannot see which person or task was behind it, and the account almost always holds more access than any single run needs. NIST's identity and authorization theme is about authenticating the agent and limiting its permissions to the job in front of it. This is the single most common failure I see: broad credentials handed to an agent for convenience, then never scoped down.

Action logging and auditability

The EU AI Act's Article 12 and NIST's monitoring and logging topics point at the same requirement. There must be a durable, traceable record of what the agent did, not just what the model said. A trace of prompts and completions is useful for debugging the model. It is not an audit log of actions. When a regulator or an incident review asks what the agent changed in the system of record at 14:32, a token trace does not answer the question.

Containment boundaries

Containment is the boundary that decides what an autonomous agent is structurally unable to do, regardless of what it decides in the moment. It maps directly to OWASP's Excessive Agency. The goal is not to make the agent smarter about restraint. It is to make the unsafe action impossible to issue, so a bad plan hits a wall instead of your production data.

What good looks like in practice

Here is the part the frameworks leave to you. Each theme maps to a concrete control you can require of a vendor or build yourself. The test for every row is the same: is it enforced where the action happens, or only written down.

• Identity and authorization. What it demands: Authenticate the agent and limit permissions to the task. Concrete control at the point of action: Run-as-invoker identity: the agent acts as the scoped identity of the person who triggered the run, so upstream RBAC is mirrored, not bypassed
• Action logging and auditability. What it demands: Durable, traceable record of actions, not just model calls. Concrete control at the point of action: Every agent action logged like any other application query, immutable, exportable to your SIEM
• Containment boundaries. What it demands: Bound what an autonomous agent can do. Concrete control at the point of action: The agent acts only through deterministic apps with typed inputs and RBAC enforced at the query layer
• Human oversight. What it demands: Escalation and approval for consequential actions. Concrete control at the point of action: Approval gates on restricted actions, routed to a person in Slack before the action commits
• Change control and traceability. What it demands: Reconstruct and roll back agent behavior. Concrete control at the point of action: Version pinning on the apps the agent runs, with rollback to a known-good version

Take the first row end to end, because it is where identity and audit meet. An agent needs to update an opportunity in Salesforce. Under run-as-invoker identity, the agent does not hold its own broad Salesforce credential. It acts as the person who started the run, carrying that user's scoped OAuth token. If the user cannot edit that record, the agent cannot either, because Salesforce's own RBAC refuses the write. The permission check is not something the agent chooses to respect. It happens upstream, at the query.

The audit record that falls out of this is the point. It reads: this user's identity, at this timestamp, invoked this app, which issued this specific write to this Salesforce object, and here is the result. That is an action log, and it is the kind of record you can audit and observe agent actions with, hand to a regulator, or replay during an incident. It exists because identity and logging were enforced at the query layer, not reconstructed afterward from a model trace.

How to stand up agent governance, in order

If you are building a program from the frameworks above, the sequence below moves from the control that fails most often to the one you add last. Each step is a thing you can verify, not a slide you can present.

1. Give every agent run a real identity. Retire shared service accounts and have the agent act as the scoped identity of the person or system that triggered it, so upstream RBAC applies without a separate policy engine.
2. Scope permissions to the task. Grant the narrowest access a run needs, then confirm the agent cannot reach systems or records outside that scope.
3. Log actions, not just tokens. Record each action the agent takes against a system of record in a durable, exportable form, and route it to your SIEM alongside your other application logs.
4. Set containment boundaries. Make unsafe actions structurally impossible to issue, so a bad plan is refused at the query rather than caught after the fact.
5. Add human approval where the stakes justify it. Route consequential actions to a named person for sign-off before they commit, and record who approved.
6. Pin versions and keep a rollback path. Version the apps the agent runs so you can reconstruct behavior and roll back to a known-good state during an incident.

Where vendors are getting this wrong

Three patterns show up repeatedly when I look at how agent governance is sold today, and each one substitutes a gesture for a control.

The first is governance as a UI dropdown. A platform offers a permissions screen and a policy toggle, and the enforcement lives in the agent's prompt or in application code that the agent can route around. A setting that the runtime does not enforce at the point of action is a preference, not a boundary.

The second is the broad service-account credential. To make an agent work across systems quickly, teams give it a single powerful identity with wide scope. Every action the agent takes is now attributed to that one account, so you lose the who, and the agent can reach far past any individual task, so you lose containment. Convenience at setup becomes an audit gap in production. The tell is simple. If your answer to "which user's authority did this action carry" is the name of a robot account, you have no identity to attribute the action to at all.

The third is treating LLM observability as agent governance. Tools that capture prompts, completions, and token counts are genuinely useful, and they are not the same as an action log. They record what the model produced. They do not record, in a durable and permissioned way, what the agent then did to your systems. A CISO who accepts a model trace as the audit trail has accepted a record of intentions in place of a record of actions.

What we are doing about it at Major

At Major, the position we build on is that governance has to be structural, enforced where actions happen, or it does not hold. Major is the enterprise platform where agents build the software they run on, and every control above applies at that app layer rather than in a policy document. Identity is run-as-invoker: a shared agent acts as the user who started the run, so upstream systems like Salesforce see that user's scoped identity and their RBAC is mirrored rather than bypassed. Auditability is a property of the layer: every agent decision is visible in run history, and every action is logged like any other app query, exportable to a SIEM. Containment comes from the same move that makes the work predictable. Because the repeatable work runs as agents with scoped permissions acting through deterministic apps with typed inputs and query-layer RBAC, the unsafe action is not a decision the model gets to make. Restricted actions can require a human approval in Slack, and the apps are version-pinned with a rollback path. Reason once. Run forever. One identity model, one audit trail, one rollback, across the agents and the apps they build.

What this does not cover: Query-layer enforcement covers identity, action logging, and containment well. It does not address model-level risks like prompt injection or hallucination, which need their own guardrails and evaluation. It does not replace data governance, classification, and retention. And it does not remove the organizational work of deciding which actions require a human, and who that human is. Those are separate programs. What the query layer gives you is the part most programs are missing: the actions an agent takes are attributable, recorded, and bounded, by construction rather than by policy.

If you are standing up agent governance, the question worth taking to your next vendor review is whether identity, audit, and containment are enforced at the query layer or only described in a policy. You can see how Major enforces run-as-invoker identity and query-layer RBAC.

Related articles

• AI Agent Governance: Controls for Autonomous Systems
• AI Agent Observability: Auditing What Agents Do
• Enterprise AI Agents: A Practical Guide