Software Factories

What Is Agentic AI Governance, and What Does a Governance Framework Look Like?

Agentic AI governance is the set of policies and infrastructure controls that decide what an AI coding agent can access, how its actions are tracked, and who is accountable for its output. A working governance framework covers access scope, an audit trail, model and harness policy, data handling, and ongoing cost and quality accountability — enforced in infrastructure, not just written in a policy document.

Why governance became urgent

The default way teams adopted coding agents was one developer, one bespoke agent, installed and configured on a laptop with access to everything that developer is logged into. That setup works until it doesn't: every agent is a separate security surface, there's no standard set of skills or MCPs across the team, and all the data the agents produce — conversations, decisions, the record of what an agent actually did — disappears the moment the session ends. Warp has called this out directly in its own account of why cloud software factories exist, describing it as a governance nightmare that scales faster than any control layer around it.

The same pressure shows up when teams move agents into the cloud piecemeal. In Adopting the software factory model: crawl, walk, run, Warp describes how one-off cloud automations each add their own setup burden and security surface, until teams want central configuration, auditing, and governance across all of them.

The five things a governance framework actually covers

Agentic AI governance isn't one control — it's five, and a team without an answer to all five doesn't have a governance framework yet:

  • Access scope — which repos, credentials, and systems a given agent configuration is allowed to touch.
  • Audit trail — whether agent conversations, evals, and memories are recorded, and who can review them after the fact.
  • Model and harness policy — which models and harnesses are approved, and whether that can be restricted or pinned org-wide.
  • Data handling and retention — where data lives, who hosts it, and whether it can be excluded from any model training.
  • Cost and quality accountability — measuring whether an agent's output over time justifies what it costs, on a recurring basis rather than a one-time review.

Governance framework at a glance

LayerQuestion it answersFailure mode if missing
Access scopeWhat can this agent touch?Agents accumulate access nobody tracks
Audit trailWhat did this agent actually do?No record when something goes wrong
Model & harness policyWhich models/harnesses are approved?Unvetted models run unnoticed
Data handling & retentionWhere does data live, and is it used for training?Sensitive data ends up in a vendor's training set
Cost & quality accountabilityIs this agent still worth running?Spend grows with no tie to output quality

How this maps to NIST and OWASP guidance

These five layers aren't a new standard; they translate existing frameworks into coding-agent terms. NIST's AI Risk Management Framework, released in January 2023, organizes AI risk work into four functions — Govern, Map, Measure, and Manage — and NIST added a Generative AI Profile (NIST AI 600-1) in July 2024. For agents specifically, the OWASP Gen AI Security Project lists excessive agency among the top risks for LLM applications and traces it to three root causes: excessive functionality, excessive permissions, and excessive autonomy. OWASP's Agentic AI – Threats and Mitigations guide, published in February 2025, extends that into a threat model for agentic systems.

LayerWhat to enforceWhere the guidance comes from
Access scopeLeast-privilege tools and credentials per agent; actions run in the requesting user's context where possibleOWASP LLM06: minimize extensions, minimize permissions, execute in the user's context
Audit trailComplete traces of prompts, tool calls, outputs, and downstream actionsOWASP LLM06: log and monitor activity; NIST AI RMF: Measure
Model & harness policyAn approved list of models and harnesses, pinned in configuration rather than chosen per laptopNIST AI RMF: Govern
Data handling & retentionWhere traces live, who hosts them, and whether they can be used for trainingNIST AI RMF: Govern
Cost & quality accountabilityRecurring scoring of agent output against cost, plus human approval for high-impact actionsOWASP LLM06: require user approval; NIST AI RMF: Measure and Manage
Mapping the five governance layers to OWASP and NIST guidance

Harness-level controls help, but they don't add up to governance on their own. Claude Code, for example, runs with read-only permissions by default, asks before editing files or running commands that modify the system, and lets organizations enforce standards through managed settings and monitor usage through OpenTelemetry. Those controls govern one tool. A governance framework has to hold across every harness a team runs, which is the argument for enforcing it in shared infrastructure rather than per laptop.

What most teams get wrong

The common mistake is treating governance as a one-time security questionnaire completed before a tool is approved, rather than a policy enforced continuously by the infrastructure the agent runs on. A questionnaire answered once tells you nothing about an agent's twentieth run six months later.

Governance is also easy to conflate with agent management — the ongoing administration of individual agents — covered in Warp's breakdown of agent orchestration vs. agent management. But governance is the policy layer underneath it: management enforces what governance decides.

A third mistake is letting the model decide what's allowed. OWASP's mitigation guidance calls for complete mediation: authorization is enforced in the downstream system — the repository, the package registry, the database — so every request an agent makes is checked against policy, whatever the prompt said. Prompt instructions and secret redaction are backstops, not boundaries.

How Warp fits

Because Warp Factories are defined as version-controlled code, governance decisions — which skills, MCPs, and permissions a given agent can use — are specified the same way any other infrastructure change is: reviewable, auditable, and reversible. The factory control room gives visibility into every agent run's cost, quality, and throughput, so accountability isn't a one-time checkbox.

Warp Factories is also built for AI sovereignty: teams can bring their own inference and hosting, host all of their own data exhaust, or prohibit any use of that data for training through Zero Data Retention.

Warp's infrastructure and security documentation spells out the boundaries. Warp runs the control plane — coordination, identity and configuration, observability, and inference routing — while execution runs in a Warp-hosted sandbox or, on Enterprise plans, on a managed self-hosted worker, so checkouts and commands stay on machines you control. Inference credentials are used only at the inference boundary and never enter the sandbox, execution secrets come from an explicit per-agent allowlist, and runs act either with the requesting user's authorization or as the agent itself. Who approves specs and merges stays a repository-policy decision, and if you bring your own model provider, retention follows your contract with that provider.

Warp reports that its own engineering team currently automates 20–30% of its PRs through factories built on this model, detailed in its guide to cloud software factories — a first-party sign that governance and throughput aren't in tension when the controls live in the infrastructure itself.

Start with one workflow

If your team doesn't have an agentic AI governance framework yet, don't try to write the whole policy before you start. Pick one agent configuration, write down exactly what it's allowed to touch, and review its run history on a set schedule — the same starting point as building good agent management.

Interested in Warp Factories? Get early access today or book a demo for a tailored walk-through of the platform. Eligible teams can receive up to $10,000 in free factory usage.

Sources

Start your software factory

Book a demo and we’ll walk you through the workflows that map to your stack.