Practices for Governing Agentic AI Systems
This white paper proposes a definition of agentic AI systems and outlines a set of baseline practices for keeping them safe, accountable, and aligned with user intent throughout their lifecycle. It examines potential benefits, explores technical and operational safeguards such as task evaluation, action-space constraints, default behaviors, legibility, monitoring, attributability, and interruptibility, and discusses broader indirect societal impacts.
Summary and content fields are drawn from the document text and are pending editorial review.
Who issued it
Who it governs
AI researchers, developers, system deployers, and policymakers should read this to understand the governance challenges and potential safety practices required for increasingly agentic AI systems.
What the document contains
The document defines agentic AI systems, outlines the human roles in their lifecycle, and details seven core governance practices for safety and accountability. It also analyzes indirect societal impacts including adoption races, labor displacement, shifting offense-defense balances, and correlated failures.
It does not provide strong recommendations or final conclusions on how accountability ought to be structured legally, nor does it offer binding regulations or specific technical solutions for ensuring absolute model alignment or cybersecurity.
- Agentic AI systems can pursue complex goals over extended periods with limited direct supervision, offering significant benefits as well as novel risks.
- Effective governance requires distinguishing and sharing responsibilities among model developers, system deployers, and users.
- A defense-in-depth approach is necessary, combining task evaluations, action constraints, default behaviors, and activity legibility.
- Automatic monitoring and reliable attributability mechanisms can help detect failures and deter malicious use.
- Maintaining human control through robust interruptibility and graceful fallback mechanisms is essential to prevent severe harms.
· 1 Introduction · 2 Definitions · 3 Potential Benefits of Agentic AI Systems · 4 Practices for Keeping Agentic AI Systems Safe and Accountable · 5 Indirect Impacts from Agentic AI Systems · 6 Conclusion · 7 Acknowledgements
How it scores
Each score is backed by a quote verified against the document text. Editorial review has not yet confirmed them.
Fairness
2/5Non-discrimination and equitable treatment
Similarly, biases in common training datasets, when used by many different model developers, could expand the biased behavior of individual AI systems into a society-wide harm
5.4 Correlated Failures
screening score · 2026-10-23
Human Dignity
1/5Respect for the inherent worth of persons
As noted by Crootof et al, [ 23], a human-in-the-loop may serve various roles beyond simply improving the reliability of the human-machine system (e.g., assigning liability, preserving human dignity).
4.2 Constraining the Action-Space and Requiring Approval
screening score · 2026-10-23
Privacy
2/5Protection of personal data and private life
If sensitive agent session data needs to be retained by the system deployer, it may create privacy risks for the user (and by extension the system deployer).
4.5 Automatic Monitoring
screening score · 2026-10-23
Enforcement
2/5Consequences and compliance mechanisms with real force
a key goal of allocating accountability for harms from agentic AI systems should be to create incentives to reduce the likelihood and severity of such harms as efficiently as possible
1 Introduction
screening score · 2026-10-23
Impact Assessment
4/5Required assessment of an AI system's risks/impacts before or during use
Either the system deployer or the user should thoroughly assess whether or not a given AI model and associated agentic AI system is appropriate for their desired use case
4.1 Evaluating Suitability for the Task
screening score · 2026-10-23