Design input: OnPermissionRequest as a governance hook for production AI agents #6147
Replies: 1 comment
|
I think IPermissionPolicy would be a useful abstraction, particularly if the goal is to make the policy independently testable and composable rather than coupling authority decisions to the agent session. One thing I would keep separate, though, is the charter's authority envelope from authorization of the exact action at execution time. For example, a charter might legitimately allow an incident-response agent to modify incidents. That establishes a capability boundary, but it does not necessarily establish that close_incident(4711) is the correct authorized action when the operator asked the agent to mark incident 4711 as investigated. I have been testing this distinction in REMORA: https://github.com/darklordVirtual/REMORA-research The execution model separates proposal, policy decision, optional approval, a fresh pre-execution check, exact-call binding, governed dispatch, and effect verification: This produced a distinction that may be useful for the questions you raise here: charter authority → permission decision → semantic authority → execution authority In particular, I think this has implications for questions 1, 4, and 5. For IPermissionPolicy, I would consider returning something richer than a Boolean allow/deny result. The decision could carry policy identity, scope, constraints, decision identity, expiry, and potentially the identity of the normalized action it authorizes. That would make the policy decision independently auditable and testable. For cross-agent permissions, my preference would be attenuation by default: child authority ⊆ parent authority A subsystem Squad could declare its own charter, but its effective authority should be the intersection of its charter, the authority delegated by the parent, and relevant runtime policy. A child should not be able to widen authority simply by declaring a broader charter. Emergency elevation is where I think this becomes especially important. I would treat temporary elevation as a bounded grant rather than a mutation of the agent's standing charter. Ideally it would carry explicit scope, actor, reason, policy identity, expiry, and audit provenance. For high-risk operations it could also be single-use or bound to the exact proposed action. There is also a TOCTOU question here. An action may be permitted when OnPermissionRequest fires, but context, policy, resource state, or delegated authority may change before the actual side effect. For consequential operations I would therefore revalidate the relevant authority immediately before dispatch rather than treating an earlier approval as permanently sufficient. One reason I think the distinction matters is that we observed structurally valid but semantically wrong tool calls in external evaluation. Across a sealed 500-case tool-selection run, REMORA went from 24/500 native wrong-tool ACCEPTs with structural-only controls, to 6/500 with contracts plus intent binding, to 0/500 after adding a conservative UNKNOWN semantic floor. There was a significant utility cost, so I do not present that as a solved problem. The negative results are published as well: https://github.com/darklordVirtual/REMORA-research/blob/master/NEGATIVE_RESULTS.md That result makes me hesitant to define permission only in terms of tool categories. READ, WRITE, DELETE, etc. are useful capability classes, but they do not establish whether the selected operation corresponds to the operator's intended effect. So for the permission taxonomy question, I would probably keep the framework taxonomy relatively small and extensible, while allowing a policy implementation to evaluate richer resource, parameter, purpose, and effect constraints. Overall, I like the charter plus OnPermissionRequest split. I would see the charter as defining the agent's standing authority envelope, while the permission policy determines whether a concrete action falls within that authority at runtime. The execution boundary can then independently ensure that the action actually dispatched is the one that was authorized. |
Uh oh!
There was an error while loading. Please reload this page.
Context
PR #6146 adds a Squad + DTS incident-response sample to this repo. While writing it I kept returning to one design question worth a broader conversation: how should production agent systems gate tool execution at the agent boundary?
The two patterns in the samples today
What OnPermissionRequest enables
In the Squad model, an agent charter defines what it is allowed to do. When an action falls outside the charter, OnPermissionRequest fires before execution. The charter is operator-authored at deploy time, reusable across sessions, auditable (versioned document), and deny-by-default.
Design questions I would love input on
Should MAF expose a first-class IPermissionPolicy interface? Right now ToolApproval is a delegate on HarnessAgentOptions. An interface would let policies be injected, composed, and tested independently.
Charter persistence: config files, database, or a policy service like OPA? For multi-tenant deployments a per-tenant policy store feels necessary.
Permission taxonomy: should MAF define a standard taxonomy of permission categories, or leave it to each agent?
Cross-agent permissions: in the SquadWithDTS sample, a parent Squad delegates to subsystem Squads. Should the subsystem inherit the parent charter, or declare its own?
Emergency override: for incident response, sometimes you need to expand permissions mid-session. Should OnPermissionRequest support temporary scope elevation with auditing?
Related
All reactions