Risk and accountability
Write down what the AI is never allowed to do
Create an authority matrix that separates answering questions from making commitments, changing records and exercising professional judgment.
Published · Collection dates are retrospective editorial dates.
An assistant that can describe a refund policy does not automatically need permission to issue refunds. A receptionist that can collect appointment preferences does not automatically need permission to cancel an existing booking. Capabilities and authority should be designed separately.
Write the forbidden actions before granting tools. That forces the business to decide where a conversation becomes a commitment and where a reversible action becomes consequential. Without an explicit boundary, the available integration features can quietly become the operating policy.
The authority matrix below is an original framework. It should be approved by the people accountable for the work and enforced in the application, not merely stated in a prompt.
Classify the action, not the topic
A topic such as billing contains very different tasks: explaining payment methods, identifying an invoice, proposing a correction, changing a balance and sending funds. “Handles billing questions” is too broad to define permission.
Describe each action using a verb and object. Then state the preconditions, evidence needed, approval role and record of completion. A permission becomes reviewable when someone can test whether those conditions were satisfied.
| Action class | Hypothetical example | Proposed boundary |
|---|---|---|
| Provide approved information | Explain published opening hours | Use maintained source only |
| Collect a request | Record preferred appointment time | Mark as request until verified |
| Change a record | Update a contact detail | Require approved identity checks |
| Make a commitment | Promise an exception or delivery date | Require authorized decision maker |
| Exercise professional judgment | Assess a legal or clinical question | Route through approved professional process |
The examples are hypothetical and do not set a universal authorization policy. A business may need stricter boundaries for any row.
Write negative cases in ordinary language
“Do not exceed authority” is hard to test. “Do not disclose another customer's appointment details” is concrete. “Do not promise that a lawyer will accept a matter” is concrete. “Do not convert a requested slot into a confirmed booking without verification” is concrete.
Include situations where a customer appears persuasive. A person claims to be the owner, says a supervisor already approved an exception or insists that a policy does not apply. The workflow should verify authority through the approved process rather than accepting the claim because it sounds plausible.
Distinguish customer permission from business permission. A customer may want the assistant to do something the business has not authorized. Their request does not expand the application's role.
Keep enforcement outside the conversation
OWASP's excessive-agency guidance identifies excessive functionality, permissions and autonomy as sources of risk. For a buyer, the practical question is which downstream controls prevent excluded actions.
Ask the implementation team to show the service account permissions, tool interface and approval checks. Can the integration perform actions the workflow does not need? Can a record identifier supplied by a customer redirect the action to someone else's account? Can a model-produced value bypass a business rule?
A clear prompt is useful for behavior. It should not be the only barrier protecting a consequential action. Test the barrier by requesting the excluded action and inspecting the actual system response.
This is a security recommendation, not a guarantee that any one control eliminates all risk.
Define the person who may approve an exception
A handoff for approval needs a named role with the necessary authority. Sending a request to any available employee can merely move the unauthorized decision to someone else.
Specify what evidence the approver receives and how approval is recorded. Tie it to the actual action, not an open-ended permission that can be reused later. If approval expires or circumstances change, the workflow should obtain a fresh decision through the approved process.
In a hypothetical rental business, approval for a particular late return should not authorize future late returns for that customer. The scope of the exception belongs in the record.
Check the language that creates commitments
Even a read-only assistant can make a promise. “You are eligible,” “the refund is approved,” or “your place is reserved” may imply a decision the system did not actually make.
Review customer wording alongside tool permissions. Define which claims require which evidence. Use accurate pending language when the decision or action remains unfinished. Do not let polished wording hide the difference between explaining a policy and applying it to a particular case.
Authority belongs in permissions, records and customer expectations at the same time.
That is an original operating principle. A mismatch between any of those layers creates work the business will eventually have to reconcile.
Revisit authority when the workflow changes
Adding a new channel, location or tool can expand the consequences of the same interaction. Recheck the matrix whenever capabilities change. Review unused permissions and remove them through the appropriate administrative process.
Ask a frontline employee and a technical owner to walk through the same forbidden cases. They should agree on what the customer hears, what the system attempts and who receives the unresolved request.
The matrix is ready when a reviewer can point to the control and evidence for each boundary. If the only answer is that the assistant has been told to behave, keep the consequential action with a person until the application provides an enforceable alternative.
Sources
Explore the workflow considerations for customer intake qualification.
Keep reading
