Skip to main content

    AI risk and compliance

    AI agent go-live: define what the agent may do before release

    An AI agent that reads systems, changes records or communicates externally operates with delegated access and the ability to act. A release decision needs to examine that whole arrangement as well as the agent's answers. Use this checklist to prepare a discussion between the business owner, technology, security, privacy and relevant specialists. It supports that decision and does not replace security testing, legal advice or certification.

    In short

    Check agent access and authority to act before release

    • Examine the agent's access and authority to act, as well as its answers, before you release it.
    • The Australian Signals Directorate (ASD) recommends restricting agentic AI to low-risk, non-sensitive situations. A completed checklist does not justify going beyond that boundary.
    • The checklist covers 7 areas. They are purpose and owner, information and permissions, action approval, untrusted content and failures, output acceptance, monitoring and recovery, and cost and change.
    • Record the decision in one of 4 forms: approved within a defined boundary, approved subject to specified actions, declined, or more evidence required. Name the triggers for re-review.

    Detail

    Agent checklist starts with a constrained use and owner

    Start with a limited use, as ASD's careful-adoption guidance advises. Release the agent only if the proposed boundary can be made appropriate for the use. Consider a less autonomous approach or the specialist work needed first.

    Purpose and accountable owner: the owner should be able to explain the outcome, users, context, included actions and exclusions. They should also name who operates the system, accepts the release and can stop it. Bring a written boundary for the agent's use and a map of who is responsible for it, approved by the relevant owners.

    Information and permissions: map what the agent can read, retrieve, write, send, delete or purchase, and which identities and systems it reaches. Limit permissions to the task, and do not let the agent inherit a powerful user's whole session. Bring a map of the permissions as built, the owner of the account the agent uses and the process for removing its access. Include technical evidence that the map is accurate.

    Meaningful action approval: decide which actions require a person to review the exact proposed change. The reviewer needs enough context, including the recipient, record, amount or information affected. Enforce restrictions outside the model where appropriate, and bring evidence that unapproved and prohibited actions are blocked.

    Untrusted content and failures: test what happens when a document, message or tool response contains hostile instructions. Also test missing records and a failed dependency. No single filter or prompt establishes immunity to prompt injection. Bring test results from typical and hostile cases, unresolved findings and the specialist's assessment of the risk that remains.

    Commercial impact

    Output needs an independent judge and a usable fallback

    Output acceptance needs an independent judge. Someone has to define a correct result, and testing should include normal, difficult and stop-or-escalate cases. The same model approving its own output is not independent verification. Bring an evaluation set, acceptance criteria, observed errors and the limits of what the test proves.

    Monitoring, interruption and recovery decide what happens when something goes wrong. Authorised operators need to identify the request, system version, action and outcome. They also need to be able to stop the system and remove its access. Define a fallback the organisation can execute. Some actions cannot be fully reversed once information is disclosed or a message sent.

    Operating cost and change are part of the same decision. Establish who maintains the system and handles exceptions, what the usage, support and review costs are and which changes require reassessment. A low model price does not establish a low cost per accepted outcome, so compare the complete process with a simpler alternative.

    In practice

    Permission scope and approval design in two contrasts

    Two contrasts show what the checklist is testing.

    Permissions: an agent with access limited to the task is a different case from one that inherits a powerful user's whole session. The stated purpose can be identical while the exposure differs.

    Approval design: an instruction to ask before doing anything risky is not a complete permission design. A meaningful approval shows the reviewer the exact proposed change, including the recipient, record, amount or information affected. It also blocks the prohibited action outside the model.

    Re-review triggers include new tools, permissions, information, users or autonomy.

    What to do

    Steps to limit an agent's first release and permissions

    • Restrict the first release to a low-risk, non-sensitive use. Treat a completed checklist as evidence, and do not treat it as permission to expand.
    • Limit permissions to the task instead of letting the agent inherit a powerful user's whole session. Test that the permissions as built match the map.
    • Block prohibited and unapproved actions outside the model, because a permission design needs more than an instruction to be careful.
    • Test with hostile as well as typical cases, and have someone other than the model judge the output.
    • Define a fallback the organisation can actually execute, and record the release decision with named triggers for re-review.

    Last reviewed

    Put this topic against your own numbers, owners and timelines.