Skip to main content

    AI Risk & Compliance

    Before an agent goes live, define what it is allowed to do.

    An AI agent that reads systems, changes records or communicates externally operates with delegated access and action capability. A release decision needs to examine that whole arrangement, not only its answers. Use this checklist to prepare a discussion between the business owner, technology, security, privacy and relevant specialists. It is a decision aid, not a security test suite, legal opinion or certification.

    In short

    Executive Summary

    • An AI agent that reads systems, changes records or communicates externally operates with delegated access and action capability, so the release decision has to examine that whole arrangement rather than only its answers.
    • ASD's careful-adoption guidance recommends restricting agentic AI to low-risk, non-sensitive situations, and a completed checklist does not justify expanding beyond that boundary.
    • The checklist covers purpose and accountable owner, information and permissions, meaningful action approval, untrusted content and failures, output acceptance, monitoring and recovery, and operating cost and change.
    • The decision is recorded as approved within a defined boundary, approved subject to specified actions, not approved, or more evidence required, with named triggers for re-review.

    Detail

    Overview

    Start with a constrained use. ASD's careful-adoption guidance recommends restricting agentic AI to low-risk, non-sensitive situations. If the proposed boundary cannot be made appropriate for the use, do not release it, and consider a less autonomous approach or the specialist work needed first.

    Purpose and accountable owner. Can the owner explain the outcome, users, context, included actions and exclusions? Who operates the system, accepts the release and can stop it? Bring a written use-case boundary and responsibility map approved by the relevant owners.

    Information and permissions. What can the agent read, retrieve, write, send, delete or purchase, and which identities and systems does it reach? Permissions should be limited to the task rather than inherited from a powerful user's whole session. Bring the implemented permission map, the identity owner and the access-removal process, with technical validation.

    Meaningful action approval. Which actions require a person to review the exact proposed change, and does the reviewer see enough context, including the recipient, record, amount or information affected? Restrictions should be enforced outside the model where appropriate. Bring evidence that unapproved and prohibited actions are blocked.

    Untrusted content and failures. What happens when a document, message or tool response contains hostile instructions, or when records are missing, information contradicts, a tool fails repeatedly or a dependency is unavailable? No single filter or prompt establishes immunity to prompt injection. Bring representative and adversarial test results, unresolved findings and the specialist assessment of residual exposure.

    Commercial impact

    Why It Matters for Organisations

    Output acceptance needs an independent judge. Someone has to define a correct result, and testing should include normal, difficult and stop-or-escalate cases. The same model approving its own output is not independent verification. Bring an evaluation set, acceptance criteria, observed errors and the limits of what the test proves.

    Monitoring, interruption and recovery decide what happens when something goes wrong. Authorised operators need to identify the request, system version, action and outcome, and to interrupt operation and revoke access. Define a fallback the organisation can execute, remembering that some actions cannot be fully reversed once information is disclosed or a message sent.

    Operating cost and change are part of the same decision. Establish who maintains the system and handles exceptions, what the usage, support and review costs are, and which changes require reassessment. A low model price does not establish a low cost per accepted outcome, so compare the complete process with a simpler alternative.

    Podcast

    Listen to how Australian executives are applying AI

    Use the podcast to pressure-test the ideas in this article against real operator conversations. Each episode focuses on what leaders are shipping, where the friction is, and what actually lands.

    The trusted source for Australian executives deciding on AI strategy, governance and adoption. I translate technical complexity into practical business outcomes: growth, margins and time to value.

    In practice

    Examples or Practical Context

    Two contrasts show what the checklist is testing.

    Permissions: an agent whose access is scoped to the task is a different proposition from one that inherits a powerful user's whole session. The stated purpose can be identical while the exposure is not.

    Approval design: an instruction to ask before doing anything risky is not a complete permission design. A meaningful approval shows the reviewer the exact proposed change, including the recipient, record, amount or information affected, and blocks the prohibited action outside the model.

    Record the decision in one of four forms: approved within a defined boundary, approved subject to specified actions, not approved, or more evidence required. Identify the triggers for re-review, including new tools, permissions, information, users or autonomy.

    What to do

    Key Takeaways

    • Restrict the first release to a low-risk, non-sensitive use, and treat a completed checklist as evidence rather than as permission to expand.
    • Scope permissions to the task instead of inheriting a powerful user's whole session, and validate the implemented map technically.
    • Enforce prohibited and unapproved actions outside the model, because an instruction to be careful is not a permission design.
    • Test with adversarial as well as representative cases, and have someone other than the model judge the output.
    • Define a fallback the organisation can actually execute, and record the release decision with named triggers for re-review.

    Newsletter

    Get the Executive Brief each week

    Stay ahead of the next board question with short, practical analysis built for Australian executives. It cuts past recycled AI news and focuses on the decisions that matter now.

    The trusted source for Australian executives deciding on AI strategy, governance and adoption. I translate technical complexity into practical business outcomes: growth, margins and time to value.

    Assessment

    Run the AI Readiness Assessment

    Check whether policy, accountability, and compliance are keeping pace with deployment. The assessment scores governance and decision control alongside four other dimensions.

    The trusted source for Australian executives deciding on AI strategy, governance and adoption. I translate technical complexity into practical business outcomes: growth, margins and time to value.

    More detail

    Additional Context

    Start with a constrained use

    ASD's careful-adoption guidance recommends restricting agentic AI to low-risk, non-sensitive situations. A checklist does not justify expanding into sensitive or high-impact activity because every row has a tick.

    If the proposed boundary cannot be made appropriate for the use, do not release it. Consider a less autonomous approach or the specialist work needed first.

    Purpose and accountable owner

    Can the owner explain the outcome, users, context, included actions and exclusions? Who operates the system, accepts the release and can stop it?

    Bring a written use-case boundary and responsibility map approved by the relevant owners.

    Information and permissions

    What can the agent read, retrieve, write, send, delete or purchase? Which identities and systems does it reach? Are permissions limited to the task rather than inherited from a powerful user's whole session?

    Bring the implemented permission map, identity owner and access-removal process, with technical validation.

    Meaningful action approval

    Which actions require a person to review the exact proposed change? Does the reviewer see enough context, including the recipient, record, amount or information affected?

    Restrictions should be enforced outside the model where appropriate. “Ask before doing anything risky” is not a complete permission design.

    Bring evidence that unapproved and prohibited actions are blocked.

    Untrusted content and failures

    What happens when a document, message or tool response contains hostile instructions? What happens with missing records, contradictory information, repeated tool failure or an unavailable dependency?

    No single filter or prompt establishes immunity to prompt injection. Bring representative and adversarial test results, unresolved findings and the specialist assessment of residual exposure.

    Output acceptance

    Who defines a correct result? Does testing include normal, difficult and stop-or-escalate cases? The same model approving its own output is not independent verification.

    Bring an evaluation set, acceptance criteria, observed errors and the limits of what the test proves.

    Monitoring, interruption and recovery

    Can authorised operators identify the request, system version, action and outcome? Can they interrupt operation and revoke access?

    Define a fallback the organisation can execute. Some actions cannot be fully reversed after information is disclosed or a message sent.

    Bring tested monitoring, escalation, interruption and recovery arrangements.

    Operating cost and change

    Who maintains the system and handles exceptions? What are usage, support and review costs? Which changes require reassessment?

    A low model price does not establish a low cost per accepted outcome. Compare the complete process with a simpler alternative.

    Record the decision

    State approved within a defined boundary, approved subject to specified actions, not approved, or more evidence required. Identify triggers for re-review, including new tools, permissions, information, users or autonomy.

    The point is a shared, evidenced understanding of what the organisation has authorised, not a completed form.