In short
Privacy frameworks written before AI often leave gaps
- Obligations on collection, use, disclosure, security and individual rights apply to AI as they apply to any other handling of personal information.
- The Office of the Australian Information Commissioner (OAIC) has published its regulatory action priorities for 2025-26. They name practices that erode privacy rights in the application of artificial intelligence.
- Privacy frameworks written before AI often leave gaps against actual AI data practices. Your privacy and legal advisers provide contract reviews and privacy impact assessments, which identify a project's privacy risks.
- AI-specific challenges include the origin of training data, vendor data sharing, automated processing at scale and retention of prompts, outputs and logs.
Detail
Existing APPs applied in facial recognition determinations
The Privacy Act 1988 has no AI exemption. If a covered organisation's AI system handles personal information, the APPs apply to that handling. That includes collection and consent, use and disclosure, data quality and security, individual rights and accountability. The Act's existing exemptions still apply, such as the one for employee records.
The Privacy Commissioner's facial recognition determinations show the existing APPs being applied to a new technology. In October 2024 the Commissioner found that Bunnings had breached the Privacy Act. On 4 February 2026 the Administrative Review Tribunal affirmed the findings on transparency and notification. It set aside the finding on collection. In August 2025 the Commissioner made a determination against Kmart, published in September 2025.
Many organisations have a privacy framework that predates their AI deployments. It was built for data processing done by people, and AI raises issues it does not cover.
The origin of training data is the first gap. Check where the data came from and whether the purpose of collection covers AI use. Collection notices written years ago may not mention AI processing, which can leave a gap today.
Vendor data sharing is the second gap. Sending personal information to an AI vendor's service may be a disclosure under the Privacy Act. If that is a disclosure to an overseas recipient, APP 8 applies. If the vendor's terms allow model training on your data, you may have disclosed personal information for a purpose your customers would not expect.
Automated processing at scale is the third gap. When AI processes thousands of records an hour, a privacy breach can affect your entire customer base. The risk profile is different from processing by people.
Retention is the fourth gap. AI systems create prompts, outputs and logs that can hold personal information. APP 11.2 requires reasonable steps to destroy or de-identify personal information that is no longer needed for any permitted purpose. Exceptions apply, so set retention periods for these records with your privacy team.
The Privacy and Other Legislation Amendment Act 2024 also added APP 1.7 to 1.9 and new civil penalty provisions. Organisations building AI data practices now should plan for further change as well as current law.
Closing these gaps takes specialist legal work and governance foundations. Privacy impact assessments, vendor contract reviews and consent framework updates are work for your privacy and legal advisers. Applied AI Australia's AI Roadmap builds the governance foundations around that work: an AI register, board reporting and a sequenced plan for operating with AI.
Commercial impact
Privacy Act penalties and reputational damage
An organisation that assumes its existing privacy compliance covers AI may miss the gaps described above. The regulator has said where it is looking.
The penalties are significant. Under section 13G of the Privacy Act 1988, a court can order a civil penalty for a serious interference with privacy. For a company, the maximum is the greater of a fixed amount set in the Act and 3 times the benefit obtained. If the court cannot determine that benefit, a share of the company's adjusted turnover in the breach turnover period replaces it. For a listed company, that is material. For a mid-market firm, it could threaten the business.
Privacy breaches through AI also damage reputation. Suppose an AI system exposes thousands of customer records because a vendor did not secure its service. Relying on the vendor will not satisfy customers or regulators. You need documented data governance: completed privacy impact assessments, vendor contracts that set data handling requirements, consent frameworks updated for AI, and audit trails.
Data governance also affects what AI produces. Suppose customer addresses sit in 2 systems that disagree. An AI tool that drafts letters may send them to the old address, and data governance decides which record to trust.
For boards, data governance for AI is an oversight question. The Australian Securities and Investments Commission (ASIC) published Report 798 in October 2024. It states that directors' and officers' duties of reasonable care and diligence 'extend to the adoption, deployment and use of AI'. A board should be able to show that it asked how AI data practices meet the Privacy Act.
In practice
Illustrative cases of privacy reviews of AI data use
Illustrative composite: a retailer's AI customer segmentation engine used purchase history collected for transactions. A privacy impact assessment by the retailer's privacy advisers found a purpose mismatch. The collection notices said the data was used to process orders and improve service, which did not cover behavioural segmentation for targeted marketing. The fix needed updated collection notices, fresh consent from existing customers and vendor contract amendments.
Illustrative composite: a financial services firm's AI credit model processed customer data through an offshore vendor. Legal review found that this was a disclosure to an overseas recipient under APP 8. The vendor's terms let it use client data to improve its service, which included model training. The firm renegotiated the contract to remove that term, add rules on where data is stored and set audit rights.
Illustrative composite: a healthcare provider planned to use an AI tool trained on patient records to support diagnosis. A privacy impact assessment found that patients' consent to treatment did not extend to training AI models. The deployment was paused for ethics committee review and a specific consent framework for AI use of clinical data.
Illustrative composite: an insurer's AI claims processing accessed far more data fields per claim than it needed. A data minimisation review cut this to the essential fields. Collecting less data also reduced the data at risk.
What to do
Steps to check AI systems and vendor contracts for privacy
- Check every AI system against all 13 Australian Privacy Principles, including those beyond security and consent.
- Make a privacy impact assessment mandatory for high-risk AI systems, and screen every other AI system that handles personal information.
- Review every AI vendor contract for data handling, cross-border disclosure, model training rights and audit provisions.
- Set retention periods for AI prompts, outputs and logs that hold personal information.
- Update collection notices and consent frameworks so they mention AI processing.
Last reviewed
Put this topic against your own numbers, owners and timelines.