Protected Health Information in AI Systems
The one-paragraph answer
HIPAA AI compliance applies whenever an AI system touches protected health information (PHI). HIPAA (Health Insurance Portability and Accountability Act) has three main rules that reach AI: the Privacy Rule governs uses and disclosures, the Security Rule requires safeguards for electronic PHI, and the Breach Notification Rule requires reporting when PHI is compromised. AI vendors processing PHI are business associates and must sign business associate agreements (BAAs). Compliance is not optional; enforcement is active.
Healthcare organizations adopting AI face immediate HIPAA questions. Can the training data include PHI? Can the vendor see PHI? Is the vendor a business associate? What safeguards does HIPAA require of AI systems? What happens if an AI system leaks PHI in a chat completion, in a log file, or in a model output? Most of these questions have clear answers under existing HIPAA law, but most healthcare organizations have never mapped their AI stack to HIPAA obligations.
Uses and disclosures of PHI must be for permitted purposes (treatment, payment, healthcare operations, or with authorization). AI systems using PHI must have documented purposes and minimum-necessary access. Marketing AI, non-clinical research AI, and third-party analytics AI often need patient authorization.
Administrative safeguards (policies, training, sanctions), physical safeguards (facility access, workstation security), and technical safeguards (access controls, audit logs, integrity controls, transmission security) all apply to AI systems handling electronic PHI. Encryption at rest and in transit is expected practice.
Unauthorized acquisition, access, use, or disclosure of unsecured PHI triggers notification obligations to affected individuals, HHS, and (for larger breaches) the media. AI incidents can qualify as breaches: a chat completion leaking PHI, a model output containing PHI, a training log exposing PHI.
AI vendors processing PHI on behalf of covered entities are business associates. BAAs must be in place before PHI is shared. BAAs impose safeguard, breach notification, and subcontractor obligations on the vendor.
De-identified data (under Safe Harbor or Expert Determination) is generally outside HIPAA. But re-identification risk is high with AI, especially generative AI. Companies training AI on de-identified data still face reputational, contractual, and (in some cases) state-law obligations. Some state laws (California, Illinois) reach beyond HIPAA's de-identification standard.
Every covered entity (health plans, healthcare providers, healthcare clearinghouses) and every business associate processing PHI is subject to HIPAA. Penalties for violations range from $137 to $2.1 million per violation category per year, plus potential criminal penalties. Enforcement by HHS OCR is active, and state attorneys general have concurrent authority.
State legislatures are beginning to pass AI-specific patient notice laws that interact with HIPAA compliance. Rhode Island H 7538 (signed June 2026) requires healthcare providers using AI for clinical documentation to notify patients that AI was used. California AB 3030 (effective January 1, 2025) requires healthcare providers using generative AI to communicate clinical information to patients to disclose the AI use and provide a path to reach a human. These laws do not amend HIPAA directly but add layered patient-notice obligations that HIPAA-covered entities must integrate with their existing privacy practices. Expect more states to follow this pattern.
The academic literature on HIPAA AI is ahead of most corporate practice, and it is unusually blunt. Two findings are worth putting in front of any executive who thinks this is a compliance formality.
“Algorithmic bias can affect AI clinical predictions and exacerbate health disparities.”
That is the gap between having AI and governing it. The second finding is the one that tends to change the room.
“a lack of a clear understanding of how to quantify benefit or ensure patient safety”
Neither of these is a fringe position. Both come from peer-reviewed work, and both describe the condition most organisations are actually in when the question about HIPAA AI arrives from the board, the buyer, or the regulator.
This is the sequence that works, and it is not the sequence most organisations choose. They start with the framework and work backwards toward reality. Start with reality.
Done in this order, HIPAA AI becomes tractable. Done out of order, it becomes a document nobody uses and a control nobody exercises.
Only if they process PHI on behalf of a covered entity or business associate. Vendors of general-purpose AI that never touch PHI are not automatically business associates.
For treatment, payment, and healthcare operations, generally yes, subject to Privacy Rule safeguards. For non-permitted purposes, patient authorization is required.
If PHI is sent to the model (even in a prompt), the model provider becomes a business associate and a BAA is required. Many popular LLM providers do offer HIPAA-compliant service tiers with BAAs.
The healthcare sector-rules addendum in Volume III of The Operating Discipline for AI Library™ addresses HIPAA-specific AI compliance. The AI Vendor Risk Inventory™ supports business associate management for AI vendors.
The authoritative texts and agency pages behind this summary. We keep this page current, but where a compliance decision turns on exact wording, read the source. Anything concerning HIPAA AI that carries legal consequence should be confirmed against the enrolled text or the issuing body, not against a secondary summary, including this one.
The AI Business Enablement Audit™ measures your organization against every framework in this library, including HIPAA and AI, and delivers a defensible governance dossier. Start or finish your audit below.
Start or finish your AI Audit →