Skip to content
ushealthcloud

Compliance

HIPAA-compliant AI platforms: a buyer’s checklist

Short answer

A HIPAA-compliant AI platform signs a Business Associate Agreement, never uses your PHI to train models shared with other customers, encrypts data with keys you can control, logs every access, de-identifies by a documented HIPAA method and keeps people in charge of high-impact actions. Use the checklist below to test each claim.

Published
Last reviewed
Reading time
4 min

Why AI changes the vendor review

A traditional vendor review asks where data is stored and who can access it. AI adds new places for PHI to go: prompts, retrieval indexes, logs of model inputs and outputs, fine-tuning datasets, and the actions agents can take in your systems. Each of these needs the same scrutiny as the database itself.

The checklist

1. Business Associate Agreement

Ask: will you sign a BAA, and does it cover the AI features, model hosting, logs and support access, not just storage? A good answer is yes, signed before any PHI is shared, with subcontractors bound by the same terms.

2. Training-data use

Ask: is our PHI, or anything derived from it, used to train or improve models shared with other customers? A good answer is never, unless you agree in writing, with each customer’s data logically isolated. Watch for “we may use data to improve our services” in the terms.

3. Encryption and key management

Ask: how is PHI encrypted at rest and in transit, and who controls the keys? Look for AES-256 at rest, TLS 1.2 or higher in transit (including between internal services), FIPS 140-validated cryptographic modules, separate keys per customer and an option for customer-managed keys.

4. Access control and audit trails

Ask: who can access PHI, prompts and outputs, and how is that access recorded? Look for least-privilege roles, multi-factor authentication, tamper-evident logs that cover prompts and model outputs as well as databases, regular access reviews, and a clear answer on how long logs are kept.

5. De-identification method

Ask: when data is de-identified, which HIPAA method is used, Safe Harbor or Expert Determination, and how are identifiers in free-text notes handled? A good answer names the method, shows how residual identifiers are measured and explains who signs off.

6. Data residency

Ask: in which regions is PHI stored and processed, and does any data, including logs and backups, leave them? For programs involving people in the EU, ask which GDPR transfer mechanism applies.

7. Model evaluation and bias testing

Ask: how was the model tested, on whose data, and how does it perform across patient groups? Look for documented intended use and limitations, results on test sets that resemble your workflow, subgroup performance and an agreed plan to evaluate on your own cases before go-live.

8. Agent permissions and human review

Ask: what can an agent do on its own? Look for explicit tool allowlists, read-only access by default, human confirmation for high-impact actions such as submitting requests or writing to the chart, and a log of every tool call.

9. Incident response and breach notification

Ask: how quickly will you notify us of a security incident or breach? HIPAA requires a business associate to notify the covered entity without unreasonable delay and no later than 60 days after discovering a breach; many BAAs set a shorter deadline. Ask for the incident response plan and when it was last tested.

10. Independent attestations

Ask: which independent assessments can you share, such as a SOC 2 Type II report, HITRUST certification or ISO 27001 certificate, and do they cover the product you are buying? They are useful evidence of controls, but none is a HIPAA certification.

11. Exit and data deletion

Ask: how do we get our data back, and what is deleted when we leave? Look for return in a usable format, deletion from production systems and backups on a stated schedule, deletion of model artifacts derived from your data, and written confirmation of destruction.

Red flags

  • Claims of being “HIPAA certified”. HHS does not certify vendors.
  • Refusing a BAA, or offering one that excludes the AI features.
  • Vague terms that allow your data to be used to “improve our services”.
  • No logging of prompts, outputs or agent actions.
  • Agents with broad write access and no approval steps.
  • Attestation reports that don’t cover the product you are buying.

Want this checklist as a one-page document for your vendor reviews? Ask us for the checklist

Frequently asked questions

Can we use a general-purpose AI chatbot with PHI?
Only if the vendor signs a BAA that covers that specific service and its configuration meets the Security Rule. Consumer versions of AI tools are generally not covered by a BAA, so PHI should not be entered into them.
Who is responsible if an AI vendor mishandles PHI?
Both parties can be. The vendor, as a business associate, is directly liable under HIPAA. The covered entity must have a BAA in place and act on known problems with the vendor.
Do we need a BAA if the vendor only receives de-identified data?
Data properly de-identified under HIPAA is not PHI, so a BAA is not required for it. Confirm the de-identification method and that no re-identification codes or identifiers are shared.

Sources

General information, not legal or medical advice.

Next step

Bring us the data problem your compliance team worries about most.

We'll map where protected health information flows today, and how to model, automate and secure it.

What the first call gives you

  1. 1A map of where PHI flows today
  2. 2One workflow worth starting with
  3. 3The controls that workflow needs

A short scoping call, then a written proposal. Please don't send patient information.