Physicians have been using AI tools in some form for many years, but in 2022 something changed. Shadow AI exploded in healthcare and doctors quickly adopted this transformative technology. One major, unexpected challenge emerged as patient’s private information leaked. And patients have zero idea for the most part.
Healthcare systems were largely unprepared for this challenge and are now attempting to play a costly game of catch up. In 2025 alone 772 healthcare data breaches exposed the personal information of nearly 140 million Americans, nearly one in two people in this country. According to IBM's 2024 Cost of a Data Breach Report the average healthcare data breach costs an affected organization $10.9 million, the highest of any industry for 13 consecutive years. With hundreds of breaches occurring annually the collective financial burden on American healthcare runs into the billions and costs that are ultimately passed on to patients through higher premiums and reduced resources for care.
If you were a patient at a hospital, and your doctor was utilizing this technology, would you like to know?
Hospitals can’t tell you either way because they often lack formal AI policies and reasonably, wouldn’t track clinicians’ personal devices. So, if a clinician is not well versed in AI, they are probably not aware this technology is not HIPAA regulated.
For example, a user friendly platform makes it easy to forget they are not interacting with a private health platform, causing a trade off between convenience and compliance. A commercial enterprise software alternative often integrates at a slow and clunky pace in comparison.
5 Ways Patient Data Gets Leaked
So, how exactly does patient data get leaked ending up in all the wrong places?
Through numerous pathways that are more difficult to protect than people might think…….
Pathway 1-Server Storage
In one scenario, a clinician types patient information into a commercial AI platform. This conversation is hardly private, rather it travels through the internet and ends up being stored on the AI server. AI conversations are often stored for the purpose of training and improving future AI models. The unconsented data transfer lacks a guarantee of deletion, even upon request. Paid versions of this technology offer more privacy than a free version. Conversations might be deleted after a period of time but many are stored indefinitely.
Pathway 2-Cybercriminals and The Black Market
Consumer AI platforms do not operate under legal requirements that prohibit the collection, retention and access of patient data. Hospitals operate under HIPPa privacy regulation where patients are aware their EHR’s are not to be shared. But stored AI is not the only potential risk, because consumer AI companies are often the target of cyber criminals with healthcare data. Full patient health records are sold from a range of $250-1000 dollars on the black market for the purpose of prescription medical fraud, identity theft, insurance fraud and other scams. Medical records are permanent as well, and cannot be reissued like a lost or stolen credit card.
Pathway 3-AI Model Training Contamination
When a clinician types in a patients personal information into consumer AI it may be stored permanently. These details can be used for reference and used as a building block for additional conversational research on the platform. Not a substantial data breach in itself. More like a tiny, irreversible, contaminant submitted unknowingly on the patients behalf.
Pathway 4-Personal Device Vulnerability
A busy doctor inputs patient's private information into a consumer AI platform on their phone or laptop that is not authorized hospital equipment. Of course a personal device would not be controlled/ monitored by company cyber security. If a device is lost or stolen patient data goes with it.
Pathway 5-Screenshots and Forwarding
In this scenario a clinician shares patient details with a colleague texted/emailed via screenshot, with exposure risk growing with every share
Why Mid-Market Hospitals are the Most Vulnerable
Large and well resourced health organizations like Mayo Clinic and Kaiser Permanente employ cohesive and well trained cyber security teams, AI governance frameworks, and company specific enterprise solutions. Mid-market hospitals, community health centers, regional health groups and private clinics operate in a different world. Lacking HIPAA compliant cyber security measures that suit organizational needs, they are often left with lower budget consumer AI tools. Integrating more sophisticated enterprise systems into existing infrastructure is another challenge. This gap in compliance is most prevalent across mid market and rural facilities where AI usage and related policies are lacking entirely. Leaving patients in this community especially vulnerable.
An era of AI awareness and basic health literacy has arrived and patient privacy is among the most pressing concerns. Surveys from organizations including the American Medical Association consistently show that privacy of personal health information ranks as a top priority for patients across the country, with large majorities viewing it as a fundamental right.
Patients should feel empowered to ask their providers directly, what AI tools does this hospital formally approve for clinical use? Does your organization have a HIPAA compliant AI usage policy? These are reasonable questions that every patient has the right to ask, and that every hospital should be prepared to answer. No patient walks into a clinic expecting to become an unpaid participant in a digital health lab experiment.
Consumer AI tools have quickly integrated into our personal lives and every major industry with healthcare being no exception. Most of us use these tools daily yet rarely ask the most important question: 'But where does all of this information go?'
Personal health information is more vulnerable than most people realize — susceptible to long term storage on consumer AI servers with no guarantee of deletion or anonymity. No patient should unknowingly become part of a permanent non-anonymous digital medical case study. Beyond storage risks consumer AI platforms are increasingly attractive targets for cybercriminals and as previously noted healthcare records command premium prices on the black market (estimated at $499 per patient breach).
Clinicians need purpose built specialized AI tools where sensitive information stays within secure clinical environments, not consumer platforms designed for general use. The answer is definitely not monitoring a clinician's cell phone, tablet, or personal computer. It's giving them better tools in the first place.
That's what The Intake Brief will keep advocating for.