HIPAA-Compliant Self Hosted AI for Healthcare
Most healthcare organizations are already using AI tools that route patient data through external APIs. Some have a BAA in place. Most have not looked closely at what that BAA actually covers, or at the breach statistics for third-party vendors. This page is for compliance officers, IT leads, and administrators who need to close that gap without shutting down AI adoption entirely.
A self hosted deployment keeps the model inside your network. ePHI never leaves. The AI satisfies HIPAA's Security Rule by architecture rather than by contract, integrates with Epic, Cerner, and other EHR systems via FHIR R4 and HL7 v2, and eliminates the BAA exposure problem at its root. Deployments typically run $8,000 to $40,000 depending on scale and workflow scope.
Why cloud AI is a HIPAA problem your BAA does not fix#
Most AI tools marketed to healthcare as "HIPAA-compliant" rely on a Business Associate Agreement and a SOC 2 certificate to make that claim. That framing leaves out what actually matters.
The third-party breach problem: 58% of healthcare records exposed in 2023 came through vendors#
In 2023, 58% of the 77.3 million individuals affected by healthcare data breaches were exposed through third-party vendor compromises, a 287% increase from 2022 (HIPAA Journal, 2024). In 2024, 276.8 million records were breached, a 64.1% increase over 2023 (HIPAA Journal, 2024). The most frequent breach vector is not your internal system. It is a third party who has access to your data.
When you route ePHI through a cloud AI API, you are adding a third party to your data flow. A BAA governs that third party's obligations. It does not eliminate the exposure.
What a BAA actually covers and what it doesn't#
A Business Associate Agreement is a contractual instrument. It creates obligations, assigns liability, and establishes an audit trail. What it does not do is prevent ePHI from transiting the vendor's infrastructure, prevent that infrastructure from being breached, or guarantee that model training pipelines exclude patient data unless those exclusions are explicitly scoped, verified, and enforced.
Many commercial AI platforms offer "zero data retention" tiers. Those are contractual commitments, not architectural ones. Your ePHI still leaves your network on every API call. The data transits infrastructure you do not control, at an endpoint operated by a third party with its own security posture, staff access policies, and government subpoena exposure.
The 2025 HIPAA Security Rule update: all safeguards are now mandatory#
The January 2025 HIPAA Security Rule update removes the "required vs. addressable" distinction that allowed covered entities to treat MFA and encryption as optional based on risk assessment. Both are now mandatory for all systems accessing ePHI, with penalties up to $1.9 million annually per violation (HHS Federal Register, January 2025). If your AI stack doesn't implement MFA and encrypted storage by architecture rather than by policy document alone, you are out of compliance regardless of what your BAA says.
What HIPAA-compliant self hosted AI looks like in practice#
The difference between "HIPAA-compliant by contract" and "HIPAA-compliant by architecture" is where the inference happens. In a self hosted deployment, the model runs inside your network. ePHI is processed by infrastructure you own and control. It never leaves.
Private inference layer: ePHI never transits external infrastructure#
The AI model, whether a general-purpose LLM fine-tuned for clinical language or a specialized coding model, runs on hardware inside your data center, private cloud, or on-premise server room. Queries from clinical staff go to your inference server. Responses come back from your inference server. No call is made to an external API. The data residency boundary is your network perimeter.
This is the only architecture that eliminates the third-party data exposure problem structurally rather than contractually.
EHR integration via FHIR R4 and HL7 v2 (Epic, Cerner, and others)#
Clinical AI is only useful if it connects to where clinical data lives. We integrate directly with your EHR through FHIR R4 APIs and HL7 v2 interface engines, the standards used by Epic, Cerner, athenahealth, and most other major systems. SMART on FHIR handles OAuth2-based authentication against your existing identity management infrastructure.
Integration scopes are defined at the deployment level. The AI system accesses only the data it needs for its configured workflows. Clinical documentation assistants get read/write access to notes. Prior authorization workflows access the patient record fields needed to populate forms. Billing automation accesses claims data. Nothing is granted broadly.
Role-based access control, MFA, and audit logging: mandatory under 2025 rules#
Every deployment includes RBAC configured against your existing LDAP or Active Directory, multi-factor authentication, and a complete audit log pipeline. The audit log captures every query, response, and user action, timestamped, attributed, and retained per your policy. This is the evidence trail HIPAA audits require and that the 2025 Security Rule updates make mandatory.
Use cases: clinical documentation, prior auth, billing automation, patient intake#
The workflows we deploy most frequently are the ones that combine high administrative burden with ePHI exposure risk:
- Clinical documentation: AI-assisted note generation from provider voice input or structured prompts, writing directly to the EHR without leaving your environment
- Prior authorization: Automated workflow that reads the patient record, generates the clinical justification, and submits to the payer without a human copying data between systems
- Medical coding: NLP-assisted ICD-10 and CPT code suggestions from clinical notes, with the human coder reviewing and confirming
- Patient intake and scheduling: Reminder workflows and intake form processing connected to your scheduling system, without routing patient data through an external platform
How we build it#
Every healthcare AI deployment follows the same four-phase process, designed to surface compliance requirements before technical decisions rather than after.
Step 1: compliance and infrastructure assessment#
We start with your current environment: what systems handle ePHI, where your network perimeter sits, and what your IT team can support operationally. This assessment produces a written deployment scope that feeds directly into your HIPAA documentation update.
Step 2: model selection and hardware sizing for clinical workloads#
Clinical workloads have specific requirements: medical terminology, abbreviation handling, note structure. We select the model (or combination of models) appropriate for your workflows, then size the inference hardware based on concurrent user count and throughput. Small practices with 5-20 clinical staff have very different hardware profiles than health systems running high-throughput coding automation.
Step 3: EHR integration and access control configuration#
This is typically the most variable phase. We configure the FHIR R4 or HL7 v2 integration layer, set up SMART on FHIR or direct LDAP authentication, define RBAC roles, and configure MFA enforcement. Everything is tested against your EHR sandbox environment before anything touches production data.
Step 4: audit logging, monitoring, and handoff documentation#
We deploy the audit log pipeline, configure retention policies, and document every component in language your compliance team can use for risk assessments and audit responses. We train your staff on the system and hand off operational runbooks so your IT team can manage it independently from day one.
Tech stack#
Every component runs inside your environment. Nothing in this stack requires an outbound connection to a cloud service.
Inference layer: Ollama (team deployments), vLLM (high-throughput production)#
Ollama is our default inference engine for medical practices and mid-size health systems: straightforward to operate, GPU-optional for smaller workloads, and manageable by an internal IT team without specialized ML expertise. For high-throughput environments processing large document volumes or many concurrent users, we deploy vLLM for its performance under load.
EHR connectivity: FHIR R4 APIs, HL7 v2 interface engines, SMART on FHIR#
We use FHIR R4 for modern EHR integrations and HL7 v2 for systems that predate FHIR or have FHIR support limited to specific modules. SMART on FHIR handles OAuth2 authentication against EHR identity providers. All integration traffic stays inside your network.
RAG and retrieval: clinical document stores with pgvector or ChromaDB#
Clinical AI often needs access to your organization's own protocols, formularies, coding guidelines, and historical documentation. We build retrieval-augmented generation (RAG) pipelines over your document corpus using pgvector (if you're running PostgreSQL) or ChromaDB for standalone vector storage. Both run entirely on-premise.
Access and compliance: Open WebUI with LDAP/AD auth, audit log pipelines, network segmentation#
Open WebUI provides the staff-facing interface with LDAP/Active Directory authentication, role-based access controls, and a user management layer your IT team can administer. Audit log pipelines capture all system activity in a format suitable for HIPAA compliance documentation. Network segmentation isolates the AI inference layer from general network traffic.
Healthcare outcomes#
Clinical documentation: AI-assisted notes without sending text to external servers#
Clinical staff using our self hosted documentation assistant spend less time on notes. Because inference runs locally, there is no latency from external API round-trips. Response times are comparable to or faster than cloud-based tools depending on hardware configuration.
Prior authorization: automated workflow that reads and writes to the EHR#
Prior authorization is one of the highest-burden administrative workflows in healthcare. Our automated prior auth workflow reads the relevant patient record fields, generates the clinical justification text, and populates the payer form, all inside your environment. Staff review and submit. ePHI never leaves your network in the process.
Medical coding: NLP-assisted coding reduced claim denials by 34% in documented deployments#
AI-assisted coding from clinical notes reduced claim denials by 34% in our documented healthcare deployments. The model suggests ICD-10 and CPT codes from the note text; the coder reviews, confirms, and submits. Denials drop because coding is more complete and consistent, not because the human is removed from the loop.
No-show reduction: appointment reminder pipelines integrated with the scheduling layer#
No-show rates dropped from 21% to 7% in deployments where we integrated AI-assisted reminder workflows with the scheduling system. Reminders are personalized based on patient communication history and sent through your existing messaging infrastructure. No patient data routes through an external automation platform.
Pricing#
Healthcare AI deployments range from $8,000 to $40,000 depending on scope, workflow count, hardware requirements, and EHR integration complexity.
A typical small-practice deployment (1-3 workflows, 5-25 staff, Ollama on existing hardware) runs at the lower end. A health system deployment with multiple EHR integrations, high-throughput inference, and custom clinical documentation workflows sits at the higher end.
Ongoing support, model updates, and operational monitoring are available as a separate retainer after initial deployment. We scope every engagement before quoting. Contact us to start with a compliance and infrastructure assessment.
FAQ#
What makes an AI system HIPAA compliant for healthcare?
HIPAA compliance for an AI system requires that ePHI never leaves a controlled environment where you can enforce access controls, audit usage, and demonstrate compliance. A self hosted deployment satisfies this by running the AI model inside your network. A cloud-based tool with a BAA satisfies the contractual requirement but does not eliminate the underlying data exposure: ePHI still transits third-party infrastructure.
Can you use self hosted AI for clinical notes and EHR workflows?
Yes. We integrate self hosted AI directly with EHR systems through FHIR R4 APIs and HL7 v2 interfaces, the standards used by Epic, Cerner, athenahealth, and other major platforms. Clinical note assistance, prior authorization workflows, and coding automation all run against your EHR data without that data leaving your environment.
Does a self hosted LLM require a Business Associate Agreement?
If the inference runs entirely on infrastructure you own and operate, and no third party has access to that infrastructure or the data processed on it, there is no business associate relationship to cover under a BAA. That is the compliance advantage of genuine on-premise deployment: the BAA gap doesn't exist because no business associate is involved. Consult your privacy officer to confirm the specific scoping for your environment and any managed services involved.
What is the risk of using cloud AI with patient data?
The primary risks are: third-party breach exposure (your vendor's infrastructure becomes part of your attack surface), model training data inclusion (unless explicitly scoped out and technically verifiable), government subpoena exposure (data held at a third-party vendor is accessible through legal process against that vendor), and regulatory penalty under the 2025 HIPAA Security Rule updates if safeguards are not implemented by architecture.
How much does HIPAA-compliant AI infrastructure cost?
Deployments typically range from $8,000 to $40,000 for initial build-out. The range reflects differences in workflow count, staff size, hardware requirements, and EHR integration complexity. We do not quote from a price list. Every engagement starts with a compliance and infrastructure assessment that produces a scoped estimate.
Start with a compliance assessment#
If you are using any AI tool that routes ePHI through an external API, or if your compliance team has flagged your current AI setup, start with an infrastructure and compliance assessment.
We will map your current ePHI data flows, identify where your AI stack creates exposure, and scope what a self hosted deployment would require for your environment.
Request a compliance and infrastructure assessment, or contact us directly to discuss your specific workflows and timeline.
See also:
