On-Premise vs Cloud AI for Healthcare: GDPR Trade-Offs

Choosing where a healthcare AI system runs looks like an infrastructure question: which servers, which region, which vendor. In practice it is a data-governance decision, and the infrastructure follows from it. The question that settles most cases is not "cloud or on-premise?" but "which data leaves our control, in what form, and who can prove what happened to it afterwards?" Answer that precisely and the deployment model usually chooses itself.

This guide is for healthcare startups, clinics, medical software companies and digital health teams in Europe who want to use AI with sensitive health data and are unsure whether cloud AI is acceptable. It compares public cloud AI, private cloud and on-premise deployment, sets out when each is defensible, and describes the architecture patterns that let most teams avoid an all-or-nothing choice. It is technical and strategic guidance, not legal advice: the legal basis for processing and the final call on transfers belong with your data protection officer and counsel.

Why does health data raise the bar for AI deployment?

Health data raises the bar for AI deployment because GDPR treats it as a special category of personal data under Article 9. Processing it is prohibited unless one of a short list of conditions applies, such as the provision of health care, public health, scientific research or explicit consent. That single classification cascades into every infrastructure choice. Large-scale processing of health data will normally require a data protection impact assessment (DPIA) under Article 35(3)(b), and healthcare providers above the size thresholds also fall under the NIS2 Directive as a sector of high criticality, which adds cybersecurity risk management on top. None of this forbids AI or cloud. It means every place patient data flows has to be justified, secured and documented, and an AI system typically adds several new flows at once: prompts, embeddings, logs, evaluation sets and model outputs. Five properties decide whether a deployment can meet that bar.

Special-category data

Special-category status means health data needs both an Article 6 lawful basis and an Article 9 condition before any processing, including processing by an AI vendor on your behalf. It covers more than diagnoses: appointment histories, prescriptions, referral letters and free-text notes that mention a condition all qualify. Teams often classify the obvious fields correctly and miss the free text, which is exactly what language models consume.

Data minimisation

Data minimisation, under Article 5(1)(c), requires that processing be limited to what is necessary for the purpose. For AI this is concrete: a model that summarises discharge letters does not need the patient's name, insurance number or address to do its job. Every field you strip before inference is a field that cannot leak, cannot be retained by a provider and does not need a transfer assessment.

Access control

Access control in an AI system has to extend to what the model can retrieve, not just who can log in. A clinical assistant that searches every document in a repository will happily surface a record the asking clinician was never authorised to see. Article 32 requires appropriate technical measures, and in practice that means the retrieval layer enforces the same role-based permissions as the source system.

Auditability

Auditability means being able to reconstruct who asked what, which data the system used and what it answered, months later. GDPR's accountability principle in Article 5(2) expects you to demonstrate compliance, and the EU AI Act requires automatic logging for high-risk systems under Article 12, with deployers keeping those logs for at least six months. A deployment you cannot audit is one you cannot defend.

Data residency

Data residency concerns where data is stored and processed, and whose law can reach it. Transfers of personal data outside the EEA need a legal mechanism under Chapter V of GDPR, and the two previous EU-US frameworks were both invalidated by the Court of Justice, in 2015 and 2020. EU hosting reduces transfer exposure but does not settle everything: a provider's parent company, support access and sub-processors can still sit elsewhere.

What is the difference between cloud AI, private cloud and on-premise deployment?

The difference between cloud AI, private cloud and on-premise deployment is who operates the infrastructure that touches patient data, and therefore whom you must trust and contractually bind. The three models are points on a spectrum rather than rival camps, and most organisations end up using more than one.

Public cloud AI means calling a model hosted by a provider over an API, or using a managed AI service in a hyperscaler region. You send data out, the provider runs inference and you get a result back. You gain the most capable models, almost no operational burden and fast iteration; you depend on the provider's contract, retention settings and sub-processors.

Private cloud means dedicated or single-tenant infrastructure, typically in an EU region, where you deploy and control the model yourself: an open-weight model on reserved GPU instances in your own cloud account, or a provider's dedicated deployment with isolated tenancy. Data still leaves your building, but not your administrative control.

On-premise means the model runs on hardware in your own data centre or hospital server room, inside your network perimeter. Nothing leaves unless you send it. You carry the full burden of hardware, patching, monitoring and model updates, and you are limited to models you are licensed to run locally.

How do cloud, private cloud and on-premise AI compare for healthcare?

Cloud, private cloud and on-premise AI compare differently on every axis that matters in healthcare, so the useful comparison is not which is "most secure" but which risks each one moves, and to whom. The table below summarises the trade-offs we see most often when scoping healthcare AI projects in Europe, with hybrid included because it is where most teams land.

Two points in the table deserve emphasis. First, security is not a property of location: a well-run cloud tenancy is often better defended than an under-staffed hospital server room, and the risks listed for on-premise are real. Second, the hybrid row is the common answer, which is why the architecture patterns later in this guide matter more than the headline choice.

When is cloud AI acceptable for healthcare data?

Cloud AI is often acceptable for healthcare data when the conditions around it, rather than the cloud itself, are right. European regulators have not declared cloud processing of health data unlawful; they expect the controller to justify it. Germany went further in 2024 and wrote cloud use into law: section 393 of the Social Code Book V permits healthcare providers and statutory health insurers to process health data in the cloud, provided processing happens in the EU, the EEA or a country with an adequacy decision, and the provider holds a current C5 attestation from the Federal Office for Information Security (BSI). The five conditions below are the ones that, in our experience, make a cloud deployment defensible. The more of them you meet, the stronger the case, and a use case that meets none of them belongs in the next section.

No patient-identifiable data leaves your environment

Cloud AI is easiest to justify when the data sent is not personal data at all: truly anonymised records, synthetic test data, published guidelines or administrative text with no patient reference. Be careful with the word "anonymised". Pseudonymised data remains personal data under Article 4(5), and free-text clinical notes are notoriously hard to anonymise because identity hides in dates, rare conditions and family details.

Strong contractual controls

Strong contractual controls start with an Article 28 data processing agreement, but for AI they must also cover what is specific to models: that your inputs and outputs are not used for training, how long prompts and completions are retained, and whether abuse monitoring stores content for human review. Many enterprise AI services offer zero or short retention on request. Get it in writing, for the specific service you actually call.

EU hosting and data residency

EU hosting should mean that both storage and inference happen in an EU region, not only the data at rest. Check the processing location of the model endpoint itself, because some services route requests to wherever capacity is available. Where a provider is headquartered outside the EU, ask how it handles government access requests, since EU hosting reduces but does not remove that exposure. Record the answer in your transfer impact assessment.

Clear processor and sub-processor terms

Clear processor and sub-processor terms mean you know every company that can touch the data, and Article 28(2) gives you the right to object to changes. AI services often chain providers: an application vendor calls a model provider, which runs on a hyperscaler. Ask for the full sub-processor list, how you are notified of changes and where each party is located. If a vendor cannot name who runs inference, that is your answer.

Human review and logging

Human review and logging make a cloud deployment defensible after the fact. Keep a clinician in the loop for anything that affects care, and log every request, the data categories involved, the model version and the output, in a store you control rather than only the provider's console. The EU AI Act expects exactly this for high-risk systems under Articles 12 and 14, and it is good practice well below that threshold.

When is on-premise or private AI deployment preferable?

On-premise or private AI deployment is preferable when the data, the organisation or the workflow makes external processing hard to justify, not merely uncomfortable. Be honest about the trade: you give up the easiest access to frontier models and take on hardware and operations. Open-weight models have narrowed the gap considerably, though. A model in the 7 to 8 billion parameter range runs on a single data-centre GPU, and a 70 billion parameter model fits on one 80 GB accelerator with 4-bit quantisation. For many clinical extraction, classification and summarisation tasks, that is enough capability. Five situations push us toward local or private deployment most often.

Highly sensitive patient data

Highly sensitive patient data, such as psychiatric notes, sexual health records, genetic data or records of minors, carries a higher impact if exposed, and your DPIA will say so. Genetic data is its own special category under Article 9. When the residual risk after every cloud safeguard is still judged high, keeping inference local is the most direct mitigation and the easiest to explain to patients and supervisory authorities.

Hospitals with strict IT and security rules

Hospitals with strict IT and security rules often decide the question before any architecture review: many forbid clinical data leaving the network, or require months of security review for each new external processor. Fighting that policy costs more time than building within it, and an on-premise system that passes review in weeks usually beats a cloud system stuck in procurement for a year.

Medical imaging workflows

Medical imaging workflows are a strong case for local inference on practical grounds as much as legal ones. A single CT study can run to hundreds of megabytes and a whole-slide pathology image can exceed a gigabyte, and the data already sits in the hospital's PACS. Moving it out for inference adds latency, bandwidth and another copy to govern. DICOM headers and burned-in annotations also carry identifiers that are easy to miss.

Internal clinical knowledge systems

Internal clinical knowledge systems, such as assistants that answer questions over local protocols, formularies and patient records, need retrieval over data that is both sensitive and permission-controlled. Running the retrieval index and the model inside the network keeps the access model simple: the system inherits existing directory permissions and never needs to replicate records to an external store just to search them.

Data that cannot leave the environment

Data that cannot leave the environment is sometimes a contractual fact rather than a risk judgement. Research data governed by an ethics approval, data shared under a hospital partnership, or records covered by national rules on medical secrecy may carry explicit restrictions on external processing. When the answer is fixed by contract or law, the only architecture question left is how to deliver useful AI within that boundary.

Which architecture patterns make healthcare AI both useful and GDPR-aware?

The architecture patterns that make healthcare AI both useful and GDPR-aware all apply the same idea: decide which data each component actually needs, and put the boundary of your control exactly there. Article 25 of GDPR calls this data protection by design and by default, and it is far easier to build in on day one than to retrofit after a DPIA flags it. Article 32 names pseudonymisation and encryption explicitly as appropriate security measures, and the first pattern below is built directly on the former. These five patterns cover most of what we build for healthcare and medical software teams, and they are designed to be combined rather than chosen between.

De-identification before cloud processing

De-identification before cloud processing strips or replaces direct identifiers inside your environment, sends only the reduced text to a cloud model, and re-attaches identifiers locally on the way back. Names, dates, addresses and record numbers become consistent placeholders, so the model can still reason about "patient A" across a document. Treat the de-identifier as safety-critical and measure its recall on your own documents, because a missed name is a disclosure.

Retrieval-augmented generation over controlled document stores

Retrieval-augmented generation (RAG) keeps clinical knowledge in a document store you control and passes only the few relevant passages to the model at query time, instead of training on the data. In healthcare that has three benefits: nothing sensitive is baked into model weights, access control is enforced at retrieval, and every answer can cite the source passages it used, which makes outputs traceable for clinicians and auditors alike.

Local inference for sensitive workflows

Local inference for sensitive workflows runs an open-weight model on your own hardware or private tenancy for the steps that touch identifiable data: extraction from referral letters, coding support, triage summaries. Our healthcare case study is one example, a document intelligence pipeline deployed entirely on-premise that extracts structured data from medical documents, routes them to the right department and integrates with the existing hospital information system.

Hybrid architectures

Hybrid architectures route each task to the least exposed place that can do it well. A typical split: identifiable documents are processed locally, de-identified text goes to a stronger cloud model for complex reasoning, and non-patient content such as guidelines and policies uses cloud services freely. The routing rule must be explicit, tested and logged, because the boundary between the two sides is exactly where hybrid systems fail.

Audit logs and role-based access

Audit logs and role-based access tie the whole design together. Every request should record the user, their role, the data categories touched, the documents retrieved, the model and version, and the output, in an append-only store with its own retention policy. Roles should be inherited from the organisation's identity system rather than reinvented. For how long to keep what, see our guide to AI system document retention under GDPR and the EU AI Act.

Does on-premise deployment change your EU AI Act obligations?

On-premise deployment does not change your EU AI Act obligations, because the Act classifies AI systems by intended purpose, not by where they run. A diagnostic support tool that qualifies as a medical device under the MDR at class IIa or higher is high-risk whether it runs in a hospital basement or a hyperscaler region. After the Digital Omnibus amendments, obligations for AI in regulated products such as medical devices apply from 2 August 2028, and those for Annex III use cases such as emergency patient triage from 2 December 2027. What deployment does change is how easily you can meet the requirements: logging, human oversight and data governance are simpler to evidence when you control the whole stack. Where GDPR and the Act pull in different directions, for example minimisation against the Act's demand for representative data, our comparison of GDPR and the EU AI Act covers the tension, and what healthcare AI teams need to prepare covers the obligations. Our EU AI Act compliance work starts with classification.

How should you decide between cloud and on-premise AI for healthcare?

Deciding between cloud and on-premise AI for healthcare works best as a structured assessment of one concrete use case, not a policy debate about cloud in general. The same organisation may rightly run triage summaries locally and a guideline assistant in the cloud. An AI Readiness Audit answers the question in a fixed sequence:

1. Map the data flow. Which data categories the use case needs, where they live today, and every place they would travel: prompts, retrieval indexes, logs and evaluation sets.
2. Minimise before choosing. Remove every field the task does not need, and test whether de-identified input still performs acceptably. This step alone often moves a use case from on-premise to cloud.
3. Check the hard constraints. Hospital IT policy, contracts, ethics approvals and national rules that fix the answer regardless of preference.
4. Compare candidate models on your own data. Benchmark an open-weight model against a cloud model on a representative sample, using the discipline in our medical AI validation checklist. If the local model meets the threshold, the case for sending data out weakens sharply.
5. Assess operating capacity. Name who will patch, monitor and update a local system in year two. An on-premise deployment without an owner is a security risk, not a safeguard.
6. Write the decision down. Record the reasoning in the DPIA and the architecture record, so the choice can be defended and revisited when models or rules change.

The output is a recommended deployment model per use case, with the reasons and the residual risks named, ready for your DPO and security team to review. If the answer is a pilot, designing a 30 to 90 day proof of value covers how to test it against a threshold. Our AI integration service then builds the chosen architecture into your existing systems, and our healthcare AI practice describes the wider work with medical and digital health teams.

Frequently asked questions

Can you use cloud AI services with patient data under GDPR?

Yes, it can be lawful, but not by default. You need an Article 9 condition for processing health data, a processor agreement covering training and retention, an EU or otherwise lawful processing location, and usually a DPIA. Consumer chat tools rarely meet those conditions; enterprise services configured for them can. Your DPO should confirm the specific setup before any patient data is sent.

Is on-premise AI automatically GDPR compliant?

No. On-premise deployment removes third-party transfer questions, but every other GDPR obligation still applies: a lawful basis, data minimisation, access control, security under Article 32, retention limits and patients' rights. A poorly secured local server can be riskier than a well-run cloud tenancy. Location is one control among many, not a compliance status.

Does pseudonymisation make patient data safe to send to the cloud?

Pseudonymisation reduces risk but does not take data out of GDPR. Under Article 4(5), pseudonymised data remains personal data while the means to re-identify it exist, even if only you hold them. It is still a valuable safeguard that GDPR explicitly encourages, and it shrinks what a provider could expose. Treat it as a mitigation within a lawful cloud setup, not a way around one.

Can open-weight models running locally match cloud AI for clinical tasks?

For many narrow tasks, yes. Extraction, classification, coding support and summarisation of clinical documents are often handled well by open-weight models running locally, especially combined with retrieval. Frontier cloud models still lead on complex, open-ended reasoning. The only reliable way to know is to benchmark both on a representative sample of your own documents before deciding.