Local AI vs Cloud AI: What Actually Happens to Your Data (and What Each Really Costs)

First, two quick disambiguations, because this phrase is genuinely confusing: "local AI" here means AI models running on your own computer — a desktop in your office instead of a provider's data centre. It has nothing to do with AI for local-business marketing, and it is not the same thing as "LocalAI," which is the name of one specific open-source product. Local AI is a way of running AI; ChatGPT, Copilot, and Gemini are the opposite way — cloud AI, where the model runs on the vendor's servers.

With that settled, the short answer: the real difference between local AI and cloud AI is custody of your text. With cloud AI, everything you type travels to a third party's servers, where it is processed and stored under that vendor's policies — which you'd need to read, and which change. With local AI, the text is processed on hardware you own and — when run offline — never leaves your building. Cloud wins on capability, convenience, and up-front cost; local wins on custody, and costs you hardware and upkeep instead of subscriptions. Everything below is the detail behind that sentence, with the vendors' and regulators' own documents linked at every claim.

Where Does Your Data Actually Go? The Custody Chain, Step by Step

Skip the abstractions and trace what physically happens to a paragraph of client text in each model.

Cloud AI (ChatGPT and similar):

1. You paste text into the chat window. It's encrypted and sent over the internet to the vendor's servers — OpenAI encrypts in transit with TLS 1.2+ and at rest with AES-256 (enterprise privacy page). Encryption protects the text from everyone except the vendor: it arrives at their servers and is processed there in readable form. That's how the model can answer you.
2. The conversation is stored on the vendor's systems. On ChatGPT's consumer plans, it may also be used to train future models — OpenAI's own documentation says ChatGPT "improves by further training on the conversations people have with it, unless you opt out" (data-usage page). The opt-out is the "Improve the model for everyone" toggle under Settings → Data Controls; Temporary Chats go further — deleted from OpenAI's systems after 30 days, not used for training, reviewed only for abuse monitoring.
3. Business tiers change the defaults, not the custody. OpenAI states that, by default, it does "not train our models on your data" for ChatGPT Business, Enterprise, or the API, retains API data up to 30 days (with zero-data-retention available for eligible endpoints), and can sign HIPAA Business Associate Agreements with qualifying customers (enterprise privacy page). Better terms — but your text still lives, for some period, on someone else's machines.
4. Because it lives there, legal process aimed at the vendor can reach it. During The New York Times' lawsuit, a court order required OpenAI to retain consumer ChatGPT content — including deleted chats. By OpenAI's own account, that obligation ended on September 26, 2025, though a limited set of April–September 2025 data remains under legal hold; Enterprise and zero-data-retention API customers were never affected. The order is over — the structural lesson isn't: a lawsuit you're not a party to can override the delete button on servers you don't own.

Local AI:

1. You paste text into a chat window that looks nearly identical. The model — downloaded once, run by free software such as Ollama or LM Studio — processes it on your own machine. LM Studio's homepage puts it in two words — "Natively local" — an app built to run open models on your own hardware. Ollama states it can "run entirely offline for mission critical work" — meaning you can unplug the network cable and it still answers.
2. That's the whole chain. Your staff → your machine. No vendor terms to read, no training default to toggle, no retention policy to monitor, no third party for anyone's court order to land on. (Legal process aimed at you still reaches your own hardware, as it always has — local AI changes who else holds your data, not your own obligations.)

One honest asterisk: this custody claim holds for genuinely local setups. Some local-AI tools also offer optional cloud features — Ollama, for instance, has a separate cloud offering with its own promises — and the moment you use those, you're back in the cloud column. The test is simple: if it works with the internet off, it's local.

Why Does Custody Matter More Than the Privacy Policy?

Because for offices with confidentiality obligations, regulators keep saying the duty attaches to the disclosure itself, not to what the vendor does afterwards.

The clearest named anchor is healthcare. HHS's Office for Civil Rights says that when a covered entity uses a cloud service to "create, receive, maintain, or transmit" electronic patient health information, "the CSP is a business associate under HIPAA" and a HIPAA-compliant business associate agreement is required — even if the cloud provider only stores encrypted data it cannot read (HHS OCR guidance on HIPAA and cloud computing). In plain English: the moment patient data touches a third party's servers, HIPAA treats that third party as part of your compliance problem, paperwork and all.

Law is converging on the same shape. The State Bar of California's practical guidance on generative AI — whose 2026 update addresses agentic AI at the request of the California Supreme Court — says a lawyer "must not input any confidential information of the client into a generative AI solution that may present material risks to confidentiality or security, absent informed client consent," and adds that "reasonable efforts require more than reliance on generalized marketing assurances." You have to actually read the terms, not the ad copy. And the point generalizes beyond any one country — Canada's federal privacy regulator puts it plainly: generative AI tools "do not occupy a space outside of current legislative frameworks" (OPC principles). Pasting personal information into one is a use of data your existing privacy obligations already govern, wherever your building is.

Local AI doesn't make those obligations disappear — it collapses the hardest part of them. There is no third-party processor to vet, contract with, or monitor, because there is no third party. What remains is the part your office already knows how to do: control who can use the machine, and be sensible with the outputs. The mental model is covered in What Is Local AI?

How Do Local AI and Cloud AI Actually Compare? The Honest Trade-Off Table

	Cloud AI (e.g. ChatGPT)	Local AI (your own hardware)
Where your text is processed	Vendor's servers, under vendor's terms	Your own machine, in your building
Training on your data	Consumer plans: yes by default, opt-out available; business plans: no by default (OpenAI data-usage page)	Nothing is sent anywhere when run offline
Retention	Vendor policy plus legal exceptions; API data up to 30 days (enterprise privacy)	Whatever you decide; it's your disk
Software cost	Free $0, Go $8/mo, Plus $20/mo, Pro from $100/mo, per user (OpenAI pricing); Business/Enterprise priced per user per month, quote-based	Typically $0 — Ollama and LM Studio are free; open models are free to download
Hardware cost	None beyond any computer with a browser	Up front: a capable small machine from $799 (base M4 Mac mini); from $1,599 for M4 Pro headroom; more for a shared office server
Ongoing costs	Subscription per seat, indefinitely	Electricity and eventual hardware replacement — small, and they vary by setup, so budget them as a modest estimated line rather than zero
Capability ceiling	Highest — frontier models (the largest, most capable models) are cloud-only	Lower — open models on office hardware; strong for drafting, summarizing, document Q&A; weaker on the hardest reasoning tasks
Maintenance	Vendor's problem	Your problem (updates, backups, access control)
Works offline	No	Yes
Compliance posture	A third-party processor to vet, contract with (a BAA or DPA — the signed contract governing how a vendor handles your data — where required), and monitor	No third-party processor; your existing internal policies govern

Two things this table refuses to claim: that local is always cheaper — for one or two users, subscriptions usually win on pure dollars, which is why the cost guide and worksheet exist — and that local matches frontier capability. It doesn't.

When Is Cloud AI Genuinely the Right Choice?

Often. If the work involves nothing confidential — marketing drafts, public research, rewriting your own website copy, brainstorming — cloud AI is more capable, cheaper to start, and maintained by someone else, and there's no custody problem because nothing sensitive is in custody. If you need the strongest available model for a hard problem, that model lives in a data centre, full stop. And if your office runs a business tier with a signed DPA or BAA and has actually read the terms, many obligations can be met that way — the standard cloud-plus-contracts route is legitimate.

The failure mode isn't "using cloud AI." It's unsorted use — confidential and harmless material flowing through the same consumer chat window with nobody deciding which is which. Practitioners admit this openly among themselves; as one commenter put it on r/Accounting, "I guarantee a lot of people put clients info into ChatGPT." Our ChatGPT safety guide covers the sorting exercise in detail.

What Would Going Local Actually Look Like?

Illustration — a fictional office, showing the pattern.

A three-lawyer firm sorts its AI use into two piles. Public-material work stays on a $20/month cloud plan. For the confidential pile — summarizing client files, drafting from privileged documents — they buy one Mac mini (theirs, configured with extra memory, lands at the $1,599 M4 Pro tier rather than the $799 base), install Ollama, and put the machine on the office network. Software cost: zero. Monthly fees for that machine: zero. The custody chain for privileged material is now staff → office network → office hardware. It's the setup one trial lawyer on r/LawFirm was pointing at when he wrote: "sharing that information with ChatGPT (OpenAI), is not a good idea. Using your own self-hosted language model would be better."

The honest close: that option costs real money up front, gives you a machine to maintain, and runs models a step behind the frontier. In exchange, the question "what is the vendor doing with our client data?" stops existing, because there is no vendor — wherever your building is. Whether the trade is worth it depends on how much of your daily work touches confidential material, which is exactly what the readiness quiz estimates in two minutes.