What Is Ollama — and How Do Local AI Models Actually Run on Your Own Computer?

If you've started reading about keeping AI work in-house, one name shows up again and again: Ollama. But nearly every page explaining it is written for software developers — GitHub pages, terminal commands, talk of "weights" and "quantization" — and none of them answer the question a clinic or firm owner is actually asking, which is simply: what is this thing, and does it really mean my client data stays in my building? This is that explanation, in plain language.

The short answer: Ollama is free software that runs an AI model on a computer you own, instead of on a company's servers. You install it once, download a model, and from then on your questions are answered by your own machine. When you run it with the internet off, your text never leaves the building — because there's no server for it to travel to. It's published openly on GitHub under the MIT license, which means anyone can inspect the code, and it installs on Mac, Windows, and Linux (ollama.com/download). Everything below is the detail behind that sentence.

What Exactly Is Ollama?

Think of it this way. A "cloud" AI like ChatGPT is a service you rent: you type into a website, your text travels to the vendor's data centre, their computer does the thinking, and the answer comes back. You never touch the actual model — it lives on their hardware, and so, for a while, does your text.

Ollama flips that around. It's a small, free program you install on your own computer whose entire job is to run AI models locally — the word "local" here just meaning "on this machine, in this room." The people who make it describe a model as able to "run entirely offline for mission critical work", and they state plainly that "your data is never trained on" and "your data stays yours." Those aren't marketing softeners; they're a direct consequence of where the computing happens. If the work is done on your desk, there's no third party in the loop to train on anything or keep anything.

Two things Ollama is not, because the naming trips people up. It is not an AI model itself — it's the software that runs models, the way a music player isn't the songs. And it is not a subscription: the software is free, it's open source, and the models it runs are free to download. There are no per-seat fees. (This is the same distinction we draw in What Is Local AI? — local AI is a way of running AI, and Ollama is one of the most common tools for doing it.)

How Does a Local AI Model Actually Run on Your Own Computer?

Here's the part that sounds like magic until you see the mechanism, at which point it becomes almost boring — in a good way.

An AI model is, underneath, an enormous file full of numbers. Those numbers are called the model's weights, and they're the result of the model's original training. When you run a command like Ollama's own example, ollama run gemma4 (from its GitHub page), the software downloads that file to your hard drive — once. That download is the only step that needs the internet.

After that, answering your question is just arithmetic performed on your own processor, using the file already sitting on your disk. Nothing is fetched from a server to produce each reply. That's the whole reason a local model works with the network unplugged — it's the mechanism behind the "runs entirely offline" claim. You could disconnect the internet, ask it to summarize a client file, and it would answer, because everything it needs is already on the machine in front of you. (We go deeper on this in Can AI Work Without Internet?)

The trade-off lives in that same fact. Because the thinking happens on your hardware, the size and speed of the model depend on how capable your computer is. The largest, most powerful models still live in vendors' data centres — you won't run those at home. But the open models Ollama can run (its GitHub page names families like Gemma and Llama) are genuinely useful for the everyday office jobs: drafting, summarizing, rewriting, and answering questions about documents you provide.

Is Ollama Safe?

This is one of the most-searched questions about Ollama, and it has two honest halves.

The first half — is the software itself trustworthy? — is easier to answer than for most tools, because Ollama is open source under the MIT license. "Open source" means the actual code is published for anyone to read, so it isn't a sealed black box you have to take on faith; security-minded people can and do inspect it. And its makers' own data promises — "your data is never trained on," "your data stays yours" — are backed by the structure, not just the wording: a model running on your machine has no channel to send your text anywhere.

The second half — is your setup safe? — is on you, and this is the honest asterisk. Running a model locally removes the vendor from the picture, but it doesn't remove the ordinary responsibilities you already have with any office computer: controlling who can log in and use the machine, keeping backups, and being sensible about what you do with the outputs. And there's one specific thing to watch: some local-AI tools, Ollama included, also offer optional cloud features. The moment you switch one of those on, your text is going off your machine again, and you're back to reading someone's terms. The test is simple and physical — if it still works with the internet off, your data stayed home.

Ollama vs LM Studio: Which Local AI Tool Should a Non-Technical Office Choose?

Ollama isn't the only on-ramp. The other name you'll see most is LM Studio, which describes itself as "natively local" — an all-in-one desktop app. The practical difference, for someone who doesn't want to touch a command line, is mostly about what the screen looks like.

	Ollama	LM Studio
What it is	Free, open-source software for running open models locally (GitHub, MIT license)	Free all-in-one desktop app, "natively local"
Runs on	macOS, Windows, Linux (download page)	macOS, Windows, Linux
Feel	Often driven by a short typed command (e.g. ollama run gemma4)	A polished window with buttons and menus
Cost	Free; open models are free to download	Free; open models are free to download
Where your text goes	Your own machine; nothing sent when run offline	Your own machine; nothing sent when run offline
Best for	Owners fine with one simple command, or an IT helper setting it up	Owners who want to click, not type

Both are free, both keep the processing on hardware you own, and both run the same families of open models. So the choice isn't really about privacy — on that they're equal — it's about the interface you'd rather look at every day. For a fuller field of options, see Private AI Options Compared.

What Does It Cost to Run AI This Way?

The software side is genuinely free — Ollama, LM Studio, and the open models themselves cost nothing. What you pay for is the computer capable enough to do the thinking. A capable small machine starts around $799 for a base M4 Mac mini, with more headroom (useful for larger models or a machine several people share) from $1,599 for the M4 Pro. That's an up-front purchase rather than a monthly subscription, which flips the usual cloud math: no per-seat fees, but you own the hardware and its upkeep. Whether that trade favours you depends on head-count and how many years you run the machine — the honest arithmetic is laid out in How Much Does Local AI Cost?

Does Running Locally Make My Office Automatically Compliant?

No — and it's important to be precise here, because "private" and "compliant" are not the same word.

Running a model on your own hardware removes the hardest compliance problem, which is the third-party processor: there's no vendor receiving your data, so there are no vendor terms, training defaults, or retention policies to vet, contract around, or monitor. That genuinely erases a large chunk of the work.

But your office's own obligations don't vanish just because the vendor did. Canada's federal privacy regulator makes the general shape of this clear: generative AI tools "do not occupy a space outside of current legislative frameworks," and organizations using them "are obligated to ensure that their activities comply with applicable privacy laws." Running locally removes the outside party — it doesn't remove your duty to control access, protect the machine, and handle personal information responsibly. The principle generalizes well beyond any one country: the rules that governed your filing cabinet still govern the computer that replaced it, wherever your building is.

The Honest Bottom Line

Ollama is the plainest possible answer to "how do I keep AI work in the building": free, open-source software that runs a model on a computer you own, so that — with the internet off — your text has nowhere to travel. It's not magic and it's not free of trade-offs. You buy the hardware up front, you maintain the machine yourself, and the open models you'll run are a step behind the very largest cloud models on the hardest problems.

In exchange, the question that keeps confidentiality-minded owners up at night — what is the vendor doing with our client data? — simply stops applying, because there is no vendor. For plenty of offices the right answer is still a well-configured cloud plan with the terms actually read; for the confidential pile, keeping the sensitive work on hardware you own is a real, honest option, and Ollama is one of the most common front doors to it.