The Local AI Glossary: Every Term, in Plain Language

Every term you'll meet on this site — or in a vendor pitch — translated. Terms are defined by what they mean for your office, not by their computer-science definition.

The Core Terms

LLM (large language model) — the kind of AI model behind chat tools. It reads text and produces text: drafts, summaries, answers. When this site says "model," this is what it means.

Local AI / on-premise ("on-prem") AI — running the model on hardware you own, inside your premises, instead of using a provider's cloud service. The subject of this entire site.

Private AI — usually a synonym for local AI. Caution: some vendors use it for cloud services with privacy features. The disambiguating question is always "where does the model run?"

Inference — the act of the model producing an answer. "Inference happens locally" = the thinking occurs on your machine.

Open model / open-weights model — a model released for anyone to download and run. This is what makes local AI possible: the model file lives on your machine like any other software.

Runtime — the software that loads a model and runs it (e.g., Ollama). Think of it as the engine mount between the model and your hardware.

RAG (retrieval-augmented generation) — the technique behind "chat with your own documents": the system first finds the relevant passages in your files, then has the model answer using them. In a local setup, both the search and the answering happen on your hardware.

Quantization — shrinking a model so it runs on smaller hardware, at a small quality cost. Why a useful model fits on an office workstation at all.

Token — the unit models read and write text in; roughly three-quarters of a word. Cloud AI bills by the token — which is why heavy use gets expensive, and why the cost worksheet asks about usage volume.

Where Your Data Lives

Data residency — which country's soil your data is stored and processed on, which determines whose laws and courts apply to it. Cloud AI can put your data in another jurisdiction; local AI makes the question trivial — it resides where your building is.

Data custody — who physically and legally controls the systems holding your data. The custody chain for local AI is: your staff → your network → your hardware. Full stop.

Processing — in privacy law, nearly anything done to personal information. When a cloud provider runs AI on your client data, the provider is processing it; that's what triggers most of the legal machinery. No provider, no third-party processing.

PII / personal information — information about an identifiable person. What privacy statutes protect.

Hardware Words You'll Meet

GPU (graphics card) — the chip that makes model inference fast. The main determinant of what size model you can run.

VRAM — the GPU's own memory; the practical ceiling on model size. Vendor spec sheets lead with it; the comparison page translates it into "what this actually runs."

Workstation-class machine — a strong desktop computer, the realistic starting hardware for a small office's local AI. Not a server room.