Can you run AI on your own computer?

What you give away when you use a chatbot online

When you type into an online AI service, your words go to the company's computers. The company sets the price, the rules on what the AI will discuss, and how long it keeps the conversation. It can change any of those tomorrow.

What a company keeps is not always its own choice. In May 2025, in a copyright case brought by The New York Times, a United States court ordered OpenAI to keep ChatGPT conversations it would otherwise have deleted. The order was later narrowed, but logs saved under it stayed available to the case (OpenAI, 2025a; Bloomberg Law, 2025).

A model on your own computer has none of those problems. Nothing is sent, so nothing is stored elsewhere.

What "open weights" means

An AI model is a large file of numbers, called weights, learned from training. Most big services keep that file on their own servers. Some companies publish it so anyone can download and run it.

• DeepSeek-R1 came out in January 2025 under the MIT licence, one of the most permissive there is. The full model is huge, but DeepSeek also released six smaller versions, from 1.5 to 70 billion parameters (DeepSeek, 2025).
• gpt-oss, released by OpenAI in August 2025 under the Apache 2.0 licence, includes a 20-billion-parameter model built to run in 16 GB of memory (OpenAI, 2025b).

"Open weights" is not the same as open source. You get the finished model, not the training data or the recipe to rebuild it. The Open Source Initiative, which maintains the open source definition, draws this line clearly (Open Source Initiative, n.d.). For running a model privately, open weights is enough.

What you need

The limit is memory. A model's size in billions of parameters sets how much it needs. Compression called quantisation stores each weight in 2 to 8 bits instead of 16, cutting memory use several times over for a small loss in quality (Gerganov et al., n.d.).

Your computer	What it runs comfortably
8 GB of memory	Small models, around 1 to 4 billion parameters. Good for summarising, rewording and simple questions.
16 GB	Models around 7 to 20 billion parameters, including gpt-oss-20b. Good for most writing and explaining.
32 GB or more, or a graphics card with lots of memory	Larger models, closer to the paid services.

As a rough guide, a model needs about its parameter count multiplied by the bits per weight, divided by eight, in bytes: an 8-billion-parameter model at 4 bits takes about 4 GB, plus room to work. Newer Macs share memory between processor and graphics, which suits this work well.

How to start

1. Install Ollama, a free, open-source program for Mac, Windows and Linux that downloads and runs models for you (Ollama, n.d.).
2. Pick a model that fits your memory from the table above, and download it once.
3. Turn off Wi-Fi and ask it something. If it answers, it is running on your machine alone.

People who prefer the command line can use llama.cpp, the engine many of these tools build on (Gerganov et al., n.d.).

What you give up

• Quality. A model that fits in a laptop is smaller than the largest online services, and makes more mistakes on hard questions. Check facts it gives you, as you should with any AI.
• Speed. On older machines, answers arrive a few words at a time.
• Freshness. A downloaded model knows nothing after its training date and cannot search the web unless you add that.
• Upkeep. You choose updates. Nobody improves it for you.

Sharing one machine

The biggest open models need hardware few households own. The same models can run on one powerful computer shared by a school, library, club or small business, on its own network. The people using it then decide the rules, and the conversations stay in the building.

What to check

Before you paste something private into an online AI, read its privacy page for three things: whether it trains on your chats, how long it keeps them, and who can be made to hand them over. If the answers worry you, the same question can go to a model on your own desk.