Run the model at home: a hands-on guide to local models with Ollama and LM Studio
Cloud models charge by use and take your data out the door; local models cost nothing per call once downloaded. This guide covers both roads — Ollama and LM Studio: installing, picking a model, wiring it into your coding tools, and which jobs go local and which stay in the cloud.

Let me draw the boundary first. This site already has two news items about local models: one about Copilot CLI gaining local-model support, about one feature in one command-line tool; another about Microsoft squeezing a 137B in-house coding model onto your computer, about one specific model. Those are news — they tell you what happened. This is a guide, about what you do yourself: run a model on your own machine, then wire it into the way you write code. There are also two guides about cloud models, on model routing and on token costs — how to pick between cloud models and how to count the cloud bill. This one is the other road: the model does not run in someone else's data center, it runs on your own machine. And the self-hosting guide is about deploying your product to your own server; this one is about running the model on the machine you code on. Different place, different purpose.
The "Skylark Recipes" project used below is a fictional practice project: a personal recipe app one person built with AI help, which stores recipes and suggests dishes from whatever is left in the fridge. Every operation, conversation, and error message shown with it is invented to explain how things work; it is not a real record. Every command, port, model name, and size for Ollama and LM Studio comes from the two products' official documentation and official model library pages, listed at the end.
First, ask what a local model actually saves you
Do not install anything yet. Ask yourself three questions; if even one answer is yes, a local model is worth an afternoon.
First: does your code or your prompts contain anything you would rather not send out? A client's source code, a product idea you have not shipped yet, debugging notes with real user data in them. A cloud model means sending your stuff to someone else's data center to be computed; however friendly the terms of service, the data has left your computer. A local model keeps the data at home: Ollama's own wording is that when a model runs locally, it does not see your prompts and does not take them. That does not mean local is automatically safe — there are three red lines later in this guide — but "the data never leaves this machine" is something local gives you by construction, while cloud gives it to you by contract. Those are not the same thing.
Second: do you often write code with no network or a weak one? On a high-speed train, on a plane, in a cafe where the signal is down to one bar, a cloud model stops working the moment the connection drops. A local model works as long as the computer is on.
Third: is your monthly cloud-model bill already big enough to hurt? A local model's marginal cost is zero — once you have downloaded it, asking it to change code a thousand times adds nothing to any bill. The cost sits on the other side: electricity, disk space, and the extra money you spent on the computer. Whether it is worth it depends on your own cloud bill; there is a section at the end that works through that math.
And there are three situations where I would not bother with a local model. First, your machine is too small — an old laptop with 8 GB of memory will struggle even with a 7B model, and the experience will be poor. Second, everything you do right now is "think it through in one go" work — a new feature across a dozen files, a hard algorithm, a critical code review — where a small local model is not yet good enough. Third, you are a team of more than one person who needs to share the same prompts and the same review records; a local model is single-machine by nature, and collaboration is smoother in the cloud. In those three cases, keep using the cloud; the local model can wait.
Two roads: Ollama or LM Studio
There are two mainstream tools for running models locally. Ollama is a command-line tool: one binary plus a background service. LM Studio is a desktop app with windows, buttons, and a chat interface. Both can download models, run them, and serve them over an OpenAI-compatible local API; the difference is how you operate them and who they suit.
Pick by who you are. If you live in the terminal and want to type commands once installed, pick Ollama: one command pulls a model, one command starts a chat, and everything else goes over an HTTP API that scripts nicely. If the terminal makes you sigh, or you want to click around first — see what a model looks like, try different quantization versions — pick LM Studio: its built-in model browser connects straight to Hugging Face, so you search a name, download, click Load, and chat, without memorizing commands.
Before either road, check whether your machine is up to it. Ollama's official README gives a blunt rule of thumb: at least 8 GB of memory for a 7B model, 16 GB for 13B, 32 GB for 33B. That line is a floor, not a recommendation — sit exactly on it and a long context will fill your memory and slow everything down. A practical way to judge: take your memory, subtract what the system and browser use day to day, then take 70 percent of what is left; that is what you can give a model. A laptop with 16 GB of memory can comfortably run a 7B to 8B model; for 13B and up, make sure you have 32 GB of memory or a decent graphics card. Disk space matters too: an 8B model is about 4.7 GB, a 70B model about 40 GB, so check your free space before you download.
Ollama: from install to first conversation
The Ollama road takes under half an hour. Step one is installing. On a Mac, download the installer from the official site; on Windows, download the setup program; on Linux, run one official script in the terminal: curl -fsSL https://ollama.com/install.sh | sh. Once installed, the service starts itself in the background; there is nothing to configure. If you use Docker, you can also pull the official ollama/ollama image, but installing directly on the host is simplest the first time, and easier to debug when something goes wrong.
Step two is pulling a model. Browse the official model library at ollama.com/library and do not be greedy at first. For your first try, pull the 3B version of llama3.2 with a single line: ollama pull llama3.2. That download is about 2.0 GB; the 1B version is smaller, about 1.3 GB. Once pulled, start chatting with ollama run llama3.2 — if the model is not on your machine yet, the run command pulls it first, so you can skip the pull step and go straight to run. For a model built for coding, try qwen2.5-coder, which the library carries from 0.5B to 32B; start with the 7B one: ollama pull qwen2.5-coder:7b.
Step three is confirming the service is running. Ollama listens by default only on your own machine, at 127.0.0.1 port 11434. Visit http://localhost:11434/api/tags in a browser or terminal; if you see the list of models you have downloaded, the service is healthy. Remember this address — your scripts and your coding tools will talk to the local model through this port.
When choosing a model, this short list from the official library and the official README helps. For everyday chat and light code changes, start small: llama3.2 3B at 2.0 GB runs on almost any recent computer; gemma3 4B at 3.3 GB and 12B at 8.1 GB are good picks if you want the strongest small model; for coding, qwen2.5-coder from 0.5B to 32B lets you pick by size, and 7B is the balanced choice. For a reasoning model, deepseek-r1 ranges from 1.5B to 671B, but the full 671B version needs 404 GB — not for a normal machine; the 1.5B and 7B versions are the ones that run locally. For work that genuinely needs a big model, such as the 70B llama3.1 (40 GB) or llama3.3 (43 GB), confirm your memory and disk first. One rule: get it running first, then worry about running it well. For your first try, pick one size smaller rather than one size bigger.
Day-to-day management is five commands: ollama list shows the models you have downloaded and how much space each takes; ollama ps shows which model is currently loaded in memory; ollama stop stops one; ollama rm deletes a model you no longer use and frees space; ollama show prints a model's parameter count, quantization, and default context length. When a model sits idle, Ollama unloads it from memory after a while on its own; the OLLAMA_KEEP_ALIVE environment variable controls how long it stays. When the disk fills up, delete models first: they live in Ollama's own directory, and the rm command is the clean way to remove them.
LM Studio: the graphical road
If you picked LM Studio, the flow looks different but does the same things. Download the installer for your operating system from the official site; macOS, Windows, and Linux are all covered. Open it, go to the Discover tab, and the built-in model browser searches Hugging Face directly: type a name such as qwen3 and you will see the model in several quantization versions. Quantization is how much the model has been compressed; common labels are Q3_K_S, Q4_K_M, Q8_0, and bigger numbers mean closer to the original precision and bigger files. LM Studio's official docs give blunt advice: if your machine can run it, pick a 4-bit option (Q4_K_M) or higher. Do not agonize the first time — Q4_K_M is fine.
Once downloaded, click Load, the model reads into memory, and when the top shows Ready you can chat in the chat window. To switch models, unload the current one first, then load another. On Apple Silicon Macs, LM Studio supports Apple's MLX format as well as GGUF; if a model comes in both, the MLX one is usually faster on an Apple Silicon machine. On Windows and Linux, pick GGUF. One detail people miss: downloading a model is not the same as loading it. Downloading stores it on disk; loading reads it into memory; you cannot chat with a model that is only downloaded.
The killer feature is in the Developer tab: click Start Server and the loaded model becomes a local service, by default at http://localhost:1234/v1. The official docs list the endpoints, and they match OpenAI's: GET /v1/models lists models, POST /v1/chat/completions chats, POST /v1/embeddings embeds, POST /v1/completions completes, POST /v1/responses handles agent-style calls. In other words, any code that used to call OpenAI keeps working if you change only the base URL to http://localhost:1234/v1; the request format stays the same. LM Studio also ships an lms command-line tool, so you can manage models without opening the window: lms server start starts the server, lms ls lists downloaded models, lms ps shows what is loaded, lms load and lms unload manage memory, lms get downloads by name. The window and the command line are the same thing underneath; use whichever you like.
Wiring the local model into your coding tools
Running a model is only the first half. The second half is getting it to actually help you write code. There are three layers, shallow to deep.
The first layer is calling the API directly from a script. Ollama's native endpoint is http://localhost:11434/api/chat — send a model name and a list of messages and you get an answer. Easier still is its OpenAI-compatible endpoint at http://localhost:11434/v1/chat/completions, where code that used to call OpenAI works after changing only the base URL. LM Studio is the same at http://localhost:1234/v1. Take Skylark Recipes as an example: its owner wrote a small script that every night asks the local model to read the day's code changes and write a Chinese summary into a draft CHANGELOG in the project. That job does not need the smartest model — a local 7B handles it — and the summary can mention features that have not shipped yet, so routing it through the local API keeps the data at home.
The second layer is plugging into a coding tool you already use. LM Studio's official docs state plainly that Codex works with its service, because LM Studio implements the OpenAI-compatible POST /v1/responses endpoint, and the docs carry a dedicated "Use Codex with LM Studio" guide. For Claude Code, LM Studio offers an Anthropic-compatible /v1/messages endpoint, and the official docs list integrations with Claude Code and Codex. The wiring is the same in both cases: point the tool's request address at the local service, fill in the model identifier exactly as LM Studio shows it (note that LM Studio's model names look like long Hugging Face repository paths, not Ollama's short names), and put any string in as the API key placeholder — the local service does not check keys by default, so anything works. The first time you connect, test the endpoint with curl or a minimal Python script to confirm the local service is actually responding, and only then change the coding tool's configuration.
The third layer is splitting work between the local model and the cloud model instead of choosing one. Give your tool two models: small everyday jobs go local, big important jobs go to the cloud. Some tools let you write two model addresses in the configuration; others need you to switch by hand. Either way, the cheaper switching is, the better — if every switch means editing five places, you will eventually stop switching and use only one. Once set up, the workflow becomes: the local model edits first and runs the tests locally, and once the tests pass, the cloud model reviews the change.
The split: what goes local, what stays in the cloud
A local model's biggest weakness is a low capability ceiling — a small model writing a big job will get things wrong or miss things. So split by the size and risk of the job, not by mood. This table is the one set for a personal project like Skylark Recipes; adapt it to yours.
Five kinds of work go local. First, writing tests: adding unit tests for a function you just wrote follows a fairly fixed pattern, small models do it decently, and if they get it wrong the tests themselves fail, so the risk is contained. Second, copy and documentation: translating an English error message into Chinese, adding a Chinese paragraph to the README, writing a summary for a commit — low reasoning demand, and mistakes are easy to spot. Third, small refactors: a change under two or three hundred lines in a single file, such as extracting repeated code into a function. Fourth, explaining code: asking the model to read a piece of code you did not write and explain in plain language what it does. Fifth, offline work: with no network, everything goes local; there is no choice to make.
Five kinds of work stay in the cloud. First, designing a new feature: a job that touches a dozen files and needs the architecture thought through first — a small model loses track. Second, hard reasoning: algorithms, performance problems, bugs that span modules, where the model has to think many steps ahead at once. Third, critical review: the code review and security check before a release, where review quality decides whether production breaks — do not save money here. Fourth, long context: jobs that need tens of thousands of lines read before you can start, where a small model's context window and attention cannot keep up. Fifth, work that needs the latest knowledge: a just-released framework or interface that is not in a small model's training data — what it answers may be invented.
Once the split is set, add one handover rule: when can something the local model changed go straight in, and when must it pass a cloud review? The rule for Skylark Recipes is: changes to tests, documentation, and copy can be submitted after the local model runs the tests; changes to business code get a cloud-model look at the change summary before submitting; changes to authentication, payments, or database migrations must be read by a human in the diff before submitting. Write the rule into the project's AGENTS.md so the agent reads it before every job.
One honest note: read what a local model writes with an extra careful eye. Not because it is always wrong, but because it fails differently from a big cloud model — a big model fails by thinking wrong, a small model fails by not thinking far enough. A piece of code from a local model can look smooth and pass its tests while missing an edge case or using a deprecated interface. Make it a habit: before merging anything a local model produced, read the diff once, run the tests once, then submit.
Three red lines for privacy
A local model's biggest selling point is privacy, but privacy is not automatic once you install it. There are three red lines to guard yourself.
First: do not expose the local service beyond your own machine. Ollama listens by default only on 127.0.0.1, meaning it accepts requests from your machine only; nobody else can reach your port 11434, and that is a good thing. Do not get tempted to set OLLAMA_HOST=0.0.0.0 and open it up — once open, anyone on the local network can call your API without authentication, list your models, pull new ones, delete yours, and run inference on your graphics card and your electricity bill. LM Studio is the same: its local service has no password by default, and anyone who can reach your port 1234 can use it directly. If you genuinely need to share it on the local network, LM Studio's Server Settings let you turn on "Require Authentication" and create a token, so only requests carrying the token are accepted; Ollama has no built-in authentication, so the correct approach is to forward the port through an SSH tunnel rather than exposing it. Remember: a local service with no authentication belongs on 127.0.0.1 only.
Second: tell local models apart from cloud models. Some models in Ollama's library carry a :cloud tag in their names; those do not run on your computer, they run on Ollama's servers, and your prompts go out. Check the name before you use one, and do not feed private material to a model you thought was local but is actually in the cloud. LM Studio does not have this issue — it only runs models you downloaded locally — but if you connect any other cloud service inside LM Studio, keep clear about which line is local and which is cloud.
Third: local does not mean you can feed it anything. Some things should not be fed to a model even if they never leave the machine: real user passwords, keys, identity numbers, a full backup of the production database. A local model will not send these out, but they stay in your conversation history, logs, and model cache, and they leave with the computer if it is lost or lent out. The correct habit is to strip sensitive data before it reaches the model: replace keys with placeholders, replace real data with invented fake data. Scrubbing is the same whether the model is local or cloud — local removes one leak, not every leak.
Five common failures
The first failure: the model will not download. A progress bar stuck in place means first checking the network, then the disk space. An 8B model is 4.7 GB and a 70B model is 40 GB, and running out of disk halfway through a download is the most common way to die. Fix it by deleting models you no longer use — ollama rm does it, and LM Studio has a delete button in the model list. Behind a corporate proxy, set HTTPS_PROXY before downloading, or the download will fail silently.
The second failure: the model runs too slowly to use. Check memory first: if the system monitor shows memory nearly full, the model is too big, so switch to a smaller one. Switching to a smaller model is the most effective speedup there is, better than tuning any setting. Also check whether two models are running at once: Ollama unloads idle models on its own, but if you have two services open, or two tools each holding a model, memory gets tight.
The third failure: the coding tool cannot reach the local service. Test the API directly with curl first: if curl http://localhost:11434/api/tags returns your model list, Ollama itself is fine and the problem is in the tool's configuration; if it returns nothing, the service is not up, so restart Ollama. For LM Studio, first confirm the server is on and the model is loaded (status Ready), then confirm the model name in the tool matches the identifier LM Studio shows character for character — case and slashes included; one wrong character reports the model as missing.
The fourth failure: the model answers beside the point, and the code it writes does not match the job. Before blaming the model, look at your prompt: is it too vague? "Help me fix this function" is a prompt a big cloud model can fill in by guessing your intent; a small local model guesses badly. Fix it by writing the prompt concretely: which file, what the result should look like, which parts must not move, and what counts as passing the test, one item at a time. A small model is not unable to work — it needs the work broken into smaller pieces and spelled out more clearly.
The fifth failure: the context gets long and the model starts making things up. A small model's context window is limited; paste in a few thousand lines of code and it starts forgetting the beginning and inventing answers. Fix it by pasting less: only the one file or function the model needs, not the whole project. The approach that works for Skylark Recipes is to first let the model read the README and the directory structure, decide which file to change, and then send only that file's contents — step by step beats dumping everything at once.
A one-page checklist, and the cost math
Here is the whole guide as a checklist. Before installing: confirm the memory is enough (7B needs at least 8 GB, 13B at least 16 GB, 33B at least 32 GB), confirm the disk is enough (an 8B model is 4.7 GB, a 70B model is 40 GB), and confirm you are solving at least one of the three problems: privacy, offline use, or cost. While installing: pull a small model first (llama3.2 3B or qwen2.5-coder 7B) and get the first conversation working; do not start with a big one. While connecting tools: test the local API with curl first (Ollama: http://localhost:11434/api/tags; LM Studio: http://localhost:1234/v1/models), then change the coding tool's configuration; check the model name character by character. While splitting work: tests, documentation, copy, small refactors, and explaining code go local; new feature design, hard reasoning, critical review, long context, and latest knowledge stay in the cloud; changes to authentication, payments, or database migrations always get a human read of the diff. While guarding privacy: bind the service to 127.0.0.1 only and do not expose the port; tell :cloud-tagged cloud models apart from local ones; strip sensitive data before it reaches the model.
Now the cost math. A cloud bill is metered: you pay for how many calls you make and how many words come out, and the bill says so plainly. A local model's cost is paid up front: the extra money spent on the computer (more memory, a better graphics card), electricity, and disk. Compare against your last three months of cloud bills: if a large chunk of each month went to small jobs like tests, documentation, and copy, moving that chunk local may save enough in a year to pay for a memory upgrade; if your cloud bill was already small, or most of it went to critical reviews and new feature design that a local model cannot do, a local model will not save you much money, and the reason to install it should be privacy and offline use, not saving money.
Sources: the official Ollama README and the official model library at ollama.com/library (read 2026-11-17); the official LM Studio documentation at lmstudio.ai/docs (read 2026-11-17).
Comments (0)
Sources
Related articles
On 17 November 2026, Microsoft launched MAI Code 2 and Code 2 Flash: the larger model has 77 billion active parameters and a one-million-token context, averaging 63.2% across four benchmarks — close to GPT-6.1 Sol and Claude Opus 5.5 — at $0.309 per task. Read the three caveats first: promotional pricing, the lowest score of the three, and no long-horizon work.
On 17 November 2026, Perplexity launched the Orchestrator API: one interface reaching OpenAI, Anthropic, Google, xAI and open-weight models such as GLM, Kimi and DeepSeek, billed at the answering model's published rate with no platform fee. When one provider fails, the same model retries on another deployment, and preview routing assigns requests to models by task.
You already call functions every day; this guide names what you have been doing. Following MDN's Functions, Arrow function expressions, and Rest parameters: declarations versus expressions, parameters and arguments, default values, rest parameters, spread syntax, return values, arrow function shorthand, the this trap, callbacks, and scope.