Local LLMs for coding keep failing the same test: actual work.
After spending several weeks using local models for personal coding projects - specifically Qwen 2.5 27B and Gemma 4 31B, currently among the strongest models that run on consumer hardware - one developer found the productivity gap versus cloud models too wide to justify. The comparison benchmark was Claude Code, used daily for professional work. That's a deliberately demanding standard: Claude Code isn't a chat interface but an agentic coding tool that reads entire codebases, writes and edits files, runs tests, and iterates on problems autonomously.
Where Local Models Break Down
The failure modes were consistent across multiple setups and agentic frameworks:
Context length. Most local models running on consumer hardware cap at 8k to 32k tokens (a token is roughly three-quarters of a word - so 32k tokens covers around 75 pages of text). Modern codebases routinely need 100k+ tokens to reason across multiple files simultaneously. When a model can't hold the whole picture in memory, errors compound: fixing one function while breaking something it can no longer see.
Reasoning depth. Qwen 27B and Gemma 4 31B are genuinely capable models for their size. Isolated coding problems? They hold up. Multi-step debugging - trace the error, find the root cause three files away, fix it without breaking adjacent behavior, verify the fix - requires sustained logical chains that smaller models struggle to maintain across long sessions.
Agentic scaffolding. Running a local model alone isn't enough. You need a framework to handle file reads, shell commands, and error feedback loops. Multiple setups were tried. None matched the reliability of purpose-built cloud tools.
The Real Cost Calculation
The case for local models rests on three pillars: privacy, cost, and offline capability. Those advantages are real. But the math only holds if the productivity loss is acceptable.
Running Qwen 27B requires at least 16GB of VRAM (the video memory on a GPU) - a mid-to-high-end hardware setup most people don't have. Slower, less accurate responses compound across a workday. For hobby projects with no time pressure, the tradeoff might be fine. For any work where time has meaningful value, it usually isn't.
The honest framing isn't "local model for $0 versus cloud model for money." It's local-model-at-reduced-productivity versus cloud-model-at-full-productivity. Stated that way, the economics frequently favor cloud.
Local models do have a genuine place: short self-contained tasks, air-gapped environments where cloud APIs aren't an option, projects with hard privacy constraints. The failure point is sustained agentic work across large codebases. The models capable of competing at that level - 405B parameter models, for example - require infrastructure that isn't available to most individuals.
For practitioners without strong offline or privacy requirements, the field evidence keeps pointing the same direction: the productivity gap is real, and it's not closing fast enough to matter for day-to-day work.