Running a local LLM that can search the web in real time - without sending your queries to OpenAI or Anthropic - just got more practical.
The setup combines llama.cpp (the popular open-source tool for running language models locally on your own hardware) with Brave Search through the Model Context Protocol (MCP). MCP is an open standard, originally developed by Anthropic, that lets AI models call external tools and data sources in a structured way. Think of it as a universal plugin system for AI assistants.
What This Actually Does
Normally, local LLMs are limited to whatever knowledge was baked into their training data. They can't look anything up. By connecting Brave Search via MCP, the model can run web searches during a conversation and incorporate fresh results into its responses - similar to how ChatGPT's browsing mode or Perplexity work, but running entirely on your machine.
Brave Search is a solid choice here because it offers a free API tier with reasonable limits, and Brave as a company has positioned itself as the privacy-focused alternative to Google.
The Privacy Angle
The appeal is straightforward: your prompts and conversations never leave your computer, while you still get access to current web information. For anyone working with sensitive internal data, proprietary code, or just philosophically opposed to cloud AI, this is a meaningful capability upgrade over a purely offline local model.
The tradeoff is that local models still lag behind GPT-4o and Claude in raw quality, especially for complex reasoning tasks. But for research, summarization, and quick factual lookups, a capable local model with web search access covers a surprising amount of daily use cases.