Running a local LLM - a large language model like the ones powering ChatGPT, but running on your own hardware instead of a cloud server - means your data never leaves your machine. No subscription, no internet required, no third-party servers logging your prompts. The friction isn't the concept; it's picking the right model. Download one that needs 16GB of GPU memory (the dedicated video RAM on your graphics card) for a laptop that has 6GB and you get crashes, not results.
llmscan is a new open-source command-line tool that handles that compatibility check before you waste hours. It scans your machine - RAM, GPU VRAM, CPU specs, available disk space - and returns a list of local models your hardware can actually run. Llama 3, Mistral, Phi-3, Gemma, and similar models all have different memory requirements, and llmscan does the matching automatically.
Install via pip: pip install llmscan, then run llmscan in your terminal. You get back a recommended list ordered by what fits your setup.
This is a narrow tool with a clear purpose. It's not a model runner like Ollama or LM Studio - those handle downloading and running models. llmscan covers the step before that: figuring out which models are even worth attempting. For anyone setting up local AI for the first time, that's the specific question you need answered before committing to a large download.
The project is early-stage on GitHub, built by a single developer. Model coverage may lag as new options ship faster than the list updates. But as capable models keep arriving in smaller sizes - the 1-7B parameter range, where parameters roughly measure a model's scale and capability - consumer hardware compatibility becomes a real selection factor rather than an obvious one. A fast scan before you start is a useful starting point.