The "local is cheaper" argument for running AI models on your own hardware gets repeated constantly in the self-hosting community. The math doesn't actually support it.
Consider a fairly typical home server setup for running large language models: two used RTX 3090 GPUs (~$1,400), a Ryzen 7900X processor, 64GB of DDR5 RAM - total hardware spend around $2,800. Under load, that rig pulls roughly 700 watts. At average electricity rates, you're spending about $0.21 per hour just to keep the thing serving requests. Add GPU depreciation spread over three years, and the marginal cost per active compute hour climbs further.
Compare that to API pricing. Claude Haiku 3 runs at $0.25 per million input tokens. GPT-4o mini is $0.15 per million. For personal projects or moderate business use, the API math comes out ahead - often significantly.
What the Electricity Bill Actually Tells You
The $0.21/hour electricity figure sounds small, but it compounds. Eight hours a day of active inference (the process of actually generating responses from a model) is $1.68/day. That's $50/month before hardware depreciation. A $2,800 investment amortized over three years adds another ~$78/month. You're sitting at roughly $130/month for a setup that's competitive with mid-tier models - and APIs have gotten cheap enough that $130/month buys a substantial number of tokens.
A break-even case for self-hosting does exist, but it requires genuinely high request volume. If you're running thousands of requests per hour for a production application, per-token API costs can eventually exceed hardware costs. Most individual developers and small teams never get close to that threshold.
The Honest Case for Running Your Own Hardware
The defensible argument for self-hosting isn't cost - it's control. There are real reasons to run models locally:
- Privacy: Your data never leaves your machines. For legal, medical, or sensitive business work, that matters more than price.
- No rate limits: APIs throttle requests. A local server does not.
- Offline operation: No API dependency means no outages, no latency from network round-trips, no service terms changes.
- Fine-tuning access: You can retrain a model on your own data to specialize it for specific tasks - something API providers rarely expose.
- High-volume economics: If you're genuinely running heavy workloads continuously, the per-token math does eventually flip in your favor.
These are legitimate reasons. "It's cheaper" mostly isn't one of them - and treating it like it is does real harm to people making hardware purchasing decisions based on bad arithmetic.
The self-hosting community builds impressive things and has valid reasons to exist. But the talking points need to match the actual economics. Someone paying $0.15 per million tokens isn't getting ripped off. They're making a rational choice based on their usage patterns.
If you're considering local inference, go in clear-eyed: you're buying control and privacy, not a lower bill. For most workloads, you're paying a premium for that control. That's a completely valid tradeoff - just stop calling it a discount.