llama.cpp - the open-source C++ engine that lets you run AI language models on your own hardware without a cloud connection - has added Multi-Token Prediction (MTP) support in beta, contributed by community developers.
Standard AI text generation produces one token at a time. A token is roughly three-quarters of a word, which means generating a 500-word response involves producing around 670 sequential steps. MTP changes this by having the model predict several upcoming tokens simultaneously, then confirming them in a single pass - think of it like a typist who plans several characters ahead rather than hunting one key at a time. Similar approaches have shown generation speed improvements of 30-50% depending on hardware and model.
MTP requires support at the model architecture level, so not every model benefits equally. DeepSeek models are the clearest current example - they were trained explicitly with multi-token prediction built in, which means users running local DeepSeek models stand to gain the most immediately.
The feature is in beta, which means rough edges are expected. Production deployments should wait for a stable release. For personal use and local experimentation, it's worth enabling and testing now.
For everyday users, llama.cpp sits underneath tools like Ollama and similar local inference wrappers - meaning speed improvements at the llama.cpp layer flow up to anything built on top of it. Even people who never touch llama.cpp directly may eventually notice faster responses in their local AI setups as this change matures.
Slower token output compared to cloud APIs has been one of the consistent complaints about running AI locally on consumer hardware. This addresses that gap directly without changing what models you can run.