Why Information Theory Says AI Has a Ceiling We Cannot Engineer Around
What Happened
A new article by Vishal Misra, published on Medium and surfaced on Hacker News on March 7, 2026, makes the case that the theoretical foundation powering modern AI - Shannon's information theory - has taken the field as far as it can go. The next set of hard problems, Misra argues, falls under Kolmogorov complexity, a framework that exposes fundamental limits on what statistical learning can achieve.
The core distinction is this: Shannon's information theory deals with the statistical properties of data. It tells us how to compress, transmit, and predict patterns based on probability distributions. This is exactly what large language models do. They learn statistical regularities in text and use those patterns to generate plausible outputs. Every improvement in LLMs over the past decade - better architectures, more data, longer contexts - has been an engineering refinement within Shannon's framework.
Kolmogorov complexity takes a different angle. Instead of asking "what are the statistical patterns in this data?", it asks "what is the shortest program that produces this output?" This is a measure of true algorithmic complexity, and critically, it is uncomputable. There is no general algorithm that can determine the Kolmogorov complexity of an arbitrary string. This uncomputability is not an engineering limitation. It is a mathematical proof.
Why It Matters
If you use AI tools daily, this framing helps explain a pattern you have probably noticed: LLMs are remarkably good at tasks that involve pattern matching, rephrasing, summarization, and generating text that follows established conventions. They struggle with tasks that require genuine novel reasoning, where the solution cannot be derived from statistical regularities in training data.
This is not a model size problem. Scaling up parameters and data improves performance on Shannon-type tasks - better predictions, smoother outputs, fewer factual errors. But it does not bridge the gap to Kolmogorov-type tasks, where the challenge is finding a compact, novel algorithm rather than interpolating between known patterns.
For anyone building workflows around AI tools, this distinction is practical. Tasks like drafting emails, summarizing documents, and generating boilerplate code sit firmly in Shannon territory. Tasks like solving novel engineering problems, producing genuinely original analysis, or reasoning through unfamiliar logic puzzles push into Kolmogorov territory, where current models will continue to fall short regardless of scale.
Our Take
This is not a new argument in computer science, but it is a useful one for people trying to calibrate their expectations of AI tools. The hype cycle tends to treat every capability jump as evidence that artificial general intelligence is around the corner. The Kolmogorov framing offers a clear, mathematically grounded reason to expect otherwise.
What matters practically: stop waiting for AI tools to replace judgment on novel problems. Instead, get better at using them for what they are provably good at - statistical pattern work. Structure your prompts to keep tasks within the pattern-matching domain. Use AI for first drafts, data transformation, and synthesis of existing knowledge. Keep the genuinely novel thinking for yourself.
The article also raises an uncomfortable question for AI companies selling "reasoning" capabilities. Chain-of-thought prompting and reasoning tokens improve outputs, but they are still operating within Shannon's framework - breaking problems into sub-patterns the model has seen before. When a problem truly has no analogous pattern in training data, no amount of chain-of-thought helps.
This does not make current AI tools less useful. It makes them more predictable. And predictability, for anyone building real workflows, is more valuable than hype.