What happens when the CEO of a competing AI company admits under oath that his model was trained using a rival's technology?
That question is now in front of the courts. Elon Musk testified that xAI trained Grok - his company's AI assistant - on OpenAI models. The admission lands in an already contentious legal fight between Musk and OpenAI, and it puts a spotlight on one of the most contested practices in AI development: model distillation.
What Distillation Actually Means
Distillation, in AI terms, is when you use the outputs of an already-trained model to teach a new one. Instead of collecting expensive human-labeled data from scratch, you feed a powerful model like GPT-4 thousands of prompts and use its responses as training data for your own system. The result learns to mimic the reasoning patterns of the original at a fraction of the cost.
OpenAI's terms of service explicitly prohibit using ChatGPT outputs to train competing AI models - and the reason is straightforward. If distillation goes unchecked, labs with smaller research budgets can extract the value of a $100 million training run for a fraction of the cost. The labs funding frontier research end up subsidizing their competitors.
This isn't a new fight. DeepSeek's R1 model faced distillation accusations earlier this year based on patterns in its outputs that appeared to match OpenAI's models. OpenAI said it was investigating. Musk's testimony is different in kind - it's an admission rather than an inference.
What This Could Mean for the Industry
The distillation problem has no clean solution right now. Output watermarking - embedding invisible signals in a model's responses that persist even when those responses are used for training - is still experimental and hasn't been deployed at scale by any major lab. Terms of service enforcement requires proving that training data came from a specific model, which means expensive forensic analysis that may not hold up in court. Legal action is the only lever currently available.
If Musk's testimony holds up legally, it would be among the most significant admissions in the ongoing battle over AI intellectual property. It puts pressure on every lab building models to audit their training data sourcing more carefully - because "we didn't know" becomes harder to claim once courts start taking distillation seriously.
For users of AI tools broadly, the practical impact is indirect but real. The incentive to keep investing in frontier research depends on labs being able to protect what they build. A legal framework that treats distillation as IP theft would push more development into auditable pipelines - or at least slow the practice of building competitive models cheaply on top of proprietary work. The Musk-OpenAI case was already one of the messiest legal disputes in tech. This testimony makes it more consequential.