An AI agent that can remember its past failures and learn from them sounds obvious. But most of today's AI agents start every task from scratch, with no memory of what they tried before. A new training method called EMPO² changes that, and the results are significant: 128% better performance on science simulation tasks and 11% improvement on online shopping tasks compared to previous best methods.
Here are six papers from this week that have real implications for how AI tools will work in the near future.
Agents That Remember What Failed
EMPO² (Exploratory Memory-Augmented On- and Off-Policy Optimization) gives AI agents access to a memory of past attempts during training. The agent learns from both fresh experiences and older memories simultaneously, similar to how a person might keep a notebook of approaches that did and didn't work.
The practical result: agents trained this way can figure out new tasks in just a few tries using only their memory, with no retraining needed. For anyone building AI-powered workflows or using tools like ChatGPT's custom agents, this points toward a future where your AI assistant actually gets better at your specific tasks over time instead of resetting every conversation.
Meta Tested Its Chatbot on Millions of Real Users for 9 Months
Meta published data from 15 iterations of A/B testing its LLaMA 3.1-based chatbot across Instagram, WhatsApp, and Messenger. Over nine months, conversation depth increased 19% and instruction-following jumped from 59% to 85%. Seven out of eight production deployments beat the baseline.
The interesting part is the method. Meta built a preference system where users choose between two model outputs, then used those real engagement signals instead of synthetic benchmarks to train reward models. They describe the optimization process as "climbing a mountain in fog" - sample the terrain, estimate which way is up, take a careful step, then check. The takeaway for AI tool builders: real user feedback loops beat lab benchmarks, even when the feedback data is messy.
An 8-Billion Parameter Medical AI Outperformed Its 27-Billion Parameter Rival
MediX-R1 is a medical AI system trained on 51,000 examples to answer questions about X-rays and CT scans. Despite being an 8-billion parameter model (relatively small by current standards), it outperformed a competitor with 27 billion parameters that required significantly more training data.
The trick: a composite reward system combining four scoring signals during training, including a separate AI judge for medical correctness and domain-specific embeddings from PubMedBERT. The model writes its reasoning in visible "think" tags before responding, making its diagnostic logic transparent. Smaller, specialized models continuing to beat larger general-purpose ones is a pattern worth tracking.
Stopping AI From Agreeing on Wrong Answers
When an AI model generates multiple solutions to a math problem, the standard approach is to pick the most common answer. The problem: a confused model might confidently produce the same wrong answer eight times and the right answer twice. Majority wins, and the model learns to be even more wrong.
T³RL fixes this by converting reasoning into executable Python code and running it through a code interpreter to verify answers. Verified solutions get weighted 2-3x higher in the voting process. Result: 31% improvement on hard math problems, with bigger gains on harder tasks.
Two More Worth a Quick Look
MOSAIC adds explicit safety checkpoints before AI agents execute tool calls. Before any action that modifies state (writing files, sending messages, making purchases), the agent must assess harm potential, reversibility, and intent alignment. Testing across Qwen and Phi models (4B-7B parameters) showed harmful actions dropped by 50% and refusals of malicious prompt injection attacks increased by over 20%, without degrading performance on legitimate tasks.
Code2Math uses a multi-agent system to automatically generate harder versions of existing math problems. Agents write code to explore thousands of configurations, then create new problems based on what they discover. The code execution provides automatic verification that evolved problems are actually solvable. This matters less for daily AI tool users but could improve how future models are trained on mathematical reasoning.