Your AI assistant remembers your name, your preferences, maybe even your dog's birthday. But how it stores and retrieves that information varies wildly depending on the system, and the design choices have real consequences for how useful (or creepy) the experience feels.
Developer Tom Bedor, who has spent three years building an open-source memory system called Elroy, published a detailed breakdown of the approaches different AI tools use to give agents persistent memory. The core framework is simple: every memory system needs to store information, retrieve it later, inject it into the conversation, and decide when to create new memories. The differences are in how each step works.
Graph Databases vs. Flat Files
The two dominant storage approaches split along a familiar line. Systems like Zep use graph databases, where memories are stored as connected nodes (think: "User works at Acme" connected to "Acme is a fintech company"). Zep claims this gives them strong performance on needle-in-a-haystack retrieval, where the system needs to find one specific fact buried in thousands of memories.
On the other side, tools like Letta (formerly MemGPT) and Claude Code use plain markdown files. The advantage is simplicity: you can open the file, read it, edit it, move it between systems. Claude Code stores memories as markdown with metadata headers, and uses background calls to a smaller model to decide what to retrieve.
Mem0 tries a middle path with graph integration but reportedly shows only about 2% performance improvement over simpler approaches. That is a thin margin for significant added complexity.
Long Context Windows Are Not Memory
One finding that matters for daily AI users: despite models now supporting 128,000 or even 1 million token context windows (a token is roughly 3/4 of a word), simply stuffing old conversations into the prompt is not a substitute for real memory. Research shows LLMs suffer roughly 30% performance drops when relevant information sits in the middle of a long document rather than at the beginning or end. A dedicated memory system that pulls out the right facts and places them prominently beats raw context every time.
The Hard Problems Nobody Has Solved
Bedor identifies six persistent challenges that anyone building or using AI memory should care about:
- Temporal confusion. LLMs are bad at reasoning about time. A memory from six months ago and one from yesterday look identical unless the system explicitly tags dates. His recommendation: always store absolute dates, never relative ones like "last week."
- Priority miscalibration. The model might remember that you ordered pizza on Tuesday and surface that fact in every conversation for weeks.
- Factual errors. If you tell your AI something incorrect, it remembers the mistake with the same confidence as a true fact.
- Privacy discomfort. Even people who share everything on social media report unease when an AI "remembers" personal details.
- Latency. Every memory retrieval adds processing time before you see a response.
- Transparency. Most systems inject memories invisibly, so you have no idea what the agent thinks it knows about you.
Bedor's own system, Elroy, addresses transparency by showing a panel of recalled memories alongside each response. He also uses what he calls "agenda items" instead of generic entity storage, focusing memories on actionable goals rather than trivia.
The practical takeaway: if you are evaluating AI tools that claim persistent memory, ask two questions. Can you see what the system remembers? And can you correct it when it is wrong? The architecture underneath matters less than whether you can actually trust and control the result.