Two years ago, the answer to "how do I give an AI agent memory?" involved vector databases, embedding models, chunking strategies, and retrieval-augmented generation (RAG) - a technique where you search a database of your documents and feed relevant snippets to the AI alongside your question. Companies raised millions building this infrastructure.
Now? Some of the most effective agent memory systems run on markdown files and grep.
What Actually Happened
The AI memory problem seemed like it needed sophisticated engineering. You had thousands of documents, conversations, and notes. Surely you needed semantic search (finding documents by meaning rather than exact keywords) to surface the right context at the right time.
But in practice, several things changed. Context windows grew from 4,000 tokens to 200,000 or more - that is roughly from 10 pages to 500 pages of text an AI can process at once. Models got better at finding needles in haystacks of information. And developers discovered that a well-organized folder of markdown files, searched with basic Unix tools, often outperformed elaborate vector database setups.
Claude Code's memory system is a good example. It stores user preferences and project context in plain .md files with simple frontmatter metadata. No embeddings. No vector store. The AI reads relevant files, greps for keywords when needed, and maintains an index file. It works surprisingly well for day-to-day coding assistance.
The Tradeoffs Nobody Talks About
Flat-file memory has real limits. It works when your knowledge base is hundreds of files, not hundreds of thousands. It works when the AI can afford to read several files per query without blowing through its context window. And it works best when someone - human or AI - actively curates the files, removing outdated information and keeping the index clean.
RAG still wins for large-scale retrieval. If you need to search across 50,000 support tickets or a million lines of documentation, you are not grepping your way through that. The embedding approach also handles fuzzy semantic matching better - finding a document about "authentication failures" when the user asks about "login problems."
The real issue is that neither approach solves the hard part of memory: knowing what to remember, when to forget, and how to connect related information across time. A human assistant does not just recall facts - they build a mental model of your preferences, your project history, and the relationships between things you have told them weeks apart. Current AI memory, whether RAG or flat files, is closer to a filing cabinet than a brain.
What This Means for Your AI Workflow
If you are using AI tools daily, the practical takeaway is straightforward: do not over-engineer your setup. Before building a RAG pipeline, try the simple version first. Create a folder of markdown files organized by topic. Give your AI agent access to read them. See if that is good enough.
For most individual users and small teams, it probably is. The complexity of RAG pays off at organizational scale, not personal scale. And the models themselves keep getting better at working with raw text, which means the bar for "good enough" retrieval keeps dropping.
The memory problem will eventually get solved at the model level - future AI systems will likely maintain persistent memory natively rather than relying on external hacks. Until then, the boring solution is usually the right one.