Anyone who has run cargo test or npm run build inside Claude Code or Cursor knows the problem: thousands of lines of passing tests, repetitive warnings, and full stack traces get dumped into the conversation, burning through context tokens on information the AI doesn't need. A new open-source Rust CLI called tokf aims to fix that.
tokf sits between your build commands and your AI coding tool, intercepting output and compressing it using TOML-defined filter rules. You can strip passing test results, collapse repeated warning patterns, and trim stack traces down to just the relevant failure lines. The AI gets a clean summary of what actually broke instead of scrolling through hundreds of lines of green checkmarks to find the one red X.
The tool is configurable per project - you write filter rules in TOML that match your specific build output patterns. That means it works with any language or build system, not just Rust or Node. It's a simple Unix-style pipe: run your command through tokf, and the filtered output is what lands in your AI assistant's context window (the working memory where your conversation and code live).
This solves a real pain point. Context windows are finite, and wasting them on verbose build output means your AI assistant forgets earlier parts of your conversation sooner. Tools like Claude Code already try to be smart about what they include, but they're working with whatever the terminal gives them. Filtering at the source is a cleaner approach.
tokf is early-stage and available on GitHub under an open-source license. If you regularly feed build output to AI coding assistants and find yourself hitting context limits, it's worth a look.