What Happened
A new open-source project called CloakPipe appeared on Hacker News on March 6, 2026. It's a Rust proxy that sits between your application and any OpenAI-compatible API, detecting sensitive entities in requests and replacing them with consistent pseudonyms before the request reaches the LLM provider.
The key word is "consistent" - the same input always maps to the same token. "Tata Motors" always becomes "ORG_7" across every request. This means embeddings and vector search still work because the semantic structure is preserved, but the actual entity values never leave your infrastructure.
CloakPipe detects API keys, JWTs, email addresses, IP addresses, financial amounts, and fiscal dates. It encrypts its mapping vault with AES-256-GCM and claims under 5 milliseconds of added latency per request. The proxy handles streaming responses too, rehydrating pseudonyms across SSE chunks. Custom detection rules are configurable via TOML files.
The project is available on GitHub at github.com/rohansx/cloakpipe.
Why It Matters
Every time you send a prompt to an LLM API, that request passes through multiple systems: embedding APIs, cloud vector databases, the LLM provider itself. Each hop is a potential data leak. This is the core problem that's keeping many enterprises from fully adopting LLM-powered workflows, and it's a legitimate concern for individual developers handling client data too.
Existing approaches to this problem tend to break things. Stripping entities entirely destroys context the model needs. Encryption makes the text meaningless to the LLM. Full on-premise deployment is expensive and limits model choice. CloakPipe's pseudonymization approach tries to thread the needle - the model sees structurally valid text with fake entities, produces a response using those fake entities, and the proxy swaps the real values back in.
For anyone building LLM-powered tools that process customer data, financial records, or internal communications, this addresses a real compliance pain point.
Our Take
The approach is clever and the engineering choices are solid. Rust for a proxy that needs to add minimal latency is the right call. Deterministic pseudonymization that preserves semantic structure is a better solution than the "just anonymize everything" approaches we've seen before.
But there are important caveats. Entity detection is only as good as its rules. If CloakPipe misses a sensitive value - a customer name in an unusual format, a proprietary term the TOML rules don't cover - that data goes straight to the API. You can't treat this as a compliance guarantee without thorough testing against your actual data patterns.
The 5ms overhead claim is impressive if it holds under load with complex detection rules. For most applications, that's negligible.
The bigger question is whether pseudonymization is sufficient for your compliance requirements. Some regulations require that sensitive data never leaves certain network boundaries, period. A proxy that replaces "John Smith" with "PERSON_3" still sends the surrounding context, which might itself be revealing.
That said, for teams that are currently sending raw data to LLM APIs with no protection at all - and there are a lot of them - CloakPipe is a meaningful improvement that takes minutes to deploy. It's not a complete privacy solution, but it's a practical step in the right direction.