For about a month, a man's phone didn't stop ringing. Strangers were calling asking for a lawyer. Others wanted a product designer. His number had been linked to other people's identities somewhere in Google's AI systems - and there was nothing obvious he could do to make it stop.
This is the privacy problem that AI-powered search has quietly created: systems that surface real contact information, scraped from across the web, and serve it up in response to natural-language queries - with no source link visible, no context, and no easy correction process.
How AI Search Creates This Problem
AI chatbots and AI-powered search features don't generate answers from nothing. They pull from scraped web data - old forum posts, professional directories, data broker sites, cached pages from years ago - and synthesize it into confident-sounding responses. When someone asks for a contact number for a designer in their city, the AI finds a phone number associated with a name somewhere in its data and returns it. It doesn't know the association is wrong. It doesn't caveat that the number is from a 2018 forum post.
Google's AI Overviews and AI Mode features are the specific concern. Unlike traditional search results where you'd click through to see the source and judge the context yourself, AI-generated answers present information with a false air of authority. Users trust that framing. A contact number that appears in a regular search result gets some skepticism. The same number surfaced by an AI assistant gets treated as a reliable answer.
The victim here wasn't being searched by name. His number had been incorrectly associated with professional categories - the kind of queries anyone might run when looking for a service.
The Removal Process Doesn't Cover This
If your phone number appears on a data broker site, there are removal request processes. They're slow and tedious, but they exist. If it appears in AI-generated answers, the path is far less clear.
Google has a tool for requesting removal of personal contact information from search results, but it was built for traditional results pages. Whether a successful removal request actually affects what AI Overviews surfaces is not documented. Google has not announced a specific fix.
This matters for a reason beyond the immediate nuisance. Data that was once public but is now outdated, or data that was never accurate in the first place, gets a second life in AI search - and this time it arrives with no timestamp, no source, and no context that would help someone question it.
The broader pattern here is structural, not a bug. Building AI search on scraped web data means inheriting everything messy about the web: personal information posted without consent, information that's since changed, and information that was never correct. Traditional search at least surfaced the source document. AI answers strip that layer and add authority. That combination makes incorrect information more harmful, not less.
If your number is being misrepresented in AI search results right now, the MIT Technology Review report recommends filing a removal request through Google's personal information removal tool and documenting what you experience - but there's no guaranteed timeline or outcome. For most people affected, there's currently no fast fix.