What happens when you hand your mouse and keyboard to an AI? That question stopped being hypothetical on March 23, when Anthropic launched Claude's computer use agent in research preview. Claude can now see your screen, move the cursor, type, open applications, navigate browsers, and fill in spreadsheets - all without you touching anything.
The feature is available to Claude Pro and Max subscribers on macOS. You can message Claude a task from your phone via a companion feature called Dispatch, walk away, and come back to find the work done. Anthropic describes it as having an AI coworker that keeps going while you take lunch.
How It Actually Works
Claude first checks for direct API integrations - things like Google Calendar or Slack connectors. When those don't exist, it falls back to controlling the computer visually, "like a human does, using the screen to navigate," according to Anthropic. For developers, it can make code changes in IDEs, submit pull requests, and run tests. For everyone else, it handles things like compiling competitive analyses from local files, populating spreadsheets with data from multiple sources, or testing mobile app UX in phone simulators.
The practical catch: screen-based operation is significantly slower than direct API calls. Anthropic itself admits the feature is "early compared to Claude's ability to code," and complex tasks may need multiple attempts.
The Security Trade-Off
Anthropic built a permission-first system. Claude asks before accessing new applications, and users can stop it at any time. An auto-review layer scans each action for risky behavior and prompt injection attacks (where malicious instructions hide inside content the AI processes). The company advises against granting access to sensitive data during the preview phase.
That caution is warranted. Giving any software full desktop access creates attack surface. An AI that can click anything can, in theory, be tricked into clicking the wrong thing. The open-source agent framework OpenClaw, which went viral earlier this year and caught OpenAI's attention enough to recruit its Austrian creator, demonstrates both the demand for autonomous agents and the security questions they raise.
The broader pattern is clear: every major AI company is racing to move from "chatbot that answers questions" to "agent that does your work." Anthropic, OpenAI, and Google are all building toward AI that operates your computer rather than just talking to you. The value proposition is obvious - nobody wants to manually copy data between apps. But the gap between "useful assistant" and "autonomous program with access to your files, email, and browser" is one that the industry is crossing faster than most users have thought through.