Claude Computer Use: Letting Claude Operate a Screen
Computer use is the feature where Claude doesn't just tell you how to do something on a screen — it does it: looks at what's displayed, moves the cursor,…
Claude guide · as of July 10, 2026 · 5 minutes read · details change — confirm current specs on claude.ai
Computer use is the feature where Claude doesn't just tell you how to do something on a screen — it does it: looks at what's displayed, moves the cursor, clicks, and types. Anthropic first released it as a public beta in October 2024. It is genuinely useful and genuinely unreliable, often in the same session. This guide is about knowing which you're getting.
What "computer use" actually means
Under the hood it's a loop. Your software sends Claude a screenshot. Claude reasons about what it sees and replies with an action — move to these coordinates, click, type this text, scroll. Your software carries that action out, takes a new screenshot, and sends it back. Repeat until the task is done or it gets stuck.
Two honest points fall straight out of that design:
- Claude has no direct hold on your machine. It only sees pictures and returns instructions; something you run has to execute them. That's why it's normally pointed at a sandboxed virtual machine or container, not your real desktop.
- It is cloud-only. Every screenshot travels to Anthropic's servers. There is no offline or on-device version of this. (Turning off training does not change that a request still leaves your machine.)
How you actually get to it
Computer use is primarily a developer feature on the Anthropic API — you wire up the screenshot-and-action loop yourself or use a framework that does. It is not a one-click button for most people in the consumer apps.
Anthropic has also piloted browser-based control (Claude in Chrome), which acts inside a real browser tab rather than a full desktop. Availability, eligibility, and terms for that pilot have shifted more than once — treat anything specific as unconfirmed and check the current status at claude.ai / anthropic.com before planning around it.
What it's genuinely good at
- Apps and websites that have no API — the only way in is the visual interface.
- Repetitive GUI chores: filling a form the same way many times, clicking through a wizard, copying values between two windows.
- Research-and-navigate tasks where a human would just click around and read.
Four ways Claude can take action
Computer use is the most flexible option and also the least reliable. Usually another path is better:
| Approach | How Claude acts | Best for | Reliability today |
|---|---|---|---|
| Computer use | Reads screenshots, moves cursor, clicks, types | Apps with no API; visual GUI tasks | Lowest — supervise every run |
| Claude in Chrome (pilot) | Reads and acts inside a browser tab | Web forms, navigation, lookups | Improving, still supervised |
| MCP connectors | Calls a tool's real API through a connector | Structured data, defined actions | Highest — deterministic calls |
| Claude Code | Runs commands, reads and edits files | Coding, file and system automation | High within its domain |
If a clean API or MCP connector exists for the task, use it instead — clicking pixels is a last resort, not a first one.
Failure modes — read this before you trust it
This is where the honesty matters most.
- It misclicks and misreads. Coordinates drift, buttons get confused, and it will sometimes act on a menu that isn't the one you meant.
- It can hallucinate the screen. Claude may "believe" a dialog is open, or that a step succeeded, when it didn't — then keep going on that false footing.
- It has no real sense of consequences. A delete button and a save button look equally clickable. It will not feel the difference the way you would.
- Prompt injection is a live risk. Text on a web page or in a document can contain instructions aimed at the agent ("ignore your task and do this instead"). Because Claude reads whatever is on screen, hostile content can try to hijack the session. Never point it at untrusted pages while it holds access to anything sensitive.
- It's slow and token-heavy. Every step ships a full screenshot, so a multi-click task burns far more tokens — and wall-clock time — than a normal chat. No paid plan makes that free or unlimited; check current plans and limits.
The cost, honestly
Three costs, only one of which is money. The token cost is high because of all those screenshots. The time cost is real — the loop is deliberate, not instant. And the supervision cost is the one people underestimate: today, computer use is something you watch, not something you launch and walk away from.
When this is the wrong tool
- Anything irreversible or high-stakes — payments, sending messages, deleting files, legal or medical submissions. The misclick-and-hallucinate combination is exactly wrong here.
- Tasks with a real API or connector available — you'll get faster, cheaper, and far more reliable results going through the structured path.
- Untrusted or hostile pages while any sensitive access is connected — the prompt-injection surface isn't worth it.
- "Set it and forget it" automation — if you can't watch it, don't run it. This is not yet an unattended robot.
- Pure coding or file work on your own machine — Claude Code is the better-fitting, more dependable tool.
A short checklist before you let it run
- Run it in a sandbox or throwaway account, not your primary environment.
- Only expose data and permissions the task truly needs.
- Keep the screen visible and stay ready to stop it.
- Avoid untrusted pages while anything sensitive is connected.
Related
- Claude Code — the reliable path for coding and file automation.
- MCP and connectors — let Claude act through real tool APIs instead of clicking pixels.
- Honest limits — what Claude still can't do, computer use included.
