Computer-use agent
Also known as: computer use agent, computer-using agent
A computer-use agent is an AI system that controls a computer or web browser the way a person does: it looks at screenshots, decides what to click or type, and repeats until a task is done.
That lets it work with apps and websites that have no API. While computer-use agents can automate tedious online chores, they're still slower and less reliable than people on many tasks, and reading web pages exposes them to hidden instructions designed to hijack them.
Computer-use agent, explained at three levels
At home
A computer-use agent is like having an assistant who takes over your mouse and keyboard. You ask it to book a table or fill in a form, and it clicks through the website for you.
At work
These agents can automate repetitive tasks in old or web-based systems that don't offer easy integrations.
At home
The agent takes a screenshot, figures out what's on screen, decides the next step, and acts. It usually pauses to ask before doing anything important, like entering payment details.
At work
Agents run in a virtual machine or sandboxed browser, combining vision models with action tools for clicking, typing, and scrolling. Reliability drops on long tasks, unusual interfaces, and pop-ups, so human checkpoints are essential.
At home
A malicious web page could hide text that tells the agent to do something you didn't ask. In my opinion, that's why you shouldn't let an agent loose while you're logged into banking or email.
At work
Risk controls include isolated environments, limited credentials, domain allow-lists, confirmation for irreversible actions, full action logging, and defences against prompt injection. For stable, high-volume workflows, APIs or traditional RPA are usually cheaper and more dependable.
Real-world examples
At home
Nadia asked a computer-use agent to compare prices for a flight to Halifax across three airline websites. It gathered the options in a table but stopped and asked before booking.
At work
Carlos is an operations analyst at an Edmonton property management firm. He piloted a computer-use agent to download monthly reports from an old vendor portal with no API, running it in a sandbox with read-only credentials.
How it compares
| feature | Computer-use agent | Traditional RPA |
|---|---|---|
| How it works | Sees screen and decides each step | Follows recorded scripts |
| Handles layout changes | Often adapts | Often breaks |
| Speed and reliability | Slower, less predictable | Fast and consistent when stable |
| Setup | Describe the task | Build and maintain scripts |
Computer-use agents adapt. RPA is faster and more predictable on stable tasks.
For analysts
I'd start computer-use agents on low-risk, read-only tasks with clear review steps. Anything involving payments, personal data, or deletions should need a human click.
History & origin
Robotic process automation (RPA) tools automated screen clicks in the 2010s, but relied on brittle, pre-recorded scripts. Anthropic released 'computer use' in beta with Claude 3.5 Sonnet in October 2024, one of the first general-purpose models able to operate a desktop from screenshots. Google previewed Project Mariner, a browser agent, in December 2024, and OpenAI launched Operator in January 2025, later folding it into ChatGPT's agent mode in July 2025. AI-powered browsers followed, including Perplexity's Comet and OpenAI's ChatGPT Atlas in 2025. Benchmarks such as OSWorld, introduced in 2024, measured progress on real computer tasks. In my view, computer-use agents are where AI safety stops being abstract, because the agent can actually click 'buy.'