What computer-use agents can do, and how to run them safely
How AI agents see the screen, click and type, which jobs they're good at today, and the guardrails to put in place first.
A computer-use agent is an AI model that operates a computer the way you do. It looks at screenshots, decides where to click or what to type, and repeats until the task is done.
How it works
Each step is a loop:
- The agent takes a screenshot of the screen or browser.
- The model decides on the next action, such as clicking a button or typing into a field.
- The action runs, and the agent takes a new screenshot to see what changed.
Because it works from what's on screen, it can use apps that have no API at all.
What it's good at today
- Filling in forms with data from a spreadsheet or document
- Clicking through admin dashboards to collect or change settings
- Testing your own web app the way a user would
- Repetitive research, like checking the same detail across many sites
It is slower than an API and can misread a busy screen, so it fits jobs where a person would otherwise do the clicking.
Guardrails before you start
An agent can click anything you can. Treat it like a new assistant on their first day, not like a script.
Use a separate browser profile or a VM
Keep it away from your personal email, banking and password manager.
Stay in the loop on anything irreversible
Have it stop and ask before purchases, deletions, or sending messages.
Watch for instructions on the page
Web pages can contain text aimed at the agent. Tell it to follow only your instructions, never the page's.
Coming up
Video walkthroughs of real tasks, start to finish, are on the way on our YouTube channel.