Computer use

AI agents that operate a computer the way a person does, by reading the screen and moving the mouse and keyboard, to work in software that has no API.

Definition

Computer-use agents look at screenshots, decide where to click or what to type, and repeat until a task is done. Anthropic released computer use in public beta in October 2024, and OpenAI followed with its Operator agent in January 2025; other providers offer similar capabilities.

The appeal is reach: an agent can work in legacy systems, internal web tools and websites without a custom integration. The drawbacks are speed, cost per task and reliability, which are all worse than calling an API, plus new security risks when an agent reads untrusted pages. Companies use it for tasks where no API exists or building one is not worth it.

Sources

  1. Anthropic, Introducing computer use (Oct 2024)
  2. OpenAI, Introducing Operator (Jan 2025)