Computer-use agents for software with no API
AI can now operate a computer about as well as people on a standard test. Where that helps with portals and legacy systems, and how to run it safely.
Most businesses have at least one system nothing connects to. A supplier portal you log into to download invoices one at a time. A government site where forms are filled in by hand. An old accounting package on one office PC. A marketplace seller dashboard that has to be updated separately from your own website.
For years the answer was either "someone does it by hand" or brittle automation scripts that broke whenever a button moved. A new kind of AI is getting good enough to change that.
What happened this month
On 5 March, OpenAI released GPT-5.4 with computer use built in. The model looks at screenshots, decides where to click and what to type, and works through a task step by step. OpenAI reported a 75% success rate on OSWorld-Verified, a standard test of everyday desktop tasks, slightly above the 72.4% it cites for people doing the same tasks.
On 23 March, Anthropic added computer use, as a research preview, to Claude Cowork and the Claude Code desktop app for Pro and Max subscribers, starting on Mac. Claude can open your applications, click through them and finish a job you describe, using the apps you already have.
Neither idea is new. Anthropic first showed computer use in 2024, and OpenAI and Google have shipped browser agents since. What is new is that it has become reliable enough to think about for real work.
Where it earns its keep
The best tasks share a profile: repetitive, done in software with no API, easy to check, and not catastrophic if one attempt goes wrong.
- Pulling documents out of portals. Downloading this month's invoices or statements from several supplier or bank portals and saving them into the right folders.
- Copying between systems. Taking orders from one screen and entering them in another, or updating stock on a marketplace dashboard to match your own system.
- Routine checks. Looking up the status of fifty shipments on a courier's site, or checking which listings have gone out of stock.
- Testing your own software. Clicking through your website or app the way a customer would, to see whether checkout still works after a change.
Where it still fails
A 75% success rate on a benchmark is impressive. It also means one task in four does not finish correctly. Real business software is messier than test tasks: pop-ups, slow pages, sessions that time out, layouts that change.
These agents are also slower than a script, often slower than a skilled person, and every step costs money because each screenshot is processed by the model. A task a person does in two minutes can take the agent several times longer.
So computer use is a poor fit for anything where a single mistake is expensive or hard to undo: payments, filings with legal weight, deleting records, anything sent to a customer without review.
Use the API first
If a system has a proper API or an MCP connector, use that instead. It is faster, cheaper and far more reliable than having an AI look at screenshots and click buttons. Computer use is for the gaps, the systems that will never get an API. It should not replace integrations that already exist.
How to run it safely
An agent that can use your computer can do anything you can do on it. Set it up accordingly.
- Give it its own account with the minimum permissions the task needs, never the owner's login.
- Run it in a separate machine or virtual desktop, not on a laptop that is logged into email, banking and admin panels.
- Keep logins that need OTPs or national ID logins, such as UAE Pass or Aadhaar-based sign-ins, with people.
- Require a person to approve anything irreversible: submitting, paying, sending, deleting.
- Keep a log of what it did, and review it, especially for the first few weeks.
- Check the terms of the site. Some portals forbid automated access, and breaking those rules can cost you the account.
There is also the risk we wrote about in December with AI browsers: text hidden in a web page or document can try to give the agent instructions. A separate account with limited access is the main defence. It keeps the damage small if something does go wrong.
A sensible first project
Pick one weekly task that a person on your team finds tedious, that lives in a system with no API, and whose result is easy to check. Write down the steps exactly. Run the agent on it alongside the person for two weeks and compare the results.
If it handles the routine cases and flags the odd ones for a human, you have saved those hours every week. If it does not, you have spent very little finding out, and the models are improving fast enough that it is worth trying again in a few months.
