Computer MCP / Use cases / Without taking over your screen
Let an agent work while you keep using your Mac
Three of the eighteen tools act on a background window without moving your pointer: press an element by its own accessibility action, write into a field, and wait for something to appear instead of looping on screenshots.
The usual complaint about computer-use tools is not that they are weak. It is that using one means giving up your machine: the pointer jumps, windows come forward, and you sit and watch instead of working. Three of the eighteen tools exist specifically so that does not have to happen.
The three that do not touch your pointer
| Tool | What it does instead |
|---|---|
computer_press |
Fires an element's own accessibility action, the same one a screen reader would trigger. No synthetic click, no pointer movement, and it works on a window that is behind another one. The button is pressed where it stands. |
computer_set_value |
Writes a value straight into a field through the accessibility API. No focus change, no keystrokes, no cursor. It refuses on a secure field every single time. |
computer_wait_for |
Waits for an element to appear, then returns. The alternative is a screenshot loop, and twenty polls cost one call here against twenty images the other way. |
Those three cover a surprising share of real work: press the button, fill the field, wait for the result. What they do not cover is anything that has no accessibility element behind it, such as dragging on a canvas or clicking a spot in a video. Those still need the real pointer, and there is no way around that on macOS.
A concrete run
Say an export takes four minutes and then a dialog appears. The version that takes over your screen looks like this: activate the app, click, screenshot, screenshot, screenshot, screenshot, click. Every one of those steals focus, and the screenshots cost tokens for pictures of a progress bar.
The version that leaves you alone:
computer_find role=AXButton title="Export" -> element
computer_press that element # no pointer, window stays behind
computer_wait_for role=AXSheet timeout=300 # one call, returns when it appears
computer_press the sheet's "Save" buttonFour calls, no focus change, one image at most. You carry on typing in another app the whole time.
What still takes the screen, honestly
computer_clickandcomputer_typeuse the real cursor and the real keyboard, because macOS offers no other route for a synthetic event. If the agent falls back to these, you will see it.computer_activatebrings an app forward. That is its whole job. It is a write-tier tool, so inaskmode it goes through the gate like any other.- The consent dialog itself. A native dialog is supposed to interrupt you: that is
the point of asking. If you do not want to be interrupted, that is what
allowmode is for, and the log still records everything. - A full-screen app hides every other window from both the accessibility API and the screenshot, because it gets its own Space. Leave the target windowed.
Several agents at once
Each MCP client starts its own server process, so running Claude Code and Cursor side by
side gives you two independent servers with two independent consent states. They do not
queue behind each other. Every line in the audit log carries a per-server session mark, and
CMCP_CLIENT lets you give each one a readable name, so afterwards you can tell
which agent pressed what.
The log is opened only-grows per write, which is what keeps two servers from tearing each other's lines. That is not an assumption: there is a test in the repo that runs two servers through fifty interleaved writes and fails if a single line comes out torn.
FAQ
Can an agent use my Mac without taking over the screen?
Largely, yes. computer_press fires an element's own accessibility action rather than simulating a click, so it works on a background window without moving your pointer. computer_set_value writes into a field the same way, and computer_wait_for waits for an element instead of taking screenshots in a loop. What still needs the real screen is anything with no accessibility element behind it, such as dragging on a canvas.
Does clicking a button require bringing the app forward?
Not with computer_press. It triggers the button's own AXPress action where the window stands, behind whatever else is open. computer_click does use the real cursor, so if the agent falls back to it you will see the pointer move.
Can I run two agents at the same time?
Yes. Each MCP client starts its own server process with its own consent state, so they do not queue behind each other. Every audit line carries a per-server session mark, and CMCP_CLIENT gives each one a readable name.
Will two servers corrupt the shared audit log?
No. Each write is an only-grows open, and the repo has a test that runs two servers through fifty interleaved writes and fails if a single line comes out torn.
Why does computer_wait_for matter for cost?
Because the alternative is a screenshot loop. Twenty polls through wait_for cost one tool call; twenty screenshots cost twenty images in the model's context, of a progress bar.
The agent says it cannot see a window that is open. Why?
Almost always a full-screen app. True full-screen puts an app on its own Space, and from outside that Space neither the accessibility API nor the screenshot sees the windows underneath. Leave the target app windowed.
Does the consent dialog interrupt me every time?
In ask mode the first write opens one dialog that grants the session; after that only password managers and terminals ask again, every time, regardless of mode. In allow mode nothing asks except those, and everything is still logged.