What it can and cannot do
"Can an agent do everything a human can on a Mac?" is not a yes or a no. It is a list of walls, and four different reasons for being on the wrong side of one. This page is that list. Every row is measured on a real Mac, not inferred from documentation.
How to read the columns
| Mark | Means |
|---|---|
| measured | We ran it. The row says what happened, not what should happen. |
| by design | It works, we chose not to. The reason is in the row, and it is not going to change quietly. |
| not yet | We want it, it is not built. No date, because a date we invent is worse than none. |
| will not | We will not build it. Crossing this wall would make the product something else. |
| unmeasured | We could not test it here. Said out loud rather than guessed. |
Walls that exist because the tool is not you
| Wall | What actually happens | |
|---|---|---|
| macOS permissions | by design | Accessibility and Screen Recording must be granted by a human, once, in System Settings. No tool can grant them to itself - that is the whole point of them. unmeasured: which process the grant attaches to when the server is started by an MCP client over npx, and whether upgrading re-prompts. Issue #4 tracks it. |
| Touch ID, biometrics | will not | Cannot be simulated, and should not be. A fingerprint prompt is a human saying yes with their body. |
| Passwords, 2FA codes, CAPTCHAs | by design | The agent never receives one. computer_ask_user puts the cursor in the field, shows a dialog naming the app and window, and you type on your own keyboard. It returns true or false - never text. There is deliberately no route through this server for a secret to reach a model. |
Walls in macOS itself
| Wall | What actually happens | |
|---|---|---|
| A full-screen app hides everything else | measured | A full-screen window gets its own Space, and both the accessibility API and the screenshot see only the Space in front. Measured: five windows on screen, all belonging to one full-screen editor, while four other apps reported zero. This cost us half a day and it looks exactly like a broken lookup. How to tell. |
| Electron apps with no accessibility tree | measured | Chromium builds its tree lazily and an app can ship without enabling it. Measured on two Electron apps in the same minute: one returned 2000 nodes and 80 buttons, the other 8 nodes and none - at depth 14, 30 and 60 alike. Raising the limit does not help; there is nothing behind it. |
| A window on another Space | not yet | computer_screenshot with an app name fails when that app's window is on a different Space. The error now says so instead of claiming the app is not running. |
| App-scoped screenshots | measured | Measured 19/9: asking for one app's window fails for four of five apps with a ScreenCaptureKit streaming error, in the same second that a full-screen capture succeeds. Root cause not found. Full-screen capture works; use it and crop. |
| Several displays | measured | Measured 19 Sep 2026 on a Mac with three. A screenshot captures one display, and until that day it always captured display 0 without saying so, which meant two thirds of the desktop was invisible with no error. Fixed: computer_screenshot takes a display index, and every answer now reports how many displays exist and which one it took, so an agent that cannot find a window knows where else to look. The earlier row here said "this machine has one" - it had three, and nobody had checked. |
| A busy machine | measured | A screenshot takes about 0.4 seconds at rest. Measured at load 26: 42.7 seconds, four seconds short of the timeout. On a loaded Mac you will see "the helper did not answer" on a capture that would have worked. |
Walls we clear partly, and say so
| Wall | What actually happens | |
|---|---|---|
| Redacting secrets | measured | Only what macOS marks as secure is painted out. Measured on a real web page: a password input carries role AXTextField with subrole AXSecureTextField and is redacted; a plain text field is not. A password in a text editor, or a token printed in a terminal, carries no marker and will not be hidden. Use readonly when the screen holds something the system cannot know about. |
| Working without taking over the screen | measured | computer_launch · computer_quit · computer_paste · computer_window · computer_menu · computer_press and computer_set_value act on a window behind another one and never move your pointer. computer_click, computer_type and computer_key go through the system input tap: they land wherever the keyboard focus is and drag your real cursor along. Both are here; only one is quiet. |
| Several agents at once | measured | Each MCP client runs its own server. Measured: two servers, 50 interleaved writes, zero torn lines, and every line carries a session mark. But they share one pointer and one focused window, and nothing locks between them. Give background work to computer_press. |
| Knowing where the cursor is | measured | computer_focused reports the focused element and whether it is secure. The system-wide lookup alone returns nothing - measured - so it goes through the focused application, then the frontmost app. |
| Unattended running | by design | An unanswered consent dialog refuses. So a long run stops when it reaches a terminal or a password manager. That is the promise working, not a defect - but it means "leave it running for days" is true for read-only work and for apps outside the always-ask list, and not otherwise. |
Walls we will not cross
| Wall | Why | |
|---|---|---|
| Running a shell command | will not | One line of code. A computer-control server with a shell inside it is remote access under a friendlier name. Install a shell MCP server if you want one - then the choice is yours and it is visible in your config. |
| Reading arbitrary files | will not | Your .env, your SSH keys and your browser profile are all files. |
| Fetching a URL | will not | It turns a local capability into a path off the machine. |
| Writing into a secure field | measured | computer_set_value refuses on a secure field, every time. Measured by removing the check on purpose: the modified build wrote into the password field. The check is the only thing standing there. |
| Handing a secret to the model | will not | Our sister product does this, and we are not copying it. See the first table. |
Not yet, and honestly named
| What | Note | |
|---|---|---|
| Undo | not yet | A human presses cmd-Z. There is no equivalent here, and the audit log tells you what happened rather than reversing it. |
| A lock between two running servers | not yet | They share one pointer. See the third table. |
| Capturing a window on another Space | not yet | Possible in principle; not measured. |
| Switching to another Space | tried, does not work | Measured 19 Sep 2026, and we are not shipping a tool for it. macOS has no public API to change Space, so the honest route is the system shortcut a person uses - Ctrl and an arrow key. Sending that as a synthetic event does not switch the Space: the machine has 13 Spaces, the shortcut is enabled, and the set of windows on screen is identical before and after. Getting to that answer took three attempts at verifying it, and the first two were wrong in ways worth naming. Counting accessibility windows was noise - that set went 10, 10, 9, 8, 10 across four readings with nothing changing, because the accessibility API lists an app's windows regardless of which Space they are on. Listening for NSWorkspace.activeSpaceDidChangeNotification is a real signal - it stays silent when nothing changes - but it never reaches a short-lived command-line process. What finally held still was CoreGraphics' own on-screen window list, which covers only the current Space: 31 windows, zero difference across four readings a second apart. A tool that sends a keystroke and reports that nothing happened is not a capability, so there is none. If you need the agent to reach something on another Space, switch to it yourself first. |
| Windows or Linux | will not | macOS only, and none planned. The accessibility model is the product. |
The Twenty-five tools
| Look (11) | computer_permissions · computer_displays · computer_menus · computer_apps · computer_windows · computer_screenshot · computer_inspect · computer_find · computer_wait_for · computer_focused · computer_audit |
|---|---|
| Touch (12) | computer_press · computer_set_value · computer_ask_user · computer_click · computer_type · computer_key · computer_scroll · computer_move · computer_activate |
In CMCP_MODE=readonly only the nine reading tools are listed at all. A model that can see a tool will eventually try it, and a refusal per call is noise.
If you hit a wall that is not here
Then this page is incomplete, and that is worth more to us than a bug report. Tell us what you tried - describe it rather than attaching the screenshot, since the whole point is that the image may hold something you did not mean to share.
Related
MCP security: what a server can reach, and how to bound it
Every tool, and which one to reach for
The failures that look like bugs and are not