What it can and cannot do
"Can an agent do everything a human can on a Mac?" is not a yes or a no. It is a list of walls, and four different reasons for being on the wrong side of one. This page is that list. Every row is measured on a real Mac, not inferred from documentation.
21measured
3by design
3not yet
7will not
3unmeasured
How to read the columns
| Mark | Means |
|---|---|
| measured | We ran it. The row says what happened, not what should happen. |
| by design | It works, we chose not to. The reason is in the row, and it is not going to change quietly. |
| not yet | We want it, it is not built. No date, because a date we invent is worse than none. |
| will not | We will not build it. Crossing this wall would make the product something else. |
| unmeasured | We could not test it here. Said out loud rather than guessed. |
Walls that exist because the tool is not you
| Wall | What actually happens | |
|---|---|---|
| macOS permissions | by design | Accessibility and Screen Recording must be granted by a human, once, in System Settings. No tool can grant them to itself - that is the whole point of them. unmeasured: which process the grant attaches to when the server is started by an MCP client over npx, and whether Screen Recording survives an upgrade. Accessibility did (issue #4). |
| Touch ID, biometrics | will not | Cannot be simulated, and should not be. A fingerprint prompt is a human saying yes with their body. |
| Passwords, 2FA codes, CAPTCHAs | by design | The agent never receives one. computer_ask_user puts the cursor in the field and names the app and window - in the menu bar icon in background mode, in a dialog otherwise - and you type on your own keyboard. computer_type refuses password fields, checked before every character. It returns true or false - never text. There is deliberately no route through this server for a secret to reach a model. |
Walls in macOS itself
| Wall | What actually happens | |
|---|---|---|
| A full-screen app hides everything else | measured | A full-screen window gets its own Space, and both the accessibility API and the screenshot see only the Space in front. Measured: five windows on screen, all belonging to one full-screen editor, while four other apps reported zero. This cost us half a day and it looks exactly like a broken lookup. How to tell. |
| Electron apps | measured | Chromium builds its tree lazily and an app can ship without enabling it - a bare accessibility call returned 8 nodes and no buttons at depth 14, 30 and 60 alike. This server switches the tree on itself before it looks. Measured on a VS Code fork with two windows open: 0 buttons before, 728 after, same windows, same moment. Not re-measurable on a live process, because the switch cannot be turned off again - use CMCP_INGEN_ELECTRON=1 on a fresh one. Chrome is a separate case: the attribute is an Electron API and Chrome does not implement it. Measured: 29 buttons before and after. You get the browser window, not the page in it. |
| A window on another Space | not yet | computer_screenshot with an app name fails when that app's window is on a different Space. The error now says so instead of claiming the app is not running. |
| App-scoped screenshots | measured | Measured 19/9: asking for one app's window fails for four of five apps with a ScreenCaptureKit streaming error, in the same second that a full-screen capture succeeds. Root cause not found. Full-screen capture works; use it and crop. |
| Several displays | measured | Measured 19 Sep 2026 on a Mac with three. A screenshot captures one display, and until that day it always captured display 0 without saying so, which meant two thirds of the desktop was invisible with no error. Fixed: computer_screenshot takes a display index, and every answer now reports how many displays exist and which one it took, so an agent that cannot find a window knows where else to look. The earlier row here said "this machine has one" - it had three, and nobody had checked. |
| A busy machine | measured | A screenshot takes about 0.4 seconds at rest. Measured at load 26: 42.7 seconds, four seconds short of the timeout. On a loaded Mac you will see "the helper did not answer" on a capture that would have worked. |
| Recording the screen to a file | unmeasured | measured against a stand-in helper: computer_record asks every time it starts, never hands the file to the model, and writes its ending to the audit log - unless the server itself is killed outright. unmeasured: leaving password managers out, and stopping rather than film one it cannot keep out, is the helper's job, and a real recording has not been made yet - we do not record the screen of the Mac we build on. Password fields are not blacked out in a recording the way they are in a screenshot. |
Walls we clear partly, and say so
| Wall | What actually happens | |
|---|---|---|
| Redacting secrets | measured | Only what macOS marks as secure is painted out. Measured on a real web page: a password input carries role AXTextField with subrole AXSecureTextField and is redacted; a plain text field is not. A password in a text editor, or a token printed in a terminal, carries no marker and will not be hidden. Use readonly when the screen holds something the system cannot know about. |
| Working without taking over the screen | measured | Three kinds, and the difference is how the action is delivered, not what the tool is called. Quiet by delivery: computer_press, computer_set_value and computer_menu fire the element's own accessibility action, so they work on a window behind the one you are in and never touch your pointer. The app can still bring itself forward in response - measured: File > New Finder Window did - and then press and menu answer took_screen and hand the front straight back. Quiet when you name the app: computer_click, computer_type, computer_key and computer_scroll are delivered into that app's own event queue - measured: the text arrived, the pointer stayed where it was left, and the front window did not change. computer_launch joins them with background: true - measured on a clean Mac, six of eight apps brought themselves forward as they started, and the front is handed straight back. Without an app they go through the system input tap and land wherever your keyboard focus is, so in background mode they are refused. One measured limit: keystrokes do not land in a Chromium window that is not focused, so for anything Chromium-based use computer_press and computer_set_value. Held back in background mode: computer_move, computer_drag, computer_paste, computer_activate, computer_quit, computer_space and computer_window. Each either moves your pointer, brings something to the front, or changes what you see - a window moved on a screen you are looking at is a quiet channel with a very visible result. computer_ask_user left this list on 29 September: in background mode its question waits in the menu bar icon instead of a dialog. And the held-back seven are no longer out of reach: computer_request_screen asks you to lend this one agent the screen, approved with Touch ID in the menu bar, for at most 15 minutes. While it has it, each step waits while you are using the keyboard or mouse (a step already running finishes), and one click in the menu takes the screen back and stops a step still running. Set CMCP_BACKGROUND and nothing can borrow it. |
| Several agents at once | measured | Each MCP client runs its own server. Measured: two servers, 50 interleaved writes, zero torn lines, and every line carries a session mark. Writes in the same app take turns: each holds a lock for that app, so keys and clicks never interleave within an action. Between actions they still share one pointer and one focused window; give background work to computer_press. |
| Knowing where the cursor is | measured | computer_focused reports the focused element and whether it is secure. The system-wide lookup alone returns nothing - measured - so it goes through the focused application, then the frontmost app. |
| A Save / Don't Save sheet | measured | The sheet you meet every time you close something with unsaved work. Measured: the sheet is found as AXSheet, all three buttons come back with a center and a pressable flag, and computer_find with contains: "Don't Save" returns exactly that one button - an agent looks for the words, not for a position. It is also the clearest case for the gate: a sheet can throw away work on your behalf, so in read-only mode pressing it is refused and nothing reaches the machine. Measured against a sheet the test puts up itself, in a fully transparent window, so proving it costs nobody their screen. |
| The Dock and the status icons | measured | Wi-Fi, the clock, the battery, and every app in the Dock were invisible until today, and the reason was not that they are special: they simply live outside any window. Measured: the Dock has zero windows and 32 dock items hanging off the application itself; Control Center has zero windows and nine menu bar items. The search walked down through windows only, so everything outside one was unreachable. It now falls back to the application element when an app has no windows - computer_find returns them like anything else, with a center you can click and a pressable flag. Nothing changes for apps that do have windows. |
| Whether an update breaks your permissions | measured | Not for Accessibility. The grant belongs to the host - the app that launches the server, which is your terminal, your editor or Claude Desktop - and not to our binary. Measured directly: the same helper run from a path macOS had never seen still reports Accessibility, and so does a copy with a byte appended, which gives it a different signature entirely. So upgrading the package did not send anyone back to System Settings for Accessibility; Screen Recording after an upgrade has not been measured separately. The flip side is worth knowing too: the permission you granted covers anything that host launches, not this tool alone. |
| Knowing that something is waiting for you | measured | If the server may never take the screen, it cannot knock either. A write that would need a dialog is refused, and the explanation goes back to the model - which only reaches you if you read that particular conversation. computer_pending makes it visible instead: what was asked for, when, and why it stopped. It is a list, not a button - nothing there can be approved from there, because a consent granted without a human is the one thing the gate exists to prevent. To allow the work you change the mode, and the queue simply says what is waiting. |
| Never taking over the screen at all | measured | This is the default. The server runs in background mode unless you set CMCP_BACKGROUND=0, and a typo does not turn it off - only 0, false, no or off count as a no.What that means for the tools. Until 21 September background mode hid all thirteen of those that could take the screen, which made it the product minus half its hands. That was because "takes the screen" was a property of the tool's name. It is not - it is a property of the delivery. An event sent to the global input stream lands in whatever window you are using; the same event delivered into one app's own queue does not. So four of them - computer_type, computer_key, computer_scroll and computer_click - now take an app, and with it they are offered in background mode. Without one the call is refused with an explanation of what to pass instead. Measured: 25 of the 32 are offered in background, eleven of them writing - up from 15 and three.Measured on a machine in use, through the shipped product: text typed into an app that was not in front arrived in that app, the pointer stayed where the person had left it, and the front window did not change. Typing goes into the field that app has focus in and is read back; a field that does not accept that gets keystrokes instead, which an app in the background can drop, and the answer then says the text was not confirmed. A password field gets nothing - checked before every character, so a Tab into one halfway through stops the typing there. That measurement covers typing and key presses. Clicks and scrolls take the same route and have not been measured - a mouse event carries no window number, and whether an app accepts one it never saw the pointer reach is a question we have not answered. The answer still reports on the screen honestly; it does not claim the click landed. Every key press, click, scroll and typed string now carries took_screen in its own answer, derived from what the server did rather than from what happened around it - so it cannot mistake the person moving their own mouse for the server moving it.Still hidden in background mode, and some always will be: computer_move is the pointer, computer_activate and computer_space exist to move you. computer_ask_user left this list on 29 September: its question now waits in the menu bar icon. computer_launch joins the quiet side with background: true - the app starts behind what you are doing. quit, window, drag and paste have not been rebuilt yet.One thing it does to other apps, and you should know it. To find a password field in Slack, Notion or Discord at all, Chromium's accessibility tree has to be on - so taking a screenshot switches it on in every app on screen, not only the one you asked about. Without that, the blackout is blind in exactly the apps where people type passwords. The cost is carried by those apps: they build and keep a tree they would not otherwise have. CMCP_INGEN_ELECTRON=1 turns it off, and the blackout is then blind there.The honest limit that does not change: this is a promise about what we do. Press a button and the app may open a window of its own; that is its choice, not ours. |
| Password managers, in the tree as well as the image | measured | Screenshots black out Keychain Access, Apple Passwords, 1Password, Bitwarden, LastPass and Dashlane entirely. Until today the accessibility tree did not: computer_inspect and computer_find would hand over the contents of the very window the image hides - as a read, so with no consent, in read-only mode, and in background mode. Anything not marked as a secure field came with it: a revealed password sitting in a static field, a one-time code, a secure note, every username. Both now stop at the app: find returns nothing, inspect returns one node saying denied and why - silence with a reason beats silence. Measured on a running app added to the list: 3 matches and 33 nodes became 0 and 1, with zero values. |
| Acting quietly inside an app | measured | The quiet route - computer_press and computer_set_value, which work on a window behind another one without moving your pointer - only exists where the app publishes an accessibility tree. Measured on five running apps: Chrome exposes 321 elements across 4 windows, 148 of them things you can actually press or type into. An Electron app measured before its tree was switched on exposed its 2 windows and 16 elements inside them, none usable; with the tree on, the same two windows gave 728 buttons. Finder's desktop window is 103 icons and nothing pressable. Where the tree is empty the only route left is a screenshot and a click by coordinate - which does move the real cursor. So "works on a window behind another one" is true of the app, not of every app. |
| Reaching a full-screen app | measured | computer_space switches desktop the way Control+Arrow does, because macOS gives every full-screen window its own desktop - a window the agent cannot find is often simply on another one. It is the one tool here that deliberately does move what you are looking at, so it asks every time, in every mode, like quitting an app. Two honest limits, both measured: if the system shortcut for switching desktops is off on this machine it says so and sends nothing rather than pressing a dead key; and it reports whether the desktop actually changed by comparing which windows are on screen before and after, so it never claims a switch it cannot prove. Two earlier ways of checking that were wrong - counting accessibility windows drifted 10 to 8 with nothing changing, and the system notification never reaches a short-lived command-line process. |
| An agent stuck in a loop | measured | An agent that cannot see why it is not getting anywhere does the same thing again. On a computer server each retry is a real click on a real person's machine, so the same write repeated more than ten times inside a minute is refused - and the refusal says what it saw and asks for a screenshot, because what a stuck agent is missing is sight. A guard that fires on legitimate repetition would be worse than none, so scrolling, key presses and typing are exempt: doing those ten times is simply what a person does. Both directions are measured, including that twelve identical scrolls pass untouched. |
| Unattended running | by design | An unanswered consent dialog refuses. So a long run stops when it reaches a terminal or a password manager. That is the promise working, not a defect - but it means "leave it running for days" is true for read-only work and for apps outside the always-ask list, and not otherwise. |
Walls we will not cross
| Wall | Why | |
|---|---|---|
| Running a shell command | will not | One line of code. A computer-control server with a shell inside it is remote access under a friendlier name. Install a shell MCP server if you want one - then the choice is yours and it is visible in your config. |
| Reading arbitrary files | will not | Your .env, your SSH keys and your browser profile are all files. |
| Fetching a URL | will not | It turns a local capability into a path off the machine. |
| Writing into a secure field | measured | computer_set_value refuses on a secure field, every time. Measured by removing the check on purpose: the modified build wrote into the password field. The check is the only thing standing there. |
| Handing a secret to the model | will not | Our sister product does this, and we are not copying it. See the first table. |
Not yet, and honestly named
| What | Note | |
|---|---|---|
| Undo | not yet | A human presses cmd-Z. There is no equivalent here, and the audit log tells you what happened rather than reversing it. |
| A lock between two running servers | measured | Two servers writing into the same app now take turns: each holds a per-app lock for the length of one action, and a lock held by a process that has died is broken rather than waited on. Measured with two agents typing into one window at once: 153 switches between their texts before, 1 after - one text, then the other. Two servers writing into different apps never wait for each other. |
| Capturing a window on another Space | not yet | Possible in principle; not measured. |
| Switching to another Space | tried, does not work | Measured 19 Sep 2026, on one machine. macOS has no public API to change Space, so the honest route is the system shortcut a person uses - Ctrl and an arrow key. Sending that as a synthetic event does not switch the Space: the machine has 13 Spaces, the shortcut is enabled, and the set of windows on screen is identical before and after. Getting to that answer took three attempts at verifying it, and the first two were wrong in ways worth naming. Counting accessibility windows was noise - that set went 10, 10, 9, 8, 10 across four readings with nothing changing, because the accessibility API lists an app's windows regardless of which Space they are on. Listening for NSWorkspace.activeSpaceDidChangeNotification is a real signal - it stays silent when nothing changes - but it never reaches a short-lived command-line process. What finally held still was CoreGraphics' own on-screen window list, which covers only the current Space: 31 windows, zero difference across four readings a second apart. That verification is what computer_space now uses, and it shipped later the same day: it sends the shortcut and then reports whether the desktop actually changed, rather than claiming a switch it cannot prove. On a machine like the one measured here, expect it to say that nothing changed. See Reaching a full-screen app above. |
| Windows or Linux | will not | macOS only, and none planned. The accessibility model is the product. |
The thirty-two tools
| Look (14) | computer_permissions · computer_record · computer_apps · computer_windows · computer_displays · computer_menus · computer_screenshot · computer_inspect · computer_pending · computer_audit · computer_find · computer_wait_for · computer_focused · computer_learning |
|---|---|
| Touch (18) | computer_request_screen · computer_launch · computer_open · computer_quit · computer_drag · computer_space · computer_paste · computer_window · computer_menu · computer_set_value · computer_press · computer_ask_user · computer_click · computer_move · computer_scroll · computer_type · computer_key · computer_activate |
In CMCP_MODE=readonly only the fourteen reading tools are listed at all. A model that can see a tool will eventually try it, and a refusal per call is noise.
If you hit a wall that is not here
Then this page is incomplete, and that is worth more to us than a bug report. Tell us what you tried - describe it rather than attaching the screenshot, since the whole point is that the image may hold something you did not mean to share.
Related
MCP security: what a server can reach, and how to bound it
Every tool, and which one to reach for
The failures that look like bugs and are not