desktop-tools GitHub

Live checks

The live checks in #42 need an unlocked Mac. npm run live-check runs the ones that can be automated, through the same MCP server (dist/mcp.js) and host that agents use:

npm run live-check                    # the installed app
npm run dev-app                       # build and open the development app (see build.md)
DESKTOP_TOOLS_STATE=~/.local/state/desktop-tools-dev npm run live-check -- --only 82,88
npm run live-check -- --list          # list the checks without running them
npm run live-check -- --json /tmp/live.json

It drives the installed Desktop Tools, or the dev app whose state folder DESKTOP_TOOLS_STATE names. Use npm run dev-app for the dev app, never a copy launched from a build folder. That app must already allow the harness’s caller, the terminal or agent that runs npm run live-check, as it would any agent (see approved agents); otherwise the harness stops before opening anything and says so. Each check prints PASS, FAIL or SKIP with its evidence, and a summary groups them by PR. The exit status is 1 when any check fails, and 2 when nothing could run.

What it touches:

• It refuses to start while the screen is locked, the session is off the console, the screensaver runs or the displays sleep.
• It makes its own fixtures in a new /tmp/desktop-tools-live-* folder: a plain and a rich TextEdit document, a Finder folder of eight files and a subfolder, a PNG, and a web page with a React-style controlled input, a date input, a contenteditable, a table with <th> headers and a grid of inputs. The folder is removed at the end unless --keep-fixtures is passed.
• It opens those fixtures in TextEdit, Finder and Safari with desktop_open, and acts only on the new windows. At the end, even after a failure or Control-C, it closes each window it opened and quits only the apps it launched itself. A fixture document that asks to save is answered with Delete or Don’t Save, on that document only.
• Web checks are skipped while Safari has windows open, since a fixture page could open as a tab in one of them.
• The Chromium checks (#107, #110 and #111) each launch their own Chrome on the fixture page, isolated from the user’s: a new temporary profile (--user-data-dir), with --no-first-run and --no-default-browser-check. They run alongside a Chrome the user already has open, never touch its windows, and quit only the process they started before deleting its profile, so each check starts with web accessibility off. They are skipped when Google Chrome isn’t in /Applications or ~/Applications.
• The Finder paste checks (#114) paste the fixtures’ own files into the fixture folder, and the folder goes when the fixtures do.
• Clipboard checks run only when the clipboard holds plain text, and that text is written back at the end. The input-source check selects another enabled keyboard layout, then the original again.
• The caller checks (#120) list the windows of a running Desktop Tools and ask to activate it, which must be refused. They then run a second MCP server under /usr/bin/perl, which the app asks you about once, and while that prompt is open check that input to a fixture window is refused too. The harness waits up to two minutes for you to answer; choose Don’t Allow, and later runs only check that perl is still refused. If you allowed perl, the check is skipped until you choose Forget for it under Approved Agents. They are skipped on a host that doesn’t approve callers.
• Modifier clicks (#77) and desktop_system_key (#89) use the real keyboard and pointer, as they always do; nothing else needs the foreground.

Still manual

These stay manual, because they are destructive, change system settings, need a second display, Space or password, or need an app or account the harness can’t create:

• #80 media, volume and brightness keys: they change the output volume, playback and display brightness system-wide, and each delivered key is unknown.
• #43 window outline: what the outline looks like, during a Space switch, on a full-screen window and while dragging; the menu toggle; clicks through the margin and label.
• #78 full screen, #66 Spaces and #1 other_space: each needs a second Space.
• #1 incomplete Accessibility retry: the #111 check reads a Chrome page the moment it opens, which passes whether or not that first read raced the page. A read that comes back incomplete twice can’t be produced on demand; when one does, it should fail with retried: true.
• #2 two displays: negative-origin displays, rearranging a display between inspect and click, and SIGKILL of a client mid-drag.
• #3 clipboard lifecycle: SIGKILL of the MCP client during an insert, and quitting Desktop Tools after an uncertain insert.
• #99 wedge and permission cases: an image into a plain-text TextEdit document (holds the clipboard until TextEdit quits), clipboard: "replace" into Slack or a browser, Mail and Notes, a 10 MiB TIFF, and clipboard read access set to Ask.
• #100 Pause All Input and #98 Recent Activity: both are read and toggled in the app’s menu. Pause is the user’s alone, so the harness never chooses it through agent input, not even in a test; the app now refuses such input anyway.
• #120 Approved Agents: the first prompt for an agent, Allow, the submenu’s Revoke, Allow and Forget, the Recent Activity name (claude-code for Claude Code), and a refusal after Revoke. Only the user answers the app’s prompts and menus, so these stay manual.
• #74 notifications: they need banners from a real app, and closing or opening one acts on it.
• #41 Dock and status menus: the Dock right-click moves the pointer, and a third-party status menu belongs to the user.
• #57, #4 and #9 Open and Save panels: a Save sheet can write files outside the fixtures if answered wrongly.
• #64 modifier drags: Option-drag in Finder copies files and Command-drag moves them.
• #90 input methods: typing in an input method’s mode, such as Japanese Hiragana, needs that method enabled.
• #86 and #92 in other apps: Numbers, Excel, Mail and Chrome, and a long Finder folder of 300 or more files.
• #88 in other apps: Chrome, read-only native text views and Preview PDFs.
• #107 web accessibility in Electron apps: inspect a Slack, VS Code or Discord window and check that web_accessibility is enabled, with web_accessibility_via manual, once per launch and the web content’s fields are listed, that the app’s window still moves and resizes normally, that All Activity lists Turn on web accessibility once, and that it still runs while input is paused.
• #5 background paste into web content (spike, before any patch): with a WebKit app such as Safari in the background and a textarea focused, answer: (1) is Edit > Paste AXEnabled, with and without the host’s background key-window preparation; (2) does an Accessibility press of Edit > Paste paste into the focused textarea; (3) must the textarea get AXFocused first?
• #27 Codex: adding the server to Codex’s configuration.
• Anything that asks for a password or an administrator’s approval.

Edit this page on GitHub