Desktop backends

computer-use.ts owns action sequencing, screenshot IDs/recall, and tool rendering. Each backend handles desktop actions and closes its per-call input session.

• Wayland: WayVNC on 127.0.0.1:5900 for input, grim for screenshots, Firefox for new tabs. The existing 1920×1080 desktop assumption is unchanged.
• Native Windows: inbox Windows PowerShell (powershell.exe) compiles windows.cs and runs a JSON-lines helper for each tool call. Input uses SendInput; capture uses GDI/System.Drawing. No VNC server or additional packages are required. Keep windows.ps1 and windows.cs beside windows.ts.

Windows screenshots include the cursor and the entire virtual desktop, scaled into 1920×1080. Pointer coordinates use that same space, including monitors with negative native origins and differing DPI settings. Unicode text is injected without modifying the clipboard; named keys retain the tool's XKB-style spelling. URLs open through the default browser, which decides whether to reuse a tab.

Run Pi in an interactive, unlocked Windows desktop session. Input cannot cross Windows integrity-level boundaries: controlling an elevated application requires Pi to be elevated too. UAC secure desktops and login/locked screens are unsupported. Do not elevate Pi unless you need that access. Enterprise policy can prevent the PowerShell helper from running. Closing stdin lets it release its held inputs; if it hangs, cleanup forcibly terminates it after five seconds.

pnpm test tests shared sequencing/recall and the Windows helper transport without interacting with the desktop. To also test real Windows capture, run in PowerShell:

$env:PI_TEST_WINDOWS_DESKTOP = '1'
pnpm test

The native smoke test only captures a screenshot in memory; it does not click, type, open a browser, or save the screenshot.