uia

UI Automation CLI

Let an AI agent (or any script) see and drive native Windows apps.

uia attaches to a running Windows app and exposes it over a simple JSON protocol on stdin/stdout. With it you can:

• Find apps: list the processes that have a window, by name or title.
• See the UI: get the control tree (buttons, text boxes, menus, lists) or search for specific controls.
• Read and change values: text, checkboxes, sliders, expanders, and selections.
• Act: click buttons, move focus, scroll, type text, press key chords, and copy, cut, or paste.
• Wait for the UI: block until a control shows a value or gets focus, or until the app stops changing.
• Watch what happens: get a live stream of UI events such as clicks, focus changes, windows opening, and values changing.
• Take screenshots: capture a window or a single control, optionally with colored outlines around the controls you care about.
• Manage windows: activate, maximize, minimize, restore, or close the app.

It works with the Windows accessibility layer that screen readers use, so it covers most Win32, WinForms, WPF, and WinUI apps without any changes to them.

Install

uia runs on Windows x64 and arm64. It runs through ndx and doesn't need .NET installed.

# once, if you don't have ndx yet
irm https://github.com/devlooped/ndx/releases/latest/download/install.ps1 | iex

ndx uia

For preview builds, add the preview feed:

ndx uia --prerelease --source https://kzu.blob.core.windows.net/nuget/index.json

Use it from an agent

This repo includes a SKILL.md that teaches an agent how to install, start, and drive uia. With a harness that supports skills:

npx skills add devlooped/uia

Or copy SKILL.md into your agent's skills folder. The skill doesn't hardcode commands; it tells the agent to read the tool's own schema, so it stays up to date.

Try it

Ask it what it can do. This prints every command, its parameters, and examples as JSON:

ndx uia --help

Send one request and get one answer:

'{"jsonrpc":"2.0","id":1,"method":"processes/list","params":{"name":"notepad"}}' | ndx uia

{"jsonrpc":"2.0","id":1,"result":[{"pid":1234,"name":"Notepad","title":"Untitled - Notepad"}]}

To drive an app, keep uia running and write one JSON request per line to its stdin. Connect to an app first. After that, uia also writes UI events as they happen, in between the answers to your requests. Each answer carries the id of the request it belongs to.

Add --verbose to see a trace of what it's doing on stderr. Stdout is left for the protocol.

Interactive demo

To explore an app by hand, the repo includes a full-screen console that lists running apps, connects to the one you pick, and shows its UI events live. From there you can take screenshots, dump the control tree, and read or set control values.

dotnet run --project src/Demo

Once both packages are packed, dnx uia.demo runs that console and picks up the native uia executable for the current architecture.

Type to filter the app list and press Enter to connect. Type /help to see the available commands. Ctrl+Q quits.

Good to know

• Only controls currently on screen are found. Items inside a collapsed menu, tree, or long scrolling list show up once they're expanded or scrolled into view.
• Typing, key presses, and clipboard actions go to the app only when its window is in the foreground.
• Passwords are never read or written.

Sponsors

Learn more about GitHub Sponsors