General skillProductivity and office

screen-vision

Reads desktop UI controls and coordinates from accessibility data, with optional verified clicks.

Stars
0
Forks
0
License
MIT
Updated
Updated Jun 26, 2026

Checking live repository facts…

Project overview

screen-vision treats the operating system’s accessibility tree as the primary source for desktop UI names, states, types, and rectangles, using OCR or icon vision only to fill gaps. Its CLI probes the host, captures a screenshot plus structured elements, and can click by element ID with dry-run as the default. It targets native, Win32, WinUI, Electron, game, or remote-desktop windows—not browser pages, which should use Playwright.

Repository facts

Primary language
Python
License
MIT
Repository updated
Jun 26, 2026
Default branch
master

Resource types

General skill

Use cases

Productivity and office

Platforms

Claude Code, Codex, and more

Runtime

Command lineDesktop

Capabilities

Computer useVerification and evals

Related projects

Browse more similar projects