Repolode
Submit
openclaw

openclaw/Peekaboo

A macOS CLI and menu-bar app that captures screens and app windows, inspects UI elements, and automates clicks and typing for AI agents or MCP clients.

View on GitHubWebsite
Stars
5,352
Forks
411
Open issues
9
Last commit
11 hours ago

What is it?

Peekaboo is a macOS command-line tool and menu-bar app for screen capture, accessibility inspection, and native UI automation. A typical loop observes the current screen, selects an element from a structured UI map with opaque IDs, and then acts on it with clicks, typing, key presses, scrolling, or menu commands.

Beyond direct CLI use, Peekaboo includes an agent that plans multi-step tasks from natural-language instructions, and an MCP server that exposes the same tools to clients such as Codex, Claude Code, and Cursor. Screenshots can be sent to configured local or remote AI models for visual question answering. Targeted input can be delivered to a specific window in the background, so the target app does not need to come to the front.

The CLI and app require macOS 15 or later, and source builds need Swift 6.2 and Node.js 22. Screen Recording and Accessibility permissions must be granted before use.

Key features

  • 01Captures the full screen, a single app, or a specific window to PNG files.
  • 02Builds a structured UI map of running apps with element IDs for targeting.
  • 03Clicks, types, presses keys, scrolls, and drags against specific elements or windows.
  • 04Runs an agent that turns natural-language requests into multi-step actions.
  • 05Exposes its tools through an MCP server for AI clients.
  • 06Supports background input delivery to targeted windows.

Use cases

  • Letting an AI agent operate native macOS apps like Safari or Finder.
  • Automating repetitive clicks and text entry in desktop software without scripting APIs.
  • Capturing app or window screenshots for documentation, testing, or bug reports.
  • Giving an MCP-enabled coding assistant the ability to inspect and verify UI state.

Quick start

  1. 1

    Install the CLI with Homebrew

    brew install openclaw/tap/peekaboo
  2. 2

    Check the permissions Peekaboo has

    peekaboo permissions status
  3. 3

    Take a screenshot of the screen

    peekaboo see --no-elements --mode screen --path /tmp/peekaboo-screen.png
  4. 4

    Inspect a running app to get a structured UI map

    peekaboo see --app Finder --json

Things to know

  • Requires macOS 15 or later, so it does not run on Windows or Linux.
  • Screen Recording and Accessibility permissions must be granted manually before it works fully.
  • Agent runs need a configured model provider.

Topics

#macos#screenshots#automation#mcp#accessibility#swift#ai-agent#ai

FAQ

What is Peekaboo?
A macOS CLI and menu-bar app that captures screens and app windows, inspects UI elements, and automates clicks and typing for AI agents or MCP clients. Peekaboo is a macOS command-line tool and menu-bar app for screen capture, accessibility inspection, and native UI automation. A typical loop observes the current screen, selects an element from a structured UI map with opaque IDs, and then acts on it with clicks, typing, key presses, scrolling, or menu commands.
Is Peekaboo free to use? What license does it have?
Peekaboo is open source under the MIT license. Check the license terms before using it in commercial products.
Is Peekaboo still maintained?
Actively maintained. The last commit to Peekaboo was on Oct 20, 2026.
What can I use Peekaboo for?
Letting an AI agent operate native macOS apps like Safari or Finder. Automating repetitive clicks and text entry in desktop software without scripting APIs. Capturing app or window screenshots for documentation, testing, or bug reports. Giving an MCP-enabled coding assistant the ability to inspect and verify UI state.
How do I install Peekaboo?
Install the CLI with Homebrew: brew install openclaw/tap/peekaboo Check the permissions Peekaboo has: peekaboo permissions status Take a screenshot of the screen: peekaboo see --no-elements --mode screen --path /tmp/peekaboo-screen.png Inspect a running app to get a structured UI map: peekaboo see --app Finder --json
Where can I get Peekaboo?
The source code and installation instructions are on GitHub: https://github.com/openclaw/Peekaboo

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects