Python library for automating Windows GUI applications by controlling windows, dialogs and controls through Win32 API or MS UI Automation.
microsoft/UFO
Microsoft's UFO³ framework coordinates GUI agents across multiple devices, using DAG-based task decomposition, with UFO² as its Windows desktop agent.
- Stars
- 10,178
- Forks
- 1,082
- Open issues
- 95
- Last commit
- 8 days ago
What is it?
UFO³ is a research framework from Microsoft for running AI agents across several devices at once. Its Galaxy component, the multi-device orchestration layer, takes a user request, breaks it into a directed acyclic graph (DAG) of tasks, assigns each task to a device based on its platform and capabilities, and runs independent tasks in parallel. The graph can be rewritten at runtime as results come back.
Devices connect to the orchestrator through a WebSocket-based protocol called AIP. The repository covers Windows, Linux and Android device agents, with MCP (Model Context Protocol) support for adding tools to them. UFO² is the older single-device component: a Windows desktop agent that combines UI Automation with API calls and is described as the stable, long-term-support option for Windows-only automation.
The project is configured with YAML files for the LLM provider (OpenAI, Azure OpenAI and others) and for the device pool. Python 3.10 or 3.11 is required.
Key features
- 01Decomposes user requests into a DAG of tasks with dependencies and runs independent tasks in parallel.
- 02Assigns tasks to devices by platform and capability, across Windows, Linux and Android.
- 03Lets the task graph be edited at runtime as execution results arrive.
- 04Connects devices to the orchestrator over the WebSocket-based AIP protocol with automatic reconnection.
- 05Integrates MCP so device agents can use external tools.
- 06Includes UFO², a Windows desktop agent that combines GUI control with API actions.
Use cases
- Running a workflow that starts on a Windows PC and continues on a Linux server or Android phone.
- Automating routine Windows desktop tasks with a single agent via UFO².
- Researching or prototyping multi-agent GUI automation across heterogeneous devices.
Quick start
- 1
Install dependencies from requirements file
pip install -r requirements.txt - 2
Copy the Galaxy agent config template
copy config\galaxy\agent.yaml.template config\galaxy\agent.yaml - 3
Edit the config file and add your API keys, and register your devices in config\galaxy\devices.yaml
- 4
Start the device agents as described in the platform-specific guides, then launch Galaxy
python -m galaxy --interactive
Things to know
- Setup requires configuring LLM API keys and several device agents, which takes effort.
- The Galaxy multi-device framework is under active development and may change.
- The UFO² Windows agent is the stable option; Galaxy's device support is limited to Windows, Linux, and Android.
In collections
Topics
FAQ
- What is UFO?
- Microsoft's UFO³ framework coordinates GUI agents across multiple devices, using DAG-based task decomposition, with UFO² as its Windows desktop agent. UFO³ is a research framework from Microsoft for running AI agents across several devices at once. Its Galaxy component, the multi-device orchestration layer, takes a user request, breaks it into a directed acyclic graph (DAG) of tasks, assigns each task to a device based on its platform and capabilities, and runs independent tasks in parallel. The graph can be rewritten at runtime as results come back.
- Is UFO free to use? What license does it have?
- UFO is open source under the MIT license. Check the license terms before using it in commercial products.
- Is UFO still maintained?
- Actively maintained. The last commit to UFO was on Oct 30, 2026.
- What can I use UFO for?
- Running a workflow that starts on a Windows PC and continues on a Linux server or Android phone. Automating routine Windows desktop tasks with a single agent via UFO². Researching or prototyping multi-agent GUI automation across heterogeneous devices.
- How do I install UFO?
- Install dependencies from requirements file: pip install -r requirements.txt Copy the Galaxy agent config template: copy config\galaxy\agent.yaml.template config\galaxy\agent.yaml Edit the config file and add your API keys, and register your devices in config\galaxy\devices.yaml Start the device agents as described in the platform-specific guides, then launch Galaxy: python -m galaxy --interactive
- Where can I get UFO?
- The source code and installation instructions are on GitHub: https://github.com/microsoft/UFO
Summary generated by AI from the project's README. GitHub stats refresh daily.
Related projects
An AI agent that controls Windows at the GUI level by reading the UI Automation tree and letting any LLM choose clicks, typing and commands.
An open-source runtime where a persistent queen agent plans work, spawns parallel worker agents, and tracks results in a shared SQLite tracker.
Open-source multi-agent AI assistant that runs tasks across web, desktop, files and chat apps like Telegram, Feishu and WeCom.
A Go, Rust and Python multi-agent orchestration framework for running LLM agents in production, with Temporal workflows and token budgets.
An Azure-based reference accelerator where multiple specialized AI agents plan and execute user-specified organizational tasks, built on Microsoft Agent Framework and Azure AI Foundry.