xlang-ai/OSWorld

What is it?

Key features

• 01Runs tasks in Ubuntu or Windows virtual machines with real desktop applications.
• 02Supports VMware, VirtualBox, Docker with KVM, Modal, Daytona and AWS providers.
• 03Evaluates results with task-specific scripts after each agent run.
• 04Runs many tasks in parallel across multiple environments.
• 05Includes baseline agent code and a result viewer for per-domain scores.

Use cases

• Benchmarking a computer-use agent against a shared set of desktop tasks.
• Reproducing published OSWorld results for a new model.
• Manually checking whether a specific benchmark task and its evaluator behave correctly.
• Running large-scale agent evaluations on cloud infrastructure.

Quick start

1. 1

   Clone the repository

   git clone https://github.com/xlang-ai/OSWorld
2. 2

   Change into the cloned directory

   cd OSWorld
3. 3

   Install required dependencies

   pip install -r requirements.txt
4. 4

   Run the quickstart example (requires VMware or Docker set up first)

   python quickstart.py

Things to know

• !Needs a local VMware or VirtualBox setup, or Docker with KVM support, before the quickstart runs.
• !Some benchmark tasks need Google account OAuth setup and may need proxy configuration.
• !Running the baseline agent requires a paid OpenAI API key.

Topics

FAQ

What is OSWorld?

A benchmark and execution environment for evaluating multimodal AI agents on open-ended computer tasks in real desktop operating systems. OSWorld is a benchmark for testing whether AI agents can complete everyday computer tasks by operating real desktop applications. Each task runs inside a virtual machine running Ubuntu or Windows, where an agent receives screenshots or accessibility information and issues mouse and keyboard actions, such as clicking, typing, or running scripts.

Is OSWorld free to use? What license does it have?

OSWorld is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.

Is OSWorld still maintained?

Actively maintained. The last commit to OSWorld was on Oct 20, 2026.

What can I use OSWorld for?

Benchmarking a computer-use agent against a shared set of desktop tasks. Reproducing published OSWorld results for a new model. Manually checking whether a specific benchmark task and its evaluator behave correctly. Running large-scale agent evaluations on cloud infrastructure.

How do I install OSWorld?

Clone the repository: git clone https://github.com/xlang-ai/OSWorld Change into the cloned directory: cd OSWorld Install required dependencies: pip install -r requirements.txt Run the quickstart example (requires VMware or Docker set up first): python quickstart.py

Where can I get OSWorld?

The source code and installation instructions are on GitHub: https://github.com/xlang-ai/OSWorld