Repolode
Submit
Nehanth

Nehanth/pooled

Peer-to-peer LLM inference in the browser: pool laptops, desktops and phones over WebRTC to run large open models, with a built-in coding agent.

View on GitHubWebsite
Stars
634
Forks
96
Open issues
67
Last commit
5 days ago

What is it?

Pooled runs one open language model across the devices in a room. Each device holds a share of the model's layers and runs them on its own GPU through WebGPU, with hidden states passed between browser tabs over direct WebRTC connections. Opening a shared link is enough to join; there is no install step, no account, and no server doing the inference.

The project offers chat with the pooled model, and a Code mode where the model writes files, serves them in a sandboxed local preview, reads console errors and fixes them. A room can also expose its model through OpenAI- and Anthropic-compatible APIs using a small CLI bridge that runs on another machine. Models listed include Qwen 3.6 35B MoE, Qwen 3.8 27B and a small Qwen3 1.7B, with an experimental 125B Qwen3.8 Flash that streams experts from disk.

The project is MIT licensed and written mainly in JavaScript, with its WebGPU kernels in WGSL. Its documentation includes architecture notes, benchmarks on specific hardware, and a threat model stating that peers in a room can see prompts and that hidden states are not private against a determined peer.

Key features

  • 01Splits a model's layers across browser tabs on laptops, desktops and phones.
  • 02Runs computation on each device's GPU with WebGPU and custom WGSL kernels.
  • 03Passes hidden states between devices over direct WebRTC connections, with a relay fallback.
  • 04Code mode builds, previews and fixes small web apps inside the browser.
  • 05Exposes the room's model through OpenAI and Anthropic compatible APIs via a CLI bridge.

Use cases

  • Pooling a few personal devices to run a larger open model without cloud accounts.
  • Sharing one model with a small group of friends or colleagues through a single link.
  • Prototyping small web apps with an in-browser coding agent.
  • Pointing local coding agents or SDK code at a model hosted in a room.

Quick start

  1. 1

    Clone the repository

    git clone https://github.com/Nehanth/pooled && cd pooled
  2. 2

    Serve the folder locally and open the room page

    npx -y serve -l 8080 .
  3. 3

    Join a room from the command line to lend a computer (optional)

    npx @pooled/cli join <ROOM CODE>

Things to know

  • Needs a WebGPU-capable GPU in the browser for inference.
  • The Qwen3.8 Flash model is experimental and downloads about 99 GB on first run.
  • Peer-to-peer rooms can be blocked by some work networks unless the site has a relay configured.

In collections

Run LLMs locally

Topics

#llm#webgpu#webrtc#peer-to-peer#distributed-inference#browser#local-llm#inference#speculative-decoding#wgsl

FAQ

What is pooled?
Peer-to-peer LLM inference in the browser: pool laptops, desktops and phones over WebRTC to run large open models, with a built-in coding agent. Pooled runs one open language model across the devices in a room. Each device holds a share of the model's layers and runs them on its own GPU through WebGPU, with hidden states passed between browser tabs over direct WebRTC connections. Opening a shared link is enough to join; there is no install step, no account, and no server doing the inference.
Is pooled free to use? What license does it have?
pooled is open source under the MIT license. Check the license terms before using it in commercial products.
Is pooled still maintained?
Actively maintained. The last commit to pooled was on Nov 3, 2026.
What can I use pooled for?
Pooling a few personal devices to run a larger open model without cloud accounts. Sharing one model with a small group of friends or colleagues through a single link. Prototyping small web apps with an in-browser coding agent. Pointing local coding agents or SDK code at a model hosted in a room.
How do I install pooled?
Clone the repository: git clone https://github.com/Nehanth/pooled && cd pooled Serve the folder locally and open the room page: npx -y serve -l 8080 . Join a room from the command line to lend a computer (optional): npx @pooled/cli join <ROOM CODE>
Where can I get pooled?
The source code and installation instructions are on GitHub: https://github.com/Nehanth/pooled

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects