Repolode
Submit
addyosmani

addyosmani/chatty

A browser-based chat app that runs open-source LLMs locally on your GPU via WebGPU, so conversations stay on your device.

Open-source alternative to:ChatGPT

View on GitHub
Stars
841
Forks
107
Open issues
9
Last commit
3 months ago

What is it?

Chatty is a Next.js web app for chatting with large language models that run entirely inside the browser. It uses WebGPU to execute models such as Gemma, Llama and Mistral on the user's own hardware, so prompts and responses are processed locally rather than on a server. After a model is downloaded once, it can be used without an active internet connection.

Beyond basic chat, the app includes saved chat history, Markdown and code highlighting, regenerating responses, and light and dark themes. Users can load PDFs or other non-binary files and ask questions about them, with embeddings generated in the browser using a Xenova transformer model and a LangChain in-memory vector store. Custom instructions can be added to personalize responses, and conversations can be exported as JSON or Markdown.

The project is built on the WebLLM project and can be run locally with npm or Docker. Running 7B models requires a GPU with roughly 6GB of memory, and 3B models need about 3GB.

Key features

  • 01Runs LLMs locally in the browser via WebGPU, with no server-side inference.
  • 02Works offline after a model has been downloaded once.
  • 03Loads PDFs and text-based files for local question answering.
  • 04Saves chat history and exports conversations as JSON or Markdown.
  • 05Supports custom instructions, voice input, and response regeneration.

Use cases

  • Chatting with an open-source model when prompts must not leave your machine.
  • Asking questions about private documents without uploading them to a cloud service.
  • Using an LLM chat interface on a laptop with no internet connection.
  • Studying how in-browser WebGPU inference and file embeddings work in a Next.js app.

Quick start

  1. 1

    Clone the repository

    git clone https://github.com/addyosmani/chatty
  2. 2

    Open the project folder

    cd chatty
  3. 3

    Install dependencies

    npm install
  4. 4

    Start the development server and open localhost:3000

    npm run dev

Things to know

  • Requires a GPU with enough memory; 7B models need about 6GB.
  • Requires a browser with WebGPU support, such as Chrome or Edge.

In collections

Run LLMs locallyPrivacy & local-first toolsAI chatbots & chat clients

Topics

#llm#webgpu#chatbot#local-ai#privacy#offline#browser#nextjs#ai#huggingface

FAQ

What is chatty?
A browser-based chat app that runs open-source LLMs locally on your GPU via WebGPU, so conversations stay on your device. Chatty is a Next.js web app for chatting with large language models that run entirely inside the browser. It uses WebGPU to execute models such as Gemma, Llama and Mistral on the user's own hardware, so prompts and responses are processed locally rather than on a server. After a model is downloaded once, it can be used without an active internet connection.
Is chatty free to use? What license does it have?
chatty is open source under the MIT license. Check the license terms before using it in commercial products.
Is chatty still maintained?
Updated within 6 months. The last commit to chatty was on Aug 9, 2026.
What can I use chatty for?
Chatting with an open-source model when prompts must not leave your machine. Asking questions about private documents without uploading them to a cloud service. Using an LLM chat interface on a laptop with no internet connection. Studying how in-browser WebGPU inference and file embeddings work in a Next.js app.
How do I install chatty?
Clone the repository: git clone https://github.com/addyosmani/chatty Open the project folder: cd chatty Install dependencies: npm install Start the development server and open localhost:3000: npm run dev
Where can I get chatty?
The source code and installation instructions are on GitHub: https://github.com/addyosmani/chatty

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects