Repolode
Submit
tinygrad

tinygrad/tinygrad

A minimal deep learning framework with a PyTorch-like tensor API, autograd, a compiler, and support for many accelerator backends.

Open-source alternative to:PyTorch

View on GitHub
Stars
33,780
Forks
4,382
Open issues
173
Last commit
19 hours ago

What is it?

tinygrad is an end-to-end deep learning stack built around a tensor library with autograd, an IR and compiler that fuse and lower kernels, JIT and graph execution, and nn, optim, and dataset modules for training models. Its ergonomics follow PyTorch, while the whole compiler and IR are kept small enough to read and modify.

The project runs lazily: operations are fused into kernels before execution, and a debug setting exposes the generated code. Accelerators supported include CPU, OpenCL, Metal, CUDA, AMD, NV, QCOM, and WebGPU, and the project notes that adding a new accelerator requires implementing about 25 low-level ops.

The repository ships a minimal MNIST-style training example and a quick comparison with PyTorch showing equivalent gradient code. Installation is from source with pip.

Key features

  • 01Tensor library with autograd and a PyTorch-like eager API.
  • 02Lazy execution that fuses operations into kernels, with a debug mode to inspect generated code.
  • 03JIT and graph execution for replaying captured kernels.
  • 04Built-in nn, optim, and dataset modules for training.
  • 05Supports CPU, OpenCL, Metal, CUDA, AMD, NV, QCOM, and WebGPU backends.

Use cases

  • Studying how a deep learning compiler and autograd engine work end to end.
  • Running small training jobs on a range of GPUs or accelerators.
  • Prototyping a new hardware backend with a small set of low-level ops.
  • Experimenting with a compact framework that is easier to modify than PyTorch.

Quick start

  1. 1

    Clone the repository

    git clone https://github.com/tinygrad/tinygrad.git
  2. 2

    Enter the project directory

    cd tinygrad
  3. 3

    Install tinygrad in editable mode from source

    python3 -m pip install -e .

Things to know

  • The README recommends installing from source rather than a packaged release.
  • Accelerator support varies by backend, so check your hardware is supported before relying on it.
  • The library is not yet 1.0, so APIs may change.

Topics

#deep-learning#machine-learning#autograd#compiler#tensor#gpu#neural-network

FAQ

What is tinygrad?
A minimal deep learning framework with a PyTorch-like tensor API, autograd, a compiler, and support for many accelerator backends. tinygrad is an end-to-end deep learning stack built around a tensor library with autograd, an IR and compiler that fuse and lower kernels, JIT and graph execution, and nn, optim, and dataset modules for training models. Its ergonomics follow PyTorch, while the whole compiler and IR are kept small enough to read and modify.
Is tinygrad free to use? What license does it have?
tinygrad is open source under the MIT license. Check the license terms before using it in commercial products.
Is tinygrad still maintained?
Actively maintained. The last commit to tinygrad was on Oct 20, 2026.
What can I use tinygrad for?
Studying how a deep learning compiler and autograd engine work end to end. Running small training jobs on a range of GPUs or accelerators. Prototyping a new hardware backend with a small set of low-level ops. Experimenting with a compact framework that is easier to modify than PyTorch.
How do I install tinygrad?
Clone the repository: git clone https://github.com/tinygrad/tinygrad.git Enter the project directory: cd tinygrad Install tinygrad in editable mode from source: python3 -m pip install -e .
Where can I get tinygrad?
The source code and installation instructions are on GitHub: https://github.com/tinygrad/tinygrad

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects