Repolode
Submit
RightNow-AI

RightNow-AI/autokernel

An autonomous agent pipeline that profiles a PyTorch model and optimizes its bottleneck GPU kernels in Triton or CUDA C++.

View on GitHubWebsite
Stars
1,619
Forks
164
Open issues
16
Last commit
2 days ago

What is it?

AutoKernel takes an arbitrary PyTorch model, profiles it with torch.profiler to rank kernels by GPU time, and extracts the top bottlenecks into standalone Triton or CUDA C++ kernels. A coding agent then edits one kernel file at a time, runs a fixed benchmark with five stages of correctness checks plus roofline analysis, and keeps or reverts each change. An orchestrator uses Amdahl's law to decide when to move to the next kernel.

The project targets NVIDIA GPUs and includes starter kernels for matmul, softmax, layernorm, rmsnorm, flash attention, fused MLP, cross entropy, rotary embeddings and reduce. It also ships self-contained GPT-2, LLaMA and BERT model definitions, and an integration with the KernelBench benchmark for iterative optimization of its problems. Results are logged to a TSV file, and optimized kernels can be exported to the HuggingFace Hub.

Key features

  • 01Profiles any PyTorch model and ranks kernels by GPU time, classifying them as compute- or memory-bound.
  • 02Extracts bottleneck kernels into Triton or CUDA C++ files via a --backend flag.
  • 03Runs a fixed benchmark with multi-stage correctness checks before measuring performance.
  • 04Schedules kernel optimization by estimated end-to-end impact using Amdahl's law.
  • 05Verifies optimized kernels inside the full model and reports total speedup.
  • 06Integrates with KernelBench and exports kernels to the HuggingFace Hub.

Use cases

  • Speeding up inference or training of a custom PyTorch model by optimizing its slowest GPU operations overnight.
  • Experimenting with autonomous coding agents on a well-defined GPU performance task.
  • Benchmarking AI-generated kernels against PyTorch references using KernelBench problems.
  • Generating and sharing hand-tuned Triton or CUDA kernels as HuggingFace kernel packages.

Quick start

  1. 1

    Install uv (or follow the official uv install instructions from the README)

  2. 2

    Clone the repository

    git clone https://github.com/RightNow-AI/autokernel.git
  3. 3

    Install dependencies from inside the cloned folder

    uv sync
  4. 4

    Run the one-time setup for test data and baselines, then verify the benchmark

    uv run prepare.py

Things to know

  • Requires an NVIDIA GPU; the README reports testing only on H100, A100 and RTX 4090.
  • Needs Python 3.10+ and uv.
  • Running the optimization agent requires a separate coding agent such as Claude or Codex.

In collections

Run and train AI on GPUs

Topics

#gpu#cuda#triton#pytorch#kernel-optimization#autonomous-agent#performance#deep-learning#autoresearch

FAQ

What is autokernel?
An autonomous agent pipeline that profiles a PyTorch model and optimizes its bottleneck GPU kernels in Triton or CUDA C++. AutoKernel takes an arbitrary PyTorch model, profiles it with torch.profiler to rank kernels by GPU time, and extracts the top bottlenecks into standalone Triton or CUDA C++ kernels. A coding agent then edits one kernel file at a time, runs a fixed benchmark with five stages of correctness checks plus roofline analysis, and keeps or reverts each change. An orchestrator uses Amdahl's law to decide when to move to the next kernel.
Is autokernel free to use? What license does it have?
autokernel is open source under the MIT license. Check the license terms before using it in commercial products.
Is autokernel still maintained?
Actively maintained. The last commit to autokernel was on Nov 5, 2026.
What can I use autokernel for?
Speeding up inference or training of a custom PyTorch model by optimizing its slowest GPU operations overnight. Experimenting with autonomous coding agents on a well-defined GPU performance task. Benchmarking AI-generated kernels against PyTorch references using KernelBench problems. Generating and sharing hand-tuned Triton or CUDA kernels as HuggingFace kernel packages.
How do I install autokernel?
Install uv (or follow the official uv install instructions from the README) Clone the repository: git clone https://github.com/RightNow-AI/autokernel.git Install dependencies from inside the cloned folder: uv sync Run the one-time setup for test data and baselines, then verify the benchmark: uv run prepare.py
Where can I get autokernel?
The source code and installation instructions are on GitHub: https://github.com/RightNow-AI/autokernel

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects