Repolode
Submit
BBuf

BBuf/how-to-optim-algorithm-in-cuda

A personal study and engineering notebook of hand-written CUDA kernels, CUTLASS/CuTe, Triton examples and LLM optimization notes.

View on GitHub
Stars
3,324
Forks
297
Open issues
8
Last commit
10 days ago

What is it?

This repository is a collection of CUDA kernel implementations and GPU optimization notes. It covers handwritten kernels for common operations such as reduce, softmax, elementwise ops, GEMV, indexing, atomic add, upsampling and linear attention, organized under a cuda-kernels directory.

Beyond raw kernels, it gathers material on CUTLASS and CuTe (GEMM, TMA, WGMMA, swizzling), Triton kernels with PyTorch interop, PTX ISA study notes, and PyTorch internals related to CUDA. It also includes notes from the CUDA-MODE lecture series, LLM serving and training optimization, and GPU architecture paper summaries.

The repository is maintained by its author as a study and engineering notebook. Some older material is written in Chinese and is being gradually replaced with English entry points.

Key features

  • 01Hand-written CUDA kernels for reduce, softmax, GEMV and other common operations.
  • 02CUTLASS and CuTe notes covering GEMM, TMA, WGMMA and swizzling.
  • 03Triton kernel examples with PyTorch interop.
  • 04PTX ISA study notes and PyTorch CUDA internals.
  • 05LLM inference and training optimization notes.

Use cases

  • Studying how common GPU operators are implemented in CUDA step by step.
  • Learning CUTLASS/CuTe and Triton for high-performance GEMM work.
  • Reviewing PTX-level details when tuning GPU kernels.
  • Looking up LLM inference optimization background.

Things to know

  • No license is specified, so reuse rights are unclear.
  • Older notes are mostly in Chinese and are being consolidated, so some material may be incomplete or inconsistent.

Topics

#cuda#gpu#kernel-optimization#cutlass#triton#llm#pytorch

FAQ

What is how-to-optim-algorithm-in-cuda?
A personal study and engineering notebook of hand-written CUDA kernels, CUTLASS/CuTe, Triton examples and LLM optimization notes. This repository is a collection of CUDA kernel implementations and GPU optimization notes. It covers handwritten kernels for common operations such as reduce, softmax, elementwise ops, GEMV, indexing, atomic add, upsampling and linear attention, organized under a cuda-kernels directory.
Is how-to-optim-algorithm-in-cuda free to use? What license does it have?
how-to-optim-algorithm-in-cuda has no declared license. Its code is public, but without a license you have no explicit permission to reuse it; ask the author before relying on it.
Is how-to-optim-algorithm-in-cuda still maintained?
Actively maintained. The last commit to how-to-optim-algorithm-in-cuda was on Oct 11, 2026.
What can I use how-to-optim-algorithm-in-cuda for?
Studying how common GPU operators are implemented in CUDA step by step. Learning CUTLASS/CuTe and Triton for high-performance GEMM work. Reviewing PTX-level details when tuning GPU kernels. Looking up LLM inference optimization background.
Where can I get how-to-optim-algorithm-in-cuda?
The source code and installation instructions are on GitHub: https://github.com/BBuf/how-to-optim-algorithm-in-cuda

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects