Repolode
Submit
stas00

stas00/ml-engineering

An open book of methods, scripts and step-by-step guides for training, fine-tuning and running inference on large language and multimodal models.

View on GitHubWebsite
Stars
19,566
Forks
1,299
Open issues
1
Last commit
5 days ago

What is it?

This is an open collection of methodologies, tools and step-by-step instructions for training and fine-tuning large language models and multimodal models, and for running their inference. The material is aimed at LLM and VLM training engineers and operators, and contains many copy-and-paste commands and scripts.

The content is based on hands-on experience training BLOOM-176B, IDEFICS-80B and RAG models. It is organized into parts covering hardware (compute, storage, network), orchestration (including SLURM), training, inference, debugging, and testing, plus a collection of publicly available training logbooks.

The repository also provides ebook builds in PDF and EPUB formats, and two SKILL.md files for teaching AI agents how to train models or evaluate a GPU cluster.

Key features

  • 01Covers accelerators, CPU memory, storage and inter-node networking for large-scale training
  • 02Includes SLURM cheatsheets and orchestration guidance
  • 03Provides tools for benchmarking network throughput and testing inter-node connectivity
  • 04Offers debugging recipes for hanging or failing PyTorch applications
  • 05Includes guides for choosing cloud providers and deciding on GPU upgrades

Use cases

  • Setting up and operating a multi-node GPU cluster for LLM training
  • Debugging hanging or crashing distributed PyTorch training jobs
  • Estimating whether a GPU generation upgrade is cost-effective
  • Preparing an acceptance test for a newly delivered GPU cluster

Quick start

  1. 1

    Clone the repository

    git clone https://github.com/stas00/ml-engineering
  2. 2

    Open the book's table of contents and read the guide you need (the book is markdown files, no install required)

Things to know

  • This is a collection of notes and guides rather than installable software, so there is nothing to run.
  • Content is written mainly from the author's personal experience with specific large-scale training projects.

Topics

#machine-learning#llm#gpus#pytorch#training#inference#slurm#distributed-training#ai#debugging

FAQ

What is ml-engineering?
An open book of methods, scripts and step-by-step guides for training, fine-tuning and running inference on large language and multimodal models. This is an open collection of methodologies, tools and step-by-step instructions for training and fine-tuning large language models and multimodal models, and for running their inference. The material is aimed at LLM and VLM training engineers and operators, and contains many copy-and-paste commands and scripts.
Is ml-engineering free to use? What license does it have?
ml-engineering is open source under the CC-BY-SA-4.0 license. Check the license terms before using it in commercial products.
Is ml-engineering still maintained?
Actively maintained. The last commit to ml-engineering was on Oct 16, 2026.
What can I use ml-engineering for?
Setting up and operating a multi-node GPU cluster for LLM training Debugging hanging or crashing distributed PyTorch training jobs Estimating whether a GPU generation upgrade is cost-effective Preparing an acceptance test for a newly delivered GPU cluster
How do I install ml-engineering?
Clone the repository: git clone https://github.com/stas00/ml-engineering Open the book's table of contents and read the guide you need (the book is markdown files, no install required)
Where can I get ml-engineering?
The source code and installation instructions are on GitHub: https://github.com/stas00/ml-engineering

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects