Repolode
Submit
datawhalechina

datawhalechina/happy-llm

A Chinese-language open tutorial that teaches LLM principles by building a small LLaMA2-style model from scratch, covering Transformer, pretraining, fine-tuning, RAG and agents.

View on GitHubWebsite
Stars
34,463
Forks
3,253
Open issues
73
Last commit
2 months ago

What is it?

Happy-LLM is a systematic learning tutorial on large language models, published by the Datawhale community in Chinese. It starts from basic NLP concepts and works up through the Transformer architecture, pretrained language model families, and the training process of LLMs, pairing theory with hands-on code.

The practical chapters walk through implementing a LLaMA2-style model in PyTorch, training a tokenizer, and running pretraining and supervised fine-tuning, then move to training with the Transformers framework, LoRA/QLoRA parameter-efficient tuning, model evaluation, retrieval-augmented generation, and agents. A later chapter covers Agentic reinforcement learning methods such as GRPO and Search-R1. The project also links small pretrained and SFT models of about 215M parameters on ModelScope.

The tutorial is available as a readable online site and as a PDF, with chapter-specific environment notes and a separate Extra Chapter where community blog posts are accepted.

Key features

  • 01Builds a LLaMA2-style model from scratch in PyTorch
  • 02Covers Transformer, attention and encoder-decoder architectures
  • 03Includes tokenizer training, pretraining and supervised fine-tuning
  • 04Explains LoRA and QLoRA parameter-efficient fine-tuning
  • 05Introduces RAG, agents and Agentic RL methods such as GRPO

Use cases

  • Learning how LLMs work internally before reading research papers
  • Reproducing a small LLM pretraining and fine-tuning pipeline
  • Preparing for LLM application work with RAG and agents
  • Teaching LLM fundamentals in a university or training course

Quick start

  1. 1

    Open the online reading site from the README

  2. 2

    Read the study and environment preparation guide before running code

  3. 3

    Download the pretrained 215M model from ModelScope if you want to try it

Things to know

  • Content is written in Chinese, so English-speaking readers may find it hard to follow.
  • The project is a tutorial rather than a production library, and each chapter needs its own Python environment.
  • The license is CC BY-NC-SA 4.0, which prohibits commercial use.

Topics

#llm#transformer#tutorial#pytorch#fine-tuning#rag#agent#nlp

FAQ

What is happy-llm?
A Chinese-language open tutorial that teaches LLM principles by building a small LLaMA2-style model from scratch, covering Transformer, pretraining, fine-tuning, RAG and agents. Happy-LLM is a systematic learning tutorial on large language models, published by the Datawhale community in Chinese. It starts from basic NLP concepts and works up through the Transformer architecture, pretrained language model families, and the training process of LLMs, pairing theory with hands-on code.
Is happy-llm free to use? What license does it have?
happy-llm is open source under the Other license. Check the license terms before using it in commercial products.
Is happy-llm still maintained?
Updated within 6 months. The last commit to happy-llm was on Aug 8, 2026.
What can I use happy-llm for?
Learning how LLMs work internally before reading research papers Reproducing a small LLM pretraining and fine-tuning pipeline Preparing for LLM application work with RAG and agents Teaching LLM fundamentals in a university or training course
How do I install happy-llm?
Open the online reading site from the README Read the study and environment preparation guide before running code Download the pretrained 215M model from ModelScope if you want to try it
Where can I get happy-llm?
The source code and installation instructions are on GitHub: https://github.com/datawhalechina/happy-llm

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects