Repolode
Submit
aa1143

aa1143/NamiGPT

A Chinese-language beginner tutorial that builds GPT-2 from scratch in PyTorch, explaining each step with a tiny 'cat eats fish' example.

View on GitHubWebsite
Stars
195
Forks
20
Open issues
0
Last commit
10 days ago

What is it?

NamiGPT is a step-by-step tutorial for learning how large language models work by writing a GPT-2 implementation in PyTorch. The model code in gpt2/model.py is under 200 lines and does not use any ready-made Transformer modules. It can load the official OpenAI 124M weights and run inference and text generation.

The course is organized as 15 chapters covering tokenization, embeddings, positional encoding, self-attention, multi-head attention, residual connections, LayerNorm, the feed-forward network, activations, Transformer blocks and stacking, weight loading, text generation, and interpretability. Each chapter has a runnable script built around the three-word example "猫吃鱼". The project also includes a Gradio demo that shows continuations with per-step candidate probabilities, and pytest tests that compare outputs against the HuggingFace implementation.

The documentation is in Chinese and the course assumes basic Python and linear algebra, with no prior LLM background needed.

Key features

  • 01Builds GPT-2 from scratch in PyTorch without using built-in Transformer modules.
  • 02Loads official OpenAI GPT-2 124M weights for inference and text generation.
  • 03Provides 15 chapters, each with a runnable script using a three-word example.
  • 04Includes a Gradio demo that shows next-token probabilities at each step.
  • 05Has pytest tests that check outputs against the HuggingFace implementation.

Use cases

  • Learning how attention, residual connections and LayerNorm fit together in a GPT model.
  • Teaching a study group or course on Transformer internals with a small, readable codebase.
  • Checking your own GPT implementation against a reference that matches HuggingFace outputs.
  • Running GPT-2 text continuation locally to observe sampling settings like temperature and top-k.

Quick start

  1. 1

    Clone the repository

    git clone https://github.com/aa1143/NamiGPT.git
  2. 2

    Enter the project folder

    cd NamiGPT
  3. 3

    Install dependencies

    pip install -r requirements.txt
  4. 4

    Run a single tutorial script, for example the multi-head attention demo

    python chapters/01-inference/ep06_multi_head_attention.py

Things to know

  • Written as a beginner tutorial, so it is not a training framework or production inference library.
  • Articles under docs/ are licensed CC BY-NC 4.0, which prohibits commercial use.
  • Uses GPT-2 weights, which do not follow instructions and are mostly English-trained.

Topics

#tutorial#pytorch#transformer#gpt-2#deep-learning#attention#from-scratch#chinese#large-language-models#llm

FAQ

What is NamiGPT?
A Chinese-language beginner tutorial that builds GPT-2 from scratch in PyTorch, explaining each step with a tiny 'cat eats fish' example. NamiGPT is a step-by-step tutorial for learning how large language models work by writing a GPT-2 implementation in PyTorch. The model code in gpt2/model.py is under 200 lines and does not use any ready-made Transformer modules. It can load the official OpenAI 124M weights and run inference and text generation.
Is NamiGPT free to use? What license does it have?
NamiGPT is open source under the MIT license. Check the license terms before using it in commercial products.
Is NamiGPT still maintained?
Actively maintained. The last commit to NamiGPT was on Oct 28, 2026.
What can I use NamiGPT for?
Learning how attention, residual connections and LayerNorm fit together in a GPT model. Teaching a study group or course on Transformer internals with a small, readable codebase. Checking your own GPT implementation against a reference that matches HuggingFace outputs. Running GPT-2 text continuation locally to observe sampling settings like temperature and top-k.
How do I install NamiGPT?
Clone the repository: git clone https://github.com/aa1143/NamiGPT.git Enter the project folder: cd NamiGPT Install dependencies: pip install -r requirements.txt Run a single tutorial script, for example the multi-head attention demo: python chapters/01-inference/ep06_multi_head_attention.py
Where can I get NamiGPT?
The source code and installation instructions are on GitHub: https://github.com/aa1143/NamiGPT

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects