A from-scratch PyTorch walkthrough and codebase for training a transformer LLM, covering pretraining, SFT, reward modeling, DPO/PPO/GRPO and generation.
12.3K1.8KPythonMITUpdated 18 days ago
#pretraining
Open-source projects tagged “pretraining”
Curated GitHub repositories related to pretraining, with AI summaries and key stats.