An open-source system to run, manage, and scale AI workloads across Kubernetes, Slurm, and 20+ clouds or GPU providers with one interface.
10.7K1.3KPythonApache-2.0Updated 2 days ago
kubeflow/trainer
Kubernetes-native platform for distributed AI model training and LLM fine-tuning across PyTorch, JAX, XGBoost and other frameworks.
- Stars
- 2,251
- Forks
- 1,087
- Open issues
- 251
- Last commit
- 15 hours ago
What is it?
Kubeflow Trainer is a Kubernetes-native platform for training AI models and fine-tuning large language models. It runs jobs across frameworks including PyTorch, MLX, HuggingFace, DeepSpeed, Megatron-LM, JAX and XGBoost, and is built around the TrainJob and Runtime APIs.
The project brings MPI to Kubernetes so multi-node, multi-GPU jobs can communicate with low latency. It integrates with Kueue and Volcano for topology-aware scheduling, Slurm Bridge for hybrid Kubernetes and Slurm clusters, and KAI Scheduler for GPU-aware placement. It reuses JobSet and LeaderWorkerSet for orchestration.
A distributed data cache streams large datasets to GPU nodes with zero-copy transfer. Users can drive jobs through the Kubeflow Python SDK. The project is currently in alpha, and its APIs may change.
Key features
- 01Runs distributed training and LLM fine-tuning jobs on Kubernetes.
- 02Supports PyTorch, JAX, XGBoost, HuggingFace, DeepSpeed and Megatron-LM.
- 03Orchestrates multi-node, multi-GPU jobs with MPI.
- 04Integrates with Kueue, Volcano, Slurm Bridge and KAI Scheduler for scheduling.
- 05Provides a distributed data cache with zero-copy transfer to GPU nodes.
- 06Offers a Python SDK for submitting TrainJobs.
Use cases
- Fine-tuning large language models on a Kubernetes GPU cluster.
- Running multi-node MPI-based training on HPC-style infrastructure.
- Scheduling AI training alongside Slurm workloads in a hybrid cluster.
Quick start
- 1
Follow the official Getting Started guide in the Kubeflow Trainer documentation to install it
Things to know
- The project is in alpha status and its APIs may change.
- It is built for Kubernetes, so you need a Kubernetes cluster to run it.
Topics
FAQ
- What is trainer?
- Kubernetes-native platform for distributed AI model training and LLM fine-tuning across PyTorch, JAX, XGBoost and other frameworks. Kubeflow Trainer is a Kubernetes-native platform for training AI models and fine-tuning large language models. It runs jobs across frameworks including PyTorch, MLX, HuggingFace, DeepSpeed, Megatron-LM, JAX and XGBoost, and is built around the TrainJob and Runtime APIs.
- Is trainer free to use? What license does it have?
- trainer is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
- Is trainer still maintained?
- Actively maintained. The last commit to trainer was on Oct 21, 2026.
- What can I use trainer for?
- Fine-tuning large language models on a Kubernetes GPU cluster. Running multi-node MPI-based training on HPC-style infrastructure. Scheduling AI training alongside Slurm workloads in a hybrid cluster.
- How do I install trainer?
- Follow the official Getting Started guide in the Kubeflow Trainer documentation to install it
- Where can I get trainer?
- The source code and installation instructions are on GitHub: https://github.com/kubeflow/trainer
Summary generated by AI from the project's README. GitHub stats refresh daily.
Related projects
A PyTorch-based distributed training and inference system for large AI models, offering parallelism, memory optimization and heterogeneous training.
AI & Machine Learning#deep-learning#distributed-training#model-parallelism#pipeline-parallelism#pytorch
41.4K4.5KPythonApache-2.0Updated yesterday
A Python framework for building, scaling and deploying AI and ML workflows, from notebook prototypes to production orchestrators.
10.3K1.4KPythonApache-2.0Updated 6 days ago
A deep learning optimization library for making distributed training and inference of large models easier and more efficient.
43.2K5KPythonApache-2.0Updated 3 days ago
Python library that runs large language models layer by layer so 70B-class models can infer on a single 4GB GPU, without quantization.
39.5K4KJupyter NotebookApache-2.0Updated 3 days ago
A GPU-over-IP bridge that lets CPU-only machines use CUDA GPUs installed on remote servers over the network.
2.5K140C++Apache-2.0Updated 6 days ago