Repolode
Submit
awslabs

awslabs/awsome-distributed-ai

Reference architectures, templates and examples for running distributed AI training and inference on AWS clusters.

View on GitHubWebsite
Stars
495
Forks
215
Open issues
38
Last commit
23 hours ago

What is it?

This repository collects reference architectures, deployment templates and runnable examples for distributed AI training and inference on AWS. It covers cluster platforms including Amazon SageMaker HyperPod, AWS ParallelCluster, AWS Parallel Computing Service and Amazon EKS, with CloudFormation and Terraform templates for each.

The examples span training frameworks such as PyTorch DDP and FSDP, Megatron-LM and NeMo, and serving engines such as vLLM, SGLang and NVIDIA Dynamo. The repo also includes Packer and Ansible scripts for building custom machine images, cluster health validation tools, EFA and Nsight observability guides, and micro-benchmarks for NCCL, NCCOM and NVSHMEM.

Teams running multi-node GPU training or inference on AWS can use it as a starting point for cluster setup, launch scripts and performance tuning.

Key features

  • 01Reference architectures for SageMaker HyperPod, ParallelCluster, PCS and EKS
  • 02Training examples for PyTorch FSDP, Megatron-LM and NeMo
  • 03Inference examples for vLLM, SGLang and NVIDIA Dynamo
  • 04Packer and Ansible scripts for custom AMIs
  • 05GPU cluster health checks and EFA monitoring exporters
  • 06Micro-benchmarks for NCCL, NCCOM and NVSHMEM

Use cases

  • Setting up a Slurm or Kubernetes GPU cluster on AWS for large model training
  • Deploying multi-node LLM inference with vLLM or SGLang on EKS
  • Validating GPU and network performance before running long training jobs
  • Profiling distributed workloads with Nsight and EFA metrics

Quick start

  1. 1

    Browse the examples directory to choose a reference architecture or example

  2. 2

    Read the README of the chosen architecture or example for setup instructions

  3. 3

    Clone the repository to your local machine

    git clone https://github.com/awslabs/awsome-distributed-ai.git

Things to know

  • Targets AWS-specific clusters (SageMaker HyperPod, ParallelCluster, PCS, EKS), so it is not useful without an AWS account and cluster setup.
  • It is a collection of reference architectures and examples rather than a single installable package, so there is no one-step install.

In collections

Run LLMs locallyRun and train AI on GPUs

Topics

#distributed-training#gpu#kubernetes#slurm#llm-inference#aws#pytorch#hpc#distributed-inference#efa

FAQ

What is awsome-distributed-ai?
Reference architectures, templates and examples for running distributed AI training and inference on AWS clusters. This repository collects reference architectures, deployment templates and runnable examples for distributed AI training and inference on AWS. It covers cluster platforms including Amazon SageMaker HyperPod, AWS ParallelCluster, AWS Parallel Computing Service and Amazon EKS, with CloudFormation and Terraform templates for each.
Is awsome-distributed-ai free to use? What license does it have?
awsome-distributed-ai is open source under the MIT-0 license. Check the license terms before using it in commercial products.
Is awsome-distributed-ai still maintained?
Actively maintained. The last commit to awsome-distributed-ai was on Nov 6, 2026.
What can I use awsome-distributed-ai for?
Setting up a Slurm or Kubernetes GPU cluster on AWS for large model training Deploying multi-node LLM inference with vLLM or SGLang on EKS Validating GPU and network performance before running long training jobs Profiling distributed workloads with Nsight and EFA metrics
How do I install awsome-distributed-ai?
Browse the examples directory to choose a reference architecture or example Read the README of the chosen architecture or example for setup instructions Clone the repository to your local machine: git clone https://github.com/awslabs/awsome-distributed-ai.git
Where can I get awsome-distributed-ai?
The source code and installation instructions are on GitHub: https://github.com/awslabs/awsome-distributed-ai

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects