EvolvingLMMs-Lab/lmms-engine

핵심 기능

• 01Distributed training with FSDP2, tensor parallelism, Ulysses sequence parallelism, and expert parallelism for MoE models.
• 02Sequence packing with first-fit bin packing and unpadded Flash Attention.
• 03Liger Triton kernels for fused CrossEntropy, RMSNorm, RoPE, and SwiGLU.
• 04Muon optimizer with Newton-Schulz orthogonalization, combined with Adam for other parameters.
• 05YAML-driven training configs launched via torchrun, Accelerate, or single GPU.
• 06Extensible registries for models, datasets, and processors.

활용 사례

• Vision-language pretraining or supervised fine-tuning of Qwen-VL style models on large multimodal datasets.
• Training Mixture-of-Experts multimodal models with expert parallelism across GPUs.
• Experimenting with diffusion or flow-matching video and image generation models such as WanVideo or SiT.
• Pretraining language models with the Muon optimizer and streaming datasets.

토픽

자주 묻는 질문

lmms-engine은(는) 무엇인가요?

A unified training engine for multimodal models, supporting vision-language, diffusion, and MoE architectures with FSDP2, Muon, and Liger kernels.

lmms-engine은(는) 무료로 사용할 수 있나요? 라이선스는 무엇인가요?

lmms-engine에는 명시된 라이선스가 없습니다. 코드는 공개되어 있지만 라이선스가 없으면 재사용 허가가 명확하지 않으므로, 사용 전에 작성자에게 문의하는 것이 좋습니다.

lmms-engine은(는) 아직 유지보수되고 있나요?

활발히 유지보수 중. lmms-engine의 마지막 커밋은 2026년 10월 18일입니다.

lmms-engine은(는) 어디에 사용할 수 있나요?

Vision-language pretraining or supervised fine-tuning of Qwen-VL style models on large multimodal datasets. Training Mixture-of-Experts multimodal models with expert parallelism across GPUs. Experimenting with diffusion or flow-matching video and image generation models such as WanVideo or SiT. Pretraining language models with the Muon optimizer and streaming datasets.

lmms-engine은(는) 어디서 받을 수 있나요?

소스 코드와 설치 방법은 GitHub에 있습니다: https://github.com/EvolvingLMMs-Lab/lmms-engine