Is cache-dit free to use? What license does it have?

cache-dit is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.

Is cache-dit still maintained?

Actively maintained. The last commit to cache-dit was on Oct 21, 2026.

What can I use cache-dit for?

Speeding up FLUX or Qwen-Image text-to-image generation in a serving setup. Running large video DiT models such as Wan or HunyuanVideo on limited GPU memory. Adding multi-GPU parallelism to an existing Diffusers pipeline. Integrating faster DiT inference into ComfyUI or SGLang-based services.

Where can I get cache-dit?

The source code and installation instructions are on GitHub: https://github.com/vipshop/cache-dit

vipshop/cache-dit

A PyTorch inference engine for Diffusion Transformers (DiTs) that adds caching, parallelism, quantization and CPU offload on top of Diffusers.

View on GitHub Website

Stars: 1,290
Forks: 92
Open issues: 88
Last commit: 9 hours ago

What is it?

Cache-DiT is a PyTorch-native inference engine for Diffusion Transformers (DiTs), built on the Hugging Face Diffusers library. It targets the slow denoising step of image, video and audio diffusion pipelines by reusing intermediate computation through hybrid caching (DBCache, TaylorSeer, SCM and related methods) and by splitting work across GPUs with Context Parallelism, Tensor Parallelism and hybrid 2D or 3D parallelism.

The project covers a large set of Diffusers pipelines, including FLUX, Qwen-Image, Wan, HunyuanVideo and LTX families, and it is compatible with torch.compile, quantization such as W8A8 and W4A4, and layerwise CPU offload. It runs on NVIDIA GPUs, Ascend NPUs and AMD GPUs, and it integrates with projects such as SGLang Diffusion, vLLM-Omni and ComfyUI.

Developers who serve or experiment with DiT-based generation models can use it to lower latency or fit larger models onto limited hardware, typically with a few lines of wrapper code around an existing Diffusers pipeline.

Key features

01Hybrid cache acceleration with DBCache, TaylorSeer and SCM methods.
02Context, tensor and hybrid 2D/3D parallelism across multiple GPUs.
03Quantization support including W8A8 and W4A4.
04Layerwise CPU offload with low overhead.
05Works with torch.compile and supports nearly all DiT pipelines in Diffusers.

Use cases

Speeding up FLUX or Qwen-Image text-to-image generation in a serving setup.
Running large video DiT models such as Wan or HunyuanVideo on limited GPU memory.
Adding multi-GPU parallelism to an existing Diffusers pipeline.
Integrating faster DiT inference into ComfyUI or SGLang-based services.

Topics

#pytorch #diffusion-models #dit #inference #model-optimization #quantization #parallelism #diffusers #flux2-klein #flux3

FAQ

What is cache-dit?: A PyTorch inference engine for Diffusion Transformers (DiTs) that adds caching, parallelism, quantization and CPU offload on top of Diffusers. Cache-DiT is a PyTorch-native inference engine for Diffusion Transformers (DiTs), built on the Hugging Face Diffusers library. It targets the slow denoising step of image, video and audio diffusion pipelines by reusing intermediate computation through hybrid caching (DBCache, TaylorSeer, SCM and related methods) and by splitting work across GPUs with Context Parallelism, Tensor Parallelism and hybrid 2D or 3D parallelism.
Is cache-dit free to use? What license does it have?: cache-dit is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is cache-dit still maintained?: Actively maintained. The last commit to cache-dit was on Oct 21, 2026.
What can I use cache-dit for?: Speeding up FLUX or Qwen-Image text-to-image generation in a serving setup. Running large video DiT models such as Wan or HunyuanVideo on limited GPU memory. Adding multi-GPU parallelism to an existing Diffusers pipeline. Integrating faster DiT inference into ComfyUI or SGLang-based services.
Where can I get cache-dit?: The source code and installation instructions are on GitHub: https://github.com/vipshop/cache-dit

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects

pytorch / ao

GitHub

PyTorch-native library for quantization, sparsity and float8 training, covering both model training and inference serving.

AI & Machine Learning #quantization #pytorch #llm #float8 #inference

3K655PythonOtherUpdated 2 days ago

vllm-project / vllm

GitHub

A fast, memory-efficient library and server for running and serving large language models with an OpenAI-compatible API.

AI & Machine Learning #llm-serving #inference #pytorch #cuda #openai-api

94.2K23.6KPythonApache-2.0Updated yesterday

lyogavin / airllm

GitHub

Python library that runs large language models layer by layer so 70B-class models can infer on a single 4GB GPU, without quantization.

AI & Machine Learning #llm #inference #gpu #low-vram #pytorch

39.5K4KJupyter NotebookApache-2.0Updated 2 days ago

modelscope / DiffSynth-Studio

GitHub

An open-source Diffusion model engine for image, video, and audio generation, with low-VRAM inference and training support.

AI & Machine Learning #diffusion-models #image-generation #video-generation #lora #quantization

13.3K1.3KPythonApache-2.0Updated yesterday

openvinotoolkit / openvino

GitHub

Open-source toolkit for optimizing and deploying deep learning models for inference on Intel CPUs, GPUs and NPUs.

AI & Machine Learning #inference #deep-learning #model-optimization #llm-inference #computer-vision

11K3.5KC++Apache-2.0Updated yesterday

openvinotoolkit / nncf

GitHub

A Python framework for compressing neural networks via quantization, pruning and sparsity to speed up inference in OpenVINO, PyTorch and ONNX.

AI & Machine Learning #quantization #model-compression #openvino #deep-learning #inference

1.2K309PythonApache-2.0Updated 2 days ago