Repolode
Submit
MiaAI-Lab

MiaAI-Lab/DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold

Serves DeepSeek-V4.1-Flash-1017 across four NVIDIA DGX Sparks with tensor parallelism and an OpenAI-compatible API, with a switchless ring option.

View on GitHubWebsite
Stars
65
Forks
25
Open issues
11
Last commit
3 days ago

What is it?

This recipe serves DeepSeek's DeepSeek-V4.1-Flash-1017 checkpoint in its published FP8 and MXFP4 form, split across four NVIDIA DGX Sparks (GB10, 128 GB each) connected through their ConnectX-7 ports with tensor parallelism over all four nodes. It exposes an OpenAI-compatible API on port 8888 that accepts text, image and video input, supports tool calling, and lets thinking be turned on or off per request.

The setup works either with a Spark-to-Spark ring cabled without a RoCE switch (start-switchless.sh) or with a normal switched network (start.sh). Scripts prepare all four machines, download the checkpoint once on the first node, share weights over NFS, and start the ranks. Optional features include structured output via response_format or guided_* parameters, DSpark speculative decoding using the checkpoint's draft blocks, and opt-in NVFP4 or abliterated weight variants.

The engine is TensorFold's native Zig implementation, with a DeepSeek-V4.1 family built on it and a series of patches maintained in the repository.

Key features

  • 01Runs DeepSeek-V4.1-Flash-1017 with tensor parallelism across four DGX Sparks.
  • 02Provides an OpenAI-compatible chat API that accepts image and video inputs.
  • 03Supports tool calling and structured output with JSON schema or grammar constraints.
  • 04Works on a switchless ring of Sparks or on a switched RoCE network.
  • 05Includes DSpark speculative decoding from the checkpoint's draft blocks.

Use cases

  • Self-hosting a large DeepSeek model on a small cluster of DGX Sparks.
  • Serving multimodal chat with image and video input behind an OpenAI-style client.
  • Running concurrent long-context requests on local GB10 hardware.

Quick start

  1. 1

    Clone the repository on spark1

    git clone https://github.com/MiaAI-Lab/DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold.git
  2. 2

    Enter the directory and create the local config file

    cd DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold && cp scripts/local.sh.example scripts/local.sh
  3. 3

    Set WORKER, WORKER2 and WORKER3 in scripts/local.sh to spark2, spark3 and spark4

  4. 4

    Start the ring setup (use ./start.sh instead if the Sparks are on a switch)

    ./start-switchless.sh

Things to know

  • Requires four NVIDIA DGX Sparks (128 GB each) linked over ConnectX-7 ports, so it is not practical on ordinary hardware.
  • Needs roughly 830 GB of disk on spark1 for the checkpoint and weight files, plus about 100 GB of free memory on each Spark.
  • Builds on a forked inference engine (TensorFold branch) and a set of local patches, which adds maintenance and setup complexity.

Topics

#llm#inference#tensor-parallelism#deepseek#openai-compatible-api#nvidia-dgx#speculative-decoding

FAQ

What is DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold?
Serves DeepSeek-V4.1-Flash-1017 across four NVIDIA DGX Sparks with tensor parallelism and an OpenAI-compatible API, with a switchless ring option. This recipe serves DeepSeek's DeepSeek-V4.1-Flash-1017 checkpoint in its published FP8 and MXFP4 form, split across four NVIDIA DGX Sparks (GB10, 128 GB each) connected through their ConnectX-7 ports with tensor parallelism over all four nodes. It exposes an OpenAI-compatible API on port 8888 that accepts text, image and video input, supports tool calling, and lets thinking be turned on or off per request.
Is DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold free to use? What license does it have?
DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold still maintained?
Actively maintained. The last commit to DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold was on Nov 4, 2026.
What can I use DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold for?
Self-hosting a large DeepSeek model on a small cluster of DGX Sparks. Serving multimodal chat with image and video input behind an OpenAI-style client. Running concurrent long-context requests on local GB10 hardware.
How do I install DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold?
Clone the repository on spark1: git clone https://github.com/MiaAI-Lab/DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold.git Enter the directory and create the local config file: cd DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold && cp scripts/local.sh.example scripts/local.sh Set WORKER, WORKER2 and WORKER3 in scripts/local.sh to spark2, spark3 and spark4 Start the ring setup (use ./start.sh instead if the Sparks are on a switch): ./start-switchless.sh
Where can I get DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold?
The source code and installation instructions are on GitHub: https://github.com/MiaAI-Lab/DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects