Repolode
Submit
bertholomus

bertholomus/deepseek-v4.1-tensorfold-tp2-2xgb10

A recipe for serving DeepSeek-V4.1-Flash-1017 with tensor parallelism across two NVIDIA DGX Spark (GB10) nodes, using the TensorFold TP2 engine.

View on GitHubWebsite
Stars
66
Forks
6
Open issues
13
Last commit
3 days ago

What is it?

This repository is a serving recipe for DeepSeek-V4.1-Flash-1017 on two NVIDIA DGX Spark (GB10) nodes. The model is split across the two machines with tensor parallelism (TP2) and runs on the authors' TensorFold-based engine, which exposes a server with a 1,048,576-token context window per request.

The weights are a 2.5 bpw EXL3 quantization published separately by the same authors, so the model fits across the two nodes. The repository itself holds no weights; it contains the engine build steps, a kernel kit, configuration examples, and launch scripts (serve.sh and config.env.example). Server features described include structured output, forced and automatic tool calls, image inputs, and kept prompt states for resuming long conversations.

The project documents measured decode and prefill throughput on the two-node setup, with release notes comparing versions. Its tensor-parallel setup is the two-node counterpart of a separate four-node recipe by the same authors.

Key features

  • 01Serves DeepSeek-V4.1-Flash-1017 with tensor parallelism across two DGX Spark (GB10) nodes.
  • 02Provides a 1,048,576-token context window per request.
  • 03Supports structured output and tool calls in DeepSeek's DSML format.
  • 04Accepts image inputs through the vision tower.
  • 05Includes launch scripts, configuration examples, and a kernel kit.

Use cases

  • Running a large DeepSeek model locally on a pair of GB10-class machines.
  • Serving long-context prompts and resuming large conversations from saved states.
  • Benchmarking tensor-parallel decode and prefill throughput on DGX Spark hardware.

Quick start

  1. 1

    Read RUNNING.md for the engine build, weights and run-time inputs

  2. 2

    Set up config.env from config.env.example and launch with scripts/serve.sh on both nodes

Things to know

  • Requires two NVIDIA DGX Spark (GB10) nodes for the tensor-parallel setup.
  • Weights are not included and must be obtained separately from Hugging Face.
  • The Engram tables are read at run time from DeepSeek's original shards 47 and 48.

In collections

Run and train AI on GPUs

Topics

#llm#tensor-parallelism#inference#deepseek#model-serving#quantization#nvidia#gpu

FAQ

What is deepseek-v4.1-tensorfold-tp2-2xgb10?
A recipe for serving DeepSeek-V4.1-Flash-1017 with tensor parallelism across two NVIDIA DGX Spark (GB10) nodes, using the TensorFold TP2 engine. This repository is a serving recipe for DeepSeek-V4.1-Flash-1017 on two NVIDIA DGX Spark (GB10) nodes. The model is split across the two machines with tensor parallelism (TP2) and runs on the authors' TensorFold-based engine, which exposes a server with a 1,048,576-token context window per request.
Is deepseek-v4.1-tensorfold-tp2-2xgb10 free to use? What license does it have?
deepseek-v4.1-tensorfold-tp2-2xgb10 is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is deepseek-v4.1-tensorfold-tp2-2xgb10 still maintained?
Actively maintained. The last commit to deepseek-v4.1-tensorfold-tp2-2xgb10 was on Nov 4, 2026.
What can I use deepseek-v4.1-tensorfold-tp2-2xgb10 for?
Running a large DeepSeek model locally on a pair of GB10-class machines. Serving long-context prompts and resuming large conversations from saved states. Benchmarking tensor-parallel decode and prefill throughput on DGX Spark hardware.
How do I install deepseek-v4.1-tensorfold-tp2-2xgb10?
Read RUNNING.md for the engine build, weights and run-time inputs Set up config.env from config.env.example and launch with scripts/serve.sh on both nodes
Where can I get deepseek-v4.1-tensorfold-tp2-2xgb10?
The source code and installation instructions are on GitHub: https://github.com/bertholomus/deepseek-v4.1-tensorfold-tp2-2xgb10

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects