Repolode
Submit
gpustack

gpustack/gguf-parser-go

Go tool that reads GGUF model files remotely and estimates their RAM/VRAM usage and maximum tokens per second before download.

View on GitHubWebsite
Stars
310
Forks
31
Open issues
0
Last commit
8 days ago

What is it?

GGUF Parser inspects GGUF model files and reports their metadata, architecture, tokenizer details, and estimated memory requirements. It reads only the needed chunks, so it can analyze a model hosted on Hugging Face, ModelScope, or Ollama without downloading the whole file.

The estimator reports RAM and VRAM usage for a given context size, batch size, flash attention setting, and number of offloaded layers, including split-across-GPU and RPC configurations. With device metrics supplied, it can also estimate the maximum tokens per second for a language model without running it. The tool also classifies files as embedding, reranking, decision, or LoRA models, and handles sliding-window, MLA, and audio projector architectures.

The project is written in Go and distributed as a CLI binary and Docker image.

Key features

  • 01Parses GGUF metadata from remote URLs or Hugging Face and ModelScope repos without full downloads.
  • 02Estimates RAM and VRAM usage under configurable context size, batch size, and layer offload settings.
  • 03Estimates maximum tokens per second from supplied device metrics.
  • 04Distinguishes embedding, reranking, decision, and LoRA model files.
  • 05Supports multi-GPU tensor splitting and RPC remote devices in estimates.

Use cases

  • Checking whether a quantized model fits in available GPU memory before downloading it.
  • Planning how many layers to offload to GPU when running llama.cpp-based inference.
  • Comparing memory and throughput estimates across model variants and quantization levels.

Quick start

  1. 1

    Download the binary from the releases page

  2. 2

    Parse a local GGUF file

    gguf-parser --path ~/.cache/lm-studio/models/unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF/DeepSeek-R1-Distill-Qwen-7B-Q4_K_M.gguf

Things to know

  • The maximum tokens per second estimate is experimental.
  • Some model files do not follow the general.file_type naming, so the quantization label may be inaccurate.

In collections

Run LLMs locally

Topics

#gguf#llm#memory-estimation#llama-cpp#model-inspection#inference#go#llama-box#stable-diffusion-cpp

FAQ

What is gguf-parser-go?
Go tool that reads GGUF model files remotely and estimates their RAM/VRAM usage and maximum tokens per second before download. GGUF Parser inspects GGUF model files and reports their metadata, architecture, tokenizer details, and estimated memory requirements. It reads only the needed chunks, so it can analyze a model hosted on Hugging Face, ModelScope, or Ollama without downloading the whole file.
Is gguf-parser-go free to use? What license does it have?
gguf-parser-go is open source under the MIT license. Check the license terms before using it in commercial products.
Is gguf-parser-go still maintained?
Actively maintained. The last commit to gguf-parser-go was on Oct 30, 2026.
What can I use gguf-parser-go for?
Checking whether a quantized model fits in available GPU memory before downloading it. Planning how many layers to offload to GPU when running llama.cpp-based inference. Comparing memory and throughput estimates across model variants and quantization levels.
How do I install gguf-parser-go?
Download the binary from the releases page Parse a local GGUF file: gguf-parser --path ~/.cache/lm-studio/models/unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF/DeepSeek-R1-Distill-Qwen-7B-Q4_K_M.gguf
Where can I get gguf-parser-go?
The source code and installation instructions are on GitHub: https://github.com/gpustack/gguf-parser-go

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects