Repolode
Submit
ggml-org

ggml-org/ggml

A plain C/C++ tensor library for machine learning, designed to be simple, portable and efficient with minimal setup.

View on GitHubWebsite
Stars
15,591
Forks
1,920
Open issues
380
Last commit
4 days ago

What is it?

ggml is a tensor library for machine learning written in plain C and C++ with no external dependencies. It provides the low-level building blocks for running ML models, including tensor operations and a computation graph, and is designed to be easy to embed in other programs.

The library targets a wide range of hardware. It builds on x86, ARM, RISC-V, LoongArch, PowerPC, s390x and WebAssembly, includes SIMD-optimized kernels for x86, ARM and RISC-V, and offers backends for CPU, GPU, NPU and browser environments. It supports integer quantization from 2 to 8 bits as well as the MXFP4 and NVFP4 microscaling formats, and it makes no memory allocations at runtime.

The repository includes a minimal, commented matrix multiplication example under examples/simple, and documents the GGUF file format. Changes to the core library, including its CMake build system, are contributed through the llama.cpp project.

Key features

  • 01Plain C/C++ implementation with no external dependencies
  • 02SIMD-optimized kernels for x86, ARM and RISC-V
  • 03Backends for CPU, GPU, NPU and browser
  • 04Integer quantization from 2 to 8 bits, plus MXFP4 and NVFP4 formats
  • 05No memory allocations during runtime

Use cases

  • Embedding ML inference into C/C++ applications with a small footprint
  • Running tensor computations on edge devices or embedded hardware
  • Building quantized model runtimes for CPUs and accelerators
  • Running ML workloads in WebAssembly in the browser

Quick start

  1. 1

    Clone the repository

    git clone https://github.com/ggml-org/ggml
  2. 2

    Enter the project directory

    cd ggml
  3. 3

    Create a build directory and configure with CMake

    mkdir build && cd build cmake ..
  4. 4

    Build the library in Release mode

    cmake --build . --config Release -j 8

Things to know

  • Core library only, with no high-level model runtime; you must write your own code against its C API.
  • Building requires CMake and a C/C++ compiler.

Topics

#tensor-library#c-plus-plus#quantization#inference#simd#wasm#edge-ai#machine-learning

FAQ

What is ggml?
A plain C/C++ tensor library for machine learning, designed to be simple, portable and efficient with minimal setup. ggml is a tensor library for machine learning written in plain C and C++ with no external dependencies. It provides the low-level building blocks for running ML models, including tensor operations and a computation graph, and is designed to be easy to embed in other programs.
Is ggml free to use? What license does it have?
ggml is open source under the MIT license. Check the license terms before using it in commercial products.
Is ggml still maintained?
Actively maintained. The last commit to ggml was on Nov 4, 2026.
What can I use ggml for?
Embedding ML inference into C/C++ applications with a small footprint Running tensor computations on edge devices or embedded hardware Building quantized model runtimes for CPUs and accelerators Running ML workloads in WebAssembly in the browser
How do I install ggml?
Clone the repository: git clone https://github.com/ggml-org/ggml Enter the project directory: cd ggml Create a build directory and configure with CMake: mkdir build && cd build cmake .. Build the library in Release mode: cmake --build . --config Release -j 8
Where can I get ggml?
The source code and installation instructions are on GitHub: https://github.com/ggml-org/ggml

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects