Repolode
Submit
SciSharp

SciSharp/LLamaSharp

A C#/.NET library for running LLaMA-family and other GGUF language models locally on CPU or GPU, built on llama.cpp.

View on GitHubWebsite
Stars
3,816
Forks
515
Open issues
20
Last commit
5 days ago

What is it?

LLamaSharp is a cross-platform .NET library for running LLaMA models and others locally. It wraps llama.cpp native backends so that C# and .NET applications can load GGUF model files and run inference without a separate server, with support for both CPU and GPU execution.

The library provides higher-level APIs for chat sessions, executors and sampling, along with RAG support, so developers can embed an LLM into a desktop, web or game application. Integrations with Semantic Kernel and other .NET AI frameworks are available, and multimodal LLaVA models are supported.

Backend packages are published on NuGet for Windows, Linux and macOS, covering CPU, CUDA 11/12/13, Metal and Vulkan, so most users do not need to compile C++ themselves.

Key features

  • 01Loads GGUF model files and runs LLaMA-family models locally.
  • 02Offers CPU, CUDA, Metal and Vulkan backends installable via NuGet.
  • 03Provides chat session, executor and sampling APIs for C# applications.
  • 04Supports RAG and multimodal LLaVA models.
  • 05Integrates with Semantic Kernel and other .NET AI frameworks.

Use cases

  • Adding an offline chatbot to a .NET desktop or web application.
  • Running a private LLM on a local GPU without sending data to a cloud API.
  • Building RAG features in a C# codebase with a local model.
  • Experimenting with GGUF models from Hugging Face inside .NET.

Quick start

  1. 1

    Install the LLamaSharp package from NuGet

    PM> Install-Package LLamaSharp
  2. 2

    Install one or more backend packages (CPU, CUDA, Vulkan) that match your device

  3. 3

    Download a GGUF model file, such as one from Huggingface

  4. 4

    Put the README chat example code in your project and set your model path

Things to know

  • Models must be in GGUF format, and older GGUF files may not work with newer versions.
  • GPU acceleration requires a backend package that matches your CUDA or Vulkan setup, and unsupported devices may need a self-compiled backend.

Topics

#llm#llama-cpp#csharp#dotnet#local-inference#gguf#rag#chatbot#gpt#llama

FAQ

What is LLamaSharp?
A C#/.NET library for running LLaMA-family and other GGUF language models locally on CPU or GPU, built on llama.cpp. LLamaSharp is a cross-platform .NET library for running LLaMA models and others locally. It wraps llama.cpp native backends so that C# and .NET applications can load GGUF model files and run inference without a separate server, with support for both CPU and GPU execution.
Is LLamaSharp free to use? What license does it have?
LLamaSharp is open source under the MIT license. Check the license terms before using it in commercial products.
Is LLamaSharp still maintained?
Actively maintained. The last commit to LLamaSharp was on Oct 17, 2026.
What can I use LLamaSharp for?
Adding an offline chatbot to a .NET desktop or web application. Running a private LLM on a local GPU without sending data to a cloud API. Building RAG features in a C# codebase with a local model. Experimenting with GGUF models from Hugging Face inside .NET.
How do I install LLamaSharp?
Install the LLamaSharp package from NuGet: PM> Install-Package LLamaSharp Install one or more backend packages (CPU, CUDA, Vulkan) that match your device Download a GGUF model file, such as one from Huggingface Put the README chat example code in your project and set your model path
Where can I get LLamaSharp?
The source code and installation instructions are on GitHub: https://github.com/SciSharp/LLamaSharp

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects