kalavai-net/kalavai-client

What is it?

Key features

• 01Pools GPUs and CPUs from multiple machines into one control plane.
• 02Supports NVIDIA and AMD GPUs, plus ARM64 and AMD64 CPUs.
• 03Allows fractional GPU sharing and multi-node, multi-GPU deployments.
• 04Provides ready-made templates for inference engines such as vLLM and llama.cpp.
• 05Includes a CLI and a local GUI for managing pools.

Use cases

• Serving large language models on a set of spare GPUs across several machines.
• Running CPU-based GGUF model inference with llama.cpp on ordinary computers.
• Deploying distributed fine-tuning jobs with Ray or axolotl.
• Hosting a unified model portal for a team's local and cloud compute.

Quick start

1. 1

   Install gcc compiler and python3-dev prerequisites

   sudo apt install gcc g++ python3-dev
2. 2

   Install the kalavai-client Python package

   pip install kalavai-client
3. 3

   Start a local private pool

   kalavai pool start
4. 4

   Start the GUI and open http://localhost:49153

   kalavai gui start

Things to know

• !The project is at an early stage, and breaking changes may occur before version 1.0.
• !Seed nodes require a Linux machine with Docker and root access.
• !Some AI engines and templates are listed as coming soon or experimental.

In collections

Topics

FAQ

What is kalavai-client?

Open-source platform and Python client that pools spare GPU and CPU capacity across machines into one control plane for running AI workloads. Kalavai aggregates compute from local, on-prem and multi-cloud machines into a single pool that can run large AI workloads. A pool has one or more seed nodes that act as the central control plane, and worker nodes that connect to it and carry out jobs, generally on machines with a GPU.

Is kalavai-client free to use? What license does it have?

kalavai-client is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.

Is kalavai-client still maintained?

Actively maintained. The last commit to kalavai-client was on Oct 17, 2026.

What can I use kalavai-client for?

Serving large language models on a set of spare GPUs across several machines. Running CPU-based GGUF model inference with llama.cpp on ordinary computers. Deploying distributed fine-tuning jobs with Ray or axolotl. Hosting a unified model portal for a team's local and cloud compute.

How do I install kalavai-client?

Install gcc compiler and python3-dev prerequisites: sudo apt install gcc g++ python3-dev Install the kalavai-client Python package: pip install kalavai-client Start a local private pool: kalavai pool start Start the GUI and open http://localhost:49153: kalavai gui start

Where can I get kalavai-client?

The source code and installation instructions are on GitHub: https://github.com/kalavai-net/kalavai-client