kaito-project/aikit

What is it?

Key features

• 01OpenAI API compatible endpoint for use with existing clients
• 02Pre-made images for Qwen, Llama, Gemma, Phi and GPT-OSS models
• 03Fine-tuning interface with Unsloth support
• 04Package models as OCI artifacts following CNCF ModelPack
• 05Runs on AMD64 and ARM64 CPUs, with NVIDIA CUDA GPU acceleration
• 06Images include SBOMs, provenance attestations and signatures

Use cases

• Running an open LLM locally with Docker and no GPU
• Deploying models to a Kubernetes cluster behind an OpenAI-style API
• Fine-tuning a model and packaging the result as an OCI artifact
• Serving models in air-gapped environments using pre-mirrored registries

Quick start

1. 1

   Start the Qwen 3.5 4B model in Docker (CPU, no GPU needed)

   docker run -d --rm -p 8080:8080 ghcr.io/kaito-project/aikit/qwen3.5:4b
2. 2

   Open the WebUI at http://localhost:8080/chat

3. 3

   Send a chat request to the OpenAI-compatible API

   curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "qwen-3.5-4b", "messages": [{"role": "user", "content": "explain kubernetes in a sentence"}]}'

Things to know

• !Pre-made models carry their own licenses, such as the Llama license, which differ from the project's MIT license.
• !AMD ROCm GPU support is experimental and pre-made model images are CUDA-based, so AMD users must build their own images.
• !Runner images that download models at startup are not air-gapped by default.

In collections

Topics

FAQ

What is aikit?

Docker-based toolkit to host, build, fine-tune and package open-source LLMs behind an OpenAI-compatible API. AIKit is a platform for running open large language models locally or in Kubernetes. It uses LocalAI for inference and exposes an OpenAI API compatible endpoint, so existing OpenAI clients can send requests to open models without code changes. Pre-made container images bundle specific models such as Qwen, Llama, Gemma, Phi and GPT-OSS, and can be started with a single docker run command.

Is aikit free to use? What license does it have?

aikit is open source under the MIT license. Check the license terms before using it in commercial products.

Is aikit still maintained?

Actively maintained. The last commit to aikit was on Nov 2, 2026.

What can I use aikit for?

Running an open LLM locally with Docker and no GPU Deploying models to a Kubernetes cluster behind an OpenAI-style API Fine-tuning a model and packaging the result as an OCI artifact Serving models in air-gapped environments using pre-mirrored registries

How do I install aikit?

Start the Qwen 3.5 4B model in Docker (CPU, no GPU needed): docker run -d --rm -p 8080:8080 ghcr.io/kaito-project/aikit/qwen3.5:4b Open the WebUI at http://localhost:8080/chat Send a chat request to the OpenAI-compatible API: curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "qwen-3.5-4b", "messages": [{"role": "user", "content": "explain kubernetes in a sentence"}]}'

Where can I get aikit?

The source code and installation instructions are on GitHub: https://github.com/kaito-project/aikit