StanfordSpezi/SpeziLLM

What is it?

Key features

• 01Runs open-source LLMs locally on Apple devices through Swift APIs.
• 02Calls OpenAI GPT models with a declarative DSL for function calling.
• 03Discovers and dispatches inference to Fog nodes on the local network via Bonjour.
• 04Provides a download manager and onboarding view for local models from Hugging Face.
• 05Includes a ready-made SwiftUI chat view for LLM conversations.

Use cases

• Adding an on-device chatbot to an iOS or macOS app built on Spezi.
• Building a SwiftUI app that calls OpenAI models with tool-style function calling.
• Running inference on lab or clinic hardware within a local network from a mobile app.

Quick start

1. 1

   Add the SpeziLLM Swift package to your app in Xcode or as a Swift package dependency

2. 2

   Follow the setup steps for the individual target you need (e.g. SpeziLLMLocal or SpeziLLMOpenAI)

Things to know

• !Spezi LLM Local does not run in simulators because it requires a modern Metal GPU.
• !Some local models require the Increase Memory Limit entitlement.
• !The OpenAI target depends on OpenAI's API service and requires an auth token.

In collections

Topics

FAQ

What is SpeziLLM?

Swift package of LLM modules for the Spezi ecosystem, covering on-device models, OpenAI, and Fog node inference. SpeziLLM is a Swift Package for adding large language model features to Apple-platform apps built on the Spezi framework. It is split into targets: a base SpeziLLM runner, SpeziLLMLocal for running open-source models such as Llama, Phi, Gemma, and DeepSeek-R1 directly on device, SpeziLLMLocalDownload for fetching models from Hugging Face with an onboarding view, SpeziLLMOpenAI for calling OpenAI GPT models, and SpeziLLMFog for dispatching inference jobs to fog nodes on the local network.

Is SpeziLLM free to use? What license does it have?

SpeziLLM is open source under the MIT license. Check the license terms before using it in commercial products.

Is SpeziLLM still maintained?

Updated within 6 months. The last commit to SpeziLLM was on Sep 27, 2026.

What can I use SpeziLLM for?

Adding an on-device chatbot to an iOS or macOS app built on Spezi. Building a SwiftUI app that calls OpenAI models with tool-style function calling. Running inference on lab or clinic hardware within a local network from a mobile app.

How do I install SpeziLLM?

Add the SpeziLLM Swift package to your app in Xcode or as a Swift package dependency Follow the setup steps for the individual target you need (e.g. SpeziLLMLocal or SpeziLLMOpenAI)

Where can I get SpeziLLM?

The source code and installation instructions are on GitHub: https://github.com/StanfordSpezi/SpeziLLM