Repolode
Submit
cactus-compute

cactus-compute/needle

A tiny on-device foundation model for tool calling, structured extraction, embeddings and speech-to-text on phones, wearables, robots and microcontrollers.

View on GitHubWebsite
Stars
14,099
Forks
1,091
Open issues
4
Last commit
12 hours ago

What is it?

Needle is a small foundation model for automation on constrained hardware. The full model is an 8 to 29 MB binary, and it is built to pick the right functions from a list you expose, fill their arguments from what a user said, and return an empty list instead of guessing when no tool fits. It also extracts typed fields from messy text against a schema you declare, and returns a vector for a sentence so an app can search or route locally.

The Python package exposes a decorator-based API: a function's signature defines argument types, its docstring becomes the tool description, and a run call executes the matched function and returns its results. Each response includes the function calls, reasoning and a confidence score. Whistle, a companion speech-to-text model in a 16.9 MB file, shares the same runtime, so audio can go in and tool calls come out in one call. Whistle transcribes English, German, French, Spanish, Italian, Dutch and Polish.

Needle supports local fine-tuning with LoRA adapters, plus a hosted fine-tuning path, and ships prebuilt engines for multiple platform folders, a C API, browser and WASI targets. A community-maintained npm package, cactus-needle, runs it in Node.js and browsers.

Key features

  • 01Selects tool calls and fills their arguments from user requests, returning an empty list when no tool applies.
  • 02Extracts typed fields from text according to a declared schema, with output guaranteed to parse.
  • 03Returns text embeddings for local search, matching and routing.
  • 04Transcribes speech with Whistle, including word timestamps, keyword biasing and live streaming.
  • 05Provides a Python decorator API that runs matched functions and returns their results.
  • 06Supports local LoRA fine-tuning and hosted fine-tuning across model depths.

Use cases

  • Voice or text assistants on phones that trigger app actions locally.
  • Smart home and wearable devices that map spoken commands to device functions.
  • Extracting invoices, bookings or forms from messy text on an offline device.
  • Local speech-to-text on an always-on microphone paired with a wake word or voice activity detector.

Topics

#edge-ai#on-device-ai#tool-calling#function-calling#tinyml#speech-to-text#embeddings#cactus#foundation-model#llm

FAQ

What is needle?
A tiny on-device foundation model for tool calling, structured extraction, embeddings and speech-to-text on phones, wearables, robots and microcontrollers. Needle is a small foundation model for automation on constrained hardware. The full model is an 8 to 29 MB binary, and it is built to pick the right functions from a list you expose, fill their arguments from what a user said, and return an empty list instead of guessing when no tool fits. It also extracts typed fields from messy text against a schema you declare, and returns a vector for a sentence so an app can search or route locally.
Is needle free to use? What license does it have?
needle is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is needle still maintained?
Actively maintained. The last commit to needle was on Oct 18, 2026.
What can I use needle for?
Voice or text assistants on phones that trigger app actions locally. Smart home and wearable devices that map spoken commands to device functions. Extracting invoices, bookings or forms from messy text on an offline device. Local speech-to-text on an always-on microphone paired with a wake word or voice activity detector.
Where can I get needle?
The source code and installation instructions are on GitHub: https://github.com/cactus-compute/needle

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects