Repolode
Предложить
cactus-compute

cactus-compute/needle

A tiny on-device foundation model for tool calling, structured extraction, embeddings and speech-to-text on phones, wearables, robots and microcontrollers.

Открыть на GitHubСайт
Звёзды
14,099
Форки
1,091
Открытые issues
4
Последний коммит
13 часов назад

Что это?

Needle is a small foundation model for automation on constrained hardware. The full model is an 8 to 29 MB binary, and it is built to pick the right functions from a list you expose, fill their arguments from what a user said, and return an empty list instead of guessing when no tool fits. It also extracts typed fields from messy text against a schema you declare, and returns a vector for a sentence so an app can search or route locally.

The Python package exposes a decorator-based API: a function's signature defines argument types, its docstring becomes the tool description, and a run call executes the matched function and returns its results. Each response includes the function calls, reasoning and a confidence score. Whistle, a companion speech-to-text model in a 16.9 MB file, shares the same runtime, so audio can go in and tool calls come out in one call. Whistle transcribes English, German, French, Spanish, Italian, Dutch and Polish.

Needle supports local fine-tuning with LoRA adapters, plus a hosted fine-tuning path, and ships prebuilt engines for multiple platform folders, a C API, browser and WASI targets. A community-maintained npm package, cactus-needle, runs it in Node.js and browsers.

Ключевые возможности

  • 01Selects tool calls and fills their arguments from user requests, returning an empty list when no tool applies.
  • 02Extracts typed fields from text according to a declared schema, with output guaranteed to parse.
  • 03Returns text embeddings for local search, matching and routing.
  • 04Transcribes speech with Whistle, including word timestamps, keyword biasing and live streaming.
  • 05Provides a Python decorator API that runs matched functions and returns their results.
  • 06Supports local LoRA fine-tuning and hosted fine-tuning across model depths.

Сценарии использования

  • Voice or text assistants on phones that trigger app actions locally.
  • Smart home and wearable devices that map spoken commands to device functions.
  • Extracting invoices, bookings or forms from messy text on an offline device.
  • Local speech-to-text on an always-on microphone paired with a wake word or voice activity detector.

Темы

#edge-ai#on-device-ai#tool-calling#function-calling#tinyml#speech-to-text#embeddings#cactus#foundation-model#llm

Частые вопросы

Что такое needle?
A tiny on-device foundation model for tool calling, structured extraction, embeddings and speech-to-text on phones, wearables, robots and microcontrollers. Needle is a small foundation model for automation on constrained hardware. The full model is an 8 to 29 MB binary, and it is built to pick the right functions from a list you expose, fill their arguments from what a user said, and return an empty list instead of guessing when no tool fits. It also extracts typed fields from messy text against a schema you declare, and returns a vector for a sentence so an app can search or route locally.
needle можно использовать бесплатно? Какая у него лицензия?
needle распространяется с открытым исходным кодом под лицензией Apache-2.0. Перед использованием в коммерческих продуктах проверьте условия лицензии.
needle всё ещё поддерживается?
Активно поддерживается. Последний коммит в needle — 18 окт. 2026 г..
Для чего можно использовать needle?
Voice or text assistants on phones that trigger app actions locally. Smart home and wearable devices that map spoken commands to device functions. Extracting invoices, bookings or forms from messy text on an offline device. Local speech-to-text on an always-on microphone paired with a wake word or voice activity detector.
Где скачать needle?
Исходный код и инструкции по установке — на GitHub: https://github.com/cactus-compute/needle

Описание создано ИИ на основе README проекта. Статистика GitHub обновляется ежедневно.

Похожие проекты