Repolode
Submit
valentinfrlch

valentinfrlch/ha-llmvision

A Home Assistant integration that uses multimodal LLMs to analyze camera images, videos, live feeds and Frigate events.

View on GitHubWebsite
Stars
1,523
Forks
154
Open issues
51
Last commit
2 months ago

What is it?

LLM Vision adds AI-based visual analysis to Home Assistant. It sends snapshots, video files, live camera feeds or Frigate events to a multimodal large language model, then answers questions or returns descriptions based on a prompt you write.

The integration works with a range of providers, including OpenRouter, OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure, Groq, Ollama, Open WebUI, LocalAI, and any endpoint compatible with the OpenAI API. Analyzed events are stored in a timeline that can be shown on a dashboard through an optional Timeline Card or queried through Assist. Sensors can also be updated from data extracted from camera streams, images or videos.

The project is distributed through HACS and includes a blueprint for camera notifications that summarize events with AI.

Key features

  • 01Supports many AI providers, including cloud APIs and local servers such as Ollama and LocalAI.
  • 02Answers prompts about images, video files, live camera feeds and Frigate events.
  • 03Remembers people, pets and objects.
  • 04Keeps a timeline of camera events for dashboards and Assist queries.
  • 05Updates Home Assistant sensors from data extracted from camera content.

Use cases

  • Getting AI-written summaries of camera motion or Frigate detections as phone notifications.
  • Asking natural-language questions about what a camera currently sees.
  • Showing a searchable history of analyzed camera events on a Home Assistant dashboard.
  • Running vision models locally with Ollama or LocalAI to keep camera footage on your own network.

Quick start

  1. 1

    Install LLM Vision from HACS

  2. 2

    Restart Home Assistant

  3. 3

    Search for LLM Vision in Settings > Devices & services and submit with default settings

  4. 4

    Add an AI provider entry from the LLM Vision integration page

Things to know

  • Requires an API key or a self-hosted multimodal LLM provider to analyze anything.
  • Runs only inside Home Assistant, so you need an existing Home Assistant setup.
  • Setup needs a media folder mounted at /media when running Home Assistant Container.

In collections

Computer vision libraries & tools

Topics

#home-assistant#smart-home#llm#computer-vision#multimodal#camera#notifications#ai#cctv-detection#hacs-integration

FAQ

What is ha-llmvision?
A Home Assistant integration that uses multimodal LLMs to analyze camera images, videos, live feeds and Frigate events. LLM Vision adds AI-based visual analysis to Home Assistant. It sends snapshots, video files, live camera feeds or Frigate events to a multimodal large language model, then answers questions or returns descriptions based on a prompt you write.
Is ha-llmvision free to use? What license does it have?
ha-llmvision is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is ha-llmvision still maintained?
Updated within 6 months. The last commit to ha-llmvision was on Sep 17, 2026.
What can I use ha-llmvision for?
Getting AI-written summaries of camera motion or Frigate detections as phone notifications. Asking natural-language questions about what a camera currently sees. Showing a searchable history of analyzed camera events on a Home Assistant dashboard. Running vision models locally with Ollama or LocalAI to keep camera footage on your own network.
How do I install ha-llmvision?
Install LLM Vision from HACS Restart Home Assistant Search for LLM Vision in Settings > Devices & services and submit with default settings Add an AI provider entry from the LLM Vision integration page
Where can I get ha-llmvision?
The source code and installation instructions are on GitHub: https://github.com/valentinfrlch/ha-llmvision

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects