Repolode
Submit
cocoindex-io

cocoindex-io/cocoindex

Incremental data pipeline framework in Python with a Rust core that keeps vector indexes and knowledge graphs in sync with changing sources for AI agents and RAG apps.

View on GitHubWebsite
Stars
11,721
Forks
921
Open issues
103
Last commit
yesterday

What is it?

CocoIndex is a framework for building data pipelines that turn sources such as codebases, documents, Slack, meeting notes, PDFs and videos into continuously updated context for AI agents and LLM applications. You declare the desired target state as a Python function over your source data, and the engine tracks what changed and recomputes only the affected rows, files or outputs instead of reprocessing the whole corpus.

The engine caches function results by hashing both inputs and function code, so editing a transformation only re-runs the outputs that depend on it. Each target row can be traced back to its source, which supports debugging and auditing. Targets include relational databases such as Postgres with vector indexes, along with vector stores like LanceDB and graph databases like Neo4j and Kuzu, and the project publishes examples for code embedding, PDF RAG, knowledge graph extraction and Kafka streaming.

The project is in Python with a Rust core, and the documentation covers installation via pip, a quickstart, connectors, transformations and a Claude Code and Cursor skill for generating correct code.

Key features

  • 01Incremental processing that recomputes only changed sources and their dependent outputs.
  • 02Function-level caching keyed on input hashes and code hashes.
  • 03Vector index targets such as Postgres with pgvector and LanceDB.
  • 04Graph database targets including Neo4j and Kuzu.
  • 05Lineage tracing from each target row back to its source.
  • 06Source connectors for local files, S3 and Google Drive, plus a Kafka target.

Use cases

  • Keeping a RAG index over a document corpus fresh without re-embedding unchanged files.
  • Building a live code search index that updates on each commit for coding agents.
  • Extracting entities and relations from meeting transcripts into a knowledge graph.
  • Streaming changed CSV rows into a Kafka topic.

Quick start

  1. 1

    Install CocoIndex from PyPI

    pip install -U cocoindex
  2. 2

    Declare your target and run the incremental update once to backfill

    coco.App(coco.AppConfig(name="docs"), main, src="./docs").update_blocking()

Things to know

  • The quickstart needs a Postgres database and an embedding function, which the example uses but does not set up for you.
  • The project is pre-1.0-style v1 API material with a fast-moving surface, so code may change between releases.

Topics

#rag#incremental-processing#vector-search#llm#etl#knowledge-graph#data-pipeline#ai-agents#agentic-data-framework#ai

FAQ

What is cocoindex?
Incremental data pipeline framework in Python with a Rust core that keeps vector indexes and knowledge graphs in sync with changing sources for AI agents and RAG apps. CocoIndex is a framework for building data pipelines that turn sources such as codebases, documents, Slack, meeting notes, PDFs and videos into continuously updated context for AI agents and LLM applications. You declare the desired target state as a Python function over your source data, and the engine tracks what changed and recomputes only the affected rows, files or outputs instead of reprocessing the whole corpus.
Is cocoindex free to use? What license does it have?
cocoindex is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is cocoindex still maintained?
Actively maintained. The last commit to cocoindex was on Oct 19, 2026.
What can I use cocoindex for?
Keeping a RAG index over a document corpus fresh without re-embedding unchanged files. Building a live code search index that updates on each commit for coding agents. Extracting entities and relations from meeting transcripts into a knowledge graph. Streaming changed CSV rows into a Kafka topic.
How do I install cocoindex?
Install CocoIndex from PyPI: pip install -U cocoindex Declare your target and run the incremental update once to backfill: coco.App(coco.AppConfig(name="docs"), main, src="./docs").update_blocking()
Where can I get cocoindex?
The source code and installation instructions are on GitHub: https://github.com/cocoindex-io/cocoindex

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects