Repolode
Submit
superlinked

superlinked/sie

Self-hosted inference server that serves embedding, reranking, extraction, OCR, guardrail, and LLM models from one cluster in your own cloud.

Open-source alternative to:Hugging Face Inference EndpointsOpenAI APICohere Rerank

View on GitHubWebsite
Stars
3,410
Forks
322
Open issues
31
Last commit
7 hours ago

What is it?

SIE (Superlinked Inference Engine) is an open-source inference server meant to run all the models an agent needs behind a single API. Instead of running a separate model server for each task, it loads any of its 100+ configured models on demand and evicts least-recently-used ones when memory is needed.

The server exposes OpenAI-compatible endpoints for embeddings, chat completions, completions, and responses, and ships a Python SDK and a TypeScript SDK. Supported tasks include semantic search (dense, sparse, and late-interaction embeddings plus reranking), document-to-markdown conversion, zero-shot classification, entity and structured extraction, content safety checks, translation, transcription, and image captioning or detection.

For production, the project includes a load-balancing gateway and Helm charts for Kubernetes, with KEDA autoscaling and Grafana dashboards. Integration guides cover LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDB.

Key features

  • 01OpenAI-compatible embeddings, chat, completions, and responses endpoints
  • 02On-demand loading of many models with LRU eviction on one server
  • 03Kubernetes deployment with Helm charts, KEDA autoscaling, and Grafana dashboards
  • 04Python and TypeScript SDKs with a single client for encoding, scoring, and extraction
  • 05Pre-configured models for search, OCR, classification, guardrails, translation, and transcription

Use cases

  • Running embedding and reranking models for a retrieval-augmented generation pipeline in your own cloud
  • Converting PDFs and scanned documents to markdown before indexing them
  • Extracting structured fields from support tickets or other free text with a schema
  • Adding content safety checks to an LLM application without sending data to a third-party API

Topics

#inference-server#embeddings#semantic-search#reranking#rag#kubernetes#openai-compatible#llm#bge#colbert

FAQ

What is sie?
Self-hosted inference server that serves embedding, reranking, extraction, OCR, guardrail, and LLM models from one cluster in your own cloud. SIE (Superlinked Inference Engine) is an open-source inference server meant to run all the models an agent needs behind a single API. Instead of running a separate model server for each task, it loads any of its 100+ configured models on demand and evicts least-recently-used ones when memory is needed.
Is sie free to use? What license does it have?
sie is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is sie still maintained?
Actively maintained. The last commit to sie was on Oct 18, 2026.
What can I use sie for?
Running embedding and reranking models for a retrieval-augmented generation pipeline in your own cloud Converting PDFs and scanned documents to markdown before indexing them Extracting structured fields from support tickets or other free text with a schema Adding content safety checks to an LLM application without sending data to a third-party API
Where can I get sie?
The source code and installation instructions are on GitHub: https://github.com/superlinked/sie

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects