Repolode
Submit
MihaiValentin

MihaiValentin/lunr-languages

Stemmers and stopword lists for Lunr.js that add multilingual full-text search across dozens of languages in the browser or Node.js.

View on GitHub
Stars
459
Forks
168
Open issues
16
Last commit
15 hours ago

What is it?

Lunr Languages is an extension for Lunr.js, a client-side full-text search library. It adds language-specific tokenization, stemming and stopword filtering so that indexes built with Lunr can match word forms correctly across a wide range of languages, including German, French, Spanish, Russian, Japanese, Arabic and Hindi.

Each language is shipped as a separate module that registers with Lunr, and a multi-language mode lets one index combine several languages. Chinese tokenization uses Intl.Segmenter with CJK bigrams by default, and can switch to @node-rs/jieba in Node.js for Jieba segmentation. Type declarations are included, and the modules can be imported with CommonJS or ES module syntax.

Because it runs entirely in the browser or Node.js, it suits static site and documentation search, offline apps, and keyword pre-filtering before vector search or a reranker in retrieval pipelines.

Key features

  • 01Language-specific tokenization, stemming and stopword lists for many languages
  • 02Multi-language mode to index content in several languages in one index
  • 03Chinese segmentation via Intl.Segmenter, with optional Jieba support in Node.js
  • 04Optional accent-folding pipeline function for Spanish indexes
  • 05TypeScript type declarations and both CommonJS and ES module imports

Use cases

  • Adding client-side search to a multilingual static website or documentation site
  • Building an offline or on-device knowledge base search in the browser
  • Pre-filtering documents by keyword before embedding or reranking in a RAG pipeline

Quick start

  1. 1

    Install the package from npm

    npm install lunr-languages
  2. 2

    Load Lunr and the German language module in Node.js

    const lunr = require('lunr'); require('lunr-languages/lunr.stemmer.support')(lunr); require('lunr-languages/lunr.de')(lunr);
  3. 3

    Create an index with the German pipeline and add documents

Things to know

  • Chinese tokenization without the optional package uses bigrams, which is less precise than Jieba.
  • Chinese support in browsers requires Intl.Segmenter and has no fallback.

In collections

Natural language processing tools & libraries

Topics

#search#full-text-search#stemming#multilingual#lunr#nlp#javascript#i18n#language-stemmer#localization

FAQ

What is lunr-languages?
Stemmers and stopword lists for Lunr.js that add multilingual full-text search across dozens of languages in the browser or Node.js. Lunr Languages is an extension for Lunr.js, a client-side full-text search library. It adds language-specific tokenization, stemming and stopword filtering so that indexes built with Lunr can match word forms correctly across a wide range of languages, including German, French, Spanish, Russian, Japanese, Arabic and Hindi.
Is lunr-languages free to use? What license does it have?
lunr-languages is open source under the Other license. Check the license terms before using it in commercial products.
Is lunr-languages still maintained?
Actively maintained. The last commit to lunr-languages was on Nov 6, 2026.
What can I use lunr-languages for?
Adding client-side search to a multilingual static website or documentation site Building an offline or on-device knowledge base search in the browser Pre-filtering documents by keyword before embedding or reranking in a RAG pipeline
How do I install lunr-languages?
Install the package from npm: npm install lunr-languages Load Lunr and the German language module in Node.js: const lunr = require('lunr'); require('lunr-languages/lunr.stemmer.support')(lunr); require('lunr-languages/lunr.de')(lunr); Create an index with the German pipeline and add documents
Where can I get lunr-languages?
The source code and installation instructions are on GitHub: https://github.com/MihaiValentin/lunr-languages

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects