Repolode
Submit
explosion

explosion/spacy-transformers

spaCy components and architectures that plug Hugging Face transformer models such as BERT, RoBERTa and XLNet into spaCy pipelines.

View on GitHubWebsite
Stars
1,411
Forks
177
Open issues
3
Last commit
3 days ago

What is it?

spacy-transformers provides spaCy components and model architectures that use transformer models from Hugging Face's transformers library inside spaCy. Once installed, a pipeline can build on pretrained models such as BERT, RoBERTa, XLNet and GPT-2 instead of training every component from scratch.

The package aligns transformer output with spaCy's tokenization, so downstream components like NER or text classification receive features at the token level. A single transformer can feed several pipeline components, and gradients from those components can update the shared model during training. Training uses spaCy v3's config system, and users can control which transformer data is stored on the Doc object and how long documents are split for processing.

The current release requires spaCy v3, PyTorch 1.5 or later and Python 3.6 or later. Task-specific heads such as token or text classification are not provided by this component; for those, the project points to a separate spacy-huggingface-pipelines package.

Key features

  • 01Uses pretrained transformers such as BERT, RoBERTa and XLNet within a spaCy pipeline.
  • 02Shares one transformer across several pipeline components for multi-task learning.
  • 03Aligns transformer outputs to spaCy's tokenization automatically.
  • 04Lets you configure which transformer data is stored on the Doc object.
  • 05Supports training through spaCy v3's config system and includes serialization and packaging.

Use cases

  • Training a spaCy NER or text classifier that uses transformer features for higher accuracy.
  • Sharing one transformer backbone across several spaCy components in a single pipeline.
  • Packaging a custom transformer-based spaCy model for deployment.

Quick start

  1. 1

    Install the package with pip, which pulls in spaCy and PyTorch

    pip install 'spacy[transformers]'
  2. 2

    For GPU support, check your CUDA version with nvcc and add the matching bracket option (see the spaCy GPU docs)

  3. 3

    If PyTorch fails to install, follow the official PyTorch instructions for your system

Things to know

  • Requires spaCy v3 and Python 3.6+ with PyTorch v1.5+.
  • The transformer component does not provide task-specific heads for classification.

In collections

Natural language processing tools & libraries

Topics

#nlp#transformers#spacy#pytorch#bert#language-model#transfer-learning#google#gpt-2#huggingface

FAQ

What is spacy-transformers?
spaCy components and architectures that plug Hugging Face transformer models such as BERT, RoBERTa and XLNet into spaCy pipelines. spacy-transformers provides spaCy components and model architectures that use transformer models from Hugging Face's transformers library inside spaCy. Once installed, a pipeline can build on pretrained models such as BERT, RoBERTa, XLNet and GPT-2 instead of training every component from scratch.
Is spacy-transformers free to use? What license does it have?
spacy-transformers is open source under the MIT license. Check the license terms before using it in commercial products.
Is spacy-transformers still maintained?
Actively maintained. The last commit to spacy-transformers was on Nov 3, 2026.
What can I use spacy-transformers for?
Training a spaCy NER or text classifier that uses transformer features for higher accuracy. Sharing one transformer backbone across several spaCy components in a single pipeline. Packaging a custom transformer-based spaCy model for deployment.
How do I install spacy-transformers?
Install the package with pip, which pulls in spaCy and PyTorch: pip install 'spacy[transformers]' For GPU support, check your CUDA version with nvcc and add the matching bracket option (see the spaCy GPU docs) If PyTorch fails to install, follow the official PyTorch instructions for your system
Where can I get spacy-transformers?
The source code and installation instructions are on GitHub: https://github.com/explosion/spacy-transformers

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects