google-research/text-to-text-transfer-transformer

What is it?

Key features

• 01Defines NLP tasks as text-to-text data via Task, TfdsTask and TextLineTask.
• 02Mixes multiple tasks into Mixtures for multi-task training with configurable rates.
• 03Provides command-line tools for pre-training, fine-tuning, evaluation and decoding.
• 04Runs on Google Cloud TPUs with Mesh TensorFlow, with GPU instructions included.
• 05Releases pre-trained checkpoints from T5-Small (60M) to T5-11B (11B parameters).

Use cases

• Reproducing the experiments from the T5 paper on transfer learning.
• Fine-tuning released T5 checkpoints on a custom text-to-text dataset.
• Building custom multi-task NLP training pipelines with mixed datasets.
• Studying pre-training objectives and dataset preprocessing for sequence-to-sequence models.

Quick start

1. 1

   Install the T5 package

   pip install t5[gcp]
2. 2

   Try T5 in the free Colab tutorial (TPU, no local setup)

3. 3

   List the available tasks and mixtures

   python -c "import t5; print(t5.data.MixtureRegistry.names())"

Things to know

• !The README says T5 on TensorFlow with MeshTF is no longer actively developed and recommends T5X instead.
• !Training and fine-tuning are built around Google Cloud TPUs and GCP storage.
• !Preparing the C4 dataset needs about 7 TB of downloads and roughly 335 CPU-days of compute.

In collections

Topics

FAQ

What is text-to-text-transfer-transformer?

Google Research's T5 codebase for reproducing the 'Text-to-Text Transfer Transformer' experiments, covering data preprocessing, training, fine-tuning and evaluation on TPUs. T5 is the codebase accompanying the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer". It frames every NLP task as text in, text out, and the library is organized mainly around loading, preprocessing, mixing and evaluating datasets. Users define Tasks from TensorFlow Datasets, text files or custom sources, attach preprocessors, SentencePiece tokenization and metrics, and combine them into Mixtures for multi-task training.

Is text-to-text-transfer-transformer free to use? What license does it have?

text-to-text-transfer-transformer is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.

Is text-to-text-transfer-transformer still maintained?

Actively maintained. The last commit to text-to-text-transfer-transformer was on Oct 12, 2026.

What can I use text-to-text-transfer-transformer for?

Reproducing the experiments from the T5 paper on transfer learning. Fine-tuning released T5 checkpoints on a custom text-to-text dataset. Building custom multi-task NLP training pipelines with mixed datasets. Studying pre-training objectives and dataset preprocessing for sequence-to-sequence models.

How do I install text-to-text-transfer-transformer?

Install the T5 package: pip install t5[gcp] Try T5 in the free Colab tutorial (TPU, no local setup) List the available tasks and mixtures: python -c "import t5; print(t5.data.MixtureRegistry.names())"

Where can I get text-to-text-transfer-transformer?

The source code and installation instructions are on GitHub: https://github.com/google-research/text-to-text-transfer-transformer