Open-source projects tagged “transformer-alternative”

Mamba is a PyTorch sequence-modeling architecture built on selective state space models, offering a linear-time alternative to Transformers for language and other sequence tasks.

Official training and reference code for RWKV, an RNN-style language model architecture that can also be trained in parallel like a GPT transformer.