attention-mechanism

Sinkhorn Transformer - Practical implementation of Sparse Sinkhorn Attention

A Deep-Learning-Based Persian Speech Recognition System

[MedIA Journal] An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites

PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"

Build a Large Language Model (From Scratch) book and Finetuned Models

A Python package housing a collection of deep-learning multi-modal data fusion method pipelines! From data loading, to training, to evaluation - fusilli's got you covered 🌸

Curso práctico: NLP de cero a cien 🤗

Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways - in Jax (Equinox framework)

Transformer architecture explained step by step - the full architecture, every attention variant, positional embeddings, and every layer inside a Transformer.

Implementation of Fast Transformer in Pytorch

Implementation of TransGanFormer, an all-attention GAN that combines the finding from the recent GanFormer and TransGan paper

Implementation of the new SOTA for model based RL, from the paper "Improving Transformer World Models for Data-Efficient RL", in Pytorch

transformers go brrr...

Learn Generative AI with PyTorch (Manning Publications, 2024)

[CVPR'22] HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpening

A minimal implementation of a denoising diffusion model in PyTorch.

Implementation of STAM (Space Time Attention Model), a pure and simple attention model that reaches SOTA for video classification

Implementation of RQ Transformer, proposed in the paper "Autoregressive Image Generation using Residual Quantization"

Implementation of Memformer, a Memory-augmented Transformer, in Pytorch

Implementation of Mimic-Video, Video-Action Models for SOTA Generalizable Robot Control Beyond VLAs