transformers

An unofficial pytorch implementation of 'Efficient Infinite Context Transformers with Infini-attention'

🤖 Python implementations of some of the fundamental Machine Learning models and algorithms from scratch with interactive Jupyter demos and math being explained.

[KDD 2024] "ImputeFormer: Low Rankness-Induced Transformers for Generalizable Spatiotemporal Imputation"

An Education Tutoring Chatbot based on Learning Science Principles powered by Large Language Models

📚 Text Classification with LoRA (Low-Rank Adaptation) of Language Models - Efficiently fine-tune large language models for text classification tasks using the Stanford Sentiment Treebank (SST-2) dataset and the LoRA technique.

Tokenizer POS-Tagger and Dependency-parser with BERT/RoBERTa/DeBERTa/GPT models for Japanese and other languages

several types of attention modules written in PyTorch for learning purposes

Model zoo for different kinds of uncertainty quantification methods used in Natural Language Processing, implemented in PyTorch.

Character-level Korean ELECTRA Model (음절 단위 한국어 ELECTRA)

An open-source pipeline for training natural language understanding models

Recent Advances on Efficient Vision Transformers

A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task, including architectures such as: Siamese LSTM Siamese BiLSTM with Attention Siamese Transformer Siamese BERT.

Implementation of Cross Transformer for spatially-aware few-shot transfer, in Pytorch

My own attempt at a long context genomics model, leveraging recent advances in long context attention modeling (Flash Attention + other hierarchical methods)

Quranic Lexical/Semantic Search

Transformers are Graph Neural Networks!

Implementation of Multistream Transformers in Pytorch

CryptoAgent is a professional, enterprise-grade solution designed to fetch, analyze, and summarize real-time cryptocurrency data.

[NeurIPS 2025] ScaleKV: Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression

An introduction to basic concepts of Transformers and key techniques of their recent advances.