attention

Implementation of a framework for Genie2 in Pytorch

Visual knowledge bank for understanding large language models, with 180 concept cards from tokenization to deployment.

SciFive: a text-text transformer model for biomedical literature

🚀 LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training

Implementation of Perceiver, General Perception with Iterative Attention

First open-source implementation of Google TurboQuant (ICLR 2026) -- near-optimal KV cache compression for LLM inference. 5x compression with near-zero quality loss.

My implementation of Kosmos2.5 from the paper: "KOSMOS-2.5: A Multimodal Literate Model"

A deep learning library based on Pytorch focussed on low resource language research and robustness

Vision Transformers for image classification, image segmentation, and object detection.

Implementation of the transformer from the paper: "Real-World Humanoid Locomotion with Reinforcement Learning"

Code for the paper "On the Expressivity Role of LayerNorm in Transformers' Attention" (Findings of ACL'2023)

several types of attention modules written in PyTorch for learning purposes

Flexible Python library providing building blocks (layers) for reproducible Transformers research (Tensorflow ✅, Pytorch 🔜, and Jax 🔜)

End-to-end pipeline for seeing how LLMs actually process your prompts. Capture attention across every layer, render heatmaps and cooking curves, compare variants with evidence — not vibes.

Official implementation of the paper: Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers

The notebook explains the various steps to obtain the results of publication: "Is Space-Time Attention All You Need for Video Understanding?"

An open source community implementation of the model from "DIFFERENTIAL TRANSFORMER" paper by Microsoft.

A simple and intuitive library for diffusion models using Flax and Jax. Includes detailed notebooks on DDPM, DDIM, and EDM with simplified mathematical explanations. Made as part of my journey for learning and experimenting with generative AI.

A clean, single-file PyTorch implementation of Attention Residuals (Kimi Team, MoonshotAI, 2026), integrated with Grouped Query Attention (GQA), SwiGLU feed-forward networks, and Rotary Position Embeddings (RoPE).

The open source implementation of the cross attention mechanism from the paper: "JOINTLY TRAINING LARGE AUTOREGRESSIVE MULTIMODAL MODELS"