jax

Official implementation of 'Transformer-VQ: Linear-Time Transformers via Vector Quantization' (ICLR 2024)

Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways - in Jax (Equinox framework)

Notebooks for the Practicals at the Deep Learning Indaba 2022.

Unofficial JAX implementations of deep learning research papers

A multi-modal CLIP model trained on the medical dataset ROCO

Universal Python SDK to run AI workloads on Kubernetes

Contains code for the paper "Vision Transformers are Robust Learners" (AAAI 2022).

Multimodal Masked Autoencoders (M3AE): A JAX/Flax Implementation

This is the official repository for the paper "Flora: Low-Rank Adapters Are Secretly Gradient Compressors" in ICML 2024.

SciFive: a text-text transformer model for biomedical literature

A JAX Research Toolkit for Visualizing, Manipulating, and Understanding Gemma Models with Multi-modal Support based on Penzai.

Chef Transformer 🍲 .

NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference

TrAVis: Visualise BERT attention in your browser

Train GEMMA on TPU/GPU! (Codebase for training Gemma-Ko Series)

Serialize JAX, Flax, Haiku, or Objax model params with 🤗`safetensors`

A simple and intuitive library for diffusion models using Flax and Jax. Includes detailed notebooks on DDPM, DDIM, and EDM with simplified mathematical explanations. Made as part of my journey for learning and experimenting with generative AI.

RoBERTa Marathi Language model trained from scratch during huggingface 🤗 x flax community week

Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.

Semantic alignment of astronomical data with natural language using multi-modal models. (Jax) Code associated with https://arxiv.org/abs/2403.08851 (COLM 2023).