transformers

An open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data. Supports NLP, pattern matching, and customizable pipelines.

Open source real-time translation app for Android that runs locally

An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

A straightforward method for training your LLM, from downloading data to generating text.

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.

SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer

[NeurIPS 2024 Best Paper Award][GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-simple, user-friendly yet state-of-the-art* codebase for autoregressive image generation!

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

BertViz: Visualize Attention in Transformer Models

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

Leveraging BERT and c-TF-IDF to create easily interpretable topics.

A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX framework, providing efficient speech analysis on Apple Silicon.

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

SD.Next: All-in-one WebUI for AI generative image and video creation, captioning and processing

This repository is a curated collection of links to various courses and resources about Artificial Intelligence (AI)

Open-source offline translation library written in Python

On-device Speech AI for Apple Silicon

Chinese version of CLIP which achieves Chinese cross-modal retrieval and representation generation.

A concise but complete full-attention transformer with a set of promising experimental features from various papers