model-parallelism

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Making large AI models cheaper, faster and more accessible

Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*

NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference

Distributed peer-to-peer LLM inference. Your prompt never leaves your device in clear text.