gpu

Pytorch domain library for recommendation systems

Deep Learning Server and CLI for Torch and TensorRT

2-2000x faster ML algos, 50% less memory usage, works on all hardware - new and old.

OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

Vendor-agnostic orchestration for training, inference and agentic workloads across NVIDIA, AMD, TPU, and Tenstorrent on clouds, Kubernetes, and bare metal.

Distributed AI Model Training and LLM Fine-Tuning on Kubernetes

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

BlazingSQL is a lightweight, GPU accelerated, SQL engine for Python. Built on RAPIDS cuDF.

Deep learning in Rust, with shape checked tensors and neural networks

Ultrafast serverless GPU inference, sandboxes, and background jobs

Deep learning for recommender systems

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

🔥 Real-time NVIDIA GPU dashboard

High-level Deep Learning Framework written in Kotlin and inspired by Keras

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

Examples of programs built using Modal

Sudoless Apple Silicon system monitor (native SwiftUI GUI) with ANE / Media Engine / memory-bandwidth tracking

Estimate whether a Hugging Face model fits and fine-tunes on your local GPU.

A terminal workbench in pure Rust: shells, persistent sessions, SSH, coding agents. GPU-rendered on Zed's gpui, VT core from Alacritty.