tensorrt

一款简单易用和高性能的AI部署框架 | An Easy-to-Use and High-Performance AI Deployment Framework

🚀🚀🚀 A collection of some awesome public YOLO object detection series projects and the related object detection datasets.

Stable Diffusion in NCNN with c++, supported txt2img and img2img

TurboOCR, >200 img/s OmnidocBench. TensorRT FP16, PP-OCRv6, HTTP + gRPC

从 NLP 到 LLM 的算法全栈教程，在线阅读地址：https://datawhalechina.github.io/base-llm/

Stable diffusion webui based on diffusers.

An Optimized Speech-to-Text Pipeline for the Whisper Model Supporting Multiple Inference Engine

Real-time inference for Stable Diffusion - 0.88s latency. Covers AITemplate, nvFuser, TensorRT, FlashAttention. Join our Discord communty: https://discord.com/invite/TgHXuSJEk6

A project that optimizes OWL-ViT for real-time inference with NVIDIA TensorRT.

Easy and fast 2d human and animal multi pose estimation using SOTA ViTPose [Y. Xu et al., 2022] Real-time performances and multiple skeletons supported.

2-4x faster ComfyUI Image Upscaling using Tensorrt ⚡

Beginner's Guide to reComputer Jetson

Deep Learning Deployment Framework: Supports tf/torch/trt/trtllm/vllm and other NN frameworks. Support dynamic batching, and streaming modes. It is dual-language compatible with Python and C++, offering scalability, extensibility, and high performance. It helps users quickly deploy models and provide services through HTTP/RPC interfaces.

ComfyUI Depth Anything (v1/v2/v3/distill-any-depth) Tensorrt Custom Node (up to 14x faster) ⚡

Deploy stable diffusion model with onnx/tenorrt + tritonserver

attempt to use TensorRT with ComfyUI

stable diffusion, controlnet, tensorrt, accelerate

GPU 性能与 AI Infra 学习项目：CUDA/Triton 算子、NCU/NSYS、vLLM/SGLang/TRT-LLM/ms-swift、PyTorch/DeepSpeed/ms-swift 训练、并行架构

Whisper in TensorRT-LLM