multimodal

The Official Repo for "Quick Start Guide to Large Language Models"

Visual Med-Alpaca is an open-source, multi-modal foundation model designed specifically for the biomedical domain, built on the LLaMa-7B.

Creates an index of images, queries a local LLM and adds tags to the image metadata

Virtual Sparse Convolution for Multimodal 3D Object Detection

[IEEE TPAMI 2026] Simulating the Real World: Survey & Resources, which contains our survey "Simulating the Real World: A Unified Survey of Multimodal Generative Models" (IEEE TPAMI, 2026) and Awesome-Text2X-Resources. Watch this repository for the latest updates! 🔥

Code for the paper "LLark: A Multimodal Instruction-Following Language Model for Music" by Josh Gardner, Simon Durand, Daniel Stoller, and Rachel Bittner.

Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.

LLaVA-Interactive-Demo

PyTorch Implementation for Paper "Emotionally Enhanced Talking Face Generation" (ICCVW'23 and ACM-MMW'23)

🏭 Mega Scale Multimodal DataPipeline for SOTA Foundation Models

A native .NET LLM inference engine for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/Linux with full GPU capability

Joint speech-language model - respond directly to audio!

A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, llava-onevision, llama-3.2-vision, qwen-vl, qwen2-vl, phi3-v etc.

多模态情感分析——基于BERT+ResNet的多种融合方法

Paper List of Pre-trained Foundation Recommender Models

Open-source data management for multimodal AI. Query, trace, and govern with a lineage-native, format-agnostic lakehouse for agents and teams. With support for biological formats and registries by the creators of Scanpy. 🍊YC S22

An open source implementation of "Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning", an all-new multi modal AI that uses just a decoder to generate both text and images

Scientific Large Language Models: A Survey on Biological & Chemical Domains

Research Trends in LLM-guided Multimodal Learning.

A customizable pipeline for multimodal data extraction from MIMIC-IV!