multimodal

A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.

Multimodal-GPT

Visual intelligence for your home.

日本語LLMまとめ - Overview of Japanese LLMs

This repository is the official implementation of Disentangling Writer and Character Styles for Handwriting Generation (CVPR 2023)

A curated list of Multimodal Related Research.

Offline inference engine for art, real-time voice conversations, LLM powered chatbots and automated workflows

Curated visual catalog of 155+ vision-language model (VLM/MLLM) architectures: papers, diagrams, training recipes, datasets, and a release timeline for multimodal AI agents.

Give Claude the ability to watch and understand videos — Claude Code plugin with frame extraction and multimodal audio analysis

Pocket-Sized Multimodal AI for content understanding and generation across multilingual texts, images, and 🔜 video, up to 5x faster than OpenAI CLIP and LLaVA 🖼️ & 🖋️

Xtreme1 is an all-in-one data labeling and annotation platform for multimodal data training and supports 3D LiDAR point cloud, image, and LLM.

Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch

Turn documents into AI-ready Markdown with visual understanding

Unifying 3D Mesh Generation with Language Models

The Self-Coding System for Your App — Alan AI SDK for Cordova

WIT (Wikipedia-based Image Text) Dataset is a large multimodal multilingual dataset comprising 37M+ image-text sets with 11M+ unique images across 100+ languages.

MOVA: Towards Scalable and Synchronized Video–Audio Generation

为纯文本模型"看图“设计更好的视觉工具箱和技能，支持多图理解，图片问答，前端UI还原、GUI 自动化等，并可选无缝接入多个主流agent，直接识别粘贴图片｜ A vision toolkit and skill designed for text-only llms — image Q&A, long-screenshot OCR, frontend UI restoration, and GUI automation, with optional seamless integration for Codex, Claude Code, Pi, Oh My Pi, and OpenCode

Codebase for Aria - an Open Multimodal Native MoE

🩺 首个会看胸部X光片的中文多模态医学大模型 | The first Chinese Medical Multimodal Model that Chest Radiographs Summarization.