vision-language

[ECCV 2024] Official implementation of the paper "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection"

Chinese version of CLIP which achieves Chinese cross-modal retrieval and representation generation.

Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

An open-source implementaion for fine-tuning Qwen-VL series by Alibaba Cloud.

A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.

日本語LLMまとめ - Overview of Japanese LLMs

[ECCV 2024 Oral] DriveLM: Driving with Graph Visual Question Answering

A general representation model across vision, audio, language modalities. Paper: ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities

A Framework of Small-scale Large Multimodal Models

Official implementation of SEED-LLaMA (ICLR 2024).

多模态中文LLaMA&Alpaca大语言模型（VisualCLA）

[IEEE TPAMI 2026] Simulating the Real World: Survey & Resources, which contains our survey "Simulating the Real World: A Unified Survey of Multimodal Generative Models" (IEEE TPAMI, 2026) and Awesome-Text2X-Resources. Watch this repository for the latest updates! 🔥

A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, llava-onevision, llama-3.2-vision, qwen-vl, qwen2-vl, phi3-v etc.

Experiments and data for the paper "When and why vision-language models behave like bags-of-words, and what to do about it?" Oral @ ICLR 2023

Official Repository of paper VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding

💐Kaleido-BERT: Vision-Language Pre-training on Fashion Domain

Florence-2 is a novel vision foundation model with a unified, prompt-based representation for a variety of computer vision and vision-language tasks.

MixGen: A New Multi-Modal Data Augmentation

[ECCV 2022] Official Pytorch Implementation of the paper : " Zero-Shot Temporal Action Detection via Vision-Language Prompting "

Awesome Multimodal Assistant is a curated list of multimodal chatbots/conversational assistants that utilize various modes of interaction, such as text, speech, images, and videos, to provide a seamless and versatile user experience.