multimodal

Language Models Can See: Plugging Visual Controls in Text Generation

Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images - ICCV 2021

Official implementation of GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents

EPNet: Enhancing Point Features with Image Semantics for 3D Object Detection(ECCV 2020)

MedEvalKit: A Unified Medical Evaluation Framework

[CVPR 2022 Oral & TPAMI 2023] Learning Optical Flow and Scene Flow with Bidirectional Camera-LiDAR Fusion

OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks

A web UI Project In order to learn the large language model. This project includes features such as chat, quantization, fine-tuning, prompt engineering templates, and multimodality.

A Continuously Updated Library for Advanced Models for Multimodal Recommendation

UniMMAD: Unified Multi-Modal and Multi-Class Anomaly Detection via MoE-Driven Feature Decompression

Pytorch implementation of the models RT-1-X and RT-2-X from the paper: "Open X-Embodiment: Robotic Learning Datasets and RT-X Models"

A comprehensive code domain benchmark review of LLM researches.

The official implementation of 1. Achieving Cross Modal Generalization with Multimodal Unified Representation (NeurIPS '23) 2. Open-set Cross Modal Generalization via Multimodal Unified Representation (ICCV '25) 3. Enhancing Multimodal Unified Representations for Cross Modal Generalization (ACL ‘25 Findings)

Official code for Paper "Mantis: Multi-Image Instruction Tuning" [TMLR 2024 Best Paper]

[CVPR'22 Best Paper Finalist] Official PyTorch implementation of the method presented in "Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentation"

[ICLR 2026] Uni-CoT: Towards Unified Chain-of-Thought Reasoning Across Text and Vision

OmniFusion — a multimodal model to communicate using text and images

💐Kaleido-BERT: Vision-Language Pre-training on Fashion Domain

DeepVerse: 4D Autoregressive Video Generation as a World Model

Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling