multimodal

An Interactive Game-based Vision Planning benchmark

OMGM: Orchestrate Multiple Granularities and Modalities for Efficient Multimodal Retrieval (ACL 2025 Main Conference)

[ICRA26] OmniVLA: Physically-Grounded Multimodal VLA with Unified Multi-Sensor Perception for Robotic Manipulation

🧠🔥 Adaptive and inclusive neurotech platform for real-time multimodal data — born from the Olympic flame project.

This repository provides the code for a multimodal realistic simulation framework using CARLA and MATLAB.

[NLPCC'23] ZeroGen: Zero-shot Multimodal Controllable Text Generation with Multiple Oracles PyTorch Implementation

A 5-way embedding model for text, audio, image, video, and 3D point clouds.

Code for paper "Modality Plug-and-Play: Elastic Modality Adaptation in Multimodal LLMs for Embodied AI"

ThingiPano: A Large-Scale Dataset of 3D Printing Metadata, Images, and Panoramic Renderings for Exploring Design Reuse

AI-powered disaster response platform with offline-first architecture using Gemma 3n. Provides computer vision hazard detection, voice analysis with emergency keywords, PDF report generation, and multi-user coordination - all working without internet access.

A dataset for multimodal machine translation

Multimodal Argument Mining Toolkit

The repository for the exclusive Daily Paper Club hosted at Agora every 10pm NYC time at this discord: https://discord.gg/Gnzh6dnzyz

Engage in a semantic segmentation challenge for land cover description using multimodal remote sensing earth observation data, delving into real-world scenarios with a dataset comprising 70,000+ aerial imagery patches and 50,000 Sentinel-2 satellite acquisitions.

My implementation of the model KosmosG from "KOSMOS-G: Generating Images in Context with Multimodal Large Language Models"

This repository provides the official resources for EMNLP 2025 Paper Grounding Multilingual Multimodal LLMs With Cultural Knowledge

Capability-aware vision + paste extension for the pi coding agent. Delegates image analysis only when the active model is text-only; passes through natively for multimodal models (zero delegation).

FeelingBlue: A Corpus for Understanding the Emotional Connotation of Color in Context, accepted at TACL 2022, presented at ACL 2023

MCP server that gives text-only LLM coding agents vision — analyze images via any multimodal model (mimo, Claude, Gemini, OpenAI-compatible). Works with Claude Code, Codex, Kimi, opencode, PI.

Pytorch implementation of Multimodal Neural Machine Translation(MNMT).