multimodal

[CVPR 2024 CVinW] Multi-Agent VQA: Exploring Multi-Agent Foundation Models on Zero-Shot Visual Question Answering

LipNet with gluon

Dataset and code implementation for the paper "Decoding the Underlying Meaning of Multimodal Hateful Memes" (IJCAI'23).

MultiModal Sentiment Analysis (Text and Audio) (Pytorch)

[IROS'24 Oral] Dataset of Moving Object Segmentation; Official Implementation of "Event-Free Moving Object Segmentation from Moving Ego Vehicle"

Google's Conceptual Captions Dataset translated into Korean

An open-source implementaion for fine-tuning Pixtral by MistralAI.

[NeurIPS 2025] TL;DR: Aligning pretrained unimodal models with the proposed framework using limited paired data yields ~52% gains in cross-modality zero-shot classification and ~92% in retrieval.

[ICLR 2026] MergeMix: A Unified Augmentation Paradigm for Visual and Multi-Modal Understanding

Automatically describe images sent by users on popular media platforms, incredibly useful for the visually impaired and for complicated imagery.

The code repository for "MB2C: Multimodal Bidirectional Cycle Consistency for Learning Robust Visual Neural Representations" in PyTorch

Official Repository for NeurIPS'25 Paper "Tool-Augmented Spatiotemporal Reasoning for Streamlining Video Question Answering Task"

LLMs based multi-model framework for building AI apps.

XVO: Generalized Visual Odometry via Cross-Modal Self-Training [ICCV 2023]

Utils and modules for Speech Language and Multimodal processing using pytorch and pytorch lightning

collection of mass spectrometry imaging workflows

[NeurIPS 2025] Official repository of the paper "Unlocking Aha Moments via Reinforcement Learning: Advancing Collaborative Visual Comprehension and Generation"

Yet another Ph.D. adventure.

[ICIP 2025] Official implementation of RT-X Net: RGB-Thermal cross attention network for Low-Light Image Enhancement

We introduce new approach, Token Reduction using CLIP Metric (TRIM), aimed at improving the efficiency of MLLMs without sacrificing their performance.