multimodal

Awesome-HCI （Ubiquitous, LLM, MLLM, Agent, RAG, Embodied-AI, RLHF)

Code for Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense? [COLM 2024]

[NAACL 2025] Beyond End-to-End VLMs: Leveraging Intermediate Text Representations for Superior Flowchart Understanding

ZimaOS Blue - A Local-First Agent Runtime for Bold Builders. Out-of-the-Box, Open-Source, Universal, Vendor-Neutral

Self-hosted AI knowledge base with hybrid semantic search (pgvector + FTS + RRF), MCP server, multi-provider LLM inference (Ollama, OpenAI, OpenRouter, llama.cpp), multimodal ingestion (vision, audio transcription, speaker diarization), and knowledge graph. Rust + PostgreSQL.

VL-JEPA inspired pipeline — compress images/text locally via Ollama, send compact payloads to any LLM API. Cut token costs by ~80%.

OpenAI-compatible reverse proxy for free multimodal AI APIs (chat / images / videos / audio / 3d)

[ACL 2023] Delving into the Openness of CLIP

The Multimodal Sarcasm Detection System detects sarcasm in multimedia content using image-caption generation and NLP. It achieved 1st place in UITC2024 by classifying sarcasm into four categories: image, text, multi sarcasm, and not sarcasm.

One Mac process. Many models. Real speed. Multi-model LLM serving with prefix reuse, MTP acceleration, and OpenAI APIs — built for Apple Silicon, measured against mlx-lm and llama.cpp.

🚁 Can Vision-Language Models Think from the Sky? UAVReason for Aerial Reasoning and Generation

[ICRA 2023] Official implementation of the paper "ViPFormer: Efficient Vision-and-Pointcloud Transformer for Unsupervised Pointcloud Understanding"

Fast dataset format and loader

Implementation for the different ML tasks on Kaggle platform with GPUs.

[ICLR 2026] - Spectral Concept Selection and Cross-modal Representation Learning for Generalized Category Discovery

Multimodal summarization of user-generated videos from wearable cameras

Telephony Server is a powerful bridge that connects telephony providers (Twilio, Vonage, Plivo, etc.) with real-time communication platforms (LiveKit, Jay.so, Pipecat, etc.). It enables seamless call routing, robust metrics collection, and observability features for enhanced telephony operations.

【NeurIPS 2024】The implementation of LIVE: Learnable In-Context Vector for Visual Question Answering https://arxiv.org/abs/2406.13185

A question-conditioned, reasoning-aware image editor designed to serve as a decoupled visual reasoning assistant for Multimodal Large Language Models (MLLMs).

Android 16 fork. AI as a platform primitive. Twelve capabilities, one shared runtime, every app. OEM-pluggable. Apache 2.0.