multi-modal

The TypeScript library for building AI applications.

Parsing-free RAG supported by VLMs

Use late-interaction multi-modal models such as ColPali in just a few lines of code.

[CVPR 2026🔥] 🧑‍🎨 OmniLottie, an open-sourced multi-modal instructed vector animation generator that produces Lottie JSONs.

Multi-modal OCR pipeline optimized for ML training (text, figure, math, tables, diagrams)

Multi-model DAG-driven parallel AI film generation — parallel speedup scales with scene independence; Generate film scenes simultaneously instead of one by one; "把影视生成的执行图从拓扑序变成关键路径最优调度" ; 唯一把场景叙事依赖建模为 DAG、以 CPM 算法驱动并行调度的影视生成引擎

Chat2Graph: Graph Native Agentic System.

A streaming multimodal database for Edge AI, and Edge Computing.

[NeurIPS 2024] MeshXL: Neural Coordinate Field for Generative 3D Foundation Models, a 3D fundamental model for mesh generation

[MIR-2023-Survey] A continuously updated paper list for multi-modal pre-trained big models

[CVPR2020] Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation

[ICCV2025] Referring any person or objects given a natural language description. Code base for RexSeek and HumanRef Benchmark

Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows

😎 curated list of awesome LMM hallucinations papers, methods & resources.

A large-scale multi-modal pre-trained model

152 open-source tools to run LLMs 100% locally – no cloud, no API keys, no censorship

A Python framework for multi-modal document understanding with Amazon Bedrock

[InterSpeech 2023] The official PyTorch implementation of: "AudioToken: Adaptation of Text-Conditioned Diffusion Models for Audio-to-Image Generation"

Qualcomm Cloud AI SDK (Platform and Apps) enable high performance deep learning inference on Qualcomm Cloud AI platforms delivering high throughput and low latency across Computer Vision, Object Detection, Natural Language Processing and Generative AI models.

GenDB, an LLM-Powered Generative Query Engine Built for the Future