multimodal-deep-learning

LAVIS - A One-stop Library for Language-Vision Intelligence

FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models 🚀

[ICLR 2024] Official implementation of " 🦙 Time-LLM: Time Series Forecasting by Reprogramming Large Language Models"

(ෆ`꒳´ෆ) A Survey on Text-to-Image Generation/Synthesis.

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.

Official implementation for "Blended Latent Diffusion" [SIGGRAPH 2023]

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"

Towards Generalist Biomedical AI

Paper List of Pre-trained Foundation Recommender Models

A curated list of awesome Multimodal studies.

Code and Pretrained Models for ICLR 2023 Paper "Contrastive Audio-Visual Masked Autoencoder".

Implementation of CVPR 2023 paper "Prompting Large Language Models with Answer Heuristics for Knowledge-based Visual Question Answering".

My implementation of "Patch n’ Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution"

Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images - ICCV 2021

[CVPR'22 Best Paper Finalist] Official PyTorch implementation of the method presented in "Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentation"

A Python package housing a collection of deep-learning multi-modal data fusion method pipelines! From data loading, to training, to evaluation - fusilli's got you covered 🌸

End-to-end Training for Multimodal Recommendation Systems

Corpus of resources for multimodal machine learning with physiological signals (mmps).

Swarming algorithms like PSO, Ant Colony, Sakana, and more in PyTorch 😊