foundation-models

A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, llava-onevision, llama-3.2-vision, qwen-vl, qwen2-vl, phi3-v etc.

Production-ready data processing made easy and shareable

[ECCV 2024] Improving 2D Feature Representations by 3D-Aware Fine-Tuning

[WACV 2024 Survey Paper] Multimodal Large Language Models for Autonomous Driving

Reliable, minimal and scalable library for pretraining foundation and world models

[ICLR'24] Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning

Official Code for "Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents"

Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.

[NeurIPS-2023] Annual Conference on Neural Information Processing Systems

mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video (ICML 2023)

Immersive workshop showcasing the remarkable potential of integrating SoTA foundation models to enhance product experiences and streamline backend workflows. Leverages Microsoft's Copilot stack, Microsoft Agent Framework and Azure primitives to offer an engaging and comprehensive introduction to AI-infused app development and deployment

An Ollama-compatible CLI for Apple's Foundation Models

A professional list on Multi-modal Data Fusion Models and Key Datasets for Urban Computing.

End-to-end Training for Multimodal Recommendation Systems

Agentic AI research papers, benchmarks, frameworks, and tools curated across 24 domains.

The official code for "TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting (ICLR 2024)". TEMPO is one of the very first open source Time Series Foundation Models for forecasting task v1.0 version.

Mantis: Lightweight Foundation Model for Time Series Classification

An Integrated Library for Tuning, Deploying and Interpreting Genomic Models

[TMLR 2025 → ICML 2026] RouteFinder: Towards Foundation Models for Vehicle Routing Problems

🦑 Unified Swift SDK for LLM inference across local and cloud providers