pre-training

The official GitHub page for the survey paper "A Survey of Large Language Models".

Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

Open Source Pre-training Model Framework in PyTorch & Pre-trained Model Zoo

Awesome resources for in-context learning and prompt engineering: Mastery of the LLMs such as ChatGPT, GPT-3, and FlanT5, with up-to-date and cutting-edge updates.

Papers about pretraining and self-supervised learning on Graph Neural Networks (GNN).

Unified Training of Universal Time Series Forecasting Transformers

Tencent Pre-training framework in PyTorch & Pre-trained Model Zoo

Research code for ECCV 2020 paper "UNITER: UNiversal Image-TExt Representation Learning"

Paper List of Pre-trained Foundation Recommender Models

Probing the representations of Vision Transformers.

[MIR-2023-Survey] A continuously updated paper list for multi-modal pre-trained big models

💐Kaleido-BERT: Vision-Language Pre-training on Fashion Domain

Official implementation for "UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction" (KDD 2024)

A curated list of papers on pre-training for graph neural networks (Pre-train4GNN).

Reproducible scaling laws for contrastive language-image learning (https://arxiv.org/abs/2212.07143)

[ICLR 2024] Domain-Agnostic Molecular Generation with Chemical Feedback

A collection of Audio and Speech pre-trained models.

An implementation of masked language modeling for Pytorch, made as concise and simple as possible

Official code for NeurIPS 2024 paper LRM-Zero: Training Large Reconstruction Models with Synthesized Data

Implementation of Marge, Pre-training via Paraphrasing, in Pytorch