pretraining

The most impactful papers related to contrastive pretraining for multimodal models!

🌹 Rose: Range-Of-Slice Equilibration PyTorch optimizer. Stateless optimization through range-normalized gradient updates.

[NeurIPS 2023 Main Track] This is the repository for the paper titled "Don’t Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learner"

[ECCV 2024] Dyadic Interaction Modeling for Social Behavior Generation

Wav2vec 2.0 Self-Supervised Pretraining

使用LLaMA-Factory微调多模态大语言模型的示例代码 Demo of Finetuning Multimodal LLM with LLaMA-Factory

《Smol 训练手册》：打造世界级大模型的秘诀

Taught by AI genius Andrew NG, this course entails the cutting edge topics such as, How generative AI works including what it can and can't do, Common uses cases such as Reading, Writing, and Chatting, Life Cycle of GenAI projects, Advanced Technology options such as RAG, Fine tunning, and Pre-Training, Implications of GenAI on business & Society.

Train transformer-based models.

[AAAI'25] The implementation of paper "Federated Foundation Models on Heterogeneous Time Series" | The first work to explore time series foundation models on federated setting.

Comparing Selective Masking Methods for Depression Detection in Social Media