pretrained-language-model

Foundation Architecture for (M)LLMs

Summarization Papers

The official code for "TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting (ICLR 2024)". TEMPO is one of the very first open source Time Series Foundation Models for forecasting task v1.0 version.

[NeurIPS 2021] COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining

CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks

[ACL2023] Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inference