roberta

[NAACL 2021] This is the code for our paper `Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach'.

Simple State-of-the-Art BERT-Based Sentence Classification with Keras / TensorFlow 2. Built with HuggingFace's Transformers.

PyTorch implementation of "data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language" from Meta AI

BERT classification model for processing texts longer than 512 tokens. Text is first divided into smaller chunks and after feeding them to BERT, intermediate results are pooled. The implementation allows fine-tuning.

Language Quantized AutoEncoders

The official implementation of "EmoBERTa: Speaker-Aware Emotion Recognition in Conversation with RoBERTa"

Tensorflow and Keras implementation of the state of the art researches in Dialog System NLU

Scripts and Approach for Amazon ML Challenge

Abstractive and Extractive Text summarization using Transformers.

Official implementation of the paper "IteraTeR: Understanding Iterative Revision from Human-Written Text" (ACL 2022)

OOD Generalization and Detection (ACL 2020)

📚 Text Classification with LoRA (Low-Rank Adaptation) of Language Models - Efficiently fine-tune large language models for text classification tasks using the Stanford Sentiment Treebank (SST-2) dataset and the LoRA technique.

NLP model zoo for Russian

Modular Rust transformer/LLM library using Candle

Master thesis with code investigating methods for incorporating long-context reasoning in low-resource languages, without the need to pre-train from scratch. We investigated if multilingual models could inherit these properties by making it an Efficient Transformer (s.a. the Longformer architecture).

Distilling Task-Specific Knowledge from Teacher Model into BiLSTM

RoBERTa Marathi Language model trained from scratch during huggingface 🤗 x flax community week

Plan and train German transformer models.

Question Answering model based on DistilBERT, trained and evaluated on the SQUAD dataset

🚀 Unified NLP Pipelines for Language Models