distillation

Code for Deep Learning for Modern AI

Interpretable and efficient predictors using pre-trained language models. Scikit-learn compatible.

Distilling Task-Specific Knowledge from Teacher Model into BiLSTM

Question Answering model based on DistilBERT, trained and evaluated on the SQUAD dataset