Repo for Diversity induced weighted mutual learning (DWML) knowledge distillation
-
Updated
Mar 14, 2025 - Python
#
sample-efficient-ml
Here are 2 public repositories matching this topic...
This repository contains the training and evaluation code accompanying the thesis "Learning with Less: Contrastive Weight Tying on the BabyLM Challenge" (Ino van de Wouw, VU Amsterdam, 2025). The project studies headless language models, models pretrained with Contrastive Weight Tying (CWT) (Godey et al., 2024) instead of a standard cross-entropy
glue slurm pytorch transformer computational-linguistics gpt bert blimp huggingface pretrained-language-model pytorch-lightning contrastive-learning sample-efficient-rl llm babylm sample-efficient-ml contrastive-weight-tying headless-language-models sample-efficient-learning
-
Updated
Jul 26, 2026 - Python
Add this topic to your repo
To associate your repository with the sample-efficient-ml topic, visit your repo's landing page and select "manage topics."