DeepCluster on MNIST

A re-implementation of the core idea behind Deep Clustering for Unsupervised Learning of Visual Features (Caron, Bojanowski, Joulin, Douze. 2018), applied to MNIST with PyTorch.

Idea: instead of labels, use k-Means on the network's own features to generate pseudo-labels. Then train the network to predict them, and repeat.

All code is in: notebooks/deepcluster.ipynb.

Notebook

• Supervisd Baseline: 4-layer CNN trained on the real MNIST label. This is then used as a baseline reference.
• k-Means: Implementation of the k-Means algorithm and a plot of within-cluster variance for k = 1..30 on raw pixels.
• DeepCluster loop: Each step encodes a batch, normalises the features, clusters them with k-Means, and trains the classifier head to predict the cluster index.
• Sanity Check: The encoder is frozen, the head is replaced with a new initialised one and trained on the real labels.

Results

Setup	Test accuracy
Supervised CNN (trained on labels)	0.9872
DeepCluster encoder (trained on cluster labels)	0.9582

Setup

git clone https://github.com/DMHamann/deepcluster.git
cd deepcluster

python -m venv .venv
source .venv/bin/activate       
pip install -r requirements.txt

jupyter lab notebooks/deepcluster.ipynb

MNIST is downloaded automatically into data/raw/ on the first run.

Reference

Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018). Deep Clustering for Unsupervised Learning of Visual Features. ECCV. arXiv:1807.05520