This repository provides keras implementation of the paper "TEMPORAL ENSEMBLING FOR SEMI-SUPERVISED LEARNING" by S. Laine et al.
The implementation includes Temporal Ensembling and PI-Model using CIFAR-10. Both methods are proposed in the paper As the paper, the semi-supervised training is done by 4000 supervised data(400 per class) and 46,000 unsupervised data. 10,000 data are left for evaluation.
keras, tensorflow, scikit-learn
The versions of test environment :
Keras==2.1.6, tensorflow-gpu==1.7.0, scikit-learn==0.19.1
Prepare data and labels to use. For instance, CIFAR-10 is composed of 10 classes and each label should express unique class and be integer. These prepared data should be placed in the data directory.
You can download CIFAR-10 data via :
https://www.kaggle.com/janzenliu/cifar-10-batches-py
Put them in "data" directory and run the following code to compress them into NPZ file.
python make_cifar10_npz.py
After running this code, you can get cifar10.npz under "data" directory.
After data is prepared, run the following script to train and evaluation.
Please look into the script about other settable parameters or run "python main.py --help". Although the most of this implementation follows the description of the paper, there are some differences. Please see the note below.
# Temporal Ensembling
python main_temporal_ensembling.py
# PI-model
python main_pi_model.py
Evaluation is done at intervals of 5 epochs. In the test, Temporal ensembling and PI-model achieved about 87.3% accuracy and about 86.7% accuracy respectively at the end of epoch.
The differences between the paper and this implementation:
- The learning rate is changed to 0.001 instead of 0.003 because of non-convergence issue
- The training epoch is changed to 350 instead of 300 to achieve higher accuracy
-
Paper
Samuli Laine and Timo Aila : TEMPORAL ENSEMBLING FOR SEMI-SUPERVISED LEARNING, ICLR, 2017.
https://arxiv.org/pdf/1610.02242.pdf -
Official implementation in Theano(lasagne) by the authors
https://github.com/smlaine2/tempens