Modular Graph Transformer Networks for Multi-Label Image Classification (MGTN)

This project implements the multi-learning based on Modular Graph Transformer Networks (MGTN).

Official GitHub: https://github.com/ReML-AI/MGTN

Requirements

Please, install the following packages

• numpy
• pytorch (1.*)
• torchnet
• torchvision
• tqdm
• networkx

Download best checkpoints

checkpoint/coco/mgtn_final_86.9762.pth.tar (Dropbox) or (BaiDu),(password:kknm)

Performance

Method	mAP	CP	CR	CF1	OP	OR	OF1
CNN-RNN	61.2	-	-	-	-	-	-
SRN	77.1	81.6	65.4	71.2	82.7	69.9	75.8
Baseline(ResNet101)	77.3	80.2	66.7	72.8	83.9	70.8	76.8
Multi-Evidence	–	80.4	70.2	74.9	85.2	72.5	78.4
ML-GCN (2019)	82.4	84.4	71.4	77.4	85.8	74.5	79.8
ML-GCN(ResNeXt50 with ImageNet)	86.2	85.8	77.3	81.3	86.2	79.7	82.8
A-GCN	83.1	84.7	72.3	78.0	85.6	75.5	80.3
KSSNet	83.7	84.6	73.2	77.2	87.8	76.2	81.5
SGTN (Our**)	86.6	77.2	82.2	79.6	76.0	82.6	79.2
MGTN(Base)	86.91	89.38	74.46	81.25	90.91	76.27	82.95
MGTN(Final}	86.98	86.11	77.85	81.77	87.71	79.40	83.35

** SGTN (Our): https://github.com/ReML-AI/sgtn

MGTN on COCO

python mgtn.py data/coco --image-size 448 --workers 8 --batch-size 32 --lr 0.03 --learning-rate-decay 0.1 --epoch_step 20 30 --embedding model/embedding/coco_glove_word2vec_80x300_ec.pkl --adj-strong-threshold 0.4 --adj-weak-threshold 0.2 --device_ids 0 1 2 3

How to cite this work?

@inproceedings{Nguyen:AAAI:2021,
	author = {Nguyen, Hoang D. and Vu, Xuan-Son and Le, Duc-Trong},
	title = {Modular Graph Transformer Networks for Multi-Label Image Classification},
	booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
	series = {AAAI '21},
	year = {2021},
	publisher = {AAAI}
}

Reference

This project is based on the following implementations:

• https://github.com/ReML-AI/sgtn
• https://github.com/durandtibo/wildcat.pytorch
• https://github.com/tkipf/pygcn
• https://github.com/Megvii-Nanjing/ML_GCN/
• https://github.com/seongjunyun/Graph_Transformer_Networks