JSeg

02/27/2024

CLIP-RC have been accepted by CVPR2024

09/27/2022

External Attention have been accepted by TPAMI.

09/19/2022

SegNeXt have been accepted by NeurIPS'2022.

Introduction

JSeg is a Semantic segmentation toolbox based on MMSegmentation, Jittor and JDet

Install

JSeg environment requirements:

• System: Linux(e.g. Ubuntu/CentOS/Arch), macOS, or Windows Subsystem of Linux (WSL)
• Python version >= 3.7
• CPU compiler (require at least one of the following)
  • g++ (>=5.4.0)
  • clang (>=8.0)
• GPU compiler (optional)
  • nvcc (>=10.0 for g++ or >=10.2 for clang)
• GPU library: cudnn-dev (recommend tar file installation, reference link)

Step 1: Install the requirements

git clone https://github.com/Jittor/JSeg
cd JSeg
python -m pip install -r requirements.txt

If you have any installation problems for Jittor, please refer to Jittor

Step 2: Install JSeg

cd JSeg
# suggest this 
python setup.py develop
# or
python setup.py install

If you don't have permission for install,please add --user.

Getting Started

Datasets

Please check the dataset_prepare for dataset preparation, The preparation of the dataset comes from MMSegmentation.

Config

JSeg defines the used model, dataset and training/testing method by config-file, please check the config.md to learn how it works.

Train

We support single-machine single-gpu, single-machine multi-gpu training, multi-machine training is not supported for the time being. Multi-gpu dependence can be referred to here

python tools/run_net.py --config-file=path/to/config --task=train

# For example
# Single GPU
python tools/run_net.py --config-file=project/fcn/fcn_r50-d8_512x1024_cityscapes_80k.py --task=train

# Multiple GPUs
mpirun -n 8 python tools/run_net.py --config-file=project/fcn/fcn_r50-d8_512x1024_cityscapes_80k.py --task=train

Val

We provide an evaluation script to evaluate the dataset. If there is not enough CPU memory, you can save CPU memory by setting --efficient_val to store the evaluation results in a local file.

python tools/run_net.py --config-file=path/to/config --resume=path/to/ckp --task=val

# For example
python tools/run_net.py --config-file=project/fcn/fcn_r50-d8_512x1024_cityscapes_80k.py --resume=work_dirs/fcn_r50-d8_512x1024_cityscapes_80k/checkpoints/ckpt_80000.pkl --task=val

Test for save result

We provide a test scripts to save the inference results of the data set, which can be saved in the specified location by setting --save-dir.

python tools/run_net.py --config-file=path/to/config --resume=path/to/ckp --save-dir=path/to/save_dir --task=test

# For example
python tools/run_net.py --config-file=project/fcn/fcn_r50-d8_512x1024_cityscapes_80k.py --resume=work_dirs/fcn_r50-d8_512x1024_cityscapes_80k/checkpoints/ckpt_80000.pkl --save-dir=./ --task=test

Demo

We provide a demo that can predict a single picture. For more information, please see here

from jseg.utils.inference import InferenceSegmentor


def main():
    config_file = 'project/fcn/fcn_r50-d8_512x1024_cityscapes_80k.py'
    ckp_file = 'work_dirs/fcn_r50-d8_512x1024_cityscapes_80k/checkpoints/ckpt_80000.pkl'
    save_dir = './'
    image = 'cityscapes/leftImg8bit/val/munster/munster_000069_000019_leftImg8bit.png'

    inference_segmentor = InferenceSegmentor(config_file, ckp_file, save_dir)
    inference_segmentor.infer(image)


if __name__ == "__main__":
    main()

Supported backbones:

• ✔️ ResNet (CVPR'2016)
• ✔️ ResNeXt (CVPR'2017)
• ✔️ MobileNetV2 (CVPR'2018)
• ✔️ ResNeSt (ArXiv'2020)
• ✔️ Vision Transformer (ICLR'2021)
• ✔️ Swin Transformer (ICCV'2021)
• ✔️ ConvNeXt (CVPR'2022)
• ✔️ BEiT (ICLR'2022)
• ✔️ MAE (CVPR'2022)

Supported methods:

• ✔️ FCN (CVPR'2015/TPAMI'2017)
• ✔️ PSPNet (CVPR'2017)
• ✔️ DeepLabV3 (ArXiv'2017)
• ✔️ DeepLabV3+ (CVPR'2018)
• ✔️ UPerNet (ECCV'2018)
• ✔️ NonLocal Net (CVPR'2018)
• ✔️ ANN (ICCV'2019)
• ✔️ DANet (CVPR'2019)
• ✔️ CCNet (ICCV'2019)
• ✔️ EMANet (ICCV'2019)
• ✔️ GCNet (ICCVW'2019/TPAMI'2020)
• ✔️ PointRend (CVPR'2020)
• ✔️ SegFormer (NeurIPS'2021)
• ✔️ EANet (TPAMI)
• ✔️ SegNeXt (NeurIPS'2022)

Supported datasets:

• ✔️ ADE20K
• ✔️ Cityscapes
• ✔️ PASCAL VOC
• ✔️ iSAID
• ✔️ LoveDA
• ✔️ Potsdam
• ✔️ Vaihingen

Contact Us

Website: http://cg.cs.tsinghua.edu.cn/jittor/

Email: [email protected]

File an issue: https://github.com/Jittor/jittor/issues

QQ Group: 761222083

The Team

JSeg is currently maintained by the Tsinghua CSCG Group. If you are also interested in JSeg and want to improve it, Please join us!

Citation

@article{hu2020jittor,
  title={Jittor: a novel deep learning framework with meta-operators and unified graph execution},
  author={Hu, Shi-Min and Liang, Dun and Yang, Guo-Ye and Yang, Guo-Wei and Zhou, Wen-Yang},
  journal={Science China Information Sciences},
  volume={63},
  number={222103},
  pages={1--21},
  year={2020}
}

Reference

1. mmsegmentation
2. Jittor
3. JDet
4. mmcv
5. timm