OA-DG: Object-Aware Domain Generalization

OA-DG is an effective method for single-domain object detection generalization (S-DGOD). It consists of two components: OA-Mix for data augmentation and OA-Loss for reducing domain gaps.

Object-Aware Domain Generalization for Object Detection, Wooju Lee^* , Dasol Hong^* , Hyungtae Lim^†, and Hyun Myung^†, AAAI 2024 (arXiv:2312.12133)

✨Features

• OA-DG consists of OA-Mix for data augmentation and OA-Loss for reducing the domain gap.

• OA-Mix increases image diversity while preserving important semantic feature with multi-level transformations and object-aware mixing.

  👀 View some example images

  

• OA-Loss reduces the domain gap by training semantic relations of foreground and background instances from multi-domain.

• Extensive experiments on standard benchmarks (Cityscapes-C and Diverse Weather Dataset) show that OA-DG outperforms SOTA methods on unseen target domains.

• OA-DG can be generally applied to improve robustness regardless of the augmentation set and object detector architectures.

🚣 Getting Started

Follow these steps to set up the project on your local machine for training and testing.

Prerequisites

Ensure you have the following prerequisites installed on your local system.

1. Install mmdetection: There are several installation guides. Follow one of the below:

   Our code is forked from mmdetection 2.28.x version.

   a. Customize Installation (recommended)

   # Install MMCV using MIM.
   
   $ pip install -U openmim
   $ mim install mmcv-full
   
   # Clone this repository
   $ git clone https://github.com/WoojuLee24/OA-DG.git
   
   # Go into the repository
   $ cd OA-DG
   
   # Install mmdetection
   $ pip install -v -e .

   b. Refer to the mmdetection's installation instructions.

   c. Use Dockerfile from mmdetection to setup the environment.

2. Install other dependencies

   # For image processing operations.
   $ pip install Pillow
   # For spectral-residual saliency algorithm in OA-Mix.
   $ pip install opencv-python
   $ pip install opencv-contrib-python

3. Prepare the following datasets:

   • Cityscapes: A dataset that contains urban street scenes from 50 cities with detailed annotations.
   • Diverse Weather Dataset: This dataset includes various weather conditions for robust testing and development of models, essential for applications in autonomous driving.

🏃 How To Run

Training

python3 -u tools/train.py $(CONFIG).py --work-dir $(WORK_DIR)

Example: OA-DG trained on Cityscapes dataset

python3 -u tools/train.py configs/OA-DG/cityscapes/faster_rcnn_r50_fpn_1x_cityscapes_oadg.py --work-dir /ws/data/cityscapes/faster_rcnn_r50_fpn_1x_cityscapes_oadg

Example: OA-DG trained on DWD dataset

python3 -u tools/train.py configs/OA-DG/dwd/faster_rcnn_r101_dc5_1x_dwd_oadg.py --work-dir /ws/data/dwd/faster_rcnn_r101_dc5_1x_dwd_oadg

Evaluation

• Cityscapes-C

   python3 -u tools/analysis_tools/test_robustness.py \
     $(CONFIG).py $(CKPT_FILE).pth --out $(OUT_PATH).pkl \
     --corruptions benchmark --eval bbox

  Example: OA-DG evaluated on Cityscapes-C dataset

   python3 -u tools/analysis_tools/test_robustness.py \
     configs/OA-DG/cityscapes/faster_rcnn_r50_fpn_1x_cityscapes_oadg.py \
     /ws/data/cityscapes/faster_rcnn_r50_fpn_1x_cityscapes_oadg/epoch_2.pth \ 
     --out /ws/data/cityscapes/faster_rcnn_r50_fpn_1x_cityscapes_oadg/test_robustness_result_2epoch.pkl \
     --corruptions benchmark --eval bbox

• Diverse Weather Dataset (DWD)

   python3 -u tools/test_dwd.py \
     $(CONFIG).py $(CKPT_FILE).pth --out $(OUT_PATH).pkl \
     --eval mAP

  Example: OA-DG evaluated on DWD dataset

   python3 -u tools/analysis_tools/test_dwd.py \
     configs/OA-DG/dwd/faster_rcnn_r101_dc5_1x_dwd_oadg.py \
     /ws/data/dwd/faster_rcnn_r101_dc5_1x_dwd_oadg/epoch_10.pth \ 
     --out /ws/data/dwd/faster_rcnn_r101_dc5_1x_dwd_oadg/test_robustness_result_10epoch.pkl \
     --eval mAP

Demo

You can run demo.

Results

We evaluated the robustness of our method for common corruptions and various weather conditions in urban scenes. mPC is an evaluation metric of robustness against out-of-distribution (OOD).

• Cityscapes-C:
• DWD:

TODO list

• Add instructions
• Add codes
• Add checkpoint files
• Add configuration files for DWD
• Pull request to MMDetection (Please click here to review the pull request.)

📢 License

Please see the LICENSE.md file.

📫 Contact Information

If you have any questions, please do not hesitate to contact us:

• Wooju Lee ✉️ [email protected]
• Dasol Hong ✉️ [email protected]

📎 Citation

@inproceedings{lee2024object,
  title={Object-Aware Domain Generalization for Object Detection},
  author={Lee, Wooju and Hong, Dasol and Lim, Hyungtae and Myung, Hyun},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={38},
  number={4},
  pages={2947--2955},
  year={2024}
}
@misc{lee2023objectaware,
      title={Object-Aware Domain Generalization for Object Detection}, 
      author={Wooju Lee and Dasol Hong and Hyungtae Lim and Hyun Myung},
      year={2023},
      eprint={2312.12133},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}