MemSeg: Memory-based semantic segmentation for off-road unstructured natural environments

Introduction

This repository is a PyTorch implementation of Memory-based semantic segmentation for off-road unstructured natural environments. This work is based on semseg.

The codebase mainly uses ResNet18, ResNet50 and MobileNet-V2 as backbone with ASPP module and can be easily adapted to other basic semantic segmentation structures.

Sample experimented dataset is RUGD.

Requirement

Hardware: >= 11G GPU memory

Software: PyTorch>=1.0.0, python3

Usage

For installation, follow installation steps below or recommend you to refer to the instructions described here.

For its pretrained ResNet50 backbone model, you can download from URL.

Getting Started

Installation

1. Clone this repository.

git clone https://github.com/youngsjjn/MemSeg.git

2. Install Python dependencies.

pip install -r requirements.txt

Implementation

1. Download datasets (i.e. RUGD) and change the root of data path in config.

Download data list of RUGD here.

2. Inference If you want to inference on pretrained models, download pretrained network in my drive and save them in ./exp/rugd/.

Inference "ResNet50 + Deeplabv3" without the memory module

sh tool/test.sh rugd deeplab50

Inference "ResNet50 + Deeplabv3" with the memory module

sh tool/test_mem.sh rugd deeplab50mem

3. Train (Evaluation is included at the end of the training) Train "ResNet50 + Deeplabv3" without the memory module

sh tool/train.sh rugd deeplab50

Train "ResNet50 + Deeplabv3" without the memory module

sh tool/train_mem.sh rugd deeplab50mem

Here, the example is for training or testing on "ResNet50 + Deeplabv3". If you want to train other networks, please change "deeplab50" or "deeplab50mem" as a postfix of a config file name.

For example, train "ResNet18 + PSPNet" with the memory module:

sh tool/train_mem.sh rugd pspnet18mem

Citation

If you like our work and use the code or models for your research, please cite our work as follows.

@article{DBLP:journals/corr/abs-2108-05635,
  author    = {Youngsaeng Jin and
               David K. Han and
               Hanseok Ko},
  title     = {Memory-based Semantic Segmentation for Off-road Unstructured Natural
               Environments},
  journal   = {CoRR},
  volume    = {abs/2108.05635},
  year      = {2021},
  url       = {https://arxiv.org/abs/2108.05635},
  eprinttype = {arXiv},
  eprint    = {2108.05635},
  timestamp = {Wed, 18 Aug 2021 19:45:42 +0200},
  biburl    = {https://dblp.org/rec/journals/corr/abs-2108-05635.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}