FaPN: Feature-aligned Pyramid Network for Dense Image Prediction [arXiv] [Project Page]

@inproceedings{
  huang2021fapn,
  title={{FaPN}: Feature-aligned Pyramid Network for Dense Image Prediction},
  author={Shihua Huang and Zhichao Lu and Ran Cheng and Cheng He},
  booktitle={International Conference on Computer Vision (ICCV)},
  year={2021}
}

Overview

FaPN vs. FPN	Before vs. After Alignment

This project provides the whole official implementation for our ICCV2021 paper "FaPN: Feature-aligned Pyramid Network for Dense Image Prediction" based on Detectron2, PanoticFCN, and MaskFormer. FaPN is a simple yet effective top-down pyramidal architecture to generate multi-scale features for dense image prediction. Comprised of a feature alignment module (FAM) and a feature selection module (FSM), FaPN addresses the issue of feature alignment in the original FPN, leading to substaintial improvements on various dense prediction tasks, such as object detection, semantic, instance, panoptic segmentation, etc.

Installation

This project is based on Detectron2, which can be constructed as follows.

• Install Detectron2 following the instructions.
• Setup the dataset following the structure.
• Install DCNv2 following Install DCNv2.md.

Results

COCO Object Detection

Faster R-CNN + FaPN:

Cityscapes Semantic Segmentation

PointRend + FaPN:

ADE20K-150 Semantic Segmentation

MaskFormer + FaPN:

COCOStuff-10K Semantic Segmentation

MaskFormer + FaPN:

COCO Instance Segmentation

Mask R-CNN + FaPN:

PointRend + FaPN:

COCO Panoptic Segmentation

PanopticFPN + FaPN:

PanopticFCN + FaPN: