Less is More: Learning from Synthetic Data with Fine-grained Attributes for Person Re-Identification

Suncheng Xiang

Shanghai Jiao Tong University

Overview

In this paper, we construct and label a large-scale synthetic person dataset named FineGPR with fine-grained attribute distribution. Moreover, aiming to fully exploit the potential of FineGPR and promote the efficient training from millions of synthetic data, we propose an attribute analysis pipeline AOST to learn attribute distribution in target domain, then apply style transfer network to eliminate the gap between synthetic and real-world data and thus is freely deployed to new scenarios. Experiments conducted on benchmarks demonstrate that FineGPR with AOST outperforms (or is on par with) existing real and synthetic datasets, which suggests its feasibility for re-ID and proves the proverbial less-is-more principle. We hope this fine-grained dataset could advance research towards re-ID in real scenarios.

[Paper] [Video Sample] [Related Project]

🔥 NEWS 🔥

• [10/2021] 📣 The first FineGPR-C caption dataset involving human describing event is coming !

• [09/2021] 📣 The large-scale synthetic person dataset FineGPR with fine-grained attribute distribution is released !

Table of Contents 👀

• FineGPR Introduction
• Comparison with existing datasets
• Link of the Dataset
• Method
• Results
• Extendibility
• FineGPR-Caption dataset
• Citation
• Ethical Considerations
• LICENSE
• Acknowledgements

FineGPR Introduction

The FineGPR dataset is generated by a popular GTA5 game engine that can synthesise images under controllable viewpoints, weathers,illuminations and backgrounds, as well as 13 fine-grained attributes at the identity level 👍 .

Our FineGPR dataset provides fine-grained and accurately configurable annotations, including 36 different viewpoints, 7 different kinds of weathers, 7 different kinds of illuminations, and 9 different kinds of backgrounds.

Viewpoint 📷

Definition of different viewpoints. Viewpoints of one identity are sampled at an interval of 10°, e.g. 0°-80° denotes that a person has 9 different angles in total.

Weather 🌨 and Illumination 🎇

The exemplars of different weather distribution (left) and illumination distribution (right) from the proposed FineGPR dataset.

Attributes at the Identity Level ⛹️‍♀️

The distributions of attributes at the identity level on FineGPR. The left figure shows the numbers of IDs for each attribute. The middle and right pies illustrate the distribution of the colors of upper-body and low-body clothes respectively.

Some visual exemplars with ID-level pedestrian attributes in the proposed FineGPR dataset, such as Wear short sleeve , Wear dress, Wear hat, Carry bag, etc.

Comparison with existing datasets

Some Mainstream Datasets for Person Re-Identification

For related FineGPR dataset (details of the previous related work, please refer to the our homepage GPR 🔎 :

Link of the Dataset

Data of FineGPR for Viewpoint Analysis

A small subset of FineGPR can be downloaded from the following links:

• SJTU Yun Drive:

  • Download Link password: qbdg

• Baidu Yun Drive:

  • Download Link password: h4k5

• Microsoft OneDrive:

  • Download Link

Directories & Files of images

FineGPR_Dataset 
├── FineGPR/   # This file is our original dataset, we provide the samples of ID=0001 and ID=0003 in this file folder.
│   ├── 0001
│   │   ├── 0001_c01_w01_l01_p01.jpg 
│   │	├── 0001_c01_w01_l02_p01.jpg  
│   │   ├── 0001_c01_w01_l03_p01.jpg
│   │   └── ...
│   ├── 0003/
│   │   ├── 0003_c01_w01_l01_p06.jpg  
│   │   ├── 0003_c01_w01_l02_p06.jpg
│   │   ├── 0003_c01_w01_l03_p06.jpg	   
│   │   └── ...
│   └── ...
├── FineGPR_subset   # This file is the subset of FineGPR dataset, each Identity contains 4 images. 
│   ├── 0001_c01_w03_l05_p03.jpg 
│   ├── 0001_c10_w03_l05_p03.jpg
│   ├── 0001_c19_w03_l05_p03.jpg
│   ├── 0001_c28_w03_l05_p03.jpg
│   ├── 0003_c01_w03_l05_p08.jpg 
│   ├── 0003_c10_w03_l05_p08.jpg
│   ├── 0003_c19_w03_l05_p08.jpg
│   ├── 0003_c28_w03_l05_p08.jpg  
│   └── ...
└── README.md   # Readme file

Name of the image

Taking "0001_c01_w01_l01_p01.jpg" as an example:

• 0001 is the id of the person
• c01 is the id of the camera
• w01 is the id of the weather
• l01 is the id of the illumination
• p01 is the id of the background

Viewpoint annotations

FineGPR
├── c01：90°      ├── c10：180°      ├── c19：270°      ├── c28：0°
├── c02：100°     ├── c11：190°      ├── c20：280°      ├── c29：10°
├── c03：110°     ├── c12：200°      ├── c21：290°      ├── c30：20°
├── c04：120°     ├── c13：210°      ├── c22：300°      ├── c31：30°
├── c05：130°     ├── c14：220°      ├── c23：310°      ├── c32：40°
├── c06：140°     ├── c15：230°      ├── c24：320°      ├── c33：50°
├── c07：150°     ├── c16：240°      ├── c25：330°      ├── c34：60°
├── c08：160°     ├── c17：250°      ├── c26：340°      ├── c35：70°
└── c09：170°     └── c18：260°      └── c27：350°      └── c36：80°

Weather annotations

FineGPR
├── w01：Sunny
├── w02：Clouds    
├── w03：Overcast
├── w04：Foggy   
├── w05：Neutral
├── w06：Blizzard 
└── w07：Snowlight 	   

Illumination annotations

FineGPR
├── l01：Midnight
├── l02：Dawn    
├── l03：Forenoon
├── l04：Noon   
├── l05：Afternoon
├── l06：Dusk 
└── l07：Night 	   

Scene annotations

FineGPR
├── p01：Urban
├── p02：Urban   
├── p03：Wild
├── p04：Urban   
├── p05：Wild
├── p06：Urban
├── p07：Urban
├── p08：Wild 
└── p09：Urban 	   

Method

💡 The two-stage pipeline AOST to learn attribute distribution of target domain. Firstly, we learn attribute distribution of real domain on the basis of XGBoost & PSO learning system. Secondly, we perform style transfer to enhance the reality of optimal dataset. Finally, the transferred data are adopted for downstream re-ID task.

Results

Performance comparison with existing Real and Synthetic datasets on Market-1501, DukeMTMC-reID and CUHK03, respectively.

References

• [1] Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification. CVPR 2018.
• [2] Bag of tricks and a strong baseline for deep person re-identification. CVPRW 2019.

Extendibility

Accompanied with our FineGPR, we also provide some human body masks (Middle) and keypoint locations (Bottom) of all characters during the annotation. We hope that our synthetic dataset FineGPR can not only contribute a lot to the development of generalizable person re-ID, but also advance the research of other computer vision tasks, such as human part segmentation and pose estimation.

FineGPR-C caption dataset

On the basis of FineGPR dafaset, we introduce a dynamic strategy to generate high-quality captions with fine-grained attribute annotations for semantic-based pretraining. To be more specific, we rearrange the different attributes as word embeddings into caption formula in the different position, and then generate semantically dense caption with high-quality description, which gives rise to our newly constructed FineGPR-C caption dataset.

A small subset of FineGPR-C caption dataset can be downloaded from the following links:

• Microsoft OneDrive:
  • Download Link

Citation

If you use our FineGPR dataset for your research, please cite our Paper.

@article{xiang2021less,
  title={Less is More: Learning from Synthetic Data with Fine-grained Attributes for Person Re-Identification},
  author={Xiang, Suncheng and You, Guanjie and Guan, Mengyuan and Chen, Hao and Wang, Feng and Liu, Ting and Fu, Yuzhuo},
  journal={arXiv preprint arXiv:2109.10498},
  year={2021}
}

If you do think this FineGPR-C caption dataset is useful and have used it in your research, please cite our Paper.

@article{xiang2021vtbr,
  title={VTBR: Semantic-based Pretraining for Person Re-Identification},
  author={Xiang, Suncheng and Zhang, Zirui and Guan, Mengyuan and Chen, Hao and Yan, Binjie and Liu, Ting and Fu, Yuzhuo},
  journal={arXiv preprint arXiv:2110.05074},
  year={2021}
}

Ethical Considerations

Our task and dataset were created with careful attention to ethical questions, which we encountered throughout our work. Access to our dataset will be provided for research purposes only and with restrictions on redistribution. Additionally, as we filtered out the sensitive attribute name in our fine-grained attribute annotation, our dataset cannot be easily repurposed for unintended tasks. Importantly, we are very cautious of human-annotation procedure of large scale datasets towards the social and ethical implications. Furthermore, we do not consider the datasets for developing non-research systems without further processing or augmentation. We hope this fine-grained dataset will shed light into potential tasks for the research community to move forward.

LICENSE

• The FineGPR Dataset and FineGPR-C caption is made available for non-commercial purposes only.
• You will not, directly or indirectly, reproduce, use, or convey the FineGPR dataset and FineGPR-C caption dataset or any Content, or any work product or data derived therefrom, for commercial purposes.

Permissions of this strong copyleft license (GNU General Public License v3.0) are conditioned on making available complete source code of licensed works and modifications, which include larger works using a licensed work, under the same license. Copyright and license notices must be preserved. Contributors provide an express grant of patent rights.

Acknowledgements

This research was supported by the National Natural Science Foundation of China under Project (Grant No. 61977045). We would like to thank authors of FineGPR, and FineGPR-Caption dataset for their work. They provide tremendous efforts in these dataset to advance the research in this field. We also appreciate Zefang Yu, Mingye Xie and Guanjie You for insightful feedback and discussion.

For further questions and suggestions about our datasets and methods, please feel free to contact Suncheng Xiang: [email protected]