Code of paper Interact, Embed, and EnlargE (IEEE): Boosting Modality-specific Representations for Multi-Modal Person Re-identification.

Overview

Interact, Embed, and EnlargE (IEEE): Boosting Modality-specific Representations for Multi-Modal Person Re-identification

We provide the codes for reproducing result of our paper Interact, Embed, and EnlargE (IEEE): Boosting Modality-specific Representations for Multi-Modal Person Re-identification.

Installation

  1. Basic environments: python3.6, pytorch1.8.0, cuda11.1.

  2. Our codes structure is based on Torchreid. (More details can be found in link: https://github.com/KaiyangZhou/deep-person-reid , you can download the packages according to Torchreid requirements.)

# create environment
cd AAAI2022_IEEE/
conda create --name ieeeReid python=3.6
conda activate ieeeReid

# install dependencies
# make sure `which python` and `which pip` point to the correct path
pip install -r requirements.txt

# install torch and torchvision (select the proper cuda version to suit your machine)
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge

# install torchreid (don't need to re-build it if you modify the source code)
python setup.py develop

Get start

  1. You can use the setting in im_r50_softmax_256x128_amsgrad_RGBNT_ieee_part_margin.yaml to get the results of full IEEE.

    python ./scripts/mainMultiModal.py --config-file ./configs/im_r50_softmax_256x128_amsgrad_RGBNT_ieee_part_margin.yaml --seed 40
  2. You can run other methods by using following configuration file:

    # MLFN
    ./configs/im_r50_softmax_256x128_amsgrad_RGBNT_mlfn.yaml
    
    # HACNN
    ./configs/im_r50_softmax_256x128_amsgrad_RGBNT_hacnn.yaml
    
    # OSNet
    ./configs/im_r50_softmax_256x128_amsgrad_RGBNT_osnet.yaml
    
    # HAMNet
    ./configs/im_r50_softmax_256x128_amsgrad_RGBNT_hamnet.yaml
    
    # PFNet
    ./configs/im_r50_softmax_256x128_amsgrad_RGBNT_hamnet.yaml
    
    # full IEEE
    ./configs/im_r50_softmax_256x128_amsgrad_RGBNT_ieee_part_margin.yaml

Details

  1. The details of our Cross-modal Interacting Module (CIM) and Relation-based Embedding Module (REM) can be found in .\torchreid\models\ieee3modalPart.py. The design of Multi-modal Margin Loss(3M loss) can be found in .\torchreid\losses\multi_modal_margin_loss_new.py.

  2. Ablation study settings.

    You can control these two modules and the loss by change the corresponding codes.

    1. Cross-modal Interacting Module (CIM) and Relation-based Embedding Module (REM)
    # change the code in .\torchreid\models\ieee3modalPart.py
    
    class IEEE3modalPart(nn.Module):
        def __init__(···
        ):
            modal_number = 3
            fc_dims = [128]
            pooling_dims = 768
            super(IEEE3modalPart, self).__init__()
            self.loss = loss
            self.parts = 6
            
            self.backbone = nn.ModuleList(···
            )
    		
    		  # using Cross-modal Interacting Module (CIM)
            self.interaction = True
            # using channel attention in CIM
            self.attention = True
            
            # using Relation-based Embedding Module (REM)
            self.using_REM = True
            
            ···
    1. Multi-modal Margin Loss(3M loss)
    # change the code in .\configs\your_config_file.yaml
    
    # using Multi-modal Margin Loss(3M loss), you can change the margin by modify the parameter of "ieee_margin".
    ···
    loss:
      name: 'margin'
      softmax:
        label_smooth: True
      ieee_margin: 1
      weight_m: 1.0
      weight_x: 1.0
    ···
    
    # using only CE loss
    ···
    loss:
      name: 'softmax'
      softmax:
        label_smooth: True
      weight_x: 1.0
    ···
Real-time pose estimation accelerated with NVIDIA TensorRT

trt_pose Want to detect hand poses? Check out the new trt_pose_hand project for real-time hand pose and gesture recognition! trt_pose is aimed at enab

NVIDIA AI IOT 803 Jan 06, 2023
Updated for TTS(CE) = Also Known as TTN V3. The code requires the first server to be 'ttn' protocol.

Updated Updated for TTS(CE) = Also Known as TTN V3. The code requires the first server to be 'ttn' protocol. Introduction This balenaCloud (previously

Remko 1 Oct 17, 2021
The codebase for Data-driven general-purpose voice activity detection.

Data driven GPVAD Repository for the work in TASLP 2021 Voice activity detection in the wild: A data-driven approach using teacher-student training. S

Heinrich Dinkel 75 Nov 27, 2022
Election Exit Poll Prediction and U.S.A Presidential Speech Analysis using Machine Learning

Machine_Learning Election Exit Poll Prediction and U.S.A Presidential Speech Analysis using Machine Learning This project is based on 2 case-studies:

Avnika Mehta 1 Jan 27, 2022
Segmentation models with pretrained backbones. PyTorch.

Python library with Neural Networks for Image Segmentation based on PyTorch. The main features of this library are: High level API (just two lines to

Pavel Yakubovskiy 6.6k Jan 06, 2023
Toward Realistic Single-View 3D Object Reconstruction with Unsupervised Learning from Multiple Images (ICCV 2021)

Table of Content Introduction Getting Started Datasets Installation Experiments Training & Testing Pretrained models Texture fine-tuning Demo Toward R

VinAI Research 42 Dec 05, 2022
Implementation of paper "Towards a Unified View of Parameter-Efficient Transfer Learning"

A Unified Framework for Parameter-Efficient Transfer Learning This is the official implementation of the paper: Towards a Unified View of Parameter-Ef

Junxian He 216 Dec 29, 2022
Implementation of the paper All Labels Are Not Created Equal: Enhancing Semi-supervision via Label Grouping and Co-training

SemCo The official pytorch implementation of the paper All Labels Are Not Created Equal: Enhancing Semi-supervision via Label Grouping and Co-training

42 Nov 14, 2022
Cossim - Sharpened Cosine Distance implementation in PyTorch

Sharpened Cosine Distance PyTorch implementation of the Sharpened Cosine Distanc

Istvan Fehervari 10 Mar 22, 2022
ThunderGBM: Fast GBDTs and Random Forests on GPUs

Documentations | Installation | Parameters | Python (scikit-learn) interface What's new? ThunderGBM won 2019 Best Paper Award from IEEE Transactions o

Xtra Computing Group 647 Jan 04, 2023
🔥 Cannlytics-powered artificial intelligence 🤖

Cannlytics AI 🔥 Cannlytics-powered artificial intelligence 🤖 🏗️ Installation 🏃‍♀️ Quickstart 🧱 Development 🦾 Automation 💸 Support 🏛️ License ?

Cannlytics 3 Nov 11, 2022
本项目是一个带有前端界面的垃圾分类项目,加载了训练好的模型参数,模型为efficientnetb4,暂时为40分类问题。

说明 本项目是一个带有前端界面的垃圾分类项目,加载了训练好的模型参数,模型为efficientnetb4,暂时为40分类问题。 python依赖 tf2.3 、cv2、numpy、pyqt5 pyqt5安装 pip install PyQt5 pip install PyQt5-tools 使用 程

4 May 04, 2022
⚖️🔁🔮🕵️‍♂️🦹🖼️ Code for *Measuring the Contribution of Multiple Model Representations in Detecting Adversarial Instances* paper.

Measuring the Contribution of Multiple Model Representations in Detecting Adversarial Instances This repository contains the code for Measuring the Co

Daniel Steinberg 0 Nov 06, 2022
[Preprint] ConvMLP: Hierarchical Convolutional MLPs for Vision, 2021

Convolutional MLP ConvMLP: Hierarchical Convolutional MLPs for Vision Preprint link: ConvMLP: Hierarchical Convolutional MLPs for Vision By Jiachen Li

SHI Lab 143 Jan 03, 2023
TagLab: an image segmentation tool oriented to marine data analysis

TagLab: an image segmentation tool oriented to marine data analysis TagLab was created to support the activity of annotation and extraction of statist

Visual Computing Lab - ISTI - CNR 49 Dec 29, 2022
Face Recognition plus identification simply and fast | Python

PyFaceDetection Face Recognition plus identification simply and fast Ubuntu Setup sudo pip3 install numpy sudo pip3 install cmake sudo pip3 install dl

Peyman Majidi Moein 16 Sep 22, 2022
Deep Learning ❤️ OneFlow

Deep Learning with OneFlow made easy 🚀 ! Carefree? carefree-learn aims to provide CAREFREE usages for both users and developers. User Side Computer V

21 Oct 27, 2022
The easiest tool for extracting radiomics features and training ML models on them.

Simple pipeline for experimenting with radiomics features Installation git clone https://github.com/piotrekwoznicki/ClassyRadiomics.git cd classrad pi

Piotr Woźnicki 17 Aug 04, 2022
"MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction" (CVPRW 2022) & (Winner of NTIRE 2022 Challenge on Spectral Reconstruction from RGB)

MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction (CVPRW 2022) Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Z

Yuanhao Cai 274 Jan 05, 2023
Implementation of our paper 'RESA: Recurrent Feature-Shift Aggregator for Lane Detection' in AAAI2021.

RESA PyTorch implementation of the paper "RESA: Recurrent Feature-Shift Aggregator for Lane Detection". Our paper has been accepted by AAAI2021. Intro

137 Jan 02, 2023