[IROS2021] NYU-VPR: Long-Term Visual Place Recognition Benchmark with View Direction and Data Anonymization Influences

Overview

NYU-VPR

This repository provides the experiment code for the paper Long-Term Visual Place Recognition Benchmark with View Direction and Data Anonymization Influences.

Here is a graphical user interface (GUI) for using VPR methods on custom datasets: https://github.com/ai4ce/VPR-GUI-Tool

Requirements

To install requirements:

pip install -r requirements.txt

Data Processing

1. Image Anonymization

To install mseg-api:

cd segmentation
cd mseg-api
pip install -e .

Make sure that you can run python -c "import mseg" in python.

To install mseg-semantic:

cd segmentation
cd apex
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

cd ../mseg-semantic
pip install -e .

Make sure that you can run python -c "import mseg_semantic" in python.

Finally:

input_file=/path/to/my/directory
model_name=mseg-3m
model_path=mseg_semantic/mseg-3m.pth
config=mseg_semantic/config/test/default_config_360_ms.yaml
python -u mseg_semantic/tool/universal_demo.py --config=${config} model_name {model_name} model_path ${model_path} input_file ${input_file}

2. Image Filtration

Inside the process folder, use whiteFilter.py to filter images with white pixel percentage.

Methods

1. VLAD+SURF

Modify vlad_codebook_generation.py line 157 - 170 to fit the dataset.

cd test/vlad
python vlad_codebook_generation.py
python query_image_closest_image_generation.py

*Notice: the processing may take a few hours.

2. VLAD+SuperPoint

cd test/vlad_SP
python main.py
python find_closest.py

*Notice: the processing may take a few hours.

3. NetVLAD

4. PoseNet

Copy the train_image_paths.txt and test_image_paths.txt to test/posenet.

Obtain the latitude and longtitude of training images and convert them to normalized Universal Transverse Mercator (UTM) coordinates.

cd test/posenet
python getGPS.py
python mean.py

Start training. This may take several hours. Suggestion: use slurm to run the process.

python train.py --image_path path_to_train_images/ --metadata_path trainNorm.txt

Generate the input file for testing from test_image_paths.txt.

python gen_test_txt.py

Start testing.

python single_test.py --image_path path_to_test_images/ --metadata_path test.txt --weights_path models_trainNorm/best_net.pth

The predicted normalized UTM coordinates of test images is in the image_name.txt. Match the test images with the training images based on their location.

python match.py

The matching result is in the match.txt.

5. DBoW

Copy the train_image_paths.txt and test_image_paths.txt to test/DBow3/utils. Copy and paste the content of test_image_paths.txt at the end of train_image_paths.txt and save the text file as total_images_paths.txt.

Open test/DBow3/utils/demo_general.cpp file. Change the for loop range at line 117 and line 123. Both ranges are the range of lines in total_images_paths.txt. The first for loop range is the range of test images and the second range is the range of training images. To run with multi-thread, you may run the code multiple times with small ranges of test images where the sum of ranges equals to the number of lines in test_image_paths.txt.

Compile and run the code.

cd test/DBow3
cmake .
cd utils
make
./demo_general a b

The result of each test image and its top-5 matched training images is in the output.txt.

Owner
Automation and Intelligence for Civil Engineering (AI4CE) Lab @ NYU
Automation and Intelligence for Civil Engineering (AI4CE) Lab @ NYU
The source codes for TME-BNA: Temporal Motif-Preserving Network Embedding with Bicomponent Neighbor Aggregation.

TME The source codes for TME-BNA: Temporal Motif-Preserving Network Embedding with Bicomponent Neighbor Aggregation. Our implementation is based on TG

2 Feb 10, 2022
一个运行在 𝐞𝐥𝐞𝐜𝐕𝟐𝐏 或 𝐪𝐢𝐧𝐠𝐥𝐨𝐧𝐠 等定时面板的签到项目

定时面板上的签到盒 一个运行在 𝐞𝐥𝐞𝐜𝐕𝟐𝐏 或 𝐪𝐢𝐧𝐠𝐥𝐨𝐧𝐠 等定时面板的签到项目 𝐞𝐥𝐞𝐜𝐕𝟐𝐏 𝐪𝐢𝐧𝐠𝐥𝐨𝐧𝐠 特别声明 本仓库发布的脚本及其中涉及的任何解锁和解密分析脚本,仅用于测试和学习研究,禁止用于商业用途,不能保证其合

Leon 1.1k Dec 30, 2022
Code and data for paper "Deep Photo Style Transfer"

deep-photo-styletransfer Code and data for paper "Deep Photo Style Transfer" Disclaimer This software is published for academic and non-commercial use

Fujun Luan 9.9k Dec 29, 2022
A deep learning framework for historical document image analysis

DIVA-DAF Description A deep learning framework for historical document image analysis. How to run Install dependencies # clone project git clone https

9 Aug 04, 2022
Official implementation for "Image Quality Assessment using Contrastive Learning"

Image Quality Assessment using Contrastive Learning Pavan C. Madhusudana, Neil Birkbeck, Yilin Wang, Balu Adsumilli and Alan C. Bovik This is the offi

Pavan Chennagiri 67 Dec 30, 2022
Code for ICCV2021 paper PARE: Part Attention Regressor for 3D Human Body Estimation

PARE: Part Attention Regressor for 3D Human Body Estimation [ICCV 2021] PARE: Part Attention Regressor for 3D Human Body Estimation, Muhammed Kocabas,

Muhammed Kocabas 277 Jan 03, 2023
A PyTorch Toolbox for Face Recognition

FaceX-Zoo FaceX-Zoo is a PyTorch toolbox for face recognition. It provides a training module with various supervisory heads and backbones towards stat

JDAI-CV 1.6k Jan 06, 2023
Official PyTorch Implementation of Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity

UnRigidFlow This is the official PyTorch implementation of UnRigidFlow (IJCAI2019). Here are two sample results (~10MB gif for each) of our unsupervis

Liang Liu 28 Nov 16, 2022
CodeContests is a competitive programming dataset for machine-learning

CodeContests CodeContests is a competitive programming dataset for machine-learning. This dataset was used when training AlphaCode. It consists of pro

DeepMind 1.6k Jan 08, 2023
Torch-based tool for quantizing high-dimensional vectors using additive codebooks

Trainable multi-codebook quantization This repository implements a utility for use with PyTorch, and ideally GPUs, for training an efficient quantizer

Daniel Povey 41 Jan 07, 2023
Self-supervised Multi-modal Hybrid Fusion Network for Brain Tumor Segmentation

JBHI-Pytorch This repository contains a reference implementation of the algorithms described in our paper "Self-supervised Multi-modal Hybrid Fusion N

FeiyiFANG 5 Dec 13, 2021
[CIKM 2021] Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive Learning

Enhancing Aspect-Based Sentiment Analysis with Supervised Contrastive Learning. This repo contains the PyTorch code and implementation for the paper E

Akuchi 18 Dec 22, 2022
using STGCN to achieve egg classification task

EEG Classification   The task requires us to classify electroencephalography(EEG) into six categories, including human body, human face, animal body,

4 Jun 13, 2022
Merlion: A Machine Learning Framework for Time Series Intelligence

Merlion: A Machine Learning Library for Time Series Table of Contents Introduction Installation Documentation Getting Started Anomaly Detection Foreca

Salesforce 2.8k Dec 30, 2022
Implementation of Deep Deterministic Policy Gradiet Algorithm in Tensorflow

ddpg-aigym Deep Deterministic Policy Gradient Implementation of Deep Deterministic Policy Gradiet Algorithm (Lillicrap et al.arXiv:1509.02971.) in Ten

Steven Spielberg P 247 Dec 07, 2022
Research Artifact of USENIX Security 2022 Paper: Automated Side Channel Analysis of Media Software with Manifold Learning

Automated Side Channel Analysis of Media Software with Manifold Learning Official implementation of USENIX Security 2022 paper: Automated Side Channel

Yuanyuan Yuan 175 Jan 07, 2023
LAnguage Model Analysis

LAMA: LAnguage Model Analysis LAMA is a probe for analyzing the factual and commonsense knowledge contained in pretrained language models. The dataset

Meta Research 960 Jan 08, 2023
🔀 Visual Room Rearrangement

AI2-THOR Rearrangement Challenge Welcome to the 2021 AI2-THOR Rearrangement Challenge hosted at the CVPR'21 Embodied-AI Workshop. The goal of this cha

AI2 55 Dec 22, 2022
The Official Implementation of the ICCV-2021 Paper: Semantically Coherent Out-of-Distribution Detection.

SCOOD-UDG (ICCV 2021) This repository is the official implementation of the paper: Semantically Coherent Out-of-Distribution Detection Jingkang Yang,

Jake YANG 62 Nov 21, 2022