SeqNet: Learning Descriptors for Sequence-Based Hierarchical Place Recognition
[ArXiv+Supplementary] [IEEE Xplore RA-L 2021] [ICRA 2021 YouTube Video]
and
SeqNetVLAD vs PointNetVLAD: Image Sequence vs 3D Point Clouds for Day-Night Place Recognition
[ArXiv] [CVPR 2021 Workshop 3DVR]
Sequence-Based Hierarchical Visual Place Recognition.
News:
Jun 23: CVPR 2021 Workshop 3DVR paper, "SeqNetVLAD vs PointNetVLAD", now available on arXiv. Oxford dataset to be released soon.
Jun 02: SeqNet code release with the Nordland dataset.
Setup (One time)
Conda
conda create -n seqnet python=3.8 mamba -c conda-forge -y
conda activate seqnet
mamba install numpy pytorch=1.8.0 torchvision tqdm scikit-learn faiss tensorboardx h5py -c conda-forge -y
Download
Run bash download.sh to download single image NetVLAD descriptors (3.4 GB) for the Nordland-clean dataset [a] and corresponding model files (1.5 GB) [b].
Run
Train
To train sequential descriptors through SeqNet:
python main.py --mode train --pooling seqnet --dataset nordland-sw --seqL 10 --w 5 --outDims 4096 --expName "w5"
To (re-)train single descriptors through SeqNet:
python main.py --mode train --pooling seqnet --dataset nordland-sw --seqL 1 --w 1 --outDims 4096 --expName "w1"
Test
python main.py --mode test --pooling seqnet --dataset nordland-sf --seqL 5 --split test --resume ./data/runs/Jun03_15-22-44_l10_w5/
The above will reproduce results for SeqNet (S5) as per Supp. Table III on Page 10.
To obtain other results from the same table, expand this.
# Raw Single (NetVLAD) Descriptor
python main.py --mode test --pooling single --dataset nordland-sf --seqL 1 --split test
# SeqNet (S1)
python main.py --mode test --pooling seqnet --dataset nordland-sf --seqL 1 --split test --resume ./data/runs/Jun03_15-07-46_l1_w1/
# Raw + Smoothing
python main.py --mode test --pooling smooth --dataset nordland-sf --seqL 5 --split test
# Raw + Delta
python main.py --mode test --pooling delta --dataset nordland-sf --seqL 5 --split test
# Raw + SeqMatch
python main.py --mode test --pooling single+seqmatch --dataset nordland-sf --seqL 5 --split test
# SeqNet (S1) + SeqMatch
python main.py --mode test --pooling s1+seqmatch --dataset nordland-sf --seqL 5 --split test --resume ./data/runs/Jun03_15-07-46_l1_w1/
# HVPR (S5 to S1)
# Run S5 first and save its predictions by specifying `resultsPath`
python main.py --mode test --pooling seqnet --dataset nordland-sf --seqL 5 --split test --resume ./data/runs/Jun03_15-22-44_l10_w5/ --resultsPath ./data/results/
# Now run S1 + SeqMatch using results from above (the timestamp of `predictionsFile` would be different in your case)
python main.py --mode test --pooling s1+seqmatch --dataset nordland-sf --seqL 5 --split test --resume ./data/runs/Jun03_15-07-46_l1_w1/ --predictionsFile ./data/results/Jun03_16-07-36_l5_0.npz
Acknowledgement
The code in this repository is based on Nanne/pytorch-NetVlad. Thanks to Tobias Fischer for his contributions to this code during the development of our project QVPR/Patch-NetVLAD.
Citation
@article{garg2021seqnet,
title={SeqNet: Learning Descriptors for Sequence-based Hierarchical Place Recognition},
author={Garg, Sourav and Milford, Michael},
journal={IEEE Robotics and Automation Letters},
volume={6},
number={3},
pages={4305-4312},
year={2021},
publisher={IEEE},
doi={10.1109/LRA.2021.3067633}
}
@misc{garg2021seqnetvlad,
title={SeqNetVLAD vs PointNetVLAD: Image Sequence vs 3D Point Clouds for Day-Night Place Recognition},
author={Garg, Sourav and Milford, Michael},
howpublished={CVPR 2021 Workshop on 3D Vision and Robotics (3DVR)},
month={Jun},
year={2021},
}
Other Related Projects
Patch-NetVLAD (2021); Delta Descriptors (2020); CoarseHash (2020); seq2single (2019); LoST (2018)
[a] This is the clean version of the dataset that excludes images from the tunnels and red lights, exact image names can be obtained from here.
[b] These will automatically save to ./data/, you can modify this path in download.sh and get_datasets.py to specify your workdir.
5 Oct 20, 2022
40 Dec 10, 2022
5 Feb 04, 2022
16 Jul 27, 2022
393 Dec 22, 2022
6 Mar 06, 2022
36 Dec 04, 2022
16 Nov 21, 2022
34 Dec 28, 2022
1 Jan 30, 2022
133 Jan 05, 2023
32 Dec 02, 2022
5 May 26, 2022
15 Dec 24, 2022
61 Nov 07, 2022
101 Dec 30, 2022
14 Dec 20, 2022
294 Dec 30, 2022
2 Jan 05, 2022
25 Dec 04, 2022