Skip to content

Learning Camera Localization via Dense Scene Matching, CVPR2021

Notifications You must be signed in to change notification settings

Tangshitao/Dense-Scene-Matching

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

This repository contains code of our CVPR 2021 paper - "Learning Camera Localization via Dense Scene Matching" by Shitao Tang, Chengzhou Tang, Rui Huang, Siyu Zhu and Ping Tan.

This paper presents a new method for scene agnostic camera localization using dense scene matching (DSM), where a cost volume is constructed between a query image and a scene. The cost volume and the corresponding coordinates are processed by a CNN to predict dense coordinates. Camera poses can then be solved by PnP algorithms.

If you find this project useful, please cite:

@inproceedings{Tang2021Learning,
  title={Learning Camera Localization via Dense Scene Matching},
  author={Shitao Tang, Chengzhou Tang, Rui Huang, Siyu Zhu and Ping Tan},
  booktitle={Computer Vision and Pattern Recognition (CVPR)},
  year={2021}
}

Usage

Environment

  • The codes are tested along with
    • pytorch=1.4.0
    • lmdb (optional)
    • yaml
    • skimage
    • opencv
    • numpy=1.17
    • tensorboard

Installation

  • Build PyTorch operations
      cd libs/model/ops
      python setup.py install
    
  • Build PnP algorithm
      cd libs/utils/lm_pnp
      mkdir build
      cd build
      cmake ..
      make all
    

Train and Test

  • Download

    You can download the trained models and label files for 7scenes, Cambridge, Scannet.

    For 7scenes, you can use the prepared data in the following.

    Chess Fire Heads Office Pumpkin Kitchen Stairs

    For Cambridge landmarks, you can download image files here, and depths here.

  • Test

    Please refer to configs/7scenes.yaml for detailed explaination of how to set label file path and image file path.

    • 7scenes
      python tools/video_test.py --config configs/7scenes.yaml
      
    • Camrbrige
      python tools/video_test.py --config configs/cambridge.yaml
      
  • Train

    We use ResNet-FPN pretrained model.

      python tools/train_net.py
    

License

The MIT License (MIT)

Copyright (c) 2022 Shitao Tang

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

About

Learning Camera Localization via Dense Scene Matching, CVPR2021

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published