code for Multi-scale Matching Networks for Semantic Correspondence, ICCV

Related tags

Deep LearningMMNet
Overview

MMNet

This repo is the official implementation of ICCV 2021 paper "Multi-scale Matching Networks for Semantic Correspondence.".

Pre-requisite

conda create -n mmnet python==3.8.0
conda activate mmnet
conda install torch==1.8.1 torchvision==0.9.1
pip install matplotlib scikit-image pandas

for installation of gluoncvth (fcn-resnet101):

git clone https://github.com/StacyYang/gluoncv-torch.git
cd gluoncv-torch
python setup.py install

Reproduction

for test

Trained models are available on [google drive].

pascal with fcn-resnet101 backbone([email protected]:81.6%):

python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name path\to\ckp_pascal_fcnres101.pth --resize 224,320

spair with fcn-resnet101 backbone([email protected]:46.6%):

python test.py --alpha 0.05 --benchmark spair --backbone fcn-resnet101 --ckp_name path\to\ckp_spair_fcnres101.pth --resize 224,320

Bibtex

If you use this code for your research, please consider citing:

@article{zhao2021multi,
  title={Multi-scale Matching Networks for Semantic Correspondence},
  author={Zhao, Dongyang and Song, Ziyang and Ji, Zhenghao and Zhao, Gangming and Ge, Weifeng and Yu, Yizhou},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2021}
}
You might also like...
A Pytorch implementation of
A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generative Modeling" (ICCV 2021)

Manifold Matching via Deep Metric Learning for Generative Modeling A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generat

null 69 Dec 10, 2022
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)

Hierarchical Memory Matching Network for Video Object Segmentation Hongje Seong, Seoung Wug Oh, Joon-Young Lee, Seongwon Lee, Suhyeon Lee, Euntai Kim

Hongje Seong 72 Dec 14, 2022
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)
Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)

Hierarchical Memory Matching Network for Video Object Segmentation Hongje Seong, Seoung Wug Oh, Joon-Young Lee, Seongwon Lee, Suhyeon Lee, Euntai Kim

Hongje Seong 26 Sep 26, 2021
Multi-Scale Geometric Consistency Guided Multi-View Stereo

ACMM [News] The code for ACMH is released!!! [News] The code for ACMP is released!!! About ACMM is a multi-scale geometric consistency guided multi-vi

Qingshan Xu 118 Jan 4, 2023
Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task
Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task

Siamese Deep Neural Networks for Semantic Text Similarity PyTorch A repository c

Shahrukh Khan 32 Dec 15, 2022
A PyTorch implementation of
A PyTorch implementation of "Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning", IJCAI-21

MERIT A PyTorch implementation of our IJCAI-21 paper Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning. Depen

Graph Analysis & Deep Learning Laboratory, GRAND 32 Jan 2, 2023
Implementation of Memory-Efficient Neural Networks with Multi-Level Generation, ICCV 2021
Implementation of Memory-Efficient Neural Networks with Multi-Level Generation, ICCV 2021

Memory-Efficient Multi-Level In-Situ Generation (MLG) By Jiaqi Gu, Hanqing Zhu, Chenghao Feng, Mingjie Liu, Zixuan Jiang, Ray T. Chen and David Z. Pan

Jiaqi Gu 2 Jan 4, 2022
Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)
Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)

Official implementation of GOCor This is the official implementation of our paper : GOCor: Bringing Globally Optimized Correspondence Volumes into You

Prune Truong 71 Nov 18, 2022
《Dual-Resolution Correspondence Network》(NeurIPS 2020)
《Dual-Resolution Correspondence Network》(NeurIPS 2020)

Dual-Resolution Correspondence Network Dual-Resolution Correspondence Network, NeurIPS 2020 Dependency All dependencies are included in asset/dualrcne

Active Vision Laboratory 45 Nov 21, 2022
Comments
  • NaN during training

    NaN during training

    Hi, congrats on your paper! I was trying to run your training code (with resnet 101 on pf-pascal) but directly after a couple of iterations, nan appear in the input. Have you ever seen this issue? Thanks

    opened by PruneTruong 2
  • In def calLayer1,i do not know where are self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1

    In def calLayer1,i do not know where are self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1

    Hello,this paper is very nice,i am very love it. I read your code,in Model.py, def calLayer1(self, feats): sum1 = self.conv1_1_down(self.msblock1_1(feats[1])) +
    self.conv1_2_down(self.msblock1_2(feats[2])) +
    self.conv1_3_down(self.msblock1_3(feats[3])) sum1 = self.wa_1(sum1) return sum1 I do not find where are these operation,self.conv1_1_down,self.conv1_2_down,self.conv1_3_down,self.wa_1,so where are these ,in which document.Thank you,looking forward to your reply.

    opened by liang532 1
  • How to prepare the PF-Pascal dataset?

    How to prepare the PF-Pascal dataset?

    I downloaded the PF-dataset-Pascal.zip from the Proposal Flow paper's web page, extracted it, and run the next line of command, but get errors about missing data files.

    Input:

    python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name assets/model/mmnet_fcnresnet101_pascal.pth --resize 224,320

    Expected output: some results about the benchmark results.

    Actual output:

    currently executing test.py file.
2021-11-19 02:01:59,172 - INFO - Options listed below:----------------
2021-11-19 02:01:59,172 - INFO - name: framework_train
2021-11-19 02:01:59,172 - INFO - benchmark: pfpascal
2021-11-19 02:01:59,172 - INFO - thresh_type: auto
2021-11-19 02:01:59,172 - INFO - backbone_name: fcn-resnet101
2021-11-19 02:01:59,172 - INFO - ms_rate: 4
2021-11-19 02:01:59,173 - INFO - feature_channel: 21
2021-11-19 02:01:59,173 - INFO - batch: 5
2021-11-19 02:01:59,173 - INFO - gpu: 0
2021-11-19 02:01:59,173 - INFO - data_path: /data/SC_Dataset
2021-11-19 02:01:59,173 - INFO - ckp_path: ./checkpoints_debug
2021-11-19 02:01:59,173 - INFO - visualization_path: visualization
2021-11-19 02:01:59,173 - INFO - model_type: MMNet
2021-11-19 02:01:59,173 - INFO - ckp_name: assets/model/mmnet_fcnresnet101_pascal.pth
2021-11-19 02:01:59,173 - INFO - log_path: ./logs/
2021-11-19 02:01:59,173 - INFO - resize: 224,320
2021-11-19 02:01:59,173 - INFO - max_kps_num: 50
2021-11-19 02:01:59,173 - INFO - split_type: test
2021-11-19 02:01:59,173 - INFO - alpha: 0.05
2021-11-19 02:01:59,173 - INFO - resolution: 2
2021-11-19 02:01:59,173 - INFO - Options all listed.------------------
2021-11-19 02:01:59,173 - INFO - ckp file: assets/model/mmnet_fcnresnet101_pascal.pth
Traceback (most recent call last):
  File "/home/runner/MMNet/test.py", line 127, in <module>
    test(logger, options)
  File "/home/runner/MMNet/test.py", line 65, in test
    test_dataset = Dataset.CorrespondenceDataset(
  File "/home/runner/MMNet/data/PascalDataset.py", line 32, in __init__
    self.train_data = pd.read_csv(self.spt_path)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 482, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 811, in __init__
    self._engine = self._make_engine(self.engine)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1040, in _make_engine
    return mapping[engine](self.f, **self.options)  # type: ignore[call-arg]
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 51, in __init__
    self._open_handles(src, kwds)
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/parsers/base_parser.py", line 222, in _open_handles
    self.handles = get_handle(
  File "/home/runner/miniconda3/lib/python3.9/site-packages/pandas/io/common.py", line 702, in get_handle
    handle = open(
FileNotFoundError: [Errno 2] No such file or directory: '/data/SC_Dataset/PF-PASCAL/test_pairs.csv'

    P.S. Output of executing ls /data/SC_Dataset/PF-PASCAL/:

    Annotations  html  index.html  JPEGImages  parsePascalVOC.mat  ShowMatchingPairs
    opened by tjyuyao 2
  • How to reproduce the reported test accuracy?

    How to reproduce the reported test accuracy?

    By running given following command with code on the main branch:

    python test.py --alpha 0.05 --backbone fcn-resnet101 --ckp_name assets/model/mmnet_fcnresnet101_spair.pth --resize 224,320 --benchmark spair

    I expect to get the reported accuracy in the Table.2 of paper, i.e. 50.4 "all" accuracy, or spair with fcn-resnet101 backbone([email protected]:46.6%): as noted in the README.md file. However I get the following output, finding nowhere the related results. Can you point out the steps to reproduce the test accuracy?

    2021-11-19 00:49:54,452 - INFO - Options listed below:----------------
2021-11-19 00:49:54,452 - INFO - name: framework_train
2021-11-19 00:49:54,453 - INFO - benchmark: spair
2021-11-19 00:49:54,453 - INFO - thresh_type: auto
2021-11-19 00:49:54,454 - INFO - backbone_name: fcn-resnet101
2021-11-19 00:49:54,455 - INFO - ms_rate: 4
2021-11-19 00:49:54,455 - INFO - feature_channel: 21
2021-11-19 00:49:54,456 - INFO - batch: 5
2021-11-19 00:49:54,456 - INFO - gpu: 0
2021-11-19 00:49:54,457 - INFO - data_path: /data/SC_Dataset
2021-11-19 00:49:54,457 - INFO - ckp_path: ./checkpoints_debug
2021-11-19 00:49:54,458 - INFO - visualization_path: visualization
2021-11-19 00:49:54,458 - INFO - model_type: MMNet
2021-11-19 00:49:54,459 - INFO - ckp_name: assets/model/mmnet_fcnresnet101_spair.pth
2021-11-19 00:49:54,459 - INFO - log_path: ./logs/
2021-11-19 00:49:54,460 - INFO - resize: 224,320
2021-11-19 00:49:54,460 - INFO - max_kps_num: 50
2021-11-19 00:49:54,461 - INFO - split_type: test
2021-11-19 00:49:54,461 - INFO - alpha: 0.05
2021-11-19 00:49:54,462 - INFO - resolution: 2
2021-11-19 00:49:54,462 - INFO - Options all listed.------------------
2021-11-19 00:49:54,463 - INFO - ckp file: assets/model/mmnet_fcnresnet101_spair.pth
2021-11-19 00:50:04,950 - INFO - [    0/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] aeroplane
2021-11-19 00:50:04,953 - INFO - [    1/12234]: 	 [Pair PCK: 0.100]	[Average: 0.217] aeroplane
2021-11-19 00:50:04,956 - INFO - [    2/12234]: 	 [Pair PCK: 0.308]	[Average: 0.247] aeroplane
2021-11-19 00:50:04,958 - INFO - [    3/12234]: 	 [Pair PCK: 0.364]	[Average: 0.276] aeroplane
2021-11-19 00:50:04,960 - INFO - [    4/12234]: 	 [Pair PCK: 0.000]	[Average: 0.221] aeroplane
2021-11-19 00:50:05,575 - INFO - [    5/12234]: 	 [Pair PCK: 0.200]	[Average: 0.217] aeroplane
2021-11-19 00:50:05,577 - INFO - [    6/12234]: 	 [Pair PCK: 0.250]	[Average: 0.222] aeroplane
2021-11-19 00:50:05,580 - INFO - [    7/12234]: 	 [Pair PCK: 0.308]	[Average: 0.233] aeroplane
2021-11-19 00:50:05,583 - INFO - [    8/12234]: 	 [Pair PCK: 0.182]	[Average: 0.227] aeroplane
2021-11-19 00:50:05,585 - INFO - [    9/12234]: 	 [Pair PCK: 0.636]	[Average: 0.268] aeroplane
2021-11-19 00:50:06,153 - INFO - [   10/12234]: 	 [Pair PCK: 0.667]	[Average: 0.304] aeroplane
2021-11-19 00:50:06,156 - INFO - [   11/12234]: 	 [Pair PCK: 0.385]	[Average: 0.311] aeroplane
2021-11-19 00:50:06,158 - INFO - [   12/12234]: 	 [Pair PCK: 0.455]	[Average: 0.322] aeroplane
2021-11-19 00:50:06,160 - INFO - [   13/12234]: 	 [Pair PCK: 0.250]	[Average: 0.317] aeroplane
2021-11-19 00:50:06,163 - INFO - [   14/12234]: 	 [Pair PCK: 0.615]	[Average: 0.337] aeroplane
2021-11-19 00:50:06,731 - INFO - [   15/12234]: 	 [Pair PCK: 0.000]	[Average: 0.316] aeroplane
...
2021-11-19 01:13:47,264 - INFO - [12216/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,265 - INFO - [12217/12234]: 	 [Pair PCK: 0.200]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,266 - INFO - [12218/12234]: 	 [Pair PCK: 0.250]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,268 - INFO - [12219/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,837 - INFO - [12220/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,838 - INFO - [12221/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,848 - INFO - [12222/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,850 - INFO - [12223/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:47,853 - INFO - [12224/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,422 - INFO - [12225/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,424 - INFO - [12226/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,425 - INFO - [12227/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,427 - INFO - [12228/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,429 - INFO - [12229/12234]: 	 [Pair PCK: 0.222]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,896 - INFO - [12230/12234]: 	 [Pair PCK: 0.333]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,899 - INFO - [12231/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,899 - INFO - [12232/12234]: 	 [Pair PCK: 0.000]	[Average: 0.333] tvmonitor
2021-11-19 01:13:48,901 - INFO - [12233/12234]: 	 [Pair PCK: 0.111]	[Average: 0.333] tvmonitor
    opened by tjyuyao 1
Releases(v0.1.0)
Owner
joey zhao
Master in Computer Sciences and Technology at Fudan University
joey zhao
GitHub Repository
Code for the KDD 2021 paper 'Filtration Curves for Graph Representation'

Filtration Curves for Graph Representation This repository provides the code from the KDD'21 paper Filtration Curves for Graph Representation. Depende

Machine Learning and Computational Biology Lab 16 Oct 16, 2022
A PyTorch implementation of Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks

SVHNClassifier-PyTorch A PyTorch implementation of Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks If

Potter Hsu 182 Jan 03, 2023
Let Python optimize the best stop loss and take profits for your TradingView strategy.

TradingView Machine Learning TradeView is a free and open source Trading View bot written in Python. It is designed to support all major exchanges. It

Robert Roman 473 Jan 09, 2023
Simply enable or disable your Nvidia dGPU

EnvyControl (WIP) Simply enable or disable your Nvidia dGPU Usage First clone this repo and install envycontrol with sudo pip install . CLI Turn off y

Victor Bayas 292 Jan 03, 2023
We have made you a wrapper you can't refuse

We have made you a wrapper you can't refuse We have a vibrant community of developers helping each other in our Telegram group. Join us! Stay tuned fo

20.6k Jan 09, 2023
GE2340 project source code without credentials.

GE2340-Project-Public GE2340 project source code without credentials. Run the bot.py to start the bot Telegram: @jasperwong_ge2340_bot If the bot does

0 Feb 10, 2022
This is an official implementation for "Video Swin Transformers".

Video Swin Transformer By Ze Liu*, Jia Ning*, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin and Han Hu. This repo is the official implementation of "V

Swin Transformer 981 Jan 03, 2023
Geometric Vector Perceptron --- a rotation-equivariant GNN for learning from biomolecular structure

Geometric Vector Perceptron Code to accompany Learning from Protein Structure with Geometric Vector Perceptrons by B Jing, S Eismann, P Suriana, RJL T

Dror Lab 85 Dec 29, 2022
Code from PropMix, accepted at BMVC'21

PropMix: Hard Sample Filtering and Proportional MixUp for Learning with Noisy Labels This repository is the official implementation of Hard Sample Fil

6 Dec 21, 2022
CURL: Contrastive Unsupervised Representations for Reinforcement Learning

CURL Rainbow Status: Archive (code is provided as-is, no updates expected) This is an implementation of CURL: Contrastive Unsupervised Representations

Aravind Srinivas 46 Dec 12, 2022
Source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated Recurrent Memory Network

KaGRMN-DSG_ABSA This repository contains the PyTorch source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated

XingBowen 4 May 20, 2022
Transformer model implemented with Pytorch

transformer-pytorch Transformer model implemented with Pytorch Attention is all you need-[Paper] Architecture Self-Attention self_attention.py class

Mingu Kang 12 Sep 03, 2022
Plugin adapted from Ultralytics to bring YOLOv5 into Napari

napari-yolov5 Plugin adapted from Ultralytics to bring YOLOv5 into Napari. Training and detection can be done using the GUI. Training dataset must be

2 May 05, 2022
Official implement of Paper：A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sening images

A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images 深度监督影像融合网络DSIFN用于高分辨率双时相遥感影像变化检测 Of

Chenxiao Zhang 135 Dec 19, 2022
Norm-based Analysis of Transformer

Norm-based Analysis of Transformer Implementations for 2 papers introducing to analyze Transformers using vector norms: Kobayashi+'20 Attention is Not

Goro Kobayashi 52 Dec 05, 2022
Easy and comprehensive assessment of predictive power, with support for neuroimaging features

Documentation: https://raamana.github.io/neuropredict/ News As of v0.6, neuropredict now supports regression applications i.e. predicting continuous t

Pradeep Reddy Raamana 93 Nov 29, 2022
Instance Semantic Segmentation List

Instance Semantic Segmentation List This repository contains lists of state-or-art instance semantic segmentation works. Papers and resources are list

bighead 87 Mar 06, 2022
Supporting code for the paper "Dangers of Bayesian Model Averaging under Covariate Shift"

Dangers of Bayesian Model Averaging under Covariate Shift This repository contains the code to reproduce the experiments in the paper Dangers of Bayes

Pavel Izmailov 25 Sep 21, 2022
Effect of Different Encodings and Distance Functions on Quantum Instance-based Classifiers

Effect of Different Encodings and Distance Functions on Quantum Instance-based Classifiers The repository contains the code to reproduce the experimen

Alessandro Berti 4 Aug 24, 2022
Luminous is a framework for testing the performance of Embodied AI (EAI) models in indoor tasks.

Luminous is a framework for testing the performance of Embodied AI (EAI) models in indoor tasks. Generally, we intergrete different kind of functional

28 Jan 08, 2023