Pytorch implementation of MaskFlownet

Related tags

Deep Learningcomputer-visiondeep-learningcnndnnpytorchconvolutional-neural-networksoptical-flowkitti-datasetmaskflownet
Overview

MaskFlownet-Pytorch

Unofficial PyTorch implementation of MaskFlownet (https://github.com/microsoft/MaskFlownet).

Tested with:

  • PyTorch 1.5.0
  • CUDA 10.1

Install

The correlation package must be installed first:

cd model/correlation_package
python setup.py install

Inference

Right now, I implemented the inference script for KITTI 2012/2015, MPI Sintel and FlyingChairs.

python predict.py CONFIG -c CHECKPOINT --dataset_cfg DATASET -f ROOT_FOLDER [-b BATCH_SIZE]

For example:

  • python predict.py MaskFlownet.yaml -c 5adNov03-0005_1000000.pth --dataset_cfg sintel.yaml -f ./SINTEL -b 4
  • python predict.py MaskFlownet.yaml -c 8caNov12-1532_300000.pth --dataset_cfg kitti.yaml -f ./KITTI -b 4
  • python predict.py MaskFlownet_S.yaml -c 771Sep25-0735_500000.pth --dataset_cfg chairs.yaml -f ./FLYINGCHAIRS -b 4
  • python predict.py MaskFlownet_S.yaml -c dbbSep30-1206_1000000.pth --dataset_cfg sintel.yaml -f ./SINTEL -b 4

Differences with the original implementation

The results are slightly different from the original implementation:

Checkpoint Network Implementation KITTI2012 KITTI2015 Sintel Clean Sintel Final FlyingChairs
771Sep25 MaskFlownet_S

Original AEPE:
PyTorch AEPE:

4.12
4.18

11.52
11.82

3.38
3.38

4.71
4.70

1.84
1.83
dbbSep30 MaskFlownet_S

Original AEPE:
PyTorch AEPE:

1.27
1.28

1.92
1.93

2.76
2.78

3.29
3.32

2.36
2.36
5adNov03 MaskFlownet

Original AEPE:
PyTorch AEPE:

1.16
1.18

1.66
1.68

2.58
2.59

3.14
3.17

2.23
2.23
8caNov12 MaskFlownet

Original AEPE:
PyTorch AEPE:

0.82
0.82

1.38
1.38

4.34
4.40

5.27
5.33

4.01
3.99

Examples

KITTI Original implementation:

original_visualization

KITTI This implementation:

this_visualization

Sintel Original implementation:

original_visualization

Sintel This implementation:

this_visualization

FlyingChairs Original implementation:

original_visualization

FlyingChairs This implementation:

this_visualization

Notes

If you use my implementation for training, it might happen that you encounter this error:

CUDA error: an illegal memory access was encountered

This is due to a bug in the torchvision implementation of deformable convolutions. (still present in version 0.7.0)

To solve it, you need to use the nightly version of torchvision.

Acknowledgment

Original MXNet implementation: here

correlation_package was taken from flownet2

Owner
Daniele Cattaneo
PostDoc at University of Freiburg. Focus on deep learning for vision-based and LiDAR-based localization, self-driving cars, and sensor fusion.
Daniele Cattaneo
GitHub Repository
DA2Lite is an automated model compression toolkit for PyTorch.

DA2Lite (Deep Architecture to Lite) is a toolkit to compress and accelerate deep network models. ⭐ Star us on GitHub — it helps!! Frameworks & Librari

Sinhan Kang 7 Mar 22, 2022
Automatic Idiomatic Expression Detection

IDentifier of Idiomatic Expressions via Semantic Compatibility (DISC) An Idiomatic identifier that detects the presence and span of idiomatic expressi

5 Jun 09, 2022
Dataset para entrenamiento de yoloV3 para 4 clases

Deteccion de objetos en video Este repo basado en el proyecto PyTorch YOLOv3 para correr detección de objetos sobre video. Construí sobre este proyect

1 Nov 01, 2021
Code implementing "Improving Deep Learning Interpretability by Saliency Guided Training"

Saliency Guided Training Code implementing "Improving Deep Learning Interpretability by Saliency Guided Training" by Aya Abdelsalam Ismail, Hector Cor

8 Sep 22, 2022
An unofficial styleguide and best practices summary for PyTorch

A PyTorch Tools, best practices & Styleguide This is not an official style guide for PyTorch. This document summarizes best practices from more than a

IgorSusmelj 1.5k Jan 05, 2023
The implementation of "Optimizing Shoulder to Shoulder: A Coordinated Sub-Band Fusion Model for Real-Time Full-Band Speech Enhancement"

SF-Net for fullband SE This is the repo of the manuscript "Optimizing Shoulder to Shoulder: A Coordinated Sub-Band Fusion Model for Real-Time Full-Ban

Guochen Yu 36 Dec 02, 2022
Repository for "Space-Time Correspondence as a Contrastive Random Walk" (NeurIPS 2020)

Space-Time Correspondence as a Contrastive Random Walk This is the repository for Space-Time Correspondence as a Contrastive Random Walk, published at

A. Jabri 239 Dec 27, 2022
A CROSS-MODAL FUSION NETWORK BASED ON SELF-ATTENTION AND RESIDUAL STRUCTURE FOR MULTIMODAL EMOTION RECOGNITION

CFN-SR A CROSS-MODAL FUSION NETWORK BASED ON SELF-ATTENTION AND RESIDUAL STRUCTURE FOR MULTIMODAL EMOTION RECOGNITION The audio-video based multimodal

skeleton 15 Sep 26, 2022
GPU-Accelerated Deep Learning Library in Python

Hebel GPU-Accelerated Deep Learning Library in Python Hebel is a library for deep learning with neural networks in Python using GPU acceleration with

Hannes Bretschneider 1.2k Dec 21, 2022
Implementation of CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification

CrossViT : Cross-Attention Multi-Scale Vision Transformer for Image Classification This is an unofficial PyTorch implementation of CrossViT: Cross-Att

Rishikesh (ऋषिकेश) 103 Nov 25, 2022
Sdf sparse conv - Deep Learning on SDF for Classifying Brain Biomarkers

Deep Learning on SDF for Classifying Brain Biomarkers To reproduce the results f

1 Jan 25, 2022
Preparation material for Dropbox interviews

Dropbox-Onsite-Interviews A guide for the Dropbox onsite interview! The Dropbox interview question bank is very small. The bank has been in a Chinese

386 Dec 31, 2022
Multiple paper open-source codes of the Microsoft Research Asia DKI group

📫 Paper Code Collection (MSRA DKI Group) This repo hosts multiple open-source codes of the Microsoft Research Asia DKI Group. You could find the corr

Microsoft 249 Jan 08, 2023
[ICCV 2021] Our work presents a novel neural rendering approach that can efficiently reconstruct geometric and neural radiance fields for view synthesis.

MVSNeRF Project page | Paper This repository contains a pytorch lightning implementation for the ICCV 2021 paper: MVSNeRF: Fast Generalizable Radiance

Anpei Chen 529 Dec 30, 2022
Neural Scene Flow Fields using pytorch-lightning, with potential improvements

nsff_pl Neural Scene Flow Fields using pytorch-lightning. This repo reimplements the NSFF idea, but modifies several operations based on observation o

AI葵 178 Dec 21, 2022
Official Implement of CVPR 2021 paper “Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Counting”

RGBT Crowd Counting Lingbo Liu, Jiaqi Chen, Hefeng Wu, Guanbin Li, Chenglong Li, Liang Lin. "Cross-Modal Collaborative Representation Learning and a L

37 Dec 08, 2022
Code for CVPR2021 paper 'Where and What? Examining Interpretable Disentangled Representations'.

PS-SC GAN This repository contains the main code for training a PS-SC GAN (a GAN implemented with the Perceptual Simplicity and Spatial Constriction c

Xinqi/Steven Zhu 40 Dec 16, 2022
Tensorflow implementation of Human-Level Control through Deep Reinforcement Learning

Human-Level Control through Deep Reinforcement Learning Tensorflow implementation of Human-Level Control through Deep Reinforcement Learning. This imp

Devsisters Corp. 2.4k Dec 26, 2022
Machine Learning Privacy Meter: A tool to quantify the privacy risks of machine learning models with respect to inference attacks, notably membership inference attacks

ML Privacy Meter Machine learning is playing a central role in automated decision making in a wide range of organization and service providers. The da

Data Privacy and Trustworthy Machine Learning Research Lab 357 Jan 06, 2023
This repository contains the source code of our work on designing efficient CNNs for computer vision

Efficient networks for Computer Vision This repo contains source code of our work on designing efficient networks for different computer vision tasks:

Sachin Mehta 386 Nov 26, 2022