Neural Residual Flow Fields for Efficient Video Representations

1. Download MPI sintel dataset

Download MPI sintel dataset from here

2. GMA optical flow estimator

To obtain optical flow estimations for pretraining, we are using GMA from here. Note that it dose not have to do with our identity.

3. Training

Training neural residual flow fields (NRFF)

# frame 0 - 6
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 0 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start0_jq98_hf96
# frame 7 - 13
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 7 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start7_jq98_hf96
# frame 14 - 20
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 14 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start14_jq98_hf96
# frame 21 - 27
python train_video_flow_midkey.py --use-estimator --lr 0.0005 --training-step 30000 --data-dir {sintel dataset training directory} --video-name alley_1 --start-frame 21 --num-frames 7 --jpeg-quality 98 --hidden-features 96 --use-estimator --tag start21_jq98_hf96

Training baseline (SIREN)

python train_video.py --data-dir {sintel dataset training directory} --video-name alley_1 --hidden-features 256 --num-frames 28 --lr 0.001 --training-step 30000 --tag baseline_siren_hf256

4. Examples

alley_2.mp4

HoneyBee.mp4

Eff video representation - Efficient video representation through neural fields

Related tags

Overview

Neural Residual Flow Fields for Efficient Video Representations

1. Download MPI sintel dataset

2. GMA optical flow estimator

3. Training

4. Examples

Owner

Attack classification models with transferability, black-box attack; unrestricted adversarial attacks on imagenet

Python implementation of cover trees, near-drop-in replacement for scipy.spatial.kdtree

The code release of paper Low-Light Image Enhancement with Normalizing Flow

An implementation of "MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing" (ICML 2019).

Code for our paper A Transformer-Based Feature Segmentation and Region Alignment Method For UAV-View Geo-Localization,

Yolov5+SlowFast: Realtime Action Detection Based on PytorchVideo

Transfer Learning Remote Sensing

Identifying a Training-Set Attack’s Target Using Renormalized Influence Estimation

Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image classification, in Pytorch

PyTorch implementation of Trust Region Policy Optimization

Keras implementation of Real-Time Semantic Segmentation on High-Resolution Images

A Deep Learning based project for creating line art portraits.

This repo contains the implementation of YOLOv2 in Keras with Tensorflow backend.

TransVTSpotter: End-to-end Video Text Spotter with Transformer

TransFGU: A Top-down Approach to Fine-Grained Unsupervised Semantic Segmentation

Transformers based fully on MLPs

PyTorch DepthNet Training on Still Box dataset

A repo for Causal Imitation Learning under Temporally Correlated Noise

Code release of paper "Deep Multi-View Stereo gone wild"

Dynamical Wasserstein Barycenters for Time Series Modeling