Open Source Light Field Toolbox for Super-Resolution

Overview

BasicLFSR

BasicLFSR is an open-source and easy-to-use Light Field (LF) image Super-Ressolution (SR) toolbox based on PyTorch, including a collection of papers on LF image SR and a benchmark to comprehensively evaluate the performance of existing methods. We also provided simple pipelines to train/valid/test state-of-the-art methods to get started quickly, and you can transform your methods into the benchmark.

Note: This repository will be updated on a regular basis, and the pretrained models of existing methods will be open-sourced one after another. So stay tuned!

Methods

Methods Paper Repository
LFSSR Light Field Spatial Super-Resolution Using Deep Efficient Spatial-Angular Separable Convolution. TIP2018 spatialsr/
DeepLightFieldSSR
resLF Residual Networks for Light Field Image Super-Resolution. CVPR2019 shuozh/resLF
HDDRNet High-Dimensional Dense Residual Convolutional Neural Network for Light Field Reconstruction. TPAMI2019 monaen/
LightFieldReconstruction
LF-InterNet Spatial-Angular Interaction for Light Field Image Super-Resolution. ECCV2019 YingqianWang/
LF-InterNet
LFSSR-ATO Light field spatial super-resolution via deep combinatorial geometry embedding and structural consistency regularization. CVPR2020 jingjin25/
LFSSR-ATO
LF-DFnet Light field image super-resolution using deformable convolution. TIP2020 YingqianWang/
LF-DFnet
MEG-Net End-to-End Light Field Spatial Super-Resolution Network using Multiple Epipolar Geometry. TIP2021 shuozh/MEG-Net

Datasets

We used the EPFL, HCInew, HCIold, INRIA and STFgantry datasets for both training and test. Please first download our datasets via Baidu Drive (key:7nzy) or OneDrive, and place the 5 datasets to the folder ./datasets/.

  • After downloading, you should find following structure:

    ├──./datasets/
    │    ├── EPFL
    │    │    ├── training
    │    │    │    ├── Bench_in_Paris.mat
    │    │    │    ├── Billboards.mat
    │    │    │    ├── ...
    │    │    ├── test
    │    │    │    ├── Bikes.mat
    │    │    │    ├── Books__Decoded.mat
    │    │    │    ├── ...
    │    ├── HCI_new
    │    ├── ...
    
  • Run Generate_Data_for_Training.m to generate training data. The generated data will be saved in ./data_for_train/ (SR_5x5_2x, SR_5x5_4x).

  • Run Generate_Data_for_Test.m to generate test data. The generated data will be saved in ./data_for_test/ (SR_5x5_2x, SR_5x5_4x).

Benchmark

We benchmark several methods on above datasets, and PSNR and SSIM metrics are used for quantitative evaluation.

PSNR and SSIM values achieved by different methods for 2xSR:

Method Scale #Params. EPFL HCInew HCIold INRIA STFgantry Average
Bilinear x2 -- 28.479949/0.918006 30.717944/0.919248 36.243278/0.970928 30.133901/0.945545 29.577468/0.931030 31.030508/0.936951
Bicubic x2 -- 29.739509/0.937581 31.887011/0.935637 37.685776/0.978536 31.331483/0.957731 31.062631/0.949769 32.341282/0.951851
VDSR x2
EDSR x2 33.088922/0.962924 34.828374/0.959156 41.013989/0.987400 34.984982/0.976397 36.295865/0.981809
RCSN x2
resLF x2
LFSSR x2 33.670594/0.974351 36.801555/0.974910 43.811050/0.993773 35.279443/0.983202 37.943969/0.989818
LF-ATO x2 34.271635/0.975711 37.243620/0.976684 44.205264/0.994202 36.169943/0.984241 39.636445/0.992862
LF-InterNet x2
LF-DFnet x2
MEG-Net x2
LFT x2

PSNR and SSIM values achieved by different methods for 4xSR:

Method Scale #Params. EPFL HCInew HCIold INRIA STFgantry Average
Bilinear x4 -- 24.567490/0.815793 27.084949/0.839677 31.688225/0.925630 26.226265/0.875682 25.203262/0.826105 26.954038/0.856577
Bicubic x4 -- 25.264206/0.832389 27.714905/0.851661 32.576315/0.934428 26.951718/0.886740 26.087451/0.845230 27.718919/0.870090
VDSR x4
EDSR x4
RCSN x4
resLF x4
LFSSR x4
LF-ATO x4
LF-InterNet x4
LF-DFnet x4
MEG-Net x4
LFT x4

Train

  • Run train.py to perform network training. Example for training [model_name] on 5x5 angular resolution for 2x/4x SR:
    $ python train.py --model_name [model_name] --angRes 5 --scale_factor 2 --batch_size 8
    $ python train.py --model_name [model_name] --angRes 5 --scale_factor 4 --batch_size 4
    
  • Checkpoints and Logs will be saved to ./log/, and the ./log/ has following structure:
    ├──./log/
    │    ├── SR_5x5_2x
    │    │    ├── [dataset_name]
    │    │         ├── [model_name]
    │    │         │    ├── [model_name]_log.txt
    │    │         │    ├── checkpoints
    │    │         │    │    ├── [model_name]_5x5_2x_epoch_01_model.pth
    │    │         │    │    ├── [model_name]_5x5_2x_epoch_02_model.pth
    │    │         │    │    ├── ...
    │    │         │    ├── results
    │    │         │    │    ├── VAL_epoch_01
    │    │         │    │    ├── VAL_epoch_02
    │    │         │    │    ├── ...
    │    │         ├── [other_model_name]
    │    │         ├── ...
    │    ├── SR_5x5_4x
    

Test

  • Run test.py to perform network inference. Example for test [model_name] on 5x5 angular resolution for 2x/4xSR:

    $ python test.py --model_name [model_name] --angRes 5 --scale_factor 2  
    $ python test.py --model_name [model_name] --angRes 5 --scale_factor 4 
    
  • The PSNR and SSIM values of each dataset will be saved to ./log/, and the ./log/ is following structure:

    ├──./log/
    │    ├── SR_5x5_2x
    │    │    ├── [dataset_name]
    │    │        ├── [model_name]
    │    │        │    ├── [model_name]_log.txt
    │    │        │    ├── checkpoints
    │    │        │    │   ├── ...
    │    │        │    ├── results
    │    │        │    │    ├── Test
    │    │        │    │    │    ├── evaluation.xls
    │    │        │    │    │    ├── [dataset_1_name]
    │    │        │    │    │    │    ├── [scene_1_name]
    │    │        │    │    │    │    │    ├── [scene_1_name]_CenterView.bmp
    │    │        │    │    │    │    │    ├── [scene_1_name]_SAI.bmp
    │    │        │    │    │    │    │    ├── views
    │    │        │    │    │    │    │    │    ├── [scene_1_name]_0_0.bmp
    │    │        │    │    │    │    │    │    ├── [scene_1_name]_0_1.bmp
    │    │        │    │    │    │    │    │    ├── ...
    │    │        │    │    │    │    │    │    ├── [scene_1_name]_4_4.bmp
    │    │        │    │    │    │    ├── [scene_2_name]
    │    │        │    │    │    │    ├── ...
    │    │        │    │    │    ├── [dataset_2_name]
    │    │        │    │    │    ├── ...
    │    │        │    │    ├── VAL_epoch_01
    │    │        │    │    ├── ...
    │    │        ├── [other_model_name]
    │    │        ├── ...
    │    ├── SR_5x5_4x
    

Recources

We provide some original super-resolved images and useful resources to facilitate researchers to reproduce the above results.

Other Recources

Contact

Any question regarding this work can be addressed to [email protected].

Owner
Squidward
Squidward
"3D Human Texture Estimation from a Single Image with Transformers", ICCV 2021

Texformer: 3D Human Texture Estimation from a Single Image with Transformers This is the official implementation of "3D Human Texture Estimation from

XiangyuXu 193 Dec 05, 2022
This is the source code for our ICLR2021 paper: Adaptive Universal Generalized PageRank Graph Neural Network.

GPRGNN This is the source code for our ICLR2021 paper: Adaptive Universal Generalized PageRank Graph Neural Network. Hidden state feature extraction i

Jianhao 92 Jan 03, 2023
Code repository for the paper Computer Vision User Entity Behavior Analytics

Computer Vision User Entity Behavior Analytics Code repository for "Computer Vision User Entity Behavior Analytics" Code Description dataset.csv As di

Sameer Khanna 2 Aug 20, 2022
MixRNet(Using mixup as regularization and tuning hyper-parameters for ResNets)

MixRNet(Using mixup as regularization and tuning hyper-parameters for ResNets) Using mixup data augmentation as reguliraztion and tuning the hyper par

Bhanu 2 Jan 16, 2022
Dyalog-apl-docset - Dyalog APL Dash Docset Generator

Dyalog APL Dash Docset Generator o alasa e kili sona kepeken tenpo lili a A Dash

Maciej Goszczycki 1 Jan 10, 2022
Source code for "Progressive Transformers for End-to-End Sign Language Production" (ECCV 2020)

Progressive Transformers for End-to-End Sign Language Production Source code for "Progressive Transformers for End-to-End Sign Language Production" (B

58 Dec 21, 2022
Fit Fast, Explain Fast

FastExplain Fit Fast, Explain Fast Installing pip install fast-explain About FastExplain FastExplain provides an out-of-the-box tool for analysts to

8 Dec 15, 2022
Based on the given clinical dataset, Predict whether the patient having Heart Disease or Not having Heart Disease

Heart_Disease_Classification Based on the given clinical dataset, Predict whether the patient having Heart Disease or Not having Heart Disease Dataset

Ashish 1 Jan 30, 2022
PyTorch common framework to accelerate network implementation, training and validation

pytorch-framework PyTorch common framework to accelerate network implementation, training and validation. This framework is inspired by works from MML

Dongliang Cao 3 Dec 19, 2022
Code & Experiments for "LILA: Language-Informed Latent Actions" to be presented at the Conference on Robot Learning (CoRL) 2021.

LILA LILA: Language-Informed Latent Actions Code and Experiments for Language-Informed Latent Actions (LILA), for using natural language to guide assi

Sidd Karamcheti 11 Nov 25, 2022
Modeling Category-Selective Cortical Regions with Topographic Variational Autoencoders

Modeling Category-Selective Cortical Regions with Topographic Variational Autoencoders

1 Oct 11, 2021
Simple torch.nn.module implementation of Alias-Free-GAN style filter and resample

Alias-Free-Torch Simple torch module implementation of Alias-Free GAN. This repository including Alias-Free GAN style lowpass sinc filter @filter.py A

이준혁(Junhyeok Lee) 64 Dec 22, 2022
NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go

NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go This repository provides our implementation of the CVPR 2021 paper NeuroMorp

Meta Research 35 Dec 08, 2022
This is code of book "Learn Deep Learning with PyTorch"

深度学习入门之PyTorch Learn Deep Learning with PyTorch 非常感谢您能够购买此书,这个github repository包含有深度学习入门之PyTorch的实例代码。由于本人水平有限,在写此书的时候参考了一些网上的资料,在这里对他们表示敬意。由于深度学习的技术在

Xingyu Liao 2.5k Jan 04, 2023
Graph Attention Networks

GAT Graph Attention Networks (Veličković et al., ICLR 2018): https://arxiv.org/abs/1710.10903 GAT layer t-SNE + Attention coefficients on Cora Overvie

Petar Veličković 2.6k Jan 05, 2023
Label-Free Model Evaluation with Semi-Structured Dataset Representations

Label-Free Model Evaluation with Semi-Structured Dataset Representations Prerequisites This code uses the following libraries Python 3.7 NumPy PyTorch

8 Oct 06, 2022
Using some basic methods to show linkages and transformations of robotic arms

roboticArmVisualizer Python GUI application to create custom linkages and adjust joint angles. In the future, I plan to add 2d inverse kinematics solv

Sandesh Banskota 1 Nov 19, 2021
《Train in Germany, Test in The USA: Making 3D Object Detectors Generalize》(CVPR 2020)

Train in Germany, Test in The USA: Making 3D Object Detectors Generalize This paper has been accpeted by Conference on Computer Vision and Pattern Rec

Xiangyu Chen 101 Jan 02, 2023
MDMM - Learning multi-domain multi-modality I2I translation

Multi-Domain Multi-Modality I2I translation Pytorch implementation of multi-modality I2I translation for multi-domains. The project is an extension to

Hsin-Ying Lee 107 Nov 04, 2022
Applying PVT to Semantic Segmentation

Applying PVT to Semantic Segmentation Here, we take MMSegmentation v0.13.0 as an example, applying PVTv2 to SemanticFPN. For details see Pyramid Visio

35 Nov 30, 2022