CVPR 2021 Challenge on Super-Resolution Space

Overview

Learning the Super-Resolution Space Challenge
NTIRE 2021 at CVPR

Learning the Super-Resolution Space challenge is held as a part of the 6th edition of NTIRE: New Trends in Image Restoration and Enhancement workshop in conjunction with CVPR 2021. The goal of this challenge is to develop a super-resolution method that can actively sample from the space of plausible super-resolutions.

How to participate?

To participate in this challenge, please sign up using the following link and clone this repo to benchmark your results. Challenge participants can submit their paper to this CVPR 2021 Workshop.

CVPR 2021 Challenge Signup

Tackling the ill-posed nature of Super-Resolution

CVPR 2021 Challenge

Usually, super-resolution (SR) is trained using pairs of high- and low-resolution images. Infinitely many high-resolution images can be downsampled to the same low-resolution image. That means that the problem is ill-posed and cannot be inverted with a deterministic mapping. Instead, one can frame the SR problem as learning a stochastic mapping, capable of sampling from the space of plausible high-resolution images given a low-resolution image. This problem has been addressed in recent works [1, 2, 3]. The one-to-many stochastic formulation of the SR problem allows for a few potential advantages:

  • The development of more robust learning formulations that better accounts for the ill-posed nature of the SR problem.
  • Multiple predictions can be sampled and compared.
  • It opens the potential for controllable exploration and editing in the space of SR predictions.
Super-Resolution with Normalizing Flow Explorable SR Screenshot 2021-01-12 at 16 05 43
[Paper] [Project] [Paper] [Project] [Paper] [Project]
[1] SRFlow: Learning the Super-Resolution Space with Normalizing Flow. Lugmayr et al., ECCV 2020. [2] Explorable Super-Resolution. Bahat & Michaeli, CVPR 2020. [3] DeepSEE: Deep Disentangled Semantic Explorative Extreme Super-Resolution. Bühler et al., ACCV 2020.

CVPR 2021 Challenge on Learning the Super-Resolution Space

We organize this challenge to stimulate research in the emerging area of learning one-to-many SR mappings that are capable of sampling from the space of plausible solutions. Therefore the task is to develop a super-resolution method that:

  1. Each individual SR prediction should achieve highest possible photo-realism, as perceived by humans.
  2. Is capable of sampling an arbitrary number of SR images capturing meaningful diversity, corresponding to the uncertainty induced by the ill-posed nature of the SR problem together with image priors.
  3. Each individual SR prediction should be consistent with the input low-resolution image.

The challenge contains two tracks, targeting 4X and 8X super-resolution respectively. You can download the training and validation data in the table below. At a later stage, the low-resolution of the test set will be released.

  Training Validation
  Low-Resolution High-Resolution Low-Resolution High-Resolution
Track 4X 4X LR Train 4X HR Train 4X LR Valid 4X HR Valid
Track 8X 8X LR Train 8X HR Train 8X LR Valid 8X HR Valid

Challenge Rules

To guide the research towards useful and generalizable techniques, submissions need to adhere to the following rules. All participants must submit code of their solution along with the final results.

  • The method must be able to generate an arbitrary number of diverse samples. That is, your method cannot be limited to a maximum number of different SR samples (corresponding to e.g. a certain number of different output network heads).
  • All SR samples must be generated by a single model. That is, no ensembles are allowed.
  • No self-ensembles during inference (e.g. flipping and rotation).
  • All SR samples must be generated using the same hyper-parameters. That is, the generated SR samples shall not be the result of different choices of hyper-parameters during inference.
  • We accept submissions of deterministic methods. However, they will naturally score zero in the diversity measure and therefore not be able to win the challenge.
  • Other than the validation and test split of the DIV2k dataset, any training data or pre-training is allowed. You are not allowed to use DIV2K validation or test sets (low- and high-resolution images) for training.

Evaluation Protocol

A method is evaluated by first predicting a set of 10 randomly sampled SR images for each low-resolution image in the dataset. From this set of images, evaluation metrics corresponding to the three criteria above will be considered. The participating methods will be ranked according to each metric. These ranks will then be combined into a final score. The three evaluation metrics are described next.

git clone --recursive https://github.com/andreas128/NTIRE21_Learning_SR_Space.git
python3 measure.py OutName path/to/Ground-Truch path/to/Super-Resolution n_samples scale_factor

# n_samples = 10
# scale_factor = 4 for 4X and 8 for 8X

How we measure Photo-realism?

To assess the photo-realism, a human study will be performed on the test set for the final submission.

Automatically assessing the photo-realism and image quality is an extremely difficult task. All existing methods have severe shortcomings. As a very rough guide, you can use the LPIPS distance. Note: LPIPS will not be used to score photo-realism of you final submission. So beware of overfitting to LPIPS, as that can lead to worse results. LPIPS is integrated in our provided toolkit in measure.py.

How we measure the spanning of the SR Space?

The samples of the developed method should provide a meaningful diversity. To measure that, we define the following score. We sample 10 images, densely calculate a metric between the samples and the ground truth. To obtain the local best we pixel-wise select the best score out of the 10 samples and take the full image's average. The global best is obtained by averaging the whole image's score and selecting the best. Finally, we calculate the score using the following formula:

score = (global best - local best)/(global best) * 100

ESRGAN SRFlow
Track 4X 0 25.36
Track 8X 0 10.62

How we measure the Low Resolution Consistency

To measure how much information is preserved in the super-resloved image from the low-resolution image, we measure the LR-PSNR. The goal in this challenge is to obtain a LR-PSNR of 45dB. All approaches that have an average PSNR above this value will be ranked equally in terms of this criteria.

ESRGAN SRFlow
Track 4X 39.01 49.91
Track 8X 31.28 50.0

Important Dates

Date Event
2021.03.01 Final test data release (inputs only)
2021.03.08 test result submission deadline
2021.03.09 fact sheet / code / model submission deadline
2021.03.11 test preliminary score release to the participants
2021.03.28 challenge paper submission deadline
2021.04.13 camera-ready deadline
2021.06.15 workshop day

Submission of Final Test Results

After the final testing phase, participants will be asked to submit:

  • SR predictions on the test set.
  • Code.
  • A fact sheet describing their method.

Details will follow when the test phase starts ...

Issues and questions

In case of any questions about the challenge or the toolkit, feel free to open an issue on Github.

Organizers

CVPR 2021 NTIRE Terms and conditions

The terms and conditions for participating in the challenge are provided here

How to participate?

To participate in this challenge, please sign up using following link and clone this repo to benchmark your results. Challenge participants can submit their paper to this CVPR 2021 Workshop.

CVPR 2021 Challenge Signup

Owner
andreas
andreas
High performance Cross-platform Inference-engine, you could run Anakin on x86-cpu,arm, nv-gpu, amd-gpu,bitmain and cambricon devices.

Anakin2.0 Welcome to the Anakin GitHub. Anakin is a cross-platform, high-performance inference engine, which is originally developed by Baidu engineer

514 Dec 28, 2022
Just Randoms Cats with python

Random-Cat Just Randoms Cats with python.

OriCode 2 Dec 21, 2021
Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation

Unseen Object Clustering: Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation Introduction In this work, we propose a new method

NVIDIA Research Projects 132 Dec 13, 2022
Code Release for Learning to Adapt to Evolving Domains

EAML Code release for "Learning to Adapt to Evolving Domains" (NeurIPS 2020) Prerequisites PyTorch = 0.4.0 (with suitable CUDA and CuDNN version) tor

23 Dec 07, 2022
This repository contains the code for the paper in EMNLP 2021: "HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression".

HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression This repository contains the code for the paper in EM

Chenhe Dong 2 Mar 24, 2022
The official code of "SCROLLS: Standardized CompaRison Over Long Language Sequences".

SCROLLS This repository contains the official code of the paper: "SCROLLS: Standardized CompaRison Over Long Language Sequences". Links Official Websi

TAU NLP Group 39 Dec 23, 2022
Implementation of "Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency"

Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency (ICCV2021) Paper Link: https://arxiv.org/abs/2107.11355 This implementation bui

32 Nov 17, 2022
A check for whether the dependency jobs are all green.

alls-green A check for whether the dependency jobs are all green. Why? Do you have more than one job in your GitHub Actions CI/CD workflows setup? Do

Re:actors 33 Jan 03, 2023
Measuring Coding Challenge Competence With APPS

Measuring Coding Challenge Competence With APPS This is the repository for Measuring Coding Challenge Competence With APPS by Dan Hendrycks*, Steven B

Dan Hendrycks 218 Dec 27, 2022
利用python脚本实现微信、支付宝账单的合并,并保存到excel文件实现自动记账,可查看可视化图表。

KeepAccounts_v2.0 KeepAccounts.exe和其配套表格能够实现微信、支付宝官方导出账单的读取合并,为每笔帐标记类型,并按月份和类型生成可视化图表。再也不用消费一笔记一笔,每月仅需10分钟,记好所有的帐。 作者: MickLife Bilibili: https://spac

159 Jan 01, 2023
python debugger and anti-vm that checks if you're in a virtual machine or if someones trying to debug your file

Anti-Debug was made by Love ❌ code ✅ 🎉 ・What it checks for ・ Kills tools that can be used to debug your file ・ Exits if ran in vm (supports different

Rdimo 31 Aug 09, 2022
This is a Keras-based Python implementation of DeepMask- a complex deep neural network for learning object segmentation masks

NNProject - DeepMask This is a Keras-based Python implementation of DeepMask- a complex deep neural network for learning object segmentation masks. Th

189 Nov 16, 2022
Locationinfo - A script helps the user to show network information such as ip address

Description This script helps the user to show network information such as ip ad

Roxcoder 1 Dec 30, 2021
Tool which allow you to detect and translate text.

Text detection and recognition This repository contains tool which allow to detect region with text and translate it one by one. Description Two pretr

Damian Panek 176 Nov 28, 2022
Deep Text Search is an AI-powered multilingual text search and recommendation engine with state-of-the-art transformer-based multilingual text embedding (50+ languages).

Deep Text Search - AI Based Text Search & Recommendation System Deep Text Search is an AI-powered multilingual text search and recommendation engine w

19 Sep 29, 2022
[TIP 2021] SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

SADRNet Paper link: SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction Requirements python

Multimedia Computing Group, Nanjing University 99 Dec 30, 2022
PatchMatch-RL: Deep MVS with Pixelwise Depth, Normal, and Visibility

PatchMatch-RL: Deep MVS with Pixelwise Depth, Normal, and Visibility Jae Yong Lee, Joseph DeGol, Chuhang Zou, Derek Hoiem Installation To install nece

31 Apr 19, 2022
Companion repository to the paper accepted at the 4th ACM SIGSPATIAL International Workshop on Advances in Resilient and Intelligent Cities

Transfer learning approach to bicycle sharing systems station location planning using OpenStreetMap Companion repository to the paper accepted at the

Politechnika Wrocławska - repozytorium dla informatyków 4 Oct 24, 2022
Using machine learning to predict and analyze high and low reader engagement for New York Times articles posted to Facebook.

How The New York Times can increase Engagement on Facebook Using machine learning to understand characteristics of news content that garners "high" Fa

Jessica Miles 0 Sep 16, 2021
Implements Gradient Centralization and allows it to use as a Python package in TensorFlow

Gradient Centralization TensorFlow This Python package implements Gradient Centralization in TensorFlow, a simple and effective optimization technique

Rishit Dagli 101 Nov 01, 2022