NCNN implementation of Real-ESRGAN. Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration.

Overview

Real-ESRGAN ncnn Vulkan

CI License: MIT Open issue Closed issue

This project is the ncnn implementation of Real-ESRGAN. Real-ESRGAN ncnn Vulkan heavily borrows from realsr-ncnn-vulkan. Many thanks to nihui, ncnn and realsr-ncnn-vulkan 😁

Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration. We also optimize it for anime images.

Contents


If Real-ESRGAN is helpful in your photos/projects, please help to this repo or recommend it to your friends. Thanks 😊
Other recommended projects:
▶️ Real-ESRGAN: A practical algorithm for general image restoration
▶️ GFPGAN: A practical algorithm for real-world face restoration
▶️ BasicSR: An open-source image and video restoration toolbox
▶️ facexlib: A collection that provides useful face-relation functions.
▶️ HandyView: A PyQt5-based image viewer that is handy for view and comparison.

📖 Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data

[Paper]   [Project Page]   [Demo]
Xintao Wang, Liangbin Xie, Chao Dong, Ying Shan
Tencent ARC Lab; Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences

TODO List

  • Support further cheap arbitrary resize (e.g., bicubic, bilinear) for the model outputs
  • Bug: Some PCs will output black images
  • Add the guidance for ncnn model conversion
  • Support face restoration - GFPGAN

💻 Usages

Example Command

realesrgan-ncnn-vulkan.exe -i input.jpg -o output.png -n realesrgan-x4plus-anime

Full Usages

Usage: realesrgan-ncnn-vulkan.exe -i infile -o outfile [options]...

  -h                   show this help
  -v                   verbose output
  -i input-path        input image path (jpg/png/webp) or directory
  -o output-path       output image path (jpg/png/webp) or directory
  -s scale             upscale ratio (4, default=4)
  -t tile-size         tile size (>=32/0=auto, default=0) can be 0,0,0 for multi-gpu
  -m model-path        folder path to pre-trained models(default=models)
  -n model-name        model name (default=realesrgan-x4plus, can be realesrgan-x4plus | realesrgan-x4plus-anime | realesrnet-x4plus)
  -g gpu-id            gpu device to use (default=0) can be 0,1,2 for multi-gpu
  -j load:proc:save    thread count for load/proc/save (default=1:2:2) can be 1:2,2,2:2 for multi-gpu
  -x                   enable tta mode
  -f format            output image format (jpg/png/webp, default=ext/png)
  • input-path and output-path accept either file path or directory path
  • scale = scale level, 4 = upscale 4x
  • tile-size = tile size, use smaller value to reduce GPU memory usage, default selects automatically
  • load:proc:save = thread count for the three stages (image decoding + model upscaling + image encoding), using larger values may increase GPU usage and consume more GPU memory. You can tune this configuration with "4:4:4" for many small-size images, and "2:2:2" for large-size images. The default setting usually works fine for most situations. If you find that your GPU is hungry, try increasing thread count to achieve faster processing.
  • format = the format of the image to be output, png is better supported, however webp generally yields smaller file sizes, both are losslessly encoded

If you encounter crash or error, try to upgrade your GPU driver

🌏 Other Open-Source Code Used

📜 BibTeX

@InProceedings{wang2021realesrgan,
    author    = {Xintao Wang and Liangbin Xie and Chao Dong and Ying Shan},
    title     = {Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data},
    booktitle = {International Conference on Computer Vision Workshops (ICCVW)},
    date      = {2021}
}

📧 Contact

If you have any question, please email [email protected] or [email protected].

Comments
  • problem running on aws

    problem running on aws

    I downloaded the ubuntu zip on a g3s.xlarge and the result is a black image. Is the zip missing files?

    
    ./realesrgan-ncnn-vulkan-v0.2.0-ubuntu/realesrgan-ncnn-vulkan -i input.jpg -o out/output.jpg -n realesrgan-x4plus -s 4 
    
    [0 Tesla M60]  queueC=0[16]  queueG=0[16]  queueT=1[2]
    [0 Tesla M60]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
    [0 Tesla M60]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/1
    [0 Tesla M60]  subgroup=32  basic=1  vote=1  ballot=1  shuffle=1
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  queueC=0[1]  queueG=0[1]  queueT=0[1]
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/0
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  subgroup=8  basic=1  vote=1  ballot=1  shuffle=0
    fopen /home/ubuntu/realesrgan-ncnn-vulkan-v0.2.0-ubuntu/models/realesrgan-x4plus.param failed
    fopen /home/ubuntu/realesrgan-ncnn-vulkan-v0.2.0-ubuntu/models/realesrgan-x4plus.bin failed
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    0.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    4.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    8.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    12.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    16.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    20.83%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    25.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    29.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    33.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    37.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    41.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    45.83%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    50.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    54.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    58.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    62.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    66.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    70.83%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    75.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    79.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    83.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    87.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    91.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    95.83%
    
    opened by kmulvey 2
  • How do I switch from integrated gpu to nvedia gpu

    How do I switch from integrated gpu to nvedia gpu

    [0 Intel(R) UHD Graphics 630] queueC=0[1] queueG=0[1] queueT=0[1] [0 Intel(R) UHD Graphics 630] bugsbn1=0 bugbilz=3 bugcopc=0 bugihfa=0 [0 Intel(R) UHD Graphics 630] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1 [0 Intel(R) UHD Graphics 630] subgroup=32 basic=1 vote=1 ballot=1 shuffle=1

    When upscaling images it only uses integrated gpu I want to use my gtx 1650 how do I switch this?

    opened by csAshish 0
  • in directory mode, option to skip if destination already exist

    in directory mode, option to skip if destination already exist

    if you stop the upscaler while processing a directory and you have to restart it, it will start from the beginning and overwrite existing files

    overwriting in file mode is fine but should be an option for direcotry mode ... so I propose a flag to disable overwrite and skip if exist

    opened by 6543 0
  • models: consider adding into README how to get them.

    models: consider adding into README how to get them.

    Hey,

    Please consider adding to README a section how to get the pre-trained model files. Currently you'd need to extract them from the following links

    • https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesrgan-ncnn-vulkan-20220424-ubuntu.zip
    • https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesr-animevideov3.pth
    • https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesr-general-x4v3.pth

    It took me a moment to figure out how to get a hold of them and I suspect many others would face similar challenge.

    opened by slashbeast 0
  • Segfault realesrnet-x4plus

    Segfault realesrnet-x4plus

    %  ./realesrgan-ncnn-vulkan -i FdLbqKcWIAAaCeY.jpeg  -o FdLbqKcWIAAaCeY.png -n realesrnet-x4plus
    zsh: segmentation fault  ./realesrgan-ncnn-vulkan -i FdLbqKcWIAAaCeY.jpeg -o FdLbqKcWIAAaCeY.png -n 
    
    opened by atomical 1
Owner
Xintao
Researcher at Tencent ARC Lab, (Applied Research Center)
Xintao
Code for the paper "Generative design of breakwaters usign deep convolutional neural network as a surrogate model"

Generative design of breakwaters usign deep convolutional neural network as a surrogate model This repository contains the code for the paper "Generat

2 Apr 10, 2022
Full Transformer Framework for Robust Point Cloud Registration with Deep Information Interaction

Full Transformer Framework for Robust Point Cloud Registration with Deep Information Interaction. arxiv This repository contains python scripts for tr

12 Dec 12, 2022
Face recognize and crop them

Face Recognize Cropping Module Source 아이디어 Face Alignment with OpenCV and Python Requirement 필요 라이브러리 imutil dlib python-opence (cv2) Usage 사용 방법 open

Cho Moon Gi 1 Feb 15, 2022
PatchMatch-RL: Deep MVS with Pixelwise Depth, Normal, and Visibility

PatchMatch-RL: Deep MVS with Pixelwise Depth, Normal, and Visibility Jae Yong Lee, Joseph DeGol, Chuhang Zou, Derek Hoiem Installation To install nece

31 Apr 19, 2022
Repository for RNNs using TensorFlow and Keras - LSTM and GRU Implementation from Scratch - Simple Classification and Regression Problem using RNNs

RNN 01- RNN_Classification Simple RNN training for classification task of 3 signal: Sine, Square, Triangle. 02- RNN_Regression Simple RNN training for

Nahid Ebrahimian 13 Dec 13, 2022
Torch implementation of SegNet and deconvolutional network

Torch implementation of SegNet and deconvolutional network

Fedor Chervinskii 5 Jul 17, 2020
Learning based AI for playing multi-round Koi-Koi hanafuda card games. Have fun.

Koi-Koi AI Learning based AI for playing multi-round Koi-Koi hanafuda card games. Platform Python PyTorch PySimpleGUI (for the interface playing vs AI

Sanghai Guan 10 Nov 20, 2022
High-fidelity 3D Model Compression based on Key Spheres

High-fidelity 3D Model Compression based on Key Spheres This repository contains the implementation of the paper: Yuanzhan Li, Yuqi Liu, Yujie Lu, Siy

5 Oct 11, 2022
PyTorch code for: Learning to Generate Grounded Visual Captions without Localization Supervision

Learning to Generate Grounded Visual Captions without Localization Supervision This is the PyTorch implementation of our paper: Learning to Generate G

Chih-Yao Ma 41 Nov 17, 2022
A Benchmark For Measuring Systematic Generalization of Multi-Hierarchical Reasoning

Orchard Dataset This repository contains the code used for generating the Orchard Dataset, as seen in the Multi-Hierarchical Reasoning in Sequences: S

Bill Pung 1 Jun 05, 2022
Linescanning - Package for (pre)processing of anatomical and (linescanning) fMRI data

line scanning repository This repository contains all of the tools used during the acquisition and postprocessing of line scanning data at the Spinoza

Jurjen Heij 4 Sep 14, 2022
Malmo Collaborative AI Challenge - Team Pig Catcher

The Malmo Collaborative AI Challenge - Team Pig Catcher Approach The challenge involves 2 agents who can either cooperate or defect. The optimal polic

Kai Arulkumaran 66 Jun 29, 2022
Train robotic agents to learn pick and place with deep learning for vision-based manipulation in PyBullet.

Ravens is a collection of simulated tasks in PyBullet for learning vision-based robotic manipulation, with emphasis on pick and place. It features a Gym-like API with 10 tabletop rearrangement tasks,

Google Research 367 Jan 09, 2023
Implementation for Shape from Polarization for Complex Scenes in the Wild

sfp-wild Implementation for Shape from Polarization for Complex Scenes in the Wild project website | paper Code and dataset will be released soon. Int

Chenyang LEI 41 Dec 23, 2022
PyTorch implementation of "Optimization Planning for 3D ConvNets"

Optimization-Planning-for-3D-ConvNets Code for the ICML 2021 paper: Optimization Planning for 3D ConvNets. Authors: Zhaofan Qiu, Ting Yao, Chong-Wah N

Zhaofan Qiu 2 Jan 12, 2022
Code for WSDM 2022 paper, Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation.

DuoRec Code for WSDM 2022 paper, Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation. Usage Download datasets fr

Qrh 46 Dec 19, 2022
Dogs classification with Deep Metric Learning using some popular losses

Tsinghua Dogs classification with Deep Metric Learning 1. Introduction Tsinghua Dogs dataset Tsinghua Dogs is a fine-grained classification dataset fo

QuocThangNguyen 45 Nov 09, 2022
This repository contains the code used in the paper "Prompt-Based Multi-Modal Image Segmentation".

Prompt-Based Multi-Modal Image Segmentation This repository contains the code used in the paper "Prompt-Based Multi-Modal Image Segmentation". The sys

Timo Lüddecke 305 Dec 30, 2022
PyTorch implementation of ICLR 2022 paper PiCO: Contrastive Label Disambiguation for Partial Label Learning

PiCO: Contrastive Label Disambiguation for Partial Label Learning This is a PyTorch implementation of ICLR 2022 paper PiCO: Contrastive Label Disambig

王皓波 147 Jan 07, 2023
Registration Loss Learning for Deep Probabilistic Point Set Registration

RLLReg This repository contains a Pytorch implementation of the point set registration method RLLReg. Details about the method can be found in the 3DV

Felix Järemo Lawin 35 Nov 02, 2022