NCNN implementation of Real-ESRGAN. Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration.

Overview

Real-ESRGAN ncnn Vulkan

CI License: MIT Open issue Closed issue

This project is the ncnn implementation of Real-ESRGAN. Real-ESRGAN ncnn Vulkan heavily borrows from realsr-ncnn-vulkan. Many thanks to nihui, ncnn and realsr-ncnn-vulkan 😁

Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration. We also optimize it for anime images.

Contents


If Real-ESRGAN is helpful in your photos/projects, please help to this repo or recommend it to your friends. Thanks 😊
Other recommended projects:
▶️ Real-ESRGAN: A practical algorithm for general image restoration
▶️ GFPGAN: A practical algorithm for real-world face restoration
▶️ BasicSR: An open-source image and video restoration toolbox
▶️ facexlib: A collection that provides useful face-relation functions.
▶️ HandyView: A PyQt5-based image viewer that is handy for view and comparison.

📖 Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data

[Paper]   [Project Page]   [Demo]
Xintao Wang, Liangbin Xie, Chao Dong, Ying Shan
Tencent ARC Lab; Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences

TODO List

  • Support further cheap arbitrary resize (e.g., bicubic, bilinear) for the model outputs
  • Bug: Some PCs will output black images
  • Add the guidance for ncnn model conversion
  • Support face restoration - GFPGAN

💻 Usages

Example Command

realesrgan-ncnn-vulkan.exe -i input.jpg -o output.png -n realesrgan-x4plus-anime

Full Usages

Usage: realesrgan-ncnn-vulkan.exe -i infile -o outfile [options]...

  -h                   show this help
  -v                   verbose output
  -i input-path        input image path (jpg/png/webp) or directory
  -o output-path       output image path (jpg/png/webp) or directory
  -s scale             upscale ratio (4, default=4)
  -t tile-size         tile size (>=32/0=auto, default=0) can be 0,0,0 for multi-gpu
  -m model-path        folder path to pre-trained models(default=models)
  -n model-name        model name (default=realesrgan-x4plus, can be realesrgan-x4plus | realesrgan-x4plus-anime | realesrnet-x4plus)
  -g gpu-id            gpu device to use (default=0) can be 0,1,2 for multi-gpu
  -j load:proc:save    thread count for load/proc/save (default=1:2:2) can be 1:2,2,2:2 for multi-gpu
  -x                   enable tta mode
  -f format            output image format (jpg/png/webp, default=ext/png)
  • input-path and output-path accept either file path or directory path
  • scale = scale level, 4 = upscale 4x
  • tile-size = tile size, use smaller value to reduce GPU memory usage, default selects automatically
  • load:proc:save = thread count for the three stages (image decoding + model upscaling + image encoding), using larger values may increase GPU usage and consume more GPU memory. You can tune this configuration with "4:4:4" for many small-size images, and "2:2:2" for large-size images. The default setting usually works fine for most situations. If you find that your GPU is hungry, try increasing thread count to achieve faster processing.
  • format = the format of the image to be output, png is better supported, however webp generally yields smaller file sizes, both are losslessly encoded

If you encounter crash or error, try to upgrade your GPU driver

🌏 Other Open-Source Code Used

📜 BibTeX

@InProceedings{wang2021realesrgan,
    author    = {Xintao Wang and Liangbin Xie and Chao Dong and Ying Shan},
    title     = {Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data},
    booktitle = {International Conference on Computer Vision Workshops (ICCVW)},
    date      = {2021}
}

📧 Contact

If you have any question, please email [email protected] or [email protected].

Comments
  • problem running on aws

    problem running on aws

    I downloaded the ubuntu zip on a g3s.xlarge and the result is a black image. Is the zip missing files?

    
    ./realesrgan-ncnn-vulkan-v0.2.0-ubuntu/realesrgan-ncnn-vulkan -i input.jpg -o out/output.jpg -n realesrgan-x4plus -s 4 
    
    [0 Tesla M60]  queueC=0[16]  queueG=0[16]  queueT=1[2]
    [0 Tesla M60]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
    [0 Tesla M60]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/1
    [0 Tesla M60]  subgroup=32  basic=1  vote=1  ballot=1  shuffle=1
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  queueC=0[1]  queueG=0[1]  queueT=0[1]
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/0
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  subgroup=8  basic=1  vote=1  ballot=1  shuffle=0
    fopen /home/ubuntu/realesrgan-ncnn-vulkan-v0.2.0-ubuntu/models/realesrgan-x4plus.param failed
    fopen /home/ubuntu/realesrgan-ncnn-vulkan-v0.2.0-ubuntu/models/realesrgan-x4plus.bin failed
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    0.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    4.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    8.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    12.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    16.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    20.83%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    25.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    29.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    33.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    37.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    41.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    45.83%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    50.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    54.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    58.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    62.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    66.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    70.83%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    75.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    79.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    83.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    87.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    91.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    95.83%
    
    opened by kmulvey 2
  • How do I switch from integrated gpu to nvedia gpu

    How do I switch from integrated gpu to nvedia gpu

    [0 Intel(R) UHD Graphics 630] queueC=0[1] queueG=0[1] queueT=0[1] [0 Intel(R) UHD Graphics 630] bugsbn1=0 bugbilz=3 bugcopc=0 bugihfa=0 [0 Intel(R) UHD Graphics 630] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1 [0 Intel(R) UHD Graphics 630] subgroup=32 basic=1 vote=1 ballot=1 shuffle=1

    When upscaling images it only uses integrated gpu I want to use my gtx 1650 how do I switch this?

    opened by csAshish 0
  • in directory mode, option to skip if destination already exist

    in directory mode, option to skip if destination already exist

    if you stop the upscaler while processing a directory and you have to restart it, it will start from the beginning and overwrite existing files

    overwriting in file mode is fine but should be an option for direcotry mode ... so I propose a flag to disable overwrite and skip if exist

    opened by 6543 0
  • models: consider adding into README how to get them.

    models: consider adding into README how to get them.

    Hey,

    Please consider adding to README a section how to get the pre-trained model files. Currently you'd need to extract them from the following links

    • https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesrgan-ncnn-vulkan-20220424-ubuntu.zip
    • https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesr-animevideov3.pth
    • https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesr-general-x4v3.pth

    It took me a moment to figure out how to get a hold of them and I suspect many others would face similar challenge.

    opened by slashbeast 0
  • Segfault realesrnet-x4plus

    Segfault realesrnet-x4plus

    %  ./realesrgan-ncnn-vulkan -i FdLbqKcWIAAaCeY.jpeg  -o FdLbqKcWIAAaCeY.png -n realesrnet-x4plus
    zsh: segmentation fault  ./realesrgan-ncnn-vulkan -i FdLbqKcWIAAaCeY.jpeg -o FdLbqKcWIAAaCeY.png -n 
    
    opened by atomical 1
Owner
Xintao
Researcher at Tencent ARC Lab, (Applied Research Center)
Xintao
Detecting and Tracking Small and Dense Moving Objects in Satellite Videos: A Benchmark

This dataset is a large-scale dataset for moving object detection and tracking in satellite videos, which consists of 40 satellite videos captured by Jilin-1 satellite platforms.

Qingyong 87 Dec 22, 2022
Semi-Autoregressive Transformer for Image Captioning

Semi-Autoregressive Transformer for Image Captioning Requirements Python 3.6 Pytorch 1.6 Prepare data Please use git clone --recurse-submodules to clo

YE Zhou 23 Dec 09, 2022
The official implementation of paper "Finding the Task-Optimal Low-Bit Sub-Distribution in Deep Neural Networks" (IJCV under review).

DGMS This is the code of the paper "Finding the Task-Optimal Low-Bit Sub-Distribution in Deep Neural Networks". Installation Our code works with Pytho

Runpei Dong 3 Aug 28, 2022
NAS Benchmark in "Prioritized Architecture Sampling with Monto-Carlo Tree Search", CVPR2021

NAS-Bench-Macro This repository includes the benchmark and code for NAS-Bench-Macro in paper "Prioritized Architecture Sampling with Monto-Carlo Tree

35 Jan 03, 2023
PyTorch implementation for "Mining Latent Structures with Contrastive Modality Fusion for Multimedia Recommendation"

MIRCO PyTorch implementation for paper: Latent Structures Mining with Contrastive Modality Fusion for Multimedia Recommendation Dependencies Python 3.

Big Data and Multi-modal Computing Group, CRIPAC 9 Dec 08, 2022
Code for paper "Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs"

This is the codebase for the paper: Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs Directory Structur

Peter Hase 19 Aug 21, 2022
An adaptive hierarchical energy management strategy for hybrid electric vehicles

An adaptive hierarchical energy management strategy This project contains the source code of an adaptive hierarchical EMS combining heuristic equivale

19 Dec 13, 2022
An official source code for paper Deep Graph Clustering via Dual Correlation Reduction, accepted by AAAI 2022

Dual Correlation Reduction Network An official source code for paper Deep Graph Clustering via Dual Correlation Reduction, accepted by AAAI 2022. Any

yueliu1999 109 Dec 23, 2022
Crab is a flexible, fast recommender engine for Python that integrates classic information filtering recommendation algorithms in the world of scientific Python packages (numpy, scipy, matplotlib).

Crab - A Recommendation Engine library for Python Crab is a flexible, fast recommender engine for Python that integrates classic information filtering r

python-recsys 1.2k Dec 21, 2022
Research using Cirq!

ReCirq Research using Cirq! This project contains modules for running quantum computing applications and experiments through Cirq and Quantum Engine.

quantumlib 230 Dec 29, 2022
Official code for UnICORNN (ICML 2021)

UnICORNN (Undamped Independent Controlled Oscillatory RNN) [ICML 2021] This repository contains the implementation to reproduce the numerical experime

Konstantin Rusch 21 Dec 22, 2022
source code for https://arxiv.org/abs/2005.11248 "Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics"

Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics This work will be published in Nature Biomedical

International Business Machines 71 Nov 15, 2022
Official code for the paper "Self-Supervised Prototypical Transfer Learning for Few-Shot Classification"

Self-Supervised Prototypical Transfer Learning for Few-Shot Classification This repository contains the reference source code and pre-trained models (

EPFL INDY 44 Nov 04, 2022
HINet: Half Instance Normalization Network for Image Restoration

HINet: Half Instance Normalization Network for Image Restoration Liangyu Chen, Xin Lu, Jie Zhang, Xiaojie Chu, Chengpeng Chen Paper: https://arxiv.org

303 Dec 31, 2022
A Pytorch Implementation for Compact Bilinear Pooling.

CompactBilinearPooling-Pytorch A Pytorch Implementation for Compact Bilinear Pooling. Adapted from tensorflow_compact_bilinear_pooling Prerequisites I

169 Dec 23, 2022
《Lerning n Intrinsic Grment Spce for Interctive Authoring of Grment Animtion》

Learning an Intrinsic Garment Space for Interactive Authoring of Garment Animation Overview This is the demo code for training a motion invariant enco

YuanBo 213 Dec 14, 2022
NP DRAW paper released code

NP-DRAW: A Non-Parametric Structured Latent Variable Model for Image Generation This repo contains the official implementation for the NP-DRAW paper.

ZENG Xiaohui 22 Mar 13, 2022
Inkscape extensions for figure resizing and editing

Academic-Inkscape: Extensions for figure resizing and editing This repository contains several Inkscape extensions designed for editing plots. Scale P

192 Dec 26, 2022
MNIST, but with Bezier curves instead of pixels

bezier-mnist This is a work-in-progress vector version of the MNIST dataset. Samples Here are some samples from the training set. Note that, while the

Alex Nichol 15 Jan 16, 2022
GPU implementation of $k$-Nearest Neighbors and Shared-Nearest Neighbors

GPU implementation of kNN and SNN GPU implementation of $k$-Nearest Neighbors and Shared-Nearest Neighbors Supported by numba cuda and faiss library E

Hyeon Jeon 7 Nov 23, 2022