Implementation for the IJCAI2021 work "Beyond the Spectrum: Detecting Deepfakes via Re-synthesis"

Overview

Beyond the Spectrum

Implementation for the IJCAI2021 work "Beyond the Spectrum: Detecting Deepfakes via Re-synthesis" by Yang He, Ning Yu, Margret Keuper and Mario Fritz.

Pretrained Models

We release the model trained on CelebA-HQ dataset with image resolution 1024x1024. For the super resolution, we use 25,000 real images from CelebA-HQ to train it. For the detectors, we use 25,000 real images and 25,000 fake images to train a binary classifier based on ResNet-50.

We release some models as examples to show how to apply our models based on pixel-level or stage5-level reconstruction errors to detect deepfakes. Download link: https://drive.google.com/file/d/1FeIgABjBpjtnXT-Hl6p5a5lpZxINzXwv/view?usp=sharing.

If you have further questions regarding the trained models, please feel free to contact.

Train

  1. Train the super resolution model.

We use Residual Dense Network (RDN) in our work. The following script shows the hyperparameters used in our experiments. To be noticed, we only use 4 images to show how to run the script. For simplicity, you can download the pretrained model from the above link.

bash script/train_super_resolution_celeba.sh [GPU_ID]
  1. Train the detectors.

After obtaining the super resolution, we use pixel-level or stage5-level L1 based recontruction error to train a classifier. The following scripts use 10 training example to show how to train a classifier with a given super resolution model. You may need to adjust the learning rate and number of training epochs in your case.

bash script/train_pixel_pggan.sh [GPU_ID]
  1. Finetune with auxiliary tasks

In order to improve the robustness of our detectors, we introduce auxiliary tasks (i.e., colorization or denoising) into the super resolution model training and finetune the whole model end-to-end. The following scripts show how to train a model with those tasks.

bash script/train_pixel_pggan_colorization.sh [GPU_ID]
bash script/train_stage5_stylegan_denoising.sh [GPU_ID]

Test

Please download our models. You can use pixel-level or stage5-level to perform deepfakes detection.

bash script/test_pixel_celeba.sh [GPU_ID]
bash script/test_stage5_celeba.sh [GPU_ID]

Citation

If our work is useful for you, please cite our paper:

@inproceedings{yang_ijcai21,
  title={Beyond the Spectrum: Detecting Deepfakes via Re-synthesis},
  author={Yang He and Ning Yu and Margret Keuper and Mario Fritz},
  booktitle={30th International Joint Conference on Artificial Intelligence (IJCAI)},
  year={2021}
}

Contact: Yang He ([email protected])

Last update: 08-22-2021

Owner
Yang He
Applied Scientist in Amazon Last Mile PostDoc in CISPA Helmholtz Center for Information Security / PhD in Max Planck Institute for Informatics
Yang He
State-of-the-art data augmentation search algorithms in PyTorch

MuarAugment Description MuarAugment is a package providing the easiest way to a state-of-the-art data augmentation pipeline. How to use You can instal

43 Dec 12, 2022
Using contrastive learning and OpenAI's CLIP to find good embeddings for images with lossy transformations

The official code for the paper "Inverse Problems Leveraging Pre-trained Contrastive Representations" (to appear in NeurIPS 2021).

Sriram Ravula 26 Dec 10, 2022
Library for 8-bit optimizers and quantization routines.

bitsandbytes Bitsandbytes is a lightweight wrapper around CUDA custom functions, in particular 8-bit optimizers and quantization functions. Paper -- V

Facebook Research 687 Jan 04, 2023
Torchreid: Deep learning person re-identification in PyTorch.

Torchreid Torchreid is a library for deep-learning person re-identification, written in PyTorch. It features: multi-GPU training support both image- a

Kaiyang 3.7k Jan 05, 2023
Solve a Rubiks Cube using Python Opencv and Kociemba module

Rubiks_Cube_Solver Solve a Rubiks Cube using Python Opencv and Kociemba module Main Steps Get the countours of the cube check whether there are tota

Adarsh Badagala 176 Jan 01, 2023
PyTorch(Geometric) implementation of G^2GNN in "Imbalanced Graph Classification via Graph-of-Graph Neural Networks"

This repository is an official PyTorch(Geometric) implementation of G^2GNN in "Imbalanced Graph Classification via Graph-of-Graph Neural Networks". Th

Yu Wang (Jack) 13 Nov 18, 2022
Planar Prior Assisted PatchMatch Multi-View Stereo

ACMP [News] The code for ACMH is released!!! [News] The code for ACMM is released!!! About This repository contains the code for the paper Planar Prio

Qingshan Xu 127 Dec 31, 2022
Face and other object detection using OpenCV and ML Yolo

Object-and-Face-Detection-Using-Yolo- Opencv and YOLO object and face detection is implemented. You only look once (YOLO) is a state-of-the-art, real-

Happy N. Monday 3 Feb 15, 2022
This repo is about to create the Streamlit application for given ML model.

HR-Attritiion-using-Streamlit This repo is about to create the Streamlit application for given ML model. Problem Statement: Managing peoples at workpl

Pavan Giri 0 Dec 10, 2021
PyTorch implementation for the visual prior component (i.e. perception module) of the Visually Grounded Physics Learner [Li et al., 2020].

VGPL-Visual-Prior PyTorch implementation for the visual prior component (i.e. perception module) of the Visually Grounded Physics Learner (VGPL). Give

Toru 8 Dec 29, 2022
An original implementation of "MetaICL Learning to Learn In Context" by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh Hajishirzi

MetaICL: Learning to Learn In Context This includes an original implementation of "MetaICL: Learning to Learn In Context" by Sewon Min, Mike Lewis, Lu

Meta Research 141 Jan 07, 2023
The challenge for Quantum Coalition Hackathon 2021

Qchack 2021 Google Challenge This is a challenge for the brave 2021 qchack.io participants. Instructions Hello, intrepid qchacker, welcome to the G|o

quantumlib 18 May 04, 2022
source code for https://arxiv.org/abs/2005.11248 "Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics"

Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics This work will be published in Nature Biomedical

International Business Machines 71 Nov 15, 2022
SWA Object Detection

SWA Object Detection This project hosts the scripts for training SWA object detectors, as presented in our paper: @article{zhang2020swa, title={SWA

237 Nov 28, 2022
This code reproduces the results of the paper, "Measuring Data Leakage in Machine-Learning Models with Fisher Information"

Fisher Information Loss This repository contains code that can be used to reproduce the experimental results presented in the paper: Awni Hannun, Chua

Facebook Research 43 Dec 30, 2022
Proto-RL: Reinforcement Learning with Prototypical Representations

Proto-RL: Reinforcement Learning with Prototypical Representations This is a PyTorch implementation of Proto-RL from Reinforcement Learning with Proto

Denis Yarats 74 Dec 06, 2022
Implementation of Continuous Sparsification, a method for pruning and ticket search in deep networks

Continuous Sparsification Implementation of Continuous Sparsification (CS), a method based on l_0 regularization to find sparse neural networks, propo

Pedro Savarese 23 Dec 07, 2022
Implementation of a Transformer that Ponders, using the scheme from the PonderNet paper

Ponder(ing) Transformer Implementation of a Transformer that learns to adapt the number of computational steps it takes depending on the difficulty of

Phil Wang 65 Oct 04, 2022
Label-Free Model Evaluation with Semi-Structured Dataset Representations

Label-Free Model Evaluation with Semi-Structured Dataset Representations Prerequisites This code uses the following libraries Python 3.7 NumPy PyTorch

8 Oct 06, 2022
Setup freqtrade/freqUI on Heroku

UNMAINTAINED - REPO MOVED TO https://github.com/p-zombie/freqtrade Creating the app git clone https://github.com/joaorafaelm/freqtrade.git && cd freqt

João 51 Aug 29, 2022