Provided is code that demonstrates the training and evaluation of the work presented in the paper: "On the Detection of Digital Face Manipulation" published in CVPR 2020.

Overview

FFD Source Code

Provided is code that demonstrates the training and evaluation of the work presented in the paper: "On the Detection of Digital Face Manipulation" published in CVPR 2020.

The proposed network framework with attention mechanism

Project Webpage

See the MSU CVLab website for project details and access to the DFFD dataset.

http://cvlab.cse.msu.edu/project-ffd.html

Notes

This code is provided as example code, and may not reflect a specific combination of hyper-parameters presented in the paper.

Description of contents

  • xception.py: Defines the Xception network with the attention mechanism
  • train*.py: Train the model on the train data
  • test*.py: Evaluate the model on the test data

Acknowledgements

If you use or refer to this source code, please cite the following paper:

@inproceedings{cvpr2020-dang,
  title={On the Detection of Digital Face Manipulation},
  author={Hao Dang, Feng Liu, Joel Stehouwer, Xiaoming Liu, Anil Jain},
  booktitle={In Proceeding of IEEE Computer Vision and Pattern Recognition (CVPR 2020)},
  address={Seattle, WA},
  year={2020}
}
Comments
  • Is it possible to release the script for generating edited images by FaceApp?

    Is it possible to release the script for generating edited images by FaceApp?

    Hi, Thanks for releasing the code and dataset! Part of your dataset is generated by FaceApp (using automated scripts running on android devices). I am wondering if you could also release this android script? I also plan to generate some edited images using FaceApp, and an automated script will be quite helpful!! Thanks!

    opened by zjxgithub 2
  • Question about mask images in dataset

    Question about mask images in dataset

    Thank you for releasing the code and the DFFD dataset!

    I noticed that in the "faceapp" part of the dataset, there is a ground-truth manipulation masks image for each fake image. How are these mask images generated?

    The paper mentioned that the ground-truth manipulation mask were calculated by source images and fake images, but I still did not understand how.

    Thank you for answering my question. :)

    opened by piddnad 2
  • Serveral question about dataset

    Serveral question about dataset

    Thanks for releasing the code and the dataset. I have some questions for the dataset,

    • In align_faces/align_faces.m inside scripts.zip, there is a file called box.txt. But I can't find it anywhere. It seems crucial to align and crop the images.

    image

    • All of the images in dataset are in the resolution of 299x299. I wonder how did you process the images in CelebA. I remember the aligned and cropped image in CelebA are in the resolution of 128x128.
    opened by wheatdog 2
  • attention map and gt mask matching

    attention map and gt mask matching

    Hi, thanks for your work. I have a small question. The attention map size is 19x19, but the gt mask (diff image) is 299x299. Are they matched by downsampling gt mask?

    opened by neverUseThisName 1
  • Are label information leaked in testing process?

    Are label information leaked in testing process?

    Thanks for uploading your code and dataset. After a short view I'm considering your predicting process is like: generating masks with scripts on test data, using test data and their masks to feed into trained model to predict. But I was confused that in your test.py file, you get dataset like this:

    def get_dataset():
      return Dataset('test', BATCH_SIZE, CONFIG['img_size'], CONFIG['map_size'], CONFIG['norms'], SEED)
    

    then you differ masks of real and fake photos by using their labels in dataset.py:

      def __getitem__(self, index):
        im_name = self.images[index]
        img = self.load_image(im_name)
        if self.label_name == 'Real':
          msk = torch.zeros(1,19,19)
        else:
          msk = self.load_mask(im_name.replace('Fake/', 'Mask/'))
        return {'img': img, 'msk': msk, 'lab': self.label, 'im_name': im_name}
    

    Is it fair to distinguish masks by label_name in the testing process? I also wonder how to create Mask/ folder when you predict fake images that donot have corresponding real images?

    If i misunderstand anything please correct me, thanks a lot!

    opened by insomnia1996 0
  • May I know where I can find the imagenet pretrained model?

    May I know where I can find the imagenet pretrained model?

    Hi,

    For using pretrained model: xception-b5690688.pth, may I know where I can find the model specified here: https://github.com/JStehouwer/FFD_CVPR2020/blob/master/xception.py#L243

    Thanks.

    opened by ilovecv 2
  • Error in get_batch in train.py

    Error in get_batch in train.py

    Greetings,

    Many thanks to your wok. I am very interested in your work and I want to try out your model. When I ran the train*.py, I encounter the following issue , here are part of the error messages.

    batch = [next(_.generator, None) for _ in self.datasets]
    

    File "D:\Fake Detector\attention_map_to_detect_manipulation\FFD_CVPR2020\dataset.py", line 91, in self = reduction.pickle.load(from_parent)batch = [next(_.generator, None) for _ in self.datasets]

    File "D:\Fake Detector\attention_map_to_detect_manipulation\FFD_CVPR2020\dataset.py", line 73, in get_batch EOFError: Ran out of input

    and reduction.dump(process_obj, to_child) File "C:\Users\xxx\anaconda3\envs\d2l\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) TypeError: cannot pickle 'generator' object

    What I did is just make directory data/train/Real(Fake) and place my images dataset into the corresponding folder and then ran the train.py. However, it seems it can't work. May I ask whether I missed anything. I am running the program in windows system and I don't know that will affect as well.

    opened by bitrookie 1
  • Use pretrained model to classify own data?

    Use pretrained model to classify own data?

    Hi @JStehouwer - thank you so much for the awesome code (v2.1)!

    I am trying to use your pretrained model on my own images in order to try out the classifier.

    Are you able to confirm:

    • Filename and format of pretrained model
    • Whether anything else is needed to perform the above classification

    Thanks again

    opened by jtlz2 4
  • dataset questions

    dataset questions

    1、 Whether the published dataset ( FFHQ、FaceAPP、StarGAN、PGGAN、StyleGAN ) has been randomly selected ? And How to generate starGAN mask, how to determine the specific CelebA picture used ? 2、 I have downloaded the FF++、CelebA and DeepFaceLab dataset, how to randomly select the training set, test set and verification set ? And how to set the random seed ? 3、 Which data sets need align processing, and how, please specify ?

    Thank you for your work, it is very good, I will follow your work, but now the problem of dataset makes my work difficult, I hope to get your help.

    opened by miaoct 2
Releases(v2.1)
ZeroGen: Efficient Zero-shot Learning via Dataset Generation

ZEROGEN This repository contains the code for our paper “ZeroGen: Efficient Zero

Jiacheng Ye 31 Dec 30, 2022
Pytorch Implementation of Residual Vision Transformers(ResViT)

ResViT Official Pytorch Implementation of Residual Vision Transformers(ResViT) which is described in the following paper: Onat Dalmaz and Mahmut Yurt

ICON Lab 41 Dec 08, 2022
Code for "Optimizing risk-based breast cancer screening policies with reinforcement learning"

Tempo: Optimizing risk-based breast cancer screening policies with reinforcement learning Introduction This repository was used to develop Tempo, as d

Adam Yala 12 Oct 11, 2022
Code for How To Create A Fully Automated AI Based Trading System With Python

AI Based Trading System This code works as a boilerplate for an AI based trading system with yfinance as data source and RobinHood or Alpaca as broker

Rubén 196 Jan 05, 2023
OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers (NeurIPS 2021)

OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers (NeurIPS 2021) This is an PyTorch implementation of OpenMatc

Vision and Learning Group 38 Dec 26, 2022
An easy-to-use app to visualise attentions of various VQA models.

Ask Me Anything: A tool for visualising Visual Question Answering (AMA) An easy-to-use app to visualise attentions of various VQA models. Please click

Apoorve 37 Nov 13, 2022
code for TCL: Vision-Language Pre-Training with Triple Contrastive Learning, CVPR 2022

Vision-Language Pre-Training with Triple Contrastive Learning, CVPR 2022 News (03/16/2022) upload retrieval checkpoints finetuned on COCO and Flickr T

187 Jan 02, 2023
A simple API wrapper for Discord interactions.

Your ultimate Discord interactions library for discord.py. About | Installation | Examples | Discord | PyPI About What is discord-py-interactions? dis

james 641 Jan 03, 2023
Code for our paper Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation

CorDA Code for our paper Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation Prerequisite Please create and activate the follo

Qin Wang 60 Nov 30, 2022
Data reduction pipeline for KOALA on the AAT.

KOALA KOALA, the Kilofibre Optical AAT Lenslet Array, is a wide-field, high efficiency, integral field unit used by the AAOmega spectrograph on the 3.

4 Sep 26, 2022
ADOP: Approximate Differentiable One-Pixel Point Rendering

ADOP: Approximate Differentiable One-Pixel Point Rendering Abstract: We present a novel point-based, differentiable neural rendering pipeline for scen

Darius Rückert 1.9k Jan 06, 2023
Fully Convolutional Refined Auto Encoding Generative Adversarial Networks for 3D Multi Object Scenes

Fully Convolutional Refined Auto-Encoding Generative Adversarial Networks for 3D Multi Object Scenes This repository contains the source code for Full

Yu Nishimura 106 Nov 21, 2022
Siamese TabNet

Raifhack-DS-2021 https://raifhack.ru/ - Команда Звёздочка Siamese TabNet Сиамская TabNet предсказывает стоимость объекта недвижимости с price_type=1,

Daniel Gafni 15 Apr 16, 2022
A PyTorch toolkit for 2D Human Pose Estimation.

PyTorch-Pose PyTorch-Pose is a PyTorch implementation of the general pipeline for 2D single human pose estimation. The aim is to provide the interface

Wei Yang 1.1k Dec 30, 2022
Few-NERD: Not Only a Few-shot NER Dataset

Few-NERD: Not Only a Few-shot NER Dataset This is the source code of the ACL-IJCNLP 2021 paper: Few-NERD: A Few-shot Named Entity Recognition Dataset.

THUNLP 319 Dec 30, 2022
MediaPipeのPythonパッケージのサンプルです。2020/12/11時点でPython実装のある4機能(Hands、Pose、Face Mesh、Holistic)について用意しています。

mediapipe-python-sample MediaPipeのPythonパッケージのサンプルです。 2020/12/11時点でPython実装のある以下4機能について用意しています。 Hands Pose Face Mesh Holistic Requirement mediapipe 0.

KazuhitoTakahashi 217 Dec 12, 2022
An offline deep reinforcement learning library

d3rlpy: An offline deep reinforcement learning library d3rlpy is an offline deep reinforcement learning library for practitioners and researchers. imp

Takuma Seno 817 Jan 02, 2023
✂️ EyeLipCropper is a Python tool to crop eyes and mouth ROIs of the given video.

EyeLipCropper EyeLipCropper is a Python tool to crop eyes and mouth ROIs of the given video. The whole process consists of three parts: frame extracti

Zi-Han Liu 9 Oct 25, 2022
Data manipulation and transformation for audio signal processing, powered by PyTorch

torchaudio: an audio library for PyTorch The aim of torchaudio is to apply PyTorch to the audio domain. By supporting PyTorch, torchaudio follows the

1.9k Dec 28, 2022
Scalable training for dense retrieval models.

Scalable implementation of dense retrieval. Training on cluster By default it trains locally: PYTHONPATH=.:$PYTHONPATH python dpr_scale/main.py traine

Facebook Research 90 Dec 28, 2022