Fake videos detection by tracing the source using video hashing retrieval.

Related tags

Deep Learningdeep-learningpytorchhashing-algorithmfake-video-detection
Overview

Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos 🎉️

📜 Directory

✨ Introduction

VTL

Video Tracing and Tampering Localization (VTL). A novel framework to detect fake video (clipping, cropping, blur, etc.) by tracing the source video of fake video. 1) Training hash centers as HCs. 2) Finding index of source video from HCs. 3) Masking the different between fake video and source video as a result of comparison (auxiliary information).

Trace Samples and Acc of HashBits

Although the source videos are very similar, we can accurately find the source videos of the fake video clips.

DFTL Dataset Samples

Same person with different scenes. You can download full 16 minutes videos of source video and fake video by follows link.

Different fake videos from same source.

Source Video

Fake Videos of Different Face Swap Methods

DAVIS2016-TL Dataset Samples

The first gif of boat is source video, and remaining five videos generated by different inpainting methods.

🔬 Train or Test

Datasets Download

BaiduNetdisk code：VTLs

  • actors: Source videos and fake videos of full 16 minutes. You can use these videos to make richer datasets.
  • DFTL: Dataset of DFTL, the DFTL build from actors.
  • DAVIS2016-TL: Extension of DAVIS2016

Extract to the same directory as the code (vtl). Example:

├─other files
├─project
│  ├─vrf: dataset of DFTL
│  ├─inpainting: dataset of DAVIS2016-TL
│  └─vtl: our code
│      ├─CSQ: Central Similarity Quantization for Efficient Image and Video Retrieval
│      ├─dmac: Compared method of Localization
│      └─codes

Train

Pretrained models and hash centers

pip install -r requirements.txt

Model DFTL DAVIS2016-TL
ViTHash 64-1024bits 64-1024bits
Generator link link

Parameters

  • local_rank: gpu id
  • path: dataset path
  • type: choice dataloader
    • 0: DFTL dataloader, dir name is vrf
    • 1: DAVIS2016-TL dataloader, dir name is inpainting

Train ViTHash

python train_h.py --local_rank=0 --path=../vrf --type=0 --bits=128

Train Generator

python train_g.py --local_rank=0 --path=../vrf --type=0

Test

Test IOU

The test script will test Generator of VTL and DMAC together on DFTL and DAVIS2016-TL. You can modify it for yourself.

python test_iou.py

Test ViTHash

  1. type: choice dataloader
    • 0: DFTL dataloader, dir name is vrf
    • 1: DAVIS2016-TL dataloader, dir name is inpainting
  2. path: dataset path
  3. hashbits: 128 256 512 or 1024, will load different pre-trained model and hash JSON file.
python test.py 1 ../inpainting 512

Test CSQ

  1. cd ./CSQ
  2. run test script
python hash_test_vrf.py --dataset=Inpainting --pretrained_3d=./Inpainting_64bits.pth

🚀️ Tracing

Trace Samples

👀️ Localization

Localization Samples

DAVIS2016-TL

DFTL

You might also like...
Implementation of accepted AAAI 2021 paper: Deep Unsupervised Image Hashing by Maximizing Bit Entropy
Implementation of accepted AAAI 2021 paper: Deep Unsupervised Image Hashing by Maximizing Bit Entropy

Deep Unsupervised Image Hashing by Maximizing Bit Entropy This is the PyTorch implementation of accepted AAAI 2021 paper: Deep Unsupervised Image Hash

null 62 Dec 30, 2022
Official implementation of NeurIPS 2021 paper "One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective"

Official implementation of NeurIPS 2021 paper "One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective"

Ng Kam Woh 71 Dec 22, 2022
Tracing Versus Freehand for Evaluating Computer-Generated Drawings (SIGGRAPH 2021)
Tracing Versus Freehand for Evaluating Computer-Generated Drawings (SIGGRAPH 2021)

Tracing Versus Freehand for Evaluating Computer-Generated Drawings (SIGGRAPH 2021) Zeyu Wang, Sherry Qiu, Nicole Feng, Holly Rushmeier, Leonard McMill

Zach Zeyu Wang 23 Dec 9, 2022
Enhancing Knowledge Tracing via Adversarial Training

Enhancing Knowledge Tracing via Adversarial Training This repository contains source code for the paper "Enhancing Knowledge Tracing via Adversarial T

Xiaopeng Guo 14 Oct 24, 2022
Implementation of light baking system for ray tracing based on Activision's UberBake

Vulkan Light Bakary MSU Graphics Group Student's Diploma Project Treefonov Andrey [GitHub] [LinkedIn] Project Goal The goal of the project is to imple

Andrey Treefonov 7 Dec 27, 2022
Ray tracing of a Schwarzschild black hole written entirely in TensorFlow.

TensorGeodesic Ray tracing of a Schwarzschild black hole written entirely in TensorFlow. Dependencies: Python 3 TensorFlow 2.x numpy matplotlib About

null 5 Jan 15, 2022
Space-event-trace - Tracing service for spaceteam events
Space-event-trace - Tracing service for spaceteam events

space-event-trace Tracing service for TU Wien Spaceteam events. This service is

TU Wien Space Team 2 Jan 4, 2022
Activity image-based video retrieval

Cross-modal-retrieval Our approach is focus on Activity Image-to-Video Retrieval (AIVR) task. The compared methods are state-of-the-art single modalit

BCMI 75 Oct 21, 2021
A Joint Video and Image Encoder for End-to-End Retrieval
A Joint Video and Image Encoder for End-to-End Retrieval

Frozen️ in Time ❄️ ️️️️ ⏳ A Joint Video and Image Encoder for End-to-End Retrieval project page | arXiv | webvid-data Repository containing the code,

null 225 Dec 25, 2022
Comments
  • the pre-trained model

    the pre-trained model

    I downloaded the provided pre-trained model and the test acc in DFTL and DAVIS2016-TL reached 100%, is this correct? Why is it so much higher than in the article?

    opened by ymhzyj 1
Releases(v1.0)
Owner
GitHub Repository
[NeurIPS 2021] ORL: Unsupervised Object-Level Representation Learning from Scene Images

Unsupervised Object-Level Representation Learning from Scene Images This repository contains the official PyTorch implementation of the ORL algorithm

Jiahao Xie 55 Dec 03, 2022
[CVPR22] Official codebase of Semantic Segmentation by Early Region Proxy.

RegionProxy Figure 2. Performance vs. GFLOPs on ADE20K val split. Semantic Segmentation by Early Region Proxy Yifan Zhang, Bo Pang, Cewu Lu CVPR 2022

Yifan 54 Nov 29, 2022
Where2Act: From Pixels to Actions for Articulated 3D Objects

Where2Act: From Pixels to Actions for Articulated 3D Objects The Proposed Where2Act Task. Given as input an articulated 3D object, we learn to propose

Kaichun Mo 69 Nov 28, 2022
Contains code for the paper "Vision Transformers are Robust Learners".

Vision Transformers are Robust Learners This repository contains the code for the paper Vision Transformers are Robust Learners by Sayak Paul* and Pin

Sayak Paul 103 Jan 05, 2023
🧑‍🔬 verify your TEAL program by experiment and observation

Graviton - Testing TEAL with Dry Runs Tutorial Local Installation The following instructions assume that you have make available in your local environ

Algorand 18 Jan 03, 2023
This is a code repository for paper OODformer: Out-Of-Distribution Detection Transformer

OODformer: Out-Of-Distribution Detection Transformer This repo is the official the implementation of the OODformer: Out-Of-Distribution Detection Tran

34 Dec 02, 2022
Cross-view Transformers for real-time Map-view Semantic Segmentation (CVPR 2022 Oral)

Cross View Transformers This repository contains the source code and data for our paper: Cross-view Transformers for real-time Map-view Semantic Segme

Brady Zhou 363 Dec 25, 2022
UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Language

UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Language This repository contains UA-GEC data and an accompanying Python lib

Grammarly 226 Dec 29, 2022
Annotated, understandable, and visually interpretable PyTorch implementations of: VAE, BIRVAE, NSGAN, MMGAN, WGAN, WGANGP, LSGAN, DRAGAN, BEGAN, RaGAN, InfoGAN, fGAN, FisherGAN

Overview PyTorch 0.4.1 | Python 3.6.5 Annotated implementations with comparative introductions for minimax, non-saturating, wasserstein, wasserstein g

Shayne O'Brien 471 Dec 16, 2022
(Py)TOD: Tensor-based Outlier Detection, A General GPU-Accelerated Framework

(Py)TOD: Tensor-based Outlier Detection, A General GPU-Accelerated Framework Background: Outlier detection (OD) is a key data mining task for identify

Yue Zhao 127 Jan 05, 2023
NeWT: Natural World Tasks

NeWT: Natural World Tasks This repository contains resources for working with the NeWT dataset. ❗ At this time the binary tasks are not publicly avail

Visipedia 26 Oct 18, 2022
A variational Bayesian method for similarity learning in non-rigid image registration (CVPR 2022)

A variational Bayesian method for similarity learning in non-rigid image registration We provide the source code and the trained models used in the re

daniel grzech 14 Nov 21, 2022
Implementation of QuickDraw - an online game developed by Google, combined with AirGesture - a simple gesture recognition application

QuickDraw - AirGesture Introduction Here is my python source code for QuickDraw - an online game developed by google, combined with AirGesture - a sim

Viet Nguyen 89 Dec 18, 2022
Neural networks applied in recognizing guitar chords using python, AutoML.NET with C# and .NET Core

Chord Recognition Demo application The demo application is written in C# with .NETCore. As of July 9, 2020, the only version available is for windows

Andres Mauricio Rondon Patiño 24 Oct 22, 2022
Code for "LASR: Learning Articulated Shape Reconstruction from a Monocular Video". CVPR 2021.

LASR Installation Build with conda conda env create -f lasr.yml conda activate lasr # install softras cd third_party/softras; python setup.py install;

Google 157 Dec 26, 2022
Learning Efficient Online 3D Bin Packing on Packing Configuration Trees

Learning Efficient Online 3D Bin Packing on Packing Configuration Trees This repository is being continuously updated, please stay tuned！ Any code con

86 Dec 28, 2022
Official implementation of Long-Short Transformer in PyTorch.

Long-Short Transformer (Transformer-LS) This repository hosts the code and models for the paper: Long-Short Transformer: Efficient Transformers for La

NVIDIA Corporation 198 Dec 29, 2022
A python toolbox for predictive uncertainty quantification, calibration, metrics, and visualization

Website, Tutorials, and Docs    Uncertainty Toolbox A python toolbox for predictive uncertainty quantification, calibration, metrics, and visualizatio

Uncertainty Toolbox 1.4k Dec 28, 2022
ArcaneGAN by Alex Spirin

ArcaneGAN by Alex Spirin

Alex 617 Dec 28, 2022
Official tensorflow implementation for CVPR2020 paper “Learning to Cartoonize Using White-box Cartoon Representations”

Tensorflow implementation for CVPR2020 paper “Learning to Cartoonize Using White-box Cartoon Representations”.

3.7k Dec 31, 2022