Official implementation of "Membership Inference Attacks Against Self-supervised Speech Models"

Last update: Nov 01, 2022

Related tags

Overview

Introduction

Official implementation of "Membership Inference Attacks Against Self-supervised Speech Models".

In this work, we demonstrate that existing self-supervised speech model such as HuBERT, wav2vec 2.0, CPC and TERA are vulnerable to membership inference attack (MIA) and thus could reveal sensitive informations related to the training data.

Requirements

Python >= 3.6
Install sox on your OS
Install s3prl on your OS

git clone https://github.com/s3prl/s3prl
cd s3prl
pip install -e ./

Install the specific fairseq

pip install [email protected]+https://github.com//pytorch/[email protected]#egg=fairseq

Preprocessing

First, extract the self-supervised feature of utterances in each corpus according to your needs.

Currently, only LibriSpeech is available.

BASE_PATH=/path/of/the/corpus
OUTPUT_PATH=/path/to/save/feature
MODEL=wav2vec2
SPLIT=train-clean-100 # you should extract train-clean-100, dev-clean, dev-other, test-clean, test-other

python preprocess_feature_LibriSpeech.py \
    --base_path $BATH_PATH \
    --output_path $OUTPUT_PATH \
    --model $MODEL \
    --split $SPLIT

Speaker-level MIA

After extracting the features, you can apply the attack against the models using either basic attack and improved attack.

Noted that you should run the basic attack to generate the .csv file with similarity scores before performing improved attack.

Basic Attack

SEEN_BASE_PATH=/path/you/save/feature/of/seen/corpus
UNSEEN_BASE_PATH=/path/you/save/feature/of/unseen/corpus
OUTPUT_PATH=/path/to/output/results
MODEL=wav2vec2

python predefined-speaker-level-MIA.py \
    --seen_base_path $SEEN_BATH_PATH \
    --unseen_base_path $UNSEEN_BATH_PATH \
    --output_path $OUTPUT_PATH \
    --model $MODEL \

Improved Attack

python train-speaker-level-similarity-model.py \
    --seen_base_path $UNSEEN_BATH_PATH \
    --output_path $OUTPUT_PATH \
    --model $MODEL \
    --speaker_list "${OUTPUT_PATH}/${MODEL}-customized-speaker-level-attack-similarity.csv"

python customized-speaker-level-MIA.py \
    --seen_base_path $SEEN_BATH_PATH \
    --unseen_base_path $UNSEEN_BATH_PATH \
    --output_path $OUTPUT_PATH \
    --model $MODEL \
    --similarity_model_path "${OUTPUT_PATH}/customized-speaker-similarity-model-${MODEL}.pt"

Utterance-level MIA

The process for utterance-level MIA is similar to that of speaker-level:

Basic Attack

SEEN_BASE_PATH=/path/you/save/feature/of/seen/corpus
UNSEEN_BASE_PATH=/path/you/save/feature/of/unseen/corpus
OUTPUT_PATH=/path/to/output/results
MODEL=wav2vec2

python predefined-utterance-level-MIA.py \
    --seen_base_path $SEEN_BATH_PATH \
    --unseen_base_path $UNSEEN_BATH_PATH \
    --output_path $OUTPUT_PATH \
    --model $MODEL \

Improved Attack

python train-utterance-level-similarity-model.py \
    --seen_base_path $UNSEEN_BATH_PATH \
    --output_path $OUTPUT_PATH \
    --model $MODEL \
    --speaker_list "${OUTPUT_PATH}/${MODEL}-customized-utterance-level-attack-similarity.csv"

python customized-utterance-level-MIA.py \
    --seen_base_path $SEEN_BATH_PATH \
    --unseen_base_path $UNSEEN_BATH_PATH \
    --output_path $OUTPUT_PATH \
    --model $MODEL \
    --similarity_model_path "${OUTPUT_PATH}/customized-utterance-similarity-model-${MODEL}.pt"

Citation

If you find our work useful, please cite:

Official implementation of "Membership Inference Attacks Against Self-supervised Speech Models"

Related tags

Overview

Introduction

Requirements

Preprocessing

Speaker-level MIA

Basic Attack

Improved Attack

Utterance-level MIA

Basic Attack

Improved Attack

Citation

Owner

Wei-Cheng Tseng

Paddle Graph Learning (PGL) is an efficient and flexible graph learning framework based on PaddlePaddle

AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation

PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.

Code and Data for the paper: Molecular Contrastive Learning with Chemical Element Knowledge Graph [AAAI 2022]

A Sign Language detection project using Mediapipe landmark detection and Tensorflow LSTM's

Pixel Consensus Voting for Panoptic Segmentation (CVPR 2020)

✅ How Robust are Fact Checking Systems on Colloquial Claims?. In NAACL-HLT, 2021.

FFTNet vocoder implementation

Pytorch implementation of "Get To The Point: Summarization with Pointer-Generator Networks"

New AidForBlind - Various Libraries used like OpenCV and other mentioned in Requirements.txt

Official implementation of the paper ``Unifying Nonlocal Blocks for Neural Networks'' (ICCV'21)

The codes reproduce the figures and statistics in the paper, "Controlling for multiple covariates," by Mark Tygert.

ONNX Command-Line Toolbox

Multitask Learning Strengthens Adversarial Robustness

Official PyTorch implementation of "Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning" (AAAI 2021)

The pytorch implementation of the paper "text-guided neural image inpainting" at MM'2020

An implementation of Deep Forest 2021.2.1.

Code for project: "Learning to Minimize Remainder in Supervised Learning".

The Official PyTorch Implementation of DiscoBox.

Structural Constraints on Information Content in Human Brain States