Introduction
Official implementation of "Membership Inference Attacks Against Self-supervised Speech Models".
In this work, we demonstrate that existing self-supervised speech model such as HuBERT, wav2vec 2.0, CPC and TERA are vulnerable to membership inference attack (MIA) and thus could reveal sensitive informations related to the training data.
Requirements
- Python >= 3.6
- Install sox on your OS
- Install s3prl on your OS
git clone https://github.com/s3prl/s3prl
cd s3prl
pip install -e ./
- Install the specific fairseq
pip install [email protected]+https://github.com//pytorch/[email protected]#egg=fairseq
Preprocessing
First, extract the self-supervised feature of utterances in each corpus according to your needs.
Currently, only LibriSpeech is available.
BASE_PATH=/path/of/the/corpus
OUTPUT_PATH=/path/to/save/feature
MODEL=wav2vec2
SPLIT=train-clean-100 # you should extract train-clean-100, dev-clean, dev-other, test-clean, test-other
python preprocess_feature_LibriSpeech.py \
--base_path $BATH_PATH \
--output_path $OUTPUT_PATH \
--model $MODEL \
--split $SPLIT
Speaker-level MIA
After extracting the features, you can apply the attack against the models using either basic attack and improved attack.
Noted that you should run the basic attack to generate the .csv file with similarity scores before performing improved attack.
Basic Attack
SEEN_BASE_PATH=/path/you/save/feature/of/seen/corpus
UNSEEN_BASE_PATH=/path/you/save/feature/of/unseen/corpus
OUTPUT_PATH=/path/to/output/results
MODEL=wav2vec2
python predefined-speaker-level-MIA.py \
--seen_base_path $SEEN_BATH_PATH \
--unseen_base_path $UNSEEN_BATH_PATH \
--output_path $OUTPUT_PATH \
--model $MODEL \
Improved Attack
python train-speaker-level-similarity-model.py \
--seen_base_path $UNSEEN_BATH_PATH \
--output_path $OUTPUT_PATH \
--model $MODEL \
--speaker_list "${OUTPUT_PATH}/${MODEL}-customized-speaker-level-attack-similarity.csv"
python customized-speaker-level-MIA.py \
--seen_base_path $SEEN_BATH_PATH \
--unseen_base_path $UNSEEN_BATH_PATH \
--output_path $OUTPUT_PATH \
--model $MODEL \
--similarity_model_path "${OUTPUT_PATH}/customized-speaker-similarity-model-${MODEL}.pt"
Utterance-level MIA
The process for utterance-level MIA is similar to that of speaker-level:
Basic Attack
SEEN_BASE_PATH=/path/you/save/feature/of/seen/corpus
UNSEEN_BASE_PATH=/path/you/save/feature/of/unseen/corpus
OUTPUT_PATH=/path/to/output/results
MODEL=wav2vec2
python predefined-utterance-level-MIA.py \
--seen_base_path $SEEN_BATH_PATH \
--unseen_base_path $UNSEEN_BATH_PATH \
--output_path $OUTPUT_PATH \
--model $MODEL \
Improved Attack
python train-utterance-level-similarity-model.py \
--seen_base_path $UNSEEN_BATH_PATH \
--output_path $OUTPUT_PATH \
--model $MODEL \
--speaker_list "${OUTPUT_PATH}/${MODEL}-customized-utterance-level-attack-similarity.csv"
python customized-utterance-level-MIA.py \
--seen_base_path $SEEN_BATH_PATH \
--unseen_base_path $UNSEEN_BATH_PATH \
--output_path $OUTPUT_PATH \
--model $MODEL \
--similarity_model_path "${OUTPUT_PATH}/customized-utterance-similarity-model-${MODEL}.pt"
Citation
If you find our work useful, please cite:
111 Dec 11, 2022
26 Dec 21, 2022
32 Dec 11, 2022
6 Jan 24, 2022
12 Dec 30, 2022
17 Sep 03, 2022
62 Dec 30, 2022
189 Dec 26, 2022
32 Oct 07, 2022
393 Dec 29, 2022
67 Oct 19, 2022
67 Dec 17, 2022
11 Jan 04, 2023
99 Dec 09, 2022
3 Sep 26, 2022
175 Jan 07, 2023
30 Nov 28, 2022
356 Dec 23, 2022
10 Apr 04, 2022
9 Dec 12, 2022