audioLIME
This repository contains the Python package audioLIME, a tool for creating listenable explanations for machine learning models in music information retrival (MIR). audioLIME is based on the method lime (local interpretable model-agnostic explanations) work presented in this paper and uses source separation estimates in order to create interpretable components.
Citing
If you use audioLIME in your work, please cite it:
@misc{haunschmid2020audiolime,
title={{audioLIME: Listenable Explanations Using Source Separation}},
author={Verena Haunschmid and Ethan Manilow and Gerhard Widmer},
year={2020},
eprint={2008.00582},
archivePrefix={arXiv},
primaryClass={cs.SD},
howpublished={13th International Workshop on Machine Learning and Music}
}
Publications
audioLIME is introduced/used in the following publications:
-
Verena Haunschmid, Ethan Manilow and Gerhard Widmer, audioLIME: Listenable Explanations Using Source Separation
- paper: arxiv
- code: branch
mml2020and mml2020-experiments
-
Verena Haunschmid, Ethan Manilow and Gerhard Widmer, Towards Musically Meaningful Explanations Using Source Separation
- paper: arxiv
Installation
The audioLIME package is not on PyPI yet. For installing it, clone the git repo and install it using setup.py.
git clone https://github.com/CPJKU/audioLIME.git # HTTPS
git clone [email protected]:CPJKU/audioLIME.git # SSH
cd audioLIME
python setup.py install
To install a version for development purposes check out this article.
Tests
To test your installation, the following test are available:
python -m unittest tests.test_SpleeterFactorization
python -m unittest tests.test_DataProviders
Note on Requirements
To keep it lightweight, not all possible dependencies are contained in setup.py. Depending on the factorization you want to use, you might need different packages, e.g. nussl or spleeter.
Installation & Usage of spleeter
pip install spleeter==2.0.2
When you're using spleeter for the first time, it will download the used model in a directory pretrained_models. You can only change the location by setting an environment variable MODEL_PATH before spleeter is imported. There are different ways to set an environment variable, for example:
export MODEL_PATH=/share/home/verena/experiments/spleeter/pretrained_models/
Available Factorizations
Currently we have the following factorizations implemented:
- SpleeterFactorization based on the source separation system spleeter (code)
- SoundLIMEFactorization: time-frequency segmentation based on SoundLIME (the original implementation was not flexible enough for our experiments)
Usage Example
Here we demonstrate how we can explain the prediction of FCN (code, Choi 2016, Won 2020) using SpleeterFactorization.
For this to work you need to install the requirements found in the above mentioned repo of the tagger and spleeter:
pip install -r examples/requirements.txt
data_provider = RawAudioProvider(audio_path)
spleeter_factorization = SpleeterFactorization(data_provider,
n_temporal_segments=10,
composition_fn=None,
model_name='spleeter:5stems')
explainer = lime_audio.LimeAudioExplainer(verbose=True, absolute_feature_sort=False)
explanation = explainer.explain_instance(factorization=spleeter_factorization,
predict_fn=predict_fn,
top_labels=1,
num_samples=16384,
batch_size=32
)
For the details on setting everything up, see example_using_spleeter_fcn.
Listen to the input and explanation.
TODOs
- upload to pypi.org (to allow installation via
pip) - usage example for
SoundLIMEFactorization - tutorial in form of a Jupyter Notebook
- more documentation
31 Dec 20, 2022
185 Dec 26, 2022
2.2k Jan 1, 2023
56 Nov 15, 2022
2 Nov 8, 2022
33 May 31, 2022
96 Dec 15, 2022
281 Dec 22, 2022
174 Dec 19, 2022
437 Oct 09, 2022
3 Apr 07, 2022
1 Jun 14, 2022
16 Oct 22, 2022
0 Jan 20, 2022
40 Dec 09, 2022
1 Oct 26, 2021
182 Dec 23, 2022
6.1k Jan 04, 2023
9 Nov 06, 2022
11 Jun 23, 2022
118 Dec 30, 2022
26 Nov 19, 2022
12 Aug 27, 2022
17 Dec 08, 2022
189 Dec 02, 2022
479 Dec 08, 2022
15 Apr 21, 2022
45 Dec 08, 2022