commonfate 📦commonfate 📦 - Common Fate Model and Transform.

Related tags

Audiocommonfate
Overview

Common Fate Transform and Model for Python

This package is a python implementation of the Common Fate Transform and Model to be used for audio source separation as described in an ICASSP 2016 paper "Common Fate Model for Unison source Separation".

Common Fate Transform

cft

The Common Fate Transform is based on a signal representation that divides a complex spectrogram into a grid of patches of arbitrary size. These complex patches are then processed by a two-dimensional discrete Fourier transform, forming a tensor representation which reveals spectral and temporal modulation textures.

Common Fate Model

cfm

An adapted factorization model similar to the PARAFAC/CANDECOMP factorisation allows to decompose the common fate transform tesnor into different time-varying harmonic sources based on their particular common modulation profile: hence the name Common Fate Model.

Usage

See the full API documentation at http://aliutkus.github.io/commonfate.

Applying the Common Fate Transform

import commonfate

# # forward transform

# STFT Parameters

framelength = 1024
hopsize = 256
X = commonfate.transform.forward(signal, framelength, hopsize)

# Patch Parameters
W = (32, 48)
mhop = (16, 24)

Z = commonfate.transform.forward(X, W, mhop, real=False)

# inverse transform of cft
Y = commonfate.transform.inverse(
    Z, fdim=2, hop=mhop, shape=X.shape, real=False
)
# back to time domain
y = commonfate.transform.inverse(
    Y, fdim=1, hop=hopsize, shape=x.shape
)

Fitting the Common Fate Model

import commonfate

# initialiase and fit the common fate model
cfm = commonfate.model.CFM(z, nb_components=10, nb_iter=100).fit()

# get the fitted factors
(A, H, C) = cfm.factors

# returns the of z approximation using the fitted factors
z_hat = cfm.approx()

Decompose an audio signal using CFT and CFM

commonfate has a built-in wrapper which computes the Common Fate Transform, fits the model according to the Common Fate Model and return the synthesised time domain signal components obtained through wiener / soft mask filtering.

The following example requires to install pysoundfile.

import commonfate
import soundfile as sf

# loading signal
(audio, fs) = sf.read(filename, always_2d=True)

# decomposes the audio signal into
# (nb_components, nb_samples, nb_channels)
components = decompose.process(
    audio,
    nb_iter=100,
    nb_components=10,
    n_fft=1024,
    n_hop=256,
    cft_patch=(32, 48),
    cft_hop=(16, 24)
)

# write out the third component to wave file
sf.write(
    "comp_3.wav",
    components[2, ...],
    fs
)

Optimisations

The current common fate model implementation makes heavily use of the Einstein Notation. We use the numpy einsum module which can be slow on large tensors. To speed up the computation time we recommend to install Daniel Smith's opt_einsum package.

Installation via pip
pip install -e 'git+https://github.com/dgasmith/opt_einsum.git#egg=opt_einsum'

commonfate automatically detects if the package is installed.

References

You can download and read the paper here. If you use this package, please reference to the following publication:

@inproceedings{stoeter2016cfm,
  TITLE = {{Common Fate Model for Unison source Separation}},
  AUTHOR = {St{\"o}ter, Fabian-Robert and Liutkus, Antoine and Badeau, Roland and Edler, Bernd and Magron, Paul},
  BOOKTITLE = {{41st International Conference on Acoustics, Speech and Signal Processing (ICASSP)}},
  ADDRESS = {Shanghai, China},
  PUBLISHER = {{IEEE}},
  SERIES = {Proceedings of the 41st International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  YEAR = {2016},
  KEYWORDS = {Non-Negative tensor factorization ; Sound source separation ; Common Fate Model},
}
You might also like...
C++ library for audio and music analysis, description and synthesis, including Python bindings

Essentia Essentia is an open-source C++ library for audio analysis and audio-based music information retrieval released under the Affero GPL license.

An app made in Python using the PyTube and Tkinter libraries to download videos and MP3 audio.

yt-dl (GUI Edition) An app made in Python using the PyTube and Tkinter libraries to download videos and MP3 audio. How do I download this? Windows: Fi

Small Python application that links a Digico console and Reaper, handling automatic marker insertion and tracking.
Small Python application that links a Digico console and Reaper, handling automatic marker insertion and tracking.

Digico-Reaper-Link This is a small GUI based helper application designed to help with using Digico's Copy Audio function with a Reaper DAW used for re

Anki vector Music ❤ is the best and only Telegram VC player with playlists, Multi Playback, Channel play and more
Anki vector Music ❤ is the best and only Telegram VC player with playlists, Multi Playback, Channel play and more

Anki Vector Music 🎵 A bot that can play music on Telegram Group and Channel Voice Chats Available on telegram as @Anki Vector Music Features 🔥 Thumb

Just-Music - Spotify API Driven Music Web app, that allows to listen and control and share songs

Just Music... Just Music Is A Web APP That Allows Users To Play Song Using Spoti

Audio fingerprinting and recognition in Python
Audio fingerprinting and recognition in Python

dejavu Audio fingerprinting and recognition algorithm implemented in Python, see the explanation here: How it works Dejavu can memorize audio by liste

Python library for audio and music analysis

librosa A python package for music and audio analysis. Documentation See https://librosa.org/doc/ for a complete reference manual and introductory tut

?️ Open Source Audio Matching and Mastering
?️ Open Source Audio Matching and Mastering

Matching + Mastering = ❤️ Matchering 2.0 is a novel Containerized Web Application and Python Library for audio matching and mastering. It follows a si

Python Audio Analysis Library: Feature Extraction, Classification, Segmentation and Applications
Python Audio Analysis Library: Feature Extraction, Classification, Segmentation and Applications

A Python library for audio feature extraction, classification, segmentation and applications This doc contains general info. Click here for the comple

Comments
  • Change factorisation notation so that it matches the paper

    Change factorisation notation so that it matches the paper

    https://github.com/aliutkus/commonfate/blob/master/commonfate/model.py#L121

    is different to paper where we state:

    notation

    the output should therefore be changed to (A, H, C)

    enhancement 
    opened by faroit 1
  • Raised a MemoryError after called decompose.process

    Raised a MemoryError after called decompose.process

    code: (audio, fs) = sf.read('1.wav', always_2d=True) components = commonfate.decompose.process( audio, nb_components=10, ) sf.write( "comp_3.wav", components[2, ...], fs ) raise errors: Traceback (most recent call last): File "F:/py_project/independent-project.git/music_ctl_lettin/music_feature_engineering/process_music.py", line 190, in nb_components=10, File "C:\Users\Besitzer\AppData\Local\Programs\Python\Python36-32\lib\site-packages\commonfate\decompose.py", line 63, in process n_hop, File "C:\Users\Besitzer\AppData\Local\Programs\Python\Python36-32\lib\site-packages\commonfate\transform.py", line 325, in forward stft = fftFunction(stft, frameShape, axes=range(len(frameShape))) File "C:\Users\Besitzer\AppData\Local\Programs\Python\Python36-32\lib\site-packages\numpy\fft\fftpack.py", line 1099, in rfftn a = rfft(a, s[-1], axes[-1], norm) File "C:\Users\Besitzer\AppData\Local\Programs\Python\Python36-32\lib\site-packages\numpy\fft\fftpack.py", line 372, in rfft _real_fft_cache) File "C:\Users\Besitzer\AppData\Local\Programs\Python\Python36-32\lib\site-packages\numpy\fft\fftpack.py", line 83, in _raw_fft r = work_function(a, wsave) MemoryError

    Any ideas?

    opened by okideal 1
Releases(0.1.3)
Owner
Fabian-Robert Stöter
Audio-ML researcher
Fabian-Robert Stöter
Inner ear models for Python

cochlea cochlea is a collection of inner ear models. All models are easily accessible as Python functions. They take sound signal as input and return

98 Jan 05, 2023
Code for csig audio deepfake detection

FMFCC Audio Deepfake Detection Solution This repo provides an solution for the 多媒体伪造取证大赛. Our solution achieve the 1st in the Audio Deepfake Detection

BokingChen 9 Jun 04, 2022
Music player and music library manager for Linux, Windows, and macOS

Ex Falso / Quod Libet - A Music Library / Editor / Player Quod Libet is a music management program. It provides several different ways to view your au

Quod Libet 1.2k Jan 07, 2023
Audio2midi - Automatic Audio-to-symbolic Arrangement

Automatic Audio-to-symbolic Arrangement This is the repository of the project "Audio-to-symbolic Arrangement via Cross-modal Music Representation Lear

Ziyu Wang 24 Dec 05, 2022
Music player - endlessly plays your music

Music player First, if you wonder about what is supposed to be a music player or what makes a music player different from a simple media player, read

Albert Zeyer 482 Dec 19, 2022
pedalboard is a Python library for adding effects to audio.

pedalboard is a Python library for adding effects to audio. It supports a number of common audio effects out of the box, and also allows the use of VST3® and Audio Unit plugin formats for third-party

Spotify 3.9k Jan 02, 2023
Official implementation of A cappella: Audio-visual Singing VoiceSeparation, from BMVC21

Y-Net Official implementation of A cappella: Audio-visual Singing VoiceSeparation, British Machine Vision Conference 2021 Project page: ipcv.github.io

Juan F. Montesinos 12 Oct 22, 2022
python wrapper for rubberband

pyrubberband A python wrapper for rubberband. For now, this just provides lightweight wrappers for pitch-shifting and time-stretching. All processing

Brian McFee 106 Nov 28, 2022
ianZiPu is a way to write notation for Guqin (古琴) music.

PyBetween Wrapper for Between - 비트윈을 위한 파이썬 라이브러리 Legal Disclaimer 오직 교육적 목적으로만 사용할수 있으며, 비트윈은 VCNC의 자산입니다. 악의적 공격에 이용할시 처벌 받을수 있습니다. 사용에 따른 책임은 사용자가

Nancy Yi Liang 8 Nov 25, 2022
Conferencing Speech Challenge

ConferencingSpeech 2021 challenge This repository contains the datasets list and scripts required for the ConferencingSpeech challenge. For more detai

73 Nov 29, 2022
:speech_balloon: SpeechPy - A Library for Speech Processing and Recognition: http://speechpy.readthedocs.io/en/latest/

SpeechPy Official Project Documentation Table of Contents Documentation Which Python versions are supported Citation How to Install? Local Installatio

Amirsina Torfi 870 Dec 27, 2022
Make an audio file (really) long-winded

longwind Make an audio file (really) long-winded Daily repetitions are an illusion anyway.

Vincent Lostanlen 2 Sep 12, 2022
Audio library for modelling loudness

Loudness Loudness is a C++ library with Python bindings for modelling perceived loudness. The library consists of processing modules which can be casc

Dominic Ward 33 Oct 02, 2022
A Python 3 script for capturing and recording a SDR stream to a WAV file (or serving it to a HTTP audio stream).

rfsoapyfile A Python 3 script for capturing and recording a SDR stream to a WAV file (or serving it to a HTTP audio stream). The script is threaded fo

4 Dec 19, 2022
[Singing Log] Let your program learn to sing!

[Singing Log] Let your program learn to sing! You must have thought this was changelog when you saw the English title, but it's not, it's chànggēlog. What it does is allow your program to print logs

黄巍 22 Sep 03, 2022
SinGlow: Generative Flow for SVS tasks in Tensorflow 2

SinGlow is a part of my Singing voice synthesis system. It can extract features of sound, particularly songs and musics. Then we can use these features (or perfect encoding) for feature migrating tas

Haobo Yang 8 Aug 22, 2022
Python CD-DA ripper preferring accuracy over speed

Whipper Whipper is a Python 3 (3.6+) CD-DA ripper based on the morituri project (CDDA ripper for *nix systems aiming for accuracy over speed). It star

671 Jan 04, 2023
Speech Algorithms Collections

Speech Algorithms Collections

Ryuk 498 Jan 06, 2023
Nayeli: cool telegram groups vc music project

Nayeli-music Nayeli 🥀 is cool telegram 🍎 groups vc music project 🎋 . Nayeli-music Nayeli Deployment 🎋 📲 Esy deploy 🐾️ Source Owner ♥️ ❄️ He is s

Kasun bandara 2 Dec 20, 2021
A tool for retrieving audio in the past

Rewinder A tool for retrieving audio in the past. Ever felt like, I need to remember that discussion which happened 10 min back. Now you can! Rewind a

Bharat 1 Jan 24, 2022