efficient neural audio synthesis in the waveform domain

Last update: Dec 23, 2022

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

by Ben Hayes, Charalampos Saitis, György Fazekas

This repository is the official implementation of Neural Waveshaping Synthesis.

Model Architecture

Requirements

To install:

pip install -r requirements.txt
pip install -e .

We recommend installing in a virtual environment.

Data

We trained our checkpoints on the URMP dataset. Once downloaded, the dataset can be preprocessed using scripts/create_urmp_dataset.py. This will consolidate recordings of each instrument within the dataset and preprocess them according to the pipeline in the paper.

python scripts/create_urmp_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/urmp \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Alternatively, you can supply your own dataset and use the general create_dataset.py script:

python scripts/create_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/dataset \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Training

To train a model on the URMP dataset, use this command:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/urmp \
  --urmp \
  --instrument vn \  # select URMP instrument with abbreviated string
  --load-data-to-memory

Or to use a non-URMP dataset:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/data \
  --load-data-to-memory

efficient neural audio synthesis in the waveform domain

Related tags

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

Model Architecture

Requirements

Data

Training

Owner

Ben Hayes

Official PyTorch implementation of the paper Image-Based CLIP-Guided Essence Transfer.

A repository for the updated version of CoinRun used to collect MUGEN, a multimodal video-audio-text dataset.

Framework for training options with different attention mechanism and using them to solve downstream tasks.

[CVPR 2021] Exemplar-Based Open-Set Panoptic Segmentation Network (EOPSN)

YOLOPのPythonでのONNX推論サンプル

Official Pytorch implementation of paper "Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images"

A python toolbox for predictive uncertainty quantification, calibration, metrics, and visualization

The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation

Bringing sanity to world of messed-up data

USAD - UnSupervised Anomaly Detection on multivariate time series

a general-purpose Transformer based vision backbone

Pytorch implementation for Semantic Segmentation/Scene Parsing on MIT ADE20K dataset

Source code for "Roto-translated Local Coordinate Framesfor Interacting Dynamical Systems"

Pytorch modules for paralel models with same architecture. Ideal for multi agent-based systems

TCPNet - Temporal-attentive-Covariance-Pooling-Networks-for-Video-Recognition

This GitHub repo consists of Code and Some results of project- Diabetes Treatment using Gold nanoparticles. These Consist of ML Models used for prediction Diabetes and further the basic theory and working of Gold nanoparticles.

[CVPR 2020] GAN Compression: Efficient Architectures for Interactive Conditional GANs

Resources for the Ki testnet challenge

Pytorch implementation of the unsupervised object discovery method LOST.

PyTorch implementation DRO: Deep Recurrent Optimizer for Structure-from-Motion