efficient neural audio synthesis in the waveform domain

Last update: Dec 23, 2022

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

by Ben Hayes, Charalampos Saitis, György Fazekas

This repository is the official implementation of Neural Waveshaping Synthesis.

Model Architecture

Requirements

To install:

pip install -r requirements.txt
pip install -e .

We recommend installing in a virtual environment.

Data

We trained our checkpoints on the URMP dataset. Once downloaded, the dataset can be preprocessed using scripts/create_urmp_dataset.py. This will consolidate recordings of each instrument within the dataset and preprocess them according to the pipeline in the paper.

python scripts/create_urmp_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/urmp \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Alternatively, you can supply your own dataset and use the general create_dataset.py script:

python scripts/create_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/dataset \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Training

To train a model on the URMP dataset, use this command:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/urmp \
  --urmp \
  --instrument vn \  # select URMP instrument with abbreviated string
  --load-data-to-memory

Or to use a non-URMP dataset:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/data \
  --load-data-to-memory

efficient neural audio synthesis in the waveform domain

Related tags

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

Model Architecture

Requirements

Data

Training

Owner

Ben Hayes

This program writes christmas wish programmatically. It is using turtle as a pen pointer draw christmas trees and stars.

PyTorch code for ICPR 2020 paper Future Urban Scene Generation Through Vehicle Synthesis

UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation

Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolution

This is project is the implementation of the DeepShift: Towards Multiplication-Less Neural Networks paper

Implicit Graph Neural Networks

Public Implementation of ChIRo from "Learning 3D Representations of Molecular Chirality with Invariance to Bond Rotations"

A PyTorch implementation for our paper "Dual Contrastive Learning: Text Classification via Label-Aware Data Augmentation".

OpenVisionAPI server

Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learning Source Code

Defending against Model Stealing via Verifying Embedded External Features

Co-mining: Self-Supervised Learning for Sparsely Annotated Object Detection, AAAI 2021.

ML-Decoder: Scalable and Versatile Classification Head

SeqTR: A Simple yet Universal Network for Visual Grounding

Temporal-Relational CrossTransformers

Pytorch implementation of "Neural Wireframe Renderer: Learning Wireframe to Image Translations"

This is the pytorch implementation of the paper - Axiomatic Attribution for Deep Networks.

StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators

PyToch implementation of A Novel Self-supervised Learning Task Designed for Anomaly Segmentation

CURL: Contrastive Unsupervised Representations for Reinforcement Learning