The official implementation of VAENAR-TTS, a VAE based non-autoregressive TTS model.

Last update: Oct 28, 2022

Related tags

Overview

VAENAR-TTS

This repo contains code accompanying the paper "VAENAR-TTS: Variational Auto-Encoder based Non-AutoRegressive Text-to-Speech Synthesis".

Samples | Paper | Pretrained Models

Usage

0. Dataset

English: LJSpeech
Mandarin: DataBaker(标贝)

1. Environment setup

conda env create -f environment.yml
conda activate vaenartts-env

2. Data pre-processing

For English using LJSpeech:

CUDA_VISIBLE_DEVICES= python preprocess.py --dataset ljspeech --data_dir /path/to/extracted/LJSpeech-1.1 --save_dir ./ljspeech

For Mandarin using Databaker(标贝):

CUDA_VISIBLE_DEVICES= python preprocess.py --dataset databaker --data_dir /path/to/extracted/biaobei --save_dir ./databaker

3. Training

For English using LJSpeech:

CUDA_VISIBLE_DEVICES=0 TF_FORCE_GPU_ALLOW_GROWTH=true python train.py --dataset ljspeech --log_dir ./lj-log_dir --test_dir ./lj-test_dir --data_dir ./ljspeech/tfrecords/ --model_dir ./lj-model_dir

For Mandarin using Databaker(标贝):

CUDA_VISIBLE_DEVICES=0 TF_FORCE_GPU_ALLOW_GROWTH=true python train.py --dataset databaker --log_dir ./db-log_dir --test_dir ./db-test_dir --data_dir ./databaker/tfrecords/ --model_dir ./db-model_dir

4. Inference (synthesize speech for the whole test set)

For English using LJSpeech:

CUDA_VISIBLE_DEVICES=0 TF_FORCE_GPU_ALLOW_GROWTH=true python inference.py --dataset ljspeech --test_dir ./lj-test-2000 --data_dir ./ljspeech/tfrecords/ --batch_size 16 --write_wavs true --draw_alignments true --ckpt_path ./lj-model_dir/ckpt-2000

For Mandarin using Databaker(标贝):

CUDA_VISIBLE_DEVICES=0 TF_FORCE_GPU_ALLOW_GROWTH=true python inference.py --dataset databaker --test_dir ./db-test-2000 --data_dir ./databaker/tfrecords/ --batch_size 16 --write_wavs true --draw_alignments true --ckpt_path ./db-model_dir/ckpt-2000

The official implementation of VAENAR-TTS, a VAE based non-autoregressive TTS model.

Related tags

Overview

VAENAR-TTS

Samples | Paper | Pretrained Models

Usage

0. Dataset

1. Environment setup

2. Data pre-processing

3. Training

4. Inference (synthesize speech for the whole test set)

Reference

Owner

THUHCSI

Collection of common code that's shared among different research projects in FAIR computer vision team.

3D-Transformer: Molecular Representation with Transformer in 3D Space

A benchmark framework for Tensorflow

A Peer-to-peer Platform for Secure, Privacy-preserving, Decentralized Data Science

A python module for configuration of block devices

This is a official repository of SimViT.

Awesome Human Pose Estimation

End-To-End Memory Network using Tensorflow

MagFace: A Universal Representation for Face Recognition and Quality Assessment

This repo provides the base code for pytorch-lightning and weight and biases simultaneous integration.

Convolutional neural network web app trained to track our infant’s sleep schedule using our Google Nest camera.

Wider or Deeper: Revisiting the ResNet Model for Visual Recognition

Learning infinite-resolution image processing with GAN and RL from unpaired image datasets, using a differentiable photo editing model.

PyTorch implementation of a collections of scalable Video Transformer Benchmarks.

PyTorch implementation of the Flow Gaussian Mixture Model (FlowGMM) model from our paper

A pre-trained model with multi-exit transformer architecture.

The Multi-Mission Maximum Likelihood framework (3ML)

Official code for Spoken ObjectNet: A Bias-Controlled Spoken Caption Dataset

Demo notebooks for Qiskit application modules demo sessions (Oct 8 & 15):

Causal Imitative Model for Autonomous Driving