WeNet
Discussions | Docs | Papers | Runtime (x86) | Runtime (android) | Pretrained Models
We share neural Net together.
The main motivation of WeNet is to close the gap between research and production end-to-end (E2E) speech recognition models, to reduce the effort of productionizing E2E models, and to explore better E2E models for production.
Highlights
- Production first and production ready: The python code of WeNet meets the requirements of TorchScript, so the model trained by WeNet can be directly exported by Torch JIT and use LibTorch for inference. There is no gap between the research model and production model. Neither model conversion nor additional code is required for model inference.
- Unified solution for streaming and non-streaming ASR: WeNet implements Unified Two Pass (U2) framework to achieve accurate, fast and unified E2E model, which is favorable for industry adoption.
- Portable runtime: Several demos will be provided to show how to host WeNet trained models on different platforms, including server x86 and on-device android.
- Light weight: WeNet is designed specifically for E2E speech recognition, with clean and simple code. It is all based on PyTorch and its corresponding ecosystem. It has no dependency on Kaldi, which simplifies installation and usage.
Performance Benchmark
Please see examples/$dataset/s0/README.md for benchmark on different speech datasets.
- Mandarin Chinese
- English
Installation
- Clone the repo
git clone https://github.com/wenet-e2e/wenet.git
- Install Conda: please see https://docs.conda.io/en/latest/miniconda.html
- Create Conda env:
conda create -n wenet python=3.8
conda activate wenet
pip install -r requirements.txt
conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c conda-forge
- Optionally, if you want to use x86 runtime or language model(LM), you have to build the runtime as follows. Otherwise, you can just ignore this step.
# runtime build requires cmake 3.14 or above
cd runtime/server/x86
mkdir build && cd build && cmake .. && cmake --build .
Discussion & Communication
Please visit Discussions for further discussion.
For Chinese users, you can aslo scan the QR code on the left to follow our offical account of WeNet. We created a WeChat group for better discussion and quicker response. Please scan the personal QR code on the right, and the guy is responsible for inviting you to the chat group.
If you can not access the QR image, please access it on gitee.
 |
 |
|---|
Or you can directly discuss on Github Issues.
Contributors
 |
 |
 |
 |
 |
|---|
Acknowledge
- We borrowed a lot of code from ESPnet for transformer based modeling.
- We borrowed a lot of code from Kaldi for WFST based decoding for LM integration.
- We referred EESEN for building TLG based graph for LM integration.
- We referred to OpenTransformer for python batch inference of e2e models.
Citations
@inproceedings{yao2021wenet,
title={WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit},
author={Yao, Zhuoyuan and Wu, Di and Wang, Xiong and Zhang, Binbin and Yu, Fan and Yang, Chao and Peng, Zhendong and Chen, Xiaoyu and Xie, Lei and Lei, Xin},
booktitle={Proc. Interspeech},
year={2021},
address={Brno, Czech Republic }
organization={IEEE}
}
@article{zhang2020unified,
title={Unified Streaming and Non-streaming Two-pass End-to-end Model for Speech Recognition},
author={Zhang, Binbin and Wu, Di and Yao, Zhuoyuan and Wang, Xiong and Yu, Fan and Yang, Chao and Guo, Liyong and Hu, Yaguang and Xie, Lei and Lei, Xin},
journal={arXiv preprint arXiv:2012.05481},
year={2020}
}
@article{wu2021u2++,
title={U2++: Unified Two-pass Bidirectional End-to-end Model for Speech Recognition},
author={Wu, Di and Zhang, Binbin and Yang, Chao and Peng, Zhendong and Xia, Wenjing and Chen, Xiaoyu and Lei, Xin},
journal={arXiv preprint arXiv:2106.05642},
year={2021}
}
![[Draft] Cache control v2](https://avatars.githubusercontent.com/u/13466943?v=4)
85 Dec 22, 2022
2k Dec 31, 2022
0 Nov 01, 2022
71 Dec 25, 2022
441 Dec 25, 2022
201 Dec 21, 2022
1.1k Dec 30, 2022
150 Dec 14, 2022
20 Oct 24, 2022
185 Jan 09, 2023
2 Dec 14, 2021
1 Jan 18, 2022
37 Dec 30, 2022
75 Dec 26, 2022
195 Dec 29, 2022
22 Jul 12, 2022
11 Oct 18, 2022
359 Jan 01, 2023
3 Jun 28, 2022
353 Dec 27, 2022