L3DAS22 challenge supporting API

Overview

L3DAS22 challenge supporting API

This repository supports the L3DAS22 IEEE ICASSP Grand Challenge and it is aimed at downloading the dataset, pre-processing the sound files and the metadata, training and evaluating the baseline models and validating the final results. We provide easy-to-use instruction to produce the results included in our paper. Moreover, we extensively commented our code for easy customization.

For further information please refer to the challenge website and to the challenge documentation.

Installation

Our code is based on Python 3.7.

To install all required dependencies run:

pip install -r requirements.txt

Follow these instructions to properly create and place the kaggle.json file.

Dataset download

It is possible to download the entire dataset through the script download_dataset.py. This script downloads the data, extracts the archives, merges the 2 parts of task1 train360 files and prepares all folders for the preprocessing stage.

To download run this command:

python download_dataset.py --output_path ./DATASETS --unzip True

This script may take long, especially the unzipping stage.

Alternatively, it is possible to manually download the dataset from Kaggle.

The train360 section of task 1 is split in 2 downloadable files. If you manually download the dataset, you should manually merge the content of the 2 folders. You can use the function download_dataset.merge_train360(). Example:

import download_dataset

train360_path = "path_that_contains_both_train360_parts"
download_dataset.merge_train360(train360_path)

Pre-processing

The file preprocessing.py provides automated routines that load the raw audio waveforms and their correspondent metadata, apply custom pre-processing functions and save numpy arrays (.pkl files) containing the separate predictors and target matrices.

Run these commands to obtain the matrices needed for our baseline models:

python preprocessing.py --task 1 --input_path DATASETS/Task1 --training_set train100 --num_mics 1
python preprocessing.py --task 2 --input_path DATASETS/Task2 --num_mics 1 --frame_len 100

The two tasks of the challenge require different pre-processing.

For Task1 the function returns 2 numpy arrays contatining:

  • Input multichannel audio waveforms (3d noise+speech scenarios) - Shape: [n_data, n_channels, n_samples].
  • Output monoaural audio waveforms (clean speech) - Shape [n_data, 1, n_samples].

For Task2 the function returns 2 numpy arrays contatining:

  • Input multichannel audio spectra (3d acoustic scenarios): Shape: [n_data, n_channels, n_fft_bins, n_time_frames].
  • Output seld matrices containing the class ids of all sounds present in each 100-milliseconds frame alongside with their location coordinates - Shape: [n_data, n_frames, ((n_classes * n_class_overlaps) + (n_classes * n_class_overlaps * n_coordinates))], where n_class_overlaps is the maximum amount of possible simultaneous sounds of the same class (3) and n_coordinates refers to the spatial dimensions (3).

Baseline models

We provide baseline models for both tasks, implemented in PyTorch. For task 1 we use a Beamforming U-Net and for task 2 an augmented variant of the SELDNet architecture. Both models are based on the single-microphone dataset configuration. Moreover, for task 1 we used only Train100 as training set.

To train our baseline models with the default parameters run:

python train_baseline_task1.py
python train_baseline_task2.py

These models will produce the baseline results mentioned in the paper.

GPU is strongly recommended to avoid very long training times.

Alternatively, it is possible to download our pre-trained models with these commands:

python download_baseline_models.py --task 1 --output_path RESULTS/Task1/pretrained
python download_baseline_models.py --task 2 --output_path RESULTS/Task2/pretrained

These models are also available for manual download here.

We also provide a Replicate interactive demo of both baseline models.

Evaluaton metrics

Our evaluation metrics for both tasks are included in the metrics.py script. The functions task1_metric and location_sensitive_detection compute the evaluation metrics for task 1 and task 2, respectively. The default arguments reflect the challenge requirements. Please refer to the above-linked challenge paper for additional information about the metrics and how to format the prediction and target vectors.

Example:

import metrics

task1_metric = metrics.task1_metric(prediction_vector, target_vector)
_,_,_,task2_metric = metrics.location_sensitive_detection(prediction_vector, target_vector)

To compute the challenge metrics for our basiline models run:

python evaluate_baseline_task1.py
python evaluate_baseline_task2.py

In case you want to evaluate our pre-trained models, please add --model_path path/to/model to the above commands.

Submission shape validation

The script validate_submission.py can be used to assess the validity of the submission files shape. Instructions about how to format the submission can be found in the L3das website Use these commands to validate your submissions:

python validate_submission.py --task 1 --submission_path path/to/task1_submission_folder --test_path path/to/task1_test_dataset_folder

python validate_submission.py --task 2 --submission_path path/to/task2_submission_folder --test_path path/to/task2_test_dataset_folder

For each task, this script asserts if:

  • The number of submitted files is correct
  • The naming of the submitted files is correct
  • Only the files to be submitted are present in the folder
  • The shape of each submission file is as expected

Once you have valid submission folders, please follow the instructions on the link above to proceed with the submission.

Owner
L3DAS
The L3DAS project aims at providing new 3D audio datasets and encouraging the proliferation of new deep learning methods for 3D audio analysis.
L3DAS
Python written Rule34 API

Python written Rule34 API

1 Nov 11, 2021
python based bot Sends notification to your telegram whenever a new video is released on a youtube channel!

YTnotifier python based bot Sends notification to your telegram whenever a new video is released on a youtube channel! REQUIREMENTS telethon python-de

Mohamed Rizad 6 Jul 23, 2022
My beancount practice as a template

my-beancount-template 个人 Beancount 方案的模板仓库 相关博客 复式记账指北(一):What and Why? 复式记账指北(二):做账方法论 复式记账指北(三):如何打造不半途而废的记账方案 配置 详细配置请参考博客三。必须修改的配置有: Bot功能:data/be

KAAAsS 29 Nov 29, 2022
A library for demo trading | backtest and forward test simulation

Trade Engine a library for demo trading | backtest and forward test simulation Features Limit/Market orders: you can place a Limit or Market order in

Ali Moradi 7 Jul 02, 2022
A play store search telegram bot

Play-Store-Bot A play store search telegram bot Made with Python3 (C) @FayasNoushad Copyright permission under MIT License License - https://github.c

Fayas Noushad 17 Oct 28, 2022
Python API for Photoshop.

Python API for Photoshop. The example above was created with Photoshop Python API.

Hal 372 Jan 02, 2023
A telegram bot script for generating session string using pyrogram and telethon on Telegram bot

String-session-Bot Telegram Bot to generate Pyrogram and Telethon String Session. A star ⭐ from you means a lot to us! Usage Deploy to Heroku Tap on a

Wahyusaputra 8 Oct 28, 2022
Python implementation of Spotify's authorization flow.

Spotify API Apps 🎷 🎶 🎼 This repository consists of many strange codes that make you think why the hell this guy doing this. Well... I got some reas

5 Dec 17, 2021
OpenQuake's Engine for Seismic Hazard and Risk Analysis

OpenQuake Engine The OpenQuake Engine is an open source application that allows users to compute seismic hazard and seismic risk of earthquakes on a g

Global Earthquake Model 281 Dec 21, 2022
对hermit 的API进行简单的封装,做成了这个python moudle

hermit-py 对hermit 的API进行简单的封装,做成了这个Python Moudle,推荐通过wheel的方式安装。 目前对点击、滑动、模拟输入、找组件、等支持较好,支持查看页面的实时布局信息,再通过布局信息进行点击滑动等操作。 支持剪贴板相关的操作,支持设置剪贴的任意语言内容。

LookCos 40 Jun 25, 2022
Darkflame Universe Account Manager

Darkflame Universe Account Manager This is a quick and simple web application intended for account creation and management for a DLU instance created

31 Nov 29, 2022
A Telegram Bot written in Python for mirroring files on the Internet to your Google Drive

No support is going to be provided of any kind, only maintaining this for vps user on request. This is a Telegram Bot written in Python for mirroring

Sunil Kumar 42 Oct 28, 2022
API de mi aplicación de Biblioteca

BOOKSTORE API Instalación/Configuración Previo Es una buena idea crear un entorno virtual antes de instalar las dependencias. Puedes hacerlo con el si

Gabriel Morales 1 Jan 09, 2022
Telegram bot to scrape images from the reddit universe

Telegram bot to scrape images from the reddit universe

XD22 3 Sep 30, 2022
Is the CoWin website updated for registration?

CoWin-Update Is the CoWin website updated for registration? This is a very hacky PYTHON3 script to lookup the CoWin portal if they re-deployed their J

Yash Jakhotiya 5 May 10, 2021
Oussama has taken his first dose of vaccine D days ago

Oussama has taken his first dose of vaccine D days ago. He may take the second dose no less than L days and no more than R days since his first dose. Determine if Oussama is too early, too late, or i

INDIA - ENSAM Rabat 2 Feb 01, 2022
An advanced telegram language translator bot

Made with Python3 (C) @FayasNoushad Copyright permission under MIT License License - https://github.com/FayasNoushad/Translator-Bot-V3/blob/main/LICE

Fayas Noushad 19 Dec 24, 2022
This repository contains free labs for setting up an entire workflow and DevOps environment from a real-world perspective in AWS

DevOps-The-Hard-Way-AWS This tutorial contains a full, real-world solution for setting up an environment that is using DevOps technologies and practic

Mike Levan 1.6k Jan 05, 2023
A Python app which retrieves the rank and players' equipped skins during a match

VALORANT rank yoinker About The Project Usage Contributing Contact Acknowledgements Disclaimer About The Project Their Queue Current Skin Current Rank

Isaac Kenyon 270 Jan 04, 2023
Auto Join: A GitHub action script to automatically invite everyone to the organization who comment at the issue page.

Auto Invite To Org By Issue Comment A GitHub action script to automatically invite everyone to the organization who comment at the issue page. What is

Max Base 6 Jun 08, 2022