A Low Complexity Speech Enhancement Framework for Full-Band Audio (48kHz) based on Deep Filtering.

Related tags

Deep LearningDeepFilterNet
Overview

DeepFilterNet

A Low Complexity Speech Enhancement Framework for Full-Band Audio (48kHz) based on Deep Filtering.

  • libDF contains Rust code used for data loading and augmentation.
  • DeepFilterNet contains Python code including a libDF wrapper for data loading, DeepFilterNet training, testing and visualization.
  • models contains DeepFilterNet model weights and config.

Usage

System requirements are cargo and pip (Rust and Python package managers). Usage of a conda or virtualenv recommended. This framework is currently only tested under Linux.

Installation of python dependencies and libDF:

cd path/to/DeepFilterNet/  # cd into repository
# Recommended: Install or activate a python env.
pip install maturin  # Used to compile libDF and load
maturin build --release -m DeepFilterNet/Cargo.toml  # Build python wheel
# Install python wheel. Make sure to specify the correct DeepFilterNet and python version
pip install target/wheels/DeepFilterNet-0.1.0-cp39-cp39-linux_x86_64.whl
# Optional: Install cuda version of pytorch from pytorch.org
pip install -r requirements.txt  # Install remaining dependencies

To enhance noisy audio files using DeepFilterNet run

# usage: enhance.py [-h] [--output-dir OUTPUT_DIR] model_base_dir noisy_audio_files [noisy_audio_files ...]
python DeepFilterNet/df/enhance.py models/DeepFilterNet/ path/to/noisy_audio.wav

License

DeepFilterNet is free and open source! All code in this repository is dual-licensed under either:

at your option. This means you can select the license you prefer!

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Comments
  • when i training with MS-SNSD dataset, and prepared the data correctly.it aways get error: RuntimeError: DF dataloader error: ThreadJoinError(

    when i training with MS-SNSD dataset, and prepared the data correctly.it aways get error: RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }")

    config, and logs as below：

    cat base_dir/config.ini [train] seed = 42 device = model = deepfilternet3 mask_only = False df_only = False jit = False batch_size = 1 batch_size_eval = 0 overfit = False num_workers = 0 max_sample_len_s = 5.0 num_prefetch_batches = 32 global_ds_sampling_f = 1.0 dataloader_snrs = -5,0,5,10,20,40 batch_size_scheduling = max_epochs = 10 validation_criteria = loss validation_criteria_rule = min early_stopping_patience = 5 start_eval = False

    [df] sr = 48000 fft_size = 960 hop_size = 480 nb_erb = 32 nb_df = 96 norm_tau = 1 lsnr_max = 35 lsnr_min = -15 min_nb_erb_freqs = 2 df_order = 5 df_lookahead = 0 pad_mode = input

    [deepfilternet] conv_lookahead = 0 conv_ch = 16 conv_depthwise = True convt_depthwise = True conv_kernel = 1,3 conv_kernel_inp = 3,3 emb_hidden_dim = 256 emb_num_layers = 2 emb_gru_skip = none df_hidden_dim = 256 df_gru_skip = none df_pathway_kernel_size_t = 1 enc_concat = False df_num_layers = 3 df_n_iter = 1 linear_groups = 1 enc_linear_groups = 16 mask_pf = False

    [distortion] p_reverb = 0.2 p_bandwidth_ext = 0.0 p_clipping = 0.0 p_zeroing = 0.0 p_air_absorption = 0.0

    [optim] lr = 0.0005 momentum = 0 weight_decay = 0.05 optimizer = adamw lr_min = 1e-06 lr_warmup = 0.0001 warmup_epochs = 3 lr_cycle_mul = 1.0 lr_cycle_decay = 0.5 lr_cycle_epochs = -1 weight_decay_end = -1

    [maskloss] factor = 0 mask = iam gamma = 0.6 gamma_pred = 0.6 f_under = 2

    [spectralloss] factor_magnitude = 0 factor_complex = 0 factor_under = 1 gamma = 1

    [multiresspecloss] factor = 0 factor_complex = 0 gamma = 1 fft_sizes = 512,1024,2048

    [sdrloss] factor = 0

    [localsnrloss] factor = 0.0005

    logs: python df/train.py dataset_small.cfg data-dir-small/ base_dir_small/ 2022-12-21 15:31:00 | INFO | DF | Running on torch 1.12.0+cu102 2022-12-21 15:31:00 | INFO | DF | Running on host research-bj-green-x4-0 fatal: not a git repository (or any of the parent directories): .git 2022-12-21 15:31:00 | INFO | DF | Loading model settings of base_dir_small 2022-12-21 15:31:00 | INFO | DF | Running on device cuda:0 2022-12-21 15:31:00 | INFO | DF | Initializing model deepfilternet3 2022-12-21 15:31:03 | INFO | DF | Initializing dataloader with data directory data-dir-small/ 2022-12-21 15:31:03 | INFO | DF | Loading HDF5 key cache from .cache_dataset_small.cfg 2022-12-21 15:31:03 | INFO | DF | Start train epoch 0 with batch size 1 2022-12-21 15:31:04 | INFO | DF | [0] [0/9] | loss: 0.28417 | lr: 1.000E-04 thread '' panicked at 'assertion failed: k <= self.len()', /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/slice/mod.rs:3094:9 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace 2022-12-21 15:31:04 | ERROR | DF | An error has been caught in function '', process 'MainProcess' (29909), thread 'MainThread' (140548370358784): Traceback (most recent call last):

    File "df/train.py", line 638, in main() └ <function main at 0x7fd3e3bf40d0>

    File "df/train.py", line 248, in main train_loss = run_epoch( └ <function run_epoch at 0x7fd3e3bfc040>

    File "df/train.py", line 355, in run_epoch for i, batch in enumerate(loader.iter_epoch(split, seed)): │ │ │ │ │ └ 0 │ │ │ │ └ 'train' │ │ │ └ <function PytorchDataLoader.iter_epoch at 0x7fd3e3bf4e50> │ │ └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7fd3dc59a5b0> │ └ Batch of size 1: │ SNRs: 40 │ Gains: -6 │
    └ 0

    File "/home/xxx/code/deepfilter/DeepFilterNet-0.4.0/pyDF-data/libdfdata/torch_dataloader.py", line 283, in iter_epoch batch = self._get_batch() │ └ <function PytorchDataLoader._get_batch at 0x7fd3e3bf4d30> └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7fd3dc59a5b0>

    File "/home/xxx/code/deepfilter/DeepFilterNet-0.4.0/pyDF-data/libdfdata/torch_dataloader.py", line 232, in _get_batch _, batch = self.data_queue.get() │ │ └ <function PytorchDataLoader._get_worker_queue_dummy.._Queue.get at 0x7fd3da327310> │ └ <libdfdata.torch_dataloader.PytorchDataLoader._get_worker_queue_dummy.._Queue object at 0x7fd3dca65640> └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7fd3dc59a5b0>

    File "/home/xxx/code/deepfilter/DeepFilterNet-0.4.0/pyDF-data/libdfdata/torch_dataloader.py", line 214, in get self.loader.cleanup() │ │ └ <method 'cleanup' of 'builtins._FdDataLoader' objects> │ └ <builtins._FdDataLoader object at 0x7fd3dc730100> └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7fd3dc59a5b0>

    RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }")

    opened by BWMa 27
  • Question for training

    Question for training

    Hi Rikorose,

    Sorry to bother you again, I try to generate data and train the model according to the training part.

    I generated the training_set.txt (just select 10 files for test.) for speech and made the hdf5.(and so on for noise). Use python df/prepare_data.py --sr 48000 speech training_set.txt TRAIN_SET_SPEECH.hdf5.

    ~/DeepFilterNet/wav/dataset/oblomov_s009036.wav
~/DeepFilterNet/wav/dataset/oblomov_s009040.wav  
~/DeepFilterNet/wav/dataset/oblomov_s009033.wav     
~/DeepFilterNet/wav/dataset/oblomov_s009037.wav    
~/DeepFilterNet/wav/dataset/oblomov_s009041.wav  
~/DeepFilterNet/wav/dataset/oblomov_s009034.wav    
~/DeepFilterNet/wav/dataset/oblomov_s009038.wav     
~/DeepFilterNet/wav/dataset/oblomov_s009042.wav  
~/DeepFilterNet/wav/dataset/oblomov_s009035.wav     
~/DeepFilterNet/wav/dataset/oblomov_s009039.wav

    Generate the dataset.cfg as shown below,

    {
 "train": [
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_SPEECH.hdf5",
      1.0
    ],
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_NOISE.hdf5",
      1.0
    ]
  ],
  "valid": [
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_SPEECH.hdf5",
      0.2
    ],
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_NOISE.hdf5",
      0.2
    ]
  ],
  "test": [
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_SPEECH.hdf5",
      0.2
    ],
    [
      "~/DeepFilterNet/DeepFilterNet-github/DeepFilterNet/hdf5/TRAIN_SET_NOISE.hdf5",
      0.2
    ]
  ]
}

    Encounter some error as shown in the figure below, 擷取

    In addition, I have some questions:

    1. In the command python df/train.py dataset.cfg ~/wav_folder/ ./base_dir/, -Is data_dir a wav folder or an hdf5 folder? (I think is hdf5 folder) -Can base_dir/ not exist? (But we need to give config.ini, so here I enter pertrained_model/ and delete .ckpt)
    2. I found that the log says dataloader len:0, is this a problem?
    3. I remove the all 'df.' of each file while import (ex. from df.config import ... -> from config import ...), otherwise it will cause an import error.

    Thanks,

    opened by aaronhsueh0506 25
  • Reproducing results

    Reproducing results

    Hi,

    I had tried to re-train the deepfilternet model using the DNS-3 challenge dataset mentioned in your work.

    I don't have the additional 10k IR. However, the other dataset remains the same.

    On VCTK test set, using the config.ini in the pre-trained model as my training config, my "best model" on validation gives PESQ score of 2.60. It is much lower than 2.81 from the pre-trained model.

    In config.ini, Adamw is used, while in the paper Adam as optimizer is mentioned.

    Do you think any other factors would result in such a performance drop?

    Could you clarify on the 3 s sample for training? Suppose the DNS-3 sample has 10 s in a sample, do I need to split it into 3 s segments so as to utilize the entire train clip? Or just use the first 3 seconds of the clip? Alternatively, is random 3 s generated on-the-fly while training?

    In the hdf5 setup, does the speech/noise/rir need to have sample number of samples? Or is the noise and RIR sampled randomly from a list? For example, if the speech list has 1000 samples, noise list is 100 samples and rir list is 100 samples, is it okay? or should it be 1000 speech, 1000 noise, 1000 rir? Is it needed to make the duration of speech and noise samples to be the same?

    How about the reverberation parameter p_reverb = 0.05? The data augmentation is performed by default or any other config is needed? conv_lookahead = 2 in config.ini. But the paper mentions "look-ahead of l = 1 frame for both DF as well as in the DNN convolutions".

    stale 
    opened by rohithmars 15
  • RuntimeError: DF dataloader error: ThreadJoinError(

    RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }") when attempting to train DeepFilterNet

    I am trying to train DeepFilterNet but I am running into the following error when training: RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }").

    What I did

    I installed DeepFilterNet via PyPI using:

    pip3 install torch torchvision torchaudio
pip install deepfilternet[train]

    Then I generated the following HDF5 dataset files:

    TEST_SET_NOISE.hdf5
TEST_SET_SPEECH.hdf5
TRAIN_SET_NOISE.hdf5
TRAIN_SET_SPEECH.hdf5
VALID_SET_NOISE.hdf5
VALID_SET_SPEECH.hdf5

    and placed them into the same directory .../data_folder.

    I also created a config.ini file with the following contents:

    config.ini 

    [deepfilternet]
emb_hidden_dim = 256
df_hidden_dim = 256
df_num_layers = 3
conv_ch = 16
conv_lookahead = 0
conv_depthwise = True
convt_depthwise = True
conv_kernel = 1,3
conv_kernel_inp = 3,3
emb_num_layers = 2
emb_gru_skip = none
df_gru_skip = none
df_pathway_kernel_size_t = 1
enc_concat = False
df_n_iter = 1
linear_groups = 1
enc_linear_groups = 16
mask_pf = False
gru_type = grouped
gru_groups = 1
group_shuffle = True
dfop_method = real_unfold
df_output_layer = linear

[df]
fft_size = 960
nb_erb = 32
nb_df = 96
sr = 48000
hop_size = 480
norm_tau = 1
lsnr_max = 35
lsnr_min = -15
min_nb_erb_freqs = 2
df_order = 5
df_lookahead = 0
pad_mode = input

[train]
model = deepfilternet2
batch_size = 16
batch_size_eval = 16
num_workers = 4
overfit = false
lr = 0.001
max_epochs = 30
seed = 42
device = cuda:1
mask_only = False
df_only = False
jit = False
max_sample_len_s = 5.0
num_prefetch_batches = 32
global_ds_sampling_f = 1.0
dataloader_snrs = -5,0,5,10,20,40
batch_size_scheduling = 
validation_criteria = loss
validation_criteria_rule = min
early_stopping_patience = 5
start_eval = False

[distortion]
p_reverb = 0.0
p_bandwidth_ext = 0.0
p_clipping = 0.0
p_air_absorption = 0.0

[optim]
lr = 0.0005
momentum = 0
weight_decay = 0.05
optimizer = adamw
lr_min = 1e-06
lr_warmup = 0.0001
warmup_epochs = 3
lr_cycle_mul = 1.0
lr_cycle_decay = 0.5
lr_cycle_epochs = -1
weight_decay_end = -1

[maskloss]
factor = 0
mask = iam
gamma = 0.6
gamma_pred = 0.6
f_under = 2

[spectralloss]
factor_magnitude = 0
factor_complex = 0
factor_under = 1
gamma = 1

[multiresspecloss]
factor = 0
factor_complex = 0
gamma = 1
fft_sizes = 512,1024,2048

[sdrloss]
factor = 0

[localsnrloss]
factor = 0.0005

    and put it in a directory .../log_folder.

    I also created a dataset.cfg file with the following contents:

    dataset.cfg 

    {
  "train": [
    [
      "TRAIN_SET_SPEECH.hdf5",
      1.0
    ],
    [
      "TRAIN_SET_NOISE.hdf5",
      1.0
    ]
  ],
  "valid": [
    [
      "VALID_SET_SPEECH.hdf5",
      1.0
    ],
    [
      "VALID_SET_NOISE.hdf5",
      1.0
    ]
  ],
  "test": [
    [
      "TEST_SET_SPEECH.hdf5",
      1.0
    ],
    [
      "TEST_SET_NOISE.hdf5",
      1.0
    ]
  ]
}

    Lastly, I ran the command to train the model:

    python df/train.py path/to/dataset.cfg .../data_folder .../log_folder

    and this is the output that I get:

    2022-11-22 22:10:07 | INFO     | DF | Running on torch 1.13.0+cu117
2022-11-22 22:10:07 | INFO     | DF | Running on host workstation3
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
2022-11-22 22:10:07 | INFO     | DF | Loading model settings of log_folder
2022-11-22 22:10:07 | INFO     | DF | Running on device cuda:1
2022-11-22 22:10:07 | INFO     | DF | Initializing model `deepfilternet2`
2022-11-22 22:10:08 | DEPRECATED | DF | Use of `linear` for `df_ouput_layer` is marked as deprecated.
2022-11-22 22:10:08 | INFO     | DF | Initializing dataloader with data directory /home/tester/bokleong/dl_project_dfn/data_folder/
2022-11-22 22:10:08 | INFO     | DF | Loading HDF5 key cache from /home/tester/bokleong/dl_project_dfn/.cache_dataset.cfg
2022-11-22 22:10:08 | INFO     | DF | Start train epoch 0 with batch size 32
thread 'DataLoader Worker 1' panicked at 'called `Option::unwrap()` on a `None` value', libDF/src/dataset.rs:1134:69
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'DataLoader Worker 0' panicked at 'called `Option::unwrap()` on a `None` value', libDF/src/dataset.rs:1134:69
thread 'DataLoader Worker 2' panicked at 'called `Option::unwrap()` on a `None` value', libDF/src/dataset.rs:1134:69
thread 'DataLoader Worker 3' panicked at 'called `Option::unwrap()` on a `None` value', libDF/src/dataset.rs:1134:69
Exception in thread PinMemoryLoop:
Traceback (most recent call last):
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/site-packages/torch/utils/data/_utils/pin_memory.py", line 49, in _pin_memory_loop
    do_one_step()
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/site-packages/torch/utils/data/_utils/pin_memory.py", line 26, in do_one_step
    r = in_queue.get(timeout=MP_STATUS_CHECK_INTERVAL)
  File "/home/tester/miniconda3/envs/user5/lib/python3.9/site-packages/libdfdata/torch_dataloader.py", line 206, in get
    self.loader.cleanup()
RuntimeError: DF dataloader error: ThreadJoinError("Any { .. }")

    Appreciate any help on this, thank you!

    opened by AnonymousEliforp 13
  • Unable to run deepFIlter form CLI

    Unable to run deepFIlter form CLI

    Hi guys, I have followed the installation instruction using pip and now I'm stuck with SLURM and I don't know how to fix it

    I'm executing deepfilternet from colab pro+ account.

    Here is the issue.

    Traceback (most recent call last): File "/usr/local/bin/deepFilter", line 5, in from df.enhance import run File "/usr/local/lib/python3.7/dist-packages/df/enhance.py", line 18, in from df.logger import init_logger, warn_once File "/usr/local/lib/python3.7/dist-packages/df/logger.py", line 49 if (jobid := os.getenv("SLURM_JOB_ID")) is not None: ^ SyntaxError: invalid syntax

    This is which I get while trying to execute this :-> !deepFilter /content/test_audio_053830.wav --output-dir /content

    Anyone got into this kind of issue??

    DO let me know the solution / how can I run this.

    opened by Anand195 13
  • Converter to .onnx model

    Converter to .onnx model

    Hi, I've been trying to use export.py function to convert retrained model to single onnx file, but it seems that there are version mismatch of torch and onnx operation. After some modifications I've almost made it work, but now I'm stuck with

    Traceback (most recent call last):
  File "/data/code_jb/deepfilter2_git/DeepFilterNet/df/export.py", line 327, in <module>
    main(args)
  File "/data/code_jb/deepfilter2_git/DeepFilterNet/df/export.py", line 292, in main
    export(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context                                                                                                                        
    return func(*args, **kwargs)
  File "/data/code_jb/deepfilter2_git/DeepFilterNet/df/export.py", line 160, in export
    export_impl(
  File "/data/code_jb/deepfilter2_git/DeepFilterNet/df/export.py", line 100, in export_impl
    torch.onnx.export(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/__init__.py", line 350, in export
    return utils.export(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 163, in export
    _export(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 1074, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 731, in _model_to_graph
    graph = _optimize_graph(
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 308, in _optimize_graph
    graph = _C._jit_pass_onnx(graph, operator_export_type)
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/__init__.py", line 416, in _run_symbolic_function                                                                                                                      
    return utils._run_symbolic_function(*args, **kwargs)
  File "/home/i.beskrovnyy/miniconda3/envs/df/lib/python3.10/site-packages/torch/onnx/utils.py", line 1421, in _run_symbolic_function                                                                                                                        
    raise symbolic_registry.UnsupportedOperatorError(
torch.onnx.symbolic_registry.UnsupportedOperatorError: Exporting the operator ::view_as_complex to ONNX opset version 14 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub.

    Am I doing something wrong or the latest Deepfilternet2 modifications was not tested with this converter? Or maybe I should use specific versions of onnxruntime/torch?

    opened by JBloodless 12
  • Noise of typing not working well

    Noise of typing not working well

    Hi Rikorose,

    Thanks for working on version 2 of Deepfilternet. Now I can do the real-time inference process with buffer size=1, which is the same as the full signal effect. The point is that the state of the RNN needs to be inherited.

    Now I'm having a trouble with typing/keyboard noise not working well. But I only use spectral loss with c=0.3 in Deepfilternet2 now, will multi-resolution loss improve in this case? or maybe c=0.6 in preious work is better?

    Thanks, Aaron

    opened by aaronhsueh0506 12
  • Implement a real-time loop for DeepFilterNet

    Implement a real-time loop for DeepFilterNet

    Hi Hendrik, Just curious as didn't see any benchmarks but could it run chunking audio as it goes or does it need the overall file to analyse? How does it compare to https://github.com/breizhn/DTLN?

    Thanks Stuart

    enhancement pinned 
    opened by StuartIanNaylor 10
  • All Cargo Tests pass, but train.py returns error

    All Cargo Tests pass, but train.py returns error

    I have been struggling with the same error for a while now...

    I am using a conda environment in linux on a HPC solution. I previously had problems with Rust detecting multiple versions of HDF5 and (I think) I fixed it by defining HDF5_VERSION being the same as library and header (version 1.10.6) and HDF5_DIR as conda environment root. Following the instructions on https://github.com/aldanor/hdf5-rust, I also do

    $ conda env config vars set RUSTFLAGS="-C link-args=-Wl,-rpath,$HDF5_DIR/lib"

    And add the directory to path: $ conda develop /zhome/a7/0/155527/Desktop/s204161/DEEPL_PLS/DeepFilterNet-main/DeepFilterNet/

    Doing this, I manage to succesfully pass the cargo tests (It no longer complains about HDF5 version used by rust not being the same as the one in h5py):

    (DEEPL_PLS) cargo test Finished test [optimized + debuginfo] target(s) in 2.06s Running unittests src/lib.rs (target/debug/deps/libdfdata-beeffcf7e03d4848)

    running 0 tests

    test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/lib.rs (target/debug/deps/libdf-9cd65e9fa81eadda)

    running 0 tests

    test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/lib.rs (target/debug/deps/deep_filter_ladspa-c16148d79da4c5bf)

    running 0 tests

    test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/lib.rs (target/debug/deps/df-f01c984070e59647)

    running 30 tests test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_06 - should panic ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_05 - should panic ... ok test tests::test_erb_inout ... ok test reexport_dataset_modules::dataset::tests::test_mix_audio_signal ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_07 - should panic ... ok test transforms::tests::test_find_max_abs ... ok test reexport_dataset_modules::augmentations::tests::test_rand_resample ... ok test transforms::tests::test_stft_istft_delay ... ok test reexport_dataset_modules::augmentations::tests::test_low_pass ... ok test reexport_dataset_modules::augmentations::tests::test_clipping ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_10 ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_read_pcm ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_04 - should panic ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_02 ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_01 ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_03 ... ok test reexport_dataset_modules::augmentations::tests::test_gen_noise ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_08 ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_slice::case_09 ... ok test transforms::tests::test_ext_bandwidth_spectral ... ok test reexport_dataset_modules::augmentations::tests::test_reverb ... ok test reexport_dataset_modules::dataset::tests::test_fft_dataset ... ok test reexport_dataset_modules::augmentations::tests::test_filters ... ok test reexport_dataset_modules::augmentations::tests::test_compose ... ok test reexport_dataset_modules::dataset::tests::test_td_dataset ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_read_flac ... ok test transforms::tests::test_estimate_bandwidth ... ok test reexport_dataset_modules::augmentations::tests::test_air_absorption ... ok test reexport_dataset_modules::dataset::tests::test_hdf5_read_vorbis ... ok test reexport_dataset_modules::dataloader::tests::test_data_loader ... ok

    test result: ok. 30 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 6.37s

    Doc-tests df

    running 0 tests

    test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

    However when I cd into the repo and run for training it gives the error

    python DeepFilterNet/df/train.py /work3/s204161/config.cfg /work3/s204161/formatted_data/ /zhome/a7/0/155527/Desktop/s204161/DEEPL_PLS/DeepFilterNet/ --debug 2022-11-21 05:42:10 | INFO | df.logger:init_logger:44 | Running on torch 1.12.0 2022-11-21 05:42:10 | INFO | df.logger:init_logger:45 | Running on host n-62-27-19 fatal: not a git repository (or any parent up to mount point /) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). 2022-11-21 05:42:10 | INFO | df.logger:init_logger:67 | Loading model settings of DeepFilterNet 2022-11-21 05:42:10 | INFO | main:main:94 | Running on device cpu 2022-11-21 05:42:10 | INFO | df.model:init_model:21 | Initializing model deepfilternet3 2022-11-21 05:42:10 | INFO | libdfdata.torch_dataloader:init:99 | Initializing dataloader with data directory /work3/s204161/formatted_data/ 2022-11-21 05:42:10 | ERROR | main::633 | An error has been caught in function '', process 'MainProcess' (16810), thread 'MainThread' (139678983296832): Traceback (most recent call last):

    File "/zhome/a7/0/155527/Desktop/s204161/DEEPL_PLS/DeepFilterNet-main/DeepFilterNet/df/train.py", line 633, in main() └ <function main at 0x7f08f2c37b50>

    File "/zhome/a7/0/155527/Desktop/s204161/DEEPL_PLS/DeepFilterNet-main/DeepFilterNet/df/train.py", line 139, in main dataloader = DataLoader( └ <class 'libdfdata.torch_dataloader.PytorchDataLoader'>

    File "/zhome/a7/0/155527/ENTER/envs/DEEPL_PLS/lib/python3.10/site-packages/libdfdata/torch_dataloader.py", line 101, in init self.loader = _FdDataLoader( │ └ <class 'builtins._FdDataLoader'> └ <libdfdata.torch_dataloader.PytorchDataLoader object at 0x7f08f1dc6b60>

    RuntimeError: DF dataset error: Hdf5ErrorDetail { source: H5Fopen(): unable to open file: bad superblock version number, msg: "Error during File::open of dataset /work3/s204161/formatted_data/VALID_SET_SPEECH.hdf5" }

    I have no idea what is left to do. I do not have any prior rust experience. I've attached conda_info.txt should it be relevant and when I run the script:

    import h5py hdfFile = h5py.File('/work3/s204161/formatted_data/TRAIN_SET_SPEECH.hdf5', 'r') print(h5py.version.info) for name in hdfFile: print(hdfFile[name]) print(hdfFile.keys())

    Summary of the h5py configuration

    h5py 3.7.0 HDF5 1.10.6 Python 3.10.8 (main, Nov 4 2022, 13:48:29) [GCC 11.2.0] sys.platform linux sys.maxsize 9223372036854775807 numpy 1.21.5 cython (built with) 0.29.30 numpy (built against) 1.21.5 HDF5 (built against) 1.10.6 <HDF5 group "/speech" (1314 members)> <KeysViewHDF5 ['speech']>

    :(

    conda_info.txt

    opened by Student204161 9
  • About whitenoise performance

    About whitenoise performance

    Thanks for your awesome work! I have installed deepfilternet through pip and test some samples. The overall noise reduction effect is great, but in whitenoise scene, there are more residues in the speech, which leads to a poor subjective feelings. Have you noticed this phenomenon? I will attach the samples below. samples.zip

    stale 
    opened by sqummy 9
  • What is up with the canned deepFilter?

    What is up with the canned deepFilter?

    (DeepFilterNet) C:\Users\sdr\Downloads\Playground\DeepFilterNet>deepFilter test.wav Traceback (most recent call last): File "C:\Users\sdr\anaconda3\envs\DeepFilterNet\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\sdr\anaconda3\envs\DeepFilterNet\lib\runpy.py", line 86, in run_code exec(code, run_globals) File "C:\Users\sdr\anaconda3\envs\DeepFilterNet\Scripts\deepFilter.exe_main.py", line 7, in TypeError: main() missing 1 required positional argument: 'args'

    (DeepFilterNet) C:\Users\sdr\Downloads\Playground\DeepFilterNet>

    stale 
    opened by Joeweav 9
  • build(deps): bump actions/stale from 6 to 7

    build(deps): bump actions/stale from 6 to 7

    Bumps actions/stale from 6 to 7.

    Release notes

    Sourced from actions/stale's releases.

    v7.0.0

    ⚠️ This version contains breaking changes ⚠️

    What's Changed

    Breaking Changes

    • In this release we prevent this action from managing the stale label on items included in exempt-issue-labels and exempt-pr-labels
    • We decided that this is outside of the scope of this action, and to be left up to the maintainer

    New Contributors

    Full Changelog: https://github.com/actions/stale/compare/v6...v7.0.0

    v6.0.1

    Update @​actions/core to 1.10.0 #839

    Full Changelog: https://github.com/actions/stale/compare/v6.0.0...v6.0.1

    Changelog

    Sourced from actions/stale's changelog.

    Changelog

    [7.0.0]

    :warning: Breaking change :warning:

    [6.0.1]

    Update @​actions/core to v1.10.0 (#839)

    [6.0.0]

    :warning: Breaking change :warning:

    Issues/PRs default close-issue-reason is now not_planned(#789)

    [5.1.0]

    Don't process stale issues right after they're marked stale [Add close-issue-reason option]#764#772 Various dependabot/dependency updates

    4.1.0 (2021-07-14)

    Features

    4.0.0 (2021-07-14)

    Features

    Bug Fixes

    • dry-run: forbid mutations in dry-run (#500) (f1017f3), closes #499
    • logs: coloured logs (#465) (5fbbfba)
    • operations: fail fast the current batch to respect the operations limit (#474) (5f6f311), closes #466
    • label comparison: make label comparison case insensitive #517, closes #516
    • filtering comments by actor could have strange behavior: "stale" comments are now detected based on if the message is the stale message not who made the comment(#519), fixes #441, #509, #518

    Breaking Changes

    ... (truncated)

    Commits
    • 6f05e42 draft release for v7.0.0 (#888)
    • eed91cb Update how stale handles exempt items (#874)
    • 10dc265 Merge pull request #880 from akv-platform/update-stale-repo
    • 9c1eb3f Update .md files and allign build-test.yml with the current test.yml
    • bc357bd Update .github/workflows/release-new-action-version.yml
    • 690ede5 Update .github/ISSUE_TEMPLATE/bug_report.md
    • afbcabf Merge branch 'main' into update-stale-repo
    • e364411 Update name of codeql.yml file
    • 627cef3 fix print outputs step (#859)
    • 975308f Merge pull request #876 from jongwooo/chore/use-cache-in-check-dist
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies github_actions 
    opened by dependabot[bot] 0
  • When I retrain the model by models/DeepFilterNet2/config.ini, the result is strange~

    When I retrain the model by models/DeepFilterNet2/config.ini, the result is strange~

    Thanks for your awesome work! I'm trying to recurrence the training through models/DeepFilterNet2/config.ini, but when I use DeepFilterNet/df/enhance.py to test the retrained model, output wavs is nearly to silence. This is my retraining step:Thanks for your awesome work! I'm trying to recurrence the training through models/DeepFilterNet2/config.ini, but when I use DeepFilterNet/df/enhance.py to test the retrained model, output wavs is nearly to silence. This is my retraining step: (1) copy the config.ini of DeepFilterNet2

        cd DeepFilterNet
    mkdir base_dir
    cp ../models/DeepFilterNet2/config.ini ./base_dir

    (2) preparing data take VocalSet_48kHz_mono_000_NA_NA.tar.bz2 as clean data take noise_fullband/datasets_fullband.noise_fullband.audioset_000.tar.bz2 as noise data

        find ../scripts/out/datasets_fullband/clean_fullband/VocalSet_48kHz_mono -iname "*.wav" > training_set_speech.txt
    python df/scripts/prepare_data.py --sr 48000 speech training_set_speech.txt ./data_dir/TRAIN_SET_SPEECH.hdf5
    find ../scripts/out/datasets_fullband/noise_fullband -iname "*.wav" > training_set_noise.txt
    python df/scripts/prepare_data.py --sr 48000 noise training_set_noise.txt ./data_dir/TRAIN_SET_NOISE.hdf5

    this is my dataset.cfg： { "train": [ [ "TRAIN_SET_SPEECH.hdf5", 1.0 ], [ "TRAIN_SET_NOISE.hdf5", 1.0 ], [ "TRAIN_SET_RIR.hdf5", 0.0 ] ], "valid": [ [ "TRAIN_SET_SPEECH.hdf5", 0.2 ], [ "TRAIN_SET_NOISE.hdf5", 0.2 ], [ "TRAIN_SET_RIR.hdf5", 0.0 ] ], "test": [ [ "TRAIN_SET_SPEECH.hdf5", 0.2 ], [ "TRAIN_SET_NOISE.hdf5", 0.2 ], [ "TRAIN_SET_RIR.hdf5", 0.0 ] ] }

    (3) start retraining python df/train.py dataset.cfg ./data_dir/ ./base_dir/

    (4) test retrained model

    python DeepFilterNet/df/enhance.py -m DeepFilterNet/base_dir2 --output-dir DeepFilterNet/test_dir_out/ DeepFilterNet/test_dir_48k/interview_48k.wav

    but the output of the reatrained model is just like this retraing_model_results

    I’m not sure which step is wrong or missed, have you ever met the same problem?

    opened by Yancongs 6
  • Multithread ?

    Multithread ?

    Hi and thanks for a amazing work ! I tested the ladspa filter on a Allwinner H3 (ARM Cortex-A7 Quad-Core 1.296 GHz) , it was easy to install; great work The filter seems to be single-threaded, a single core seems not be enough to process. Would it be possible to involve the other cores ?

    2022-12-22T16:33:37.579Z | WARN |  deep_filter_ladspa | DF 7b7326a9e80d | Underrun detected (RTF: 2.39). Processing too slow!
overrun!!! (at least 277.761 ms long)
2022-12-22T16:33:38.903Z | WARN |  deep_filter_ladspa | DF 3bfb6693797b | Underrun detected (RTF: 2.48). Processing too slow!
overrun!!! (at least 312.537 ms long)
2022-12-22T16:33:40.179Z | WARN |  deep_filter_ladspa | DF 9f61b96d4beb | Underrun detected (RTF: 2.36). Processing too slow!
overrun!!! (at least 266.241 ms long)
2022-12-22T16:33:41.476Z | WARN |  deep_filter_ladspa | DF 54a1822b409f | Underrun detected (RTF: 2.41). Processing too slow!
overrun!!! (at least 284.917 ms long)
    wontfix 
    opened by jenskastensson 2
  • build(deps): update roots requirement from 0.0.7 to 0.0.8 in /libDF

    build(deps): update roots requirement from 0.0.7 to 0.0.8 in /libDF

    Updates the requirements on roots to permit the latest version.

    Release notes

    Sourced from roots's releases.

    December 2022

    Remove debug output, fix quartic equations, allow mutable closures.

    Changelog

    Sourced from roots's changelog.

    0.0.8 - 2022-12-21

    • Remove debug print - thanks to J-F-Liu, Ralith
    • Fixed discriminant check in quartic equations - thanks to jingnanshi
    • Allowed mutable closures for secant etc. - thanks to vron

    0.0.7 - 2021-06-17

    • SearchError becomes public - thanks to JP-Ellis
    • Unnormalized cubic equations are solved using the general formula rather than trigonometrically - thanks to Logicalshift

    0.0.6 - 2019-12-22

    • Fixed cubic equations with very small a3 - thanks to Andrew Hunter
    • Improved quartic equations with multiple roots (for f64; f32 is still a problem) - thanks to Tim Lueke
    • Removed warnings of rustc 1.40.0
    • Switched benchmarks from Bencher to Criterion

    0.0.5 - 2019-01-20

    • Trait Error implemented for SearchError - thanks to phillyfan1138
    • Find roots of higher-degree polynomials using eigenvalues - thanks to stiv-yakovenko
    • Find roots of higher-degree polynomials using Sturm's theorem recursively (experimental)
    • Inverse quadratic approximation

    0.0.4 - 2017-09-05

    • Reduced the performance overhead by using generics - thanks to aepsil0n
    • Handle special cases of quadratic equations - thanks to stiv-yakovenko

    0.0.3 - 2017-03-28

    • New version of the compiler
    • Benchmarks
    • Improved the speed of the Brent-Dekker method
    • Reduced the convergency boilerplate

    0.0.2 - 2015-06-08

    • Fight against the compiler

    [0.0.1] - 2015-03-24

    • Initial version
    Commits

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies rust 
    opened by dependabot[bot] 0
  • Run tract model in half float

    Run tract model in half float

    running

    cargo run -p deep_filter --features=tract,bin,wav-utils,transforms --bin deep-filter -- --half-floats assets/noisy_snr0.wav -o out -vv -D

    results in the following stack trace:

       Compiling deep_filter v0.4.1-pre (/home/hendrik/projects/DeepFilterNet/libDF)
    Finished dev [optimized + debuginfo] target(s) in 20.20s
     Running `target/debug/deep-filter --half-floats assets/noisy_snr0.wav -o out -vv -D`
[2022-12-13T10:09:59Z INFO  df::tract] Init encoder with delay: 0
[2022-12-13T10:09:59Z INFO  tract_linalg::x86_64_fma] mmm_f32, sigmoid_f32, tanh_f32: x86_64/fma activated
[2022-12-13T10:09:59Z INFO  tract_linalg::x86_64_fma] mmm_i8_i8 and mmm_i8_i32: x86_64/avx2 activated
[2022-12-13T10:09:59Z INFO  df::tract] Init ERB decoder with delay: 0
[2022-12-13T10:09:59Z INFO  df::tract] Init DF decoder with delay: 0
[2022-12-13T10:09:59Z INFO  df::tract] Running with model type deepfilternet3 lookahead 2
Error: Evaluating #3 "/erb_conv0/1/Conv.pad" Pad

Caused by:
    Tensor datum type error: tensor is F32, accessed as F16

Stack backtrace:
   0: tract_data::tensor::Tensor::check_for_access
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-data-0.18.5/src/tensor.rs:748:13
   1: tract_data::tensor::Tensor::to_scalar
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-data-0.18.5/src/tensor.rs:848:9
   2: tract_core::ops::array::pad::Pad::eval_t
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/ops/array/pad.rs:34:38
      <tract_core::ops::array::pad::Pad as tract_core::ops::EvalOp>::eval
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/ops/array/pad.rs:109:18
   3: tract_core::plan::eval
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/plan.rs:491:17
   4: core::ops::function::FnMut::call_mut
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/ops/function.rs:164:5
      tract_core::plan::SimpleState<F,O,M,P>::exec_plan_with_eval
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/plan.rs:264:26
   5: tract_core::plan::SimpleState<F,O,M,P>::run_plan_with_eval
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/plan.rs:199:9
      tract_core::plan::SimpleState<F,O,M,P>::run
             at /home/hendrik/.cargo/registry/src/github.com-1ecc6299db9ec823/tract-core-0.18.5/src/plan.rs:177:9
   6: df::tract::DfTract::process
             at ./libDF/src/tract.rs:401:17
   7: deep_filter::main
             at ./libDF/src/bin/enhance_wav.rs:139:13
   8: core::ops::function::FnOnce::call_once
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/ops/function.rs:248:5
      std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/sys_common/backtrace.rs:122:18
   9: std::rt::lang_start::{{closure}}
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/rt.rs:166:18
  10: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/ops/function.rs:283:13
      std::panicking::try::do_call
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:492:40
      std::panicking::try
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:456:19
      std::panic::catch_unwind
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/rt.rs:148:48
      std::panicking::try::do_call
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:492:40
      std::panicking::try
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:456:19
      std::panic::catch_unwind
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/rt.rs:148:20
  11: main
  12: __libc_start_call_main
  13: __libc_start_main_alias_2
  14: _start
    opened by Rikorose 2
Releases(v0.4.0)
Owner
Hendrik Schröter
Hendrik Schröter
GitHub Repository
Deep learning (neural network) based remote photoplethysmography: how to extract pulse signal from video using deep learning tools

Deep-rPPG: Camera-based pulse estimation using deep learning tools Deep learning (neural network) based remote photoplethysmography: how to extract pu

Terbe Dániel 138 Dec 17, 2022
CycleTransGAN-EVC: A CycleGAN-based Emotional Voice Conversion Model with Transformer

CycleTransGAN-EVC CycleTransGAN-EVC: A CycleGAN-based Emotional Voice Conversion Model with Transformer Demo emotion CycleTransGAN CycleTransGAN Cycle

24 Dec 15, 2022
Image augmentation library in Python for machine learning.

Augmentor is an image augmentation library in Python for machine learning. It aims to be a standalone library that is platform and framework independe

Marcus D. Bloice 4.8k Jan 07, 2023
A Comprehensive Empirical Study of Vision-Language Pre-trained Model for Supervised Cross-Modal Retrieval

CLIP4CMR A Comprehensive Empirical Study of Vision-Language Pre-trained Model for Supervised Cross-Modal Retrieval The original data and pre-calculate

24 Dec 26, 2022
Official implementation for "Image Quality Assessment using Contrastive Learning"

Image Quality Assessment using Contrastive Learning Pavan C. Madhusudana, Neil Birkbeck, Yilin Wang, Balu Adsumilli and Alan C. Bovik This is the offi

Pavan Chennagiri 67 Dec 30, 2022
Aircraft design optimization made fast through modern automatic differentiation

Aircraft design optimization made fast through modern automatic differentiation. Plug-and-play analysis tools for aerodynamics, propulsion, structures, trajectory design, and much more.

Peter Sharpe 394 Dec 23, 2022
[ICLR 2021] Is Attention Better Than Matrix Decomposition?

Enjoy-Hamburger 🍔 Official implementation of Hamburger, Is Attention Better Than Matrix Decomposition? (ICLR 2021) Under construction. Introduction T

Gsunshine 271 Dec 29, 2022
A resource for learning about deep learning techniques from regression to LSTM and Reinforcement Learning using financial data and the fitness functions of algorithmic trading

A tour through tensorflow with financial data I present several models ranging in complexity from simple regression to LSTM and policy networks. The s

195 Dec 07, 2022
Official repository for CVPR21 paper "Deep Stable Learning for Out-Of-Distribution Generalization".

StableNet StableNet is a deep stable learning method for out-of-distribution generalization. This is the official repo for CVPR21 paper "Deep Stable L

120 Dec 28, 2022
A Simulation Environment to train Robots in Large Realistic Interactive Scenes

iGibson: A Simulation Environment to train Robots in Large Realistic Interactive Scenes iGibson is a simulation environment providing fast visual rend

Stanford Vision and Learning Lab 493 Jan 04, 2023
Repository aimed at compiling code, papers, demos etc.. related to my PhD on 3D vision and machine learning for fruit detection and shape estimation at the university of Lincoln

PhD_3DPerception Repository aimed at compiling code, papers, demos etc.. related to my PhD on 3D vision and machine learning for fruit detection and s

lelouedec 2 Oct 06, 2022
Training a deep learning model on the noisy CIFAR dataset

Training-a-deep-learning-model-on-the-noisy-CIFAR-dataset This repository contai

1 Jun 14, 2022
Unofficial keras(tensorflow) implementation of MAE model from Masked Autoencoders Are Scalable Vision Learners

MAE-keras Unofficial keras(tensorflow) implementation of MAE model described in 'Masked Autoencoders Are Scalable Vision Learners'. This work has been

Yewon 11 Jun 12, 2022
An implementation of the BADGE batch active learning algorithm.

Batch Active learning by Diverse Gradient Embeddings (BADGE) An implementation of the BADGE batch active learning algorithm. Details are provided in o

125 Dec 24, 2022
Code-free deep segmentation for computational pathology

NoCodeSeg: Deep segmentation made easy! This is the official repository for the manuscript "Code-free development and deployment of deep segmentation

André Pedersen 26 Nov 23, 2022
The official homepage of the COCO-Stuff dataset.

The COCO-Stuff dataset Holger Caesar, Jasper Uijlings, Vittorio Ferrari Welcome to official homepage of the COCO-Stuff [1] dataset. COCO-Stuff augment

Holger Caesar 715 Dec 31, 2022
Real-world Anomaly Detection in Surveillance Videos- pytorch Re-implementation

Real world Anomaly Detection in Surveillance Videos : Pytorch RE-Implementation This repository is a re-implementation of "Real-world Anomaly Detectio

seominseok 62 Dec 08, 2022
Single-step adversarial training (AT) has received wide attention as it proved to be both efficient and robust.

Subspace Adversarial Training Single-step adversarial training (AT) has received wide attention as it proved to be both efficient and robust. However,

15 Sep 02, 2022
This program can detect your face and add an Christams hat on the top of your head

Auto_Christmas This program can detect your face and add a Christmas hat to the top of your head. just run the Auto_Christmas.py, then you can see the

3 Dec 22, 2021
The source code for CATSETMAT: Cross Attention for Set Matching in Bipartite Hypergraphs

catsetmat The source code for CATSETMAT: Cross Attention for Set Matching in Bipartite Hypergraphs To be able to run it, add catsetmat to PYTHONPATH H

2 Dec 19, 2022