Streaming over lightweight data transformations

Overview

slide

PyPI version Build Status Codecoverage Codacy Badge License DOI

Description

Data augmentation libarary for Deep Learning, which supports images, segmentation masks, labels and keypoints. Furthermore, SOLT is fast and has OpenCV in its backend. Full auto-generated docs and examples are available here: https://mipt-oulu.github.io/solt/.

Features

  • Support of Images, masks and keypoints for all the transforms (including multiple items at the time)
  • Fast and PyTorch-integrated
  • Convenient and flexible serialization API
  • Excellent documentation
  • Easy to extend
  • 100% Code coverage

Examples

Images: Cats Images + Keypoints: Cats Medical Images + Binary Masks: Brain MRI Medical Images + Multiclass Masks Knee MRI

E.g. the last row is generated using the following transforms stream.

stream = solt.Stream([
    slt.Rotate(angle_range=(-20, 20), p=1, padding='r'),
    slt.Crop((256, 256)),
    solt.SelectiveStream([
        slt.GammaCorrection(gamma_range=0.5, p=1),
        slt.Noise(gain_range=0.1, p=1),
        slt.Blur()    
    ], n=3)
])

img_aug, mask_aug = stream({'image': img, 'mask': mask})

If you want to visualize the results, you need to modify the execution of the transforms:

img_aug, mask_aug = stream({'image': img, 'mask': mask}, return_torch=False).data

Installation

The most recent version is available in pip:

pip install solt

You can fetch the most fresh changes from this repository:

pip install git+https://github.com/MIPT-Oulu/solt

Benchmark

We propose a fair benchmark based on the refactored version of the one proposed by albumentations team, but here, we also convert the results into a PyTorch tensor and do the ImageNet normalization. The following numbers support a realistic and honest comparison between the libraries (number of images per second, the higher - the better):

albumentations
0.4.3
torchvision (Pillow-SIMD backend)
0.5.0
augmentor
0.2.8
solt
0.1.9
HorizontalFlip 2253 2549 2561 3530
VerticalFlip 2380 2557 2572 3740
RotateAny 1479 1389 670 2070
Crop224 2566 1966 1981 4281
Crop128 5467 5738 5720 7186
Crop64 9285 9112 9049 10345
Crop32 11979 10550 10607 12348
Pad300 1642 109 - 2631
VHFlipRotateCrop 1574 1334 616 1889
HFlipCrop 2391 1943 1917 3572

Python and library versions: Python 3.7.0 (default, Oct 9 2018, 10:31:47) [GCC 7.3.0], numpy 1.18.1, pillow-simd 7.0.0.post3, opencv-python 4.2.0.32, scikit-image 0.16.2, scipy 1.4.1.

The code was run on AMD Threadripper 1900. Please find the details about the benchmark here.

How to contribute

Follow the guidelines described here.

Author

Aleksei Tiulpin, Research Unit of Medical Imaging, Physics and Technology, University of Oulu, Finalnd.

How to cite

If you use SOLT and cite it in your research, please, don't hesitate to sent an email to Aleksei Tiulpin. All the papers that use SOLT are listed here.

@misc{solt2019,
  author       = {Aleksei Tiulpin},
  title        = {SOLT: Streaming over Lightweight Transformations},
  month        = jul,
  year         = 2019,
  version      = {v0.1.9},
  doi          = {10.5281/zenodo.3702819},
  url          = {https://doi.org/10.5281/zenodo.3702819}
}
Comments
  • Crashed without clear error message when tranforming multiple images with different shapes

    Crashed without clear error message when tranforming multiple images with different shapes

    Code to reproduce:

    import solt
    import solt.transforms as slt
    import numpy as np
    
    
    if __name__ == "__main__":
        trf = solt.Stream([slt.Resize(resize_to=(50, 50)),
                           slt.Flip()])
        img1 = np.ones((100, 100, 3), dtype=np.int32)
        img2 = np.ones((110, 110, 3), dtype=np.int32)
        trf_img = trf({'images': (img1, img2)})
        print('Done')
    

    Adding a resize transformation at the beginning doesn't help. A clear error message is needed for this case.

    opened by hoanghng 4
  • NameError: name 'ktps' is not defined

    NameError: name 'ktps' is not defined

    While testing one of the notebook algorithm, this error occured: kpts = None for annotation_fname in glob.glob(os.path.join('Data', 'helen_annotations', '*.txt')): with open(annotation_fname) as f: if f.readline()[:-1] == fname.split('.')[0]: ktps = [] for l in f: tmp = l.split() ktps.append([float(tmp[0]), float(tmp[2])]) break kpts = np.array(ktps)

    Specific Error:

    NameError Traceback (most recent call last) in 8 ktps.append([float(tmp[0]), float(tmp[2])]) 9 break ---> 10 kpts = np.array(ktps)

    NameError: name 'ktps' is not defined

    opened by tobimichigan 3
  • Use pytorch for 3D+ data

    Use pytorch for 3D+ data

    Proof of concept.

    • Switched to pytorch for 3D+ in Flip, Crop and Pad
    • Flip now accepts multiple axes, but not axis=-1. Also, the axis order has been switched from opencv to numpy
    • Minor cleanup to the tests
    opened by soupault 2
  • nd follow-up

    nd follow-up

    • Removed height and width concepts where possible
    • Switched from shape checking decorator to function
    • Added shape validation also for masks and keypoints
    • Splitted utils into checks and serial
    opened by soupault 2
  • Bump opencv-python-headless from 4.1.2.30 to 4.2.0.32 in /ci

    Bump opencv-python-headless from 4.1.2.30 to 4.2.0.32 in /ci

    Bumps opencv-python-headless from 4.1.2.30 to 4.2.0.32.

    Release notes

    Sourced from opencv-python-headless's releases.

    4.2.0.32

    OpenCV version 4.2.0.

    Changes:

    • macOS environment updated from xcode8.3 to xcode 9.4
    • macOS uses now Qt 5 instead of Qt 4
    • Nasm version updated to Docker containers
    • multibuild updated

    Fixes:

    • don't use deprecated brew tap-pin, instead refer to the full package name when installing #267
    • replace get_config_var() with get_config_vars() in setup.py #274
    • add workaround for DLL errors in Windows Server #264
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Fix the documentation bug

    Fix the documentation bug

    https://github.com/MIPT-Oulu/solt/blob/485bfb0d471134f75e09d06aa5e9ce4b57c0e13c/solt/transforms/_transforms.py#L1018 tells something about noise. Should talk about the intensities instead.

    opened by lext 1
  • Compatibility issues with latest pytorch

    Compatibility issues with latest pytorch

    Works fine with 1.1, but not 1.5. Traceback:

        as_dict=as_dict, scale_keypoints=scale_keypoints, normalize=normalize, mean=mean, std=std,
      File ".../site-packages/solt/core/_data.py", line 273, in to_torch
        img.sub_(mean)
    RuntimeError: output with backend CPU and dtype Byte doesn't match the desired backend CPU and dtype Float
    Process finished with exit code 1
    
    opened by soupault 1
  • Can't run unit tests locally on MacOS

    Can't run unit tests locally on MacOS

    🐛 Bug

    It is not possible to run unit tests locally on MacOS X (10.12).

    To Reproduce

    Steps to reproduce the behavior:

    1. Create a conda environment and install all of the packages as suggested here.

    2. Run the following code: pytest tests

    3. The unit tests fail and generate an error:

    ImportError while importing test module '/Users/melekhi1/projects/solt/tests/test_data_core.py'.
    Hint: make sure your test modules/packages have valid Python names.
    Traceback:
    tests/test_data_core.py:4: in <module>
        import cv2
    ../../anaconda2/envs/solt_test_env/lib/python3.8/site-packages/cv2/__init__.py:3: in <module>
        from .cv2 import *
    E   ImportError: dlopen(/Users/melekhi1/anaconda2/envs/solt_test_env/lib/python3.8/site-packages/cv2/cv2.cpython-38-darwin.so, 2): Symbol not found: _inflateValidate
    E     Referenced from: /Users/melekhi1/anaconda2/envs/solt_test_env/lib/python3.8/site-packages/cv2/.dylibs/libpng16.16.dylib (which was built for Mac OS X 10.13)
    E     Expected in: /usr/lib/libz.1.dylib
    E    in /Users/melekhi1/anaconda2/envs/solt_test_env/lib/python3.8/site-packages/cv2/.dylibs/libpng16.16.dylib
    

    Expected behavior

    The unit tests should pass

    =============================== test session starts ================================
    platform darwin -- Python 3.8.2, pytest-3.6.4, py-1.8.1, pluggy-0.7.1
    rootdir: /Users/melekhi1/projects/solt, inifile:
    plugins: pep8-1.0.6, flake8-1.0.2, cov-2.6.0
    collected 701 items
    
    tests/test_base_transforms.py .............................................. [  6%]
    ............................................................................ [ 17%]
    .....                                                                        [ 18%]
    tests/test_data_core.py .................................................... [ 25%]
    ............................................................................ [ 36%]
    ............................................................................ [ 47%]
    ..................................................................           [ 56%]
    tests/test_transforms.py ................................................... [ 63%]
    ............................................................................ [ 74%]
    ............................................................................ [ 85%]
    ............................................................................ [ 96%]
    .........                                                                    [ 97%]
    tests/test_utils.py ................                                         [100%]
    
    ============================ 701 passed in 6.79 seconds ============================
    

    Suggestions

    It seems the bug is related to the latest version of opencv-python-headless (4.2.0.32) package. Apparently, its previous version, 4.1.2.30, could solve the issue at least for this particular platform (MacOS 10.12). It would be great to test this solution on more recent MacOS.

    opened by imelekhov 1
  • Incorrect serialization of a cropping transform

    Incorrect serialization of a cropping transform

    The attribute crop_size needs to be renamed into crop_to to match the constructor. Otherwise, the serialization does not work well.

    https://github.com/MIPT-Oulu/solt/blob/770e397884bcafe80a11723c229e275c1c1f8b5a/solt/transforms/_transforms.py#L715

    bug 
    opened by lext 1
  • Fair benchmark with the other libraries

    Fair benchmark with the other libraries

    the augbench package needs further development and its results need to be reported in the README.

    the benchmark needs to cover random transformations rather than the static ones. It is important to make comparison for three cases: image, image mask, and image+10 masks (instance segmentation task).

    opened by lext 1
  • Brightness needs to support percentages of the mean intensity

    Brightness needs to support percentages of the mean intensity

    The use case for this is that for some images it might be better to rely on their mean intensity and increase their intensity by n% of the mean value.

    enhancement 
    opened by lext 0
  • Allow to randomly select random number of augmentations in Selective stream

    Allow to randomly select random number of augmentations in Selective stream

    Currently, selective stream takes a parameter that tells how many transforms to select from the list of transformations. I was recently suggested to make it a range and allow to sample random number of augmentations out of this range.

    feature request 
    opened by lext 0
Releases(v0.1.9)
  • v0.1.9(Mar 10, 2020)

    • [x] SOLT is now PyTorch Native
    • [x] All the transform subtract the mean and std by default
    • [x] Implemented a proper serialization and deserialization
    • [x] Allowed to use dict instead of a data container
    • [x] Introduced shorter transform names
    • [x] Implemented a fair benchmark to compare with other libraries
    • [x] Fixed a bug of JPEGCompression (@tiulpin )
    • [x] Added IntensityRemappping transform (@soupault )

    This release is not backward comparible

    Source code(tar.gz)
    Source code(zip)
  • v0.1.8(Jul 26, 2019)

  • v0.1.7(Jul 6, 2019)

    This release has some tests-related features and also adds two new transforms.

    Detailed description:

    • [x] Improved and parametrized many more testes
    • [x] Added CutOut data augmentation
    • [x] Added KeypointsJitter class that allows to apply random displacements to keypoints
    Source code(tar.gz)
    Source code(zip)
  • v0.1.6(May 25, 2019)

  • v0.1.5(Feb 15, 2019)

  • v0.1(Nov 13, 2018)

  • v0.0.6(Sep 21, 2018)

Owner
Research Unit of Medical Imaging, Physics and Technology
Research Unit of Medical Imaging, Physics and Technology
Clean and readable code for Decision Transformer: Reinforcement Learning via Sequence Modeling

Minimal implementation of Decision Transformer: Reinforcement Learning via Sequence Modeling in PyTorch for mujoco control tasks in OpenAI gym

Nikhil Barhate 104 Jan 06, 2023
Official PyTorch implementation of Learning Intra-Batch Connections for Deep Metric Learning (ICML 2021) published at International Conference on Machine Learning

About This repository the official PyTorch implementation of Learning Intra-Batch Connections for Deep Metric Learning. The config files contain the s

Dynamic Vision and Learning Group 41 Dec 10, 2022
This repository contain code on Novelty-Driven Binary Particle Swarm Optimisation for Truss Optimisation Problems.

This repository contain code on Novelty-Driven Binary Particle Swarm Optimisation for Truss Optimisation Problems. The main directory include the code

0 Dec 23, 2021
This is the 3D Implementation of 《Inconsistency-aware Uncertainty Estimation for Semi-supervised Medical Image Segmentation》

CoraNet This is the 3D Implementation of 《Inconsistency-aware Uncertainty Estimation for Semi-supervised Medical Image Segmentation》 Environment pytor

25 Nov 08, 2022
[NeurIPS 2021] SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning

SSUL - Official Pytorch Implementation (NeurIPS 2021) SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning Sun

Clova AI Research 44 Dec 27, 2022
Code for "Searching for Efficient Multi-Stage Vision Transformers"

Searching for Efficient Multi-Stage Vision Transformers This repository contains the official Pytorch implementation of "Searching for Efficient Multi

Yi-Lun Liao 62 Oct 25, 2022
Official Repsoitory for "Activate or Not: Learning Customized Activation." [CVPR 2021]

CVPR 2021 | Activate or Not: Learning Customized Activation. This repository contains the official Pytorch implementation of the paper Activate or Not

184 Dec 27, 2022
The implementation of our CIKM 2021 paper titled as: "Cross-Market Product Recommendation"

FOREC: A Cross-Market Recommendation System This repository provides the implementation of our CIKM 2021 paper titled as "Cross-Market Product Recomme

Hamed Bonab 16 Sep 12, 2022
This is a template for the Non-autoregressive Deep Learning-Based TTS model (in PyTorch).

Non-autoregressive Deep Learning-Based TTS Template This is a template for the Non-autoregressive TTS model. It contains Data Preprocessing Pipeline D

Keon Lee 13 Dec 05, 2022
CTRL-C: Camera calibration TRansformer with Line-Classification

CTRL-C: Camera calibration TRansformer with Line-Classification This repository contains the official code and pretrained models for CTRL-C (Camera ca

57 Nov 14, 2022
This is the official implementation for the paper "(Almost) Free Incentivized Exploration from Decentralized Learning Agents" in NeurIPS 2021.

Observe then Incentivize Experiments This is the code used for the paper "(Almost) Free Incentivized Exploration from Decentralized Learning Agents",

Cong Shen Research Group 0 Mar 08, 2022
Code for the KDD 2021 paper 'Filtration Curves for Graph Representation'

Filtration Curves for Graph Representation This repository provides the code from the KDD'21 paper Filtration Curves for Graph Representation. Depende

Machine Learning and Computational Biology Lab 16 Oct 16, 2022
Object classification with basic computer vision techniques

naive-image-classification Object classification with basic computer vision techniques. Final assignment for the computer vision course I took at univ

2 Jul 01, 2022
Code & Models for 3DETR - an End-to-end transformer model for 3D object detection

3DETR: An End-to-End Transformer Model for 3D Object Detection PyTorch implementation and models for 3DETR. 3DETR (3D DEtection TRansformer) is a simp

Facebook Research 487 Dec 31, 2022
This is the implementation of "SELF SUPERVISED REPRESENTATION LEARNING WITH DEEP CLUSTERING FOR ACOUSTIC UNIT DISCOVERY FROM RAW SPEECH" submitted to ICASSP 2022

CPC_DeepCluster This is the implementation of "SELF SUPERVISED REPRESENTATION LEARNING WITH DEEP CLUSTERING FOR ACOUSTIC UNIT DISCOVERY FROM RAW SPEEC

LEAP Lab 2 Sep 15, 2022
A little software to generate and save Julia or Mandelbrot's Fractals.

Julia-Mandelbrot-s-Fractals A little software to generate and save Julia or Mandelbrot's Fractals. Dependencies : Python 3.7 or more. (Also possible t

Olivier 0 Jul 09, 2022
Out of Distribution Detection on Natural Adversarial Examples

OOD-on-NAE Research project on out of distribution detection for the Computer Vision course by Prof. Rob Fergus (CSCI-GA 2271) Paper out on arXiv - ht

Anugya 1 Jun 08, 2022
Implementation of ICCV21 paper: PnP-DETR: Towards Efficient Visual Analysis with Transformers

Implementation of ICCV 2021 paper: PnP-DETR: Towards Efficient Visual Analysis with Transformers arxiv This repository is based on detr Recently, DETR

twang 113 Dec 27, 2022
Cross-modal Deep Face Normals with Deactivable Skip Connections

Cross-modal Deep Face Normals with Deactivable Skip Connections Victoria Fernández Abrevaya*, Adnane Boukhayma*, Philip H. S. Torr, Edmond Boyer (*Equ

72 Nov 27, 2022
What can linearized neural networks actually say about generalization?

What can linearized neural networks actually say about generalization? This is the source code to reproduce the experiments of the NeurIPS 2021 paper

gortizji 11 Dec 09, 2022