scAR (single-cell Ambient Remover) is a package for data denoising in single-cell omics.

Overview

scAR

scAR single-cell omics machine learning variational autoencoders denoising

scAR (single cell Ambient Remover) is a package for denoising multiple single cell omics data. It can be used for multiple tasks, such as, sgRNA assignment for scCRISPRseq, identity barcode assignment for cell indexing, protein denoising for CITE-seq, mRNA denoising for scRNAseq, and etc... It is built using probabilistic deep learning, illustrated as follows:

Table of Contents

Installation

Clone this repository,

$ git clone https://github.com/Novartis/scAR.git

Enter the cloned directory:

$ cd scAR

To install the dependencies, create a conda environment:

Please use scAR-gpu if you have an nvidia graphis card and the corresponging driver installed.

$ conda env create -f scAR-gpu.yml

or

Please use scAR-cpu if you don't have a graphis card availalble.

$ conda env create -f scAR-cpu.yml

To activate the scAR conda environment run:

$ conda activate scAR

Usage

There are two ways to run scAR.

  1. Use scAR API if you are Python users
>>> from scAR import model
>>> scarObj = model(adata.X.to_df(), empty_profile)
>>> scarObj.train()
>>> scarObj.inference()
>>> adata.layers["X_scAR_denoised"] = scarObj.native_counts
>>> adata.obsm["X_scAR_assignment"] = scarObj.feature_assignment  # feature assignment, e.g., sgRNAs, tags, and etc.. Only available in 'cropseq' mode

See the tutorials

  1. Run scAR from the command line
$ scar raw_count_matrix.pickle -t technology -e empty_profile.pickle -o output

raw_count_matrix.pickle, a pickle-formatted raw count matrix (MxN) with cells in rows and features in columns
empty_profile.pickle, a pickle-formatted feature frequencies (Nx1) in empty droplets
technology, a string, either 'scRNAseq' or 'CROPseq' or 'CITEseq'

Use scar --help command to see other optional arguments and parameters.

The output folder contains four (or five) files:

output
├── denoised_counts.pickle		# denoised count matrix
├── expected_noise_ratio.pickle	# estimated noise ratio
├── BayesFactor.pickle			# bayesian factor of ambient contamination
├── expected_native_freq.pickle	# estimated native frequencies
└── assignment.pickle			# feature assignment, e.g., sgRNAs, tags, and etc.. Gernerated under 'cropseq' mode

Dependencies

PyTorch 1.8 Python 3.8.6 torchvision 0.9.0 tqdm 4.62.3 scikit-learn 1.0.1

Resources

License

This project is licensed under the terms of License.
Copyright 2022 Novartis International AG.

Reference

If you use scAR in your research, please consider citing our manuscript,

@article {Sheng2022.01.14.476312,
	author = {Sheng, Caibin and Lopes, Rui and Li, Gang and Schuierer, Sven and Waldt, Annick and Cuttat, Rachel and Dimitrieva, Slavica and Kauffmann, Audrey and Durand, Eric and Galli, Giorgio G and Roma, Guglielmo and de Weck, Antoine},
	title = {Probabilistic modeling of ambient noise in single-cell omics data},
	elocation-id = {2022.01.14.476312},
	year = {2022},
	doi = {10.1101/2022.01.14.476312},
	publisher = {Cold Spring Harbor Laboratory},
	URL = {https://www.biorxiv.org/content/early/2022/01/14/2022.01.14.476312},
	eprint = {https://www.biorxiv.org/content/early/2022/01/14/2022.01.14.476312.full.pdf},
	journal = {bioRxiv}
}
Comments
  • Stochastic rounding to integers for downstream use in TotalVI/SCVI

    Stochastic rounding to integers for downstream use in TotalVI/SCVI

    Hi Caibin,

    I tried using scar's output as input for TotalVI/SCVI. As expected, those gave an error because the input is not integer anymore. I would suggest implementing stochastic rounding to integers as done in SoupX.

    Let me know if you're interested and I can find the time to implement it.

    Regards, Mikhael

    enhancement 
    opened by mdmanurung 9
  • BiocondaBot not triggered

    BiocondaBot not triggered

    Hi @fgypas , I made a new release v0.4.1 but bioconda somehow is not triggered upon the new release.

    In the new release, some codes related to building process have been refactored.

    • All information in setup.py (deleted) is integrated into setup.cfg.
    • An extra pyproject.toml file is added.

    I am wondering whether these affect the bioconda-recipes.

    Many thanks, Caibin

    opened by CaibinSh 7
  • New release

    New release

    Hi @fgypas ,

    I am making a new release. There are mainly three changes: 1) addition of a readthedocs; 2) code reformatting via black and pylint (pylint now can score >7, so I have increase the standard in the Action test from 0.5 to 6); 3) renaming 'scAR' to 'scar'.

    I have a couple of questions regarding whether these changes influence the bioconda recipe.

    • Will renaming package name (scAR) require modification in bioconda PR? All uppercase ('scAR') is changed to lowercase ('scar') in everywhere possible (inc. folder, environment, and etc.) But the repo name may stay as 'scAR' for a while, as renaming repo name requires permission from Nick.

    • Should we exclude the folder of datasets in the conda recipe? In addition, a folder, named 'datasets' contains >100 MBs data is added for the tutorial. Should we exclude it?

    question 
    opened by CaibinSh 3
  • Implementation in scvi-tools

    Implementation in scvi-tools

    Hi scAR team,

    I'm reaching out to gauge interest in having a mirror implementation in scvi-tools for scAR. Given the existing infrastructure in the scvi-tools repository, I was able to create a port of scAR quite easily as an external module. Of course, the implementation will link to this repository as the original and cites the paper in the docs. On top of that, the port would allow users of scvi-tools to use the pretrained scAR encoder for doublet detection using the solo model.

    Here's the pending pull request so you can check out what it would look like in the final implementation: https://github.com/scverse/scvi-tools/pull/1683

    Please let me know what you think!

    opened by ricomnl 2
  • Positive-valued denoising results for ADTs with raw 0 counts

    Positive-valued denoising results for ADTs with raw 0 counts

    Hi scar team!

    Thank you for developing this interesting package. I had a question about the resulting denoised values for CITE-seq experiments.

    I've noticed that some cells that originally have a 0 value for an ADT (as a raw count) will have a positive value (>0) for that ADT after the denoising procedure. Below, I show this case for the CD25 ADT in the 10xPBMC5k CITE-seq dataset (from the tutorial at https://scar-tutorials.readthedocs.io/en/latest/tutorials/scAR_tutorial_denoising_CITEseq.html).

    I'm a bit confused about how to best interpret these values and how they are occurring. Should these be set to 0 after the denoising procedure?

    Screen Shot 2022-05-25 at 1 16 37 AM question 
    opened by diegoalexespi 2
  • Sparsity values for mRNA decontamination?

    Sparsity values for mRNA decontamination?

    Hello,

    I was wondering what the recommendations for the sparsity value would be in denoising mRNA? Specifically if we don't know too much of the data besides UMI/nGenes in the cells etc.? I noticed its generally set at 1 for sgRNA decontamination, but what would the general recommended value be for mRNA?

    Thanks, Chang

    question 
    opened by cnk113 1
  • Number of training epochs + batch size

    Number of training epochs + batch size

    Dear scAR-Team,

    thank you for developing this package. I am currently exploring it and I would like to ask you

    1. how do you determine the number of epochs the user should use for feature_type = "mRNA"? In your tutorials you used 400 epochs and in your paper you mentioned that you fixed the epochs to 800. I applied it for various batch sizes (up to 1000) and noticed that the model is sensitive to it.

    2. I noticed that you use rather small batch-size - is scAR sensitive to the batch-size, it is just due to computational limitations or due to better perfromance?

    Thank you in advance!

    Best,

    question 
    opened by KalinNonchev 1
  • bump to version 0.3.2

    bump to version 0.3.2

    fix(*): changelog docs: adding docstring in documentation docs: adding Release notes in documentation docs: adding docstring in documentation test: adding semantic release refactor: further refactoring codes fix semantic release

    opened by CaibinSh 1
  • ask for permission of Webhooks

    ask for permission of Webhooks

    Hi @kliatsko ,

    We are currently refactoring and adding functionalities to scAR.

    Could you please grant the Webhooks permission for us to automate the documentation?

    Many thanks in advance. Best regards, Caibin on behalf of the scar team @fgypas @Tobias-Ternent @mr-nvs @AlexMTYZ.

    help wanted 
    opened by CaibinSh 1
  • New release

    New release

    • Additions of readthedocs
    • Code refactoring
    1. Renaming module names, e.g. changing "scAR" -> "scar"
    2. Renaming parameter names, e.g.

    changing "scRNAseq_tech" -> "feature_type" changing "model" -> "count_model" changing "scRNAseq_tech" -> "feature_type"

    • Black and Pylint re-formatting the code
    enhancement 
    opened by CaibinSh 1
  • Black github action

    Black github action

    Addition of black github action that runs on every push and every pull request. It shows in the stdout all the changes that need to be made (--diff), but returns exit code 0, even if errors are observed.

    opened by fgypas 1
Releases(v0.4.4)
  • v0.4.4(Aug 9, 2022)

    Documentation

    • Update dependency (03cf19e)
    • Update dependencies (9bd7f1c)
    • Update documentations (418996c)
    • Update dependencies (1bde351)
    • main: Add link to anndata and scanpy (8436e05)
    • main: Update dependencies (984df35)
    • main: Update documentation for .h5 file (2a309e0)
    • Add a link of binary installers (2faed3e)
    • Update documentations (e26a6e9)
    • Add competing methods (8564b2b)
    • scar: Add versionadded directives for parameter sparsity and round_to_int (33e35ca)
    • Update docs (a4da539)
    • Update introduction (a036b24)
    • Change readthedocs template (421e52f)
    • data_generator: Update docs (1f8f668)
    • data_generator: Re-style docs (afef9fb)
    • *: Re-style docs (2d550fa)

    Performance

    • main: Command line tool supports a new input: filtered_feature_bc_matrix.h5 (73bc13e)
    • setup: Add an error raise statement (f4fb1a8)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.3(Jun 15, 2022)

    Fix

    • setup: Fix a bug to allow sample reasonable numbers of droplets (ef6f7e4)
    • main: Fix a bug in main to set default NN number (794ff17)

    Documentation

    • main: Add scanpy as dependency (252a492)

    Performance

    • main: Set a separate batchsize_infer parameter for inference (8727f04)
    • setup: Add an option of random sampling droplets to speed up calculation (ce042dd)
    • setup: Enable manupulate large-scale emptydroplets (15f1840)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.2(Jun 7, 2022)

  • v0.4.1(May 19, 2022)

    What's Changed

    Feature

    • inference: add a round_to_int parameter to round the counts (float) for easy interpretation and better integration into other methods (#47) (902a2b9) (8694239)

    Build

    • setup: replace setup.py with setup.cfg and pyproject.toml (#51) (3dc999a)

    Chore

    Documentation

    • readthedocs: add scAR_logo image (#51) (c34f362)
    • tutorials: add ci=None to speed up plotting (#51) (902a2b9)

    Contributor

    @CaibinSh and @mdmanurung

    Full Changelog: https://github.com/Novartis/scar/compare/v0.4.0...v0.4.1

    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(May 5, 2022)

  • v0.3.5(May 3, 2022)

  • v0.3.4(May 1, 2022)

  • v0.3.3(May 1, 2022)

  • v0.3.1(Apr 29, 2022)

  • v0.3.0(Apr 27, 2022)

    What's Changed

    Renaming module names, e.g. changing "scAR" -> "scar" Renaming parameter names, e.g.

    "scRNAseq_tech" -> "feature_type" "model" -> "count_model" "empty_profile" -> "ambient_profile" ...

    • Black and Pylint re-formatting the code
    • New release by @CaibinSh in https://github.com/Novartis/scAR/pull/26

    Contributor

    @CaibinSh @fgypas @mr-nvs @Tobias-Ternent

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.3...v0.3.0

    Source code(tar.gz)
    Source code(zip)
  • v0.2.3(Apr 20, 2022)

    • Add integration test
    • Black formating
    • Bump version to 0.2.3

    Contributors: @fgypas , @mr-nvs and @CaibinSh

    What's Changed

    • Develop by @CaibinSh in https://github.com/Novartis/scAR/pull/19

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.2...v0.2.3

    Source code(tar.gz)
    Source code(zip)
  • v0.2.2(Apr 4, 2022)

    v0.2.2

    • Remove torchaudio
    • Add test data for integration tests
    • Bump version to 0.2.2

    Contributors: @CaibinSh @fgypas

    What's Changed

    • Remove torchaudio, add test data and bump version to 0.2.2 by @fgypas in https://github.com/Novartis/scAR/pull/15

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.1-beta...v0.2.2

    Source code(tar.gz)
    Source code(zip)
  • v0.2.1-beta(Apr 1, 2022)

    • fix a typo in scAR-gpu.yml
    • reorganise init.py files

    Contributor: @CaibinSh

    What's Changed

    • Develop by @CaibinSh in https://github.com/Novartis/scAR/pull/12

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.0-beta...v0.2.1-beta

    Source code(tar.gz)
    Source code(zip)
  • v0.2.0-beta(Apr 1, 2022)

    • Support for training of the model with CPUs
    • Addition of two yaml files for CPU/GPU installation
    • Refactor of setup.py and structure of the package
    • Addition of tests with pytest
    • Addition of lint checks
    • Automate build with github actions (install package and run lint checks and pytest)
    • Update documentation
    • Version 0.2.0

    Co-authored-by: @CaibinSh @mr-nvs @Tobias-Ternent @fgypas

    What's Changed

    • 0.2.0-release by @fgypas in https://github.com/Novartis/scAR/pull/11

    Full Changelog: https://github.com/Novartis/scAR/commits/v0.2.0-beta

    Source code(tar.gz)
    Source code(zip)
A simple rest api serving a deep learning model that classifies human gender based on their faces. (vgg16 transfare learning)

this is a simple rest api serving a deep learning model that classifies human gender based on their faces. (vgg16 transfare learning)

crispengari 5 Dec 09, 2021
Python implementation of NARS (Non-Axiomatic-Reasoning-System)

Python implementation of NARS (Non-Axiomatic-Reasoning-System)

Bowen XU 11 Dec 20, 2022
Implementation of CaiT models in TensorFlow and ImageNet-1k checkpoints. Includes code for inference and fine-tuning.

CaiT-TF (Going deeper with Image Transformers) This repository provides TensorFlow / Keras implementations of different CaiT [1] variants from Touvron

Sayak Paul 9 Jun 26, 2022
An educational AI robot based on NVIDIA Jetson Nano.

JetBot Looking for a quick way to get started with JetBot? Many third party kits are now available! JetBot is an open-source robot based on NVIDIA Jet

NVIDIA AI IOT 2.6k Dec 29, 2022
PyDEns is a framework for solving Ordinary and Partial Differential Equations (ODEs & PDEs) using neural networks

PyDEns PyDEns is a framework for solving Ordinary and Partial Differential Equations (ODEs & PDEs) using neural networks. With PyDEns one can solve PD

Data Analysis Center 220 Dec 26, 2022
This is a model made out of Neural Network specifically a Convolutional Neural Network model

This is a model made out of Neural Network specifically a Convolutional Neural Network model. This was done with a pre-built dataset from the tensorflow and keras packages. There are other alternativ

9 Oct 18, 2022
Perspective: Julia for Biologists

Perspective: Julia for Biologists 1. Examples Speed: Example 1 - Single cell data and network inference Domain: Single cell data Methodology: Network

Elisabeth Roesch 55 Dec 02, 2022
LSTMs (Long Short Term Memory) RNN for prediction of price trends

Price Prediction with Recurrent Neural Networks LSTMs BTC-USD price prediction with deep learning algorithm. Artificial Neural Networks specifically L

5 Nov 12, 2021
An implementation of a sequence to sequence neural network using an encoder-decoder

Keras implementation of a sequence to sequence model for time series prediction using an encoder-decoder architecture. I created this post to share a

Luke Tonin 195 Dec 17, 2022
Character Controllers using Motion VAEs

Character Controllers using Motion VAEs This repo is the codebase for the SIGGRAPH 2020 paper with the title above. Please find the paper and demo at

Electronic Arts 165 Jan 03, 2023
Neighborhood Contrastive Learning for Novel Class Discovery

Neighborhood Contrastive Learning for Novel Class Discovery This repository contains the official implementation of our paper: Neighborhood Contrastiv

Zhun Zhong 56 Dec 09, 2022
Facebook AI Image Similarity Challenge: Descriptor Track

Facebook AI Image Similarity Challenge: Descriptor Track This repository contains the code for our solution to the Facebook AI Image Similarity Challe

Sergio MP 17 Dec 14, 2022
Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. It can use GPUs and perform efficient symbolic differentiation.

============================================================================================================ `MILA will stop developing Theano https:

9.6k Jan 06, 2023
PyTorch implementation of SampleRNN: An Unconditional End-to-End Neural Audio Generation Model

samplernn-pytorch A PyTorch implementation of SampleRNN: An Unconditional End-to-End Neural Audio Generation Model. It's based on the reference implem

DeepSound 261 Dec 14, 2022
PyTorch implementation of D2C: Diffuison-Decoding Models for Few-shot Conditional Generation.

D2C: Diffuison-Decoding Models for Few-shot Conditional Generation Project | Paper PyTorch implementation of D2C: Diffuison-Decoding Models for Few-sh

Jiaming Song 90 Dec 27, 2022
Zero-shot Learning by Generating Task-specific Adapters

Code for "Zero-shot Learning by Generating Task-specific Adapters" This is the repository containing code for "Zero-shot Learning by Generating Task-s

INK Lab @ USC 11 Dec 17, 2021
Action Recognition for Self-Driving Cars

Action Recognition for Self-Driving Cars This repo contains the codes for the 2021 Fall semester project "Action Recognition for Self-Driving Cars" at

VITA lab at EPFL 3 Apr 07, 2022
PyTorch Implementation for Deep Metric Learning Pipelines

Easily Extendable Basic Deep Metric Learning Pipeline Karsten Roth ([email 

Karsten Roth 543 Jan 04, 2023
Let Python optimize the best stop loss and take profits for your TradingView strategy.

TradingView Machine Learning TradeView is a free and open source Trading View bot written in Python. It is designed to support all major exchanges. It

Robert Roman 473 Jan 09, 2023
Realtime Face Anti Spoofing with Face Detector based on Deep Learning using Tensorflow/Keras and OpenCV

Realtime Face Anti-Spoofing Detection 🤖 Realtime Face Anti Spoofing Detection with Face Detector to detect real and fake faces Please star this repo

Prem Kumar 86 Aug 03, 2022