A meta plugin for processing timelapse data timepoint by timepoint in napari

Overview

napari-time-slicer

License PyPI Python Version tests codecov napari hub

A meta plugin for processing timelapse data timepoint by timepoint. It enables a list of napari plugins to process 2D+t or 3D+t data step by step when the user goes through the timelapse. Currently, these plugins are using napari-time-slicer:

napari-time-slicer enables inter-plugin communication, e.g. allowing to combine the plugins listed above in one image processing workflow for segmenting a timelapse dataset:

If you want to convert a 3D dataset into as 2D + time dataset, use the menu Tools > Utilities > Convert 3D stack to 2D timelapse (time-slicer). It will turn the 3D dataset to a 4D datset where the Z-dimension (index 1) has only 1 element, which will in napari be displayed with a time-slider. Note: It is recommended to remove the original 3D dataset after this conversion.

Usage for plugin developers

Plugins which implement the napari_experimental_provide_function hook can make use the @time_slicer. At the moment, only functions which take napari.types.ImageData, napari.types.LabelsData and basic python types such as int and float are supported. If you annotate such a function with @time_slicer it will internally convert any 4D dataset to a 3D dataset according to the timepoint currently selected in napari. Furthermore, when the napari user changes the current timepoint or the input data of the function changes, a re-computation is invoked. Thus, it is recommended to only use the time_slicer for functions which can provide [almost] real-time performance. Another constraint is that these annotated functions have to have a viewer parameter. This is necessary to read the current timepoint from the viewer when invoking the re-computions.

Example

import napari
from napari_time_slicer import time_slicer

@time_slicer
def threshold_otsu(image:napari.types.ImageData, viewer: napari.Viewer = None) -> napari.types.LabelsData:
    # ...

You can see a full implementations of this concept in the napari plugins listed above.


This napari plugin was generated with Cookiecutter using @napari's cookiecutter-napari-plugin template.

Installation

You can install napari-time-slicer via pip:

pip install napari-time-slicer

To install latest development version :

pip install git+https://github.com/haesleinhuepf/napari-time-slicer.git

Contributing

Contributions are very welcome. Tests can be run with tox, please ensure the coverage at least stays the same before you submit a pull request.

License

Distributed under the terms of the BSD-3 license, "napari-time-slicer" is free and open source software

Issues

If you encounter any problems, please file an issue along with a detailed description.

Comments
  • pyqt5 dependency

    pyqt5 dependency

    The dependency on pyqt5 which gets installed via pip can create trouble if napari has been installed via conda (see https://napari.org/plugins/best_practices.html#don-t-include-pyside2-or-pyqt5-in-your-plugin-s-dependencies). Is there any reason for this dependency? As this plugin is itself a dependency of other plugins like napari-segment-blobs-and-things-with-membranes this can create trouble down the chain.

    opened by guiwitz 7
  • PyQt5 version requirement breaks environment

    PyQt5 version requirement breaks environment

    Hi @haesleinhuepf ,

    I wanted to ask whether it is really strictly necessary to use the current PyQt5 requirement?

    pyqt5>=5.15.0
    

    It collides with current Spyder versions that only support PyQt up to 5.13:

    spyder 5.1.5 requires pyqtwebengine<5.13, which is not installed.
    spyder 5.1.5 requires pyqt5<5.13, but you have pyqt5 5.15.6 which is incompatible.
    

    Since the time slicer is used downstream in quite a few plugins of yours (e.g., segment-blobs-and-things-with-membranes, etc.) this is quite a restriction.

    opened by jo-mueller 5
  • Bug report: `KeyError: 'viewer'`

    Bug report: `KeyError: 'viewer'`

    Hi @haesleinhuepf ,

    I am getting an error in this notebook in the 5th cell on this command:

    surface = nppas.largest_label_to_surface(labels)
    

    where nppas is napari-process-points-and-surfaces. Labels is a regular label image as made with skimage.measure.label().

    Thanks for looking at it!

    opened by jo-mueller 2
  • Make dask arrays instead of computing slice for slice

    Make dask arrays instead of computing slice for slice

    Hey @haesleinhuepf! this is the first implementation of the time slicer wrapper using dask instead of computing the time slices based on the current time index. I could re-use some a little of the previous code but the wrappers start to differ from eachother pretty soon. At the moment I'm also unsure if this wrapper can replace the original time slicer function as a substitute so I kept both your old version and the dask version. An idea that I had which could be useful for saving the dask images is a function which processes each time slice and saves it as a separate image (If images are saved one by one it's really easy to load them as dask arrays!)

    opened by Cryaaa 1
  • Tests failing

    Tests failing

    source:

     if sys.platform.startswith('linux') and running_as_bundled_app():
      .tox/py37-linux/lib/python3.7/site-packages/napari/utils/misc.py:65: in running_as_bundled_app
          metadata = importlib_metadata.metadata(app_module)
      .tox/py37-linux/lib/python3.7/site-packages/importlib_metadata/__init__.py:1005: in metadata
      return Distribution.from_name(distribution_name).metadata
      .tox/py37-linux/lib/python3.7/site-packages/importlib_metadata/__init__.py:562: in from_name
      raiseValueError("A distribution name is required.")
      E   ValueError: A distribution name is required.
    

    See also:

    https://github.com/napari/napari/issues/4797

    opened by haesleinhuepf 0
  • Have 4D dask arrays as result of time-sliced functions

    Have 4D dask arrays as result of time-sliced functions

    This turns result of time-slicer annotated functions into 4D delayed dask arrays as proposed by @Cryaaa in #5

    This PR doesn't fully work yet in the interactive napari user-interface. After setting up a workflow and when going through time, it crashes sometimes with a KeyError while saving the duration of an operation. This is related to a computation finishing while the result has already be replaced. Basically multiple threads writing to the same result. It's this error: https://github.com/dask/dask/issues/896

    Reproduce:

    • Start napari
    • Open the Example dataset clEsperanto > CalibZapwfixed
    • Turn it into a 2D+t dataset using Tools > Utilities
    • Open the assistant
    • Setup a workflow, e.g. Denoise, Threshold, Label
    • Move the time-bar a couple of times until it crashes.

    I'm not sure yet how to solve this.

    opened by haesleinhuepf 8
  • Aggregate points and surfaces in 4D

    Aggregate points and surfaces in 4D

    Hi Robert @haesleinhuepf ,

    I am seeing some issues with using the timeslicer on 4D points/surface data in napari. For instance, using the label_to_surface() function from napari-process-points-and-surfaces throws an error:

    ValueError: Input volume should be a 3D numpy array.
    

    which comes from the marching_cubes function under the hood. Here is a small example script to reproduce the error:

    import napari
    import napari_process_points_and_surfaces as nppas
    # Make a blurry sphere
    s = 100
    data = np.zeros((s, s, s), dtype=float)
    x0 = 50
    radius = 15
    
    for x in range(s):
        for y in range(s):
            for z in range(s):
                if np.sqrt((x-x0)**2 + (y-x0)**2 + (z-x0)**2) < radius:
                    data[x, y, z] = 1.0
    
    viewer = make_napari_viewer()
    viewer.add_image(image)
    
    segmentation = image > filters.threshold_otsu(image)
    viewer.add_labels(segmentation)
    
    surf = nppas.label_to_surface(segmentation.astype(int))
    viewer.add_surface(surf)
    

    When introspecting the call to marching_cubes within the time_slicer function it is also evident that the image is somehow still a 4D image.

    opened by jo-mueller 4
Releases(0.4.9)
Owner
Robert Haase
Computational Microscopist, BioImage Analyst, Code Jockey
Robert Haase
Includes all files needed to satisfy hw02 requirements

HW 02 Data Sets Mean Scale Score for Asian and Hispanic Students, Grades 3 - 8 This dataset provides insights into the New York City education system

7 Oct 28, 2021
Clean and reusable data-sciency notebooks.

KPACUBO KPACUBO is a set Jupyter notebooks focused on the best practices in both software development and data science, namely, code reuse, explicit d

Matvey Morozov 1 Jan 28, 2022
A Python package for the mathematical modeling of infectious diseases via compartmental models

A Python package for the mathematical modeling of infectious diseases via compartmental models. Originally designed for epidemiologists, epispot can be adapted for almost any type of modeling scenari

epispot 12 Dec 28, 2022
An ETL Pipeline of a large data set from a fictitious music streaming service named Sparkify.

An ETL Pipeline of a large data set from a fictitious music streaming service named Sparkify. The ETL process flows from AWS's S3 into staging tables in AWS Redshift.

1 Feb 11, 2022
Option Pricing Calculator using the Binomial Pricing Method (No Libraries Required)

Binomial Option Pricing Calculator Option Pricing Calculator using the Binomial Pricing Method (No Libraries Required) Background A derivative is a fi

sammuhrai 1 Nov 29, 2021
[CVPR2022] This repository contains code for the paper "Nested Collaborative Learning for Long-Tailed Visual Recognition", published at CVPR 2022

Nested Collaborative Learning for Long-Tailed Visual Recognition This repository is the official PyTorch implementation of the paper in CVPR 2022: Nes

Jun Li 65 Dec 09, 2022
VHub - An API that permits uploading of vulnerability datasets and return of the serialized data

VHub - An API that permits uploading of vulnerability datasets and return of the serialized data

André Rodrigues 2 Feb 14, 2022
General Assembly's 2015 Data Science course in Washington, DC

DAT8 Course Repository Course materials for General Assembly's Data Science course in Washington, DC (8/18/15 - 10/29/15). Instructor: Kevin Markham (

Kevin Markham 1.6k Jan 07, 2023
Data processing with Pandas.

Processing-data-with-python This is a simple example showing how to use Pandas to create a dataframe and the processing data with python. The jupyter

1 Jan 23, 2022
Python package for analyzing behavioral data for Brain Observatory: Visual Behavior

Allen Institute Visual Behavior Analysis package This repository contains code for analyzing behavioral data from the Allen Brain Observatory: Visual

Allen Institute 16 Nov 04, 2022
Orchest is a browser based IDE for Data Science.

Orchest is a browser based IDE for Data Science. It integrates your favorite Data Science tools out of the box, so you don’t have to. The application is easy to use and can run on your laptop as well

Orchest 3.6k Jan 09, 2023
Fit models to your data in Python with Sherpa.

Table of Contents Sherpa License How To Install Sherpa Using Anaconda Using pip Building from source History Release History Sherpa Sherpa is a modeli

134 Jan 07, 2023
Data Intelligence Applications - Online Product Advertising and Pricing with Context Generation

Data Intelligence Applications - Online Product Advertising and Pricing with Context Generation Overview Consider the scenario in which advertisement

Manuel Bressan 2 Nov 18, 2021
Tkinter Izhikevich Neuron Model With Python

TKINTER IZHIKEVICH NEURON MODEL WITH PYTHON Hodgkin-Huxley Model It is a mathematical model for the generation and transmission of action potentials i

Rabia KOÇ 8 Jul 16, 2022
A Python 3 library making time series data mining tasks, utilizing matrix profile algorithms

MatrixProfile MatrixProfile is a Python 3 library, brought to you by the Matrix Profile Foundation, for mining time series data. The Matrix Profile is

Matrix Profile Foundation 302 Dec 29, 2022
Analytical view of olist e-commerce in Brazil

Analysis of E-Commerce Public Dataset by Olist The objective of this project is to propose an analytical view of olist e-commerce in Brazil. For this

Gurpreet Singh 1 Jan 11, 2022
Extract data from a wide range of Internet sources into a pandas DataFrame.

pandas-datareader Up to date remote data access for pandas, works for multiple versions of pandas. Installation Install using pip pip install pandas-d

Python for Data 2.5k Jan 09, 2023
X-news - Pipeline data use scrapy, kafka, spark streaming, spark ML and elasticsearch, Kibana

X-news - Pipeline data use scrapy, kafka, spark streaming, spark ML and elasticsearch, Kibana

Nguyễn Quang Huy 5 Sep 28, 2022
SNV calling pipeline developed explicitly to process individual or trio vcf files obtained from Illumina based pipeline (grch37/grch38).

SNV Pipeline SNV calling pipeline developed explicitly to process individual or trio vcf files obtained from Illumina based pipeline (grch37/grch38).

East Genomics 1 Nov 02, 2021
PyEmits, a python package for easy manipulation in time-series data.

PyEmits, a python package for easy manipulation in time-series data. Time-series data is very common in real life. Engineering FSI industry (Financial

Thompson 5 Sep 23, 2022