A meta plugin for processing timelapse data timepoint by timepoint in napari

Overview

napari-time-slicer

License PyPI Python Version tests codecov napari hub

A meta plugin for processing timelapse data timepoint by timepoint. It enables a list of napari plugins to process 2D+t or 3D+t data step by step when the user goes through the timelapse. Currently, these plugins are using napari-time-slicer:

napari-time-slicer enables inter-plugin communication, e.g. allowing to combine the plugins listed above in one image processing workflow for segmenting a timelapse dataset:

If you want to convert a 3D dataset into as 2D + time dataset, use the menu Tools > Utilities > Convert 3D stack to 2D timelapse (time-slicer). It will turn the 3D dataset to a 4D datset where the Z-dimension (index 1) has only 1 element, which will in napari be displayed with a time-slider. Note: It is recommended to remove the original 3D dataset after this conversion.

Usage for plugin developers

Plugins which implement the napari_experimental_provide_function hook can make use the @time_slicer. At the moment, only functions which take napari.types.ImageData, napari.types.LabelsData and basic python types such as int and float are supported. If you annotate such a function with @time_slicer it will internally convert any 4D dataset to a 3D dataset according to the timepoint currently selected in napari. Furthermore, when the napari user changes the current timepoint or the input data of the function changes, a re-computation is invoked. Thus, it is recommended to only use the time_slicer for functions which can provide [almost] real-time performance. Another constraint is that these annotated functions have to have a viewer parameter. This is necessary to read the current timepoint from the viewer when invoking the re-computions.

Example

import napari
from napari_time_slicer import time_slicer

@time_slicer
def threshold_otsu(image:napari.types.ImageData, viewer: napari.Viewer = None) -> napari.types.LabelsData:
    # ...

You can see a full implementations of this concept in the napari plugins listed above.


This napari plugin was generated with Cookiecutter using @napari's cookiecutter-napari-plugin template.

Installation

You can install napari-time-slicer via pip:

pip install napari-time-slicer

To install latest development version :

pip install git+https://github.com/haesleinhuepf/napari-time-slicer.git

Contributing

Contributions are very welcome. Tests can be run with tox, please ensure the coverage at least stays the same before you submit a pull request.

License

Distributed under the terms of the BSD-3 license, "napari-time-slicer" is free and open source software

Issues

If you encounter any problems, please file an issue along with a detailed description.

Comments
  • pyqt5 dependency

    pyqt5 dependency

    The dependency on pyqt5 which gets installed via pip can create trouble if napari has been installed via conda (see https://napari.org/plugins/best_practices.html#don-t-include-pyside2-or-pyqt5-in-your-plugin-s-dependencies). Is there any reason for this dependency? As this plugin is itself a dependency of other plugins like napari-segment-blobs-and-things-with-membranes this can create trouble down the chain.

    opened by guiwitz 7
  • PyQt5 version requirement breaks environment

    PyQt5 version requirement breaks environment

    Hi @haesleinhuepf ,

    I wanted to ask whether it is really strictly necessary to use the current PyQt5 requirement?

    pyqt5>=5.15.0
    

    It collides with current Spyder versions that only support PyQt up to 5.13:

    spyder 5.1.5 requires pyqtwebengine<5.13, which is not installed.
    spyder 5.1.5 requires pyqt5<5.13, but you have pyqt5 5.15.6 which is incompatible.
    

    Since the time slicer is used downstream in quite a few plugins of yours (e.g., segment-blobs-and-things-with-membranes, etc.) this is quite a restriction.

    opened by jo-mueller 5
  • Bug report: `KeyError: 'viewer'`

    Bug report: `KeyError: 'viewer'`

    Hi @haesleinhuepf ,

    I am getting an error in this notebook in the 5th cell on this command:

    surface = nppas.largest_label_to_surface(labels)
    

    where nppas is napari-process-points-and-surfaces. Labels is a regular label image as made with skimage.measure.label().

    Thanks for looking at it!

    opened by jo-mueller 2
  • Make dask arrays instead of computing slice for slice

    Make dask arrays instead of computing slice for slice

    Hey @haesleinhuepf! this is the first implementation of the time slicer wrapper using dask instead of computing the time slices based on the current time index. I could re-use some a little of the previous code but the wrappers start to differ from eachother pretty soon. At the moment I'm also unsure if this wrapper can replace the original time slicer function as a substitute so I kept both your old version and the dask version. An idea that I had which could be useful for saving the dask images is a function which processes each time slice and saves it as a separate image (If images are saved one by one it's really easy to load them as dask arrays!)

    opened by Cryaaa 1
  • Tests failing

    Tests failing

    source:

     if sys.platform.startswith('linux') and running_as_bundled_app():
      .tox/py37-linux/lib/python3.7/site-packages/napari/utils/misc.py:65: in running_as_bundled_app
          metadata = importlib_metadata.metadata(app_module)
      .tox/py37-linux/lib/python3.7/site-packages/importlib_metadata/__init__.py:1005: in metadata
      return Distribution.from_name(distribution_name).metadata
      .tox/py37-linux/lib/python3.7/site-packages/importlib_metadata/__init__.py:562: in from_name
      raiseValueError("A distribution name is required.")
      E   ValueError: A distribution name is required.
    

    See also:

    https://github.com/napari/napari/issues/4797

    opened by haesleinhuepf 0
  • Have 4D dask arrays as result of time-sliced functions

    Have 4D dask arrays as result of time-sliced functions

    This turns result of time-slicer annotated functions into 4D delayed dask arrays as proposed by @Cryaaa in #5

    This PR doesn't fully work yet in the interactive napari user-interface. After setting up a workflow and when going through time, it crashes sometimes with a KeyError while saving the duration of an operation. This is related to a computation finishing while the result has already be replaced. Basically multiple threads writing to the same result. It's this error: https://github.com/dask/dask/issues/896

    Reproduce:

    • Start napari
    • Open the Example dataset clEsperanto > CalibZapwfixed
    • Turn it into a 2D+t dataset using Tools > Utilities
    • Open the assistant
    • Setup a workflow, e.g. Denoise, Threshold, Label
    • Move the time-bar a couple of times until it crashes.

    I'm not sure yet how to solve this.

    opened by haesleinhuepf 8
  • Aggregate points and surfaces in 4D

    Aggregate points and surfaces in 4D

    Hi Robert @haesleinhuepf ,

    I am seeing some issues with using the timeslicer on 4D points/surface data in napari. For instance, using the label_to_surface() function from napari-process-points-and-surfaces throws an error:

    ValueError: Input volume should be a 3D numpy array.
    

    which comes from the marching_cubes function under the hood. Here is a small example script to reproduce the error:

    import napari
    import napari_process_points_and_surfaces as nppas
    # Make a blurry sphere
    s = 100
    data = np.zeros((s, s, s), dtype=float)
    x0 = 50
    radius = 15
    
    for x in range(s):
        for y in range(s):
            for z in range(s):
                if np.sqrt((x-x0)**2 + (y-x0)**2 + (z-x0)**2) < radius:
                    data[x, y, z] = 1.0
    
    viewer = make_napari_viewer()
    viewer.add_image(image)
    
    segmentation = image > filters.threshold_otsu(image)
    viewer.add_labels(segmentation)
    
    surf = nppas.label_to_surface(segmentation.astype(int))
    viewer.add_surface(surf)
    

    When introspecting the call to marching_cubes within the time_slicer function it is also evident that the image is somehow still a 4D image.

    opened by jo-mueller 4
Releases(0.4.9)
Owner
Robert Haase
Computational Microscopist, BioImage Analyst, Code Jockey
Robert Haase
Office365 (Microsoft365) audit log analysis tool

Office365 (Microsoft365) audit log analysis tool The header describes it all WHY?? The first line of code was written long time before other colleague

Anatoly 1 Jul 27, 2022
Airflow ETL With EKS EFS Sagemaker

Airflow ETL With EKS EFS & Sagemaker (en desarrollo) Diagrama de la solución Imp

1 Feb 14, 2022
SNV calling pipeline developed explicitly to process individual or trio vcf files obtained from Illumina based pipeline (grch37/grch38).

SNV Pipeline SNV calling pipeline developed explicitly to process individual or trio vcf files obtained from Illumina based pipeline (grch37/grch38).

East Genomics 1 Nov 02, 2021
Intake is a lightweight package for finding, investigating, loading and disseminating data.

Intake: A general interface for loading data Intake is a lightweight set of tools for loading and sharing data in data science projects. Intake helps

Intake 851 Jan 01, 2023
Lale is a Python library for semi-automated data science.

Lale is a Python library for semi-automated data science. Lale makes it easy to automatically select algorithms and tune hyperparameters of pipelines that are compatible with scikit-learn, in a type-

International Business Machines 293 Dec 29, 2022
Leverage Twitter API v2 to analyze tweet metrics such as impressions and profile clicks over time.

Tweetmetric Tweetmetric allows you to track various metrics on your most recent tweets, such as impressions, retweets and clicks on your profile. The

Mathis HAMMEL 29 Oct 18, 2022
Karate Club: An API Oriented Open-source Python Framework for Unsupervised Learning on Graphs (CIKM 2020)

Karate Club is an unsupervised machine learning extension library for NetworkX. Please look at the Documentation, relevant Paper, Promo Video, and Ext

Benedek Rozemberczki 1.8k Jan 09, 2023
Conduits - A Declarative Pipelining Tool For Pandas

Conduits - A Declarative Pipelining Tool For Pandas Traditional tools for declaring pipelines in Python suck. They are mostly imperative, and can some

Kale Miller 7 Nov 21, 2021
Statsmodels: statistical modeling and econometrics in Python

About statsmodels statsmodels is a Python package that provides a complement to scipy for statistical computations including descriptive statistics an

statsmodels 8k Dec 29, 2022
Powerful, efficient particle trajectory analysis in scientific Python.

freud Overview The freud Python library provides a simple, flexible, powerful set of tools for analyzing trajectories obtained from molecular dynamics

Glotzer Group 195 Dec 20, 2022
OpenDrift is a software for modeling the trajectories and fate of objects or substances drifting in the ocean, or even in the atmosphere.

opendrift OpenDrift is a software for modeling the trajectories and fate of objects or substances drifting in the ocean, or even in the atmosphere. Do

OpenDrift 167 Dec 13, 2022
A set of functions and analysis classes for solvation structure analysis

SolvationAnalysis The macroscopic behavior of a liquid is determined by its microscopic structure. For ionic systems, like batteries and many enzymes,

MDAnalysis 19 Nov 24, 2022
MS in Data Science capstone project. Studying attacks on autonomous vehicles.

Surveying Attack Models for CAVs Guide to Installing CARLA and Collecting Data Our project focuses on surveying attack models for Connveced Autonomous

Isabela Caetano 1 Dec 09, 2021
Parses data out of your Google Takeout (History, Activity, Youtube, Locations, etc...)

google_takeout_parser parses both the Historical HTML and new JSON format for Google Takeouts caches individual takeout results behind cachew merge mu

Sean Breckenridge 27 Dec 28, 2022
A lightweight, hub-and-spoke dashboard for multi-account Data Science projects

A lightweight, hub-and-spoke dashboard for cross-account Data Science Projects Introduction Modern Data Science environments often involve many indepe

AWS Samples 3 Oct 30, 2021
A Pythonic introduction to methods for scaling your data science and machine learning work to larger datasets and larger models, using the tools and APIs you know and love from the PyData stack (such as numpy, pandas, and scikit-learn).

This tutorial's purpose is to introduce Pythonistas to methods for scaling their data science and machine learning work to larger datasets and larger models, using the tools and APIs they know and lo

Coiled 102 Nov 10, 2022
Dbt-core - dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

Dbt-core - dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

dbt Labs 6.3k Jan 08, 2023
PrimaryBid - Transform application Lifecycle Data and Design and ETL pipeline architecture for ingesting data from multiple sources to redshift

Transform application Lifecycle Data and Design and ETL pipeline architecture for ingesting data from multiple sources to redshift This project is composed of two parts: Part1 and Part2

Emmanuel Boateng Sifah 1 Jan 19, 2022
Produces a summary CSV report of an Amber Electric customer's energy consumption and cost data.

Amber Electric Usage Summary This is a command line tool that produces a summary CSV report of an Amber Electric customer's energy consumption and cos

Graham Lea 12 May 26, 2022
ASOUL直播间弹幕抓取&&数据分析

ASOUL直播间弹幕抓取&&数据分析(更新中) 这些文件用于爬取ASOUL直播间的弹幕(其他直播间也可以)和其他信息,以及简单的数据分析生成。

159 Dec 10, 2022