🌾 PASTIS 🌾 Panoptic Agricultural Satellite TIme Series

Overview

🌾 PASTIS 🌾 Panoptic Agricultural Satellite TIme Series (optical and radar)

The PASTIS Dataset

  • Dataset presentation

PASTIS is a benchmark dataset for panoptic and semantic segmentation of agricultural parcels from satellite time series. It contains 2,433 patches within the French metropolitan territory with panoptic annotations (instance index + semantic labelfor each pixel). Each patch is a Sentinel-2 multispectral image time series of variable lentgh.

We propose an official 5 fold split provided in the dataset's metadata, and evaluated several of the top-performing image time series networks. You are welcome to use our numbers and to submit your own entries to the leaderboard!

  • Dataset in numbers
▶️ 2,433 time series ▶️ 124,422 individual parcels ▶️ 18 crop types
▶️ 128x128 pixels / images ▶️ 38-61 acquisitions / series ▶️ 10m / pixel
▶️ 10 spectral bands ▶️ covers ~4,000 km² ▶️ over 2B pixels
  • 🔥 NEW: Radar extension (PASTIS-R)

We also propose an extended version of PASTIS which contains all radar observations of Sentinel-1 for all 2433 patches in addition to the Sentinel-2 images. For each patch, approximately 70 observations of Sentinel-1 in ascending orbit, and 70 observations in descending orbit are added to the dataset. The PASTIS-R extension can thus be used to evaluate optical-radar fusion methods for parcel-based classification, semantic segmentation, and panoptic segmentation.
For more details on PASTIS-R, refer to our recent paper on multi-modal fusion with attention-based models (link coming soon).

Usage

  • Download

The dataset can be downloaded from zenodo in different formats:

  1. PASTIS (29 GB zipped) : The original PASTIS dataset for semantic and panoptic segmentation on Sentinel-2 time series (format used for the ICCV 2021 paper). DOI
  2. PASTIS-R (54 GB zipped) : The extended version with Sentinel-1 observations. DOI
  3. PASTIS-R (pixel-set format) (27 GB zipped) : The PASTIS-R dataset prepared in pixel-set format for parcel-based classification only. See this repo and paper for more details on this format. DOI
  • Data loading

This repository also contains a PyTorch dataset class in code/dataloader.py that can be readily used to load data for training models on PASTIS and PASTIS-R. For the pixel-set dataset, use the dataloader in code/dataloader_pixelset.py. The time series contained in PASTIS have variable lengths. The code/collate.py contains a pad_collate function that you can use in the pytorch dataloader to temporally pad shorter sequences. The demo.ipynb notebook shows how to use these classes and methods to load data from PASTIS.

  • Metrics

A PyTorch implementation is also given in code/panoptic_metrics.py to compute the panoptic metrics. In order to use these metrics, the model's output should contain an instance prediction and a semantic prediction. The first one allocates an instance id to each pixel of the image, and the latter a semantic label.

Leaderboard

Please open an issue to submit new entries. Do mention if the work has been published and wether the code accessible for reproducibility. We require that at least a preprint is available to present the method used.

Semantic Segmentation

Optical only (PASTIS)

Model name #Params OA mIoU Published
U-TAE 1.1M 83.2% 63.1% ✔️ link
Unet-3d* 1.6M 81.3% 58.4% ✔️ link
Unet-ConvLSTM* 1.5M 82.1% 57.8% ✔️ link
FPN-ConvLSTM* 1.3M 81.6% 57.1% ✔️ link
Models that we re-implemented ourselves are denoted with a star (*).

Optical+Radar fusion (PASTIS-R)

Model name #Params OA mIoU Published
Late Fusion (U-TAE) + Aux + TempDrop 1.7M 84.2% 66.3% ✔️ link
Early Fusion (U-TAE) + TempDrop 1.6M 83.8% 65.9% ✔️ link

Panoptic Segmentation

Optical only (PASTIS)

Model name #Params SQ RQ PQ Published
U-TAE + PaPs 1.3M 81.3 49.2 40.4 ✔️ link

Optical+Radar fusion (PASTIS-R)

Model name #Params SQ RQ PQ Published
Early Fusion (U-TAE + PaPs) + Aux + TempDrop 1.8M 82.2 50.6 42.0 ✔️ link
Late Fusion (U-TAE + PaPs) + TempDrop 2.4M 81.6 50.5 41.6 ✔️ link

Documentation

The agricultural parcels are grouped into 18 different crop classes as shown in the table below. The backgroud class corresponds to non-agricultural land, and the void label for parcels that are mostly outside their patch. drawing

Additional information about the dataset can be found in the documentation/pastis-documentation.pdf document.

References

If you use PASTIS please cite the related paper:

@article{garnot2021panoptic,
  title={Panoptic Segmentation of Satellite Image Time Series
with Convolutional Temporal Attention Networks},
  author={Sainte Fare Garnot, Vivien  and Landrieu, Loic },
  journal={ICCV},
  year={2021}
}

For the PASTIS-R optical-radar fusion dataset, please also cite this paper:

@article{garnot2021mmfusion,
  title={Multi-Modal Temporal Attention Models for Crop Mapping from Satellite Time Series},
  author={Sainte Fare Garnot, Vivien  and Landrieu, Loic and Chehata, Nesrine },
  journal={arxiv},
  year={2021}
}

Credits

  • The satellite imagery used in PASTIS was retrieved from THEIA: "Value-added data processed by the CNES for the Theia www.theia.land.fr data cluster using Copernicus data. The treatments use algorithms developed by Theia’s Scientific Expertise Centres. "

  • The annotations used in PASTIS stem from the French land parcel identification system produced by IGN, the French mapping agency.

  • This work was partly supported by ASP, the French Payment Agency.

  • We also thank Zenodo for hosting the datasets.

DimReductionClustering - Dimensionality Reduction + Clustering + Unsupervised Score Metrics

Dimensionality Reduction + Clustering + Unsupervised Score Metrics Introduction

11 Nov 15, 2022
Code for the paper: Sketch Your Own GAN

Sketch Your Own GAN Project | Paper | Youtube | Slides Our method takes in one or a few hand-drawn sketches and customizes an off-the-shelf GAN to mat

677 Dec 28, 2022
Official repository for CVPR21 paper "Deep Stable Learning for Out-Of-Distribution Generalization".

StableNet StableNet is a deep stable learning method for out-of-distribution generalization. This is the official repo for CVPR21 paper "Deep Stable L

120 Dec 28, 2022
Official release of MSHT: Multi-stage Hybrid Transformer for the ROSE Image Analysis of Pancreatic Cancer axriv: http://arxiv.org/abs/2112.13513

MSHT: Multi-stage Hybrid Transformer for the ROSE Image Analysis This is the official page of the MSHT with its experimental script and records. We de

Tianyi Zhang 53 Dec 27, 2022
A PyTorch Lightning solution to training OpenAI's CLIP from scratch.

train-CLIP 📎 A PyTorch Lightning solution to training CLIP from scratch. Goal ⚽ Our aim is to create an easy to use Lightning implementation of OpenA

Cade Gordon 396 Dec 30, 2022
TResNet: High Performance GPU-Dedicated Architecture

TResNet: High Performance GPU-Dedicated Architecture paperV2 | pretrained models Official PyTorch Implementation Tal Ridnik, Hussam Lawen, Asaf Noy, I

426 Dec 28, 2022
BMN: Boundary-Matching Network

BMN: Boundary-Matching Network A pytorch-version implementation codes of paper: "BMN: Boundary-Matching Network for Temporal Action Proposal Generatio

qinxin 260 Dec 06, 2022
Technical Indicators implemented in Python only using Numpy-Pandas as Magic - Very Very Fast! Very tiny! Stock Market Financial Technical Analysis Python library . Quant Trading automation or cryptocoin exchange

MyTT Technical Indicators implemented in Python only using Numpy-Pandas as Magic - Very Very Fast! to Stock Market Financial Technical Analysis Python

dev 34 Dec 27, 2022
Weighted QMIX: Expanding Monotonic Value Function Factorisation

This repo contains the cleaned-up code that was used in "Weighted QMIX: Expanding Monotonic Value Function Factorisation"

whirl 82 Dec 29, 2022
Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networks

Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networks arXiv preprint: https://arxiv.org/abs/2201.02143. Architec

19 Nov 30, 2022
Implementation of the paper titled "Using Sampling to Estimate and Improve Performance of Automated Scoring Systems with Guarantees"

Using Sampling to Estimate and Improve Performance of Automated Scoring Systems with Guarantees Implementation of the paper titled "Using Sampling to

MIDAS, IIIT Delhi 2 Aug 29, 2022
Object Detection using YOLO from PyImageSearch

Object Detection using YOLO from PyImageSearch By applying object detection, you’ll not only be able to determine what is in an image, but also where

Mohamed NIANG 1 Feb 09, 2022
PyTorch Implementation for Fracture Detection in Wrist Bone X-ray Images

wrist-d PyTorch Implementation for Fracture Detection in Wrist Bone X-ray Images note: Paper: Under Review at MPDI Diagnostics Submission Date: Novemb

Fatih UYSAL 5 Oct 12, 2022
Constructing interpretable quadratic accuracy predictors to serve as an objective function for an IQCQP problem that represents NAS under latency constraints and solve it with efficient algorithms.

IQNAS: Interpretable Integer Quadratic programming Neural Architecture Search Realistic use of neural networks often requires adhering to multiple con

0 Oct 24, 2021
This repository contains the code to replicate the analysis from the paper "Moving On - Investigating Inventors' Ethnic Origins Using Supervised Learning"

Replication Code for 'Moving On' - Investigating Inventors' Ethnic Origins Using Supervised Learning This repository contains the code to replicate th

Matthias Niggli 0 Jan 04, 2022
Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF From a Single Image

Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF From a Single Image (Project page) Zhengqin Li, Mohammad Sha

209 Jan 05, 2023
Code and real data for the paper "Counterfactual Temporal Point Processes", available at arXiv.

counterfactual-tpp This is a repository containing code and real data for the paper Counterfactual Temporal Point Processes. Pre-requisites This code

Networks Learning 11 Dec 09, 2022
Latex code for making neural networks diagrams

PlotNeuralNet Latex code for drawing neural networks for reports and presentation. Have a look into examples to see how they are made. Additionally, l

Haris Iqbal 18.6k Jan 01, 2023
Vector AI — A platform for building vector based applications. Encode, query and analyse data using vectors.

Vector AI is a framework designed to make the process of building production grade vector based applications as quickly and easily as possible. Create

Vector AI 267 Dec 23, 2022
[NeurIPS 2020] Blind Video Temporal Consistency via Deep Video Prior

pytorch-deep-video-prior (DVP) Official PyTorch implementation for NeurIPS 2020 paper: Blind Video Temporal Consistency via Deep Video Prior TensorFlo

Yazhou XING 90 Oct 19, 2022