🌾 PASTIS 🌾 Panoptic Agricultural Satellite TIme Series

Overview

🌾 PASTIS 🌾 Panoptic Agricultural Satellite TIme Series (optical and radar)

The PASTIS Dataset

  • Dataset presentation

PASTIS is a benchmark dataset for panoptic and semantic segmentation of agricultural parcels from satellite time series. It contains 2,433 patches within the French metropolitan territory with panoptic annotations (instance index + semantic labelfor each pixel). Each patch is a Sentinel-2 multispectral image time series of variable lentgh.

We propose an official 5 fold split provided in the dataset's metadata, and evaluated several of the top-performing image time series networks. You are welcome to use our numbers and to submit your own entries to the leaderboard!

  • Dataset in numbers
▶️ 2,433 time series ▶️ 124,422 individual parcels ▶️ 18 crop types
▶️ 128x128 pixels / images ▶️ 38-61 acquisitions / series ▶️ 10m / pixel
▶️ 10 spectral bands ▶️ covers ~4,000 km² ▶️ over 2B pixels
  • 🔥 NEW: Radar extension (PASTIS-R)

We also propose an extended version of PASTIS which contains all radar observations of Sentinel-1 for all 2433 patches in addition to the Sentinel-2 images. For each patch, approximately 70 observations of Sentinel-1 in ascending orbit, and 70 observations in descending orbit are added to the dataset. The PASTIS-R extension can thus be used to evaluate optical-radar fusion methods for parcel-based classification, semantic segmentation, and panoptic segmentation.
For more details on PASTIS-R, refer to our recent paper on multi-modal fusion with attention-based models (link coming soon).

Usage

  • Download

The dataset can be downloaded from zenodo in different formats:

  1. PASTIS (29 GB zipped) : The original PASTIS dataset for semantic and panoptic segmentation on Sentinel-2 time series (format used for the ICCV 2021 paper). DOI
  2. PASTIS-R (54 GB zipped) : The extended version with Sentinel-1 observations. DOI
  3. PASTIS-R (pixel-set format) (27 GB zipped) : The PASTIS-R dataset prepared in pixel-set format for parcel-based classification only. See this repo and paper for more details on this format. DOI
  • Data loading

This repository also contains a PyTorch dataset class in code/dataloader.py that can be readily used to load data for training models on PASTIS and PASTIS-R. For the pixel-set dataset, use the dataloader in code/dataloader_pixelset.py. The time series contained in PASTIS have variable lengths. The code/collate.py contains a pad_collate function that you can use in the pytorch dataloader to temporally pad shorter sequences. The demo.ipynb notebook shows how to use these classes and methods to load data from PASTIS.

  • Metrics

A PyTorch implementation is also given in code/panoptic_metrics.py to compute the panoptic metrics. In order to use these metrics, the model's output should contain an instance prediction and a semantic prediction. The first one allocates an instance id to each pixel of the image, and the latter a semantic label.

Leaderboard

Please open an issue to submit new entries. Do mention if the work has been published and wether the code accessible for reproducibility. We require that at least a preprint is available to present the method used.

Semantic Segmentation

Optical only (PASTIS)

Model name #Params OA mIoU Published
U-TAE 1.1M 83.2% 63.1% ✔️ link
Unet-3d* 1.6M 81.3% 58.4% ✔️ link
Unet-ConvLSTM* 1.5M 82.1% 57.8% ✔️ link
FPN-ConvLSTM* 1.3M 81.6% 57.1% ✔️ link
Models that we re-implemented ourselves are denoted with a star (*).

Optical+Radar fusion (PASTIS-R)

Model name #Params OA mIoU Published
Late Fusion (U-TAE) + Aux + TempDrop 1.7M 84.2% 66.3% ✔️ link
Early Fusion (U-TAE) + TempDrop 1.6M 83.8% 65.9% ✔️ link

Panoptic Segmentation

Optical only (PASTIS)

Model name #Params SQ RQ PQ Published
U-TAE + PaPs 1.3M 81.3 49.2 40.4 ✔️ link

Optical+Radar fusion (PASTIS-R)

Model name #Params SQ RQ PQ Published
Early Fusion (U-TAE + PaPs) + Aux + TempDrop 1.8M 82.2 50.6 42.0 ✔️ link
Late Fusion (U-TAE + PaPs) + TempDrop 2.4M 81.6 50.5 41.6 ✔️ link

Documentation

The agricultural parcels are grouped into 18 different crop classes as shown in the table below. The backgroud class corresponds to non-agricultural land, and the void label for parcels that are mostly outside their patch. drawing

Additional information about the dataset can be found in the documentation/pastis-documentation.pdf document.

References

If you use PASTIS please cite the related paper:

@article{garnot2021panoptic,
  title={Panoptic Segmentation of Satellite Image Time Series
with Convolutional Temporal Attention Networks},
  author={Sainte Fare Garnot, Vivien  and Landrieu, Loic },
  journal={ICCV},
  year={2021}
}

For the PASTIS-R optical-radar fusion dataset, please also cite this paper:

@article{garnot2021mmfusion,
  title={Multi-Modal Temporal Attention Models for Crop Mapping from Satellite Time Series},
  author={Sainte Fare Garnot, Vivien  and Landrieu, Loic and Chehata, Nesrine },
  journal={arxiv},
  year={2021}
}

Credits

  • The satellite imagery used in PASTIS was retrieved from THEIA: "Value-added data processed by the CNES for the Theia www.theia.land.fr data cluster using Copernicus data. The treatments use algorithms developed by Theia’s Scientific Expertise Centres. "

  • The annotations used in PASTIS stem from the French land parcel identification system produced by IGN, the French mapping agency.

  • This work was partly supported by ASP, the French Payment Agency.

  • We also thank Zenodo for hosting the datasets.

The repository is for safe reinforcement learning baselines.

Safe-Reinforcement-Learning-Baseline The repository is for Safe Reinforcement Learning (RL) research, in which we investigate various safe RL baseline

172 Dec 19, 2022
Testbed of AI Systems Quality Management

qunomon Description A testbed for testing and managing AI system qualities. Demo Sorry. Not deployment public server at alpha version. Requirement Ins

AIST AIRC 15 Nov 27, 2021
A New Approach to Overgenerating and Scoring Abstractive Summaries

We provide the source code for the paper "A New Approach to Overgenerating and Scoring Abstractive Summaries" accepted at NAACL'21. If you find the code useful, please cite the following paper.

Kaiqiang Song 4 Apr 03, 2022
Pytorch domain adaptation package

DomainAdaptation This package is created to tackle the problem of domain shifts when dealing with two domains of different feature distributions. In d

Institute of Computational Perception 7 Oct 22, 2022
Repositorio oficial del curso IIC2233 Programación Avanzada 🚀✨

IIC2233 - Programación Avanzada Evaluación Las evaluaciones serán efectuadas por medio de actividades prácticas en clases y tareas. Se calculará la no

IIC2233 @ UC 0 Dec 15, 2022
🙄 Difficult algorithm, Simple code.

🎉TensorFlow2.0-Examples🎉! "Talk is cheap, show me the code." ----- Linus Torvalds Created by YunYang1994 This tutorial was designed for easily divin

1.7k Dec 25, 2022
Can we learn gradients by Hamiltonian Neural Networks?

Can we learn gradients by Hamiltonian Neural Networks? This project was carried out as part of the Optimization for Machine Learning course (CS-439) a

2 Aug 22, 2022
Dataset VSD4K includes 6 popular categories: game, sport, dance, vlog, interview and city.

CaFM-pytorch ICCV ACCEPT Introduction of dataset VSD4K Our dataset VSD4K includes 6 popular categories: game, sport, dance, vlog, interview and city.

96 Jul 05, 2022
Pocsploit is a lightweight, flexible and novel open source poc verification framework

Pocsploit is a lightweight, flexible and novel open source poc verification framework

cckuailong 208 Dec 24, 2022
YOLOv5 in PyTorch > ONNX > CoreML > TFLite

This repository represents Ultralytics open-source research into future object detection methods, and incorporates lessons learned and best practices evolved over thousands of hours of training and e

Ultralytics 34.1k Dec 31, 2022
Repository for training material for the 2022 SDSC HPC/CI User Training Course

hpc-training-2022 Repository for training material for the 2022 SDSC HPC/CI Training Series HPC/CI Training Series home https://www.sdsc.edu/event_ite

sdsc-hpc-training-org 21 Jul 27, 2022
StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion

StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion Yinghao Aaron Li, Ali Zare, Nima Mesgarani We pres

Aaron (Yinghao) Li 282 Jan 01, 2023
Implementation of Monocular Direct Sparse Localization in a Prior 3D Surfel Map (DSL)

DSL Project page: https://sites.google.com/view/dsl-ram-lab/ Monocular Direct Sparse Localization in a Prior 3D Surfel Map Authors: Haoyang Ye, Huaiya

Haoyang Ye 93 Nov 30, 2022
Colour detection is necessary to recognize objects, it is also used as a tool in various image editing and drawing apps.

Colour Detection On Image Colour detection is the process of detecting the name of any color. Simple isn’t it? Well, for humans this is an extremely e

Astitva Veer Garg 1 Jan 13, 2022
Classification models 1D Zoo - Keras and TF.Keras

Classification models 1D Zoo - Keras and TF.Keras This repository contains 1D variants of popular CNN models for classification like ResNets, DenseNet

Roman Solovyev 12 Jan 06, 2023
Application of K-means algorithm on a music dataset after a dimensionality reduction with PCA

PCA for dimensionality reduction combined with Kmeans Goal The Goal of this notebook is to apply a dimensionality reduction on a big dataset in order

Arturo Ghinassi 0 Sep 17, 2022
A facial recognition doorbell system using a Raspberry Pi

Facial Recognition Doorbell This project expands on the person-detecting doorbell system to allow it to identify faces, and announce names accordingly

rydercalmdown 22 Apr 15, 2022
Age Progression/Regression by Conditional Adversarial Autoencoder

Age Progression/Regression by Conditional Adversarial Autoencoder (CAAE) TensorFlow implementation of the algorithm in the paper Age Progression/Regre

Zhifei Zhang 603 Dec 22, 2022
Official pytorch implementation of "Feature Stylization and Domain-aware Contrastive Loss for Domain Generalization" ACMMM 2021 (Oral)

Feature Stylization and Domain-aware Contrastive Loss for Domain Generalization This is an official implementation of "Feature Stylization and Domain-

22 Sep 22, 2022
Deep Dual Consecutive Network for Human Pose Estimation (CVPR2021)

Beanie - is an asynchronous ODM for MongoDB, based on Motor and Pydantic. It uses an abstraction over Pydantic models and Motor collections to work wi

295 Dec 29, 2022