A PyTorch implementation of SIN: Superpixel Interpolation Network

Related tags

Deep LearningSIN
Overview

SIN: Superpixel Interpolation Network

This is is a PyTorch implementation of the superpixel segmentation network introduced in our PRICAI-2021 paper:

SIN: Superpixel Interpolation Network

Prerequisites

The training code was mainly developed and tested with python 3.6, PyTorch 1.4, CUDA 10, and Ubuntu 18.04.

Demo

The demo script run_demo.py provides the superpixels with grid size 16 x 16 using our pre-trained model (in /pretrained_ckpt). Please feel free to provide your own images by copying them into /demo/inputs, and run

python run_demo.py --data_dir=./demo/inputs --data_suffix=jpg --output=./demo 

The results will be generate in a new folder under /demo called spixel_viz.

Data preparation

To generate training and test dataset, please first download the data from the original BSDS500 dataset, and extract it to . Then, run

cd data_preprocessing
python pre_process_bsd500.py --dataset=
   
     --dump_root=
    
     
python pre_process_bsd500_ori_sz.py --dataset=
     
       --dump_root=
      
       
cd ..

      
     
    
   

The code will generate three folders under the , named as /train, /val, and /test, and three .txt files record the absolute path of the images, named as train.txt, val.txt, and test.txt.

Training

Once the data is prepared, we should be able to train the model by running the following command

python main.py --data=
   
     --savepath=
    

    
   

if we wish to continue a train process or fine-tune from a pre-trained model, we can run

python main.py --data=
   
     --savepath=
    
      --pretrained=
      

     
    
   

The code will start from the recorded status, which includes the optimizer status and epoch number.

The training log can be viewed from the tensorboard session by running

tensorboard --logdir=
   
     --port=8888

   

Testing

We provide test code to generate: 1) superpixel visualization and 2) the.csv files for evaluation.

To test on BSDS500, run

python run_infer_bsds.py --data_dir=
   
     --output=
    
      --pretrained=
     

     
    
   

To test on NYUv2, please follow the intruction on the superpixel benchmark to generate the test dataset, and then run

python run_infer_nyu.py --data_dir=
   
     --output=
    
      --pretrained=
     

     
    
   

To test on other datasets, please first collect all the images into one folder , and then convert them into the same format (e.g. .png or .jpg) if necessary, and run

python run_demo.py --data_dir=
   
     --data_suffix=
    
      --output=
     
       --pretrained=
      

      
     
    
   

Superpixels with grid size 16 x 16 will be generated by default. To generate the superpixel with a different grid size, we simply need to resize the images into the approporate resolution before passing them through the code. Please refer to run_infer_nyu.py for the details.

Evaluation

We use the code from superpixel benchmark for superpixel evaluation. A detailed instruction is available in the repository, please

(1) download the code and build it accordingly;

(2) edit the variables $SUPERPIXELS, IMG_PATH and GT_PATH in /eval_spixel/my_eval.sh,

(3) run

cp /eval_spixel/my_eval.sh 
   
    /examples/bash/
cd  
    
     /examples/
bash my_eval.sh

    
   

several files should be generated in the map_csv folders in the corresponding test outputs;

(4) run

cd eval_spixel
python copy_resCSV.py --src=
   
     --dst=
    

    
   

(5) open /eval_spixel/plot_benchmark_curve.m , set the our1l_res_path as and modify the num_list according to the test setting. The default setting is for our BSDS500 test set.;

(6) run the plot_benchmark_curve.m, the ASA Score, CO Score, and BR-BP curve of our method should be shown on the screen. If you wish to compare our method with the others, you can first run the method and organize the data as we state above, and uncomment the code in the plot_benchmark_curve.m to generate a similar figure shown in our papers.

Acknowledgement

The code is implemented based on superpixel_fcn. We would like to express our sincere thanks to the contributors.

Cite

If you use SIN in your work please cite our paper:

@article{yuan2021sin,
title={SIN: Superpixel Interpolation Network},
author={Qing Yuan, Songfeng Lu, Yan Huang, Wuxin Sha},
booktitle={PRICAI},
year={2021}
}

A GridMixup augmentation, inspired by GridMask and CutMix

GridMixup A GridMixup augmentation, inspired by GridMask and CutMix Easy install pip install git+https://github.com/IlyaDobrynin/GridMixup.git Overvie

IlyaDo 42 Dec 28, 2022
NCVX (NonConVeX): A User-Friendly and Scalable Package for Nonconvex Optimization in Machine Learning.

NCVX NCVX: A User-Friendly and Scalable Package for Nonconvex Optimization in Machine Learning. Please check https://ncvx.org for detailed instruction

SUN Group @ UMN 28 Aug 03, 2022
Online Pseudo Label Generation by Hierarchical Cluster Dynamics for Adaptive Person Re-identification

Online Pseudo Label Generation by Hierarchical Cluster Dynamics for Adaptive Person Re-identification

TANG, shixiang 6 Nov 25, 2022
Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

Apache MXNet (incubating) for Deep Learning Apache MXNet is a deep learning framework designed for both efficiency and flexibility. It allows you to m

The Apache Software Foundation 20.2k Jan 05, 2023
[ICCV'21] Neural Radiance Flow for 4D View Synthesis and Video Processing

NeRFlow [ICCV'21] Neural Radiance Flow for 4D View Synthesis and Video Processing Datasets The pouring dataset used for experiments can be download he

44 Dec 20, 2022
Source code for paper "ATP: AMRize Than Parse! Enhancing AMR Parsing with PseudoAMRs" @NAACL-2022

ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs Hi this is the source code of our paper "ATP: AMRize Then Parse! Enhancing AMR Parsing w

Chen Liang 13 Nov 23, 2022
CenterNet:Objects as Points目标检测模型在Pytorch当中的实现

CenterNet:Objects as Points目标检测模型在Pytorch当中的实现

Bubbliiiing 267 Dec 29, 2022
SNIPS: Solving Noisy Inverse Problems Stochastically

SNIPS: Solving Noisy Inverse Problems Stochastically This repo contains the official implementation for the paper SNIPS: Solving Noisy Inverse Problem

Bahjat Kawar 35 Nov 09, 2022
Code for the paper "TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks"

TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks This is a Python3 / Pytorch implementation of TadGAN paper. The associated

Arun 92 Dec 03, 2022
An energy estimator for eyeriss-like DNN hardware accelerator

Energy-Estimator-for-Eyeriss-like-Architecture- An energy estimator for eyeriss-like DNN hardware accelerator This is an energy estimator for eyeriss-

HEXIN BAO 2 Mar 26, 2022
A synthetic texture-invariant dataset for object detection of UAVs

A synthetic dataset for object detection of UAVs This repository contains a synthetic datasets accompanying the paper Sim2Air - Synthetic aerial datas

LARICS Lab 10 Aug 13, 2022
TensorFlow implementation of original paper : https://github.com/hszhao/PSPNet

Keras implementation of PSPNet(caffe) Implemented Architecture of Pyramid Scene Parsing Network in Keras. For the best compability please use Python3.

VladKry 386 Dec 29, 2022
A repository for storing njxzc final exam review material

文档地址,请戳我 👈 👈 👈 ☀️ 1.Reason 大三上期末复习软件工程的时候,发现其他高校在GitHub上开源了他们学校的期末试题,我很受触动。期末

GuJiakai 2 Jan 18, 2022
PyTorch wrappers for using your model in audacity!

audacitorch This package contains utilities for prepping PyTorch audio models for use in Audacity. More specifically, it provides abstract classes for

Hugo Flores García 130 Dec 14, 2022
Implementation of Basic Machine Learning Algorithms on small datasets using Scikit Learn.

Basic Machine Learning Algorithms All the basic Machine Learning Algorithms are implemented in Python using libraries Acknowledgements Machine Learnin

Piyal Banik 47 Oct 16, 2022
render sprites into your desktop environment as shaped windows using GTK

spritegtk render static or animated sprites into your desktop environment as dynamic shaped windows using GTK requires pycairo and PYGobject: pip inst

hermit 20 Oct 27, 2022
git《Commonsense Knowledge Base Completion with Structural and Semantic Context》(AAAI 2020) GitHub: [fig1]

Commonsense Knowledge Base Completion with Structural and Semantic Context Code for the paper Commonsense Knowledge Base Completion with Structural an

AI2 96 Nov 05, 2022
Implementation of StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation in PyTorch

StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation Implementation of StyleSpace Analysis: Disentangled Controls for StyleGAN Ima

Xuanchi Ren 86 Dec 07, 2022
Official repo of the paper "Surface Form Competition: Why the Highest Probability Answer Isn't Always Right"

Surface Form Competition This is the official repo of the paper "Surface Form Competition: Why the Highest Probability Answer Isn't Always Right" We p

Peter West 46 Dec 23, 2022
Ensemble Visual-Inertial Odometry (EnVIO)

Ensemble Visual-Inertial Odometry (EnVIO) Authors : Jae Hyung Jung, Yeongkwon Choe, and Chan Gook Park 1. Overview This is a ROS package of Ensemble V

Jae Hyung Jung 95 Jan 03, 2023