Multi-task yolov5 with detection and segmentation based on yolov5

Last update: Dec 30, 2022

Related tags

Deep Learning yolov5ds

Overview

YOLOv5DS

Multi-task yolov5 with detection and segmentation based on yolov5(branch v6.0)

decoupled head
anchor free
segmentation head

README中文

Ablation experiment

All experiments is trained on a small dataset with 47 classes ,2.6k+ images for training and 1.5k+ images for validation:

model	P	R	[email protected]	[email protected]:95
yolov5s	0.536	0.368	0.374	0.206
yolov5s+train scrach	0.452	0.314	0.306	0.152
yolov5s+decoupled head	0.555	0.375	0.387	0.214
yolov5s + decoupled head+class balance weights	0.541	0.392	0.396	0.217
yolov5s + decoupled head+class balance weights	0.574	0.386	0.403	0.22
yolov5s + decoupled head+seghead	0.533	0.383	0.396	0.212

The baseline model is yolov5s. and decoupled head, add class balance weights all helps to improve MAP.

Adding a segmentation head can still get equivalent MAP as single detection model.

Training Method

python trainds.py

As VOC dataset do not offer the box labels and mask labels, so we forward this model with a detection batch and a segmention batch , and accumulate the gradient , than update the whole model parameters.

MAP

To compare with the SSD512, we use VOC07+12 training set as the detection training set, VOC07 test data as detection test data, for segmentation ,we use VOC12 segmentation datset as training and test set.

the input size is 512(letter box).

model	VOC2007 test
SSD512	79.8
yolov5s+seghead(512)	79.2

The above results only trained less than 200 epoch, weights

demo

see detectds.py.

Train custom data

Use labelme to label box and mask on your dataset;

the box label format is voc, you can use voc2yolo.py to convert to yolo format,

the mask label is json files , you should convert to mask .png image labels,like VOC2012 segmentation labels.

see how to arrange your detection dataset with yolov5 , then arrange your segmentation dataset same as yolo files , see data/voc.yaml:


# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: .  # dataset root dir
train: VOC/det/images/train  # train images (relative to 'path') 118287 images
val: VOC/det/images/test  # train images (relative to 'path') 5000 images
road_seg_train: VOC/seg/images/train   # road segmentation data
road_seg_val: VOC/seg/images/val

# Classes
nc: 20  # number of classes
segnc: 20

names: ['aeroplane', 'bicycle', 'bird', 'boat',
           'bottle', 'bus', 'car', 'cat', 'chair',
           'cow', 'diningtable', 'dog', 'horse',
           'motorbike', 'person', 'pottedplant',
           'sheep', 'sofa', 'train', 'tvmonitor']  # class names

segnames: ['aeroplane', 'bicycle', 'bird', 'boat',
           'bottle', 'bus', 'car', 'cat', 'chair',
           'cow', 'diningtable', 'dog', 'horse',
           'motorbike', 'person', 'pottedplant',
           'sheep', 'sofa', 'train', 'tvmonitor']

change the config in trainds.py and :

python trainds.py

test image folder with :
```
python detectds.py
```

Comments

请问我在对训好的模型运行val.py时出现这个错误可能是什么问题

im = cv2.resize(im, new_unpad, interpolation=cv2.INTER_LINEAR) cv2.error: OpenCV(4.1.2) C:\projects\opencv-python\opencv\modules\imgproc\src\resize.cpp:3723: error: (-215:Assertion failed) inv_scale_x > 0 in function 'cv::resize'

opened by zhangfx123 0

Multi-task yolov5 with detection and segmentation based on yolov5

Related tags

Overview

YOLOv5DS

Ablation experiment

Training Method

MAP

demo

Train custom data

Reference

You might also like...

a basic code repository for basic task in CV(classification,detection,segmentation)

A novel Engagement Detection with Multi-Task Training (ED-MTT) system

YOLOv5 Series Multi-backbone, Pruning and quantization Compression Tool Box.

A Python training and inference implementation of Yolov5 helmet detection in Jetson Xavier nx and Jetson nano

YOLOv5 🚀 is a family of object detection architectures and models pretrained on the COCO dataset

Implementation of PyTorch-based multi-task pre-trained models

Drone detection using YOLOv5

YOLOv5 detection interface - PyQt5 implementation

YOLOv5 + ROS2 object detection package

Comments

请问我在对训好的模型运行val.py时出现这个错误可能是什么问题

Releases(v6.0)

v6.0(Dec 16, 2021)

Owner

Video Instance Segmentation using Inter-Frame Communication Transformers (NeurIPS 2021)

This code implements constituency parse tree aggregation

🥇 LG-AI-Challenge 2022 1위 솔루션 입니다.

The repository for our EMNLP 2021 paper "Finnish Dialect Identification: The Effect of Audio and Text"

CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation

A pytorch &keras implementation and demo of Fastformer.

Code for EMNLP 2021 paper Contrastive Out-of-Distribution Detection for Pretrained Transformers.

Elevation Mapping on GPU.

Convert Pytorch model to onnx or tflite, and the converted model can be visualized by Netron

A research toolkit for particle swarm optimization in Python

Qt-GUI implementation of the YOLOv5 algorithm (ver.6 and ver.5)

WatermarkRemoval-WDNet-WACV2021

ICRA 2021 - Robust Place Recognition using an Imaging Lidar

Public Models considered for emotion estimation from EEG

Framework for abstracting Amiga debuggers and access to AmigaOS libraries and devices.

Train neural network for semantic segmentation (deep lab V3) with pytorch in less then 50 lines of code

🤗 Paper Style Guide

2D&3D human pose estimation

Code for paper "Document-Level Argument Extraction by Conditional Generation". NAACL 21'

Pytorch implementation of Distributed Proximal Policy Optimization: https://arxiv.org/abs/1707.02286