Multi-task yolov5 with detection and segmentation based on yolov5

Last update: Dec 30, 2022

Related tags

Deep Learning yolov5ds

Overview

YOLOv5DS

Multi-task yolov5 with detection and segmentation based on yolov5(branch v6.0)

decoupled head
anchor free
segmentation head

README中文

Ablation experiment

All experiments is trained on a small dataset with 47 classes ,2.6k+ images for training and 1.5k+ images for validation:

model	P	R	[email protected]	[email protected]:95
yolov5s	0.536	0.368	0.374	0.206
yolov5s+train scrach	0.452	0.314	0.306	0.152
yolov5s+decoupled head	0.555	0.375	0.387	0.214
yolov5s + decoupled head+class balance weights	0.541	0.392	0.396	0.217
yolov5s + decoupled head+class balance weights	0.574	0.386	0.403	0.22
yolov5s + decoupled head+seghead	0.533	0.383	0.396	0.212

The baseline model is yolov5s. and decoupled head, add class balance weights all helps to improve MAP.

Adding a segmentation head can still get equivalent MAP as single detection model.

Training Method

python trainds.py

As VOC dataset do not offer the box labels and mask labels, so we forward this model with a detection batch and a segmention batch , and accumulate the gradient , than update the whole model parameters.

MAP

To compare with the SSD512, we use VOC07+12 training set as the detection training set, VOC07 test data as detection test data, for segmentation ,we use VOC12 segmentation datset as training and test set.

the input size is 512(letter box).

model	VOC2007 test
SSD512	79.8
yolov5s+seghead(512)	79.2

The above results only trained less than 200 epoch, weights

demo

see detectds.py.

Train custom data

Use labelme to label box and mask on your dataset;

the box label format is voc, you can use voc2yolo.py to convert to yolo format,

the mask label is json files , you should convert to mask .png image labels,like VOC2012 segmentation labels.

see how to arrange your detection dataset with yolov5 , then arrange your segmentation dataset same as yolo files , see data/voc.yaml:


# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: .  # dataset root dir
train: VOC/det/images/train  # train images (relative to 'path') 118287 images
val: VOC/det/images/test  # train images (relative to 'path') 5000 images
road_seg_train: VOC/seg/images/train   # road segmentation data
road_seg_val: VOC/seg/images/val

# Classes
nc: 20  # number of classes
segnc: 20

names: ['aeroplane', 'bicycle', 'bird', 'boat',
           'bottle', 'bus', 'car', 'cat', 'chair',
           'cow', 'diningtable', 'dog', 'horse',
           'motorbike', 'person', 'pottedplant',
           'sheep', 'sofa', 'train', 'tvmonitor']  # class names

segnames: ['aeroplane', 'bicycle', 'bird', 'boat',
           'bottle', 'bus', 'car', 'cat', 'chair',
           'cow', 'diningtable', 'dog', 'horse',
           'motorbike', 'person', 'pottedplant',
           'sheep', 'sofa', 'train', 'tvmonitor']

change the config in trainds.py and :

python trainds.py

test image folder with :
```
python detectds.py
```

Comments

请问我在对训好的模型运行val.py时出现这个错误可能是什么问题

im = cv2.resize(im, new_unpad, interpolation=cv2.INTER_LINEAR) cv2.error: OpenCV(4.1.2) C:\projects\opencv-python\opencv\modules\imgproc\src\resize.cpp:3723: error: (-215:Assertion failed) inv_scale_x > 0 in function 'cv::resize'

opened by zhangfx123 0

Multi-task yolov5 with detection and segmentation based on yolov5

Related tags

Overview

YOLOv5DS

Ablation experiment

Training Method

MAP

demo

Train custom data

Reference

You might also like...

a basic code repository for basic task in CV(classification,detection,segmentation)

A novel Engagement Detection with Multi-Task Training (ED-MTT) system

YOLOv5 Series Multi-backbone, Pruning and quantization Compression Tool Box.

A Python training and inference implementation of Yolov5 helmet detection in Jetson Xavier nx and Jetson nano

YOLOv5 🚀 is a family of object detection architectures and models pretrained on the COCO dataset

Implementation of PyTorch-based multi-task pre-trained models

Drone detection using YOLOv5

YOLOv5 detection interface - PyQt5 implementation

YOLOv5 + ROS2 object detection package

Comments

请问我在对训好的模型运行val.py时出现这个错误可能是什么问题

Releases(v6.0)

v6.0(Dec 16, 2021)

Owner

PyTorch implementation of Deformable Convolution

Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learning

Biomarker identification for COVID-19 Severity in BALF cells Single-cell RNA-seq data

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

Dynamic Capacity Networks using Tensorflow

PyTorch code of my ICDAR 2021 paper Vision Transformer for Fast and Efficient Scene Text Recognition (ViTSTR)

General Multi-label Image Classification with Transformers

Transferable Unrestricted Attacks, which won 1st place in CVPR’21 Security AI Challenger: Unrestricted Adversarial Attacks on ImageNet.

Cave Generation using metaballs in Blender. Originally created by sdfgeoff, Edited by Myself (Archie Jaskowicz).

City-Scale Multi-Camera Vehicle Tracking Guided by Crossroad Zones Code

Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch

This is the repo for our work "Towards Persona-Based Empathetic Conversational Models" (EMNLP 2020)

Code for Mesh Convolution Using a Learned Kernel Basis

FCAF3D: Fully Convolutional Anchor-Free 3D Object Detection

MMGeneration is a powerful toolkit for generative models, based on PyTorch and MMCV.

Code for our TKDE paper "Understanding WeChat User Preferences and “Wow” Diffusion"

Deep Q-network learning to play flappybird.

Cours d'Algorithmique Appliquée avec Python pour BTS SIO SISR

Towards Debiasing NLU Models from Unknown Biases

Open-CyKG: An Open Cyber Threat Intelligence Knowledge Graph