Skip to content

3D position tracking for soccer players with multi-camera videos

Notifications You must be signed in to change notification settings

SherryJYC/3D-Tracking-MVS

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

3D Player Tracking with Multi-View Stream

Project for 3DV 2021 Spring @ ETH Zurich [Report Link]


This repo contains a full pipeline to support 3D position tracking of soccer players, with multi-view calibrated moving/fixed video sequences as inputs.
- In single-camera tracking stage, Tracktor++ is used to get 2D positions.
- In multi-camera tracking stage, 2D positions are projected into 3D positions. Then across-camera association is achieved as an optimization problem with spatial, temporal and visual constraints.
- In the end, visualization in 2D, 3D and a voronoi visualization for sports coaching purpose are provided.
3D Tracking Sports Coaching
demo demo

Demo

Check demo scripts as examples

Currently, processed data is under protection due to legal issues.

  • Run the demo visualization on the moving cameras
bash script/demo_moving.sh
  • Run the demo visualization on the fixed cameras
bash script/demo_fix.sh

Preprocessing

  • Split video into image frames
python src/utils/v2img.py --pathIn=data/0125-0135/CAM1/CAM1.mp4 --pathOut=data/0125-0135/CAM1/img --splitnum=1
  • Estimate football pitch homography (size 120m * 90m ref)

FIFA official document

python src/utils/computeHomo.py --img=data/0125-0135/RIGHT/img/image0000.jpg --out_dir=data/0125-0135/RIGHT/
  • Handle moving cameras
python src/utils/mov2static.py --calib_file=data/calibration_results/0125-0135/CAM1/calib.txt --img_dir=data/0125-0135/CAM1/img --output_dir=data/0125-0135/CAM1/img_static
  • Convert ground truth/annotation json to text file
python src/utils/json2txt.py --jsonfile=data/0125-0135/0125-0135.json

Single-camera tracking

  • Object Detector: frcnn_fpn
    Train object detector and generate detection results with this Google Colab notebook. [pretrained model]
  • Run Tracktor++
    Put trainded object detector model_epoch_50.model into src/tracking_wo_bnw/output/faster_rcnn_fpn_training_soccer/.
    Put data and calibration results into src/tracking_wo_bnw/.
cd src/tracking_wo_bnw
python experiments/scripts/test_tracktor.py
  • Run ReID(team id) model
python src/team_classification/team_svm.py PATH_TO_TRACKING_RESULT PATH_TO_IMAGES
  • Convert tracking results to coordinates on the pitch

Equation to find the intersection of a line with a plane (ref)

python src/calib.py --calib_path=PATH_TO_CALIB --res_path=PATH_TO_TRACKING_RESULT --xymode --reid

# also plot the camera positions for fixed cameras
python src/calib.py --calib_path=PATH_TO_CALIB --res_path=PATH_TO_TRACKING_RESULT --viz

Across-camera association

  • Run two-cam tracker
python src/runMCTRacker.py 

# add team id constraint
python src/runMCTRacker.py --doreid
  • Run multi-cam tracker (e.g. 8 cams)
python src/runTreeMCTracker.py --doreid

Evaluation

  • Produce quatitative results (visualize results)

visualize 2d bounding box

# if format <x, y, w, h>
python src/utils/visualize.py --img_dir=data/0125-0135/RIGHT/img --result_file=output/tracktor/16m_right_prediction.txt 
# if format <x1, y1, x2, y2>
python src/utils/visualize.py --img_dir=data/0125-0135/RIGHT/img --result_file=output/iou/16m_right.txt --xymode
# if with team id
python src/utils/visualize.py --img_dir=data/0125-0135/RIGHT/img --result_file=output/tracktor/16m_right_prediction.txt --reid
# if 3d mode
python src/utils/visualize.py --img_dir=data/0125-0135/RIGHT/img --result_file=output/tracktor/RIGHT.txt --calib_file=data/calibration_results/0125-0135/RIGHT/calib.txt  --pitchmode

visualize 3d tracking result with ground truth and voronoi diagram

python src/utils/visualize_on_pitch.py --result_file=PATH_TO_TRACKING_RESULT --ground_truth=PATH_TO_GROUND_TRUTH

visualize 3d ground truth on camera frames (reprojection)

python src/utils/visualize_tracab --img_path=PATH_TO_IMAGES --calib_path=PATH_TO_CALIB --gt_path=PATH_TO_TRACAB_GT --output_path=PATH_TO_OUTPUT_VIDEO
  • Produce quantitative result
# 2d <frame id, objid, x, y, w, h, .., ...>
python src/motmetrics/apps/eval_motchallenge.py data/0125-0135/ output/tracktor_filtered

# 3d
python src/utils/eval3d.py --pred=output/pitch/EPTS_3_pitch.txt_EPTS_4_pitch.txt.txt --fixcam  --gt=data/fixedcam/gt_pitch_550.txt
python src/utils/eval3d.py --fixcam --boxplot

Acknowledgement

We would like to thank the following Github repos or softwares:

Authors

Yuchang Jiang, Tianyu Wu, Ying Jiao, Yelan Tao

About

3D position tracking for soccer players with multi-camera videos

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published