Two-stage CenterNet

Overview

Probabilistic two-stage detection

Two-stage object detectors that use class-agnostic one-stage detectors as the proposal network.

Probabilistic two-stage detection,
Xingyi Zhou, Vladlen Koltun, Philipp Krähenbühl,
arXiv technical report (arXiv 2103.07461)

Contact: [email protected]. Any questions or discussions are welcomed!

Abstract

We develop a probabilistic interpretation of two-stage object detection. We show that this probabilistic interpretation motivates a number of common empirical training practices. It also suggests changes to two-stage detection pipelines. Specifically, the first stage should infer proper object-vs-background likelihoods, which should then inform the overall score of the detector. A standard region proposal network (RPN) cannot infer this likelihood sufficiently well, but many one-stage detectors can. We show how to build a probabilistic two-stage detector from any state-of-the-art one-stage detector. The resulting detectors are faster and more accurate than both their one- and two-stage precursors. Our detector achieves 56.4 mAP on COCO test-dev with single-scale testing, outperforming all published results. Using a lightweight backbone, our detector achieves 49.2 mAP on COCO at 33 fps on a Titan Xp.

Summary

  • Two-stage CenterNet: First stage estimates object probabilities, second stage conditionally classifies objects.

  • Resulting detector is faster and more accurate than both traditional two-stage detectors (fewer proposals required), and one-stage detectors (lighter first stage head).

  • Our best model achieves 56.4 mAP on COCO test-dev.

  • This repo also includes a detectron2-based CenterNet implementation with better accuracy (42.5 mAP at 70FPS) and a new FPN version of CenterNet (40.2 mAP with Res50_1x).

Main results

All models are trained with multi-scale training, and tested with a single scale. The FPS is tested on a Titan RTX GPU. More models and details can be found in the MODEL_ZOO.

COCO

Model COCO val mAP FPS
CenterNet-S4_DLA_8x 42.5 71
CenterNet2_R50_1x 42.9 24
CenterNet2_X101-DCN_2x 49.9 8
CenterNet2_R2-101-DCN-BiFPN_4x+4x_1560_ST 56.1 5
CenterNet2_DLA-BiFPN-P5_24x_ST 49.2 38

LVIS

Model val mAP box
CenterNet2_R50_1x 26.5
CenterNet2_FedLoss_R50_1x 28.3

Objects365

Model val mAP
CenterNet2_R50_1x 22.6

Installation

Our project is developed on detectron2. Please follow the official detectron2 installation. All our code is under projects/CenterNet2/. In theory, you should be able to copy-paste projects/CenterNet2/ to the latest detectron2 release or your own detectron2 repo to run our project. There might be API changes in future detectron2 releases that make it incompatible.

Demo

We use the default detectron2 demo script. To run inference on an image folder using our pre-trained model, run

python projects/CenterNet2/demo/demo.py --config-file projects/CenterNet2/configs/CenterNet2_R50_1x.yaml --input path/to/image/ --opts MODEL.WEIGHTS models/CenterNet2_R50_1x.pth

Benchmark evaluation and training

Please check detectron2 GETTING_STARTED.md for running evaluation and training. Our config files are under projects/CenterNet2/configs and the pre-trained models are in the MODEL_ZOO.

License

Our code under projects/CenterNet2/ is under Apache 2.0 license. projects/CenterNet2/centernet/modeling/backbone/bifpn_fcos.py are from AdelaiDet, which follows the original non-commercial license. The code from detectron2 follows the original Apache 2.0 license.

Citation

If you find this project useful for your research, please use the following BibTeX entry.

@inproceedings{zhou2021probablistic,
  title={Probabilistic two-stage detection},
  author={Zhou, Xingyi and Koltun, Vladlen and Kr{\"a}henb{\"u}hl, Philipp},
  booktitle={arXiv preprint arXiv:2103.07461},
  year={2021}
}
Issues
  • What are the difference Custom ROI head layers?

    What are the difference Custom ROI head layers?

    📚 Documentation

    Hi! Again. Thanks for opensouring the code. I wonder the main differences between "CustomCascadeROIHeads" "CustomFastRCNNOutputLayers" and original "CascadeROIHeads" and "FastRCNNOutputLayers" ?

    What are the results of using original modules by replacing RPN with CenterNet ?

    documentation 
    opened by lxtGH 7
  • problem with registering custom dataset

    problem with registering custom dataset

    command Line Args: Namespace(config_file='/content/CenterNet2/projects/CenterNet2/configs/CenterNet2_R2-101-DCN-BiFPN_4x+4x_1560_ST.yaml', dist_url='tcp://127.0.0.1:23538', eval_only=False, machine_rank=0, manual_device='', num_gpus=1, num_machines=1, opts=[], resume=False) Config '/content/CenterNet2/projects/CenterNet2/configs/CenterNet2_R2-101-DCN-BiFPN_4x+4x_1560_ST.yaml' has no VERSION. Assuming it to be compatible with latest v2. Traceback (most recent call last): File "/content/CenterNet2/projects/CenterNet2/train_net.py", line 227, in args=(args,), File "/usr/local/lib/python3.7/dist-packages/detectron2/engine/launch.py", line 62, in launch main_func(*args) File "/content/CenterNet2/projects/CenterNet2/train_net.py", line 187, in main cfg = setup(args) File "/content/CenterNet2/projects/CenterNet2/train_net.py", line 175, in setup cfg.merge_from_file(args.config_file) File "/usr/local/lib/python3.7/dist-packages/detectron2/config/config.py", line 54, in merge_from_file self.merge_from_other_cfg(loaded_cfg) File "/usr/local/lib/python3.7/dist-packages/fvcore/common/config.py", line 123, in merge_from_other_cfg return super().merge_from_other_cfg(cfg_other) File "/usr/local/lib/python3.7/dist-packages/yacs/config.py", line 217, in merge_from_other_cfg _merge_a_into_b(cfg_other, self, self, []) File "/usr/local/lib/python3.7/dist-packages/yacs/config.py", line 474, in _merge_a_into_b v = _check_and_coerce_cfg_value_type(v, b[k], k, full_key) File "/usr/local/lib/python3.7/dist-packages/yacs/config.py", line 537, in _check_and_coerce_cfg_value_type original_type, replacement_type, original, replacement, full_key ValueError: Type mismatch (<class 'detectron2.config.config.CfgNode'> vs. <class 'str'>) with values (PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000 PRECOMPUTED_PROPOSAL_TOPK_TRAIN: 2000 PROPOSAL_FILES_TEST: () PROPOSAL_FILES_TRAIN: () TEST: () TRAIN: () vs. TRAIN = ('my_dataset_train',) TEST = ('my_dataset_test',)) for config key: DATASETS

    thanks for this amazing work , I am running into this issue after I register my dataset i am using colab and before running !python /content/CenterNet2/projects/CenterNet2/train_net.py --config /content/CenterNet2/projects/CenterNet2/configs/CenterNet2_R2-101-DCN-BiFPN_4x+4x_1560_ST.yaml I am registering the dataset like this from detectron2.data.datasets import register_coco_instances register_coco_instances("my_dataset_train", {}, "/content/results/yolo2coco/train.json", "/content/results/yolo2coco/train") #register_coco_instances("my_dataset_val", {}, "/content/results/yolo2coco/test.json", "/content/results/yolo2coco/valid") register_coco_instances("my_dataset_test", {}, "/content/results/yolo2coco/test.json", "/content/results/yolo2coco/test") and adding the name my_dataset_train and my_dataset_test in config Could you please help me out .

    opened by gauravlochab 7
  • KeyError: 'Non-existent config key: MODEL.ROI_BOX_HEAD.MULT_PROPOSAL_SCORE'

    KeyError: 'Non-existent config key: MODEL.ROI_BOX_HEAD.MULT_PROPOSAL_SCORE'

    when I am trying to train the center2 with my own data, here is an error.

    File "/home/atesi3/anaconda3/envs/torch18/lib/python3.6/site-packages/yacs/config.py", line 491, in _merge_a_into_b raise KeyError("Non-existent config key: {}".format(full_key)) KeyError: 'Non-existent config key: MODEL.ROI_BOX_HEAD.MULT_PROPOSAL_SCORE'

    Which version of the yacs did you use? or which version of the detectron2? Plese help.

    opened by lbf4616 4
  • Custom dataset classes

    Custom dataset classes

    📚 Documentation

    • Links to the relevant documentation/comment: https://github.com/facebookresearch/detectron2/blob/master/datasets/README.md

    Hi, I wonder how to register custom dataset classes, since I already tried to train with custom dataset and getting this error.

    Traceback (most recent call last):
      File "train_net.py", line 228, in <module>
        args=(args,),
      File "/usr/local/lib/python3.7/dist-packages/detectron2/engine/launch.py", line 62, in launch
        main_func(*args)
      File "train_net.py", line 208, in main
        do_train(cfg, model, resume=args.resume)
      File "train_net.py", line 108, in do_train
        data_loader = build_detection_train_loader(cfg, mapper=mapper)
      File "/usr/local/lib/python3.7/dist-packages/detectron2/config/config.py", line 201, in wrapped
        explicit_args = _get_args_from_config(from_config, *args, **kwargs)
      File "/usr/local/lib/python3.7/dist-packages/detectron2/config/config.py", line 238, in _get_args_from_config
        ret = from_config_func(*args, **kwargs)
      File "/usr/local/lib/python3.7/dist-packages/detectron2/data/build.py", line 314, in _train_loader_from_config
        proposal_files=cfg.DATASETS.PROPOSAL_FILES_TRAIN if cfg.MODEL.LOAD_PROPOSALS else None,
      File "/usr/local/lib/python3.7/dist-packages/detectron2/data/build.py", line 227, in get_detection_dataset_dicts
        dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in names]
      File "/usr/local/lib/python3.7/dist-packages/detectron2/data/build.py", line 227, in <listcomp>
        dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in names]
      File "/usr/local/lib/python3.7/dist-packages/detectron2/data/catalog.py", line 58, in get
        return f()
      File "/usr/local/lib/python3.7/dist-packages/detectron2/data/datasets/coco.py", line 489, in <lambda>
        DatasetCatalog.register(name, lambda: load_coco_json(json_file, image_root, name))
      File "/usr/local/lib/python3.7/dist-packages/detectron2/data/datasets/coco.py", line 80, in load_coco_json
        meta.thing_classes = thing_classes
      File "/usr/local/lib/python3.7/dist-packages/detectron2/data/catalog.py", line 150, in __setattr__
        "to a different value!\n{} != {}".format(key, self.name, oldval, val)
    AssertionError: Attribute 'thing_classes' in the metadata of 'coco_2017_train' cannot be set to a different value!
    ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light', 'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard', 'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone', 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear', 'hair drier', 'toothbrush'] != ['class1', 'class2', 'class3']
    
    documentation 
    opened by salmanhiro 3
  • I got an error when I train CenterNet2_R2-101-DCN-BiFPN_4x+4x_1560_ST

    I got an error when I train CenterNet2_R2-101-DCN-BiFPN_4x+4x_1560_ST

    I trained : python ./tools/train_net.py --config-file ./configs/COCO-Detection/faster_rcnn_R_50_C4_1x.yaml --num-gpus 1 SOLVER.IMS_PER_BATCH 2 SOLVER.BASE_LR 0.0025 image Susccessful!! But I run : python ./projects/CenterNet2/train_net.py --config-file ./projects/CenterNet2/configs/CenterNet2_R2-101-DCN-BiFPN_4x+4x_1560_ST.yaml --num-gpus 1 SOLVER.IMS_PER_BATCH 2

    image There is an error, I don't know why?

    opened by marcovwu 3
  • CenterTrack2?

    CenterTrack2?

    Hi, just curious, will you be implementing a CenterTrack2 to go along with this?

    enhancement 
    opened by austinmw 2
  • GPU memory is not fully used.

    GPU memory is not fully used.

    I have "cuda out of memory error" whereas in pytorch uses only 60% of my GPU memory. Can you help me with this issue? My GPU is RTX3090 wtih 24GB VRAM. However model only uses up to 15-16GB.

    opened by Shoxa-Mir 2
  • CenterNet2_DLA-BiFPN-P5_640_24x_ST Config Missing

    CenterNet2_DLA-BiFPN-P5_640_24x_ST Config Missing

    📚 Documentation

    It looks like CenterNet2_DLA-BiFPN-P5_640_24x_ST config is missing and the link in the ModelZoo table is broken config. Is this configuration deprecated?

    documentation 
    opened by bakerada 2
  • Error :AssertionError: Box tensor contains infinite or NaN!

    Error :AssertionError: Box tensor contains infinite or NaN!

    📚 Documentation

    I used the coco dataset to train centernet2, but got a error:

    Traceback (most recent call last):
      File "./train_net.py", line 245, in <module>
        args=(args,),
      File "/dataset/datacode/code/CenterNet2/detectron2/engine/launch.py", line 62, in launch
        main_func(*args)
      File "./train_net.py", line 226, in main
        do_train(cfg, model, resume=args.resume)
      File "./train_net.py", line 128, in do_train
        loss_dict = model(data)
      File "/home/das/anaconda3/envs/centernet2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
        result = self.forward(*input, **kwargs)
      File "/dataset/datacode/code/CenterNet2/detectron2/modeling/meta_arch/rcnn.py", line 166, in forward
        _, detector_losses = self.roi_heads(images, features, proposals, gt_instances)
      File "/home/das/anaconda3/envs/centernet2/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
        result = self.forward(*input, **kwargs)
      File "/dataset/datacode/code/CenterNet2/projects/CenterNet2/centernet/modeling/roi_heads/custom_roi_heads.py", line 166, in forward
        losses = self._forward_box(features, proposals, targets)
      File "/dataset/datacode/code/CenterNet2/projects/CenterNet2/centernet/modeling/roi_heads/custom_roi_heads.py", line 116, in _forward_box
        proposals = self._create_proposals_from_boxes(prev_pred_boxes, image_sizes)
      File "/dataset/datacode/code/CenterNet2/detectron2/modeling/roi_heads/cascade_rcnn.py", line 290, in _create_proposals_from_boxes
        boxes_per_image.clip(image_size)
      File "/dataset/datacode/code/CenterNet2/detectron2/structures/boxes.py", line 200, in clip
        assert torch.isfinite(self.tensor).all(), "Box tensor contains infinite or NaN!"
    AssertionError: Box tensor contains infinite or NaN!
    

    where is the main wrong? I think the coco dataset is OK

    documentation 
    opened by Lg955 2
  • what is the suitable iter for my dataset

    what is the suitable iter for my dataset

    📚 Documentation

    I have 10000 images and 5 classes, what is the suitable MAX_ITER for training Centernet2?

    documentation 
    opened by Lg955 2
  • Import add_centernet_config from config.py to __init__.py

    Import add_centernet_config from config.py to __init__.py

    While using CenterNet2 in Detectron2, if add_centernet_config is not imported to init.py, it gives - KeyError: 'Non-existent config key: MODEL.ROI_BOX_HEAD.MULT_PROPOSAL_SCORE'.

    As per other projects under Detectron2 (for example - panoptic_deeplab), we can see that init.py imports add_panoptic_deeplab_config from its own config.py and then we call add_panoptic_deeplab_config and pass the YAML path to this function.

    In the case of panoptic_deeplab,

    from detectron2.projects import panoptic_deeplab
    panoptic_deeplab.add_panoptic_deeplab_config (cfg_path)
    

    Since 'add_centernet_config' was missing from centernet's init.py , I added it and the error was resolved.

    opened by SahilChachra 0
  • Inference using demo.py issue

    Inference using demo.py issue

    Hello, When I try to inference using demo.py file I am getting all the results however, I get only class_ids on top of the bounding boxes. How can I modify the code to put class_name instead?

    I have registered dataset using detectron2 tutorial and using coco format json annotations for my custom dataset.

    opened by Shoxa-Mir 0
  • run demo.py for video output error

    run demo.py for video output error

    python demo.py --video-input /workspace/xdy/CenterNet2-master/output/video/2021111616185900.mp4 [12/09 01:25:50 detectron2]: Arguments: Namespace(config_file='/workspace/xdy/CenterNet2-master/projects/CenterNet2/configs/CenterNet2_DLA-BiFPN-P5_640_16x_ST.yaml', webcam=False, video_input='/workspace/xdy/CenterNet2-master/output/video/2021111616185900.mp4', input=None, output='/workspace/xdy/CenterNet2-master/output/val/', confidence_threshold=0.3, opts=[]) WARNING [12/09 01:25:50 d2.config.compat]: Config '/workspace/xdy/CenterNet2-master/projects/CenterNet2/configs/CenterNet2_DLA-BiFPN-P5_640_16x_ST.yaml' has no VERSION. Assuming it to be compatible with latest v2. Loading pretrained DLA! 0%| | 0/7501 [00:01<?, ?it/s] Traceback (most recent call last): File "/workspace/xdy/CenterNet2-master/projects/CenterNet2/demo.py", line 185, in for vis_frame in tqdm.tqdm(demo.run_on_video(video), total=num_frames): File "/root/miniconda3/envs/detectron2/lib/python3.9/site-packages/tqdm/std.py", line 1180, in iter for obj in iterable: File "/workspace/xdy/CenterNet2-master/projects/CenterNet2/predictor.py", line 152, in run_on_video yield process_predictions(frame, self.predictor(frame)) File "/workspace/xdy/CenterNet2-master/projects/CenterNet2/predictor.py", line 115, in process_predictions vis_frame = video_visualizer.draw_instance_predictions(frame, predictions) File "/workspace/xdy/detectron2/detectron2/utils/video_visualizer.py", line 109, in draw_instance_predictions else [y[0] for y in filter(lambda x: x[1], zip(labels, visibilities))] TypeError: 'NoneType' object is not iterable

    I run demo.py for video output,but something wrong here.But when I run demo.py with Base-CenterNet2.yaml,it work.how can i fix it?

    opened by Dannyxu1031 3
  • About Gaussian kernel calculation

    About Gaussian kernel calculation

    https://github.com/xingyizhou/CenterNet2/blob/9ba5c8e6ba6b44afc1c19d7767b0ec72cb314586/projects/CenterNet2/centernet/modeling/dense_heads/centernet.py#L306 I can't understand this part of the code well,can you provide complete mathematical formulas?How to calculate the Gaussian kernel and apply it in the heatmap?Thanks very much.

    opened by shenjiantao 1
  • Initialization of  bias of

    Initialization of bias of "bbox_pred" in CenterNetHead

    📚 Documentation

    • Links to the relevant documentation/comment: https://github.com/xingyizhou/CenterNet2/blob/9ba5c8e6ba6b44afc1c19d7767b0ec72cb314586/projects/CenterNet2/centernet/modeling/dense_heads/centernet_head.py#L87

    Hi! Thank you for your inspiring work!

    Currently I am doing some modification upon the codes and before that I have to understand them. I have run into the initialization code above. Why do you initialize bbox_pred.bias with 8? Is this some magic number?

    documentation 
    opened by greatwallet 1
  • why regression target need divide per level stride?

    why regression target need divide per level stride?

    📚 Documentation

    • Links to the relevant documentation/comment: reg_targets[l] = reg_targets[l] / float(self.strides[l])
    documentation 
    opened by 2anchao 1
  • AttributeError: 'CustomFastRCNNOutputs' object has no attribute '_predict_boxes'

    AttributeError: 'CustomFastRCNNOutputs' object has no attribute '_predict_boxes'

    Instructions To Reproduce the 🐛 Bug:

    1. Full runnable code or full changes you made:
    when i use CenterNet2_R2-101-DCN-BiFPN_4x+4x_1560_ST moudel, and make ROI_BOX_HEAD:BBOX_REG_LOSS_TYPE: "giou", it turns the below error:
    AttributeError: 'CustomFastRCNNOutputs' object has no attribute '_predict_boxes'
    
    1. What exact command you run: ${PYTHON} projects/CenterNet2/train_net.py --num-gpus 1 --config-file projects/CenterNet2/configs/CenterNet2_R2-101-DCN-BiFPN_4x+4x_1560_ST.yaml
    2. Full logs or other relevant observations:
    Traceback (most recent call last):
      File "projects/CenterNet2/train_net.py", line 232, in <module>
        args=(args,),
      File "/root/paddlejob/workspace/env_run/detectron2/engine/launch.py", line 62, in launch
        main_func(*args)
      File "projects/CenterNet2/train_net.py", line 213, in main
        do_train(cfg, model, resume=args.resume)
      File "projects/CenterNet2/train_net.py", line 125, in do_train
        loss_dict = model(data)
      File "/root/paddlejob/workspace/env_run/py36_detectron0.4/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
        return forward_call(*input, **kwargs)
      File "/root/paddlejob/workspace/env_run/detectron2/modeling/meta_arch/rcnn.py", line 166, in forward
        _, detector_losses = self.roi_heads(images, features, proposals, gt_instances)
      File "/root/paddlejob/workspace/env_run/py36_detectron0.4/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
        return forward_call(*input, **kwargs)
      File "/root/paddlejob/workspace/env_run/projects/CenterNet2/centernet/modeling/roi_heads/custom_roi_heads.py", line 166, in forward
        losses = self._forward_box(features, proposals, targets)
      File "/root/paddlejob/workspace/env_run/projects/CenterNet2/centernet/modeling/roi_heads/custom_roi_heads.py", line 128, in _forward_box
        stage_losses = predictor.losses(predictions, proposals)
      File "/root/paddlejob/workspace/env_run/projects/CenterNet2/centernet/modeling/roi_heads/custom_fast_rcnn.py", line 223, in losses
        self.freq_weight if use_advanced_loss else None, 
      File "/root/paddlejob/workspace/env_run/projects/CenterNet2/centernet/modeling/roi_heads/custom_fast_rcnn.py", line 159, in losses
        "loss_box_reg": self.box_reg_loss()
      File "/root/paddlejob/workspace/env_run/projects/CenterNet2/centernet/modeling/roi_heads/custom_fast_rcnn.py", line 142, in box_reg_loss
        self._predict_boxes()[fg_inds[:, None], gt_class_cols],
    AttributeError: 'CustomFastRCNNOutputs' object has no attribute '_predict_boxes'
    
    
    opened by secortot 1
  • Something interesting in your paper

    Something interesting in your paper

    📚 Documentation

    i noticed the formula between (2) and (3) in your original paper:

    log(P_bg) = log(P(bg | O_k = 1)P(O_k = 1) + P(O_k = 0))

    the assumption P(bg | O_k = 0) = 1 can be easily understand, but i wonder what's your understanding of $P(bg | O_k = 1)P(O_k = 1)$ in the rvalue of the formula shown above. here's my opinions:

    you use the law of full probability to decompose the entire process into 1st proposal stage and 2nd refine & classfication stage. P(O_k = 1) means the probability of foreground proposal prediction, while P(bg | O_k = 1) means given a foreground proposal but actually it belongs to background, the 2nd stage revise it to the background.

    is that understanding correct?

    /* following that, here's another question: how to model the P(bg | O_k = 1) in you network and how to prove you do not acutally model the P(bg | O_k = 0) instead? is that the assumption P(bg | O_k = 0) = 1 works? */

    i noticed something wrong with the question annotated above: the last linear layer (i.e. logistic/softmax regression) actually model the P(C_k | O_k) rather than specific condition O_k = 0 or O_k = 1, so the last question may not exist. backup for more people with the same misunderstanding.

    appreciate your reply, thanks!

    • Links to the relevant documentation/comment:
    documentation 
    opened by ShinoSpace 0
  • Error :AssertionError: Box tensor contains infinite or NaN!

    Error :AssertionError: Box tensor contains infinite or NaN!

    📚 Documentation

    I use my custom data to train CenterNet2, however i met this error after about 3hours:

    [08/09 22:12:04 d2.utils.events]: [eta: 1 day, 11:40:59 iter: 3660 total_loss: 1.185 loss_cls_stage0: 0.1144 loss_box_reg_stage0: 0.05557 loss_cls_stage1: 0.06713 loss_box_reg_stage1: 0.02371 loss_cls_stage2: 0.04383 loss_box_reg_stage2: 0.007958 loss_centernet_loc: 0.5499 loss_centernet_agn_pos: 0.3275 loss_centernet_agn_neg: 0.04878 time: 1.5286 data_time: 0.9600 lr: 0.018295 max_mem: 15129M [08/09 22:12:47 d2.utils.events]: [0m eta: 1 day, 11:42:35 iter: 3680 total_loss: 1.172 loss_cls_stage0: 0.09907 loss_box_reg_stage0: 0.05197 loss_cls_stage1: 0.06327 loss_box_reg_stage1: 0.02357 loss_cls_stage2: 0.0416 loss_box_reg_stage2: 0.008138 loss_centernet_loc: 0.538 loss_centernet_agn_pos: 0.3417 loss_centernet_agn_neg: 0.0396 time: 1.5287 data_time: 0.5869 lr: 0.018395 max_mem: 15129M [08/09 22:13:34 d2.utils.events]: [0m eta: 1 day, 11:41:24 iter: 3700 total_loss: 1.216 loss_cls_stage0: 0.1155 loss_box_reg_stage0: 0.05366 loss_cls_stage1: 0.06624 loss_box_reg_stage1: 0.02732 loss_cls_stage2: 0.04616 loss_box_reg_stage2: 0.01099 loss_centernet_loc: 0.5492 loss_centernet_agn_pos: 0.326 loss_centernet_agn_neg: 0.04868 time: 1.5288 data_time: 0.8384 lr: 0.018495 max_mem: 15129M [08/09 22:14:15 d2.utils.events]: [0m eta: 1 day, 11:40:55 iter: 3720 total_loss: 1.221 loss_cls_stage0: 0.113 loss_box_reg_stage0: 0.05246 loss_cls_stage1: 0.06204 loss_box_reg_stage1: 0.02323 loss_cls_stage2: 0.03913 loss_box_reg_stage2: 0.007532 loss_centernet_loc: 0.5236 loss_centernet_agn_pos: 0.3329 loss_centernet_agn_neg: 0.03891 time: 1.5287 data_time: 0.5345 lr: 0.018595 max_mem: 15129M No instances! torch.Size([0, 7]) torch.Size([0, 4]) 16 No instance in box reg loss No instances! torch.Size([0, 7]) torch.Size([0, 4]) 16 No instance in box reg loss Traceback (most recent call last): File "train_net.py", line 323, in args=(args,), File "/cache/user-job-dir/codes/detectron2/engine/launch.py", line 82, in launch main_func(*args) File "train_net.py", line 303, in main do_train(cfg, model, resume=args.resume) File "train_net.py", line 207, in do_train loss_dict = model(data) File "/home/work/anaconda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/cache/user-job-dir/codes/detectron2/modeling/meta_arch/rcnn.py", line 163, in forward _, detector_losses = self.roi_heads(images, features, proposals, gt_instances) File "/home/work/anaconda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/cache/user-job-dir/codes/centernet/modeling/roi_heads/custom_roi_heads.py", line 166, in forward losses = self._forward_box(features, proposals, targets) File "/cache/user-job-dir/codes/centernet/modeling/roi_heads/custom_roi_heads.py", line 116, in _forward_box proposals = self._create_proposals_from_boxes(prev_pred_boxes, image_sizes) File "/cache/user-job-dir/codes/detectron2/modeling/roi_heads/cascade_rcnn.py", line 290, in _create_proposals_from_boxes boxes_per_image.clip(image_size) File "/cache/user-job-dir/codes/detectron2/structures/boxes.py", line 200, in clip assert torch.isfinite(self.tensor).all(), "Box tensor contains infinite or NaN!" AssertionError: Box tensor contains infinite or NaN!`

    i have no idea why training diverged, since the loss is quite small.

    documentation 
    opened by pSGAme 1
  • window libtorch centernet2

    window libtorch centernet2

    🚀 Feature

    A clear and concise description of the feature proposal. C++(dla) C++(cascade)

    Motivation & Examples

    look https://github.com/ttanzhiqiang/Detectron2_CenterNetv2 Tell us why the feature is useful.

    Describe what the feature would look like, if it is implemented. Best demonstrated using code examples in addition to words. image

    enhancement 
    opened by ttanzhiqiang 0
Owner
Xingyi Zhou
CS Ph.D. student at UT Austin.
Xingyi Zhou
Code for Mining the Benefits of Two-stage and One-stage HOI Detection

Status: Archive (code is provided as-is, no updates expected) PPO-EWMA [Paper] This is code for training agents using PPO-EWMA and PPG-EWMA, introduce

OpenAI 21 Dec 24, 2021
Code for our NeurIPS 2021 paper Mining the Benefits of Two-stage and One-stage HOI Detection

CDN Code for our NeurIPS 2021 paper "Mining the Benefits of Two-stage and One-stage HOI Detection". Contributed by Aixi Zhang*, Yue Liao*, Si Liu, Mia

null 46 Jan 19, 2022
CenterNet:Objects as Points目标检测模型在Pytorch当中的实现

CenterNet:Objects as Points目标检测模型在Pytorch当中的实现

Bubbliiiing 165 Jan 20, 2022
Hybrid CenterNet - Hybrid-supervised object detection / Weakly semi-supervised object detection

Hybrid-Supervised Object Detection System Object detection system trained by hybrid-supervision/weakly semi-supervision (HSOD/WSSOD): This project is

null 2 Jan 9, 2022
Virtual Dance Reality Stage: a feature that offers you to share a stage with another user virtually

Portrait Segmentation using Tensorflow This script removes the background from an input image. You can read more about segmentation here Setup The scr

null 279 Jan 10, 2022
Two-Stage Peer-Regularized Feature Recombination for Arbitrary Image Style Transfer

Two-Stage Peer-Regularized Feature Recombination for Arbitrary Image Style Transfer Paper on arXiv Public PyTorch implementation of two-stage peer-reg

NNAISENSE 33 Jan 10, 2022
dualFace: Two-Stage Drawing Guidance for Freehand Portrait Sketching (CVMJ)

dualFace dualFace: Two-Stage Drawing Guidance for Freehand Portrait Sketching (CVMJ) We provide python implementations for our CVM 2021 paper "dualFac

Haoran XIE 35 Dec 21, 2021
A two-stage U-Net for high-fidelity denoising of historical recordings

A two-stage U-Net for high-fidelity denoising of historical recordings Official repository of the paper (not submitted yet): E. Moliner and V. Välimäk

Eloi Moliner Juanpere 12 Dec 9, 2021
PyTorch implementation of "A Two-Stage End-to-End System for Speech-in-Noise Hearing Aid Processing"

Implementation of the Sheffield entry for the first Clarity enhancement challenge (CEC1) This repository contains the PyTorch implementation of "A Two

null 3 Nov 1, 2021
The trained model and denoising example for paper : Cardiopulmonary Auscultation Enhancement with a Two-Stage Noise Cancellation Approach

The trained model and denoising example for paper : Cardiopulmonary Auscultation Enhancement with a Two-Stage Noise Cancellation Approach

ycj_project 1 Jan 18, 2022
Multi-Stage Progressive Image Restoration

Multi-Stage Progressive Image Restoration Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Sh

Syed Waqas Zamir 630 Jan 20, 2022
Code for one-stage adaptive set-based HOI detector AS-Net.

AS-Net Code for one-stage adaptive set-based HOI detector AS-Net. Mingfei Chen*, Yue Liao*, Si Liu, Zhiyuan Chen, Fei Wang, Chen Qian. "Reformulating

Mingfei Chen 39 Jan 18, 2022
(CVPR2021) DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation

DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation CVPR2021(oral) [arxiv] Requirements python3.7 pytorch==

W-zx-Y 54 Jan 17, 2022
[CVPR2021] Look before you leap: learning landmark features for one-stage visual grounding.

LBYL-Net This repo implements paper Look Before You Leap: Learning Landmark Features For One-Stage Visual Grounding CVPR 2021. Getting Started Prerequ

SVIP Lab 36 Dec 29, 2021
Optimizing DR with hard negatives and achieving SOTA first-stage retrieval performance on TREC DL Track (SIGIR 2021 Full Paper).

Optimizing Dense Retrieval Model Training with Hard Negatives Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Jiafeng Guo, Min Zhang, Shaoping Ma This repo provi

Jingtao Zhan 70 Jan 12, 2022
Single-Stage 6D Object Pose Estimation, CVPR 2020

Overview This repository contains the code for the paper Single-Stage 6D Object Pose Estimation. Yinlin Hu, Pascal Fua, Wei Wang and Mathieu Salzmann.

CVLAB @ EPFL 71 Jan 17, 2022
A Data Annotation Tool for Semantic Segmentation, Object Detection and Lane Line Detection.(In Development Stage)

Data-Annotation-Tool How to Run this Tool? To run this software, follow the steps: git clone https://github.com/Autonomous-Car-Project/Data-Annotation

TiVRA AI 8 Jan 7, 2022
TOOD: Task-aligned One-stage Object Detection, ICCV2021 Oral

One-stage object detection is commonly implemented by optimizing two sub-tasks: object classification and localization, using heads with two parallel branches, which might lead to a certain level of spatial misalignment in predictions between the two tasks.

null 202 Jan 11, 2022
Code for "Searching for Efficient Multi-Stage Vision Transformers"

Searching for Efficient Multi-Stage Vision Transformers This repository contains the official Pytorch implementation of "Searching for Efficient Multi

Yi-Lun Liao 51 Jan 12, 2022