A deep learning object detector framework written in Python for supporting Land Search and Rescue Missions.

Overview

AIR-logo

AIR: Aerial Inspection RetinaNet for supporting Land Search and Rescue Missions

DOI Docker Pulls

AIR is a deep learning based object detection solution to automate the aerial drone footage inspection task frequently carried out during search and rescue (SAR) operations with drone units. It provides a fast, convenient and reliable way to augment aerial, high-resolution image inspection for clues about human presence by highlighting relevant image regions with bounding boxes, as done in the image below. With the assistance of AIR, SAR missions with aerial drone searches can likely be carried out much faster before, and with considerably higher success rate.

This code repository is based on the master's thesis work by Pasi Pyrrö ORCID iD icon from Aalto University, School of Science with the title "AIR: Aerial Inspection RetinaNet for Land Search and Rescue Missions". This thesis work was funded by Accenture.

AIR-example

Key features

  • Supports both video and images
  • Highly accurate deep learning detector custom built for this application
  • Decent processing speed despite the high-resolution images and a very deep neural network used
  • Supports object tracking with kalman filter
  • Implements the novel MOB bounding box postprocessing algorithm
  • Implements the SAR-APD evaluation algorithm for real-world, value-oriented metric reporting in SAR aerial person detection applications
  • Plethora of options for customization (e.g., in bounding box postprocessing, image preprocessing, speed vs accuracy trade-offs, etc.)

Results on HERIDAL test set

In the table below are listed the results of the AIR detector and other notable state-of-the-art methods on the HERIDAL test set. Note that the FPS metric is a rough estimate based on the reported inference speed and hardware setup by the original authors, and should be taken as mostly directional. We did not test any of the other methods ourselves.

Method Precision Recall AP FPS
Mean shift segmentation method [1] 18.7 74.7 - -
Saliency guided VGG16 [2] 34.8 88.9 - -
Faster R-CNN [3] 67.3 88.3 86.1 1
Two-stage multimodel CNN [4] 68.9 94.7 - 0.1
SSD [4] 4.3 94.4 - -
AIR with NMS (ours) 90.1 86.1 84.6 1

It turns out AIR achieves state-of-the-art results in precison and inference speed while having comparable recall to the strongest competitors!

You can check out the full details of AIR evaluation in this Wandb report.

Bounding Box Aggregation options

NMS-vs-MOB

AIR implements both NMS and MOB algorithms for bounding box prediction postprocessing. The image above shows the main difference: MOB (c) merges bounding boxes (a) instead of eliminating them like NMS (b). Thus, choosing MOB can produce visually more pleasing predictions. Moreover, MOB is less sensitive to the choice of condifence score threshold, making it more robust under unseen data. AIR also comes with a custom SAR-APD evaluation scheme that truthfully ranks MOB-equipped object detector performance (as standard object detection metrics, such as VOC2012 AP, do not like MOB very much).

Should you choose MOB postprocessing and SAR-APD evaluation, you should see similar evaluation results (e.g., after running bash evaluate.sh)

NMS-vs-MOB

All metrics increased by over 4 percentage points from NMS & VOC2012 results with no loss of prediction visual quality, neat!

Hardware Requirements

  • x86 or x64 processor architecture (at least for the Docker installation, and if trying to install natively on ARM64 architecture, expect a lot of compiling python packages from source)
  • NVIDIA® GPU with CUDA® capability 3.5 or higher (recommended)
  • ~8 GB of RAM or more

Software Requirements

If using containers:

If using native installation:

  • Python 3.6
  • pip and setuptools
  • FFmpeg (recommended)
  • Linux or Mac OS (not tested on Windows)

Quick install instructions using Docker

  • Clone this repo
  • Either start docker CPU environment by running: bash start-air-cpu-env.sh docker
  • Or start docker GPU environment by running: bash start-air-gpu-env.sh docker
  • Inside docker container, build the AIR project by running: python setup.py build_ext --inplace

Quick install instructions using Singularity

  • Clone this repo
  • You might wanna specify your SINGULARITY_CACHEDIR env variable prior to installation, if so edit the shell scripts used in the next steps
  • Either start docker CPU environment by running: bash start-air-cpu-env.sh singularity
  • Or start docker GPU environment by running: bash start-air-gpu-env.sh singularity
  • Go to mounted directory by typing cd AIR
  • Build the AIR project by running: python setup.py build_ext --inplace

Quick native install instructions

  • Clone this repo
  • To build the AIR project for CPU, run the command: /usr/bin/python3 -m pip install air-detector[cpu]
  • To build the AIR project for GPU, run the command: /usr/bin/python3 -m pip install air-detector[gpu]

Download data and the trained model

Quick demos

Once everything is set up (installation and asset downloads), you might wanna try out these cool and simple demos to get the hang of using the AIR detector.

Running inference on HERIDAL test image folder

  • In the project root, run the command: /bin/bash infer.sh
  • Check data/predictions/dauntless-sweep-2_resnet152_pascal-enclose-inference/ folder for the output images

Evaluating AIR performance on HERIDAL test set (can be slow on CPU)

  • In the project root, run the command: /bin/bash evaluate.sh
  • Check data/predictions/dauntless-sweep-2_resnet152_pascal-enclose-sar-apd-eval/ folder for the output images

SAR-APD video detection with image output

  • In the project root, run the command: /bin/bash convert-model.sh
  • Then start the inferencing by running: /usr/bin/python3 video_detect.py -c mob_cpu_images
  • Check data/videos/Ylojarvi-gridiajo-two-guys-moving_air_output/ folder for the output images

SAR-APD video detection with video output

  • In the project root, run the command: /bin/bash convert-model.sh
  • Then start the inferencing by running: /usr/bin/python3 video_detect.py -c mob_gpu_tracking
  • Check data/videos/Ylojarvi-gridiajo-two-guys-moving_air_output_compressed.mov output video

Wandb support

  • AIR uses experiment tracking software by Weight & Biases
  • Wandb enables you to view the whole public experimentation record for AIR
  • AIR experiments can be controlled with the following environment variables:
    • WANDB_MODE="disabled"
      • disables all wandb functionality, required if not logged in to wandb
    • WANDB_MODE="dryrun"
      • wandb logging works as normal, but nothing is uploaded to cloud
    • WANDB_API_KEY=<your_api_key>
      • needed to interact with wandb web UI, you might want to put this in your ~/.bashrc or ~/.zshrc so that it's automatically included into the Docker envs. On most linux based systems, you can achieve this by running this shell command with your Wandb API key:
        echo "export WANDB_API_KEY=<your_api_key>" >> "~/.${SHELL/\/bin\//}rc"; exec $SHELL
    • WANDB_DIR=~/wandb
      • select where local log files are stored
    • WANDB_PROJECT="air-testing"
      • select your wandb cloud project
    • WANDB_ENTITY="ML-Mike"
      • select your wandb account username
    • WANDB_RUN_GROUP="cool-experiments"
      • Semantic group within a project to log experiments runs to
  • More info in wandb documentation (not all env variables are supported though)

Project folder structure

  • airutils: air utility functions that augment the standard keras-retinanet framework to incorporate aerial person detection (APD) support to it.
  • config: configuration files for video_detect.py script (easier to use than the CLI in most cases)
  • data: input and output data for the AIR detector, divided into subfolders:
    • /datasets (input for training and evaluation)
    • /images (input for general inference)
    • /predictions (output of the AIR detector)
    • /videos (input for video_detect.py)
  • keras-retinanet: a fork of the keras-retinanet repository with some AIR modifications
  • models: contains trained object detection models (both trainable and inference models)

General toubleshooting

  • Try setting the AIR_VERBOSE=1 enviroment variable to see full TF logs
  • If using docker, make sure you're allocating enough memory for the container (like at least 8 GB)
  • If some script simply gets "killed", it's a clear indicator that you have too little memory allocated
  • Tracker options in video_detect.py might need to be recalibrated for each use case for best performance
  • If you edit any of the bash scripts in the root folder (e.g., evaluate.sh), make sure there are no whitespace after linebreaks '\', bash can be picky about these things... Also avoid commenting out any command line parameters in those scripts, just delete the whole line outright
  • The actual python scripts corresponding to the bash scripts are in keras_retinanet/keras_retinanet/bin folder and they can be called directly (or you can try the installed console scripts (e.g., air-evaluate) if you ran pip install .) with approproate parameters (examples can be found in those bash scripts)
  • Using the command line parameter -h or --help usually helps (pun intended)
  • If you find a bug or have a complicated question, open a new issue here

Acknowledgements

  • Kudos to Aalto Science-IT project for providing the compute hardware for training and testing the AIR detector
  • Big thanks to contributors of keras-retinanet, and their marvellous programming efforts that have allowed AIR to exists in the first place!
  • Huge credit to the authors of the HERIDAL dataset [2], for which AIR equally well owes its very existence
  • Also thanks to Ari Nissinen from Finnish Voluntary Rescue Service (VAPEPA) for providing video material used in development and testing of the AIR detector
  • Lastly, thanks to Hassan Naseri from Accenture for instructing with the thesis work and providing valuable insights throughout the whole AIR project

How to cite

@MastersThesis{pyrro2021air, 
	title={{AIR:} {Aerial} Inspection RetinaNet for Land Search and Rescue Missions}, 
	author={Pyrr{\"o}, Pasi and Naseri, Hassan and Jung, Alexander},
	school={Aalto University, School of Science},
	year={2021}
}

Licenses

References

[1] TURIC, H., DUJMIC, H., AND PAPIC, V. Two-stage segmentation of aerial images for search and rescue. Information Technology and Control 39, 2 (2010).

[2] BOŽIC-ŠTULIC, D., MARUŠIC, Ž., AND GOTOVAC, S. Deep learning approach in aerial imagery for supporting land search and rescue missions. International Journal of Computer Vision 127, 9 (2019), 1256–1278.

[3] MARUŠIC, Ž., BOŽIC-ŠTULIC, D., GOTOVAC, S., AND MARUŠIC, T. Region proposal approach for human detection on aerial imagery. In 2018 3rd International Conference on Smart and Sustainable Technologies (SpliTech) (2018), IEEE, pp. 1–6.

[4] VASIC, M. K., AND PAPIC, V. Multimodel deep learning for person detection in aerial images. Electronics 9, 9 (2020), 1459.

Comments
  • cudnn failed to initialyze

    cudnn failed to initialyze

    Hello i tried to install the docker-gpu images, i've download the weights and the data, but when i launch /bin/bash infer.sh i get this response :

    [email protected]:/home# /bin/bash infer.sh 
    Using TensorFlow backend.
    2022-03-01 23:46:15.554674: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
    2022-03-01 23:46:15.578708: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2499950000 Hz
    2022-03-01 23:46:15.579340: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x561a790 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
    2022-03-01 23:46:15.579364: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
    2022-03-01 23:46:15.580946: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
    2022-03-01 23:46:15.636357: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2022-03-01 23:46:15.636553: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x53a4460 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
    2022-03-01 23:46:15.636569: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce GTX 1650 Ti with Max-Q Design, Compute Capability 7.5
    2022-03-01 23:46:15.636762: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2022-03-01 23:46:15.636888: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
    name: NVIDIA GeForce GTX 1650 Ti with Max-Q Design major: 7 minor: 5 memoryClockRate(GHz): 1.2
    pciBusID: 0000:01:00.0
    2022-03-01 23:46:15.637043: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
    2022-03-01 23:46:15.637856: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
    2022-03-01 23:46:15.638563: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
    2022-03-01 23:46:15.638766: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
    2022-03-01 23:46:15.639774: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
    2022-03-01 23:46:15.640478: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
    2022-03-01 23:46:15.642728: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
    2022-03-01 23:46:15.642815: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2022-03-01 23:46:15.642967: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2022-03-01 23:46:15.643040: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
    2022-03-01 23:46:15.643072: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
    2022-03-01 23:46:15.643622: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
    2022-03-01 23:46:15.643634: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0 
    2022-03-01 23:46:15.643639: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N 
    2022-03-01 23:46:15.643714: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2022-03-01 23:46:15.643833: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2022-03-01 23:46:15.643934: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2857 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce GTX 1650 Ti with Max-Q Design, pci bus id: 0000:01:00.0, compute capability: 7.5)
    Loading model, this may take a second...
    tracking <tf.Variable 'Variable:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_1:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_2:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_3:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_4:0' shape=(15, 4) dtype=float32> anchors
    Running inference on image folder: /home/data/images/test
    Running network: N/A% (0 of 4) |                                                                                                                                     | Elapsed Time: 0:00:00 ETA:  --:--:--2022-03-01 23:46:40.231228: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
    2022-03-01 23:46:40.548887: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
    2022-03-01 23:46:40.554431: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
    Traceback (most recent call last):
      File "keras_retinanet/keras_retinanet/bin/infer.py", line 215, in <module>
        main()
      File "keras_retinanet/keras_retinanet/bin/infer.py", line 192, in main
        profile=args.profile
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 250, in get_detections
        max_inflation_factor=max_inflation_factor
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 134, in run_inference_on_image
        boxes, scores, labels = model.predict_on_batch(np.expand_dims(image, axis=0))[:3]
      File "/usr/local/lib/python3.6/dist-packages/keras/engine/training.py", line 1580, in predict_on_batch
        outputs = self.predict_function(ins)
      File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/backend.py", line 3476, in __call__
        run_metadata=self.run_metadata)
      File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py", line 1472, in __call__
        run_metadata_ptr)
    tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
      (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
    	 [[{{node conv1/convolution}}]]
    	 [[filtered_detections/map/while/LoopCond/_9551]]
      (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
    	 [[{{node conv1/convolution}}]]
    0 successful operations.
    0 derived errors ignored.
    
    

    inside the container I checked under the cuda repository and noticed that the cudnn librairies where not inside , so i downloaded it and copy it inside cuda/include and cuda/lib64 but i've got the same result. I am on ubuntu 18.04 i have a GeForce GTX 1650 , nvidia-smi return cuda 11.6 nvida-driver 510 Can you help me on that ? Thank you

    help wanted 
    opened by mrba59 7
  • Box detect group of person

    Box detect group of person

    Hi, I've runned inference on my eval dataset, the results are really good and i congrat you on that it's really impressive. I just have one question. The inference detect 15 person on this image , as you can see some box contains more that one person but they are detected as one, maybe there is a parameter taking account group of person ? Is there any chance to return one box for each person ?

    frame22401-idx-7

    opened by mrba59 3
  • Dataset unavailable

    Dataset unavailable

    Hi, nice work! Unfortunately i can't reach out the HERIDAL download page. If i try from here: http://ipsar.fesb.unist.hr/HERIDAL%20database.html It prompts an error.

    Is there any chance you have an offline dump you can share with me somehow?

    Thanks

    opened by lodm94 1
  • ModuleNotFoundError: No module named 'keras.api'

    ModuleNotFoundError: No module named 'keras.api'

    When I ran !/bin/bash convert-model.sh in Google Colab, it resulted in:

    2022-03-22 09:32:42.586979: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
    Traceback (most recent call last):
      File "keras_retinanet/keras_retinanet/bin/convert_model.py", line 103, in <module>
        main()
      File "keras_retinanet/keras_retinanet/bin/convert_model.py", line 79, in main
        model = models.load_model(args.model_in, backbone_name=args.backbone)
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/models/__init__.py", line 87, in load_model
        return keras.models.load_model(filepath, custom_objects=backbone(backbone_name).custom_objects, compile=compile)
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/models/__init__.py", line 57, in backbone
        from .resnet import ResNetBackbone as b
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/models/resnet.py", line 18, in <module>
        from keras.utils import get_file
    ImportError: cannot import name 'get_file' from 'keras.utils' (/usr/local/lib/python3.7/dist-packages/keras/utils/__init__.py)
    

    After some googling I found out that I should install kashgari, so I ran

    pip install kashgari

    And when I came back to !/bin/bash convert-model.sh, I had:

    Using TensorFlow backend.
    Traceback (most recent call last):
      File "keras_retinanet/keras_retinanet/bin/convert_model.py", line 33, in <module>
        from ..utils.config import read_config_file, parse_anchor_parameters
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/config.py", line 19, in <module>
        import keras
      File "/usr/local/lib/python3.7/dist-packages/keras/__init__.py", line 3, in <module>
        from . import utils
      File "/usr/local/lib/python3.7/dist-packages/keras/utils/__init__.py", line 26, in <module>
        from .vis_utils import model_to_dot
      File "/usr/local/lib/python3.7/dist-packages/keras/utils/vis_utils.py", line 7, in <module>
        from ..models import Model
      File "/usr/local/lib/python3.7/dist-packages/keras/models.py", line 12, in <module>
        from .engine.training import Model
      File "/usr/local/lib/python3.7/dist-packages/keras/engine/__init__.py", line 8, in <module>
        from .training import Model
      File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 14, in <module>
        from . import training_utils
      File "/usr/local/lib/python3.7/dist-packages/keras/engine/training_utils.py", line 17, in <module>
        from .. import metrics as metrics_module
      File "/usr/local/lib/python3.7/dist-packages/keras/metrics.py", line 1850, in <module>
        BaseMeanIoU = tf.keras.metrics.MeanIoU
      File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/lazy_loader.py", line 58, in __getattr__
        module = self._load()
      File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/lazy_loader.py", line 41, in _load
        module = importlib.import_module(self.__name__)
      File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
    ModuleNotFoundError: No module named 'keras.api'
    

    Am I doing something wrong? Is there a way to address this error?

    help wanted 
    opened by dinarakhay 1
  • Bump opencv-python from 4.1.0.25 to 4.2.0.32

    Bump opencv-python from 4.1.0.25 to 4.2.0.32

    ⚠️ Dependabot is rebasing this PR ⚠️

    Rebasing might not happen immediately, so don't worry if this takes some time.

    Note: if you make any changes to this PR yourself, they will take precedence over the rebase.


    Bumps opencv-python from 4.1.0.25 to 4.2.0.32.

    Release notes

    Sourced from opencv-python's releases.

    4.2.0.32

    OpenCV version 4.2.0.

    Changes:

    • macOS environment updated from xcode8.3 to xcode 9.4
    • macOS uses now Qt 5 instead of Qt 4
    • Nasm version updated to Docker containers
    • multibuild updated

    Fixes:

    • don't use deprecated brew tap-pin, instead refer to the full package name when installing #267
    • replace get_config_var() with get_config_vars() in setup.py #274
    • add workaround for DLL errors in Windows Server #264

    4.1.2.30

    OpenCV version 4.1.2.

    Changes:

    • Python 3.8 builds added to the build matrix
    • Support for Python 3.4 builds dropped (Python 3.4 is in EOL)
    • multibuild updated
    • minor build logic changes
    • Docker images rebuilt

    Notes:

    Please note that Python 2.7 enters into EOL phase in January 2020. opencv-python Python 2.7 wheels won't be provided after that.

    4.1.1.26

    OpenCV version 4.1.1.

    Changes:

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump pillow from 6.2.1 to 8.3.2

    Bump pillow from 6.2.1 to 8.3.2

    ⚠️ Dependabot is rebasing this PR ⚠️

    Rebasing might not happen immediately, so don't worry if this takes some time.

    Note: if you make any changes to this PR yourself, they will take precedence over the rebase.


    Bumps pillow from 6.2.1 to 8.3.2.

    Release notes

    Sourced from pillow's releases.

    8.3.2

    https://pillow.readthedocs.io/en/stable/releasenotes/8.3.2.html

    Security

    • CVE-2021-23437 Raise ValueError if color specifier is too long [hugovk, radarhere]

    • Fix 6-byte OOB read in FliDecode [wiredfool]

    Python 3.10 wheels

    • Add support for Python 3.10 #5569, #5570 [hugovk, radarhere]

    Fixed regressions

    • Ensure TIFF RowsPerStrip is multiple of 8 for JPEG compression #5588 [kmilos, radarhere]

    • Updates for ImagePalette channel order #5599 [radarhere]

    • Hide FriBiDi shim symbols to avoid conflict with real FriBiDi library #5651 [nulano]

    8.3.1

    https://pillow.readthedocs.io/en/stable/releasenotes/8.3.1.html

    Changes

    8.3.0

    https://pillow.readthedocs.io/en/stable/releasenotes/8.3.0.html

    Changes

    ... (truncated)

    Changelog

    Sourced from pillow's changelog.

    8.3.2 (2021-09-02)

    • CVE-2021-23437 Raise ValueError if color specifier is too long [hugovk, radarhere]

    • Fix 6-byte OOB read in FliDecode [wiredfool]

    • Add support for Python 3.10 #5569, #5570 [hugovk, radarhere]

    • Ensure TIFF RowsPerStrip is multiple of 8 for JPEG compression #5588 [kmilos, radarhere]

    • Updates for ImagePalette channel order #5599 [radarhere]

    • Hide FriBiDi shim symbols to avoid conflict with real FriBiDi library #5651 [nulano]

    8.3.1 (2021-07-06)

    • Catch OSError when checking if fp is sys.stdout #5585 [radarhere]

    • Handle removing orientation from alternate types of EXIF data #5584 [radarhere]

    • Make Image.array take optional dtype argument #5572 [t-vi, radarhere]

    8.3.0 (2021-07-01)

    • Use snprintf instead of sprintf. CVE-2021-34552 #5567 [radarhere]

    • Limit TIFF strip size when saving with LibTIFF #5514 [kmilos]

    • Allow ICNS save on all operating systems #4526 [baletu, radarhere, newpanjing, hugovk]

    • De-zigzag JPEG's DQT when loading; deprecate convert_dict_qtables #4989 [gofr, radarhere]

    • Replaced xml.etree.ElementTree #5565 [radarhere]

    ... (truncated)

    Commits
    • 8013f13 8.3.2 version bump
    • 23c7ca8 Update CHANGES.rst
    • 8450366 Update release notes
    • a0afe89 Update test case
    • 9e08eb8 Raise ValueError if color specifier is too long
    • bd5cf7d FLI tests for Oss-fuzz crash.
    • 94a0cf1 Fix 6-byte OOB read in FliDecode
    • cece64f Add 8.3.2 (2021-09-02) [CI skip]
    • e422386 Add release notes for Pillow 8.3.2
    • 08dcbb8 Pillow 8.3.2 supports Python 3.10 [ci skip]
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump opencv-contrib-python from 4.1.0.25 to 4.2.0.32

    Bump opencv-contrib-python from 4.1.0.25 to 4.2.0.32

    ⚠️ Dependabot is rebasing this PR ⚠️

    Rebasing might not happen immediately, so don't worry if this takes some time.

    Note: if you make any changes to this PR yourself, they will take precedence over the rebase.


    Bumps opencv-contrib-python from 4.1.0.25 to 4.2.0.32.

    Release notes

    Sourced from opencv-contrib-python's releases.

    4.2.0.32

    OpenCV version 4.2.0.

    Changes:

    • macOS environment updated from xcode8.3 to xcode 9.4
    • macOS uses now Qt 5 instead of Qt 4
    • Nasm version updated to Docker containers
    • multibuild updated

    Fixes:

    • don't use deprecated brew tap-pin, instead refer to the full package name when installing #267
    • replace get_config_var() with get_config_vars() in setup.py #274
    • add workaround for DLL errors in Windows Server #264

    4.1.2.30

    OpenCV version 4.1.2.

    Changes:

    • Python 3.8 builds added to the build matrix
    • Support for Python 3.4 builds dropped (Python 3.4 is in EOL)
    • multibuild updated
    • minor build logic changes
    • Docker images rebuilt

    Notes:

    Please note that Python 2.7 enters into EOL phase in January 2020. opencv-python Python 2.7 wheels won't be provided after that.

    4.1.1.26

    OpenCV version 4.1.1.

    Changes:

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Unable to get working with GPU: OOM and/or hangs

    Unable to get working with GPU: OOM and/or hangs

    First of all, thanks for putting this together! I'm trying to integrate it into my volunteer search & rescue team's RPAS/drone workflow, and we're all very excited about the initial results.

    While the results under CPU mode are great, they are a bit slow for our purposes (using 4000x2250px images). Our team computer has a 3060 GPU (Windows 11, i7 CPU, 32GB RAM), which we hope can speed up processing, but we're unable to get it working in GPU mode.

    When running the default GPU Docker install, we get OOM errors:

    [email protected]:/home# /bin/bash infer.sh
    Using TensorFlow backend.
    2022-06-16 02:32:07.062753: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
    2022-06-16 02:32:07.068259: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2687995000 Hz
    2022-06-16 02:32:07.069773: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x57d5a40 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
    2022-06-16 02:32:07.069800: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
    2022-06-16 02:32:07.072249: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
    2022-06-16 02:32:08.733246: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:32:08.733462: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x570f500 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
    2022-06-16 02:32:08.733513: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 3060 Laptop GPU, Compute Capability 8.6
    2022-06-16 02:32:08.738143: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:32:08.738184: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
    name: NVIDIA GeForce RTX 3060 Laptop GPU major: 8 minor: 6 memoryClockRate(GHz): 1.702
    pciBusID: 0000:01:00.0
    2022-06-16 02:32:08.738376: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
    2022-06-16 02:32:08.739257: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
    2022-06-16 02:32:08.740589: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
    2022-06-16 02:32:08.740821: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
    2022-06-16 02:32:08.742371: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
    2022-06-16 02:32:08.743383: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
    2022-06-16 02:32:08.745999: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
    2022-06-16 02:32:08.746592: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:32:08.747027: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:32:08.747050: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
    2022-06-16 02:32:08.747094: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
    2022-06-16 02:32:08.747495: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
    2022-06-16 02:32:08.747514: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0
    2022-06-16 02:32:08.747532: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N
    2022-06-16 02:32:08.748081: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:32:08.748109: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1387] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
    2022-06-16 02:32:08.748516: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:32:08.748577: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4857 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3060 Laptop GPU, pci bus id: 0000:01:00.0, compute capability: 8.6)
    Loading model, this may take a second...
    2022-06-16 02:36:14.483391: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 1073741824 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:14.483439: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 1073741824
    2022-06-16 02:36:14.576914: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 966367744 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:14.576962: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 966367744
    2022-06-16 02:36:14.671003: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 869731072 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:14.671047: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 869731072
    2022-06-16 02:36:14.770382: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 782758144 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:14.770424: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 782758144
    2022-06-16 02:36:14.864264: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 704482304 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:14.864312: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 704482304
    2022-06-16 02:36:14.957833: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 634034176 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:14.957889: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 634034176
    2022-06-16 02:36:15.053452: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 570630912 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.053498: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 570630912
    2022-06-16 02:36:15.150759: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 513568000 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.150802: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 513568000
    2022-06-16 02:36:15.243685: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 462211328 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.243726: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 462211328
    2022-06-16 02:36:15.335976: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 415990272 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.336028: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 415990272
    2022-06-16 02:36:15.427373: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 374391296 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.427417: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 374391296
    2022-06-16 02:36:15.520526: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 336952320 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.520570: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 336952320
    2022-06-16 02:36:15.612661: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 303257088 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.612720: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 303257088
    2022-06-16 02:36:15.704694: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 272931584 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.704741: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 272931584
    2022-06-16 02:36:15.798110: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 245638656 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.798153: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 245638656
    2022-06-16 02:36:15.889580: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 221074944 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.889628: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 221074944
    2022-06-16 02:36:15.984319: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 198967552 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:15.984362: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 198967552
    2022-06-16 02:36:16.077288: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 179070976 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:16.077332: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 179070976
    2022-06-16 02:36:16.169940: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 161164032 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:16.169981: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 161164032
    2022-06-16 02:36:16.264281: E tensorflow/stream_executor/cuda/cuda_driver.cc:893] failed to alloc 145047808 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
    2022-06-16 02:36:16.264328: W ./tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 145047808
    tracking <tf.Variable 'Variable:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_1:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_2:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_3:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_4:0' shape=(15, 4) dtype=float32> anchors
    Running inference on image folder: /home/data/images/test
    Running network: N/A% (0 of 138) |                                          | Elapsed Time: 0:00:00 ETA:  --:--:--2022-06-16 02:36:21.094051: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
    2022-06-16 02:48:27.323880: W tensorflow/stream_executor/cuda/redzone_allocator.cc:312] Internal: ptxas exited with non-zero error code 65280, output: ptxas fatal   : Value 'sm_86' is not defined for option 'gpu-name'
    
    Relying on driver to perform ptx compilation. This message will be only logged once.
    2022-06-16 02:48:27.773454: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
    2022-06-16 02:49:29.518638: W tensorflow/core/common_runtime/bfc_allocator.cc:305] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature.
    Traceback (most recent call last):
      File "keras_retinanet/keras_retinanet/bin/infer.py", line 215, in <module>
        main()
      File "keras_retinanet/keras_retinanet/bin/infer.py", line 192, in main
        profile=args.profile
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 250, in get_detections
        max_inflation_factor=max_inflation_factor
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 168, in run_inference_on_image
        max_inflation_factor=max_inflation_factor)
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/../../../airutils/mob.py", line 365, in merge_boxes_per_label
        iou_threshold, max_iterations, top_k, max_inflation_factor, merge_mode)
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/../../../airutils/mob.py", line 276, in merge_overlapping_boxes
        X = 1. - compute_overlap(boxes, boxes)
      File "compute_overlap.pyx", line 27, in compute_overlap.compute_overlap
        cdef np.ndarray[double, ndim=2] overlaps = np.zeros((N, K), dtype=np.float64)
    MemoryError: Unable to allocate array with shape (100000, 100000) and data type float64
    .....
    

    I've tried some extreme configuration options to lessen the memory burden, but we still get the OOM errors:

        --image_min_side 100 
        --image_max_side 150 
        --image_tiling_dim 8 
    

    I found that by setting compile=False in the below line of keras_retinanet\keras_retinanet\bin\infer.py, I avoid getting the Cuda OOM errors, but the process just hangs and eventually errors out the same. model = models.load_model(args.model, backbone_name=args.backbone, compile=False)

    Complete output with compile=False:

    [email protected]:/home# /bin/bash infer.sh
    Using TensorFlow backend.
    2022-06-16 02:09:13.780103: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
    2022-06-16 02:09:13.786143: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2687995000 Hz
    2022-06-16 02:09:13.787804: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5b70d30 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
    2022-06-16 02:09:13.787838: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
    2022-06-16 02:09:13.790658: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
    2022-06-16 02:09:15.386150: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:09:15.386309: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x58fbfd0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
    2022-06-16 02:09:15.386346: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 3060 Laptop GPU, Compute Capability 8.6
    2022-06-16 02:09:15.389332: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:09:15.389373: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
    name: NVIDIA GeForce RTX 3060 Laptop GPU major: 8 minor: 6 memoryClockRate(GHz): 1.702
    pciBusID: 0000:01:00.0
    2022-06-16 02:09:15.389623: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
    2022-06-16 02:09:15.390896: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
    2022-06-16 02:09:15.392376: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
    2022-06-16 02:09:15.392657: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
    2022-06-16 02:09:15.395000: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
    2022-06-16 02:09:15.396610: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
    2022-06-16 02:09:15.399907: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
    2022-06-16 02:09:15.400535: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:09:15.401007: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:09:15.401035: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
    2022-06-16 02:09:15.401082: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
    2022-06-16 02:09:15.401523: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
    2022-06-16 02:09:15.401542: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0
    2022-06-16 02:09:15.401581: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N
    2022-06-16 02:09:15.402103: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:09:15.402132: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1387] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
    2022-06-16 02:09:15.402663: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:969] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
    Your kernel may have been built without NUMA support.
    2022-06-16 02:09:15.402747: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4857 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3060 Laptop GPU, pci bus id: 0000:01:00.0, compute capability: 8.6)
    Loading model, this may take a second...
    tracking <tf.Variable 'Variable:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_1:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_2:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_3:0' shape=(15, 4) dtype=float32> anchors
    tracking <tf.Variable 'Variable_4:0' shape=(15, 4) dtype=float32> anchors
    Running inference on image folder: /home/data/images/test
    Running network: N/A% (0 of 138) |                                               | Elapsed Time: 0:00:00 ETA:  --:--:--2022-06-16 02:13:11.539626: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
    2022-06-16 02:25:21.996183: W tensorflow/stream_executor/cuda/redzone_allocator.cc:312] Internal: ptxas exited with non-zero error code 65280, output: ptxas fatal   : Value 'sm_86' is not defined for option 'gpu-name'
    
    Relying on driver to perform ptx compilation. This message will be only logged once.
    2022-06-16 02:25:22.378005: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
    2022-06-16 02:26:24.908769: W tensorflow/core/common_runtime/bfc_allocator.cc:305] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you'd like to disable this feature.
    Traceback (most recent call last):
      File "keras_retinanet/keras_retinanet/bin/infer.py", line 216, in <module>
      File "keras_retinanet/keras_retinanet/bin/infer.py", line 193, in main
        )
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 250, in get_detections
        max_inflation_factor=max_inflation_factor
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/eval.py", line 168, in run_inference_on_image
        max_inflation_factor=max_inflation_factor)
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/../../../airutils/mob.py", line 365, in merge_boxes_per_label
        iou_threshold, max_iterations, top_k, max_inflation_factor, merge_mode)
      File "keras_retinanet/keras_retinanet/bin/../../keras_retinanet/utils/../../../airutils/mob.py", line 276, in merge_overlapping_boxes
        X = 1. - compute_overlap(boxes, boxes)
      File "compute_overlap.pyx", line 27, in compute_overlap.compute_overlap
        cdef np.ndarray[double, ndim=2] overlaps = np.zeros((N, K), dtype=np.float64)
    MemoryError: Unable to allocate array with shape (100000, 100000) and data type float64
    

    In case it helps:

    [email protected]:/home# nvidia-smi
    Thu Jun 16 02:29:41 2022
    +-----------------------------------------------------------------------------+
    | NVIDIA-SMI 510.73.08    Driver Version: 512.96       CUDA Version: 11.6     |
    |-------------------------------+----------------------+----------------------+
    | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
    |                               |                      |               MIG M. |
    |===============================+======================+======================|
    |   0  NVIDIA GeForce ...  On   | 00000000:01:00.0 Off |                  N/A |
    | N/A   47C    P0    24W /  N/A |      0MiB /  6144MiB |      0%      Default |
    |                               |                      |                  N/A |
    +-------------------------------+----------------------+----------------------+
    
    +-----------------------------------------------------------------------------+
    | Processes:                                                                  |
    |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
    |        ID   ID                                                   Usage      |
    |=============================================================================|
    |  No running processes found                                                 |
    +-----------------------------------------------------------------------------+
    
    

    Any suggestions would be very much appreciated, as I think I'm fully out of ideas and we're really keen to make this work!

    opened by duncanenman 2
Releases(0.0.2)
  • 0.0.2(Jan 14, 2022)

  • 0.0.1(Nov 9, 2021)

Owner
Accenture
Accenture Github site
Accenture
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search

BossNAS This repository contains PyTorch evaluation code, retraining code and pretrained models of our paper: BossNAS: Exploring Hybrid CNN-transforme

Changlin Li 127 Dec 26, 2022
Learned Initializations for Optimizing Coordinate-Based Neural Representations

Learned Initializations for Optimizing Coordinate-Based Neural Representations Project Page | Paper Matthew Tancik*1, Ben Mildenhall*1, Terrance Wang1

Matthew Tancik 127 Jan 03, 2023
A PyTorch re-implementation of the paper 'Exploring Simple Siamese Representation Learning'. Reproduced the 67.8% Top1 Acc on ImageNet.

Exploring simple siamese representation learning This is a PyTorch re-implementation of the SimSiam paper on ImageNet dataset. The results match that

Taojiannan Yang 72 Nov 09, 2022
YOLO-v5 기반 단안 카메라의 영상을 활용해 차간 거리를 일정하게 유지하며 주행하는 Adaptive Cruise Control 기능 구현

자율 주행차의 영상 기반 차간거리 유지 개발 Table of Contents 프로젝트 소개 주요 기능 시스템 구조 디렉토리 구조 결과 실행 방법 참조 팀원 프로젝트 소개 YOLO-v5 기반으로 단안 카메라의 영상을 활용해 차간 거리를 일정하게 유지하며 주행하는 Adap

14 Jun 29, 2022
[ICLR 2021] "CPT: Efficient Deep Neural Network Training via Cyclic Precision" by Yonggan Fu, Han Guo, Meng Li, Xin Yang, Yining Ding, Vikas Chandra, Yingyan Lin

CPT: Efficient Deep Neural Network Training via Cyclic Precision Yonggan Fu, Han Guo, Meng Li, Xin Yang, Yining Ding, Vikas Chandra, Yingyan Lin Accep

26 Oct 25, 2022
PiRank: Learning to Rank via Differentiable Sorting

PiRank: Learning to Rank via Differentiable Sorting This repository provides a reference implementation for learning PiRank-based models as described

54 Dec 17, 2022
Automatically erase objects in the video, such as logo, text, etc.

Video-Auto-Wipe Read English Introduction:Here   本人不定期的基于生成技术制作一些好玩有趣的算法模型,这次带来的作品是“视频擦除”方向的应用模型,它实现的功能是自动感知到视频中我们不想看见的部分(譬如广告、水印、字幕、图标等等)然后进行擦除。由于图标擦

seeprettyface.com 141 Dec 26, 2022
Baseline model for "GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping" (CVPR 2020)

GraspNet Baseline Baseline model for "GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping" (CVPR 2020). [paper] [dataset] [API] [do

GraspNet 209 Dec 29, 2022
How to Learn a Domain Adaptive Event Simulator? ACM MM, 2021

LETGAN How to Learn a Domain Adaptive Event Simulator? ACM MM 2021 Running Environment: pytorch=1.4, 1 NVIDIA-1080TI. More details can be found in pap

CVTEAM 4 Sep 20, 2022
ExCon: Explanation-driven Supervised Contrastive Learning

ExCon: Explanation-driven Supervised Contrastive Learning Contributors of this repo: Zhibo Zhang ( Zhibo (Darren) Zhang 18 Nov 01, 2022

Official implementation of CATs: Cost Aggregation Transformers for Visual Correspondence NeurIPS'21

CATs: Cost Aggregation Transformers for Visual Correspondence NeurIPS'21 For more information, check out the paper on [arXiv]. Training with different

Sunghwan Hong 120 Jan 04, 2023
A particular navigation route using satellite feed and can help in toll operations & traffic managemen

How about adding some info that can quanitfy the stress on a particular navigation route using satellite feed and can help in toll operations & traffic management The current analysis is on the satel

Ashish Pandey 1 Feb 14, 2022
Official implementation of Representer Point Selection via Local Jacobian Expansion for Post-hoc Classifier Explanation of Deep Neural Networks and Ensemble Models at NeurIPS 2021

Representer Point Selection via Local Jacobian Expansion for Classifier Explanation of Deep Neural Networks and Ensemble Models This repository is the

Yi(Amy) Sui 2 Dec 01, 2021
Cross-Image Region Mining with Region Prototypical Network for Weakly Supervised Segmentation

Cross-Image Region Mining with Region Prototypical Network for Weakly Supervised Segmentation The code of: Cross-Image Region Mining with Region Proto

LiuWeide 16 Nov 26, 2022
TensorFlow (Python) implementation of DeepTCN model for multivariate time series forecasting.

DeepTCN TensorFlow TensorFlow (Python) implementation of multivariate time series forecasting model introduced in Chen, Y., Kang, Y., Chen, Y., & Wang

Flavia Giammarino 21 Dec 19, 2022
An air quality monitoring service with a Raspberry Pi and a SDS011 sensor.

Raspberry Pi Air Quality Monitor A simple air quality monitoring service for the Raspberry Pi. Installation Clone the repository and run the following

rydercalmdown 24 Dec 09, 2022
A simple rest api that classifies pneumonia infection weather it is Normal, Pneumonia Virus or Pneumonia Bacteria from a chest-x-ray image.

This is a simple rest api that classifies pneumonia infection weather it is Normal, Pneumonia Virus or Pneumonia Bacteria from a chest-x-ray image.

crispengari 3 Jan 08, 2022
This is the official repository for our paper: ''Pruning Self-attentions into Convolutional Layers in Single Path''.

Pruning Self-attentions into Convolutional Layers in Single Path This is the official repository for our paper: Pruning Self-attentions into Convoluti

Zhuang AI Group 77 Dec 26, 2022
kullanışlı ve işinizi kolaylaştıracak bir araç

Hey merhaba! işte çok sorulan sorularının cevabı ve sorunlarının çözümü; Soru= İçinde var denilen birçok şeyi göremiyorum bunun sebebi nedir? Cevap= B

Sexettin 16 Dec 17, 2022
Unofficial implementation of "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" (https://arxiv.org/abs/2103.14030)

Swin-Transformer-Tensorflow A direct translation of the official PyTorch implementation of "Swin Transformer: Hierarchical Vision Transformer using Sh

52 Dec 29, 2022