Developing and Comparing Vision-based Algorithms for Vision-based Agile Flight

Overview

DodgeDrone: Vision-based Agile Drone Flight (ICRA 2022 Competition)

IMAGE ALT TEXT HERE

Would you like to push the boundaries of drone navigation? Then participate in the dodgedrone competition! You will get the chance to develop perception and control algorithms to navigate a drone in both static and dynamic environments. Competing in the challenge will deepen your expertise in computer vision and control, and boost your research. You can find more information at the competition website.

This codebase provides the following functionalities:

  1. A simple high-level API to evaluate your navigation policy in the Robot Operating System (ROS). This is completely independent on how you develop your algorithm.
  2. Training utilities to use reinforcement learning for the task of high-speed obstacle avoidance.

All evaluation during the competition will be performed using the same ROS evaluation, but on previously unseen environments / obstacle configurations.

Submission

  • 06 May 2022 Submission is open. Please submit your version of the file user_code.py with all needed dependencies with an email to loquercio AT berkeley DOT edu. Please use as subject ICRA 2022 Competition: Team Name. If you have specific dependencies, please provide instructions on how to install them. Feel free to switch from python to cpp if you want.

Further Details

  • We will only evaluate on the warehouse environment with spheres obstacles.
  • If you're using vision, you are free to use any sensor you like (depth, optical flow, RGB). The code has to run real-time on a desktop with 16 Intel Core i7-6900K and an NVIDIA Titan Xp.
  • If you're using vision, feel free to optimize the camera parameters for performance (e.g. field of view).
  • We will two rankings, one for vision-based and another for state-based. The top three team for each category will qualify for the finals.

Update

  • 02 May 2022 Fix a bug in the vision racing environment when computing reward function. No need to update if you are not using RL or if you have change the reward formualtion. Related to this issue #65

  • 27 March 2022 Fix a static object rendering issue. Please download the new Unity Standalone using this. Also, git pull the project.

Flight API

This library contains the core of our testing API. It will be used for evaluating all submitted policies. The API is completely independent on how you build your navigation system. You could either use our reinforcement learning interface (more on this below) or add your favourite navigation system.

Prerequisite

Before continuing, make sure to have g++ and gcc to version 9.3.0. You can check this by typing in a terminal gcc --version and g++ --version. Follow this guide if your compiler is not compatible.

In addition, make sure to have ROS installed. Follow this guide and install ROS Noetic if you don't already have it.

Installation (for ROS User)

We only support Ubuntu 20.04 with ROS noetic. Other setups are likely to work as well but not actively supported.

Start by creating a new catkin workspace.

cd     # or wherever you'd like to install this code
export ROS_VERSION=noetic
export CATKIN_WS=./icra22_competition_ws
mkdir -p $CATKIN_WS/src
cd $CATKIN_WS
catkin init
catkin config --extend /opt/ros/$ROS_VERSION
catkin config --merge-devel
catkin config --cmake-args -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-fdiagnostics-color

cd src
git clone [email protected]:uzh-rpg/agile_flight.git
cd agile_flight

Run the setup_ros.bash in the main folder of this repository, it will ask for sudo permissions. Then build the packages.

./setup_ros.bash

catkin build

Installation (for Python User)

If you want to develop algorithms using only Python, especially reinforcement learning, you need to install our library as python package.

Make sure that you have anaconda installed. This is highly recommanded.

Run the setup_py.bash in the main folder of this repository, it will ask for sudo permissions.

./setup_py.bash

Task

The task is to control a simulated quadrotor to fly through obstacle dense environments. The environment contains both static and dynamic obstacles. You can specifiy which difficulty level and environment you want to load for testing your algorithm. The yaml configuration file is located in this file. The goal is to proceed as fast as possible 60m in positive x-direction without colliding into obstacles and exiting a pre-defined bounding box. The parameters of the goal location and the bounding box can be found here.

environment:
  level: "medium" # three difficulty level for obstacle configurations [easy, medium, hard]
  env_folder: "environment_0" # configurations for dynamic and static obstacles, environment number are between [0 - 100]

unity:
  scene_id: 0 # 0 warehouse, 1 garage, 2 natureforest, 3 wasteland

Usage

The usage of this code base entails two main aspects: writing your algorithm and testing it in the simulator.

Writing your algorithm:

To facilitate coding of your algorithms, we provided a simple code structure for you, just edit the following file: envtest/ros/user_code.py. This file contains two functions, compute_command_vision_based and compute_command_state_based. In the vision-based case, you will get the current image and state of the quadrotor. In the state-based case, you will get the metric distance to obstacles and the state of the quadrotor. We strongly reccomend using the state-based version to start with, it is going to be much easier than working with pixels!

Depending on the part of the competition you are interested in, adapt the corresponding function. To immediately see something moving, both functions at the moment publish a command to fly straight forward, of course without avoiding any obstacles. Note that we provide three different control modes for you, ordered with increasing level of abstraction: commanding individual single-rotor thrusts (SRT), specifying mas-normalized collective thrust and bodyrates (CTBR), and outputting linear velocity commands and yawrate (LINVEL). The choice of control modality is up to you. Overall, the more low-level you go, the more difficult is going to be to mantain stability, but the more agile your drone will be.

Testing your approach in the simulator:

Make sure you have completed the installation of the flight API before continuing. To use the competition software, three steps are required:

  1. Start the simulator

    roslaunch envsim visionenv_sim.launch render:=True
    # Using the GUI, press Arm & Start to take off.
    python evaluation_node.py
    

    The evaluation node comes with a config file. There, the options to plot the results can be disabled if you want no plots.

  2. Start your user code. This code will generate control commands based on the sensory observations. You can toggle vision-based operation by providing the argument --vision_based.

    cd envtest/ros
    python run_competition.py [--vision_based]
    
  3. Tell your code to start! Until you publish this message, your code will run but the commands will not be executed. We use this to ensure fair comparison between approaches as code startup times can vary, especially for learning-based approaches.

    rostopic pub /kingfisher/start_navigation std_msgs/Empty "{}" -1
    

If you want to perform steps 1-3 automatically, you can use the launch_evaluation.bash N script provided in this folder. It will automatically perform N rollouts and then create an evaluation.yaml file which summarizes the rollout statistics.

Using Reinforcement Learning (Optional) We provide an easy interface for training your navigation policy using reinforcement learning. While this is not required for the competition, it could just make your job easier if you plan on using RL.

Follow this guide to know more about how to use the training code and some tips on how to develop reinforcement learning algorithms

Owner
Robotics and Perception Group
Robotics and Perception Group
HogwartsRegister - A Hogwarts Register With Python

A Hogwarts Register Installation download code git clone https://github.com/haor

0 Feb 12, 2022
A script to automatically update bot status at GitHub as well as in Telegram channel.

A simple & short repository to show your bot's status in your GitHub README.md file as well as in you channel.

Jainam Oswal 55 Dec 13, 2022
ThnoolBox - A thneed is a multi-use versatile object

ThnoolBox Have you ever wanted a collection of bodged desktop apps that are Lorax themed ? No ? Sucks to suck I guess Apps & their downsides CalculaTh

pocoyo 1 Jan 21, 2022
This an Anki add on that automatically converts Notion notes into Anki flash cards. Currently in development!

NotionFlash This is an Anki add on in development that will allow automatically convert your Notion study notes into Anki flash cards. The Anki deck c

Neeraj Patel 10 Oct 07, 2022
Pequenos programas variados que estou praticando e implementando, leia o Read.me!

my-small-programs Pequenos programas variados que estou praticando e implementando! Arquivo: automacao Automacao de processos de rotina com código Pyt

Léia Rafaela 43 Nov 22, 2022
Trackthis - This library can be used to track USPS and UPS shipments.

Trackthis - This library can be used to track USPS and UPS shipments. It has the option of returning the raw API response, or optionally, it can be used to standardize the USPS and UPS responses so t

Aaron Guzman 0 Mar 29, 2022
🍏 Make Thinc faster on macOS by calling into Apple's native Accelerate library

🍏 Make Thinc faster on macOS by calling into Apple's native Accelerate library

Explosion 81 Nov 26, 2022
ARK sõidueksami Matrixi bot

ARK Sõidueksami bot Küsib ARK-i lehelt uusimad eksami ajad ja saadab sõnumi Matrixi kanali Dev setup Linux python3 -m venv venv source venv/bin/activa

Arti Zirk 3 Jun 15, 2021
To check my COVID-19 vaccine appointment, I wrote an infinite loop that sends me a Whatsapp message hourly using Twilio and Selenium. It works on my Raspberry Pi computer.

COVID-19_vaccine_appointment To check my COVID-19 vaccine appointment, I wrote an infinite loop that sends me a Whatsapp message hourly using Twilio a

Ayyuce Demirbas 24 Dec 17, 2022
万能通用对象池,可以池化任意自定义类型的对象。

pip install universal_object_pool 此包能够将一切任意类型的python对象池化,是万能池,适用范围远大于单一用途的mysql连接池 http连接池等。 框架使用对象池包,自带实现了4个对象池。可以直接开箱用这四个对象池,也可以作为例子学习对象池用法。

12 Dec 15, 2022
adbsync - An ADB syncing helper

adbsync - An ADB syncing helper What's this? Everytime I wanted to make a backup of my phone, or restore those files onto it, I had to use everytime t

Giovanni Gualtieri 3 Aug 05, 2022
Collection of functions for working with interlaced content in VapourSynth.

vsfieldkit Collection of functions for working with interlaced content in VapourSynth. It does not have any hard dependencies outside of VapourSynth.

Justin Turner Arthur 11 May 27, 2022
A fishing bot script written in Python!

A fishing bot script written in Python!

Anel Drocic 3 Nov 03, 2021
Buildium-to-stessa - Automation to assist in converting Buildium transactions into Stessa format

Buildium Transactions - Stessa Transactions There is currently no third-party i

Austin Comstock 4 Apr 17, 2022
Python-Roadmap - Дорожная карта по изучению Python

Python Roadmap Я решил сделать что-то вроде дорожной карты (Roadmap) для изучения языка Python. Возможно, если найдутся желающие дополнять ее, модифиц

Ruslan Prokhorov 1.2k Dec 28, 2022
Object-data mapper and advanced query manager for non relational databases

Object data mapper and advanced query manager for non relational databases. The data is owned by different, configurable back-end databases and it is

Luca Sbardella 121 Aug 11, 2022
A dashboard for your code. A build system.

NOTICE: THIS REPO IS NO LONGER UPDATED Changes Changes is a build coordinator and reporting solution written in Python. The project is primarily built

Dropbox 763 Sep 09, 2022
Another Provably Rare Gem Miner 💎 (for Raritygems)

Provably Rare Gem Miner Go (for Rarity) Pull Request is strongly welcome as I don't know anything about Golang/Python/Web3. Usage Install Python 3.x i

朱里 6 Apr 22, 2022
An esoteric programming language that supports concurrency, regex, and web requests.

The Hofstadter Esoteric Programming Language Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's La

Austin Henley 19 Dec 27, 2022
Refer'd Resume Scanner

Refer'd Resume Scanner I wanted to share a free resource we built to assist applicants with resume building. Our resume scanner identifies potential s

Refer'd 74 Mar 07, 2022