Final project for ENGG 5402 Advanced Robotics in CUHK

Overview

Final project


Hi all,

Congratulations on getting into the most interesting part of this course! You could use what you have learnt in class to complete a real robot manipulation task, and this is also the chance to check whether you have understand those knowledge. Maybe you'll be interested in robotics through this practical project, then welcome to join us and do something that changes the world! So why still hesitate? Take action now!

Junjia Liu


Update

  1. 2021.11.22 - Some students told me that they do not have a computer with a Nvidia GPU. So, I tried to remove Nvidia's dependence from the docker file. However, I found that the Pybullet simulator is mainly rely on the OpenGL, which has to require a Nvidia driver.

  2. 2021.11.22 - If you have any question about this project, please try to learn and use the issues in Github to ask your questions so that I can reply in time and let other students who encounter the same problem know the solution at the same time.

    image-20211121165337801


Foundations

Notice: If you are familiar with git, Github, Docker, Linux, virtual machine, you can skip this chapter and start from the second chapter Set up your Environment.

Ubuntu virtual machine

Step 1: Since this project requires Docker and Ubuntu environment, and if you are just using a Windows PC, I highly recommend you install a virtual machine in your Windows as your first step. The virtual machine I recommend is vmware.

Notice: You need to give a large space to the virtual machine (> 30G), otherwise the space will not enough for pulling the docker image.

Step 2: After installing your virtual machine, please follow this video to install a Ubuntu 18.04 on VMware in your Windows.

Cheers! You have your first Linux system!

Ubuntu

In Ubuntu, the only thing you need to know is how to use the Terminal. Everything can be done by several terminal commands.

Step 3: Here is a command tutorial for beginners, you can take a quick look and understand the basic usage methods.

How to use Github to keep tracking the change of code version?

Notice: From here, forget about your Windows, everything is in Ubuntu!

Since this project contains large amount of dependent packages and various unexpected errors may occur during your installation and use, it is convenient for us to use Github to host our code. There are two advantages:

  1. I can modify the bug you faced and re-push (upload) the correct version of code rapidly;
  2. You can re-pull (download & update) the modification in the correct version easily, rather than the whole code repository again and again.

Step4: Here I highly recommend to use the GitKraken to clone (initial download) the project repository in your Ubuntu system and keep tracking the version changes.

Or simple git command is also recommended. Here is a tutorial.

Cheers! You're doing the same cool things as a programmer.

Docker

Step5: Get familiar with Docker [ English | Chinese ].

Set up your Environment

In order for your solution to be executable on your local machine, I provide you a docker image with some pre-installed software. You need to develop and test your solution within this docker image.

Step6: So first you need to install the following requirements for using Docker.

Simulator installation

Clone the Repository

There are some submodules in the software package, and you need to clone the repository recursively.

Step7: Clone the project repository using the Terminal.

git clone https://github.com/Skylark0924/advanced_robotics_project.git --recursive

or

git clone https://github.com/OCRTOC/OCRTOC_software_package
cd OCRTOC_software_package
git submodule update --init --recursive

Set up Docker

Step8: Pull the Docker image or build it from docker file.

Notice: This will take really a long time (about 1~3 hours), so you can leave the virtual machine alone (but not shutdown it) and do something else.

  • Option 1 (suggested, faster): Build docker image from docker file:

    # Jump to the path you clone the project repository
    cd [YOUR_REPO_PATH]
    cd docker
    bash build.sh ocrtoc_pb_g:v1
    
  • Option 2: Pull the pre-built docker image from either of the following sources:

    docker pull registry.cn-hangzhou.aliyuncs.com/ocrtoc2021/release:2.1
    

    or

    docker pull registry.us-west-1.aliyuncs.com/ocrtoc2021/release:2.1
    

Start Docker

Step9: I provide two scripts for creating and executing the docker image. Please modify the name and path in the scripts according to your own needs.

# Jump to the path you clone the project repository
cd [YOUR_REPO_PATH]

# Create docker container
bash tools/create_container.sh

# Enter the docker container
bash tools/exec_container.sh

When your host operating system restarts, the docker will not restart automatically. I also provide another script for you to restart the docker.

# Restart docker container
bash tools/start_container.sh

# Enter the docker container
bash tools/exec_container.sh

Set up Environment inside the Docker Container

Step10: Since some parts of the environment are given as source code, they cannot be installed while building the docker image. You need to run the following commands to set up the environment. However, you don't need to run them when you restart the docker container.

# Enter the docker container
bash tools/exec_container.sh

## In the docker image
cd /root/ocrtoc_ws/src/tools
bash setup_env.sh

Content of the Docker Image

  • Operating System: Ubuntu 18.04
  • ROS melodic-desktop-full
  • moveit 1.0.7
  • CUDA 11.2
  • PyBullet 3.0
  • ceres-solver 2.0
  • colmap 3.7
  • pycolmap 0.0.1
  • torch==1.8.2, torchvision==0.7.0, open3d==0.12.0 for python3 and open3d==0.9.0 for python

Now you have finished setting up the environment, and you can try to run the baseline solution.

Cheers! By now you have completed all the installation steps, let's enjoy the exploration!

Explore the simulator

Step11: Open it! Open 3 terminals and execute the following commands in each of the terminal.

# In the first terminal, run the following to open the docker image
bash tools/exec_container.sh

# In the docker image
# Start the PyBullet simulator and ros interface
roslaunch ocrtoc_task bringup_simulator_pybullet.launch task_index:=0-0


# In the second terminal
bash tools/exec_container.sh

# In the docker image
# Start moveit and rviz
roslaunch panda_moveit_config ocrtoc.launch


# In the third terminal
bash tools/exec_container.sh

# In the docker image
# Start manipulator interface node
rosrun ocrtoc_common manipulator_interface_node.py

You may find a motion planning plugins is shown in the rviz, which is a motion planning framework called MoveIt. This framework is usually used in the daily robotics research, since it contains many mature robotics algorithms such as ... and allows us to execute the robot motion without disturbed by trivia. Here is the manual of the MoveIt, you can take a look but do not waste too much time: http://docs.ros.org/en/kinetic/api/moveit_tutorials/html/doc/quickstart_in_rviz/quickstart_in_rviz_tutorial.html

However, in this project, we expect you can practice what you have learned in this course, rather than using some open source packages.

Project

Requirement

In this project, you are required to implement a kinematic control on a 7-DOF Franka Panda robot to track a desired trajectory.

Demo

demo

DH parameter of Panda robot

The Denavit–Hartenberg parameters of this robot are given as follows:

coor

image-20211121214314827

Program and test your solution

Though there are a mass of scripts in this project, the only one you need to consider and modify is ocrtoc_common/test/trajectory_tracking.py.

# Jump to the path of your solution
roscd ocrtoc_common/test

# Plot the desired traj in rviz (python2)
python test_dsr_traj_vis.py

# Run your solution (python3)
python3 trajectory_tracking.py

Notice: Every time you try to test your code, please make sure the three terminals mentioned in the Step11 is running normally! Otherwise, you will receive a plenty of errors.

Criteria

  1. (20%) Finish the forward kinematic function def fk(self, q);
  2. (20%) Finish the Jacobian function def jacobian(self, q);
  3. (55%) Finish the kinematic control function `def control(self);
  4. (5%) Plot your results as I mentioned in the def traj_vis(self) ;
  5. Bonus (+10%) Take your code and come to Academic Building 1 (2F) to test your code on the real Panda robot.

Please submit your trajectory_tracking.py and a document which contains your result figures. The deadline is behind the final exam.

Trouble shooting

  1. If you are using the recommended installation and your terminal is stuck here, don't be panic, it just takes a while.

    image-20211113152603381

  2. If you cannot find anything in \root\ocrtoc_ws\src, it probably be the mistake in tools\create_container.sh. You need to modify the name and path in the scripts according to your own needs.

    So you need to modify $HOME/OCRTOC_software_package:/root/ocrtoc_ws/src -> :/root/ocrtoc_ws/src .

  3. When you run rosrun ocrtoc_common manipulator_interface_node.py and get an error like this:

    m_i = ManipulatorInterface('panda_arm') File "/root/ocrtoc_ws/src/ocrtoc_common/src/ocrtoc_common/manipulator_interface.py", line 25, in __init__ self.move_group = moveit_commander.MoveGroupCommander(group_name) File "/opt/ros/melodic/lib/python2.7/dist-packages/moveit_commander/move_group.py", line 66, in __init__ name, robot_description, ns, wait_for_servers RuntimeErrorshutdown request: [/manipulator_interface_node] Reason: new node registered with same name : Unable to connect to move_group action server 'move_group' within allotted time (5s) ">
    Traceback (most recent call last):
      File "/root/ocrtoc_ws/src/ocrtoc_common/scripts/manipulator_interface_node.py", line 8, in 
         
          
        m_i = ManipulatorInterface('panda_arm')
      File "/root/ocrtoc_ws/src/ocrtoc_common/src/ocrtoc_common/manipulator_interface.py", line 25, in __init__
        self.move_group = moveit_commander.MoveGroupCommander(group_name)
      File "/opt/ros/melodic/lib/python2.7/dist-packages/moveit_commander/move_group.py", line 66, in __init__
        name, robot_description, ns, wait_for_servers
    RuntimeErrorshutdown request: [/manipulator_interface_node] Reason: new node registered with same name
    : Unable to connect to move_group action server 'move_group' within allotted time (5s)
    
         

    please try to exit the docker and get in again.

    # In the docker image
    exit
    
    # Out of the docker
    cd [YOUR_REPO_PATH]
    bash tools/exec_container.sh
    

How to reboot the docker

You must believe that some bugs can be fixed by just reboot the system. So, how can you reboot the Docker?

# In the docker image
exit

# Out of the docker
docker ps -a
## It will output a list of containers, find the one the 'IMAGE' is 'ocrtoc_pb_g:v1' and copy its 'CONTAINER ID'

docker stop [CONTAINER_ID_YOU_COPY]

# Out of the docker
cd [YOUR_REPO_PATH]
bash tools/start_container.sh

bash tools/exec_container.sh

How to deal with the version update?

If there are some issues in this repository, I will fix it and notify you via email. You don not need to execute Step1~Step10 again, the only two things you need to do is to pull the modification and execute Step10 to renew your environment.

# In your repo path
git pull  # you might face a merge alert, handle it yourself

# Enter the docker container
bash tools/exec_container.sh

## In the docker image
cd /root/ocrtoc_ws/src/tools
bash setup_env.sh
Owner
Junjia Liu
Ph.D. student in Robotics, CUHK
Junjia Liu
Run CodeServer on Google Colab using Inlets in less than 60 secs using your own domain.

Inlets Colab Run CodeServer on Colab using Inlets in less than 60 secs using your own domain. Features Optimized for Inlets/InletsPro Use your own Cus

2 Dec 30, 2021
A inspector to be able to view and edit Qt style sheet while an application is running

Qt Style Sheet Inspector An inspector widget to view and modify the style sheet of a Qt app at runtime. Usage In order to use the inspector widget on

ESSS 46 Dec 10, 2022
These are the scripts used for the project of ‘Assembly of a pan-genome for global cattle reveals missing sequence and novel structural variation, providing new insights into their diversity and evolution history’

script-SV-genotyping These are the scripts used for the project of ‘Assembly of a pan-genome for global cattle reveals missing sequence and novel stru

2 Aug 26, 2022
A Guide for Feature Engineering and Feature Selection, with implementations and examples in Python.

Feature Engineering & Feature Selection A comprehensive guide [pdf] [markdown] for Feature Engineering and Feature Selection, with implementations and

Yimeng.Zhang 968 Dec 29, 2022
Create rangebased on lists or values of the range itself. Range any type. Can you imagine?

funcao-allrange-for-python3 Create rangebased on lists or values of the range itself. Range any type. Can you imagine? WARNING!!! THIS MODULE DID NOT

farioso-fernando 1 Feb 09, 2022
A(Sync) Interface for internal Audible API written in pure Python.

Audible Audible is a Python low-level interface to communicate with the non-publicly Audible API. It enables Python developers to create there own Aud

mkb79 192 Jan 03, 2023
Test pour savoir si je suis capable de paratger une lib avec le monde entier !!

Data analysis Document here the project: MLproject Description: Project Description Data Source: Type of analysis: Please document the project the bet

Lucas_Penarrubia 0 Jan 18, 2022
This is a repository containing the backend and the frontend of a simple pokédex.

Pokémon This is a repository containing the backend and the frontend of a simple pokédex. This is a work in progress project! Project Structure 🗂 pok

André Rato 1 Nov 28, 2021
Santa's kitchen helper for python

Santa's Kitchen Helper Introduction/Overview Contents UX User Stories Design Wireframes Color Scheme Typography Imagery Features Exisiting Features Fe

Paul Browne 4 May 31, 2022
A simple solution for water overflow problem in Python

Water Overflow problem There is a stack of water glasses in a form of triangle as illustrated. Each glass has a 250ml capacity. When a liquid is poure

Kris 2 Oct 22, 2021
Python template for Advent of Code event

Advent of Code Python Starter A tamplate for Advent of Code write in Python. Usage The project use poetry for project manager. Clone this repository a

Leonardo Gago 6 Dec 31, 2022
A Python application that simulates the rolling of a dice, randomly picking one of the 6 faces and then displaying it.

dice-roller-app This is an application developed in Python that shuffles between the 6 faces of a dice, using buttons to shuffle and close the applica

Paddy Costelloe 0 Jul 20, 2021
A prototype COG-based tile server for sparse Mars datasets

Mars tiler Mars Tiler is a prototype web application that serves tiles from cloud-optimized GeoTIFFs, with an emphasis on supporting planetary dataset

Daven Quinn 3 Mar 23, 2022
Automatic certificate unpinning for Android apps

What is this? Script used to perform automatic certificate unpinning of an APK by adding a custom network security configuration that permits user-add

Antoine Neuenschwander 5 Jul 28, 2021
万能通用对象池,可以池化任意自定义类型的对象。

pip install universal_object_pool 此包能够将一切任意类型的python对象池化,是万能池,适用范围远大于单一用途的mysql连接池 http连接池等。 框架使用对象池包,自带实现了4个对象池。可以直接开箱用这四个对象池,也可以作为例子学习对象池用法。

12 Dec 15, 2022
Find the remote website version based on a git repository

versionshaker Versionshaker is a tool to find a remote website version based on a git repository This tool will help you to find the website version o

Orange Cyberdefense 110 Oct 23, 2022
A Notifier Program that Notifies you to relax your eyes Every 15 Minutes👀

Every 15 Minutes is an application that is used to Notify you to Relax your eyes Every 15 Minutes, This is fully made with Python and also with the us

Ashely Sato 1 Nov 02, 2021
nbsafety adds a layer of protection to computational notebooks by solving the stale dependency problem when executing cells out-of-order

nbsafety adds a layer of protection to computational notebooks by solving the stale dependency problem when executing cells out-of-order

150 Jan 07, 2023
Battle-Ship - Python-console battle ship

Battle-Ship this SHOULD work in lenux(if i spelled it wrong spam issues till I fix it) the thing that maby wont work is where it clears the screen the

pl608 2 Jan 06, 2022
Unzip Japanese Shift-JIS zip archives on non-Japanese systems.

Unzip JP GUI Unzip Japanese Shift-JIS zip archives on non-Japanese systems. This script unzips the file while converting the file names from Shift-JIS

Emile Bangma 9 Dec 07, 2022