The ABR Control library is a python package for the control and path planning of robotic arms in real or simulated environments.

Overview

https://imgur.com/4qIqbRn.jpg

ABR Control

The ABR Control library is a python package for the control and path planning of robotic arms in real or simulated environments. ABR Control provides API's for the Mujoco, CoppeliaSim (formerly known as VREP), and Pygame simulation environments, and arm configuration files for one, two, and three-joint models, as well as the UR5 and Kinova Jaco 2 arms. Users can also easily extend the package to run with custom arm configurations. ABR Control auto-generates efficient C code for generating the control signals, or uses Mujoco's internal functions to carry out the calculations.

ABR also provides an interface and config available for controlling a real Jaco 2 at the ABR_Jaco2 repository.

Installation

The ABR Control library depends on NumPy, SymPy, SciPy, CloudPickle, Cython, SetupTools, Nengo, and Matplotlib. We recommend using Anaconda. Note that installing in a clean environment will require compiling of the dependent libraries, and will take a few minutes.

To install ABR Control, clone this repository and run:

sudo apt-get install g++
sudo apt-get install python-dev
sudo apt-get install libfreetype6-dev
conda activate your_environment
python setup.py install
python setup.py develop

ABR Control is tested to work on Python 3.6+, Python 2 is not supported.

Optional Installation

Mujoco

If you would like to use the Mujoco API you will need to install a forked version of mujoco-py with hooks for exitting out of simulations with the ESC key. To use the mujoco API, make sure you are in your anaconda environment and run:

git clone https://github.com/studywolf/mujoco-py.git
cd mujoco-py
pip install -e .
pip install glfw>=1.8.3
pip install requests

Pygame

If you would like to use the Pygame API, from your anaconda environment run:

pip install pygame

Vrep

We support Vrep <=4.0. You will need to download Vrep and follow the installation instructions.

PyDMPs

Some of the path planners work through the use of dynamic movement primitives (DMPs). DMPs allow for a stable, generalizable, and easily extensible way of representing complex trajectories. Path planners making use of DMP are appended with 'dmp' in their filename and will require the installation of the pydmps repository. To install, from your Anaconda environment run:

pip install pydmps

Usage

The ABR Control repo is comprised of three parts: 1) arms, 2) controllers, and 3) interfaces.

1a) Arms: Using CoppeliaSim, Pygame, or a real arm

All of the required information about an arm model is kept in that arm's config file. To use the ABR Control library with a new arm, the user must provide the transformation matrices (written using SymPy expressions) from the robot's origin reference frame to each link's centre-of-mass (COM) and joints. These are specified sequentially, e.g. origin -> link0 COM, link0 COM -> joint0, joint0 -> link1 COM, etc. The arm config file and any simulation code is kept in a folder named the same as the arm in the abr_control/arms/ directory.

The ABR Control configuration base class uses the SymPy transform matrices to provide functions that will calculate the transforms, Jacobian, Jacobian derivative, inertia matrices, gravity forces, and centripetal and Coriolis effects for each joint and COM. These can be accessed:

from abr_control.arms import jaco2

robot_config = jaco2.Config()
# calculate the following given the arm state at joint_angles
robot_config.Tx('joint3', q=joint_angles)  # the (x, y, z) position of joint3
robot_config.M(q=joint_angles)  # calculate the inertia matrix in joint space
robot_config.J('EE', q=joint_angles)  # the Jacobian of the end-effector

By default, the use_cython parameter is set to True to allow for real-time control by generating optimized Cython code for each of the robot configuration functions. This can take a little bit of time to generate these functions, but they are saved in ~.cache/abr_control/arm_name/saved_functions where they will be loaded from for future runs. Note that a hash is saved for the config, so if any changes are made the functions will be regenerated during the next use. The cython optimization can be turned off on instantiation:

from abr_control.arms import ur5

robot_config = ur5.Config(use_cython=False)

Below are results from running the operational space controller with different controllers with use_cython=True and False.

docs/examples/timing.png

1b) Arms: Using Mujoco

When using Mujoco the process is a bit different. Mujoco handles the calculation of all the kinematics and dynamics functions, and only requires an xml config be made describing the kinematic chain. The Mujoco API page describes this in detail.

Detailed models can be created by importing 3D modeling stl files and using the mesh object type in the tag. An example of this is the abr_control/arms/jaco2/jaco2.xml. For users building their own models, you may specify the location of the xml with the folder parameter. For more details, please refer to the Mujoco documentation linked above and use the xml files in this repository as examples.

2) Controllers

Controllers make use of the robot configuration files to generate control signals that accomplish a given task (for most controllers this is reaching a target). The ABR Control library provides implementations of several primary controllers, including operational space, generalized coordinates (joint) space, sliding, and floating control.

When using an operational space controller, it is possible to also pass in secondary controllers to operate in the null space of the operational space controller. These secondary controllers can be set up to achieve secondary goals such as avoiding joint limits and obstacles, damping movement, or maintaining a configuration near a specified resting state.

In the path_planners folder there are several path planners that can be used in conjunction with the controllers. There are filters, linear and second order, which can be used to trace a path from the current position to the target without suddenly warping and causing large spikes in generated torque. The inverse kinematics planner takes in a target for the end-effector and returns a joint angle trajectory to follow. An arc path planner is also provided that creates an arcing path which can be useful when the arm has to reach over itself. This can help prevent self-collisions and odd arm configurations.

Each path planner also has the ability to generate a trajectory for end-effector orientation with the path_plannner.generate_orientation_path() function. This uses spherical linear interpolation (SLERP) to generate a set of orientations from a start to a target quaternion. The time profile will match that of the path planner instantiated (ie: a linear path planner will have a linear step in orientation over time, with a constant change in orientation, whereas a second order path planner will have a bell shaped profile with the largest steps occurring during the middle of the movement, with an acceleration and deceleration at the start and end, respectively.) In addition to filters, there is an example path planner using the dynamic movement primitives trajectory generation system.

Finally, there is an implementation of nonlinear adaptive control in the signals folder, as well as examples in Mujoco, PyGame, and CoppeliaSim showing how this class can be used to overcome unexpected forces acting on the arm.

3) Interfaces

For communications to and from the system under control, an interface API is used. The functions available in each class vary depending on the specific system, but must provide connect, disconnect, send_forces and get_feedback methods.

Putting everything together

A control loop using these three files looks like:

from abr_control.arms import jaco2
from abr_control.controllers import OSC
from abr_control.interfaces import CoppeliaSim

robot_config = jaco2.Config()
interface = CoppeliaSim(robot_config)
interface.connect()

ctrlr = OSC(robot_config, kp=20,
            # control (x, y, z) out of [x, y, z, alpha, beta, gamma]
            ctrlr_dof=[True, True, True, False, False, False])

target_xyz = [.2, .2, .5]  # in metres
target_orientation = [0, 0, 0]  # Euler angles, relevant when controlled
for ii in range(1000):
    feedback = interface.get_feedback()  # returns a dictionary with q, dq
    u = ctrlr.generate(
        q=feedback['q'],
        dq=feedback['dq'],
        target=np.hstack([target_xyz, target_orientation]))
    interface.send_forces(u)  # send forces and step CoppeliaSim sim forward

interface.disconnect()

NOTE that when using the Mujoco interface it is necessary to instantiate and connect the interface before instantiating the controller. Some parameters only get parsed from the xml once the arm config is linked to the mujoco interface, which happens upon connection.

Examples

The ABR Control repo comes with several examples that demonstrate the use of the different interfaces and controllers.

By default all of the PyGame examples run with the three-link MapleSim arm. You can also run the examples using the two-link Python arm by changing the import statement at the top of the example scripts.

To run the CoppeliaSim examples, have the most recent CoppeliaSim version open. By default, the CoppeliaSim examples all run with the UR5 or Jaco2 arm model. To change this, change which arm folder is imported at the top of the example script. The first time you run an example you will be promted to download the arm model. Simply select yes to download the file and the simulation will start once the download completes.

To run the Mujoco examples, you will be promted to download any mesh or texture files, if they are used in the xml config, similarly to the CoppeliaSim arm model. Once the download completes the simulation will start. If you are using the forked Mujoco-Py repository (See Optional Installation section) you can exit the simulation with the ESC key and pause with the spacebar.

Owner
Applied Brain Research
Applied Brain Research
A Fear and Greed index visualiser for Bitcoin on a SSD1351 OLED Screen

We're Doomed - A Bitcoin Fear and Greed index OLED visualiser Doom is a first-person-shooter from the 1990s. The health status monitor was one of the

VEEB 19 Dec 29, 2022
Nordpool_diff custom integration for Home Assistant

nordpool_diff custom integration for Home Assistant Requires https://github.com/custom-components/nordpool Applies non-causal FIR differentiator1 to N

Joonas Pulakka 45 Dec 23, 2022
A simple portable USB MIDI controller based on Raspberry-PI Pico and a 16-button keypad, written in Circuit Python

RPI-Pico-16-BTn-MIDI-Controller-using-CircuitPython A simple portable USB MIDI controller based on Raspberry-PI Pico, written in Circuit Python. Link

Rounak Dutta 3 Dec 04, 2022
Home Assistant custom integration for Yi cameras: yi-hack-MStar, yi-hack-Allwinner and yi-hack-Allwinner-v2

yi-hack Home Assistant integration Overview yi-hack Home Assistant is a custom integration for Yi cameras (or Sonoff camera) with one of the following

roleo 131 Jan 03, 2023
Setup DevTerm to be a cool non-GUI device

DevTerm hobby project I bought this amazing device: DevTerm A-0604. It has a beefy ARM processor, runs a custom version of Armbian, embraces Open Sour

Alex Shteinikov 9 Nov 17, 2022
Pure micropython ESP32 SPI driver for sdcard and screen at the same SPI bus

micropython-esp32-spi-sdcard-and-screen-driver Proof of concept of Pure micropython espidf SPI driver for sdcard with screen at the same SPI bus (exam

Thomas Favennec 7 Mar 14, 2022
Tools and documentation to aid in modifying the ADI ADALM Pluto firmware

Pluto firmware modifications This repository contains tools and documentation to aid in modifying the ADI ADALM Pluto firmware. Extraction of the Plut

Daniel Estévez 28 Dec 21, 2022
An open source two key macro-pad modeled to look like a cartoony melting popsicle

macropopsicle An open source two key macro-pad modeled to look like a cartoony melting popsicle. Build instructions Parts List -1x Top case half (3D p

17 Aug 18, 2022
Python application, displaying currently played track from Spotify on OLED display connected via I2C

RaspberryPi Spotify OLED Display This application will display currently played track on SSD1306 OLED display connected to RaspberryPi. Displayed stuf

Wojciech Olech 2 Dec 30, 2021
A versatile program that uses the raspberry pi camera and provides it as a service

PiCameleon Is a daemon program meant to provide the RaspberryPi Camera as a service while running according to a configuration.

André Esser 52 Oct 16, 2022
Unofficial Playdate reverse-engineering notes/tools - covers file formats, server API and USB commands

Unofficial Playdate reverse-engineering notes/tools - covers file formats, server API and USB commands ⚠️ This documentation is unofficial and is not

James 106 Dec 31, 2022
Automatically draw a KiCad schematic for a circuit prototyped on a breadboard.

Schematic-o-matic Schematic-o-matic automatically draws a KiCad schematic for a circuit prototyped on a breadboard. How It Works The first step in the

Nick Bild 22 Oct 11, 2022
Open-Source board for converting RaspberryPI to Brain-computer interface

The easiest way to the neuroscience world with the shield for RaspberryPi - PIEEG (website). Open-source. Crowdsupply This project is the result of se

Ildaron 436 Jan 01, 2023
USB Rubber Ducky with the Rasberry Pi pico microcontroller

pico-ducky Install Install and have your USB Rubber Ducky working in less than 5 minutes. Download CircuitPython for the Raspberry Pi Pico. Plug the d

AnOnYmOus001100 3 Oct 08, 2022
OctoPrint is the snappy web interface for your 3D printer!

OctoPrint OctoPrint provides a snappy web interface for controlling consumer 3D printers. It is Free Software and released under the GNU Affero Genera

OctoPrint 7.1k Jan 03, 2023
Testing out some (stolen) DMA code for RP2040 Micropython

RP2040_micropython_dma testing out some (stolen) DMA code for RP2040 Micropython. Heavy inspiration and some code from https://iosoft.blog/2021/10/26/

2 Dec 29, 2022
Point Density-Aware Voxels for LiDAR 3D Object Detection (CVPR 2022)

PDV PDV is LiDAR 3D object detection method. This repository is based off [OpenPCDet]. Point Density-Aware Voxels for LiDAR 3D Object Detection Jordan

Toronto Robotics and AI Laboratory 114 Dec 21, 2022
A ESP32 project template with a web interface built in React

ESP AP Webserver demo.mp4 This is my experiment with "mobile app development" for the ESP32. The project consists of two parts, the ESP32 code and the

8 Dec 15, 2022
Cow Feeder is a bot automatically execute trade on cowswap

Cow Feeder is a bot automatically execute trade on cowswap, includes functions: Monitoring Ethereum network gas price and execute trade whe

6 Apr 20, 2022
emhass: Energy Management for Home Assistant

emhass EMHASS: Energy Management for Home Assistant Context This module was conceived as an energy management optimization tool for residential electr

David 70 Dec 24, 2022