A Python library for reading, writing and visualizing the OMEGA Format

Overview

OMEGA Format - Python Library

This module is developed by ika - RWTH Aachen as a contribution to the VVM project which aims to develop test procedures and to provide frameworks and methods for the safety verification of automated vehicles. VVM is working on the use case of Urban Intersections and focuses on driving functions up to full automation of vehicles (SAE Level 4 and 5).

As part of the project a data format for storing reference and perception data from pilotings, test drives and simulation in urban traffic is developed. This module enables the creation, reading and visualization of data conforming to this data format. Additionally, it can check files for conformance and perform basic sanity checks on the data.

Data Format

The base of both, the reference data format and the perception data format is the HDF5 file format. This library utilizes h5py to interact with those.

Reference Data

The OMEGA Format reference recording format is used to store data that represents the 'true' state of road users, infrastructure information, weather and more during a piloting, testing or simulation. The representation is on an object list basis. The following diagram shows an overview of the hierarchy in the OMEGA Format reference recording format. A more detailed description can be found in the specification document and the signal list.

Perception Data

The PerceptionRecording format is used to store data that represents what a vehicle under test, sensor under test or similar perceives from its surroundings. It is designed to be compared against the ReferenceRecording format. The following diagram shows an overview of the hierarchy in the PerceptionRecording format. A more detailed description is coming soon.

Installation

The dependencies are managed with conda environments. Conda can be installed following the conda installation instructions.

To create a new conda environment omega_env and install the module run the following in your console:

conda env create -n omega_env -f environment_visualization.yml
conda activate omega_env

If you want an editable install (modifications to the files in the directory are immediately used by the module) run:

pip install -e .[visualization]

To update the existing environment upon addition of new dependencies:

conda env update -n omega_env -f environment_visualization.yml

Usage

Validate your data

To check if your file conforms to the specification run.

omega_format verify --reference <FILENAME>

or

omega_format verify --perception <FILENAME>

When using the library and creating objects or rading from an hdf5 file, by default, sanity checks are performed. To circumvent those pass validate=False to the from_hdf5 function or use or use cls.construct instead of cls as the constructor of an object. In the backend pydantic is used for the sanity checks.

Visualize your data

To visualize a data file execute the following in your conda environment:

omega_format visualize --snip --max-snippets=2 <FILENAME>

or in Python:

import omega_format
from omega_format.visualization import Visualizer, SnippetContainer
reference_recording = omega_format.ReferenceRecording.from_hdf5('path/to/the/reference_recording_file.hdf5')
visualizer = Visualizer(SnippetContainer.create_list(references=reference_recording))
visualizer.start_gui_and_visualization()

A window will open that lets you interact with and inspect your data.

Extending the visualizer

By subclassing omega_format.visualization.VisualizationModule and adding an instance of your subclass to the visualizers list of the Visualizer you can extend the functionality of the visualizer. The subclass has to implement at least one of the functions visualize_static and visualize_dynamics, returning a list of pyqt widgets to plot. For more details take a look at the omega_format.vis.VisualizationModule or the other modules defined in the visualization.modules directory.

Create a reference data

This module maps the reference and perception data file specifications to a hierarchy of python classes. The root classes are ReferenceRecording and PerceptionRecording respectively. First initializing an object from that class and fill its properties with the objects of the classes in question (e.g. Weather RoadUser, Lane). After adding all your data, call to_hdf5 on the ReferenceRecording or PerceptionRecording and a format compliant hdf5 file will be created for you.

import numpy as np
import omega_format
from datetime import datetime

rr = omega_format.ReferenceRecording(meta_data=omega_format.MetaData(recorder_number=1,
                                                                     recording_number=1,
                                                                     daytime=datetime.now()
                                                                     reference_point_lat=50.786687,
                                                                     reference_point_lon=6.046312),
                               timestamps=omega_format.Timestamps(val=np.array([0])),
                               )
rr.weather = omega_foramt.Weather()
rr.road_users[0] = omega_format.RoadUser(type=omega_format.ReferenceTypes.RoadUser.Type.CAR, sub_type=omega_format.ReferenceTypes.RoadUser.SubType.General.REGULAR,
                                    birth=0, bb=omega_format.BoundingBox(np.array([2,3,0])),
                                    tr=omega_format.Trajectory(pos_x=np.array([0]),pos_y=np.array([0]),pos_z=np.array([0]),
                                                          roll=np.array([0]),pitch=np.array([0]),heading=np.array([0])))
rr.roads[0] = omega_format.Road(location=omega_format.ReferenceTypes.RoadLocation.URBAN)
rr.to_hdf5('test.hdf5')

Further Help

Standalone viewer of hdf5 files

There are plenty of tools, e.g.

Documentation

You can create a documentation with pdoc3. To do this first install pdoc3 with pip install pdoc3 and then run pdoc3 --http localhost:8889 --template-dir .\doc\templates\ .\omega_format from the root of this repo to view the documentation in your web browser.

License

The library is published under the MIT license specified in LICENSE. An overview over the licenses of the dependencies in this library is listed in LICENSES_OF_REQUIREMENTS.md.

Contact

In case of questions regarding the format, this repository or otherwise related feel free to raise an issue or contact Michael Schuldes ([email protected]).

Acknowledgement

The research leading to these results is funded by the German Federal Ministry for Economic Affairs and Energy within the project “Verifikations- und Validierungsmethoden automatisierter Fahrzeuge im urbanen Umfeld". The authors would like to thank the consortium for the successful cooperation.

bmwi_logo

You might also like...
Script to generate a massive volume of data in sql, csv, json or xml format

DataGenerator Made with Python Open for pull requests 1. Dependencies To install required dependencies run pip install -r requirements.txt 2. Executi

Export watched content from Tautulli to the Letterboxd CSV Import Format

Export watched content from Tautulli to the Letterboxd CSV Import Format

🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.

Boltons boltons should be builtins. Boltons is a set of over 230 BSD-licensed, pure-Python utilities in the same spirit as — and yet conspicuously mis

isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type.
isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type.

isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type. It provides a command line utility, Python library and plugins for various editors to quickly sort all your imports.

ticktock is a minimalist library to view Python time performance of Python code.

ticktock is a minimalist library to view Python time performance of Python code.

RapidFuzz is a fast string matching library for Python and C++

RapidFuzz is a fast string matching library for Python and C++, which is using the string similarity calculations from FuzzyWuzzy

pydsinternals - A Python native library containing necessary classes, functions and structures to interact with Windows Active Directory.
pydsinternals - A Python native library containing necessary classes, functions and structures to interact with Windows Active Directory.

pydsinternals - Directory Services Internals Library A Python native library containing necessary classes, functions and structures to interact with W

Library for processing molecules and reactions in python way

Chython [ˈkʌɪθ(ə)n] Library for processing molecules and reactions in python way. Features: Read/write/convert formats: MDL .RDF (.RXN) and .SDF (.MOL

A simple gpsd client and python library.

gpsdclient A small and simple gpsd client and library Installation Needs Python 3 (no other dependencies). If you want to use the library, use pip: pi

Comments
  • ISO 8601

    ISO 8601

    Metadata daytime timestamp should be ISO 8601 compliant to reduce ambiguity:

    https://github.com/ika-rwth-aachen/omega_format/blob/745f67d774d2da04201de9fe24fa24468a8b191b/omega_format/meta_data.py#L81 ->

    daytime=datetime.strptime(cls.assure_string(dt), '%Y-%m-%dT%H:%M:%S.%f%z') if dt is not None else None,
    
    opened by kai-storms 0
  • Typo in FlatMarkingType enum

    Typo in FlatMarkingType enum

    FlatMarkingType.Plain should probably be FlatMarkingType.Plane. This is clear from the documentation anyhow, but just to avoid confusion ;-)

    https://github.com/ika-rwth-aachen/omega_format/blob/4bd733044128ea0008bd495cfb077d831600a4c2/omega_format/enums/reference_types.py#L217

    opened by lu-w 0
Releases(v4.0)
Owner
Institut für Kraftfahrzeuge, RWTH Aachen, ika
Institut für Kraftfahrzeuge, RWTH Aachen, ika
Control-Alt-Delete - Help Tux Escape Beastie's Jail!

Control-Alt-Delete Help Tux escape Beastie's jail by completing the following challenges! Challenges Challenge 00: Drinks: Tux needs to drink less. Ch

NDLUG 8 Oct 31, 2021
Trying to replicate (albeit unsuccessfully) the phenomenon of boids using Ursina in a naive manner.

Boids_Boi Trying to replicate (albeit unsuccessfully) the phenomenon of boids using Ursina in a naive manner. Please install the Ursina module before

2 Oct 19, 2021
A simple, console based nHentai Code Generator

nHentai Code Generator A simple, console based nHentai Code Generator. How to run? Windows Android Windows Make sure you have python and git installed

5 Jun 02, 2022
Local backup made easy, with Python and shutil

KTBackup BETA Local backup made easy, with Python and shutil Features One-command backup and restore Minimalistic (only using stdlib) Convenient direc

kelptaken 1 Dec 27, 2021
Handy Tool to check the availability of onion site and to extract the title of submitted onion links.

This tool helps is to quickly investigate a huge set of onion sites based by checking its availability which helps to filter out the inactive sites and collect the site title that might helps us to c

Balaji 13 Nov 25, 2022
About Library for extract infomation from thai personal identity card.

ThaiPersonalCardExtract Library for extract infomation from thai personal identity card. imprement from easyocr and tesseract New Feature v1.3.2 🎁 In

ggafiled 26 Nov 15, 2022
✨ Un bot Twitter totalement fait en Python par moi, et en français.

Twitter Bot ❗ Un bot Twitter totalement fait en Python par moi, et en français. Il faut remplacer auth = tweepy.OAuthHandler(consumer_key, consumer_se

MrGabin 3 Jun 06, 2021
ColorController is a Pythonic interface for managing colors by english-language name and various color values.

ColorController.py Table of Contents Encode color data in various formats. 1.1: Create a ColorController object using a familiar, english-language col

Tal Zaken 2 Feb 12, 2022
Every 2 minutes, check for visa slots at VFS website

vfs-visa-slot-germany Every 2 minutes, check for visa slots at VFS website. If there are any, send a call and a message of the format: Sent from your

12 Dec 15, 2022
Audio Steganography is a technique used to transmit hidden information by modifying an audio signal in an imperceptible manner.

Audio Steganography Audio Steganography is a technique used to transmit hidden information by modifying an audio signal in an imperceptible manner. Ab

Karan Yuvraj Singh 1 Oct 17, 2021
Analyze metadata of your Python project.

Analyze metadata of your Python projects Setup: Clone repo py-m venv venv (venv) pip install -r requirements.txt specify the folders which you want to

Pedro Monteiro de Carvalho e Silva Prado 1 Nov 10, 2021
Use generator for range function

Use the generator for the range function! installation method: pip install yrange How to use: First import yrange in your application. You can then wo

1 Oct 28, 2021
This is a python table of data implementation with styles, colors

Table This is a python table of data implementation with styles, colors Example Table adapts to the lack of data Lambda color features Full power of l

Урядов Алексей 5 Nov 09, 2021
A clock app, which helps you with routine tasks.

Clock This app helps you with routine tasks. Alarm Clock Timer Stop Watch World Time (Which city you want) About me Full name: Matin Ardestani Age: 14

Matin Ardestani 13 Jul 30, 2022
Utility to add/remove licenses to/from source files

Utility to add/remove licenses to/from source files. Supports processing any combination of globs, files, and directories (recurse). Pruning options allow skipping non-licensing files.

Eduardo Ponce Mojica 2 Jan 29, 2022
Simple profile athena generator for Fortnite Private Servers.

Profile-Athena-Generator A simple profile athena generator for Fortnite Private Servers. This profile athena generrator features: Item variants Get al

Fevers 10 Aug 27, 2022
ZX Spectrum Utilities: (zx-spectrum-utils)

Here are a few utility programs that can be used with the zx spectrum. The ZX Spectrum is one of the first home computers from the early 1980s.

Graham Oakes 4 Mar 07, 2022
✨ Un pierre feuille ciseaux totalement fait en Python par moi, et en français.

Pierre Feuille Ciseaux ❗ Un pierre feuille ciseaux totalement fait en Python par moi. 🔮 Avec l'utilisation du module "random", j'ai pu faire un choix

MrGabin 3 Jun 06, 2021
✨ Voici un code en Python par moi, et en français qui permet d'exécuter du Javascript en Python.

JavaScript In Python ❗ Voici un code en Python par moi, et en français qui permet d'exécuter du Javascript en Python. 🔮 Une vidéo pour vous expliquer

MrGabin 4 Mar 28, 2022
PyGMT - A Python interface for the Generic Mapping Tools

PyGMT A Python interface for the Generic Mapping Tools Documentation (development version) | Contact | Try Online Why PyGMT? A beautiful map is worth

The Generic Mapping Tools (GMT) 564 Dec 28, 2022