A Python library for reading, writing and visualizing the OMEGA Format

Overview

OMEGA Format - Python Library

This module is developed by ika - RWTH Aachen as a contribution to the VVM project which aims to develop test procedures and to provide frameworks and methods for the safety verification of automated vehicles. VVM is working on the use case of Urban Intersections and focuses on driving functions up to full automation of vehicles (SAE Level 4 and 5).

As part of the project a data format for storing reference and perception data from pilotings, test drives and simulation in urban traffic is developed. This module enables the creation, reading and visualization of data conforming to this data format. Additionally, it can check files for conformance and perform basic sanity checks on the data.

Data Format

The base of both, the reference data format and the perception data format is the HDF5 file format. This library utilizes h5py to interact with those.

Reference Data

The OMEGA Format reference recording format is used to store data that represents the 'true' state of road users, infrastructure information, weather and more during a piloting, testing or simulation. The representation is on an object list basis. The following diagram shows an overview of the hierarchy in the OMEGA Format reference recording format. A more detailed description can be found in the specification document and the signal list.

Perception Data

The PerceptionRecording format is used to store data that represents what a vehicle under test, sensor under test or similar perceives from its surroundings. It is designed to be compared against the ReferenceRecording format. The following diagram shows an overview of the hierarchy in the PerceptionRecording format. A more detailed description is coming soon.

Installation

The dependencies are managed with conda environments. Conda can be installed following the conda installation instructions.

To create a new conda environment omega_env and install the module run the following in your console:

conda env create -n omega_env -f environment_visualization.yml
conda activate omega_env

If you want an editable install (modifications to the files in the directory are immediately used by the module) run:

pip install -e .[visualization]

To update the existing environment upon addition of new dependencies:

conda env update -n omega_env -f environment_visualization.yml

Usage

Validate your data

To check if your file conforms to the specification run.

omega_format verify --reference <FILENAME>

or

omega_format verify --perception <FILENAME>

When using the library and creating objects or rading from an hdf5 file, by default, sanity checks are performed. To circumvent those pass validate=False to the from_hdf5 function or use or use cls.construct instead of cls as the constructor of an object. In the backend pydantic is used for the sanity checks.

Visualize your data

To visualize a data file execute the following in your conda environment:

omega_format visualize --snip --max-snippets=2 <FILENAME>

or in Python:

import omega_format
from omega_format.visualization import Visualizer, SnippetContainer
reference_recording = omega_format.ReferenceRecording.from_hdf5('path/to/the/reference_recording_file.hdf5')
visualizer = Visualizer(SnippetContainer.create_list(references=reference_recording))
visualizer.start_gui_and_visualization()

A window will open that lets you interact with and inspect your data.

Extending the visualizer

By subclassing omega_format.visualization.VisualizationModule and adding an instance of your subclass to the visualizers list of the Visualizer you can extend the functionality of the visualizer. The subclass has to implement at least one of the functions visualize_static and visualize_dynamics, returning a list of pyqt widgets to plot. For more details take a look at the omega_format.vis.VisualizationModule or the other modules defined in the visualization.modules directory.

Create a reference data

This module maps the reference and perception data file specifications to a hierarchy of python classes. The root classes are ReferenceRecording and PerceptionRecording respectively. First initializing an object from that class and fill its properties with the objects of the classes in question (e.g. Weather RoadUser, Lane). After adding all your data, call to_hdf5 on the ReferenceRecording or PerceptionRecording and a format compliant hdf5 file will be created for you.

import numpy as np
import omega_format
from datetime import datetime

rr = omega_format.ReferenceRecording(meta_data=omega_format.MetaData(recorder_number=1,
                                                                     recording_number=1,
                                                                     daytime=datetime.now()
                                                                     reference_point_lat=50.786687,
                                                                     reference_point_lon=6.046312),
                               timestamps=omega_format.Timestamps(val=np.array([0])),
                               )
rr.weather = omega_foramt.Weather()
rr.road_users[0] = omega_format.RoadUser(type=omega_format.ReferenceTypes.RoadUser.Type.CAR, sub_type=omega_format.ReferenceTypes.RoadUser.SubType.General.REGULAR,
                                    birth=0, bb=omega_format.BoundingBox(np.array([2,3,0])),
                                    tr=omega_format.Trajectory(pos_x=np.array([0]),pos_y=np.array([0]),pos_z=np.array([0]),
                                                          roll=np.array([0]),pitch=np.array([0]),heading=np.array([0])))
rr.roads[0] = omega_format.Road(location=omega_format.ReferenceTypes.RoadLocation.URBAN)
rr.to_hdf5('test.hdf5')

Further Help

Standalone viewer of hdf5 files

There are plenty of tools, e.g.

Documentation

You can create a documentation with pdoc3. To do this first install pdoc3 with pip install pdoc3 and then run pdoc3 --http localhost:8889 --template-dir .\doc\templates\ .\omega_format from the root of this repo to view the documentation in your web browser.

License

The library is published under the MIT license specified in LICENSE. An overview over the licenses of the dependencies in this library is listed in LICENSES_OF_REQUIREMENTS.md.

Contact

In case of questions regarding the format, this repository or otherwise related feel free to raise an issue or contact Michael Schuldes ([email protected]).

Acknowledgement

The research leading to these results is funded by the German Federal Ministry for Economic Affairs and Energy within the project “Verifikations- und Validierungsmethoden automatisierter Fahrzeuge im urbanen Umfeld". The authors would like to thank the consortium for the successful cooperation.

bmwi_logo

You might also like...
Script to generate a massive volume of data in sql, csv, json or xml format

DataGenerator Made with Python Open for pull requests 1. Dependencies To install required dependencies run pip install -r requirements.txt 2. Executi

Export watched content from Tautulli to the Letterboxd CSV Import Format

Export watched content from Tautulli to the Letterboxd CSV Import Format

🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.

Boltons boltons should be builtins. Boltons is a set of over 230 BSD-licensed, pure-Python utilities in the same spirit as — and yet conspicuously mis

isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type.
isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type.

isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type. It provides a command line utility, Python library and plugins for various editors to quickly sort all your imports.

ticktock is a minimalist library to view Python time performance of Python code.

ticktock is a minimalist library to view Python time performance of Python code.

RapidFuzz is a fast string matching library for Python and C++

RapidFuzz is a fast string matching library for Python and C++, which is using the string similarity calculations from FuzzyWuzzy

pydsinternals - A Python native library containing necessary classes, functions and structures to interact with Windows Active Directory.
pydsinternals - A Python native library containing necessary classes, functions and structures to interact with Windows Active Directory.

pydsinternals - Directory Services Internals Library A Python native library containing necessary classes, functions and structures to interact with W

Library for processing molecules and reactions in python way

Chython [ˈkʌɪθ(ə)n] Library for processing molecules and reactions in python way. Features: Read/write/convert formats: MDL .RDF (.RXN) and .SDF (.MOL

A simple gpsd client and python library.

gpsdclient A small and simple gpsd client and library Installation Needs Python 3 (no other dependencies). If you want to use the library, use pip: pi

Comments
  • ISO 8601

    ISO 8601

    Metadata daytime timestamp should be ISO 8601 compliant to reduce ambiguity:

    https://github.com/ika-rwth-aachen/omega_format/blob/745f67d774d2da04201de9fe24fa24468a8b191b/omega_format/meta_data.py#L81 ->

    daytime=datetime.strptime(cls.assure_string(dt), '%Y-%m-%dT%H:%M:%S.%f%z') if dt is not None else None,
    
    opened by kai-storms 0
  • Typo in FlatMarkingType enum

    Typo in FlatMarkingType enum

    FlatMarkingType.Plain should probably be FlatMarkingType.Plane. This is clear from the documentation anyhow, but just to avoid confusion ;-)

    https://github.com/ika-rwth-aachen/omega_format/blob/4bd733044128ea0008bd495cfb077d831600a4c2/omega_format/enums/reference_types.py#L217

    opened by lu-w 0
Releases(v4.0)
Owner
Institut für Kraftfahrzeuge, RWTH Aachen, ika
Institut für Kraftfahrzeuge, RWTH Aachen, ika
This is discord nitro code generator and checker made with python. This will generate nitro codes and checks if the code is valid or not. If code is valid then it will print the code leaving 2 lines and if not then it will print '*'.

Discord Nitro Generator And Checker ⚙️ Rᴜɴ Oɴ Rᴇᴘʟɪᴛ 🛠️ Lᴀɴɢᴜᴀɢᴇs Aɴᴅ Tᴏᴏʟs If you are taking code from this repository without a fork, then atleast

Vɪɴᴀʏᴀᴋ Pᴀɴᴅᴇʏ 37 Jan 07, 2023
Python Random Number Genrator

This Genrates Random Numbers. This Random Number Generator was made using python. This software uses Time and Random extension. Download the EXE file and run it to get your answer.

Krish Sethi 2 Feb 03, 2022
✨ Un chois aléatoire d'un article sur Wikipedia totalement fait en Python par moi, et en français.

Wikipedia Random Article ❗ Un chois aléatoire d'un article sur Wikipedia totalement fait en Python par moi, et en français. 🔮 Grâce a une requète a w

MrGabin 4 Jul 18, 2021
Dill_tils is a package that has my commonly used functions inside it for ease of use.

DilllonB07 Utilities Dill_tils is a package that has my commonly used functions inside it for ease of use. Installation Anyone can use this package by

Dillon Barnes 2 Dec 05, 2021
Python tool to check a web applications compliance with OWASP HTTP response headers best practices

Check Your Head A quick and easy way to check a web applications response headers!

Zak 6 Nov 09, 2021
Homebase Name Changer for Fortnite: Save the World.

Homebase Name Changer This program allows you to change the Homebase name in Fortnite: Save the World. How to use it? After starting the HomebaseNameC

PRO100KatYT 7 May 21, 2022
pydsinternals - A Python native library containing necessary classes, functions and structures to interact with Windows Active Directory.

pydsinternals - Directory Services Internals Library A Python native library containing necessary classes, functions and structures to interact with W

Podalirius 36 Dec 14, 2022
isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type.

isort is a Python utility / library to sort imports alphabetically, and automatically separated into sections and by type. It provides a command line utility, Python library and plugins for various e

Python Code Quality Authority 5.5k Jan 08, 2023
kawadi is a versatile tool that used as a form of weapon and is used to cut, shape and split wood.

kawadi kawadi (કવાડિ in Gujarati) (Axe in English) is a versatile tool that used as a form of weapon and is used to cut, shape and split wood. kawadi

Jay Vala 2 Jan 10, 2022
Bounding Boxes Python Utils

Bounding Boxes Python Utils

Vadim 4 May 01, 2022
A python script to generate wallpaper

wallpaper eits Warning You need to set the path to Robot Mono font in the source code. (Settings are in the main function) Usage A script that given a

Henrique Tsuyoshi Yara 5 Dec 02, 2021
An awesome tool to save articles from RSS feed to Pocket automatically.

RSS2Pocket An awesome tool to save articles from RSS feed to Pocket automatically. About the Project I used to use IFTTT to save articles from RSS fee

Hank Liao 10 Nov 12, 2022
Python Classes Without Boilerplate

attrs is the Python package that will bring back the joy of writing classes by relieving you from the drudgery of implementing object protocols (aka d

The attrs Cabal 4.6k Jan 06, 2023
A python module to update the console without flashing.

A python module to update the console without flashing.

Matthias 112 Dec 19, 2022
A program to convert celcius to faranheit. made with python

Temp-Converter What is Temp-Converter Temp-Converter is little program made with pyhton to convert celcius to faranheit. Needed A python interpreter P

Chandula Janith 0 Nov 27, 2021
✨ Un juste prix totalement fait en Python par moi, et en français.

Juste Prix ❗ Un juste prix totalement fait en Python par moi, et en français. 🔮 Avec l'utilisation du module "random", j'ai pu faire un choix aléatoi

MrGabin 3 Jun 06, 2021
Pass arguments by reference—in Python!

byref Pass arguments by reference—in Python! byrefis a decorator that allows Python functions to declare reference parameters, with similar semantics

9 Feb 10, 2022
Aggregating gridded data (xarray) to polygons

A package to aggregate gridded data in xarray to polygons in geopandas using area-weighting from the relative area overlaps between pixels and polygons.

Kevin Schwarzwald 42 Nov 09, 2022
Protect your eyes from eye strain using this simple and beautiful, yet extensible break reminder

Protect your eyes from eye strain using this simple and beautiful, yet extensible break reminder

Gobinath 1.2k Jan 01, 2023
✨ Un code pour voir les disponibilités des vaccins contre le covid totalement fait en Python par moi, et en français.

Vaccine Notifier ❗ Un chois aléatoire d'un article sur Wikipedia totalement fait en Python par moi, et en français. 🔮 Grâce a une requète API, on peu

MrGabin 3 Jun 06, 2021