Tools for Optuna, MLflow and the integration of both.

Related tags

Machine Learningpythonmachine-learningloggingtransformershyperparameter-optimizationhyperparametermlflowoptuna
Overview

HPOflow - Sphinx DOC

DOC MIT License Contributor Covenant Python Version pypi
pytest status Static Code Checks status Build & Deploy Doc GitHub issues

Tools for Optuna, MLflow and the integration of both.

Detailed documentation with examples can be found here: Sphinx DOC

Table of Contents

Maintainers

One Conversation
This project is maintained by the One Conversation team of Deutsche Telekom AG.

The main components are:

Installation

HPOflow is available at the Python Package Index (PyPI). It can be installed with pip:

$ pip install hpoflow

Some additional dependencies might be necessary.

To use hpoflow.optuna_mlflow.OptunaMLflow:

$ pip install mlflow GitPython

To use hpoflow.optuna_transformers.OptunaMLflowCallback:

$ pip install mlflow GitPython transformers

To install all optional dependencies use:

$ pip install hpoflow[optional]

Support and Feedback

The following channels are available for discussions, feedback, and support requests:

Reporting Security Vulnerabilities

This project is built with security and data privacy in mind to ensure your data is safe. We are grateful for security researchers and users reporting a vulnerability to us, first. To ensure that your request is handled in a timely manner and non-disclosure of vulnerabilities can be assured, please follow the below guideline.

Please do not report security vulnerabilities directly on GitHub. GitHub Issues can be publicly seen and therefore would result in a direct disclosure.

Please address questions about data privacy, security concepts, and other media requests to the [email protected] mailbox.

Contribution

Our commitment to open source means that we are enabling - in fact encouraging - all interested parties to contribute and become part of our developer community.

Contribution and feedback is encouraged and always welcome. For more information about how to contribute, as well as additional contribution information, see our Contribution Guidelines.

Code of Conduct

This project has adopted the Contributor Covenant as our code of conduct. Please see the details in our Contributor Covenant Code of Conduct. All contributors must abide by the code of conduct.

Licensing

Copyright (c) 2021 Philip May, Deutsche Telekom AG
Copyright (c) 2021 Philip May
Copyright (c) 2021 Timothy Wolff-Piggott

Licensed under the MIT License (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License by reviewing the file LICENSE in the repository.

Comments
  • review README.md and CONTRIBUTING.md

    review README.md and CONTRIBUTING.md

    Review README.md and CONTRIBUTING.md

    • is there something missing? maybe compare with optuna and transformers
    • spelling
    • idiomatic english
    • consistency
    • correctness
    • links ok?
    • ...

    PS: The real documentation is still missing and a know issue.

    opened by PhilipMay 12
  • add typing in optuna_transformers

    add typing in optuna_transformers

    @twolffpiggott can you please tell me the type of this?

    https://github.com/telekom/HPOflow/blob/e2b0943218af419a79ce95e60b67c9a4c2477349/hpoflow/optuna_transformers.py#L47

    opened by PhilipMay 6
  • add `transformers.py`

    add `transformers.py`

    @twolffpiggott should we add this here or to an other project we open source?

    https://github.com/PhilipMay/mltb/blob/master/mltb/integration/transformers.py

    enhancement 
    opened by PhilipMay 6
  • Create Sphinx documentation page

    Create Sphinx documentation page

    • [x] setup
    • [x] make GH action
    • [x] setup page
    • [x] change styling to telekom style
    • switch to MD
    • [x] add more content
    • [x] link from README to page
    • [x] link from pypi to GH page
    • [x] add impressum
    • [x] remove strange mouse over image effect
    • add version info
    documentation 
    opened by PhilipMay 4
  • Problems with direct `_imports.check()` call

    Problems with direct `_imports.check()` call

    When the __init__.py imports OMLflowCallback the optuna_transformers.py script is executed. That executes the _imports.check() call which then throws an exception if transformers or mlflow is not installed. But that should be avoided.

    See here: https://github.com/telekom/HPOflow/blob/d1cce5cbc2a84634d1484a053286000dda05b681/hpoflow/optuna_transformers.py#L11-L17

    The solution would be to put the _imports.check() call into the constructor. But that is not possible because OMLflowCallback inherits from transformers.

    The only solution I have is to put OMLflowCallback into an factory function that creates an OMLflowCallback and does the _imports.check() in there.

    @twolffpiggott what do you think?

    bug 
    opened by PhilipMay 3
  • Flake8 ignore list for Black compatibility

    Flake8 ignore list for Black compatibility

    Flake8 raises a warning for "E203" when it encounters a Black decision to insert whitespace before : in slicing syntax.

    Black's behaviour is more correct here, so my suggestion is to add "E203" to the flake8 config ignore list.

    i.e. in setup.cfg:

    [flake8]
...
extend-ignore = E203
    opened by twolffpiggott 3
  • Simple Example?

    Simple Example?

    I don't understand how to use this package. Could you provide a basic example? I don't understand the import_structure and how it relates to importing the modules? Thanks

    opened by jmrichardson 2
  • WIP prefix in contrib file

    WIP prefix in contrib file

    Should this

    Create Work In Progress [WIP] pull requests only if you need clarification or an explicit review before you can continue your work item.

    be more like this

    Add a [WIP] prefix on your pull request name if you need clarification or an explicit review before you can continue your work item.

    documentation 
    opened by PhilipMay 2
Releases(0.1.4)
Owner
Telekom Open Source Software
published by Deutsche Telekom AG and partner companies
Telekom Open Source Software
GitHub Repository https://telekom.github.io/HPOflow/
Probabilistic programming framework that facilitates objective model selection for time-varying parameter models.

Time series analysis today is an important cornerstone of quantitative science in many disciplines, including natural and life sciences as well as eco

Christoph Mark 129 Dec 24, 2022
Anomaly Detection and Correlation library

luminol Overview Luminol is a light weight python library for time series data analysis. The two major functionalities it supports are anomaly detecti

LinkedIn 1.1k Jan 01, 2023
2D fluid simulation implementation of Jos Stam paper on real-time fuild dynamics, including some suggested extensions.

Fluid Simulation Usage Download this repo and store it in your computer. Open a terminal and go to the root directory of this folder. Make sure you ha

Mariana Ávalos Arce 5 Dec 02, 2022
Karate Club: An API Oriented Open-source Python Framework for Unsupervised Learning on Graphs (CIKM 2020)

Karate Club is an unsupervised machine learning extension library for NetworkX. Please look at the Documentation, relevant Paper, Promo Video, and Ext

Benedek Rozemberczki 1.8k Jan 03, 2023
Given the names and grades for each student in a class N of students, store them in a nested list and print the name(s) of any student(s) having the second lowest grade.

Hackerank-Nested-List Given the names and grades for each student in a class N of students, store them in a nested list and print the name(s) of any s

Sangeeth Mathew John 2 Dec 14, 2021
Auto updating website that tracks closed & open issues/PRs on scikit-learn/scikit-learn.

Repository Status for Scikit-learn Live webpage Auto updating website that tracks closed & open issues/PRs on scikit-learn/scikit-learn. Running local

Thomas J. Fan 6 Dec 27, 2022
Implementation of K-Nearest Neighbors Algorithm Using PySpark

KNN With Spark Implementation of KNN using PySpark. The KNN was used on two separate datasets (https://archive.ics.uci.edu/ml/datasets/iris and https:

Zachary Petroff 4 Dec 30, 2022
machine learning model deployment project of Iris classification model in a minimal UI using flask web framework and deployed it in Azure cloud using Azure app service

This is a machine learning model deployment project of Iris classification model in a minimal UI using flask web framework and deployed it in Azure cloud using Azure app service. We initially made th

Krishna Priyatham Potluri 73 Dec 01, 2022
Code base of KU AIRS: SPARK Autonomous Vehicle Team

KU AIRS: SPARK Autonomous Vehicle Project Check this link for the blog post describing this project and the video of SPARK in simulation and on parkou

Mehmet Enes Erciyes 1 Nov 23, 2021
Module is created to build a spam filter using Python and the multinomial Naive Bayes algorithm.

Naive-Bayes Spam Classificator Module is created to build a spam filter using Python and the multinomial Naive Bayes algorithm. Main goal is to code a

Viktoria Maksymiuk 1 Jun 27, 2022
A benchmark of data-centric tasks from across the machine learning lifecycle.

A benchmark of data-centric tasks from across the machine learning lifecycle.

61 Dec 28, 2022
Python package for causal inference using Bayesian structural time-series models.

Python Causal Impact Causal inference using Bayesian structural time-series models. This package aims at defining a python equivalent of the R CausalI

Thomas Cassou 219 Dec 11, 2022
MLR - Machine Learning Research

Machine Learning Research 1. Project Topic 1.1. Exsiting research Benmark: https://paperswithcode.com/sota ACL anthology for NLP papers: http://www.ac

Charles 69 Oct 20, 2022
TIANCHI Purchase Redemption Forecast Challenge

TIANCHI Purchase Redemption Forecast Challenge

Haorui HE 4 Aug 26, 2022
A Python implementation of FastDTW

fastdtw Python implementation of FastDTW [1], which is an approximate Dynamic Time Warping (DTW) algorithm that provides optimal or near-optimal align

tanitter 651 Jan 04, 2023
Titanic Traveller Survivability Prediction

The aim of the mini project is predict whether or not a passenger survived based on attributes such as their age, sex, passenger class, where they embarked and more.

John Phillip 0 Jan 20, 2022
scikit-learn is a python module for machine learning built on top of numpy / scipy

About scikit-learn is a python module for machine learning built on top of numpy / scipy. The purpose of the scikit-learn-tutorial subproject is to le

Gael Varoquaux 122 Dec 12, 2022
Programming assignments and quizzes from all courses within the Machine Learning Engineering for Production (MLOps) specialization offered by deeplearning.ai

Machine Learning Engineering for Production (MLOps) Specialization on Coursera (offered by deeplearning.ai) Programming assignments from all courses i

Aman Chadha 173 Jan 05, 2023
Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

Petastorm Contents Petastorm Installation Generating a dataset Plain Python API Tensorflow API Pytorch API Spark Dataset Converter API Analyzing petas

Uber Open Source 1.6k Dec 31, 2022
Simple and flexible ML workflow engine.

This is a simple and flexible ML workflow engine. It helps to orchestrate events across a set of microservices and create executable flow to handle requests. Engine is designed to be configurable wit

Katana ML 295 Jan 06, 2023