The elegance of Airflow + the power of AWS

Overview

Orkestra

The elegance of Airflow + the power of AWS

Docs Codecov

PyPI PyPI - Downloads PyPI - License PyPI - Python Version GitHub issues Mentioned in Awesome CDK

examples/hello_orkestra.py

list: logger.info(item.dict()) return [item.dict()] * 10 @compose(model=Item, is_map_job=True, **default_args) def double_price(item: Item, context): item.price = item.price * 2 return item.dict() @compose(**default_args) def assert_false(event, context): assert False @compose(**default_args) def do_nothing(event, context): logger.info({"doing": "nothing"}) @compose(**default_args) def say_hello(event, context): return "hello, world" @compose(**default_args) def say_goodbye(event, context): return "goodbye" @compose(**default_args) def random_int(event, context): return random.randrange(100) @compose(**default_args) def random_float(event, context): return float(random_int(event, context)) dag() ">
import random
from typing import *
from uuid import uuid4

from aws_lambda_powertools import Logger, Tracer
from pydantic import BaseModel

from orkestra import compose
from orkestra.interfaces import Duration


def dag():
    (
        generate_item
        >> add_price
        >> copy_item
        >> double_price
        >> (do_nothing, assert_false)
        >> say_hello
        >> [random_int, random_float]
        >> say_goodbye
    )


class Item(BaseModel):
    id: str
    name: str
    price: Optional[float] = None

    @classmethod
    def random(cls):
        return cls(
            id=str(uuid4()),
            name=random.choice(
                [
                    "potato",
                    "moon rock",
                    "hat",
                ]
            ),
        )


logger = Logger()

tracer = Tracer()


default_args = dict(
    enable_powertools=True,
    timeout=Duration.seconds(6),
)


@compose(**default_args)
def generate_item(event, context):
    logger.info("generating random item")
    item = Item.random().dict()
    logger.info(item)
    tracer.put_metadata("GenerateItem", "SUCCESS")
    return item


@compose(model=Item, **default_args)
def add_price(item: Item, context):
    price = 3.14
    logger.info(
        "adding price to item",
        extra={
            "item": item.dict(),
            "price": price,
        },
    )
    item.price = price
    return item.dict()


@compose(model=Item, **default_args)
def copy_item(item: Item, context) -> list:
    logger.info(item.dict())
    return [item.dict()] * 10


@compose(model=Item, is_map_job=True, **default_args)
def double_price(item: Item, context):
    item.price = item.price * 2
    return item.dict()


@compose(**default_args)
def assert_false(event, context):
    assert False


@compose(**default_args)
def do_nothing(event, context):
    logger.info({"doing": "nothing"})


@compose(**default_args)
def say_hello(event, context):
    return "hello, world"


@compose(**default_args)
def say_goodbye(event, context):
    return "goodbye"


@compose(**default_args)
def random_int(event, context):
    return random.randrange(100)


@compose(**default_args)
def random_float(event, context):
    return float(random_int(event, context))


dag()

app.py

#!/usr/bin/env python3
from aws_cdk import core as cdk

from examples.hello_orkestra import generate_item


class HelloOrkestra(cdk.Stack):
    def __init__(self, scope, id, **kwargs):

        super().__init__(scope, id, **kwargs)

        generate_item.schedule(
            self,
            expression="rate(5 minutes)",
            state_machine_name="hello_orkestra",
        )


app = cdk.App()


app.synth()

state machine

Owner
Stephan Fitzpatrick
“Talk is cheap. Show me the code.”
Stephan Fitzpatrick
A telegram bot written in Python to fetch random SFW & NSFW anime images

Tsuzumi A telegram bot written in python to fetch both random SFW & NSFW Anime images using nekos.life & waifu.pics API Commands SFW Commands : /

Nisarga Adhikary 3 Oct 12, 2022
Telegram bot to stream videos in telegram Voice Chat for both groups and channels

Telegram bot to stream videos in telegram Voice Chat for both groups and channels. Supports live steams, YouTube videos and telegram media. Supports scheduling streams, recording and many more.

Akki ThePro 2 Sep 11, 2022
Command-line program to download videos from YouTube.com and other video sites

youtube-dl - download videos from youtube.com or other video platforms INSTALLATION DESCRIPTION OPTIONS CONFIGURATION OUTPUT TEMPLATE FORMAT SELECTION

youtube-dl 116.4k Jan 07, 2023
Yes, it's true :revolving_hearts: This repository has 301 stars.

Yes, it's true! Inspired by a similar repository from @RealPeha, but implemented using a webhook on AWS Lambda and API Gateway, so it's serv

511 Dec 30, 2022
Univerity-student oriented (lithuanian) discord bot

Univerity-student oriented (lithuanian) discord bot

3 Nov 30, 2021
Based on falcondai and fenhl's Python snowflake tool, but with documentation and simliarities to Discord.

python-snowflake-2 Based on falcondai and fenhl's Python snowflake tool, but with documentation and simliarities to Discord. Docs make_snowflake This

2 Mar 19, 2022
Slack bot to automatically delete yubisneeze / accidental yubikey presses

YubiSnooze Slack bot to automatically delete yubisneeze / accidental yubikey presses. It will search using the regex "[cbdefghijklnrtuv]{44}" and if t

Andrew MacPherson 3 Feb 09, 2022
Track player's stats, find out when they're online and grinding!

Hypixel Stats Tracker Track player's stats, find out when they're online and playing games! INFO Showcase Server: https://discord.gg/yY5qQHPar6 Suppor

4 Dec 18, 2022
Moderation By Pokemon Bot (Discord)

Moderation Bot By Pokémon Bot (Discord) Official Moderation Bot for Pokemon Bot functional and based in the Discord Server, the bot is written in Pyth

Aakash Manoj Agrawal 6 Jan 04, 2022
Cloud-native, data onboarding architecture for the Google Cloud Public Datasets program

Public Datasets Pipelines Cloud-native, data pipeline architecture for onboarding datasets to the Google Cloud Public Datasets Program. Overview Requi

Google Cloud Platform 109 Dec 30, 2022
Super simple anti-spam Discord bot

AutoAntiRaidBot Super simple anti-spam Discord bot. Will automatically kick any member with an account made under 1 day ago, and will ban any member w

Kainoa Kanter 6 Jun 27, 2022
This is a very easy to use tool developed in python that will search for free courses from multiple sites including youtube and enroll in the ones in which it can.

Free-Course-Hunter-and-Enroller This is a very easy to use tool developed in python that will search for free courses from multiple sites including yo

Zain 12 Nov 12, 2022
Chatbot with python code!

Chatbot Python Chatbot with python! How to Run Installation requirements. pip install -r requirements.txt Sample Chatbot The required files must be d

Mohammad Dori 3 Jul 15, 2022
A simple fun discord bot using discord.py that can post memes

A simple fun discord bot using discord.py * * Commands $commands - to see all commands $meme - for a random meme from the internet $cry - to make the

Dice Flip 2 Dec 20, 2021
Resources for the AMLD 2022 workshop "DevOps on AWS"

MLOPS on AWS | AMLD 2022 This repository contains all the resources necessary to follow along and reproduce the workshop "MLOps on AWS: a Hands-On Tut

xtream 8 Jun 16, 2022
trackbranch is a tool for developers that can be used to store collections of branches in the form of profiles.

trackbranch trackbranch is a tool for developers that can be used to store collections of branches in the form of profiles. This can be useful for sit

Kevin Morris 1 Oct 21, 2021
Riffdog Terraform scanner - finding 'things' in the Real World (aka AWS) which Terraform didn't put there.

riffdog Riffdog Terraform / Reality scanner - finding 'things' in the Real World which Terraform didn't put there. This project works by firstly loadi

Riffdog 4 Mar 23, 2020
Create a roles overview page for all Ansible roles/playbooks in Gitlab

ansible-create-roles-overview Overview The script ./create_roles_overview.py queries a Gitlab API for Ansible roles and playbooks. It will iterate ove

2 Oct 11, 2021
An advanced api client for python botters.

[ALPHA] pybotters An advanced api client for python botters. 📌 Description pybottersは仮想通貨botter向けのPythonライブラリです。複数取引所に対応した非同期APIクライアントであり、bot開発により素晴ら

261 Dec 31, 2022
Bot per la chat live del corso di sistemi operativi UniBO

cravattaBot TL;DR: Ho fatto un bot telegram per la chat del corso di sistemi. Indice Installazione e prerequisiti Prerequisiti Installazione Setup Con

Alessandro Frau 3 May 21, 2022