Get some python in google cloud functions

Overview

PyPI version

[NOTE]: This is a highly experimental (and proof of concept) library so do not expect all python packages to work flawlessly. Also, cloud functions are now (Summer 2018) rolling out native support for python3 in EAP so that also might be an option, check out the #functions channel on googlecloud-community.slack.com where the product managers hang around and open to help you out!

cloud-functions-python

py-cloud-fn is a CLI tool that allows you to write and deploy Google cloud functions in pure python, supporting python 2.7 and 3.5 (thanks to @MitalAshok for helping on the code compatibility). No javascript allowed! The goal of this library is to be able to let developers write light weight functions in idiomatic python without needing to worry about node.js. It works OOTB with pip, just include a file named requirements.txt that is structured like this:

pycloudfn==0.1.206
jsonpickle==0.9.4

as you normally would when building any python application. When building (for production), the library will pick up this file and make sure to install the dependencies. It will do so while caching all dependencies in a virtual environment, to speed up subsequent builds.

TLDR, look at the examples

Run pip install pycloudfn to get it. You need to have Google cloud SDK installed, as well as the Cloud functions emulator and npm if you want to test your function locally.

You also need Docker installed and running as well as the gcloud CLI. Docker is needed to build for the production environment, regardless of you local development environment.

Currently, http, pubsub and bucket events are supported (no firebase).

Usage

CLI

usage: py-cloud-fn [-h] [-p] [-f FILE_NAME] [--python_version {2.7,3.5,3.6}]
                   function_name {http,pubsub,bucket}

Build a GCP Cloud Function in python.

positional arguments:
  function_name         the name of your cloud function
  {http,pubsub,bucket}  the trigger type of your cloud function

optional arguments:
  -h, --help            show this help message and exit
  -p, --production      Build function for production environment
  -i, --production_image
                        Docker image to use for building production environment
  -f FILE_NAME, --file_name FILE_NAME
                        The file name of the file you wish to build
  --python_version {2.7,3.5}
                        The python version you are targeting, only applies
                        when building for production

Usage is meant to be pretty idiomatic:

Run py-cloud-fn <function_name> <trigger_type> to build your finished function. Run with -h to get some guidance on options. The library will assume that you have a file named main.py if not specified.

The library will create a cloudfn folder wherever it is used, which can safely be put in .gitignore. It contains build files and cache for python packages.

$DJANGO_SETTINGS_MODULE=mysite.settings py-cloud-fn my-function http -f function.py --python_version 3.5

  _____                  _                 _         __
 |  __ \                | |               | |       / _|
 | |__) |   _ ______ ___| | ___  _   _  __| |______| |_ _ __
 |  ___/ | | |______/ __| |/ _ \| | | |/ _` |______|  _| '_ \
 | |   | |_| |     | (__| | (_) | |_| | (_| |      | | | | | |
 |_|    \__, |      \___|_|\___/ \__,_|\__,_|      |_| |_| |_|
         __/ |
        |___/

Function: my-function
File: function.py
Trigger: http
Python version: 3.5
Production: False

⠴    Building, go grab a coffee...
⠋    Generating javascript...
⠼    Cleaning up...

Elapsed time: 37.6s
Output: ./cloudfn/target/index.js

Dependencies

This library works with pip OOTB. Just add your requirements.txt file in the root of the repo and you are golden. It obviously needs pycloudfn to be present.

Authentication

Since this is not really supported by google, there is one thing that needs to be done to make this work smoothly: You can't use the default clients directly. It's solvable though, just do

from cloudfn.google_account import get_credentials

biquery_client = bigquery.Client(credentials=get_credentials())

And everything is taken care off for you!! no more actions need be done.

Handling a http request

Look at the Request object for the structure

from cloudfn.http import handle_http_event, Response


def handle_http(req):
      return Response(
        status_code=200,
        body={'key': 2},
        headers={'content-type': 'application/json'},
    )


handle_http_event(handle_http)

If you don't return anything, or return something different than a cloudfn.http.Response object, the function will return a 200 OK with an empty body. The body can be either a string, list or dictionary, other values will be forced to a string.

Handling http with Flask

Flask is a great framework for building microservices. The library supports flask OOTB. If you need to have some routing / parsing and verification logic in place, flask might be a good fit! Have a look at the example to see how easy it is!

from cloudfn.flask_handler import handle_http_event
from cloudfn.google_account import get_credentials
from flask import Flask, request
from flask.json import jsonify
from google.cloud import bigquery

app = Flask('the-function')
biquery_client = bigquery.Client(credentials=get_credentials())


@app.route('/',  methods=['POST', 'GET'])
def hello():
    print request.headers
    return jsonify(message='Hello world!', json=request.get_json()), 201


@app.route('/lol')
def helloLol():
    return 'Hello lol!'


@app.route('/bigquery-datasets',  methods=['POST', 'GET'])
def bigquery():
    datasets = []
    for dataset in biquery_client.list_datasets():
        datasets.append(dataset.name)
    return jsonify(message='Hello world!', datasets={
        'datasets': datasets
    }), 201


handle_http_event(app)

Handling http with Django

Django is a great framework for building microservices. The library supports django OOTB. Assuming you have setup your django application in a normal fashion, this should be what you need. You need to setup a pretty minimal django application (no database etc) to get it working. It might be a little overkill to squeeze django into a cloud function, but there are some pretty nice features for doing request verification and routing in django using for intance django rest framework.

See the example for how you can handle a http request using django.

from cloudfn.django_handler import handle_http_event
from mysite.wsgi import application


handle_http_event(application)

Handling a bucket event

look at the Object for the structure, it follows the convention in the Storage API

from cloudfn.storage import handle_bucket_event
import jsonpickle


def bucket_handler(obj):
    print jsonpickle.encode(obj)


handle_bucket_event(bucket_handler)

Handling a pubsub message

Look at the Message for the structure, it follows the convention in the Pubsub API

from cloudfn.pubsub import handle_pubsub_event
import jsonpickle


def pubsub_handler(message):
    print jsonpickle.encode(message)


handle_pubsub_event(pubsub_handler)

Deploying a function

I have previously built go-cloud-fn, in which there is a complete CLI available for you to deploy a function. I did not want to go there now, but rather be concerned about building the function and be super light weight. Deploying a function can be done like this:

(If you have the emulator installed, just swap gcloud beta functions with npm install && functions and you are golden!).

HTTP

py-cloud-fn my-function http --production && \
cd cloudfn/target && gcloud beta functions deploy my-function \
--trigger-http --stage-bucket <bucket> && cd ../..

Storage

py-cloud-fn  my-bucket-function bucket -p && cd cloudfn/target && \
gcloud beta functions deploy my-bucket-function --trigger-bucket \
<trigger-bucket> --stage-bucket <stage-bucket> && cd ../..

Pubsub

py-cloud-fn my-topic-function bucket -p && cd cloudfn/target && \
gcloud beta functions deploy my-topic-function --trigger-topic <topic> \
--stage-bucket <bucket> && cd ../..

Adding support for packages that do not work

  • Look at the build output for what might be wrong.
  • Look for what modules might be missing.
  • Add a line-delimited file for hidden imports and a folder called cloudfn-hooks in the root of your repo, see more at Pyinstaller for how it works. Check out this for how to add hooks.

Troubleshooting

When things blow up, the first thing to try is to delete the cloudfn cache folder. Things might go a bit haywire when builds are interrupted or other circumstances. It just might save the day! Please get in touch at twitter if you bump into anything: @MartinSahlen

License

Copyright © 2017 Martin Sahlen

Distributed under the MIT License

Owner
Martin Abelson Sahlen
Engineer, Entrepreneur and hobby musician. Co-Founder @alvindotai
Martin Abelson Sahlen
My Advent of Code solutions. I also upload videos of my solves: https://www.youtube.com/channel/UCuWLIm0l4sDpEe28t41WITA

My solutions to adventofcode.com puzzles. I post videos of me solving the puzzles in real-time at https://www.youtube.com/channel/UCuWLIm0l4sDpEe28t41

195 Jan 04, 2023
Upbit(업비트) Cryptocurrency Exchange OPEN API Client for Python

Base Repository Python Upbit Client Repository Upbit OPEN API Client @Author: uJhin @GitHub: https://github.com/uJhin/upbit-client/ @Officia

Yu Jhin 37 Nov 06, 2022
Quickly and efficiently delete your entire tweet history with the help of your Twitter archive without worrying about the pointless 3200 tweet limit imposed by Twitter.

Twitter Nuke Quickly and efficiently delete your entire tweet history with the help of your Twitter archive without worrying about the puny and pointl

Mayur Bhoi 73 Dec 12, 2022
A Telegram Bot to display Codeforces Contest Ranklist

CFRankListBot A bot that displays the top ranks for a Codeforces contest. Participants' Details All the details of a participant is in the utils/__ini

Code IIEST 5 Dec 25, 2021
Nonebot2 简易群管

简易群管 ✨ NoneBot2 简易群管 ✨ _ 踢 改 禁 欢迎issue pr 权限说明:permission=SUPERUSER 安装 💿 pip install nonebot-plugin-admin 导入 📲 在bot.py 导入,语句: nonebot.load_plugin("n

幼稚园园长 74 Dec 22, 2022
A mass account list editor for python

Account-List-Editor This is an mass account list editor Usage Run the editor.py file with python (python3 ./editor.py) Press a button (1/2) and drag &

ExtremeDev 1 Dec 20, 2021
This project is a basic login system in terminal for Discord

Welcome to Discord Login System(Terminal) 👋 This project is a basic login system in terminal for Discord Author 👤 arukovic Github: @SONIC-CODEZ Show

SONIC-CODEZ 2 Feb 11, 2022
BiliBili-live-barrage-transceiver - A simple python program for sending and receiving barrage in bilibili live room

BiliBili-live-barrage-transceiver - A simple python program for sending and receiving barrage in bilibili live room

zeroy 2 Jan 18, 2022
A hyper-user friendly bot framework built on hikari

Framework A hyper-user friendly bot framework built on hikari. Framework is based off the blocking discord library disco, In both modularity and struc

Vincent 1 Jan 10, 2022
BingBot - A bot that will automate searches on bing

bingBot A bot that will automate searches on bing. To install this just download

Lukas 2 Jul 28, 2022
The modern Lavalink wrapper designed for discord.py

Pomice The modern Lavalink wrapper designed for discord.py This library is heavily based off of/uses code from the following libraries: Wavelink Slate

Gstone 1 Feb 02, 2022
Collection of script to manage WLED devices

Collection of script to manage WLED devices

Daniel Poelzleithner 4 Sep 26, 2022
Recommendation systems are among most widely preffered marketing strategies.

Recommendation systems are among most widely preffered marketing strategies. Their popularity comes from close prediction scores obtained from relationships of users and items. In this project, two r

Sübeyte 8 Oct 06, 2021
Python implementation of Spotify's authorization flow.

Spotify API Apps 🎷 🎶 🎼 This repository consists of many strange codes that make you think why the hell this guy doing this. Well... I got some reas

5 Dec 17, 2021
Ts-matterbridge - Integrate TeamSpeak Chat with MatterBridge

TeamSpeak-MatterBridge Bot You can use this bot to integrate TeamSpeak Chat with

4 Sep 25, 2022
A tool that helps keeping track of your AWS quota utilization

aws-quota-checker A tool that helps keeping track of your AWS quota utilization. It'll determine the limits of your AWS account and compare them to th

Max 63 Dec 14, 2022
MassReportBot - Discord Mass Report Bot By Dropout

Discord Mass Report Bot By Dropout Discord Report Bot, Just Re-Made The "Admin R

vanis / 1800 0 Jan 20, 2022
An anime themed telegram bot that can convert telegram media.

ShoukoKomiRobot • 𝕎𝕣𝕚𝕥𝕥𝕖𝕟 𝕀𝕟 Python3 • 𝕃𝕚𝕓𝕣𝕒𝕣𝕪 𝕌𝕤𝕖𝕕 Pyrogram • 𝕊𝕠𝕗𝕥𝕨𝕒𝕣𝕖 𝕌𝕤𝕖𝕕 Ebook-convert Deploy 𝔽𝕠𝕣𝕜 𝕥𝕙𝕚𝕤 𝕣

25 Aug 14, 2022
Fortnite Dumper for anyone's Save the World profiles.

Anyone's Fortnite Save the World Profile Dumper This program allows you to dump anyone's Fortnite Save the World Profiles. How to use it? After starti

PRO100KatYT 6 Apr 13, 2022
Python wrapper for Interactive Brokers Client Portal Web API

EasyIB: Unofficial Wrapper for Interactive Brokers API EasyIB is an unofficial python wrapper for Interactive Brokers Client Portal Web API. Features

39 Dec 13, 2022