Get some python in google cloud functions

Overview

PyPI version

[NOTE]: This is a highly experimental (and proof of concept) library so do not expect all python packages to work flawlessly. Also, cloud functions are now (Summer 2018) rolling out native support for python3 in EAP so that also might be an option, check out the #functions channel on googlecloud-community.slack.com where the product managers hang around and open to help you out!

cloud-functions-python

py-cloud-fn is a CLI tool that allows you to write and deploy Google cloud functions in pure python, supporting python 2.7 and 3.5 (thanks to @MitalAshok for helping on the code compatibility). No javascript allowed! The goal of this library is to be able to let developers write light weight functions in idiomatic python without needing to worry about node.js. It works OOTB with pip, just include a file named requirements.txt that is structured like this:

pycloudfn==0.1.206
jsonpickle==0.9.4

as you normally would when building any python application. When building (for production), the library will pick up this file and make sure to install the dependencies. It will do so while caching all dependencies in a virtual environment, to speed up subsequent builds.

TLDR, look at the examples

Run pip install pycloudfn to get it. You need to have Google cloud SDK installed, as well as the Cloud functions emulator and npm if you want to test your function locally.

You also need Docker installed and running as well as the gcloud CLI. Docker is needed to build for the production environment, regardless of you local development environment.

Currently, http, pubsub and bucket events are supported (no firebase).

Usage

CLI

usage: py-cloud-fn [-h] [-p] [-f FILE_NAME] [--python_version {2.7,3.5,3.6}]
                   function_name {http,pubsub,bucket}

Build a GCP Cloud Function in python.

positional arguments:
  function_name         the name of your cloud function
  {http,pubsub,bucket}  the trigger type of your cloud function

optional arguments:
  -h, --help            show this help message and exit
  -p, --production      Build function for production environment
  -i, --production_image
                        Docker image to use for building production environment
  -f FILE_NAME, --file_name FILE_NAME
                        The file name of the file you wish to build
  --python_version {2.7,3.5}
                        The python version you are targeting, only applies
                        when building for production

Usage is meant to be pretty idiomatic:

Run py-cloud-fn <function_name> <trigger_type> to build your finished function. Run with -h to get some guidance on options. The library will assume that you have a file named main.py if not specified.

The library will create a cloudfn folder wherever it is used, which can safely be put in .gitignore. It contains build files and cache for python packages.

$DJANGO_SETTINGS_MODULE=mysite.settings py-cloud-fn my-function http -f function.py --python_version 3.5

  _____                  _                 _         __
 |  __ \                | |               | |       / _|
 | |__) |   _ ______ ___| | ___  _   _  __| |______| |_ _ __
 |  ___/ | | |______/ __| |/ _ \| | | |/ _` |______|  _| '_ \
 | |   | |_| |     | (__| | (_) | |_| | (_| |      | | | | | |
 |_|    \__, |      \___|_|\___/ \__,_|\__,_|      |_| |_| |_|
         __/ |
        |___/

Function: my-function
File: function.py
Trigger: http
Python version: 3.5
Production: False

⠴    Building, go grab a coffee...
⠋    Generating javascript...
⠼    Cleaning up...

Elapsed time: 37.6s
Output: ./cloudfn/target/index.js

Dependencies

This library works with pip OOTB. Just add your requirements.txt file in the root of the repo and you are golden. It obviously needs pycloudfn to be present.

Authentication

Since this is not really supported by google, there is one thing that needs to be done to make this work smoothly: You can't use the default clients directly. It's solvable though, just do

from cloudfn.google_account import get_credentials

biquery_client = bigquery.Client(credentials=get_credentials())

And everything is taken care off for you!! no more actions need be done.

Handling a http request

Look at the Request object for the structure

from cloudfn.http import handle_http_event, Response


def handle_http(req):
      return Response(
        status_code=200,
        body={'key': 2},
        headers={'content-type': 'application/json'},
    )


handle_http_event(handle_http)

If you don't return anything, or return something different than a cloudfn.http.Response object, the function will return a 200 OK with an empty body. The body can be either a string, list or dictionary, other values will be forced to a string.

Handling http with Flask

Flask is a great framework for building microservices. The library supports flask OOTB. If you need to have some routing / parsing and verification logic in place, flask might be a good fit! Have a look at the example to see how easy it is!

from cloudfn.flask_handler import handle_http_event
from cloudfn.google_account import get_credentials
from flask import Flask, request
from flask.json import jsonify
from google.cloud import bigquery

app = Flask('the-function')
biquery_client = bigquery.Client(credentials=get_credentials())


@app.route('/',  methods=['POST', 'GET'])
def hello():
    print request.headers
    return jsonify(message='Hello world!', json=request.get_json()), 201


@app.route('/lol')
def helloLol():
    return 'Hello lol!'


@app.route('/bigquery-datasets',  methods=['POST', 'GET'])
def bigquery():
    datasets = []
    for dataset in biquery_client.list_datasets():
        datasets.append(dataset.name)
    return jsonify(message='Hello world!', datasets={
        'datasets': datasets
    }), 201


handle_http_event(app)

Handling http with Django

Django is a great framework for building microservices. The library supports django OOTB. Assuming you have setup your django application in a normal fashion, this should be what you need. You need to setup a pretty minimal django application (no database etc) to get it working. It might be a little overkill to squeeze django into a cloud function, but there are some pretty nice features for doing request verification and routing in django using for intance django rest framework.

See the example for how you can handle a http request using django.

from cloudfn.django_handler import handle_http_event
from mysite.wsgi import application


handle_http_event(application)

Handling a bucket event

look at the Object for the structure, it follows the convention in the Storage API

from cloudfn.storage import handle_bucket_event
import jsonpickle


def bucket_handler(obj):
    print jsonpickle.encode(obj)


handle_bucket_event(bucket_handler)

Handling a pubsub message

Look at the Message for the structure, it follows the convention in the Pubsub API

from cloudfn.pubsub import handle_pubsub_event
import jsonpickle


def pubsub_handler(message):
    print jsonpickle.encode(message)


handle_pubsub_event(pubsub_handler)

Deploying a function

I have previously built go-cloud-fn, in which there is a complete CLI available for you to deploy a function. I did not want to go there now, but rather be concerned about building the function and be super light weight. Deploying a function can be done like this:

(If you have the emulator installed, just swap gcloud beta functions with npm install && functions and you are golden!).

HTTP

py-cloud-fn my-function http --production && \
cd cloudfn/target && gcloud beta functions deploy my-function \
--trigger-http --stage-bucket <bucket> && cd ../..

Storage

py-cloud-fn  my-bucket-function bucket -p && cd cloudfn/target && \
gcloud beta functions deploy my-bucket-function --trigger-bucket \
<trigger-bucket> --stage-bucket <stage-bucket> && cd ../..

Pubsub

py-cloud-fn my-topic-function bucket -p && cd cloudfn/target && \
gcloud beta functions deploy my-topic-function --trigger-topic <topic> \
--stage-bucket <bucket> && cd ../..

Adding support for packages that do not work

  • Look at the build output for what might be wrong.
  • Look for what modules might be missing.
  • Add a line-delimited file for hidden imports and a folder called cloudfn-hooks in the root of your repo, see more at Pyinstaller for how it works. Check out this for how to add hooks.

Troubleshooting

When things blow up, the first thing to try is to delete the cloudfn cache folder. Things might go a bit haywire when builds are interrupted or other circumstances. It just might save the day! Please get in touch at twitter if you bump into anything: @MartinSahlen

License

Copyright © 2017 Martin Sahlen

Distributed under the MIT License

Owner
Martin Abelson Sahlen
Engineer, Entrepreneur and hobby musician. Co-Founder @alvindotai
Martin Abelson Sahlen
A Discord Rich Presence App to set your own custom rich presence.

discord-rich-presence A Discord Rich Presence App to set your own custom rich presence. #BUILDS Ready to use package are available inside "finalpackag

1 Nov 22, 2021
Async client API for the Telegram Group Calls

PyTgCalls This project allow to make Telegram group call with MTProto Api using Pyrogram and WebRTC, this is possible thanks to the power of NodeJS's

185 Jan 03, 2023
♻️ API to run evaluations of the FAIR principles (Findable, Accessible, Interoperable, Reusable) on online resources

♻️ FAIR enough 🎯 An OpenAPI where anyone can run evaluations to assess how compliant to the FAIR principles is a resource, given the resource identif

Maastricht University IDS 4 Oct 20, 2022
Discord bot that displays the current Swatch Internet Time (.beat) as a status.

Internet-Time-Display Discord bot that displays the current Swatch Internet Time (.beat) as a status. Visit the website! Add the bot to your server! A

2 Mar 15, 2022
Bot made by BLACKSTORM[BM] Contact Us - t.me/BLACKSTORM18

ᴡʜᴀᴛ ɪs ᴊᴀʀᴠɪs sᴇᴄᴜʀɪᴛʏ ʙᴏᴛ ᴊᴀʀᴠɪs ʙᴏᴛ ɪs ᴛᴇʟᴇɢʀᴀᴍ ɢʀᴏᴜᴘ ᴍᴀɴᴀɢᴇʀ ʙᴏᴛ ᴡɪᴛʜ ᴍᴀɴʏ ғᴇᴀᴛᴜʀᴇs. ᴛʜɪs ʙᴏᴛ ʜᴇʟᴘs ʏᴏᴜ ᴛᴏ ᴍᴀɴᴀɢᴇ ʏᴏᴜʀ ɢʀᴏᴜᴘs ᴇᴀsɪʟʏ. ᴏʀɪɢɪɴᴀʟʟʏ ᴀ

1 Dec 11, 2021
Python SDK for IEX Cloud

iexfinance Python SDK for IEX Cloud. Architecture mirrors that of the IEX Cloud API (and its documentation). An easy-to-use toolkit to obtain data for

Addison Lynch 640 Jan 07, 2023
Best badge generator API to count visitors of your Repository / Account 🥇

github visitors badge A badge generator service to count visitors of your markdown file. Hello every one! In this post, I will tell you the story of m

Sᴇɴᴜ Gᴀᴍᴇʀ Bᴏʏ 〽 3 Dec 11, 2021
🦊 Powerfull Discord Nitro Generator

🦊 Follow me here 🦊 Discord | YouTube | Github ☕ Usage 💻 Downloading git clone https://github.com/KanekiWeb/Nitro-Generator/new/main pip insta

Kaneki 104 Jan 02, 2023
Want to play What Would Rather on your Server? Invite the bot now! 😏

What is this Bot? 👀 What You Would Rather? is a Guessing game where you guess one thing. Long Description short Take this example: You typed r!rather

FSP Gang s' YT 3 Oct 18, 2021
Получение интересной информации о любой пиццерии Додо

dodopizza-abuse Получение инфорации о выбранной пиццерии Додо Установка и запуск на Linux Устанавливаем git и python: apt-get update && apt-get -y ins

Хозя 24 Nov 02, 2022
Discord bot for Shran development

shranbot A discord bot named Herbert West that will monitor the Shran development discord server. Using dotenv shranbot uses a .env file to load secre

Matt Williams 1 Jul 29, 2022
Instant messaging client in tkinter

Concord_client_tk Instant messaging client in tkinter Contributors : Ilade-s [https://github.com/Ilade-s] Doku [https://github.com/D0kuhebi] Descripti

Raphaël Merlet 2 Jun 15, 2022
A Phyton script working for stream twits from twitter by tweepy to mongoDB

twitter-to-mongo A python script that uses the Tweepy library to pull Tweets with specific keywords from Twitter's Streaming API, and then stores the

3 Feb 03, 2022
Github action for automatically determine the version for next release by using repository tags

This action will automatically determine the version for next release by using repository tags

Igor Gov 7 Oct 25, 2022
Python script to delete old / embarrassing tweets.

Delete Tweets Do you have hundreds of embarrassing tweets on your Twitter profile, that you tweeted over a decade ago as an innocent high schooler, th

Linda Zheng 9 Nov 26, 2022
un outil pour bypasser les code d'états HTTP négatif coté client ( 4xx )

4xxBypasser un outil pour bypasser les code d'états HTTP négatif coté client ( 4xx ) Liscence : MIT license Creator Installation : git clone https://g

21 Dec 25, 2022
Discord Rpc With Python And 2 Buttons

Discord-RPC-With-Python- Discord Rpc With Python And 2 Buttons Packages pypresence time Required Programs Python Latest Version Random IDE Discord :P

Kaz 4 Dec 12, 2021
Telegram bot to download almost all from Instagram

Instagram Manager Bot The most advanced Instagram Downloader Bot. Please fork this repository don't import code Made with Python3 (C) @subinps Copyrig

SUBIN 300 Dec 30, 2022
Python client for ETAPI of Trilium Note.

Python client for ETAPI of Trilium Note.

33 Dec 31, 2022
ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

240 Jan 06, 2023