Using AWS's API Gateway + Lambda + Python to run a simple websocket application. For learning/testing

Overview

aws-simple-websocket

Using AWS's API Gateway + Lambda + Python to run a simple websocket application. For learning/testing. The AWS Resources seemed overly complex and were missing some critical gotchas in setting up a system like this.

Example terminal showing usage

Using the following for guidance:

Architecture

To keep things as basic as possible we're using a bare minimum of resources and CLI helpers where possible.

A client makes a connection via Websocket to an API Gateway V2. That gateway maintains a socket connection for us, and sends events to some sort of "integration" or handler. In our case, this will be a Lambda function that will handle the incoming socket events ($connect/$disconnect). It will handle messages sent from websocket clients, and to further expand this example, an outside data source via SNS topic. The API Gateway requires us to keep track of Connection IDs, so we can programmatically and precisely send messages to specific clients.

Using Terraform (in ./deployment) the following are created:

Architecture Diagram

  1. API Gateway V2 (Websocket) - The primary Websocket management service which holds sockets for connections and can hit a variety of AWS integrations

  2. Lambda - The main executor of business logic - where all our code will live

  3. S3 - A basic Key/Value store for our connections

  4. SNS - To demonstrate an external publisher, our Lambda function is also listening to an SNS Topic

Some additional resources are needed:

  1. CloudWatch - Logging for API Gateway and Lambda function with retention periods set by default

  2. IAM - Permissions to glue everything together

Deployment

This demo repo uses Terraform to manage cloud resources. These are all stored in the ./deployment repository. NOTE: Creating resources in AWS may incur charges to your account. Ensure you have billing alarms setup and understand AWS costs. This demo repo should cost almost nothing, however.

  1. Install Terraform
  2. Change to ./deployment directory
  3. Init Terraform (terraform init)
  4. It's best practice to use "Workspaces" to namespace resources in terraform for different environments, so create a dev workspace (terraform workspace new dev)
  5. Check if you need to enable API Gateway Logging in your current region. Feel free to set ./deployment/enf.tf:init_api_gw_logging_role to false if your account already has this setup
  6. Create the resources terraform apply
  7. Run /util/lambda-deploy dev to build and deploy the lambda code. There are no external dependencies, just boto3

Usage

Contained in ./util are a lot of small CLI scripts to allow us to interact with the system without bloating the core too much with things like static web pages for the app side.

  • lambda-build - builds a zip archive for deployment to Lambda

  • lambda-deploy - runs lambda-build and deploys the archive to the provided Lambda environment. Uses terraform to get the name of our Lambda function to deploy to

  • send-data [json] - Sends a JSON payload to connected clients via SNS Topic. Uses terraform to get the name of our SNS Topic

  • tail-logs - Watches logs for the API Gateway and Lambda function. Uses terraform to get the name of our the log groups

  • connect - Connect to the websocket using a small utility, websocat

Quick Example

  1. Deploy the stack using the above instructions.
  2. ./util/connect to connect and listen to the websocket
  3. Using a new terminal session, ./util/send-data '{"hello": "world"}'
  4. View that in your first terminal with connect running, you'll see {"hello": "world"}

Users can broadcast messages to other users as well, not just through SNS. With a connected client, send:

{"action": "broadcast", "message": "Good news, everyone!"}

You will then see: {"message": "Good news, everyone!"} in all the connected clients.

Improvements

  1. Move from print() to logging module, for the sake of keeping this really simple, I left print in there
Owner
Seth Miller
Seth Miller
Get Your Localhost Online - Ngrok Alternative

Get Your Localhost Online - Ngrok Alternative

Azimjon Pulatov 442 Jan 04, 2023
Python tutorial for implementing Oxylabs' Residential Proxies with AIOHTTP

Integrating Oxylabs' Residential Proxies with AIOHTTP Requirements for the Integration For the integration to work you'll need to install aiohttp libr

Oxylabs.io 6 Sep 14, 2022
Implementing Cisco Support APIs into NetBox

NetBox Cisco Support API Plugin NetBox plugin using Cisco Support APIs to gather EoX and Contract coverage information for Cisco devices. Compatibilit

Timo Reimann 23 Dec 21, 2022
Initial code of an A3C network

A3C-network Initial code of an A3C network Open the python file named as "APL452 Project Report2" The following libraries and packages have been insta

Ayush Tanwar 0 Jun 11, 2022
Desktop application for checking sites connection in a background mode

Site connectivity checker Desktop application for checking site connection in a background mode by sending ICMP messages. Problem and solution Usually

Karina Singatullina 26 Dec 19, 2022
Ultimate transformation library that supports validation, contexts and aiohttp.

Trafaret Ultimate transformation library that supports validation, contexts and aiohttp. Trafaret is rigid and powerful lib to work with foreign data,

Mikhail Krivushin 174 Nov 27, 2022
Easy to use gRPC-web client in python

pyease-grpc Easy to use gRPC-web client in python Tutorial This package provides a requests like interface to make calls to gRPC-Web servers.

Sudipto Chandra 4 Dec 03, 2022
The World Most Fastest Proxy Checker In Python, Maybe?!

The World's Most Fastest Proxy Checker In Python, Maybe?! Features Based on Python 3.7+ Save Valid Porixes into the custom file Multi-Thread Fully Asy

Cyber 4 Feb 10, 2022
It's a little project for change MAC address, for ethical hacking purposes

MACChangerPy It's a small project for MAC address change, for ethical hacking purposes, don't use it for bad purposes, any infringement will be your r

Erick Adriano Nunes da Silva 1 Mar 11, 2022
This Tool Help To Information gathering for domain name or ip address...

Owl-Eye This Tool Help To Information gathering for domain name or ip address... follow this command $apt update && upgrade $apt install python apt in

Black Owl 6 Nov 12, 2022
Dokumentasi belajar Network automation

Repositori belajar network automation dengan Docker, Python & GNS3 Using Frameworks and integrate with: Paramiko Netmiko Telnetlib CSV SFTP Netmiko, S

Daniel.Pepuho 3 Mar 15, 2022
A Python3 discord trojan, utilizing discord webhooks for sending information.

Vape-Lite-RAT A Python3 discord trojan, utilizing discord webhooks for sending information. What you do with this code / project / idea is non of my b

NightTab 12 Oct 15, 2022
This is an open project to maintain a list of domain names that serve YouTube ads

The YouTube ads blocklist project This is an open project to maintain a list of domain names that serve YouTube ads. The original project only produce

Evan Pratten 574 Dec 30, 2022
Tool for pretty printing and optimizing Lightning Network channels.

Suez Tool for pretty printing and optimizing Lightning Network channels. Installation Install poetry poetry install poetry run ./suez Channel fee poli

Pavol Rusnak 69 Nov 03, 2022
A pretty quick and simple interface to paramiko SFTP

A pretty quick and simple interface to paramiko SFTP. Provides multi-threaded routines with progress notifications for reliable, asynchronous transfers. This is a Python3 optimized fork of pysftp wit

14 Dec 21, 2022
A tool which is capable of scanning ports as TCP & UDP and detecting open and closed ones.

PortScanner Scan All Open Ports Of The Target IP. A tool which is capable of scanning ports as TCP & UDP and detecting open and closed ones. Clone fro

Msf Nmt 17 Nov 26, 2022
JF⚑can - Super fast port scanning & service discovery using Masscan and Nmap. Scan large networks with Masscan and use Nmap's scripting abilities to discover information about services. Generate report.

Description Killing features Perform a large-scale scans using Nmap! Allows you to use Masscan to scan targets and execute Nmap on detected ports with

377 Jan 03, 2023
A socket script to obtain chinese phones-sequence for any english word

Foreign Pronunciation Generator (English-Chinese) We provide a simple socket script for acquiring Chinese pronunciation of English words (phones in ai

Ephemeroptera 5 Jul 25, 2022
This is a zeep based SOAP client wrapper for simple communication with the Bricknode SOAP API.

This is a zeep based SOAP client wrapper for simple communication with the Bricknode SOAP API.

Nord Fondkommission AB 2 Dec 15, 2021
πŸ‘¨πŸΌβ€πŸ’» β€Žβ€Žβ€Žβ€β€ A customizable man-in-the-middle TCP proxy with out-of-the-box support for HTTP & HTTPS.

πŸ‘¨β€πŸ’» mitm A customizable man-in-the-middle TCP proxy with out-of-the-box support for HTTP & HTTPS. Installing pip install mitm Note that OpenSSL 1.1

Felipe 92 Jan 05, 2023