Yet Another Workflow Parser for SecurityHub

Related tags

Data Analysisyawps
Overview

YAWPS

Yet Another Workflow Parser for SecurityHub

"Screaming pepper" by Rum Bucolic Ape is licensed with CC BY-ND 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by-nd/2.0/

Purpose

Currently SecurityHub has a ChatBot integration thats a bit lacking. All of securityhub goes to chatbot, which means a singular flooding channel of alerts.

With cloud-custodians recent support for securityhub and organizations we have a good way to send all alerts for an entire org to slack. But that means every account goes to a single channel.

This repo is part of a multi-series talk/demo on how to intelligently route account messages to differing Slack channels.

In the scenario where a team owns an account it would be nice to let cloud-custodian generate meaningful securityhub notifications that go to specific team channels.

For this talk we will simply tag AWS accounts with 2 tags account_name (a human readable name) and slack_channel (a slack channel to direct those security hub notifications to).

A blog post and KubeCon talk will be coming soon with more information

Prerequisites

The only real pre-requisite here is a working multi-account SecurityHub

Configuration

Environment Variable Description
SLACK_FALLBACK_CHANNEL Channel to fallback to if the slack_channel tag is not provided on the account
SLACK_TOKEN the path in SSM to the slack token`
SLACK_TOKEN_SSM_PATH if a SLACK_TOKEN is not found, this is where to grab it from the EC2 Param store
LOGGING_LEVEL the logging level to use. Default is INFO
ENABLE_FORK_COPY_SEVERITY Enable the ability to fork some messages to another channel by severity. Value can be True or False. Default is False
FORK_COPY_SEVERITY_VALUE If ENABLE_FORK_COPY_SEVERITY is True, what severity level to fork by. Should be an integer between 0 and 100. Default is 90
ENABLE_FORK_ONLY_SEVERITY Enable the ability to fork some messages to only another channel by severity. Value can be True or False. Default is False
FORK_ONLY_SEVERITY_VALUE If ENABLE_FORK_ONLY_SEVERITY is True, what severity level to fork by. Should be an integer between 0 and 100. Default is 100

Forking

There are a few use cases for forking.

In general (all defaults) YAWPS will only send to the channel found in the tag or the SLACK_FALLBACK_CHANNEL (because it's required).

This is great until you have rules that you want a second team (lets say security) to also see and follow up with.

Using ENABLE_FORK_COPY_SEVERITY and FORK_COPY_SEVERITY_VALUE lets you also send to that second slack channel. Lets say you set FORK_COPY_SEVERITY_VALUE to 90. This means that anything rated 90 will send to both.

Another use-case exists: not sending team specific alerts. Lets say that an alert is not actionable by the configured team, but is purely for security visibility (like failed IAM logins etc). You can use ENABLE_FORK_ONLY_SEVERITY set to, say 100, in this scenario so that custom rules can set severity to 100 and send it only to security and bypass the primary team. This is good for noise filtration and helping to keep things actionable by a singular source.

Deploy

ServerLess

TODO

Terraform

  1. Download this repository (or a released artifact)
  2. Run make zip to produce a fully deployable s3 artifact
  3. Deploy something similar to this terraform

Testing

$ poetry install
$ poetry run tox
Owner
myoung34
Cloud security engineer, tinkerer, tomato farmer
myoung34
Karate Club: An API Oriented Open-source Python Framework for Unsupervised Learning on Graphs (CIKM 2020)

Karate Club is an unsupervised machine learning extension library for NetworkX. Please look at the Documentation, relevant Paper, Promo Video, and Ext

Benedek Rozemberczki 1.8k Jan 09, 2023
pyETT: Python library for Eleven VR Table Tennis data

pyETT: Python library for Eleven VR Table Tennis data Documentation Documentation for pyETT is located at https://pyett.readthedocs.io/. Installation

Tharsis Souza 5 Nov 19, 2022
A project consists in a set of assignements corresponding to a BI process: data integration, construction of an OLAP cube, qurying of a OPLAP cube and reporting.

TennisBusinessIntelligenceProject - A project consists in a set of assignements corresponding to a BI process: data integration, construction of an OLAP cube, qurying of a OPLAP cube and reporting.

carlo paladino 1 Jan 02, 2022
A set of procedures that can realize covid19 virus detection based on blood.

A set of procedures that can realize covid19 virus detection based on blood.

Nuyoah-xlh 3 Mar 07, 2022
.npy, .npz, .mtx converter.

npy-converter Matrix Data Converter. Expand matrix for multi-thread, multi-process Divid matrix for multi-thread, multi-process Support: .mtx, .npy, .

taka 1 Feb 07, 2022
Data collection, enhancement, and metrics calculation.

l3_data_collection Data collection, enhancement, and metrics calculation. Summary Repository containing code for QuantDAO's JDT data collection task.

Ruiwyn 3 Dec 23, 2022
Produces a summary CSV report of an Amber Electric customer's energy consumption and cost data.

Amber Electric Usage Summary This is a command line tool that produces a summary CSV report of an Amber Electric customer's energy consumption and cos

Graham Lea 12 May 26, 2022
Feature engineering and machine learning: together at last

Feature engineering and machine learning: together at last! Lambdo is a workflow engine which significantly simplifies data analysis by unifying featu

Alexandr Savinov 14 Sep 15, 2022
PyChemia, Python Framework for Materials Discovery and Design

PyChemia, Python Framework for Materials Discovery and Design PyChemia is an open-source Python Library for materials structural search. The purpose o

Materials Discovery Group 61 Oct 02, 2022
TE-dependent analysis (tedana) is a Python library for denoising multi-echo functional magnetic resonance imaging (fMRI) data

tedana: TE Dependent ANAlysis TE-dependent analysis (tedana) is a Python library for denoising multi-echo functional magnetic resonance imaging (fMRI)

136 Dec 22, 2022
PipeChain is a utility library for creating functional pipelines.

PipeChain Motivation PipeChain is a utility library for creating functional pipelines. Let's start with a motivating example. We have a list of Austra

Michael Milton 2 Aug 07, 2022
Pyspark Spotify ETL

This is my first Data Engineering project, it extracts data from the user's recently played tracks using Spotify's API, transforms data and then loads it into Postgresql using SQLAlchemy engine. Data

16 Jun 09, 2022
An orchestration platform for the development, production, and observation of data assets.

Dagster An orchestration platform for the development, production, and observation of data assets. Dagster lets you define jobs in terms of the data f

Dagster 6.2k Jan 08, 2023
We're Team Arson and we're using the power of predictive modeling to combat wildfires.

We're Team Arson and we're using the power of predictive modeling to combat wildfires. Arson Map Inspiration There’s been a lot of wildfires in Califo

Jerry Lee 3 Oct 17, 2021
Data cleaning tools for Business analysis

Datacleaning datacleaning tools for Business analysis This program is made for Vicky's work. You can use it, too. 数据清洗 该数据清洗工具是为了商业分析 这个程序是为了Vicky的工作而

Lin Jian 3 Nov 16, 2021
Using Python to scrape some basic player information from www.premierleague.com and then use Pandas to analyse said data.

PremiershipPlayerAnalysis Using Python to scrape some basic player information from www.premierleague.com and then use Pandas to analyse said data. No

5 Sep 06, 2021
An experimental project I'm undertaking for the sole purpose of increasing my Python knowledge

5ePy is an experimental project I'm undertaking for the sole purpose of increasing my Python knowledge. #Goals Goal: Create a working, albeit lightwei

Hayden Covington 1 Nov 24, 2021
Pipetools enables function composition similar to using Unix pipes.

Pipetools Complete documentation pipetools enables function composition similar to using Unix pipes. It allows forward-composition and piping of arbit

186 Dec 29, 2022
Full automated data pipeline using docker images

Create postgres tables from CSV files This first section is only relate to creating tables from CSV files using postgres container alone. Just one of

1 Nov 21, 2021
PandaPy has the speed of NumPy and the usability of Pandas 10x to 50x faster (by @firmai)

PandaPy "I came across PandaPy last week and have already used it in my current project. It is a fascinating Python library with a lot of potential to

Derek Snow 527 Jan 02, 2023