Automatically block traffic on Cloudflare's side based on Nginx Log parsing.

Overview

AutoRL

This is a PoC of automatically block traffic on Cloudflare's side based on Nginx Log parsing.

It will evaluate Nginx access.log and find potential CC pattern, and block them on Cloudflare's side and send a message to Telegram Group.

Topology

With Cloudflare Argo Tunnel, we can set security group to allow inbound traffic for SSH only, this can guarantee the Host's IP will not be exposed to the Internet (ref: 使用 Cloudflare Argo Tunnel(cloudflared) 来加速和保护你的网站), however, attackers can still CC your website by sending enormous requests cocurrently, AutoRL is here trying to mitigate this problem.

Prerequisites

Since this is only a PoC, the following condition must be met to use AutoRL.

  • Python 3 installed on Host

  • Nginx used for Reverse proxy and all the logs are logged into one access.log file.

  • Nginx has the following log format (in /etc/nginx/nginx.conf)

    log_format  main  '$remote_addr $time_iso8601 "$request" '
                        '$status $body_bytes_sent "$http_referer" '
                        '"$http_user_agent" "$http_x_forwarded_for"';
    

    On this condition, the raw log should look like this:

    172.70.211.101 2022-05-02T10:44:16+08:00 "GET /grafana/api/live/ws HTTP/1.1" 400 12 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5028.0 Safari/537.36" "145.xx.xx.xxx"
    

    Where, 172.70.211.101 is Cloudflare's IP, 2022-05-02T10:44:16+08:00 stands for request datetime and "145.xx.xx.xxx" is the real visitor IP.

Usage

  1. Download the autorl.py to your host

  2. Edit the following variable in the autorl.py

    • CF_EMAIL (Your Cloudflare login email)
    • CF_AUTH_KEY (Your Cloudflare Global API Key)
    • ACCESS_LOG_PATH (Default is /var/log/nginx/access.log)
    • INTERVAL_MIN (Default is 1, then this script will evaluate for 1min's traffic)
    • RATE_PER_MINUTE (How many requests are allowed for single IP, e,g, when this is set to 600 and INTERVAL_MIN is 1, then one IP can send at most 600 requests, after that, this IP will be blocked.)
    • TG_CHAT_ID (Your Telegram Chat Group ID)
    • TG_BOT_TOKEN (You should invite a bot to your group, and fillin the bot token here)
    • IP_WHITE_LIST (If you'd like to whitelist some IP, fillin here)
  3. Create a crontab for this script, example:

    * * * * * for i in {1..6}; do /usr/bin/python3 /path/to/autorl.py & sleep 10; done
    

Demo

On Telegram side:

On Cloudflare side:

Notes

  • Blocked IP address will never gets unblocked.
  • If logrotate is not setup correctly, then parsing the whole access.log might consume a lot of system resources.
  • The attack pattern/sample is not stored so we have no idea how the attack is conducted.
Owner
Nova Kwok
43EC 6073 0BFF A16C 34BB 9EF2 8D42 A0E6 99E5 0639
Nova Kwok
A web-based app that allows easy, simple - and if desired high-throughput - analysis of qPCR data

qpcr-Analyser A web-based GUI for the qpcr package that allows easy, simple and high-throughput analysis of qPCR data. As is described in more detail

1 Sep 13, 2022
Medusa is a cross-platform agent compatible with both Python 3.8 and Python 2.7.

Medusa Medusa is a cross-platform agent compatible with both Python 3.8 and Python 2.7. Installation To install Medusa, you'll need Mythic installed o

Mythic Agents 123 Nov 09, 2022
Free,Cross-platform,Single-file mass network protocol server simulator

FaPro Free,Cross-platform,Single-file mass network protocol server simulator 中文Readme Description FaPro is a Fake Protocol Server tool, Can easily sta

FOFA Pro 1.4k Jan 06, 2023
A repository dedicated to IoT(internet of things ) and python scripts

📑 Introduction Week of Learning is a weekly program in which you will get all the necessary knowledge about Circuit-Building, Arduino and Micro-Contr

27 Nov 22, 2022
This python script can change the mac address after some attack

MAC-changer Hello people, this python script was written for people who want to change the mac address after some attack, I know there are many ways t

5 Oct 10, 2022
批量检查目标是否为cdn

🐸 Frog For Automatic Scan 🐶 Doge For Defense Evasion&Offensive Security Frog-checkCDN 批量检查目标是否为cdn Usage: python3 checkCDN.py list.txt list内可以为ip或者d

TimWhite 119 Dec 27, 2022
NSX-T infrastructure as code - SDDC deployment

Deploy NSX-T Infrastructure - Simple Topology by Nicolas MICHEL @vpackets / LinkedIn Introduction The purpose of this entire repository is to automate

21 Nov 28, 2022
A Cheap Flight Alert program sends you a SMS to notify cheap flights in next 8 months.

Flight Dealer A Cheap Flight Alert program sends you a SMS to notify cheap flights (return trips) in next 6 months. Installing Download the Python 3 i

Aytaç Kaşoğlu 2 Feb 10, 2022
A gRPC-Web implementation for Python

Sonora Sonora is a Python-first implementation of gRPC-Web built on top of standard Python APIs like WSGI and ASGI for easy integration. Why? Regular

Alex Stapleton 216 Dec 30, 2022
CSP-style concurrency for Python

aiochan Aiochan is a library written to bring the wonderful idiom of CSP-style concurrency to python. The implementation is based on the battle-tested

Ziyang Hu 127 Dec 23, 2022
Flashes keyboard leds on incoming/outgoing network packets

LED Net Capture Flashes keyboard leds on incoming/outgoing network packets Usage Requires root priviledges to run usage: ledcapture.py [-h] --keyboard

Dan Habot 56 Oct 27, 2022
Web service load balancing simulation experiment.

Web service load balancing simulation experiment.

NicestZK 1 Nov 12, 2021
This Tool can help enginners and biggener in network, the tool help you to find of any ip with subnet mask that can calucate them and show you ( Availble IP's , Subnet Mask, Network-ID, Broadcast-ID )

This Tool can help enginners and biggener in network, the tool help you to find of any ip with subnet mask that can calucate them and show you ( Availble IP's , Subnet Mask, Network-ID, Broadcast-ID

12 Dec 13, 2022
Wifijammer - Continuously jam all wifi clients/routers

wifijammer Continuously jam all wifi clients and access points within range. The effectiveness of this script is constrained by your wireless card. Al

Dan McInerney 3.5k Dec 31, 2022
Synchronised text editor over TCP, for live editing with others.

SyncTEd Synchronised text editor over TCP, for live editing with others. Written in Python with PyGame. Run Install requirements: pip install -r requi

Marko Živić 1 May 13, 2022
A TrueCharts automatic and bulk update utility

trueupdate A TrueCharts automatic and bulk update utility How to install run pip install trueupdate Please be aware you will need to reinstall after e

TrueCharts 125 Jan 04, 2023
A transport agnostic sync/async RPC library that focuses on exposing services with a well-defined API using popular protocols.

WARNING: This is from spyne's development branch. This version is not released yet! Latest stable release can be found in the 2_13 branch. If you like

1.1k Dec 23, 2022
Simple Port Scanner With Socket Module In Python 3x

PortScanner Simple Port Scanner With Socket Module In Python 3x How To Install Requirements Of This Port Scanner sudo apt install python3;sudo apt ins

1 Nov 23, 2021
A Python script that alerts via SMS when a stock is reaching an inflection point

TradeAlert Not sure what this will ultimately become, but for now, its a Python script that alerts via SMS when a stock is reaching an inflection poin

3 Feb 22, 2022
Way find out if DNS is down or your instance

DNS-PING Way to find out if DNS is down or your instance Problem: At times it happens that DNS provider services of a website URL is down and so to re

Giten Mitra 4 Nov 18, 2022