Automatically block traffic on Cloudflare's side based on Nginx Log parsing.

Overview

AutoRL

This is a PoC of automatically block traffic on Cloudflare's side based on Nginx Log parsing.

It will evaluate Nginx access.log and find potential CC pattern, and block them on Cloudflare's side and send a message to Telegram Group.

Topology

With Cloudflare Argo Tunnel, we can set security group to allow inbound traffic for SSH only, this can guarantee the Host's IP will not be exposed to the Internet (ref: 使用 Cloudflare Argo Tunnel(cloudflared) 来加速和保护你的网站), however, attackers can still CC your website by sending enormous requests cocurrently, AutoRL is here trying to mitigate this problem.

Prerequisites

Since this is only a PoC, the following condition must be met to use AutoRL.

  • Python 3 installed on Host

  • Nginx used for Reverse proxy and all the logs are logged into one access.log file.

  • Nginx has the following log format (in /etc/nginx/nginx.conf)

    log_format  main  '$remote_addr $time_iso8601 "$request" '
                        '$status $body_bytes_sent "$http_referer" '
                        '"$http_user_agent" "$http_x_forwarded_for"';
    

    On this condition, the raw log should look like this:

    172.70.211.101 2022-05-02T10:44:16+08:00 "GET /grafana/api/live/ws HTTP/1.1" 400 12 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5028.0 Safari/537.36" "145.xx.xx.xxx"
    

    Where, 172.70.211.101 is Cloudflare's IP, 2022-05-02T10:44:16+08:00 stands for request datetime and "145.xx.xx.xxx" is the real visitor IP.

Usage

  1. Download the autorl.py to your host

  2. Edit the following variable in the autorl.py

    • CF_EMAIL (Your Cloudflare login email)
    • CF_AUTH_KEY (Your Cloudflare Global API Key)
    • ACCESS_LOG_PATH (Default is /var/log/nginx/access.log)
    • INTERVAL_MIN (Default is 1, then this script will evaluate for 1min's traffic)
    • RATE_PER_MINUTE (How many requests are allowed for single IP, e,g, when this is set to 600 and INTERVAL_MIN is 1, then one IP can send at most 600 requests, after that, this IP will be blocked.)
    • TG_CHAT_ID (Your Telegram Chat Group ID)
    • TG_BOT_TOKEN (You should invite a bot to your group, and fillin the bot token here)
    • IP_WHITE_LIST (If you'd like to whitelist some IP, fillin here)
  3. Create a crontab for this script, example:

    * * * * * for i in {1..6}; do /usr/bin/python3 /path/to/autorl.py & sleep 10; done
    

Demo

On Telegram side:

On Cloudflare side:

Notes

  • Blocked IP address will never gets unblocked.
  • If logrotate is not setup correctly, then parsing the whole access.log might consume a lot of system resources.
  • The attack pattern/sample is not stored so we have no idea how the attack is conducted.
Owner
Nova Kwok
43EC 6073 0BFF A16C 34BB 9EF2 8D42 A0E6 99E5 0639
Nova Kwok
Repo used to maintain all notes and scripts developed during my DevNet Expert studies

DevNet Expert Studies Exam Date: TBD (Waiting for registration to open) This repository will be used to track my progress and maintain all notes/scrip

Dan 32 Dec 11, 2022
A simple tool to utilize the basic functionality of the Private API From Virus Total

Welcome To VT-SCAN (viurs total api) Information This is a simple tool to utilize the basic functionality of the Private API From Virus Total. with th

0X0ŽĒR∅⁰ 1 Sep 21, 2022
DataShare - Simple library for data sharing between scripts and public functions calling

DataShare - Simple library for data sharing between scripts and public functions calling. Installation. Install code, Delete LICENSE, README, readme.t

Ivan Perzhinsky. 1 Dec 17, 2021
track IP Address

ipX Table of Contents ipX Welcome Features Uses Author 📝 License Welcome find the location of an IP address. Specifically, you can get the following

Ali Shahid 15 Sep 26, 2022
This is a simple python script to collect sub-domains from hackertarget API

Domain-Scraper 🌐 This is a simple python script to collect sub-domains from hackertarget API Note : This is tool is limited to 20 Queries / day with

CHINO TECH TOOLS 4 Sep 09, 2021
Web service load balancing simulation experiment.

Web service load balancing simulation experiment.

NicestZK 1 Nov 12, 2021
PySocks lets you send traffic through SOCKS proxy servers.

PySocks lets you send traffic through SOCKS proxy servers. It is a modern fork of SocksiPy with bug fixes and extra features. Acts as a drop-i

1.1k Dec 07, 2022
A vpn that sits in your browser, accessible via a website

VPNInYourBrowser A vpn that sits in your browser, accessible via a website Example setup: https://VPNInBrowser.jaffa42.repl.co Setup Put the code onto

1 Jan 20, 2022
PcapXray - A Network Forensics Tool - To visualize a Packet Capture offline as a Network Diagram

PcapXray - A Network Forensics Tool - To visualize a Packet Capture offline as a Network Diagram including device identification, highlight important communication and file extraction

Srinivas P G 1.4k Dec 28, 2022
A Python Packages to make own chat room

Chathon A Python packages for make own chat room Install PyPI pip install chathon

1 Dec 10, 2021
The module that allows the collection of data sampling, which is transmitted with WebSocket via WIFI or serial port for CSV file.

The module that allows the collection of data sampling, which is transmitted with WebSocket via WIFI or serial port for CSV file.

Nelson Wenner 2 Apr 01, 2022
A Python script that alerts via SMS when a stock is reaching an inflection point

TradeAlert Not sure what this will ultimately become, but for now, its a Python script that alerts via SMS when a stock is reaching an inflection poin

3 Feb 22, 2022
A database-based CDN node supporting PostgreSQL and MongoDB backends.

A simple to use database-based deployable CDN node for hobbyist developers who wish to have their own CDN!

Vish M 10 Nov 19, 2022
GitHub action for sspanel automatically checks in to get free traffic quota

SSPanel_Checkin This is a dish chicken script for automatic check-in of sspanel for GitHub action, It is only applicable when there is no verification

FeedCatWithFish 7 Apr 28, 2022
Modern Denial-of-service ToolKit for python

💣 Impulse Modern Denial-of-service ToolKit 💻 Main window 📡 Methods: Method Target Description SMS PHONE Sends a massive amount of SMS messages and

1 Nov 29, 2021
Module for convenient work with TCP sockets.

m_socket-py Module for convenient work with TCP sockets. Contributing Pool Request is supported! Ask questions in the Issues section. License Copyrigh

Egor Arskiy 5 Mar 09, 2022
nettrace is a powerful tool to trace network packet and diagnose network problem inside kernel.

nettrace nettrace is is a powerful tool to trace network packet and diagnose network problem inside kernel on TencentOS. It make use of eBPF and BCC.

84 Jan 01, 2023
The AKS cluster provisioner provisions AKS clusters :-)

Overview The AKS cluster provisioner provisions AKS clusters :-) It uses the Azure CLI to configure VNet and subnets before creating the cluster itsel

Gigi Sayfan 1 Nov 10, 2021
EchoDNS - Analyze your DNS traffic super easy, shows all requested DNS traffic

EchoDNS - Analyze your DNS traffic super easy, shows all requested DNS traffic

Oli Zimmermann 1 Jan 11, 2022
IP-Escaner - A Python Tool to obtain information from an IP address

IP-Escaner Herramienta para obtener informacion de una direccion IP Opciones de

4 Apr 09, 2022