ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

Overview

ShadowClone

ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete.

You can make full use of the Free Tiers provided by cloud providers and supercharge your mundane cli tools with shadow clone jutsu (Naruto style)!

Installation

Please visit the wiki for installation and intial configuration instructions

Usage

⚡ python shadowclone.py -h
usage: shadowclone.py [-h] -i INPUT [-s SPLITNUM] [-o OUTPUT] -c COMMAND

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
  -s SPLITNUM, --split SPLITNUM
                        Number of lines per chunk of file
  -o OUTPUT, --output OUTPUT
  -c COMMAND, --command COMMAND
                        command to execute

How it works

We create a container image during the initial setup and register it as a runtime for our function in AWS/GCP/Azure whatever. When you execute ShadowClone on your computer, instances of that container are activated automatically and are only active for the duration of its execution. How many instances to activate is dynamically decided at runtime depending on the size of the input file provided and the split factor. The input is then split into chunks and equally distributed between all the instances to execute in parallel. For example, if your input file has 10,000 lines and you set the split factor to 100 lines, then it will be split into 100 chunks of 100 lines each and 100 instances will be run in parallel!

Features

  • Extremely fast
  • No need to maintain a VPS (or a fleet of it :))
  • Costs almost nothing per month
    • Compatible with free tiers of most cloud services
  • Cloud agnostic
    • Same script works with AWS, GCP, Azure etc.
  • Supports upto 1000 parallel invocations
  • Dynamically decide the number of invocations
  • Run any tool in parallel on the cloud
  • Pipe output to other tools

Comparison

This tool was inspired by the awesome Axiom and Fleex projects and goes beyond the concept of VPS for running the tools by using serverless functions and containers.

Features Axiom/Fleex ShadowClone
Instances 10-100s* 1000s
Cost Per instance/per minute Mostly Free**
Startup Time 4-5 minutes 2-3 seconds
Idle Cost $++ Free
On Demand Scalability No

*Most cloud providers do not allow spinning up too many instances by default, so you are limited to around 10-15 instances at max. You have to make a request to the support to increase this number.

** AWS & Azure allow 1 million invocations per month for free. Google allows 2 million invocations per month for free. You will be charged only if you go above these limits

Demo

DNS Bruteforcing using a 43mb file - 34 seconds

asciicast

Running httpx on 94K subdomains in 1 min

asciicast

References

Lithops documentation

Free Tiers

Cloud Provider Free Allowance Link
Google Functions 2 Million invocations, 400,000 GB-seconds per month Google Cloud Free Program
AWS Lambda 1 Million invocations, Up to 3.2 million seconds of compute time per month Free Cloud Computing Services - AWS Free Tier
Azure Functions 1 Million invocations Microsoft Azure Free Services

Obviously, you can make any number of function invocations per month. The table above only shows how many invocations are free.

Similar Tools

Disclaimer

This tool is designed as a proof-of-concept implementation and it's usage is intended for educational purpose only. Usage for attacking targets without prior mutual consent is illegal. It's the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program.

An Async Bot/API wrapper for Twitch made in Python.

TwitchIO is an asynchronous Python wrapper around the Twitch API and IRC, with a powerful command extension for creating Twitch Chat Bots. TwitchIO co

TwitchIO 590 Jan 03, 2023
Simple stock price analytics

mune · Mune is an open source python web application built to analyze stocks, named after Homma Munehisa. Currently, the forecasting component is powe

Richard Hong 14 Aug 30, 2021
Aria & Qbittorent Mirror Bot

Eunha Mirror Eunha Mirror is a multipurpose Telegram Bot writen in Python for mirroring files on the Internet to our beloved Google Drive. Features su

ovin 158 Dec 19, 2022
Send pm to Admin - Telegram

Send pm to Admin - Telegram

Ahoora 3 Nov 17, 2022
You can connect with Sanila Ranatunga using this bot😉😉

Sanila-Ranatunga-s-Assistant-Bot You can connect with Sanila Ranatunga using this bot 😉 😉 Reach me on Telegram Sanila's Assistant Bot What is Telegr

Sanila Ranatunga 5 Feb 01, 2022
An API wrapper for the file.io web service.

🗃️ File.io An API wrapper for the file.io web service. Install $ pip3 install fileio or

nkot56297 1 Dec 18, 2021
A FORKED AND Modded version of TL:GD for 🅱️3R0K🧲support

for support join here working example group Leech Here For Any Issues/Imrovements or Discussions go here or here Please Leave A star And Fork this Rep

XcodersHub 165 Mar 12, 2022
Telegram Bot to save Posts or Files that can be Accessed via Special Links

OKAERI-FILE Bot Telegram untuk menyimpan Posting atau File yang dapat Diakses melalui Link Khusus. Jika Anda memerlukan tambahan module lagi dalam rep

Wahyusaputra 5 Aug 04, 2022
EpikCord.py - This is an API Wrapper for Discord's API for Python

EpikCord.py - This is an API Wrapper for Discord's API for Python! We've decided not to fork discord.py and start completely from scratch for a new, better structuring system!

EpikHost 28 Oct 10, 2022
AWS Enumeration and Footprinting Tool

Quiet Riot 🎶 C'mon, Feel The Noise 🎶 An enumeration tool for scalable, unauthenticated validation of AWS principals; including AWS Acccount IDs, roo

Wes Ladd 89 Jan 05, 2023
Checks if Minecraft accounts are available, or taken.

MCNameChecker Checks validity of Minecraft IGN's. Using async to make it even faster. Has rate-limit detections and Proxy support Usage Q. How do I us

Dimitri Demarkus 5 Apr 22, 2022
Stream Telegram files to web

Telegram File Stream Bot A Telegram bot to stream files to web Demo Bot » Report a Bug | Request Feature Table of Contents About this Bot Original Rep

Wrench 572 Jan 09, 2023
Discord Streaming Statut (Bot/SelfBot)

Discord-Streaming-Status Discord Streaming Status For Both User Accounts And Bot Accounts. Open your cmd and enter the command: pip install discord BE

Komuro 2 Nov 02, 2021
Most Simple & Powefull web3 Trade Bot (WINDOWS LINUX) Suport BSC ETH

Most Simple & Powefull Trade Bot (WINDOWS LINUX) What Are Some Pros And Cons Of Owning A Sniper Bot? While having a sniper bot is typically an advanta

GUI BOT 4 Jan 25, 2022
A simple tool that allows you to change your default AWS CLI profile.

Select AWS Profile Select AWS Profile (slapr) is a simple tool that lets you select which AWS Profile you want to use and sets it as the default AWS p

Antoni Yanev 2 Nov 09, 2022
Multi-Branch CI/CD Pipeline using CDK Pipelines.

Using AWS CDK Pipelines and AWS Lambda for multi-branch pipeline management and infrastructure deployment. This project shows how to use the AWS CDK P

AWS Samples 36 Dec 23, 2022
GitPython is a python library used to interact with Git repositories.

Gitoxide: A peek into the future… I started working on GitPython in 2009, back in the days when Python was 'my thing' and I had great plans with it. O

3.8k Jan 03, 2023
A python crypto trading bot on Binance using RSI in 25 Lines 🚀

RSI Crypto Trading Bot - Binance A Crypto Trading Bot on Binance trading BTCUSDT and ETHUSDT using RSI in 25 Lines of Code Getting Started Note Python

Blankly Finance 10 Dec 26, 2022
Discord bot for playing blindfold chess.

Albin Discord bot for playing blindfold chess written in Python. Albin takes the moves from chat and pushes them on the board without showing it. TODO

8 Oct 14, 2022
ResolveURL - Fork of UrlResolver by eldorados, tknorris and jsergio123

ResolveURL Fork of UrlResolver by eldorados, tknorris and jsergio123 I am in no

gujal 60 Jan 03, 2023