It's a simple python script to take backup of directories (compressing) then the same to move your mentioned S3 bucket with the help of AWS IAM User.

Overview

Directory Backup Moved to S3 (Pyscript)

Build


Description

Here it's a python script that needs to use this script simply create a directory backup and moved that compressed backup file to a configured S3 bucket with the help of python script and AWS IAM User with S3 full access. So, let's roll down.


Feature

  • Easy to configure for anyone
  • It generates the directory compressed format
  • Which one we entered directory converted to a backup/compressed form to S3
  • All the steps I have provided including AWS IAM User and S3 Bucket Creation

Modules used


Pre-Requests

  • Basic Knowledge of python
  • Basic Knowledge of AWS IAM, S3 service
  • Need to change your IAM user creds and bucket name at var.py

IAM User Creation steps (with screenshot)

  1. log into your AWS account as a root user and go to IAM user
  2. goto Access Managment >> Users alt_txt
  3. Click Add User (top right corner) alt_txt
  4. Enter any username as you like and Choose "Programmatic access" >> Click Next Permissions alt_txt
  5. Set Permissions >> Select "Attach existing policies directly" >> Choose "AmazonS3FullAccess" >> Click Next Tags alt_txt
  6. Add Tags(Optional) >> Enter a key and value as you like either you can leave as blank alt_txt
  7. Review your user details and click "Create User" alt_txt
  8. Store your credentials to your local alt_txt

Reference URL:: IAM User creation article


S3 Bucket Creation (with screenshot)

  1. Go to S3 > Click Create Bucket alt_txt
  2. Any bucket name as you wish and then please enable versioning (that you upload same file multiple times or modified versions uploaded that the S3 stored as a version bases like Git) alt_txt
  3. Click create bucket

alt_txt

Reference URL:: Creating S3 bucket please use this doc and you can secure your bucket with IAM user using S3 bucket policy


Pre-Requested (Dependency packages)

yum install -y git
yum install -y python3
yum install -y python3-pip

How to get

git clone https://github.com/yousafkhamza/backup-to-s3-pyscript.git
cd backup-to-s3-pyscript
pip3 install -r requirements.txt

Change your creds and bucket name in at var.py file

Command to run the script::

python3 to-S3.py Python/test
# Pyhon/test               <------------ Directory to take backup and move the backup to S3

Output be like

$ $ python3 to-S3.py Python/httpd
Start to Upload that the httpd.tar.gz your S3 Bucket yousaf-test
Backup successfully uploaded to your S3 bucket yousaf-test

View of S3 bucket

Screenshot alt_txt


Behind the code

vim to-S3.py

import boto3
import tarfile
import os
import sys
import posixpath
import var
from boto3.s3.transfer import S3Transfer

directory = sys.argv[1]
dirname = os.path.split(directory)[-1]
if directory.endswith ('/'):
    print ('Please remove / after the directory path you have entered')
else:
    if posixpath.isdir(directory):
        tarname = '/tmp/{}.tar.gz'.format(dirname)
        tar = tarfile.open(tarname,'w:gz')
        tar.add(directory)
        tar.close()
        print('Start to Upload that the', dirname+'.tar.gz', 'your S3 Bucket', var.BUCKET_NAME)

# S3 uploading started
        client = boto3.client('s3', aws_access_key_id=var.AWS_ACCESS_KEY_ID,aws_secret_access_key=var.AWS_SECRET_ACCESS_KEY)
        transfer = S3Transfer(client)
        transfer.upload_file(tarname, var.BUCKET_NAME, 'backup/{}.tar.gz'.format(dirname))
        print('Backup succesfully uploaded to your S3 bucket', var.BUCKET_NAME)
    
# Remove temporary backup from local
        os.remove(tarname)
    else:
        print('Please enter a valid directory path')

var.py

AWS_ACCESS_KEY_ID = 'AKTBRI2N5IAT3ND'             <--------------   Replace your acess key
AWS_SECRET_ACCESS_KEY = 'asUNmMPrC99HoiiQPjehetFtVsPv'          <--------- Replace your secret key
BUCKET_NAME = 'yousaf-test'                <----------- Replace your bucket name

Conclusion

It's a simple python script to take backup of directories (compressing) then the same to move your mentioned S3 bucket with the help of AWS IAM User. this script may be helpful who had face issues moving backups to S3 so it might be useful for cloud engineers.

⚙️ Connect with Me


Owner
Yousaf K Hamza
Junior DevOps Enginner
Yousaf K Hamza
An advanced QR Code telegram bot with more features.

QR Code Bot A telegram qr code encode and decode bot Advanced Features 1. Database ( MongoDB ) Support 2. Broadcast Support 3. Status Command 4. Setti

Fayas Noushad 16 Nov 12, 2022
OpenSea-Python-Bot - OpenSea Python Bot can be used in 2 modes

OpenSea-Python-Bot OpenSea Python Bot can be used in 2 modes. When --nft paramet

49 Feb 10, 2022
Scrapping malaysianpaygap & Extracting data from the Instagram posts

Scrapping malaysianpaygap & Extracting data from the posts Recently @malaysianpaygap has gotten quite famous as a platform that enables workers throug

Yudhiesh Ravindranath 65 Nov 09, 2022
Boilerplate template for the discord-py-interactions library

discord-py-interactions_boilerplate Boilerplate template for the discord-py-interactions library Currently, this boilerplate supports discord-py-inter

Ventus 7 Dec 03, 2022
Easy-apply-bot - A LinkedIn Easy Apply bot to help with my job search.

easy-apply-bot A LinkedIn Easy Apply bot to help with my job search. Getting Started First, clone the repository somewhere onto your computer, or down

Matthew Alunni 5 Dec 09, 2022
A discord bot for tracking Iranian Minecraft servers and showing the statistics of them

A discord bot for tracking Iranian Minecraft servers and showing the statistics of them

MCTracker 20 Dec 30, 2022
Maestral is an open-source Dropbox client written in Python.

Maestral - A light-weight and open-source Dropbox client for macOS and Linux

2.6k Jan 03, 2023
A simple discord bot named atticus that sends you the timetable of your classes upon request

A simple discord bot named atticus that sends you the timetable of your classes upon request. Soon, it would you ping you before classes too!

Samhitha 3 Oct 13, 2022
Deploy your apps on any Cloud provider in just a few seconds

The simplest way to deploy your apps in the Cloud Deploy your apps on any Cloud providers in just a few seconds ⚡ Qovery Engine is an open-source abst

Qovery 1.9k Dec 26, 2022
🚀 An asynchronous python API wrapper meant to replace discord.py - Snappy discord api wrapper written with aiohttp & websockets

Pincer An asynchronous python API wrapper meant to replace discord.py ❗ The package is currently within the planning phase 📌 Links |Join the discord

Pincer 125 Dec 26, 2022
An App to get Ko-Fi payment updates on Telegram.

Deployments. Heroku.com 🚀 Replit.com 🌀 Make sure your app runs 24*7 Zeet.co 💪 Use this :~ Get Bot token from @botfather 🤖 Get ID where you want to

Jainam Oswal 16 Nov 12, 2022
A new coin listing alert bot using Python, Flask, MongoDB, Telegram API and Binance API

Bzzmans New Coin Listing Detection Bot Architecture About Project Work in progress. This bot basically gets new coin listings from Binance using Binan

Eyüp Barlas 21 May 31, 2022
An API wrapper for Henrik's Unofficial VALORANT API

ValorantAPI.py An API wrapper for Henrik's Unofficial VALORANT API Warning!! This project is still in beta and only contains barely anything yet. If y

Jakkaphat Chalermphanaphan 0 Feb 04, 2022
The programm for collecting data from Tinkoff API and building Excel table.

tinkproject The program for portfolio analysis via Tinkoff API Hello! This is my first project, please, don't judge me. This project was developed for

214 Dec 02, 2022
Bifrost C2. Open-source post-exploitation using Discord API

Bifrost Command and Control What's Bifrost? Bifrost is an open-source Discord BOT that works as Command and Control (C2). This C2 uses Discord API for

38 Dec 05, 2022
Nonebot2 简易群管

简易群管 ✨ NoneBot2 简易群管 ✨ _ 踢 改 禁 欢迎issue pr 权限说明:permission=SUPERUSER 安装 💿 pip install nonebot-plugin-admin 导入 📲 在bot.py 导入,语句: nonebot.load_plugin("n

幼稚园园长 74 Dec 22, 2022
A Wide AOI tool for discord.

drkdiscord - drk#1337 A Wide AOI tool for discord. Installation To Install you have to have python 3.x and pip installed on your system. If you have t

Darkest Surface 6 Dec 17, 2022
Space Bot, a Discord bot built for HackerSpace Club of PES University

Space Bot Space Bot, a Discord bot built for HackerSpace Club of PES University What can Space Bot do? Space Bot allows you to lookup any mentor or to

HackerSpace @PESU 7 Oct 23, 2022
Rust UserBot, Telegram istifadəsini asanlaşdıran bir proyektdir.

RUST USERBOT 🇦🇿 Rust UserBot, Telegram istifadəsini asanlaşdıran bir proyektdir. Qurulum Heroku Serverə qurulum git clone https://github.com/rustres

1 Oct 25, 2021
Microservice to extract structured information on EVM smart contracts.

Contract Serializer Microservice to extract structured information on EVM smart contract. Why? Modern NFT contracts may have different names for getPr

WeBill.io 8 Dec 19, 2022