It's a simple python script to take backup of directories (compressing) then the same to move your mentioned S3 bucket with the help of AWS IAM User.

Overview

Directory Backup Moved to S3 (Pyscript)

Build


Description

Here it's a python script that needs to use this script simply create a directory backup and moved that compressed backup file to a configured S3 bucket with the help of python script and AWS IAM User with S3 full access. So, let's roll down.


Feature

  • Easy to configure for anyone
  • It generates the directory compressed format
  • Which one we entered directory converted to a backup/compressed form to S3
  • All the steps I have provided including AWS IAM User and S3 Bucket Creation

Modules used


Pre-Requests

  • Basic Knowledge of python
  • Basic Knowledge of AWS IAM, S3 service
  • Need to change your IAM user creds and bucket name at var.py

IAM User Creation steps (with screenshot)

  1. log into your AWS account as a root user and go to IAM user
  2. goto Access Managment >> Users alt_txt
  3. Click Add User (top right corner) alt_txt
  4. Enter any username as you like and Choose "Programmatic access" >> Click Next Permissions alt_txt
  5. Set Permissions >> Select "Attach existing policies directly" >> Choose "AmazonS3FullAccess" >> Click Next Tags alt_txt
  6. Add Tags(Optional) >> Enter a key and value as you like either you can leave as blank alt_txt
  7. Review your user details and click "Create User" alt_txt
  8. Store your credentials to your local alt_txt

Reference URL:: IAM User creation article


S3 Bucket Creation (with screenshot)

  1. Go to S3 > Click Create Bucket alt_txt
  2. Any bucket name as you wish and then please enable versioning (that you upload same file multiple times or modified versions uploaded that the S3 stored as a version bases like Git) alt_txt
  3. Click create bucket

alt_txt

Reference URL:: Creating S3 bucket please use this doc and you can secure your bucket with IAM user using S3 bucket policy


Pre-Requested (Dependency packages)

yum install -y git
yum install -y python3
yum install -y python3-pip

How to get

git clone https://github.com/yousafkhamza/backup-to-s3-pyscript.git
cd backup-to-s3-pyscript
pip3 install -r requirements.txt

Change your creds and bucket name in at var.py file

Command to run the script::

python3 to-S3.py Python/test
# Pyhon/test               <------------ Directory to take backup and move the backup to S3

Output be like

$ $ python3 to-S3.py Python/httpd
Start to Upload that the httpd.tar.gz your S3 Bucket yousaf-test
Backup successfully uploaded to your S3 bucket yousaf-test

View of S3 bucket

Screenshot alt_txt


Behind the code

vim to-S3.py

import boto3
import tarfile
import os
import sys
import posixpath
import var
from boto3.s3.transfer import S3Transfer

directory = sys.argv[1]
dirname = os.path.split(directory)[-1]
if directory.endswith ('/'):
    print ('Please remove / after the directory path you have entered')
else:
    if posixpath.isdir(directory):
        tarname = '/tmp/{}.tar.gz'.format(dirname)
        tar = tarfile.open(tarname,'w:gz')
        tar.add(directory)
        tar.close()
        print('Start to Upload that the', dirname+'.tar.gz', 'your S3 Bucket', var.BUCKET_NAME)

# S3 uploading started
        client = boto3.client('s3', aws_access_key_id=var.AWS_ACCESS_KEY_ID,aws_secret_access_key=var.AWS_SECRET_ACCESS_KEY)
        transfer = S3Transfer(client)
        transfer.upload_file(tarname, var.BUCKET_NAME, 'backup/{}.tar.gz'.format(dirname))
        print('Backup succesfully uploaded to your S3 bucket', var.BUCKET_NAME)
    
# Remove temporary backup from local
        os.remove(tarname)
    else:
        print('Please enter a valid directory path')

var.py

AWS_ACCESS_KEY_ID = 'AKTBRI2N5IAT3ND'             <--------------   Replace your acess key
AWS_SECRET_ACCESS_KEY = 'asUNmMPrC99HoiiQPjehetFtVsPv'          <--------- Replace your secret key
BUCKET_NAME = 'yousaf-test'                <----------- Replace your bucket name

Conclusion

It's a simple python script to take backup of directories (compressing) then the same to move your mentioned S3 bucket with the help of AWS IAM User. this script may be helpful who had face issues moving backups to S3 so it might be useful for cloud engineers.

⚙️ Connect with Me


Owner
Yousaf K Hamza
Junior DevOps Enginner
Yousaf K Hamza
Assistant made in python to control your spotify via voice

Spotify-Assistant Assistant made in python to control your spotify via voice Overview 🚀 PLAY, PAUSE, NEXT, PREVIOUS, VOLUME COMMANDS 📝 Toast notific

Mauri 6 Jan 18, 2022
Discord-shell - Reverse shell accessible via discord.py bot

Discord-shell reverse shell on discord bot. (STILL IN DEVELOPMENT) Installation

Skultz 6 Oct 21, 2022
An async-ready Python wrapper around FerrisChat's API.

FerrisWheel An async-ready Python wrapper around FerrisChat's API. Installation Instructions Linux: $ python3.9 -m pip install -U ferriswheel Python 3

FerrisChat 8 Feb 08, 2022
Basic Discord python bot

#How to Create a Discord Bot Account In order to work with the Python library and the Discord API, we must first create a Discord Bot account. Here ar

Tustus 1 Oct 13, 2021
Chatbot construido com o framework Rasa para responder dúvidas referentes ao COVID-19.

Racom Chatbot Chatbot construido com o framework Rasa. Como executar Necessário instalar Docker e Docker Compose. Para inicializar a aplicação, basta

Vinícius Souza 4 Jul 28, 2022
Python Discord Server Nuker

Untitled Nuker Python Discord Server Nuker Features: Ban Everyone Kick Everyone Rename Everyone Spam To All Channels Delete All Channels Delete All Ro

22 Dec 22, 2022
Make WhatsApp ChatBot and use WhatsApp API to send the WhatsApp messages in python .

Ultramsg.com WhatsApp Bot using WhatsApp API and ultramsg Demo WhatsApp API ChatBot using Ultramsg API with python. Opportunities and tasks: The outpu

Ultramsg 64 Dec 29, 2022
A high level library for building Discord bots.

Qord A high level library for building Discord bots. 🚧 This library is currently in development. Questions that you are having What is this? This is

Izhar Ahmad 16 May 14, 2022
Cloud-native, data onboarding architecture for the Google Cloud Public Datasets program

Public Datasets Pipelines Cloud-native, data pipeline architecture for onboarding datasets to the Google Cloud Public Datasets Program. Overview Requi

Google Cloud Platform 109 Dec 30, 2022
A discord account nuker with lots of tools that will destroy a discord account

A discord account nuker with lots of tools that will destroy a discord account (token destroyer... and much more).

firexi 10 Apr 28, 2022
A twitter bot that simply replies with a beautiful screenshot of the tweet, powered by beautify.dhravya.dev

Poet this! Replies with a beautiful screenshot of the tweet, powered by poet.so Installation git clone https://github.com/dhravya/poet-this.git cd po

Dhravya Shah 30 Dec 04, 2022
Asynchronous Python Wrapper for the GoFile API

Asynchronous Python Wrapper for the GoFile API

Gautam Kumar 22 Aug 04, 2022
An EmbedBuilder in Python for discord.py embeds. Pip Module.

Discord.py-MaxEmbeds An EmbedBuilder for Discord bots in Python. You need discord.py to use this module. Installation Step 1 First you have to install

Max Tischberger 6 Jan 13, 2022
Ever wanted a dashboard for making your antispam? This is it.

Ever wanted a dashboard for making your antispam? This is it.

Skelmis 1 Oct 27, 2021
Telegram Google Translater Bot

Google-Translater-Bot Hey Mo Tech, I am simple Google Translater Bot. I can translate any language to you selected language Team Mo Tech Deploy To Her

21 Dec 01, 2022
Crypto Signal Provider - A web application that allows users to select a cryptocurrency

Crypto_Signal_Provider This is a web application that allows users to select a c

Raul 2 Dec 11, 2022
A module to get data about anime characters, news, info, lyrics and more.

Animec A module to get data about anime characters, news, info, lyrics and more. The module scrapes myanimelist to parse requested data. If you wish t

DriftAsimov 31 Aug 31, 2022
Revolt.py - An async library to interact with the https://revolt.chat api.

Revolt.py An async library to interact with the https://revolt.chat api. This library will be focused on making bots and i will not implement anything

Zomatree 0 Oct 08, 2022
Python 3 tools for interacting with Notion API

NotionDB Python 3 tools for interacting with Notion API: API client Relational database wrapper Installation pip install notiondb API client from noti

Viet Hoang 14 Nov 24, 2022
Host your Python Discord Bot 24/7 for free. POC

🐉 Pandore 🐉 The easiest and fastest way to host your Python3 Discord Bot 24/7 for free! 📚 Documentation 📚 If you encounter any problem while using

Billy 73 Jan 02, 2023