Distributed, blockchain based hashtables middleware for deduplication of file uploads to the cloud

Overview

distributed-blockchain-based-secure-file-dedupe

  • Searching is Distributed, Block and Access List for each upload is unique and it is stored in a single VM to reduce overhead in the metadata (blockchain based).

  • Instructions for 4 nodes (including a storage node) docker swarm setup:

    1. get.docker.com .. install docker on all VMs (run both commands under "This script is meant for the quick & easy installation")

    2. ssh into all VMs with sudo access (or add docker usergroup)

    3. Run "docker swarm init" on the leader node.

    4. Run "docker swarm join-token manager", copy paste the output to all other VMs, so that all other nodes will be added to the swarm as manager.

    5. To confirm all the nodes present in swarm, run "docker node ls"

    6. Create a common network (attachable) among all nodes on the swarm using "docker network create --driver overlay --attachable dedupe"

    7. To confirm, run "docker network ls" on all the VMs and check the scope(==swarm) and driver(==overlay)

    8. Build and push the image "docker-compose build", "docker-compose push".

    9. Pull the image for dedupe nodes separately "docker image pull abhishekvtangod/blockdedupe"

    10. Pull the image for cloud node "docker image pull abhishekvtangod/blockdedupe_cloud"

    11. Run the containers on all VMs.

      • docker container run -d --name node1 --network dedupe --env PYTHONUNBUFFERED=1 --env node_id=1 -p 5001:5000 --hostname localhost abhishekvtangod/blockdedupe
      • docker container run -d --name node2 --network dedupe --env PYTHONUNBUFFERED=1 --env node_id=2 -p 5002:5000 --hostname localhost abhishekvtangod/blockdedupe
      • docker container run -d --name node3 --network dedupe --env PYTHONUNBUFFERED=1 --env node_id=3 -p 5003:5000 --hostname localhost abhishekvtangod/blockdedupe
      • docker container run -d --name cloud --network dedupe --env PYTHONUNBUFFERED=1 -p 5000:5000 --hostname localhost abhishekvtangod/blockdedupe_cloud
    12. Access all the urls from individual ips of VM with appropriate ports(change port mappings according to your ease and avoid confusions).

Pure Blockchain Implementation: https://github.com/lanxeon

Owner
Abhishek Tangod
Like what you do; Stay passionate, focused and motivated :)
Abhishek Tangod
A web app to scan crypto markets based on candlestick pattern recognition from

Crypto_Scanner A web app to scan crypto markets based on candlestick pattern recognition from "Japanese Candlestick Charting Techniques: A Contemporar

Chris Qi 27 Jan 01, 2023
A python script for AES Angecryption in Steganography

Angecryption is an encryption or an decryption result from a file to create an other file with the same / or not type.

ISIS 3 Jul 25, 2022
Deriving RSA public keys from message-signature pairs

The repository contains: Experimental code to calculate RSA public keys based on two known message-signature pairs

Silent Signal 120 Dec 31, 2022
I coded the sha256 algorithm into python without using any modules.

sha256.py I coded the sha256 algorithm in python without using any modules. The purpose of the code was to better understand the algorithm and learn h

4 Dec 12, 2022
Python app for encrypting messages with fernet cryptography.

Fernet Encryption Python app for encrypting messages with fernet cryptography. Github repo: https://github.com/mystic-repo/FernetEncryption PyPi: http

Mystic 1 May 28, 2022
Accounting Cycle Program with Blockchain Component

In the first iteration of my accounting cycle program, I talked about adding in a blockchain component that allows the user to verify the inegrity of

J. Brandon Walker 1 Nov 29, 2021
Powerful Tool to encrypt and decrypt files using AES.

AEScryptor Tool Description Encrypt and Decrypt files with AES-128 (16bytes key). AES mode = CFB (cipher Feedback) security = super safe! Usage [1] Ch

5 Jan 12, 2022
This is a webpage that contains login and signup page by which the password is stored using elliptic curve cryptography

LoginPage_using_Elliptic_curve_cryptography- This is a webpage that contains login and signup page by which the password is stored using elliptic curv

1 Oct 15, 2021
Taishang Credential With Interactive Badges

结合数字徽章的交互式区块链证书 DApp 1 项目简介 DID 与 VC 一直是区块链研究的重要领域,也是区块链落地的重要基础,从「传统证书」到基于DID的VC证书是证书体系范式转移的重要第一步。 但是,在迈出第一步之后我们可以进行更加丰富的尝试,例如尝试将不可转移的徽章与可转移的权益与证书相结合,

1 Nov 07, 2021
Python Script for signingn LetsEncrypt certificate with certbot, and update them into Fortigate

Python Script for signingn LetsEncrypt certificate with certbot, and update them into Fortigate (to be used into the WEB VPN or Load Balancer certificate)

6 Jan 03, 2023
PeGuard - Windows PE crypter and packing utility

PEGUARD PEGUARD is a file crypter and packing utility. This project was original

11 Nov 28, 2022
Repository detailing Choice Coin's Creation and Documentation

Choice Coin V1 This Repository provides code and documentation detailing Choice Coin V1, a utility token built on the Algorand Blockchain. Choice Coin

Choice Coin 245 Dec 29, 2022
A python implementation of our standard object-oriented encryption package, shipped with most apps.

Encryption Manager (python edition) VerseGroup's native encryption manager adapted for python applications. Function Generate new set of private and p

Verse Group LLC 2 Oct 30, 2022
TON Command Line Interface - easy smart contract manipulation

toncli The Open Network cross-platform smart contract command line interface. Easy to deploy and interact with TON smart contracts. Installation Toncl

Disintar IO 100 Dec 18, 2022
A curated list of resources dedicated to reinforcement learning applied to cyber security.

Awesome Reinforcement Learning for Cyber Security A curated list of resources dedicated to reinforcement learning applied to cyber security. Note that

Kim Hammar 212 Jan 02, 2023
Modern(-ish) password hashing for your software and your servers

bcrypt Good password hashing for your software and your servers Installation To install bcrypt, simply: $ pip install bcrypt Note that bcrypt should b

Python Cryptographic Authority 947 Dec 28, 2022
Basic Ethereum Miner Lib

EthMine ⛏ Basic Ethereum Miner Library. Developers can integrate this algorithm to mine blocks from their ethereum supported chain efficiently. Instal

Jaival Patel 1 Oct 30, 2021
Deribit_Algo_Project_Python - Deribit algo project written in python trading crypto futures

This is a Algo/script trading for Deribit. You need an account with deribit, to

24 Jan 09, 2023
gcrypter: an encryption algorithm based on bytes and their correspondent numbers to encode strings

gcrypter: an encryption algorithm based on bytes and their correspondent numbers to encode strings

Nuninha-GC 1 Jan 10, 2022
Bitcoin & Lightning Container Manager for facilitating development tools

Torch-cli Bitcoin & Lightning Container Manager for facilitating development too

Gray Finance 3 Aug 22, 2022