Distributed, blockchain based hashtables middleware for deduplication of file uploads to the cloud

Overview

distributed-blockchain-based-secure-file-dedupe

  • Searching is Distributed, Block and Access List for each upload is unique and it is stored in a single VM to reduce overhead in the metadata (blockchain based).

  • Instructions for 4 nodes (including a storage node) docker swarm setup:

    1. get.docker.com .. install docker on all VMs (run both commands under "This script is meant for the quick & easy installation")

    2. ssh into all VMs with sudo access (or add docker usergroup)

    3. Run "docker swarm init" on the leader node.

    4. Run "docker swarm join-token manager", copy paste the output to all other VMs, so that all other nodes will be added to the swarm as manager.

    5. To confirm all the nodes present in swarm, run "docker node ls"

    6. Create a common network (attachable) among all nodes on the swarm using "docker network create --driver overlay --attachable dedupe"

    7. To confirm, run "docker network ls" on all the VMs and check the scope(==swarm) and driver(==overlay)

    8. Build and push the image "docker-compose build", "docker-compose push".

    9. Pull the image for dedupe nodes separately "docker image pull abhishekvtangod/blockdedupe"

    10. Pull the image for cloud node "docker image pull abhishekvtangod/blockdedupe_cloud"

    11. Run the containers on all VMs.

      • docker container run -d --name node1 --network dedupe --env PYTHONUNBUFFERED=1 --env node_id=1 -p 5001:5000 --hostname localhost abhishekvtangod/blockdedupe
      • docker container run -d --name node2 --network dedupe --env PYTHONUNBUFFERED=1 --env node_id=2 -p 5002:5000 --hostname localhost abhishekvtangod/blockdedupe
      • docker container run -d --name node3 --network dedupe --env PYTHONUNBUFFERED=1 --env node_id=3 -p 5003:5000 --hostname localhost abhishekvtangod/blockdedupe
      • docker container run -d --name cloud --network dedupe --env PYTHONUNBUFFERED=1 -p 5000:5000 --hostname localhost abhishekvtangod/blockdedupe_cloud
    12. Access all the urls from individual ips of VM with appropriate ports(change port mappings according to your ease and avoid confusions).

Pure Blockchain Implementation: https://github.com/lanxeon

Owner
Abhishek Tangod
Like what you do; Stay passionate, focused and motivated :)
Abhishek Tangod
Tracking (of choice) cryptocurrencies' daily prices and moving average.

Crypto-price-moving_average Tracking (of choice) cryptocurrencies' daily prices and moving average. About Alpha Vantage The Alpha Vantage library (htt

Thong Huynh 2 Jan 22, 2022
Bsvlib - Bitcoin SV (BSV) Python Library

bsvlib A Bitcoin SV (BSV) Python Library that is extremely simple to use but mor

Aaron 22 Dec 15, 2022
Generate a 2FA Code out of an 2FA Secret(Time-based only!)

2FA-Generator Generate a 2FA Code out of an 2FA Secret(Time-based only!) ❗ Pleas don't share this secrets/generated codes with someone. If someone kno

TheDanniCraft 1 Nov 05, 2021
Active github repos of all cryptocurrencies

This repo is to maintain the list of active repositories for all cryptocurrencies that https://codemask.org uses. The active list will be automaticall

CodeMask 5 May 20, 2022
Get the SHA256 hash of any file with this Python Script

Hashfile-SHA256 A SHA256 hash verifying script, written in python. Report Bug Table of Contents About The Project Built With Getting Started Prerequis

Ethan Gallucci 1 Nov 01, 2021
Technical_indicators_cryptos - Using technical indicators to find optimal trading strategies to deploy onto trading bot.

technical_indicators_cryptos Using technical indicators to find optimal trading strategies to deploy onto trading bot. In the Jup Notebook you wil

Van 4 Jul 03, 2022
EncryptAGit - Encrypt Your Git Repos

EncryptAGit - Encrypt Your Git Repos

midnite_runr 25 Oct 06, 2022
Cryptocurrency application that displays instant cryptocurrency prices and reads prices with the Google Text-to-Speech library.

📈 Cryptocurrency Price App 💰 ◽ Cryptocurrency application that displays instant cryptocurrency prices and reads prices with the Google Text-to-Speec

Furkan Mert 2 Nov 08, 2021
Run with one command grafana, prometheus, and a python script to collect and display cryptocurrency prices and track your wallet balance.

CryptoWatch Track your favorite crypto coin price and your wallet balance. Install Create .env: ADMIN_USER=admin ADMIN_PASSWORD=admin Configure you

Rafael Zimmermann 13 Dec 13, 2022
Vaulty - Encrypt/Decrypt with ChaCha20-Poly1305

Vaulty Encrypt/Decrypt with ChaCha20-Poly1305 Vaulty is an extremely lightweight encryption/decryption tool which uses ChaCha20-Poly1305 to provide 25

Chris Mason 1 Jul 04, 2022
An Etebase (EteSync 2.0) server so you can run your own.

Etebase - Encrypt Everything An Etebase (EteSync 2.0) server so you can run your own. Installation Requirements Etebase requires Python 3.7 or newer a

EteSync & Etebase 1.2k Dec 31, 2022
Python wrapper for the Equibles cryptos API.

Equibles Cryptos API for Python Requirements. Python 2.7 and 3.4+ Installation & Usage pip install If the python package is hosted on Github, you can

Equibles 1 Feb 02, 2022
This is a webpage that contains login and signup page by which the password is stored using elliptic curve cryptography

LoginPage_using_Elliptic_curve_cryptography- This is a webpage that contains login and signup page by which the password is stored using elliptic curv

1 Oct 15, 2021
Cryptocurrency Exchange Websocket Data Feed Handler

Cryptocurrency Exchange Websocket Data Feed Handler

Bryant Moscon 1.6k Dec 31, 2022
Programme de chiffrement et déchiffrement affine d'un message en python3.

Chiffrement Affine En Python3 Programme de chiffrement et déchiffrement affine d'un message en python3. Explication du chiffrement affine avec complex

Malik Makkes 1 Mar 26, 2022
⚡ Automatically decrypt encryptions without knowing the key or cipher, decode encodings, and crack hashes ⚡

⚡ Automatically decrypt encryptions without knowing the key or cipher, decode encodings, and crack hashes ⚡

11.2k Jan 09, 2023
A simple and secure password-based encryption & decryption algorithm based on hash functions, implemented solely based on python.

pyhcrypt A simple and secure password-based encryption & decryption algorithm based on hash functions, implemented solely based on python. Usage Pytho

Hongfei Xu 3 Feb 08, 2022
Linear encryption software programmed with python

Echoder linear encryption software programmed with python How does it work? The text in the text section runs a function with two keys entered keys mu

Emre Orhan 4 Dec 20, 2021
Highly decentralized and censorship-resistant way to store key data

Beacon coin Beacon coin is a Chia singelton coin that can store data that needs to be: always available censorship resistant versioned potentially imm

Sebastjan Trepca 24 Oct 04, 2022
An encryption format offering better security, performance and ease of use than PGP.

An encryption format offering better security, performance and ease of use than PGP. File a bug if you found anything where we are worse than our competition, and we will fix it.

27 Dec 25, 2022