We want to check several batch of web URLs (1~100 K) and find the phishing website/URL among them.

Overview

WebAttestation

[TOC]

Introduction

We want to check several batch of web URLs (1~100 K) and find the phishing website/URL among them. This module is designed to do the URL/web attestation by using the API from NUS-Phishperida-Project. The program contents 3 main parts: WebDownloader, webScreenShoter and PhishperidaPKG.

WebDownloader

This module will provide API to download the webpage component: html file, image file, javascript file, href link file based on the input URL.

Module detail doc : https://github.com/LiuYuancheng/WebAttestation/blob/main/WebDownloadReadme.md

WebScreenShoter

This module will use different web browser's driver to capture the webpage's screen shot based on the given URL.

Module detail doc :

PhishperidaPKG

This module is used to encapsulate the NUS-Phishperida project (not OOP) as a black box API for other projects to use.

NUS-Phishperida project: https://github.com/lindsey98/Phishpedia

Module detail doc :

For each URL, the program will do below steps:

  1. Use webDownloader module to download all the web components.1

  2. Use webScreenShoter module to get a webpage screenshot of the url.

  3. Pass the web components and the screen shot to PhishperidaPKG to do the siamese checking

Program Workflow

If you set the program running under single thread, the program work flow diagram will be shown as below:


Program Setup

Development Environment : python 3.7.10
Additional Lib/Software Need
  • WebDownloader: Refer to program setup section in [WebDownloaderReadme.md]
  • WebScreenShoter: Refer to program setup section in [WebScreenShoterReadme.md]
  • PhishperidaPKG: Refer to program setup section in [PhishperidaPKGReadme.md]
Hardware Needed
  • WebDownloader: N.A
  • WebScreenShoter: Computer with video output.
  • PhishperidaPKG: Computer with Nvidia graph card.
Program File List

version: v0.1

Program File Execution Env Description
src/webAttestation.py python 3.7.4 Main web Attestation execution program.
src/webScreenShoter.py python 3.7.10 Main web screen shot execution program.
src/webDownload.py python 3.7.10 Main web Downloader program API.
src/phishpediaPKG.py python 3.8.10 Encapsulated API the NUS-Phishperida project for OPP.
src/webGlobal.py python 3.7.4 Global parameters file which will be used in the other modules.
src/urllist.txt url record list.

Program Usage

Module API Usage
  • WebDownloader: Refer to program API usage section in [WebDownloaderReadme.md]
  • WebScreenShoter: Refer to program API usage section in [WebScreenShoterReadme.md]
  • PhishperidaPKG: Refer to program API usage section in [PhishperidaPKGReadme.md]
Program Execution
  1. Copy the url you want to check in the url record file "urllist.txt"

  2. Cd to the program folder and run program execution cmd:

    python webAttestation.py
    
  3. Check the result:


Last edit by LiuYuancheng([email protected]) at 26/11/2021

WinBoost: Boost your windows system.

Winboost runs a complete checkup of your entire system locating junk files, speed-reducing issues and causes of any system or application glitches or crashes. Through a lot of research and testing, w

Smit Parmar 4 Oct 01, 2021
📙 Super lightweight function registries for your library

catalogue: Super lightweight function registries for your library catalogue is a tiny, zero-dependencies library that makes it easy to add function (o

Explosion 139 Jan 02, 2023
A Sophisticated And Beautiful Doxing Tool

Garuda V1.1 A Sophisticated And Beautiful Doxing Tool Works on Android[Termux] | Linux | Windows Don't Forget to give it a star ❗ How to use ❓ First o

The Cryptonian 67 Jan 10, 2022
Translation patch for Hololive ERROR

Translation patch for Hololive ERROR How do I install the patch? Grab the Translation.zip file for the latest version from the releases page, and unzi

18 Jul 20, 2022
dotfiles - Cristian Valero Abundio

In this repository you can find various configurations to configure your Linux operating system, preferably ArchLinux and its derivatives.

Cristian Valero Abundio 1 Jan 09, 2022
The Doodle Master seeks to turn your UI mockups into real code.

Doodle Master The Doodle Master seeks to turn your UI mockups into real code. Currently this repository just serves to demonstrate a Proof Of Concept

Karanbir Chahal 2.4k Dec 09, 2022
tgEasy | Easy for a Brighter Shine | Monkey Patcher Addon for Pyrogram

tgEasy | Easy for a Brighter Shine | Monkey Patcher Addon for Pyrogram

Jayant Hegde Kageri 35 Nov 12, 2022
A domonic-like wrapper around selectolax

A domonic-like wrapper around selectolax

byteface 3 Jun 23, 2022
run-js Goal: The Easiest Way to Run JavaScript in Python

run-js Goal: The Easiest Way to Run JavaScript in Python features Stateless Async JS Functions No Intermediary Files Functional Programming CommonJS a

Daniel J. Dufour 9 Aug 16, 2022
Tutorials on advanced python topics, and literate programming framework to write them.

Advanced course on Python3 This course covers several topics Python decorators The python object system / meta classes Also see my text on Python impo

Michael Moser 59 Dec 19, 2022
Tomador de ramos UC automatico para Windows, Linux y macOS

auto-ramos v2.0 Tomador de ramos UC automatico para Windows, Linux y macOS Funcion Este script de Python tiene como principal objetivo hacer que la to

Open Source eUC 13 Jun 29, 2022
CPLib is the abbreviation of Competitive Programming Library.

CPLib CPLib is the abbreviation of Competitive Programming Library. It aims to be a general template and optimization library for competitive programm

12 Oct 16, 2021
FantasyBballHelper - Espn Fantasy Basketball Helper

ESPN FANTASY BASKETBALL HELPER The simple goal of this project is to allow fanta

1 Jan 18, 2022
Probably the best way to simulate block scopes in Python

This is a package, as it says on the tin, to emulate block scoping in Python, the lack of which being a clever design choice yet sometimes a trouble.

88 Oct 26, 2022
Weblate is a copylefted libre software web-based continuous localization system

Weblate is a copylefted libre software web-based continuous localization system, used by over 2500 libre projects and companies in more than 165 count

Weblate 7 Dec 15, 2022
dynamically create __slots__ objects with less code

slots_factory Factory functions and decorators for creating slot objects Slots are a python construct that allows users to create an object that doesn

Michael Green 2 Sep 07, 2021
Cloud Native sample microservices showcasing Full Stack Observability using AppDynamics and ThousandEyes

Cloud Native Sample Bookinfo App Observability Bookinfo is a sample application composed of four Microservices written in different languages.

Cisco DevNet 13 Jul 21, 2022
Hands-on machine learning workshop

emb-ntua-workshop This workshop discusses introductory concepts of machine learning and data mining following a hands-on approach using popular tools

ISSEL Soft Eng Team 12 Oct 30, 2022
Python script to commit to your github for a perfect commit streak. This is purely for education purposes, please don't use this script to do bad stuff.

Daily-Git-Commit Commit to repo every day for the perfect commit streak Requirments pip install -r requirements.txt Setup Download this repository. Cr

JareBear 34 Dec 14, 2022
Manjaro CN Repository

Manjaro CN Repository Automatically built packages based on archlinuxcn/repo and manjarocn/docker. Install Add manjarocn to /etc/pacman.conf: Please m

Manjaro CN 28 Jun 26, 2022