We want to check several batch of web URLs (1~100 K) and find the phishing website/URL among them.

Overview

WebAttestation

[TOC]

Introduction

We want to check several batch of web URLs (1~100 K) and find the phishing website/URL among them. This module is designed to do the URL/web attestation by using the API from NUS-Phishperida-Project. The program contents 3 main parts: WebDownloader, webScreenShoter and PhishperidaPKG.

WebDownloader

This module will provide API to download the webpage component: html file, image file, javascript file, href link file based on the input URL.

Module detail doc : https://github.com/LiuYuancheng/WebAttestation/blob/main/WebDownloadReadme.md

WebScreenShoter

This module will use different web browser's driver to capture the webpage's screen shot based on the given URL.

Module detail doc :

PhishperidaPKG

This module is used to encapsulate the NUS-Phishperida project (not OOP) as a black box API for other projects to use.

NUS-Phishperida project: https://github.com/lindsey98/Phishpedia

Module detail doc :

For each URL, the program will do below steps:

  1. Use webDownloader module to download all the web components.1

  2. Use webScreenShoter module to get a webpage screenshot of the url.

  3. Pass the web components and the screen shot to PhishperidaPKG to do the siamese checking

Program Workflow

If you set the program running under single thread, the program work flow diagram will be shown as below:


Program Setup

Development Environment : python 3.7.10
Additional Lib/Software Need
  • WebDownloader: Refer to program setup section in [WebDownloaderReadme.md]
  • WebScreenShoter: Refer to program setup section in [WebScreenShoterReadme.md]
  • PhishperidaPKG: Refer to program setup section in [PhishperidaPKGReadme.md]
Hardware Needed
  • WebDownloader: N.A
  • WebScreenShoter: Computer with video output.
  • PhishperidaPKG: Computer with Nvidia graph card.
Program File List

version: v0.1

Program File Execution Env Description
src/webAttestation.py python 3.7.4 Main web Attestation execution program.
src/webScreenShoter.py python 3.7.10 Main web screen shot execution program.
src/webDownload.py python 3.7.10 Main web Downloader program API.
src/phishpediaPKG.py python 3.8.10 Encapsulated API the NUS-Phishperida project for OPP.
src/webGlobal.py python 3.7.4 Global parameters file which will be used in the other modules.
src/urllist.txt url record list.

Program Usage

Module API Usage
  • WebDownloader: Refer to program API usage section in [WebDownloaderReadme.md]
  • WebScreenShoter: Refer to program API usage section in [WebScreenShoterReadme.md]
  • PhishperidaPKG: Refer to program API usage section in [PhishperidaPKGReadme.md]
Program Execution
  1. Copy the url you want to check in the url record file "urllist.txt"

  2. Cd to the program folder and run program execution cmd:

    python webAttestation.py
    
  3. Check the result:


Last edit by LiuYuancheng([email protected]) at 26/11/2021

奇遇淘客服务器端

奇遇淘客 APP 服务器端 警告 正在使用 v0.2.0 版本的用户,请尽快升级到 v0.2.1。 v0.2.0 版本的 Docker 镜像中包含了有问题的 aiohttp。 奇遇淘客代码库 奇遇淘客 iOS APP 奇遇淘客 Android APP 奇遇淘客文档 服务器端文档 Docker 使用

奇遇科技 92 Nov 09, 2022
A Linux webcam plugin for BGMv2 as used in our demos.

The goal of this repository is to supplement the main Real-Time High Resolution Background Matting repo with a working demo of a videoconferencing plu

Andrey Ryabtsev 144 Dec 27, 2022
This is the key combo trainer for League of Legends and Dota 2 players.

This is the key combo trainer for League of Legends and Dota 2 players. Place the mouse cursor on the blue point and press the key combo from the upper-left side of the screen.

Ilya Shpigor 1 Jan 31, 2022
The Playwright Workshop for TAU: The Homecoming

tau-playwright-workshop This repository contains the instructions and example code for the Playwright workshop for TAU: The Homecoming on December 1,

Pandy Knight 134 Dec 30, 2022
Flight Reservation App With Python

Flight Reservation App With Python

victor-h. 1 Nov 21, 2021
Decipher using Markov Chain Monte Carlo

Decipher using Markov Chain Monte Carlo

Science étonnante 43 Dec 24, 2022
Audio-analytics for music-producers! Automate tedious tasks such as musical scale detection, BPM rate classification and audio file conversion.

Click here to be re-directed to the Beat Inspect Streamlit Web-App You are a music producer? Let's get in touch via LinkedIn Fundamental Analytics for

Stefan Rummer 11 Dec 27, 2022
A simple assembly- and brainfuck-inspired stack-based language

asm-stackfuck A simple assembly- and brainfuck-inspired stack-based language. The language has a few goals: Be stack-based Look like assembly Have a s

Nils Trinity 1 Feb 06, 2022
Программа для практической работы №12 по дисциплине

Информатика: программа для практической работы №12 Код и блок-схема программы для практической работы №12 по дисциплине "Информатика" (I семестр). Сут

Vladislav 1 Dec 07, 2021
Generate PNG filles from NFO files.

Installation git clone https://github.com/pcroland/nfopng cd nfopng pip install -r requirements.txt Usage ❯ ./nfopng.py usage: nfopng.py [-h] [-v] [-i

4 Jun 26, 2022
Vaksina - Vaksina COVID QR Validation Checker With Python

Vaksina COVID QR Validation Checker Vaksina is a general purpose library intende

Michael Casadevall 33 Aug 20, 2022
🇮🇳 A Indian Flag Animation Project Made With Python

🇮🇳 A Indian Flag Animation Project Made With Python

MuFaz-TG 2 Oct 21, 2022
Control your gtps with gtps-tools!

Note Please give credit to me! Do not try to sell this app, because this app is 100% open source! Do not try to reupload and rename the creator app! S

Jesen N 6 Feb 16, 2022
Blender addon, import and update mixamo animation

This is a blender addon for import and update mixamo animations.

ywaby 7 Apr 19, 2022
GNU/Linux'u yeni kurulumu bitirmiş olarak açtığınızda sizi karşılayacak bir uygulama.

Hoş Geldiniz GNU/Linux'u yeni kurulumu bitirmiş olarak açtığınızda sizi karşılayacak bir uygulama.

Alperen İsa 96 Oct 30, 2022
Proyecto - Análisis de texto de eventos históricos

Acceder al código desde Google Colab para poder ver de manera adecuada todas las visualizaciones y poder interactuar con ellas. Link de acceso: https:

1 Jan 31, 2022
An example project that shows how to check if a certain macro is active in a file.

PlatformIO Check Compiler Flags Example Description Demonstrates the usage of an extra script and a special compilter invocation to get the active mac

Maximilian Gerhardt 1 Oct 28, 2021
Serverless demo showing users how they can capture (and obfuscate) their Lambda payloads in Datadog APM

Serverless-capture-lambda-payload-demo Serverless demo showing users how they can capture (and obfuscate) their Lambda payloads in Datadog APM This wi

Datadog, Inc. 1 Nov 02, 2021
Simple Python API for the Ergo Platform Explorer

Ergo is a "Resilient Platform for Contractual Money." It is designed to be a platform for applications with the main focus to provide an efficient, se

7 Jul 06, 2021
A python library with various gambling and gaming classes

gamble is a simple library that implements a collection of some common gambling-related classes Features die, dice, d-notation cards, decks, hands pok

Jacobi Petrucciani 16 May 24, 2022