find all the URL of a site with a specific Regex

Overview

href

this program will find all the link with a spesfic Regex pattern from a site.

what it will do

in any site there are a lots of url that may you need the file behind them, this program will find all the tag, then list the href of the tags. you can use Regex to find the special link(s)

all the finded url's have some special charater's, so the Regex pattern will try to match with all finded url, if match, the url will return. if not match, try for next url in the list.

if you do not write any pattern the program will print all link of site, defualt pattern is: .*

and the last thing is that the program is case-insensitive

how to use

usage:

    --url 'the url of site'

    --pattern 'Regex pattern'

    --load-headers` path header file

    href.py --url 'URL' --pattern 'RegegPattern' --load-headers ./headers

example:

    href.py --url 'https://guitarmusic.ir/hayedeh-songs/' --pattern '.*mp3.*'

example:

href

note:

  1. all the switch have a small way to use

--help: -h

--url: -u

--pattern: -p

--load-headers: -l

  1. use pipe

to use the program some time you need to pipe or redirect the result

some site repeated their link to preview a video or music before download them, so you can pipe the result to uniq command for prevent link duplicate.

and for having the link in a text file, you should redirect the result to a file. href.py -u "URL" -p "patternt" > links.txt

  1. run easy

to run the program witch oud cd to the source dir or wite the full path every time, you can link it to your ~//.local/bin/href do it by this command: ln -s href.py ~/.local/bin/href

and do not forget to make it executable

  1. headers

if you got any http status code try to use a the header of site, or use the default header in the directory by this switch --load-header [file]

Owner
Arya Shabane
The Only Currency Left In This World Is POWER | Knowledge is POWER
Arya Shabane
find all the URL of a site with a specific Regex

href this program will find all the link with a spesfic Regex pattern from a site. what it will do in any site there are a lots of url that may you ne

Arya Shabane 12 Dec 05, 2022
C++ library for urlencode.

liburlencode C library for urlencode.

Khaidi Chu 6 Oct 31, 2022
ShortenURL-model - The model layer class for shorten url service

ShortenURL Model The model layer class for shorten URL service Usage Complete th

TwinIsland 1 Jan 07, 2022
Simple python library to deal with URI Templates.

uritemplate Documentation -- GitHub -- Travis-CI Simple python library to deal with URI Templates. The API looks like from uritemplate import URITempl

Hyper 210 Dec 19, 2022
coURLan: Clean, filter, normalize, and sample URLs

coURLan: Clean, filter, normalize, and sample URLs Why coURLan? “Given that the bandwidth for conducting crawls is neither infinite nor free, it is be

Adrien Barbaresi 20 Dec 14, 2022
A python code for url redirect check

A python code for url redirect check

Fayas Noushad 1 Oct 24, 2021
A URL builder for genius :D

genius-url A URL builder for genius :D Usage from gurl import genius_url

ꌗᖘ꒒ꀤ꓄꒒ꀤꈤꍟ 12 Aug 14, 2021
Simple Version of ouo.io. shorten any link on the web easily

OUO.IO LINK SHORTENER This is a simple python script that made to short links. currently ouo.io doesn't have Application Programming Interface so i de

Danushka-Madushan 1 Dec 11, 2021
Qysqa - URL shortener website with python

Qysqa - shorten your URL. ~ A simple URL-shortening website. how do you pronounc

Dastan Ozgeldi 0 Nov 18, 2022
Python implementation for generating Tiny URL- and bit.ly-like URLs.

Short URL Generator Python implementation for generating Tiny URL- and bit.ly-like URLs. A bit-shuffling approach is used to avoid generating consecut

Alireza Savand 170 Dec 28, 2022
Have you ever wondered: Where does this link go? The REDLI Tool follows the path of the URL.

Have you ever wondered: Where does this link go? The REDLI Tool follows the path of the URL. It allows you to see the complete path a redirected URL goes through. It will show you the full redirectio

JAYAKUMAR 28 Sep 11, 2022
This is a no-bullshit file hosting and URL shortening service that also runs 0x0.st. Use with uWSGI.

This is a no-bullshit file hosting and URL shortening service that also runs 0x0.st. Use with uWSGI.

mia 1.6k Dec 31, 2022
Fast pattern fetcher, Takes a URLs list and outputs the URLs which contains the parameters according to the specified pattern.

Fast Pattern Fetcher (fpf) Coded with 3 by HS Devansh Raghav Fast Pattern Fetcher, Takes a URLs list and outputs the URLs which contains the paramete

whoami security 5 Feb 20, 2022
🔗 FusiShort is a URL shortener built with Python, Redis, Docker and Kubernetes

This is a playground application created with goal of applying full cycle software development using popular technologies like Python, Redis, Docker and Kubernetes.

Lucas Fusinato Zanis 7 Nov 10, 2022
Extract countries, regions and cities from a URL or text

This project is no longer being maintained and has been archived. Please check the Forks list for newer versions. Forks We are aware of two 3rd party

Ushahidi 216 Nov 18, 2022
encurtador de links feito com python

curt-link encurtador de links feito com python! instalação Linux: $ git clone https://github.com/bydeathlxncer/curt-link $ cd curt-link $ python3 url.

bydeathlxncer 5 Dec 29, 2021
A simple URL shortener built with Flask

A simple URL shortener built with Flask and MongoDB.

Mike Lowe 2 Feb 05, 2022
A simple, immutable URL class with a clean API for interrogation and manipulation.

purl - A simple Python URL class A simple, immutable URL class with a clean API for interrogation and manipulation. Supports Pythons 2.7, 3.3, 3.4, 3.

David Winterbottom 286 Jan 02, 2023
🔗 Generate Phishing URLs 🔗

URLer 🔗 Generate Phishing URLs 🔗 URLer Table Of Contents General Information Preview Installation Disclaimer Credits Social Media Bug Report General

mrblackx 5 Feb 08, 2022
Yet another URL library

Yet another URL library

aio-libs 884 Jan 03, 2023