Python script for crawling ResearchGate.net papers
This code start crawling process by urls in start.txt and give paper details in crawled.json.
First install Python. Then install these libraries:
pip install selenium
pip install webdriver-manager
MAX_FETCH_COUNT: How many pages you want to crawl?
MAX_CACHED_NUM: We renew crawled.json after crawling each MAX_CACHED_NUM papers.
covid-tracker-script A simple python script to fetch the latest covid info How it works First, get the current date in MM-DD-YYYY format. Check if the
🐛 MASS DBLTEK EXPLOIT CHECKER USING SHODAN 🕸 Python script who crawl first shodan page and check DBLTEK vulnerability
quora-plus-bypass A python script to extract answers to any question on Quora (Quora+ included) Requirements Python 3.x
poolbooru_gelscraper a simple python script for scraping images off gelbooru pools. modules required:requests_html, and os by default saves files with
Binance harvester - A Python 3 script to harvest data from the Binance socket stream and calculate popular TA indicators and produce lists of top trending coins
Aliexpress to telegram post Python script that reads Aliexpress offers urls from a Excel filename (.csv) and post then in a Telegram channel using a b
This script is useful for downloading stock market data for a wide range of companies specified by their respective tickers. The script reads in the d
Auto Invite To The Organization By Star A GitHub Action script to automatically invite everyone to your organization that stars your repository. What
AutoReserve A webdriver-based script for reserving badminton courts. 使用说明 下载 chromedriver 选择当前Chrome对应版本 安装 selenium pip install selenium 更改场次、金额信息dat
Crawler Rottentomatoes, Goodreads and IMDB sites crawler. Crawler written by beautifulsoup, selenium and lxml to gather books and films information an
Web Scraping images using Selenium and Python A propos de ce document This is a markdown document about Web scraping images and videos using Selenium
snallygaster Finds file leaks and other security problems on HTTP servers. what? snallygaster is a tool that looks for files accessible on web servers
httpmapper 👁️ Project • Technologies • Installation • How it works • License Project 🚧 For educational purposes. This is a project that I developed,
Alpha Swap English This is a simple python tool for the purpose of swapping latinic letters with cirylic ones and vice versa, in txt, docx and pdf fil
PyMDL An API for MyDramaList(MDL) based on webscraping for python. Description An API for MDL to make your life easier in retriving and working on dat
Dailyiptvlist.com scraper Info Made in python Linux only script Script requires to have wget installed Running script Clone repository with: git clone
Movies-Scraper You are probably tired of navigating through a movie website to get the right movie you'd want to watch during the weekend. There may e
Pattern Pattern is a web mining module for Python. It has tools for: Data Mining: web services (Google, Twitter, Wikipedia), web crawler, HTML DOM par
proxy scraper 🔎 Installation: git clone https://github.com/ebankoff/proxy_scraper Required pip libraries (pip install library name): lxml beautifulso
Web Crawler for "sina weibo" A web crawler for recording posts in "sina weibo" Introduction This script helps collect attributes of posts in "sina wei
高可用IP代理池 README ｜ 中文文档 本项目所采集的IP资源都来自互联网，愿景是为大型爬虫项目提供一个高可用低延迟的高匿IP代理池。 项目亮点 代理来源丰富 代理抓取提取精准 代理校验严格合理 监控完备，鲁棒性强 架构灵活，便于扩展 各个组件分布式部署 快速开始 注意，代码请在release
image retrieval This is a python code to retrieve an image from the internet, a
SCRYPTO What ? Crypto currencies scraping (At the moment, only bitcoin and ethereum crypto currencies are supported) How ? A python script is running
KKBA Intruoduction A low-code tool that generates python crawler code based on curl or url Requirement Python = 3.6 Install pip install kkba Usage Co
UsernameScraperTool Username Scraper for 40+ Social sites. How To use git clone
Scrap-mtg-top-8 - A top 8 mtg scraper using python
cybernews A package that provides you Latest Cyber/Hacker News from website using Web-Scraping. Latest Cyber/Hacker News Using Webscraping Developed b
IGLS - Instagram Like Scraper It's a web scraping command line tool based on python and selenium. Description This is a trial tool for learning purpos
feapder 是一款简单、快速、轻量级的爬虫框架。起名源于 fast、easy、air、pro、spider的缩写，以开发快速、抓取快速、使用简单、功能强大为宗旨，历时4年倾心打造。支持轻量爬虫、分布式爬虫、批次爬虫、爬虫集成，以及完善的爬虫报警机制。 之
0 Dec 15, 2021
4 Jan 9, 2022
10 Aug 18, 2022
1 Jan 2, 2022
68 Oct 8, 2022
6 Dec 6, 2022
71 Oct 4, 2022
11 Dec 11, 2022
4 Nov 9, 2021
1 Dec 30, 2021
3 Jul 01, 2022
2k Dec 28, 2022
15 Dec 05, 2021
3 May 31, 2022
6 Dec 10, 2022
1 Oct 16, 2021
1 Jan 31, 2022
8.4k Jan 08, 2023
19 Dec 07, 2022
4 Aug 20, 2022
5.2k Jan 03, 2023
1 Dec 30, 2021
15 Sep 01, 2022
8 Sep 20, 2021
1 Dec 20, 2022
1 Jan 24, 2022
4 Jun 02, 2022
5 Oct 29, 2021
1.4k Dec 29, 2022