A list of Python Bots used to extract data from several websites. Data extraction is for products on e-commerce (ecommerce) websites. Data fetched is such as the product images, title, price range, price, type of data etc. List of Website: https://www.qoovee.com/en/ https://merxu.com/en/ https://daraz.com/ https://www.nihaojewelry.com/ https://www.ecplaza.net/mask--product https://www.exportportal.com/ https://www.mallory.com/ https://www.townandcountryhardware.com/ https://www.like123.com/en/ https://www.ishopping.pk/ http://global.gmarket.co.kr/ https://shoptheglobe.co/ https://www.rannthai.com/ https://www.industrybuying.com/ https://www.ralali.com/ https://globaltradeplaza.com/ https://www.wholesalebox.in/ https://madeinindonesia.com/ https://dubaiyellowpagesonline.com/ https://www.qualitymill.com/ https://www.grainger.com/ https://www.abraa.com/ Python Libraries: Selenium, Beautiful Soup, Pandas, Scrapy, Requests, Urllib & Credentials etc. ___________________________________ This Project was based on a Python Internship Summer 2021 Dated: June-July 2021
A list of Python Bots used to extract data from several websites
Overview
Raspi-scraper is a configurable python webscraper that checks raspberry pi stocks from verified sellers
Raspi-scraper is a configurable python webscraper that checks raspberry pi stocks from verified sellers.
13 Oct 15, 2022
TarkovScrappy - A nifty little bot that lets you know if a queried item might be required for a quest at some point in the land of Tarkov!
TarkovScrappy A nifty little bot that lets you know if a queried item might be required for a quest at some point in the land of Tarkov! Hideout items
2 Apr 11, 2022
A Telegram crawler to search groups and channels automatically and collect any type of data from them.
Introduction This is a crawler I wrote in Python using the APIs of Telethon months ago. This tool was not intended to be publicly available for a numb
39 Dec 28, 2022
fork huanghyw/jd_seckill
Jd_Seckill 特别声明: 本仓库发布的jd_seckill项目中涉及的任何脚本，仅用于测试和学习研究，禁止用于商业用途，不能保证其合法性，准确性，完整性和有效性，请根据情况自行判断。 本项目内所有资源文件，禁止任何公众号、自媒体进行任何形式的转载、发布。
512 Jan 03, 2023
Example of scraping a paginated API endpoint and dumping the data into a DB
Provider API Scraper Example Example of scraping a paginated API endpoint and dumping the data into a DB. Pre-requisits Python = 3.9 Pipenv Setup # i
1 Oct 20, 2021
Scrape and display grades onto the console
WebScrapeGrades About The Project This Project is a personal project where I learned how to webscrape using python requests. Being able to get request
1 Oct 23, 2021
🕷 Phone Crawler with multi-thread functionality
Phone Crawler: Phone Crawler with multi-thread functionality Disclaimer: I'm not responsible for any illegal/misuse actions, this program was made for
3 Feb 10, 2022
Python framework to scrape Pastebin pastes and analyze them
pastepwn - Paste-Scraping Python Framework Pastebin is a very helpful tool to store or rather share ascii encoded data online. In the world of OSINT,
105 Dec 29, 2022
Using Selenium with Python to Web Scrap Popular Youtube Tech Channels.
Web Scrapping Popular Youtube Tech Channels with Selenium Data Mining, Data Wrangling, and Exploratory Data Analysis About the Data Web scrapi
0 Aug 18, 2021
This is my CS 20 final assesment.
eeeeeSpider This is my CS 20 final assesment. How to use: Open program Run to your hearts content! There are no external dependancies that you will ha
1 Jan 17, 2022
Scrapes all articles and their headlines from theonion.com
The Onion Article Scraper Scrapes all articles and their headlines from the satirical news website https://www.theonion.com Also see Clickhole Article
0 Nov 17, 2021
Creating Scrapy scrapers via the Django admin interface
django-dynamic-scraper Django Dynamic Scraper (DDS) is an app for Django which builds on top of the scraping framework Scrapy and lets you create and
1.1k Dec 17, 2022
High available distributed ip proxy pool, powerd by Scrapy and Redis
高可用IP代理池 README ｜ 中文文档 本项目所采集的IP资源都来自互联网，愿景是为大型爬虫项目提供一个高可用低延迟的高匿IP代理池。 项目亮点 代理来源丰富 代理抓取提取精准 代理校验严格合理 监控完备，鲁棒性强 架构灵活，便于扩展 各个组件分布式部署 快速开始 注意，代码请在release
5.2k Jan 03, 2023
Shopee Scraper - A web scraper in python that extract sales, price, avaliable stock, location and more of a given seller in Brazil
Shopee Scraper A web scraper in python that extract sales, price, avaliable stock, location and more of a given seller in Brazil. The project was crea
5 Nov 29, 2022
Script for scrape user data like "id,username,fullname,followers,tweets .. etc" by Twitter's search engine .
TwitterScraper Script for scrape user data like "id,username,fullname,followers,tweets .. etc" by Twitter's search engine . Screenshot Data Users Only
19 Nov 17, 2022
SearchifyX, predecessor to Searchify, is a fast Quizlet, Quizizz, and Brainly webscraper with various stealth features.
SearchifyX SearchifyX, predecessor to Searchify, is a fast Quizlet, Quizizz, and Brainly webscraper with various stealth features. SearchifyX lets you
28 Dec 20, 2022
FilmMikirAPI - A simple rest-api which is used for scrapping on the Kincir website using the Python and Flask package
FilmMikirAPI - A simple rest-api which is used for scrapping on the Kincir website using the Python and Flask package
1 Nov 17, 2022
抢京东茅台脚本，定时自动触发，自动预约，自动停止
jd_maotai 抢京东茅台脚本，定时自动触发，自动预约，自动停止 小白信用 99.6，暂时还没抢到过，朋友 80 多抢到了一瓶，所以我感觉是跟信用分没啥关系，完全是看运气的。
117 Dec 22, 2022
Snowflake database loading utility with Scrapy integration
Snowflake Stage Exporter Snowflake database loading utility with Scrapy integration. Meant for streaming ingestion of JSON serializable objects into S
0 Dec 06, 2021
Crawl BookCorpus
These are scripts to reproduce BookCorpus by yourself.
590 Jan 03, 2023