Python Webscraping using Selenium

Overview

Web Scraping with Python and Selenium

The code shows how to do web scraping using Python and Selenium.

We use as data the https://sbot.org.br/localize-o-ortopedista/ site to extract all the licensed orthopedists.

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

Prerequisites

What things you need to install the software and how to install them

Installing

A step by step series of examples that tell you how to get a development env running

Install the following Python libraries:

  • selenium - An API to write functional/acceptance tests using Selenium WebDriver.

With:

pip install -r requirements.txt

Chrome Driver

You can find install instructions in the official repository. Remember, to run the project you'll need to update your drive location. If you are on Windows, the current path is C:/chromedriver.exe by default. On Linux/MacOS X, after download the respective driver, you'll need to update 'testcase\test_page.py' (line 8) and 'webscraping.py' - at project's root, line 12.

Running the code

python webscraping.py

Running the tests (with unittest)

python testcase/test_page.py

Contributing

Feel free to submitting pull requests.

License

This project is licensed under the GNU General Public License.

Owner
Luís Miguel Massih Pereira
In love with Dart & Flutter ♥
Luís Miguel Massih Pereira
RAT-el is an open source penetration test tool that allows you to take control of a windows machine.

To prevent RATel from being detected by antivirus, please do not upload the payload to TOTAL VIRUS. Each month I will test myself if the payload gets detected by antivirus. So you’ll have a photo eve

218 Dec 16, 2022
The (Python-based) mining software required for the Game Boy mining project.

ntgbtminer - Game Boy edition This is a version of ntgbtminer that works with the Game Boy bitcoin miner. ntgbtminer ntgbtminer is a no thrills getblo

Ghidra Ninja 31 Nov 04, 2022
A pytest plugin that enables you to test your code that relies on a running Elasticsearch search engine

pytest-elasticsearch What is this? This is a pytest plugin that enables you to test your code that relies on a running Elasticsearch search engine. It

Clearcode 65 Nov 10, 2022
Auto Click by pyautogui and excel operations.

Auto Click by pyautogui and excel operations.

Janney 2 Dec 21, 2021
Python Projects - Few Python projects with Testing using Pytest

Python_Projects Few Python projects : Fast_API_Docker_PyTest- Just a simple auto

Tal Mogendorff 1 Jan 22, 2022
Hamcrest matchers for Python

PyHamcrest Introduction PyHamcrest is a framework for writing matcher objects, allowing you to declaratively define "match" rules. There are a number

Hamcrest 684 Dec 29, 2022
Automates hiketop+ crystal earning using python and appium

hikepy Works on poco x3 idk about your device deponds on resolution Prerquests Android sdk java adb Setup Go to https://appium.io/ Download and instal

4 Aug 26, 2022
MongoDB panel for the Flask Debug Toolbar

Flask Debug Toolbar MongoDB Panel Info: An extension panel for Rob Hudson's Django Debug Toolbar that adds MongoDB debugging information Author: Harry

Cenk Altı 4 Dec 11, 2019
API Rest testing FastAPI + SQLAchmey + Docker

Transactions API Rest Implement and design a simple REST API Description We need to a simple API that allow us to register users' transactions and hav

TxeMac 2 Jun 30, 2022
LuluTest is a Python framework for creating automated browser tests.

LuluTest LuluTest is an open source browser automation framework using Python and Selenium. It is relatively lightweight in that it mostly provides wr

Erik Whiting 14 Sep 26, 2022
No longer maintained, please migrate to model_bakery

Model Mommy: Smart fixtures for better tests IMPORTANT: Model Mommy is no longer maintained and was replaced by Model Bakery. Please, consider migrati

Bernardo Fontes 917 Oct 04, 2022
Load and performance benchmark tool

Yandex Tank Yandextank has been moved to Python 3. Latest stable release for Python 2 here. Yandex.Tank is an extensible open source load testing tool

Yandex 2.2k Jan 03, 2023
tidevice can be used to communicate with iPhone device

tidevice can be used to communicate with iPhone device

Alibaba 1.8k Jan 08, 2023
buX Course Enrollment Automation

buX automation BRACU - buX course enrollment automation Features: Automatically enroll into multiple courses at a time. Find courses just entering cou

Mohammad Shakib 1 Oct 06, 2022
How to Create a YouTube Bot that Increases Views using Python Programming Language

YouTube-Bot-in-Python-Selenium How to Create a YouTube Bot that Increases Views using Python Programming Language. The app is for educational purpose

Edna 14 Jan 03, 2023
0hh1 solver for the web (selenium) and also for mobile (adb)

0hh1 - Solver Aims to solve the '0hh1 puzzle' for all the sizes (4x4, 6x6, 8x8, 10x10 12x12). for both the web version (using selenium) and on android

Adwaith Rajesh 1 Nov 05, 2021
Statistical tests for the sequential locality of graphs

Statistical tests for the sequential locality of graphs You can assess the statistical significance of the sequential locality of an adjacency matrix

2 Nov 23, 2021
Python Webscraping using Selenium

Web Scraping with Python and Selenium The code shows how to do web scraping using Python and Selenium. We use as data the https://sbot.org.br/localize

Luís Miguel Massih Pereira 1 Dec 01, 2021
Docker-based integration tests

Docker-based integration tests Description Simple pytest fixtures that help you write integration tests with Docker and docker-compose. Specify all ne

Avast 326 Dec 27, 2022
Minimal example of how to use pytest with automated 'devops' style automated test runs

Pytest python example with automated testing This is a minimal viable example of pytest with an automated run of tests for every push/merge into the m

Karma Computing 2 Jan 02, 2022