Skip to content

kcsoc/society-email-scrape

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

society-email-scrape

How to automatically generate new data

  • Go to unis.yml
  • Add your uni
  • Create a Pull Request
  • GitHub Actions bot will automatically update as per your PR
  • When I approve the PR your Uni will automatically be loaded into the website
  • https://kcsoc.github.io/society-email-scrape/

How to run yourself

Add any URLs to unis.yml

PS: Don't forget to leave a trailing newline at the end of the file

git clone https://github.com/kcsoc/society-email-scrape.git
cd society-email-scrape
./main.sh

How to test for a single university

  1. Choose a university (on your own or from unis.yml)
  2. Run python main.py UNIVERSITY_URL

To run with debug mode

The program features a debug mode. To enable it, simply preface the python command with DEBUG_MODE=true

For example:

DEBUG_MODE=true python3 main.py https://www.imperialcollegeunion.org/activities/a-to-z

Or set the DEBUG_MODE variable to true globally in your shell with, export DEBUG_MODE=true

Note: DEBUG_MODE is enabled when the variable is set, to unset, use the unset keyword in bash