Social insights is an open source big data project that generates insights about various interesting topics happening every day. Currently, we are focusing on political sentiment analysis due to the upcoming elections in the United States. Data sources include the Twitter API. There is a search engine built to query the insights.
The project is broken down into 3 parts:
- dataproc: All the analytics, database work, and ML.
- search-engine: The API work and the querying/search engine system
- web: the web server to render visualization and integrate dataproc and search-engine components
To run and setup the project you need python, node.js and NPM installed for the visualizations, which can be found here. For the data processing you need python which can found here.
To get started on backend:
git clone https://github.com/dsc-umass/social-insights.git
pip3 install pipenv
pipenv shell
pipenv install
To get started on website:
git clone https://github.com/dsc-umass/social-insights.git
cd web
yarn install
Getting Started with Django Development
After creating pipenv shell to run the development server:
cd webpages
python manage.py runserver
- Twitter API - Sentiment Analysis and Data Analytics
- Flask - API Framework
- PM2 - Production Server and Load Balancing
- React - Frontend framework for the website
- NLTK - For the search engine