Current github repo provides implementation described in paper: "Identification of Twitter Bots based on an Explainable ML Framework: the US 2020 Elections Case Study" accepted in ICWSM 2022 conference.

Code folder contain all scripts for:

• feature selection
• parameter fine tuning
• final model performance evaluation

Installation of required packages: pip3 install -r requirements.txt

Developed Bot Detection ML framework was trained and tested over Twitter dataset.
The particular dataset was collected with use of Twitter API during the 2020 US Election period. During this period we manage to collect tweets/retweets that contain particular hashtags that are correlated with the 2020 US Election topic. We provide our dataset in the form of two separated files. In those files we provide already extracted features for each user that was labeled according to our labeling process.

1. data/us_2020_election_data.csv
   • contain user features and labels that was active in twitter during the month of September
   • we utilize it during our training/validation/testing phase of our implementation
2. data/us_2020_election_data_october.csv
   • is utilized in order to identify how a developed model can perform over the unseen data that was collected during the month of October.

Both files was labeled with use of Botometer and BotSentinel Bot Detection tools. We keep only a union of users where both tools agree if a particular account is bot or normal user. Created pipeline allow us to keep user labels as clear as possible. We remove user ids in order to keep private the user identity, we provide only labels and extracted features which do not provide any sensitive user information.

In the plot folder we store performance metrics results in the form of figures.

Execution flow:

1. code/feature_selection.py
   • Provide file with selected features in code folder
2. code/model_fine_tuning.py
   • Store parameters of best model after the fine-tuning
3. code/performance_measurement.py
   • Evaluate selected model and generates figures of ROC and PR AUCs. Also provide SHAP figure that explain the model decision making based on 20 most important features.

Developed framework provides state of the art performance over the Twitter US Election data. Based on the performance of unseen data we assume that is possible utilization of developed methods over data more than one month after training data, since the performance drop is not significant.

Citation:

@inproceedings{shevtsov2022identification,
  title={Identification of Twitter Bots Based on an Explainable Machine Learning Framework: The US 2020 Elections Case Study},
  author={Shevtsov, Alexander and Tzagkarakis, Christos and Antonakaki, Despoina and Ioannidis, Sotiris},
  booktitle={Proceedings of the International AAAI Conference on Web and Social Media},
  volume={16},
  pages={956--967},
  year={2022}
}