Code for the paper "Combining Textual Features for the Detection of Hateful and Offensive Language"

Last update: Aug 04, 2022

Overview

The repository provides the source code for the paper "Combining Textual Features for the Detection of Hateful and Offensive Language" submitted to HASOC 2021 English Subtask 1A.

Publication

Installation (requires >=Python 3.6 )

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

download the 'resources.zip' file here: https://drive.google.com/file/d/1X88cMrLVpAcJd5Z4Gg6MfTLclIuGF-d6/view?usp=sharing
extract the content of 'resources.zip'

Training and Evaluation on HASOC datasets (2019, 2020, 2021)

Execute the following command to train and evaluate the model. The evaluation results are saved under the folder 'results'.

python main.py -c config.json

Optimizing Hyperparameters

The "config.json" file contains hyperparameters that can be changed to train different variants of the model.

{
  "base_dir": "",
  "batch_size": 64,
  "epochs": 20,
  "epoch_patience": 5,
  "bert_model_dir": "resources/hatebert",
  "monitor": "loss",
  "tweet_text_seq_len": 80,
  "tweet_text_char_len": 128,
  "char_size": 29,
  "max_learning_rate": 0.001,
  "end_learning_rate": 0.0000001,
  "rnn_type": "lstm",
  "rnn_layer_size": 200,
  "text_models": ["char_emb", "bert", "hate_words"],
  "normalize_text": true,
  "dataset_year": "2021",
  "optimizer": "adam",
  "text_use_attention": false,
  "oversample": true,
  "feature_normalization_layer_size": 512,
  "min_feature_normalization_layer_size": 64
}

bert_model_dir

"bert_model_dir": "resources/hatebert"
     OR
"bert_model_dir": "resources/bert-base"

dataset_year

"dataset_year": "2019"
	OR
"dataset_year": "2020"
	OR
"dataset_year": "2021"

text_models

"text_models": ["hate_words"]
	OR
"text_models": ["bert"]
	OR
"text_models": ["char_emb"]
	OR
"text_models": ["char_emb", "bert", "hate_words"]

rnn_type

"rnn_type": "lstm"
	OR
"rnn_type": "gru"
	OR
"rnn_type": "bi-gru"

Code for the paper "Combining Textual Features for the Detection of Hateful and Offensive Language"

Related tags

Overview

The repository provides the source code for the paper "Combining Textual Features for the Detection of Hateful and Offensive Language" submitted to HASOC 2021 English Subtask 1A.

Publication

Installation (requires >=Python 3.6 )

Training and Evaluation on HASOC datasets (2019, 2020, 2021)

Optimizing Hyperparameters

Owner

Sherzod Hakimov

The devkit of the nuPlan dataset.

Working demo of the Multi-class and Anomaly classification model using the CLIP feature space

"Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion"(WWW 2021)

This repo contains source code and materials for the TEmporally COherent GAN SIGGRAPH project.

The dataset of tweets pulling from Twitters with keyword: Hydroxychloroquine, location: US, Time: 2020

SpeechBrain is an open-source and all-in-one speech toolkit based on PyTorch.

DFFNet: An IoT-perceptive Dual Feature Fusion Network for General Real-time Semantic Segmentation

TACTO: A Fast, Flexible and Open-source Simulator for High-Resolution Vision-based Tactile Sensors

Autoregressive Predictive Coding: An unsupervised autoregressive model for speech representation learning

Code for "Primitive Representation Learning for Scene Text Recognition" (CVPR 2021)

Official code for our ICCV paper: "From Continuity to Editability: Inverting GANs with Consecutive Images"

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

This package implements the algorithms introduced in Smucler, Sapienza, and Rotnitzky (2020) to compute optimal adjustment sets in causal graphical models.

Implementation of Kaneko et al.'s MaskCycleGAN-VC model for non-parallel voice conversion.

Experiments for Fake News explainability project

Code and models used in "MUSS Multilingual Unsupervised Sentence Simplification by Mining Paraphrases".

A flexible tool for creating, organizing, and sharing visualizations of live, rich data. Supports Torch and Numpy.

An experiment on the performance of homemade Q-learning AIs in Agar.io depending on their state representation and available actions

Privacy-Preserving Machine Learning (PPML) Tutorial Presented at PyConDE 2022

Pytorch implementation of "Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling"