BERTopic
BERTopic is a topic modeling technique that leverages
BERTopic supports guided, (semi-) supervised, and dynamic topic modeling. It even supports visualizations similar to LDAvis!
Corresponding medium posts can be found here and here.
Installation
Installation, with sentence-transformers, can be done using pypi:
pip install bertopic
You may want to install more depending on the transformers and language backends that you will be using. The possible installations are:
pip install bertopic[flair]
pip install bertopic[gensim]
pip install bertopic[spacy]
pip install bertopic[use]
Getting Started
For an in-depth overview of the features of BERTopic you can check the full documentation here or you can follow along with one of the examples below:
| Name | Link |
|---|---|
| Topic Modeling with BERTopic |  |
| (Custom) Embedding Models in BERTopic | |
| Advanced Customization in BERTopic | |
| (semi-)Supervised Topic Modeling with BERTopic | |
| Dynamic Topic Modeling with Trump's Tweets | |
| Topic Modeling arXiv Abstracts |  |
Quick Start
We start by extracting topics from the well-known 20 newsgroups dataset containing English documents:
from bertopic import BERTopic
from sklearn.datasets import fetch_20newsgroups
docs = fetch_20newsgroups(subset='all', remove=('headers', 'footers', 'quotes'))['data']
topic_model = BERTopic()
topics, probs = topic_model.fit_transform(docs)
After generating topics and their probabilities, we can access the frequent topics that were generated:
>>> topic_model.get_topic_info()
Topic Count Name
-1 4630 -1_can_your_will_any
0 693 49_windows_drive_dos_file
1 466 32_jesus_bible_christian_faith
2 441 2_space_launch_orbit_lunar
3 381 22_key_encryption_keys_encrypted
-1 refers to all outliers and should typically be ignored. Next, let's take a look at the most frequent topic that was generated, topic 0:
>>> topic_model.get_topic(0)
[('windows', 0.006152228076250982),
('drive', 0.004982897610645755),
('dos', 0.004845038866360651),
('file', 0.004140142872194834),
('disk', 0.004131678774810884),
('mac', 0.003624848635985097),
('memory', 0.0034840976976789903),
('software', 0.0034415334250699077),
('email', 0.0034239554442333257),
('pc', 0.003047105930670237)]
NOTE: Use BERTopic(language="multilingual") to select a model that supports 50+ languages.
Visualize Topics
After having trained our BERTopic model, we can iteratively go through hundreds of topics to get a good understanding of the topics that were extracted. However, that takes quite some time and lacks a global representation. Instead, we can visualize the topics that were generated in a way very similar to LDAvis:
topic_model.visualize_topics()
We can create an overview of the most frequent topics in a way that they are easily interpretable. Horizontal barcharts typically convey information rather well and allow for an intuitive representation of the topics:
topic_model.visualize_barchart()
Find all possible visualizations with interactive examples in the documentation here.
Embedding Models
BERTopic supports many embedding models that can be used to embed the documents and words:
- Sentence-Transformers
- Flair
- Spacy
- Gensim
- USE
Sentence-Transformers is typically used as it has shown great results embedding documents meant for semantic similarity. Simply select any from their documentation here and pass it to BERTopic:
topic_model = BERTopic(embedding_model="all-MiniLM-L6-v2")
Flair allows you to choose almost any
from flair.embeddings import TransformerDocumentEmbeddings
roberta = TransformerDocumentEmbeddings('roberta-base')
topic_model = BERTopic(embedding_model=roberta)
Click here for a full overview of all supported embedding models.
Dynamic Topic Modeling
Dynamic topic modeling (DTM) is a collection of techniques aimed at analyzing the evolution of topics over time. These methods allow you to understand how a topic is represented over time. Here, we will be using all of Donald Trump's tweet to see how he talked over certain topics over time:
import re
import pandas as pd
trump = pd.read_csv('https://drive.google.com/uc?export=download&id=1xRKHaP-QwACMydlDnyFPEaFdtskJuBa6')
trump.text = trump.apply(lambda row: re.sub(r"http\S+", "", row.text).lower(), 1)
trump.text = trump.apply(lambda row: " ".join(filter(lambda x:x[0]!="@", row.text.split())), 1)
trump.text = trump.apply(lambda row: " ".join(re.sub("[^a-zA-Z]+", " ", row.text).split()), 1)
trump = trump.loc[(trump.isRetweet == "f") & (trump.text != ""), :]
timestamps = trump.date.to_list()
tweets = trump.text.to_list()
Then, we need to extract the global topic representations by simply creating and training a BERTopic model:
topic_model = BERTopic(verbose=True)
topics, probs = topic_model.fit_transform(tweets)
From these topics, we are going to generate the topic representations at each timestamp for each topic. We do this by simply calling topics_over_time and pass in his tweets, the corresponding timestamps, and the related topics:
topics_over_time = topic_model.topics_over_time(tweets, topics, timestamps, nr_bins=20)
Finally, we can visualize the topics by simply calling visualize_topics_over_time():
topic_model.visualize_topics_over_time(topics_over_time, top_n_topics=6)
Overview
For quick access to common functions, here is an overview of BERTopic's main methods:
| Method | Code |
|---|---|
| Fit the model | .fit(docs) |
| Fit the model and predict documents | .fit_transform(docs) |
| Predict new documents | .transform([new_doc]) |
| Access single topic | .get_topic(topic=12) |
| Access all topics | .get_topics() |
| Get topic freq | .get_topic_freq() |
| Get all topic information | .get_topic_info() |
| Get representative docs per topic | .get_representative_docs() |
| Get topics per class | .topics_per_class(docs, topics, classes) |
| Dynamic Topic Modeling | .topics_over_time(docs, topics, timestamps) |
| Update topic representation | .update_topics(docs, topics, n_gram_range=(1, 3)) |
| Reduce nr of topics | .reduce_topics(docs, topics, nr_topics=30) |
| Find topics | .find_topics("vehicle") |
| Save model | .save("my_model") |
| Load model | BERTopic.load("my_model") |
| Get parameters | .get_params() |
For an overview of BERTopic's visualization methods:
| Method | Code |
|---|---|
| Visualize Topics | .visualize_topics() |
| Visualize Topic Hierarchy | .visualize_hierarchy() |
| Visualize Topic Terms | .visualize_barchart() |
| Visualize Topic Similarity | .visualize_heatmap() |
| Visualize Term Score Decline | .visualize_term_rank() |
| Visualize Topic Probability Distribution | .visualize_distribution(probs[0]) |
| Visualize Topics over Time | .visualize_topics_over_time(topics_over_time) |
| Visualize Topics per Class | .visualize_topics_per_class(topics_per_class) |
Citation
To cite BERTopic in your work, please use the following bibtex reference:
@misc{grootendorst2020bertopic,
author = {Maarten Grootendorst},
title = {BERTopic: Leveraging BERT and c-TF-IDF to create easily interpretable topics.},
year = 2020,
publisher = {Zenodo},
version = {v0.9.4},
doi = {10.5281/zenodo.4381785},
url = {https://doi.org/10.5281/zenodo.4381785}
}
22 Dec 29, 2022
55 Nov 28, 2022
11 Mar 28, 2022
478 Dec 25, 2022
61 Jan 01, 2023
1 Jan 20, 2022
34 Dec 18, 2022
110 Jan 01, 2023
1 Feb 16, 2022
21 Nov 24, 2022
284 Nov 25, 2022
119 Dec 30, 2022
3.2k Dec 28, 2022
136 Dec 20, 2022
226 Dec 30, 2022
32 Dec 14, 2022
121 Dec 27, 2022
1 Dec 23, 2021
50 Mar 22, 2022
94 Dec 27, 2022