TriTopic

Install

pip install git+https://github.com/SmartVisions-AI/tritopic.git
# LLM features: set an OpenAI key (only needed for TopicInterpreter and the Decisions API)
export OPENAI_API_KEY=sk-...

Python 3.9-3.13. The first fit downloads the embedding model all-MiniLM-L6-v2 (about 90 MB). Extras: [llm], [benchmark], [turftopic], [full], [dev].

Your first model

from tritopic import TriTopic

model = TriTopic()                       # n_topics="auto"
labels = model.fit_transform(documents)  # one topic id per document

model.get_topic_info()                   # Topic, Size, Keywords, Label, Coherence
model.get_representative_docs(0, n_docs=3)
model.visualize().write_html("map.html")

Then

from tritopic import TopicInterpreter
from tritopic.research import topic_reliability, saturation_curve, methods_report

TopicInterpreter(domain_hint="customer reviews").interpret(model)   # labels + descriptions (LLM)
topic_reliability(model)                                            # which topics can you trust?
saturation_curve(model)                                             # enough data?
print(methods_report(model, corpus="12,400 hotel reviews"))         # methods paragraph

Every feature with code: user guide. Runnable scripts: examples.

Task-by-task instructions with code. Every snippet comes from a runnable script in examples/; the technical documentation explains how each part works and why.

1. Installation

pip install git+https://github.com/SmartVisions-AI/tritopic.git     # 2.5 (this repository)
pip install "tritopic[benchmark] @ git+https://github.com/SmartVisions-AI/tritopic.git"  # + BERTopic, datasets

Python 3.9-3.13. The first fit downloads the embedding model (all-MiniLM-L6-v2, about 90 MB).

The LLM features (TopicInterpreter, the Decisions API functions, codebook(), intercoder_reliability()) call the OpenAI API over HTTPS and only need an API key:

export OPENAI_API_KEY=sk-...          # Windows PowerShell: $env:OPENAI_API_KEY = "sk-..."

2. Quick start

Script: examples/01_quickstart.py

from tritopic import TriTopic

model = TriTopic()                          # n_topics="auto"
labels = model.fit_transform(documents)     # one topic id per document

print(model.get_topic_info())               # Topic, Size, Keywords, Label, Coherence, ...
topic = model.get_topic(0)
print(topic.keywords, topic.size)
print(model.get_representative_docs(0, n_docs=3))
model.visualize().write_html("map.html")    # interactive 2D map

documents is a list of strings. Every document gets a topic; -1 (outlier) only appears for clusters smaller than min_cluster_size.

3. Your own data

Script: examples/02_your_own_data.py

import re
import pandas as pd
from tritopic import TriTopic, TriTopicConfig

df = pd.read_csv("my_texts.csv")
df["text"] = df["text"].fillna("").map(lambda t: re.sub(r"\s+", " ", t).strip())
df = df[df["text"].str.split().str.len() >= 10].reset_index(drop=True)   # drop very short texts
documents = df["text"].tolist()

# Optional: your own embeddings (any sentence-transformers model, GPU, caching ...)
from sentence_transformers import SentenceTransformer
embeddings = SentenceTransformer("all-MiniLM-L6-v2").encode(documents, normalize_embeddings=True)

config = TriTopicConfig(min_cluster_size=10, n_keywords=10, random_state=42, verbose=False)
model = TriTopic(config=config, n_topics=8)
df["topic"] = model.fit_transform(documents, embeddings=embeddings)

info = model.get_topic_info().set_index("Topic")
df["topic_keywords"] = df["topic"].map(info["Keywords"])
df["topic_probability"] = model.probabilities_.max(axis=1)
df.to_csv("my_texts_with_topics.csv", index=False)

Tips:

• Clean only what is noise for your question (HTML, signatures, boilerplate). Stop words are handled by language; do not stem or lemmatize, the embeddings need natural text.
• Long documents are fine. Embedding models truncate (MiniLM: 256 tokens), so for books or long reports split into paragraphs or sections first and fit on the parts.
• Pre-computed embeddings make experiments fast: encode once, fit many times.

4. How many topics?

Script: examples/03_shape_the_topics.py

TriTopic()                                            # "auto": coarsest partition with near-best keyword coherence
TriTopic(n_topics=12)                                 # exactly 12 topics
TriTopic(config=TriTopicConfig(auto_resolution=False, resolution=0.5))   # fixed Leiden resolution

model.resolution_                                     # chosen resolution
model.resolution_search_                              # [(resolution, n_topics, coherence), ...]

Auto mode tends to give more topics than a corpus has classes (BBC: about 6-14 for 5 sections). If you know the granularity you need, pass n_topics; if you know the categories, use seeds (§10).

5. Shaping the topics after fitting

model.reduce_outliers(strategy="neighbors")      # or "embeddings", or "decisions" (LLM, §9)
model.reduce_topics(10)                          # merge the most similar topics until 10 remain
model.merge_topics([4, 7])                       # merge specific topics
model.divide(topic_id=0, n_subtopics=3)          # split one topic

hierarchy = model.build_hierarchy(n_levels=3)    # coarse -> fine
for root in hierarchy.roots:
    print(root.node_id, root.size, root.keywords[:4], [c.node_id for c in root.children])
model.visualize_hierarchy_tree().write_html("hierarchy.html")

All operations refresh keywords, centroids and probabilities.

6. New documents, saving and loading

Script: examples/04_new_documents_and_saving.py

model.transform(new_docs)            # topic ids; -1 if less similar than model.outlier_threshold_
model.transform_proba(new_docs)      # (n_new, n_topics), columns in topic-id order
model.probabilities_                 # the same for the training documents

model.save("model.pkl")
model = TriTopic.load("model.pkl")   # keeps topics, labels, seeds, reducer, embeddings

The outlier threshold is calibrated on the training data (1st percentile of the training documents' similarity to their own topic). Set TriTopicConfig(outlier_threshold=0.3) for a fixed value.

7. Metadata, time and languages

Script: examples/05_metadata_time_and_languages.py

config = TriTopicConfig(use_metadata_view=True, metadata_weight=0.2)
model = TriTopic(config=config).fit(documents, metadata=df[["source", "rating"]])
pd.crosstab(df["source"], model.labels_, normalize="index")       # topic share per source

from tritopic import TopicVisualizer
TopicVisualizer().plot_topic_over_time(labels=model.labels_, timestamps=df["date"].tolist(),
                                       topics=model.topics_).write_html("over_time.html")

TriTopic(language="german")                        # german / french / spanish stop words
TriTopic(language="multilingual")                  # switches to BAAI/bge-m3 (100+ languages)

Metadata only strengthens edges between documents that are already similar in content; it never connects unrelated documents. For how topics change content over time, see topic_evolution() in §11 and example 12.

8. Let an LLM interpret the topics

Script: examples/06_llm_interpretation.py

from tritopic import TopicInterpreter

interpreter = TopicInterpreter(domain_hint="customer reviews of hotels", language="English")
results = interpreter.interpret(model)          # writes label + description into model.topics_
for tid, r in results.items():
    print(tid, r.label, r.verdict, r.description, r.aspects)

log = interpreter.refine(model, results=results)   # split topics the LLM calls "mixed"
print(interpreter.summarize(model))                # overview of the topic landscape

refine() only keeps a split when the graph split agrees with the LLM's sub-themes. Default model gpt-6-luna; pass model="..." to use another. Cost: about 1,500 input tokens per topic.

Labels only, with Anthropic or OpenAI SDKs: model.generate_labels(LLMLabeler(provider="anthropic", api_key=...)). Without any LLM: model.generate_labels(SimpleLabeler(n_words=3)).

9. Typed LLM judgements (Decisions API)

Script: examples/07_decisions_api.py

from tritopic.integrations.decisions import (DecisionsClient, word_intrusion, rate_topics,
                                             assign_documents, suggest_merges, apply_merges)

client = DecisionsClient()
keywords = [t.keywords for t in model.topics_ if t.topic_id != -1]
rate_topics(keywords, client)                               # 0-3 per topic
word_intrusion(keywords, client).intruder_probability       # automated word-intrusion test
labels, conf = assign_documents(model, new_docs, client, allow_other=True)   # -1 = none fits
model.reduce_outliers(strategy="decisions", decisions_client=client)
apply_merges(model, suggest_merges(model, client, threshold=0.5))

Merge suggestions are conservative and often empty at 0.5; lower the threshold to see borderline pairs and check them before merging.

10. Start from a codebook (seeded topics)

Script: examples/09_seeded_topics.py

When you already know (some of) the categories, describe each in one sentence or a list of words:

seeds = {
    "Space": "Space flight: NASA, rockets, satellites, orbits, the moon and planets",
    "Cars": "Cars: engines, driving, car models, dealers, tires and fuel",
    "Medicine": ["disease", "doctor", "patient", "treatment", "drug"],     # word lists work too
}
model = TriTopic().fit(documents, seeds=seeds)

model.get_topic_info()[["Topic", "Seed", "Size", "Keywords"]]
model.seed_topics_          # {seed name: topic id}; TopicInfo.seed holds the seed name
model.emergent_topics_      # ids of the topics nobody asked for: the interesting part
model.seed_anchors_         # the documents pinned to each seed

The best-matching documents of each seed are pinned to one topic; all other documents join a seeded topic or form new ones. A seed that fits no document is ignored with a warning, which shows codebook categories missing from the data.

• With pre-computed embeddings TriTopic encodes the seeds with embedding_model. If your embeddings come from another model, pass seed_embeddings=encoder.encode(list(seeds.values())).
• TriTopicConfig(seed_anchors=20) sets the number of anchors per seed (default 1% of the corpus, 5-30).
• Seed every theme you know: with a partial codebook a seeded topic can absorb a neighbouring unseeded theme. TopicInterpreter.refine() splits such mixed topics afterwards.

Effect on benchmark data: full codebook ARI 0.515 → 0.624 (dev), 0.399 → 0.498 (held-out); BBC demo with five one-line seeds NMI 0.742 → 0.801, ARI 0.631 → 0.800.

11. Research toolkit

Script: examples/10_research_toolkit.py

from tritopic.research import (topic_reliability, saturation_curve, plot_saturation,
                               bridge_documents, topic_connections, topic_prevalence, compare_groups,
                               distinctive_keywords, topic_evolution, plot_evolution,
                               topic_quotes, methods_report)

Which topics can I trust?

rel = topic_reliability(model)            # refits on 5 random 80% samples
rel[["topic", "label", "reliability", "core_share", "reliable"]]
model.document_stability_                 # per document: share of refits it stayed with its topic

Report topics with reliability below 0.7 with care, or merge them. method="consensus" is instant (uses the stored Leiden runs) but a weaker signal.

Do I have enough data?

sat = saturation_curve(model)
print(sat)                                # e.g. "all topics discoverable with 30% of the documents"
sat.summary                               # fraction, n_docs, recovered
sat.point_95, sat.point_all, sat.saturated, sat.late_topics
plot_saturation(sat).show()

If late_topics is not empty, those topics only appear with (nearly) all documents: collect more data before drawing conclusions about them.

Where do topics meet?

bridge_documents(model, top_n=10)         # doc, topic, other_topic, bridge_score, text
topic_connections(model).head()           # topic_a, topic_b, strength, bridge_docs

How big is each topic, and does it differ between groups?

topic_prevalence(model)                                   # share with 95% Wilson interval
topic_prevalence(model, groups=df["country"])             # per group
compare_groups(model, df["country"])                      # chi2 / Fisher, Cramér's V, BH-adjusted p
distinctive_keywords(model, df["country"], "AT", "DE", n=10)              # words that separate groups
distinctive_keywords(model, df["country"], "AT", "DE", topic_id=3)        # ... within one topic

How do topics develop over time?

evo = topic_evolution(model, df["date"], freq="Q")        # "M", "Q", "Y"
evo.events                                                # birth / continuation / split / merge / death
evo.events[evo.events.event == "birth"]                   # what was new, with keywords
evo.nodes.docs                                            # document indices of each period topic
evo.lineage(3)                                            # period topics of global topic 3
plot_evolution(evo).show()                                # Sankey diagram

Each period gets its own graph and is clustered finer than the global model (resolution=4 × model.resolution_), so a new event inside a broad topic shows up as a birth. Topics of consecutive periods are linked when their centroids have cosine ≥ link_threshold (0.75); lower it for fewer births, raise it for more. Use periods with at least a few hundred documents. A theme that skips a period counts as a new birth. Real-data walkthrough (HuffPost news 2012-2022, finds COVID in 2020 and the war in Ukraine in 2022): examples/12_topic_evolution_news.py.

Quotes and the methods section

topic_quotes(model, n=3)                                  # citable sentences per topic
print(methods_report(model, corpus="12,400 hotel reviews from 2023-2025"))

methods_report is written from the fitted model without an LLM, so it is reproducible. It includes the parameter table and software versions; check and adapt the wording before publishing.

12. Codebook and second coder for manual coding

Script: examples/11_codebook_and_second_coder.py

from tritopic import TopicInterpreter
from tritopic.integrations.decisions import DecisionsClient, intercoder_reliability

interp = TopicInterpreter(domain_hint="interview transcripts")
interp.interpret(model)                                   # labels + descriptions first

codebook = interp.codebook(model, n_quotes=3)             # name, definition, inclusion, exclusion,
codebook.to_excel("codebook.xlsx")                        # coding_notes, anchor_examples (real quotes)

ic = intercoder_reliability(model, DecisionsClient(), sample_size=200)
print(ic.kappa, ic.agreement)                             # Cohen's kappa, raw agreement
ic.per_topic                                              # precision / recall / F1 per topic
ic.confusion                                              # rows: TriTopic, columns: LLM

The second coder sees only what a human coder would see in a codebook (label, description, keywords), so kappa measures whether the topics can be applied consistently. Kappa above 0.8 counts as almost perfect agreement, 0.6-0.8 as substantial (Landis & Koch, 1977). Low F1 for a topic points to a category that is hard to apply: sharpen its definition or merge it.

13. Evaluation and visualization

Script: examples/08_evaluate_and_visualize.py

model.evaluate()         # coherence (NPMI, whole corpus), diversity, stability, outlier ratio

from sklearn.metrics import normalized_mutual_info_score
normalized_mutual_info_score(true_labels, model.labels_)        # if you have reference labels

model.visualize()                       # 2D document map
model.visualize_topics(n_keywords=8)    # keyword bars
model.visualize_hierarchy()             # dendrogram
model.visualize_overlap(threshold=0.1)  # topic co-occurrence
model.visualize_hierarchy_tree()        # after build_hierarchy()

All visualizations return Plotly figures (.show(), .write_html()).

14. Research charts

Script: examples/13_research_charts.py

Sixteen Plotly charts for analysing and presenting a model. Each returns a figure (.show(), .write_html()); the numbers behind it come from tritopic.research, so you can also report them as tables.

from tritopic.research import topic_reliability
from tritopic.visualization import charts

rel = topic_reliability(model)
charts.plot_topic_table(model, reliability=rel).show()     # overview tiles
charts.plot_triview(model).show()                          # what holds each topic together
charts.plot_resolution_ladder(model).show()                # how topics split when you zoom in
charts.plot_topic_onions(model).show()                     # core and edge, stability, bridges
charts.plot_keyword_barcode(model, topic_id=0).show()      # is a keyword carried by the whole topic?
charts.plot_trust(model, rel).show()                       # which topics can go into the paper

Seven of these chart types are new: the Tri-View triangle, the zoom ladder, topic onions, the keyword barcode, the birth timeline, codebook coverage and saturation with topic dots. They need what only TriTopic has: two graph views, consensus and bootstrap refits, seeds and topic evolution.

15. Configuration recipes

from tritopic import TriTopicConfig

# Short texts (tweets, reviews): denser graph, smaller clusters allowed
TriTopicConfig(n_neighbors=20, min_cluster_size=5)

# Long, technical documents: more weight on wording
TriTopicConfig(semantic_weight=0.4, lexical_weight=0.4)

# Large corpora (50k+): fewer consensus runs, fewer refinement iterations
TriTopicConfig(n_consensus_runs=5, max_iterations=3)

# Broader topics without fixing the number
TriTopicConfig(auto_resolution_tolerance=0.10)

# Reproducibility
TriTopicConfig(random_state=42)

All parameters with defaults: technical documentation §15.

16. FAQ and troubleshooting

The first fit is slow. The embedding model is downloaded and the documents are encoded once; with pre-computed embeddings a fit of 2,000 documents takes about 7-10 s. UMAP compiles with numba on first use.

Too many / too few topics. Pass n_topics=k, use reduce_topics(k), or raise/lower auto_resolution_tolerance. With known categories, use seeds.

One topic mixes two themes. model.divide(topic_id, n_subtopics=2), or let TopicInterpreter.refine() decide.

Results differ between runs. Set random_state. Consensus clustering keeps runs close (NMI spread 0.014 across seeds in the benchmark); topic_reliability() shows which topics are unstable.

Keywords contain domain boilerplate (e.g. "hotel" in hotel reviews). Words in nearly every document get a low IDF weight already; to remove them completely, delete them from the texts before fitting (single frequent words hardly change the embeddings).

No OpenAI key. All non-LLM features work without a key. SimpleLabeler gives keyword labels.

Dropbox / OneDrive folders. Keep virtual environments and model caches outside synced folders.

How TriTopic works, why it is built this way, and how to use every part of it. For a quick start see the README, for task-by-task code the user guide; for release notes the CHANGELOG.

1. Design principles

1. No single view is enough. Embeddings capture meaning but blur specific wording; TF-IDF captures wording but misses synonyms; metadata captures structure but not content. TriTopic fuses them in one graph. Removing the lexical view costs about 0.05 NMI on the development benchmarks.
2. Stochastic clustering needs stabilising. One Leiden run depends on its seed. TriTopic runs it ten times and builds a consensus; across seeds the NMI spread is 0.014 (BERTopic: 0.105).
3. Every document gets a topic. Graph clustering assigns every document; only clusters smaller than min_cluster_size become outliers, which is rare.
4. Decide on development data, report on held-out data. Every default in 2.4 was chosen on separate development splits; the benchmark numbers come from untouched evaluation splits.

2. The pipeline step by step

model.fit(documents, embeddings=None, metadata=None):

Full-dimensional, unrefined embeddings are used for centroids, probabilities, transform(), outlier reduction and merging, so new documents are always compared in the same space.

3. Graph construction

Semantic graph (graph_type="hybrid", default). kNN on the reduced embeddings with Euclidean distance (reduced_metric). UMAP output is a Euclidean layout; cosine angles around its arbitrary origin would join clusters that happen to lie on the same ray. Similarities use a self-tuning Gaussian kernel, exp(-d² / (σᵢ σⱼ)) with σ the distance to the k-th neighbour, so dense and sparse regions get comparable weights. Without dimensionality reduction the metric is cosine on the full embeddings.

• Mutual kNN: an edge only if both points are among each other's neighbours (removes noise bridges).
• SNN: weight = shared neighbours / k (structural similarity), computed as a sparse product A·Aᵀ.
• Hybrid: (1 - snn_weight) · mutual + snn_weight · SNN, each max-normalised.

Lexical graph. Mutual kNN with cosine similarity on TF-IDF (top 10,000 terms, sublinear TF, uni- and bigrams, stop words of language). It is built once per fit and cached.

Fusion. w_sem · semantic + w_lex · lexical + 0.1 · min(semantic, lexical); weights are renormalised over the active views. The last term rewards edges both views agree on.

Metadata. Columns are encoded per document (categorical: exact match, including strings and booleans; numerical and datetime: min-max scaled, similar if 1 - |Δ| > 0.8). The metadata similarity of an edge, averaged over columns, is added with weight metadata_weight to existing edges only. Connecting every pair of documents that share a category would add O(n²) edges and let metadata dominate the topics.

4. Consensus clustering

1. Leiden (RBConfigurationVertexPartition, weighted) runs n_consensus_runs times with seeds random_state + i.
2. For every graph edge, the agreement is the share of runs that put both ends in the same cluster (Lancichinetti & Fortunato, 2012).
3. Edges with agreement below 50% are dropped (each node keeps its most consistent edge), the rest are weighted by agreement × original weight, and Leiden runs again on this consensus graph.
4. Steps 2-3 repeat until all runs agree or the consensus graph stops changing (max. 5 rounds); the run with the highest mean ARI to the others is returned.
5. Clusters smaller than min_cluster_size become outliers (-1).

Memory and time grow with the number of edges. Earlier versions built an all-pairs co-occurrence matrix, which needed several GB beyond ~20k documents. stability_score_ is the mean pairwise ARI of the initial runs.

5. Choosing the resolution

The Leiden resolution sets the granularity (higher = more topics).

n_topics="auto" (with auto_resolution=True, default): on the initial graph, one Leiden run per resolution in resolution_range (default 0.01-1.0, 15 log-spaced steps). For each partition, keywords are extracted and scored by mean NPMI from the cached document-term matrix. TriTopic keeps the coarsest resolution whose coherence is within auto_resolution_tolerance (5%) of the best, among partitions whose largest topic holds at most auto_resolution_max_share (50%) of the documents. Very coarse partitions score high NPMI on generic, frequently co-occurring words; the share guard prevents collapses to two topics.

On the development splits (3 seeds) this gave NMI 0.594 / ARI 0.474, against 0.565 / 0.415 for a fixed resolution. It still tends to find more topics than a corpus has classes.

n_topics=k: log-space bisection for a resolution with k clusters of at least min_cluster_size. The bracket widens (×4 steps) while the count still moves towards k; on very clean or very noisy graphs a fixed range silently missed the target. If the result overshoots, reduce_topics(k) merges down.

Fixed: auto_resolution=False uses resolution (default 0.3; Leiden's usual 1.0 over-segments kNN graphs badly, e.g. 60 topics for 6 newsgroup categories).

6. Iterative refinement

After clustering, the graph-building embeddings (the reduced ones) are pulled towards their topic centroid and the semantic graph is re-clustered:

refined = (1 - b) · x + b · centroid,    b = blend · 1 / (1 + (d / median_d)²)

blend decays from 0.3 to 0.1 over max_iterations; documents far from the centroid move less. The loop stops when the ARI between consecutive partitions reaches convergence_threshold (0.95). Refinement works in the reduced space directly; earlier versions re-projected the full embeddings through UMAP.transform() every iteration, which was slow and noisy. On the development benchmarks refinement does not hurt but has no clear effect either; it can be switched off with use_iterative_refinement=False.

7. Keywords

The corpus is tokenized once (CountVectorizer, uni- and bigrams, stop words of language, min_df=2, max_df=0.95). Tokens must start with two letters, so 000, __ or years never become keywords. The same counts feed the lexical view.

Coverage-weighted c-TF-IDF (keyword_method="ctfidf", default):

score(t) = IDF(t) · √( tf_share(t) · coverage(t) )
tf_share(t) = share of the topic's tokens that are t
coverage(t) = share of the topic's documents that contain t
IDF(t)      = log(N / (1 + df(t)))

Term frequency alone favours words repeated in a few long documents; coverage alone favours boilerplate ("thanks", "mail address"). The geometric mean raised NPMI coherence from 0.210 to 0.305 on the development splits at equal diversity. BERTopic's formula (tf · log(1 + A/f)) scored 0.193 in the same test.

BM25 ("bm25"): average Okapi BM25 weight of a term in the topic's documents relative to the corpus average, times log(1 + frequency). KeyBERT ("keybert"): embedding-based candidates with MMR.

All topics are scored in one sparse product; build_hierarchy() and divide() reuse the cached matrix.

8. Centroids, probabilities and new documents

• topic_embeddings_: mean of the unrefined embeddings per topic.
• probabilities_ (soft_assignment_method="centroid"): softmax(cos(x, centroids) · softmax_temperature) (temperature 5). With "graph", the topic distribution of a document's weighted graph neighbours.
• transform(docs): encode, nearest centroid by cosine; below the outlier threshold → -1. By default (outlier_threshold=None) the threshold is calibrated on the training data: the 1st percentile of the training documents' similarity to their own centroid (model.outlier_threshold_). The fixed 0.35 used up to 2.4 rejected up to 40% of in-domain documents on the benchmark corpora, the calibrated rule about 1%. transform_proba(docs) returns the softmax distribution.

9. Post-fit operations

All operations refresh keywords, centroids and probabilities and reuse the cached token counts.

• reduce_outliers(strategy): "embeddings" (nearest centroid above a threshold), "neighbors" (majority vote of the nearest non-outlier documents), "decisions" (an LLM picks the topic or "other", see §13).
• reduce_topics(n, size_penalty=0.0): repeatedly merges the two topics with the most similar centroids. A size penalty (min/max size)^p is optional; the 0.3 used up to 2.4.0 merged small topics into large unrelated ones (reducing 2k → k topics: NMI 0.542 without vs. 0.591 with the fix on the evaluation splits). Graph-based agglomeration by modularity gain was tested and is worse.
• merge_topics([ids]): merges the given topics into the largest one.
• divide(topic_id, n_subtopics): re-clusters the topic's subgraph with a resolution search in both directions; the new topics get fresh ids.

10. Hierarchies and per-document analysis

build_hierarchy(n_levels=3) clusters the graph at resolutions spaced around resolution_ (/4 … ×4) and links each fine node to the coarse node holding most of its documents. TopicHierarchy.cut(depth), flatten(), get_node(id); visualize_hierarchy_tree() draws it.

get_document_topics(doc_idx, top_n) returns the top topics of one document (centroid or graph method). topic_overlap_matrix(threshold) counts how often two topics are both "active" (probability above the threshold) in the same document; visualize_overlap() shows it as a heatmap.

11. Languages

language sets the stop words for keywords, the lexical view and coherence: "english" (scikit-learn list), "german", "french", "spanish" (built-in lists) or "multilingual" (no stop words). With "multilingual" and the default embedding model, TriTopic switches to BAAI/bge-m3. The token pattern accepts letters of any alphabet.

12. LLM topic interpretation

tritopic.TopicInterpreter (OpenAI Responses API with a strict JSON schema; default model gpt-6-luna; needs OPENAI_API_KEY, no extra package).

Evidence per topic. The top 15 keywords; n_docs (8) example documents chosen by k-means on the topic's embeddings (the document nearest each cluster centre, largest cluster first), so a large sub-group gets several examples while the topic's spread is still covered; the three nearest topics for contrast; topic size and share; an optional domain_hint.

Answer (TopicInterpretation): label, description, aspects, verdict (coherent, mixed, unclear), sub_themes (name, description, supporting example documents), confidence, evidence. interpret(model) writes labels and descriptions into model.topics_ and stores all results in model.interpretations_.

refine(model) splits a topic with divide() when

1. the verdict is mixed with confidence ≥ 0.6,
2. at least two sub-themes are each backed by ≥ 2 example documents (one stray article does not make a topic mixed), and
3. the acceptance rule holds (accept="consistency", default): the example documents of different sub-themes land in different new topics, each sub-theme's examples mostly in one. "coherence" (keyword NPMI of the new topics beats the original), "both" and "none" are available.

Rejected splits are undone; the new topics are interpreted. summarize(model) returns a short overview and a grouping of related topics (model.overview_).

Acceptance rules compared on identical LLM verdicts (10 dataset settings × 2 interpretation runs):

With the default, refine() raised NMI from 0.614 to 0.658 on the development splits and from 0.606 to 0.639 on the held-out evaluation splits (BBC with 3 topics: 0.716 → 0.859).

Model choice (development splits, deliberately coarse topics, 28 topics of which 15 genuinely mixed):

Two lessons from the development: farthest-point sampling of examples over-represented spread-out minorities (the BBC politics/business topic got one politics and six business examples), and keyword NPMI is a poor referee for splits (business keywords are naturally less coherent than political ones, so the correct split was rejected). Proportional sampling and the consistency check fixed both.

13. LLM judgements via the Decisions API

tritopic.integrations.decisions uses OpenAI's Decisions API (POST /v1/decisions, model gpt-6-luna, typed answers: predicate → probability, choice → one option, score → ordered levels). DecisionsClient runs requests in parallel, retries on 429/5xx and caches identical requests.

Measured on the evaluation splits: LLM topic rating 2.21 for TriTopic vs. 1.94 (BERTopic tuned) and 0.49 (default); assignment accuracy 0.657 vs. 0.648 for nearest-centroid, about 74% on the documents it does not abstain on; merge judgements separate same-class pairs better than centroid cosine (AUC 0.63-0.75 vs. 0.61-0.65) but are conservative, so merges rarely change NMI.

14. Evaluation metrics

model.evaluate():

• coherence_mean, coherence_std: NPMI of each topic's keywords over the whole corpus (document co-occurrence; pairs that never co-occur count as -1; bigram keywords are matched with the keyword analyzer). Not comparable with versions before 2.4, which used only each topic's own documents.
• diversity: unique share of all topics' keywords.
• stability: mean pairwise ARI of the consensus runs.
• n_topics, outlier_ratio.

Standalone in tritopic.utils.metrics: compute_coherence, compute_coherence_batch, coherence_from_doc_term, compute_diversity, compute_stability, compute_silhouette, compute_downstream_score.

15. Configuration reference

16. Design decisions and the evidence behind them

All experiments are archived in benchmarks/experiments.

17. Performance

Fit times with pre-computed embeddings (one machine, sequential runs):

The one-time UMAP fit dominates on short texts, tokenization on long ones. TopicInterpreter adds about 2-3 s per topic of API time (parallelised over 8 workers).

18. Seeded topics (codebook mode)

fit(documents, seeds={...}) starts from the topics you expect. A seed is either a one-sentence description ("Sport news: football, rugby, tennis ...") or a list of words (["football", "rugby"]).

1. Anchors. The seed texts are embedded with the model's embedding model. Each document gets the score cos(document, seed), for word-list seeds plus seed_keyword_weight (0.5) × the share of the seed words present in the document. Every document competes for its best seed; the highest-scoring seed_anchors documents of each seed (default 1% of the corpus, clipped to 5-30) become its anchors. A seed that is no document's best match is ignored with a warning, which makes codebook categories that are absent from the data visible.
2. Fixed memberships. In every Leiden run (consensus, refinement, resolution search) the anchors start in one community per seed and are passed to Leiden as is_membership_fixed. All other documents move freely: they join a seeded community or form new ones.
3. Naming. A topic whose anchors mostly come from one seed gets TopicInfo.seed and, until an LLM relabels it, label = seed name. seed_topics_ and emergent_topics_ split the result; get_topic_info() has a Seed column; seed_anchors_ lists the anchor documents. Seeds survive save()/load().

With pre-computed document embeddings, pass seed_embeddings (one vector per seed, same model) or let TriTopic encode the seeds with embedding_model; a dimension mismatch raises an error.

Results: full codebook on the dev splits NMI 0.625 → 0.661, ARI 0.515 → 0.624; held-out ARI 0.399 → 0.498; 81% of the anchors belong to the seed's true class; with half of the classes seeded ARI +17%. BBC demo: NMI 0.742 → 0.801, ARI 0.631 → 0.800 with five one-line seeds. Limitation: with a partial codebook a seeded topic can absorb an unseeded neighbouring theme (BBC with three seeds: business and politics ended up in one emergent topic). Seed every theme you know, or run TopicInterpreter.refine() afterwards.

19. Research toolkit

tritopic.research turns a fitted model into the numbers and material a paper needs. Nothing here calls an LLM unless marked.

Structure analyses (new in 2.6)

20. Research charts

tritopic.visualization.charts draws every analysis above as a Plotly figure. Topics are coloured by size rank with a fixed eight-colour palette (further topics grey), so colours stay the same across charts.

2.6.0 (October 2026)

New

• tritopic.visualization.charts: sixteen research charts (Plotly) — topic table, Tri-View triangle, zoom ladder, topic onions, keyword barcode, constellation, fuzzy borders, group tilt, trust quadrant, composition, birth timeline, codebook coverage, quote wall, LLM verdict board, coder confusion grid, saturation with topic dots.
• Structure analyses in tritopic.research: view_composition(), resolution_ladder(), topic_cores(), keyword_coverage(), coassignment(), topic_discovery(), codebook_coverage().
• Website docs/index.html: how it works, user guide, all examples with code, live BBC results, the chart gallery with code for every chart, technical documentation, benchmarks.
• Example 13_research_charts.py; docs/algorithm_visual_brief.md (brief for an algorithm illustration).

Fixed

• BBC demo: topic labels in the prevalence chart were cut off; the chart now wraps long labels.

2.5.1 (October 2026)

Changed

• topic_evolution() tested on real dated news and redesigned. On HuffPost news 2012-2022 (16,400 articles) the 2.5.0 version, which clustered each period's part of the global graph, kept only about 5 of 28 edges per document and produced mostly noise (70% of period topics "born"). Each period now gets its own kNN graph in the model's layout, clustered at 4× the model's resolution (resolution= to override), and link_threshold defaults to 0.75. Result: the COVID pandemic (2020, continuing as vaccines in 2021) and the war in Ukraine (2022) are detected as births at every tested sample size, together with events such as the Flint water crisis, the Hong Kong protests and Roe v. Wade.
• TopicEvolution.nodes has a docs column (document indices); events carry keywords and size; mismatched timestamp counts raise an error.
• Example 12_topic_evolution_news.py on real dated news.

2.5.0 (October 2026)

Codebook mode and a research toolkit: start from the topics you expect, and get the reliability, saturation, group-comparison and reporting outputs a paper needs.

New

• Seeded topics (codebook mode): fit(documents, seeds={"name": "description" | [words]}). The documents that match a seed best are pinned to one Leiden community per seed (is_membership_fixed); all other documents join a seeded topic or form emergent ones. TopicInfo.seed, seed_topics_, emergent_topics_, seed_anchors_, a Seed column in get_topic_info(); seed_embeddings for pre-computed embeddings; config seed_anchors, seed_keyword_weight. Full codebook on the dev splits: NMI 0.625 → 0.661, ARI 0.515 → 0.624; held-out ARI 0.399 → 0.498.
• tritopic.research:
• topic_reliability(): bootstrap refits (default) or consensus runs; reliability and core share per topic, document_stability_ per document. Reliable topics (≥ 0.7) were 77% pure vs. 54% below 0.5.
• saturation_curve() / plot_saturation(): exact topic accumulation curve (hypergeometric), saturation points for 95% and 100% of the topics, optional refit method.
• bridge_documents(), topic_connections(): documents and links between topics.
• topic_prevalence() with Wilson or bootstrap intervals; compare_groups() with χ²/Fisher tests, Cramér's V, odds ratios and Benjamini-Hochberg correction; distinctive_keywords() (log-odds with informative prior).
• topic_evolution() / plot_evolution(): per-period topics linked into births, splits, merges, deaths.
• topic_quotes(): citable sentences per topic.
• methods_report(): methods paragraph and parameter table from the fitted model.
• TopicInterpreter.codebook(): codebook entries for qualitative content analysis (definition, inclusion/exclusion criteria, coding notes) with real anchor quotes.
• intercoder_reliability() (Decisions API): an LLM as independent second coder; Cohen's kappa, per-topic precision/recall/F1, confusion matrix. BBC demo: kappa 0.94.
• Examples 09_seeded_topics.py, 10_research_toolkit.py, 11_codebook_and_second_coder.py; user guide docs/user_guide.md; BBC demo extended with all new outputs.

Changed

• transform() outlier threshold is calibrated on the training data by default (outlier_threshold=None → 1st percentile of the training similarities, model.outlier_threshold_). The fixed 0.35 rejected up to 40% of in-domain documents; the calibrated rule about 1%.

Fixed

• fit(verbose=True) with automatic resolution crashed when printing the scan summary.

2.4.0 (October 2026)

Head-to-head with BERTopic on identical embeddings: NMI 0.582 vs. 0.439, ARI 0.440 vs. 0.285, keyword NPMI 0.262 vs. 0.117, 0% vs. 18% outliers (4 datasets × 5 topic counts × 3 seeds). See the README benchmarks.

New

• TopicInterpreter (tritopic.labeling): an LLM reads each topic (keywords, proportionally sampled example documents, neighbouring topics) and returns label, description, aspects and a verdict (coherent / mixed / unclear) with sub-themes. refine() splits mixed topics when the graph split agrees with the LLM's sub-themes; summarize() writes an overview. OpenAI Responses API with a strict JSON schema, default model gpt-6-luna.
• OpenAI Decisions API integration (tritopic.integrations.decisions, optional): LLM word-intrusion test and topic ratings, assign_documents(), reduce_outliers(strategy="decisions"), merge suggestions.
• Automatic topic count: n_topics="auto" picks the Leiden resolution by keyword coherence, skipping partitions dominated by one topic (auto_resolution, resolution_range, auto_resolution_steps, auto_resolution_tolerance, auto_resolution_max_share; results in resolution_, resolution_search_).
• Coverage-weighted c-TF-IDF: IDF · √(tf_share · coverage), +45% NPMI on the dev splits.
• Benchmark harness benchmarks/compare_bertopic.py, llm_eval.py, summarize.py, results_to_markdown.py; decision log in benchmarks/experiments/.
• BBC demo page in examples/bbc_demo/.

Performance (fit 4.5x faster than 2.3 at equal NMI)

• Iterative refinement works on the reduced embeddings instead of calling UMAP.transform() each iteration.
• Consensus clustering on graph edges (O(edges)) instead of an all-pairs co-occurrence matrix.
• The corpus is tokenized once; the lexical TF-IDF view is derived from the keyword counts (arXiv, 55k characters per document: 44 s → 22 s).
• Batched keyword extraction; build_hierarchy() and divide() no longer re-tokenize per topic; vectorized coherence and topic_overlap_matrix(); lexical kNN graph cached.

Fixes

• kNN graph on UMAP output used cosine distance; now Euclidean with a self-tuning kernel (reduced_metric). FAISS L2 distances are square-rooted.
• BM25 keywords scored the first n corpus documents instead of the topic's documents.
• Metadata connected every pair of documents sharing a category (millions of edges); it now reweights existing edges. String columns no longer fail with pandas 3.
• reduce_topics() applied a size penalty that merged small topics into large unrelated ones (2k → k: NMI 0.542 → 0.591); available as size_penalty=0.3.
• n_topics=k and divide() counted clusters below min_cluster_size and searched a fixed range; the search now widens until k is reachable and searches both directions.
• NPMI coherence used only each topic's own documents as reference and skipped bigram keywords; it now uses the whole corpus. Values are not comparable with earlier versions.
• Tokens must start with two letters (no 000, __, years as keywords).
• reduce_outliers(), reduce_topics() and representative documents use the same unrefined embedding space as the topic centroids.

Changed defaults

• resolution 1.0 → 0.3 (used for n_topics=k and auto_resolution=False).
• n_topics="auto" uses the coherence-based resolution search.

2.3.0 (February 2026)

Cross-lingual stop words (language=), automatic BAAI/bge-m3 for language="multilingual", hierarchical topics (build_hierarchy(), divide(), visualize_hierarchy_tree()), per-document topics (get_document_topics(), topic_overlap_matrix(), visualize_overlap()), graph-based soft assignment, turftopic adapter.

2.2.x

Dimensionality reduction before graph building, soft topic probabilities, reduce_outliers(), reduce_topics(), merge_topics(), save/load of the full model state.

Citation

@software{tritopic2026,
  author = {Egger, Roman},
  title  = {TriTopic: Tri-Modal Graph Topic Modeling},
  year   = {2026},
  url    = {https://github.com/SmartVisions-AI/tritopic}
}

License

MIT, see LICENSE. Issues and pull requests are welcome. Maintained by SmartVisions.

Head-to-head with BERTopic 0.17.4 on the datasets of the original TriTopic benchmark: 20 Newsgroups (2,000 docs), BBC News (1,225), AG News (2,000), arXiv (2,000); five topic counts per dataset plus automatic mode; seeds 42, 123, 456.

• Same embeddings for every model: all-MiniLM-L6-v2, pre-computed once per dataset.
• Two BERTopic variants: defaults, and "tuned" with BERTopic's keyword best practice (CountVectorizer(stop_words="english", ngram_range=(1, 2), min_df=2), ClassTfidfTransformer(reduce_frequent_words=True)); UMAP seed fixed.
• No tuning on the evaluation data: every 2.4 design decision was made on separate development splits (20NG test, BBC test, AG News train sample, arXiv validation).
• Metrics: NMI/ARI on all documents (BERTopic outliers form one class); strict NPMI over the whole corpus; fit time with pre-computed embeddings on one machine.

Fixed topic count

Automatic topic count

NMI per dataset (fixed k)

Where TriTopic does not lead: on BBC News, BERTopic has the higher ARI for k = 5-20 (NMI is close). On short texts BERTopic is 1-2 s per run faster; on long documents (arXiv) TriTopic is about twice as fast (23 s vs. 50 s). BERTopic collapsed to 4 topics on 20 Newsgroups for two of three seeds, which is part of its lower average there.

Earlier releases quoted mean NMI 0.575 vs. 0.513 for BERTopic. That benchmark gave BERTopic stronger embeddings (all-mpnet-base-v2) than TriTopic and used a lenient coherence measure; the numbers above supersede it.

Reproduce:

pip install "tritopic[benchmark]"
python benchmarks/compare_bertopic.py --pkg . --models tritopic,bertopic,bertopic_tuned --out results.csv
python benchmarks/summarize.py results.csv
python benchmarks/llm_eval.py --pkg . --out llm.csv     # LLM judgements; needs OPENAI_API_KEY, --dry-run estimates cost

The decisions behind 2.4 (keyword scoring, auto resolution, ablations, LLM experiments) are documented in benchmarks/experiments/README.md. Release notes: CHANGELOG.md.