🎮 Toxic behavior detection in gaming chats

This data application shows how LLM can automatically detect and classify toxic behavior in gaming chat messages, specifically focusing on DOTA 2 community interactions. The system analyzes chat patterns, identifies harassment and hate speech.

Analyzed 638 DOTA 2 chat messages (dataset from Huggingface)classified into three toxicity levels (0: Mid-toxic, 1: Non-toxic, 2: Toxic). Implemented both traditional NLP approaches and modern LLM-based detection using Google's Gemma model. Processing pipeline includes text preprocessing with spaCy, pattern analysis, and real-time toxicity classification.

Distribution of toxicity labels in DOTA 2 chat

Non-toxic messages dominate the dataset with 353 instances, while toxic content comprises 167 messages, showing that most player interactions remain positive.

Message length by toxicity level

Toxic messages tend to be shorter and more direct (median ~25 characters), while non-toxic messages show greater length variation, suggesting toxic players use brief, aggressive language.

Toxic message pattern clustering

The scatter plot matrix shows distinct clustering patterns where toxic messages correlate strongly with imperative commands, second-person pronouns, and profanity usage.

Message flow distribution

Message length follows a right-skewed distribution with most messages containing 2-8 words, indicating players prefer concise communication during gameplay.

Word cloud analysis

While the word cloud highlights frequent terms, individual word frequency alone is insufficient for toxicity detection - context and sentence structure analysis is required for accurate classification.

Why individual words don't determine toxicity:

Here are three examples showing how the same words can appear across different toxicity levels:

Non-toxic example:

Mid-toxic example:

Toxic example:

Word sniper

Word: fuck

Word: noob

Sentence: you low rank you complain a newplayer sniper for

Sentence: like they re fucking trying to lose

Sentence: feeded by noobs

Analysis: The word sniper here refers to a game character/role. In this context, it's used for strategic discussion or game mechanics, not as an attack on other players. The sentence structure is informative rather than aggressive.

Analysis: While fuck is profanity, in this mid-toxic context it's often used as frustration expression rather than direct harassment. The sentence may express game-related anger without personally attacking specific players, making it less severe than fully toxic messages.

Analysis: In toxic contexts, noob becomes a weapon for harassment. The sentence structure typically combines this term with other insults, demands to quit, or personal attacks. The aggressive tone and intent to hurt or exclude the player makes it toxic, not the word itself.

Toxicity detector with Gemma 3

Model details:

Parameters: Temperature: 0.7, top-p: 0.95, top-k: 40, max output tokens: 1024

Model: google-deepmind / gemma-3-27b-it (instruction-tuned variant)

Platform: Replicate

Configuration: Optimized for gaming context understanding with custom prompting for DOTA 2 terminology and slang recognition

How to use the toxicity detector:

Enter a DOTA 2 chat message in the input field

Click "Show detailed analysis" to process the message

The system will return toxicity analysis

Enter DOTA 2 chat message

The application successfully demonstrates how LLMs can provide more nuanced toxicity detection compared to traditional keyword-based approaches, understanding context, sarcasm, and gaming-specific language patterns.

.css-15w88e5{color:var(--chakra-colors-fg-neutral-primary);font-weight:inherit;letter-spacing:-0.09px;}🎮 Toxic behavior detection in gaming chats