🏆 GermEval 2025 Harmful Content Detection Model Zoo
Interactive testing interface for all 6 submitted models
Detecting harmful content in German social media
Author: Samuel Ruairí Bullard (@abullardUR) | University of Regensburg
Choose which model to test
Run classification to see the prediction.
🎯 Task Overview
The GermEval 2025 Shared Task focuses on detecting harmful content in German social media. The provided datasets primarily originate from the public German-language Twitter network of a right‑wing extremist group. Our submission addresses three subtasks:
📢 Subtask 1: Call to Action (C2A)
- Binary detection of so‑called call to actions, i.e. calls for risky actions (e.g. criminal offences, demonstrations with possible escalation potential)
🏛️ Subtask 2: Attacks on Democratic Basic Order (DBO)
- Fine‑grained classification into four forms of statements and (violent) attacks against the free and democratic basic order of the Federal Republic of Germany (Nothing, Criticism, Agitation, Subversive)
⚔️ Subtask 3: Violence Detection (VIO)
- Binary detection of disturbing, violence‑related statements in tweets
📊 Performance Summary
| Task | Class-Weighted | CW + Focal Loss |
|---|---|---|
| C2A | 0.82 | 0.82 |
| DBO | 0.63 | 0.56 |
| VIO | 0.82 | 0.81 |
Macro-F1 scores on competitions hidden test set. Bold indicates best approach per task.
🛠️ Model Architecture
- Base: LSX-UniWue/ModernGBERT_134M
- Task Heads: Binary/Multi-class classification layers
- Optimization: Bayesian hyperparameter search via Weights & Biases Sweeps
📈 Training Strategy
- Data Split: 80% train / 20% validation (stratified)
- Class Imbalance Handling:
- Inverse frequency class weights
- Focal Loss with optimized γ parameter
- Evaluation: Macro-F1 score (primary metric)