Revisiting Supervised Contrastive Learning for Microblog Classification
Research output: Contributions to collected editions/works › Article in conference proceedings › Research › peer-review
Standard
The 2024 Conference on Empirical Methods in Natural Language Processing: Proceedings of the Conference; November 12-16, 2024. ed. / Yaser Al-Onaizan; Mohit Bansal; Yun-Nung Chen. Kerrville: Association for Computational Linguistics, 2024. p. 15644-15653.
Research output: Contributions to collected editions/works › Article in conference proceedings › Research › peer-review
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - CHAP
T1 - Revisiting Supervised Contrastive Learning for Microblog Classification
AU - Huang, Junbo
AU - Usbeck, Ricardo
N1 - Conference code: 29
PY - 2024
Y1 - 2024
N2 - Microblog content (e.g., Tweets) is noisy due to its informal use of language and its lack of contextual information within each post. To tackle these challenges, state-of-the-art microblog classification models rely on pre-training language models (LMs). However, pre-training dedicated LMs is resource-intensive and not suitable for small labs. Supervised contrastive learning (SCL) has shown its effectiveness with small, available resources. In this work, we examine the effectiveness of fine-tuning transformer-based language models, regularized with a SCL loss for English microblog classification. Despite its simplicity, the evaluation on two English microblog classification benchmarks (TweetEval and Tweet Topic Classification) shows an improvement over baseline models. The result shows that, across all subtasks, our proposed method has a performance gain of up to 11.9 percentage points. All our models are open source.
AB - Microblog content (e.g., Tweets) is noisy due to its informal use of language and its lack of contextual information within each post. To tackle these challenges, state-of-the-art microblog classification models rely on pre-training language models (LMs). However, pre-training dedicated LMs is resource-intensive and not suitable for small labs. Supervised contrastive learning (SCL) has shown its effectiveness with small, available resources. In this work, we examine the effectiveness of fine-tuning transformer-based language models, regularized with a SCL loss for English microblog classification. Despite its simplicity, the evaluation on two English microblog classification benchmarks (TweetEval and Tweet Topic Classification) shows an improvement over baseline models. The result shows that, across all subtasks, our proposed method has a performance gain of up to 11.9 percentage points. All our models are open source.
KW - Business informatics
U2 - 10.18653/v1/2024.emnlp-main.876
DO - 10.18653/v1/2024.emnlp-main.876
M3 - Article in conference proceedings
SP - 15644
EP - 15653
BT - The 2024 Conference on Empirical Methods in Natural Language Processing
A2 - Al-Onaizan, Yaser
A2 - Bansal, Mohit
A2 - Chen, Yun-Nung
PB - Association for Computational Linguistics
CY - Kerrville
T2 - Conference on Empirical Methods in Natural Language Processing - EMNLP 2024
Y2 - 12 November 2024 through 16 November 2024
ER -