TY - CHAP

T1 - Revisiting Supervised Contrastive Learning for Microblog Classification

AU - Huang, Junbo

AU - Usbeck, Ricardo

N1 - Conference code: 29

PY - 2024

Y1 - 2024

N2 - Microblog content (e.g., Tweets) is noisy due to its informal use of language and its lack of contextual information within each post. To tackle these challenges, state-of-the-art microblog classification models rely on pre-training language models (LMs). However, pre-training dedicated LMs is resource-intensive and not suitable for small labs. Supervised contrastive learning (SCL) has shown its effectiveness with small, available resources. In this work, we examine the effectiveness of fine-tuning transformer-based language models, regularized with a SCL loss for English microblog classification. Despite its simplicity, the evaluation on two English microblog classification benchmarks (TweetEval and Tweet Topic Classification) shows an improvement over baseline models. The result shows that, across all subtasks, our proposed method has a performance gain of up to 11.9 percentage points. All our models are open source.

AB - Microblog content (e.g., Tweets) is noisy due to its informal use of language and its lack of contextual information within each post. To tackle these challenges, state-of-the-art microblog classification models rely on pre-training language models (LMs). However, pre-training dedicated LMs is resource-intensive and not suitable for small labs. Supervised contrastive learning (SCL) has shown its effectiveness with small, available resources. In this work, we examine the effectiveness of fine-tuning transformer-based language models, regularized with a SCL loss for English microblog classification. Despite its simplicity, the evaluation on two English microblog classification benchmarks (TweetEval and Tweet Topic Classification) shows an improvement over baseline models. The result shows that, across all subtasks, our proposed method has a performance gain of up to 11.9 percentage points. All our models are open source.

KW - Business informatics

U2 - 10.18653/v1/2024.emnlp-main.876

DO - 10.18653/v1/2024.emnlp-main.876

M3 - Article in conference proceedings

SP - 15644

EP - 15653

BT - The 2024 Conference on Empirical Methods in Natural Language Processing

A2 - Al-Onaizan, Yaser

A2 - Bansal, Mohit

A2 - Chen, Yun-Nung

PB - Association for Computational Linguistics

CY - Kerrville

T2 - Conference on Empirical Methods in Natural Language Processing - EMNLP 2024

Y2 - 12 November 2024 through 16 November 2024

ER -