Klasifikasi Multilabel Artikel Ilmiah Bidang Komputer Menggunakan SciBERT dan TextCNN

Regina Nathania Agussalim; Agustinus Jacobus; Sherwin R. U. A. Sompie

doi:10.30606/rjti.v5i2.4705

Authors

Regina Nathania Agussalim Universitas Sam Ratulangi
Agustinus Jacobus Universitas Sam Ratulangi
Sherwin R. U. A. Sompie Universitas Sam Ratulangi

DOI:

https://doi.org/10.30606/rjti.v5i2.4705

Keywords:

Artikel Ilmiah, Klasifikasi Multilabel, SciBERT, TextCNN, Threshold Tuning

Abstract

The increasing number of scientific articles makes the process of searching, filtering, and organizing relevant literature more challenging. This problem becomes more complex because one scientific article can be associated with more than one research topic. This study aims to implement and evaluate a SciBERT-TextCNN model for multilabel classification of scientific articles in the Computer Science field based on title and abstract metadata. The dataset used was arXiv article metadata obtained from Kaggle and limited to Computer Science categories. The research stages included data preprocessing, Multi-Label Random Under-Sampling, tokenization using SciBERT tokenizer, model training, threshold tuning, and evaluation. The model was tested using five scenarios: SciBERT frozen and fine-tuning the last 1, 2, 3, and 4 layers. The best performance was obtained by fine-tuning the last 2 SciBERT layers, achieving precision of 0.708, recall of 0.726, F1-score of 0.715, macro F1-score of 0.649, and Hamming Loss of 0.02333. These results show that SciBERT-TextCNN can be used to support automatic multilabel classification of scientific articles.

Downloads

Download data is not yet available.

References

National Science Board, “Publications Output: U.S. Trends and International Comparisons,” 2023.

D. Bawden and L. Robinson, “Information Overload: An Introduction,” in Oxford Research Encyclopedia of Politics, Oxford University Press, 2020. doi: 10.1093/acrefore/9780190228637.013.1360.

N. A. Sajid et al., “Single vs. Multi-Label: The Issues, Challenges and Insights of Contemporary Classification Schemes,” Applied Sciences, vol. 13, no. 11, p. 6804, Jun. 2023, doi: 10.3390/app13116804.

Autumn Toney and James Dunham, “Multi-label Classification of Scientific Research Documents Across Domains and Languages,” in Proceedings of the Third Workshop on Scholarly Document Processing, Association for Computational Linguistics, 2022, pp. 105–114.

A. N. Tarekegn, M. Giacobini, and K. Michalak, “A review of methods for imbalanced multi-label classification,” Pattern Recognit., vol. 118, p. 107965, Oct. 2021, doi: 10.1016/j.patcog.2021.107965.

S. Li and J. Ou, “Multi-Label Classification of Research Papers Using Multi-Label K-Nearest Neighbour Algorithm,” J. Phys. Conf. Ser., vol. 1994, no. 1, p. 012031, Aug. 2021, doi: 10.1088/1742-6596/1994/1/012031.

H. Fallah, P. Bellot, E. Bruno, and E. Murisasco, “Adapting Transformers for Multi-Label Text Classification,” 2022. [Online]. Available: http://ceur-ws.org

Y. Zhang, X. Li, Y. Liu, A. Li, X. Yang, and X. Tang, “A Multilabel Text Classifier of Cancer Literature at the Publication Level: Methods Study of Medical Text Classification,” JMIR Med. Inform., vol. 11, p. e44892, Oct. 2023, doi: 10.2196/44892.

N. Ahmed Sajid et al., “A Novel Metadata Based Multi-Label Document Classification Technique,” Computer Systems Science and Engineering, vol. 46, no. 2, pp. 2195–2214, 2023, doi: 10.32604/csse.2023.033844.

D. Likhareva, H. Sankaran, and S. Thiyagarajan, “Empowering Interdisciplinary Research with BERT-Based Models: An Approach Through SciBERT-CNN with Topic Modeling,” Apr. 2024.

Z. R. K. Rostam and G. Kertész, “Fine-Tuning Large Language Models for Scientific Text Classification: A Comparative Study,” in 2024 IEEE 6th International Symposium on Logistics and Industrial Informatics (LINDI), IEEE, Nov. 2024, pp. 000233--00023. [Online]. Available: http://arxiv.org/abs/2412.00098

S. Ramakrishnan and L. D. Dhinesh Babu, “Improving Multi-Label Emotion Classification on Imbalanced Social Media Data With BERT and Clipped Asymmetric Loss,” IEEE Access, vol. 13, pp. 60589–60601, 2025, doi: 10.1109/ACCESS.2025.3557091.

F. Charte, A. J. Rivera, M. J. del Jesus, and F. Herrera, “Addressing imbalance in multilabel classification: Measures and random resampling algorithms,” Neurocomputing, vol. 163, pp. 3–16, Sep. 2015, doi: 10.1016/j.neucom.2014.08.091.

I. Beltagy, K. Lo, and A. Cohan, “SciBERT: A Pretrained Language Model for Scientific Text,” Sep. 2019, [Online]. Available: http://arxiv.org/abs/1903.10676

Y. Kim, “Convolutional Neural Networks for Sentence Classification,” in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Stroudsburg, PA, USA: Association for Computational Linguistics, 2014, pp. 1746–1751. doi: 10.3115/v1/D14-1181.

T. Ridnik et al., “Asymmetric Loss For Multi-Label Classification,” 2021. [Online]. Available: https://github.com/Alibaba-MIIL/ASL.