NONE 20260102091335755 wseas/wseas content-registration-form+ja@crossref.org content-registration-form WSEAS TRANSACTIONS ON INFORMATION SCIENCE AND APPLICATIONS 1790-0832 2224-3402 Soumia Afartass Laboratory of Research in Computer Science and Telecommunications (LRIT), Faculty of Sciences Mohammed V University in Rabat, MOROCCO Fadoua Ataa Allah Computer Science Studies, Information Systems and Communication Center, The Royal Institute of Amazigh Culture in Rabat, MOROCCO Khalid Minaoui Laboratory of Research in Computer Science and Telecommunications (LRIT), Faculty of Sciences Mohammed V University in Rabat, MOROCCO Arabic, an inflectional language with a rich morphology and complex syntactic structures, demands robust approaches for effective normalization and lemmatization. While, most existing NLP models focus on English, Modern Standard Arabic remains understudied, particularly in lemmatization tasks. In this paper, we introduce AraBART, the first Arabic model to feature an end-to-end pre-trained encoder-decoder, leveraging the BART architecture. We used the Arabic-PADT UD Treebank and Farasa corpus. Performance was assessed with accuracy, precision, recall, and F1 score. The results show that AraBART surpasses strong baselines, including transformer-based models such as AraT5, mT5, and BERT, achieving a 4.72% improvement in lemmatization accuracy. More importantly, AraBART has achieved an accuracy of 94.71%, approaching the performance of Farasa (97.32%). Furthermore, incorporating parts of the Farasa corpus into our training process showed a clear improvement in accuracy, revealing AraBART's effectiveness for broad applications in Arabic NLP tasks, including text summarization and machine translation. 01 02 2026 01 02 2026 1 1 https://creativecommons.org/licenses/by/4.0/deed.en_US 10.37394/23209.2026.23.1 https://wseas.com/journals/isa/2026/a025109-001(2026).pdf S. Afartass, F. A. Allah, K. Minaoui, Transformer-based Arabic Language Lemmatization, 3rd International Conference on Embedded Systems and Artificial Intelligence (ESAI), 2024. http://dx.doi.org/10.1109/ESAI62891.2024.10 913868. Z. Ghosn, Les sites Internet gouvernementaux au Moyen-Orient, The Arab Advisors Group, 2003, [Online]. https://www.persee.fr/ (Accessed: November 5, 2024). B. Barraud, Tribune des Droits Humains, Censure de l’internet dans les pays arabes, 2006, [Online]. https://www.academia.edu/94916908/ (Accessed: April 20, 2024). O. Dakkak, N. Ghneim, A. Shalaby, S. Desouki, R. Sonbol, Arabic Language Resources in HIAST, Conference on Arabic Language Resources and Tools, 2009. A. Alghani, Lexical extraction and morphological and semantic categorization of the Arabic language dictionary (Extraction lexicale et catégorisation morphologique et sémantique du dictionnaire de la langue arabe), 2020, [Online]. https://memoirepfe.fstusmba.ac.ma/book/1454 (Accessed: November 3, 2024). I. F. Mohammed, Q. A. Dhayef, Derivation and its Effect on Meaning in English and Arabic: A Contrastive Study, International Journal of Linguistics, 2022. DOI: 10.32996/ijls.2022.2.2.20. D. Tahar, S. Benharzallah, B. Ali, The Impact of Online Indexing in Improving Arabic Information Retrieval Systems, Informatica, Vol. 42, 2018. https://doi.org/10.31449/inf.v42i4.229. M. Rizi, S. Maazaoui, A New Deep LearningBased Approach for Cloud Service Classification (Une Nouvelle approche basée deep learning pour la classification de services cloud), 2021, [Online]. https://thesesalgerie.com/ (Accessed: July 24, 2024). A. El-Sawy, H. El-Bakry, M. Loey, N. Mastorakis, An Intelligent Agent Tutor System for Detecting Arabic Children Handwriting Difficulty Based on Immediate Feedback, WSEAS Transactions on Systems, Vol. 15, 2016, pp. 63-72. A. H. Alshehri, AraQA-BERT: Towards an Arabic Question Answering System using pre-trainedBERT Models, WSEAS Transactions on Information Science and Applications, vol. 21, 2024, pp. 361-373. https://doi.org/10.37394/23209.2024.21.34. M. Rizi, S. Maazaoui, Une Nouvelle approche basée deep learning pour la classification de services cloud, Artificial Intelligence and Deep Learning: The Concept and the Applications, Vol. 6, 2011. https://doi.org/10.34874/IMIST.PRSM/reinno va-v6i23.53457. N. Habash, R. Roth, O. Rambow, R. Eskander, N. Tomeh. Morphological Analysis and Disambiguation for Dialectal Arabic. Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2013, pp. 426–432. M. Diab, Second Generation AMIRA Tools for Arabic Processing: Fast and Robust Tokenization, POS Tagging, and Base Phrase Chunking, Proceedings of the Second International Conference on Arabic Language Resources and Tools, Cairo, Egypt 2009. A. Pasha, M. Al-Badrashiny, M. Diab, A. El Kholy, R. Eskander, N. Habash, M. Pooleery, O. Rambow, R. Roth. MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), 2014, pp 1094–1101. H. Mubarak, Build Fast and Accurate Lemmatization for Arabic, QCRI, Hamad Bin Khalifa University (HBKU), Doha, Qatar, 2018. https://aclanthology.org/L18-1181/. M. Boudchiche, A. Mazroui, A hybrid approach for Arabic lemmatization, Int. J. Speech Technol, Vol. 22, 2018, pp. 568-573. https://doi.org/10.1007/s10772-018-9528-3. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, P. J. Liu, Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer, Proceedings of the 37th International Conference on Machine Learning, 2020, pp. 1-20. E. B. Nagoudi, A. Elmadany, M. AbdulMageed, AraT5: Text-to-Text Transformers for Arabic Language Generation, arXiv, 2021, pp. 1-20. https://doi.org/10.48550/arXiv.2109.12068. L. Xue, N. Constant, A. Roberts, M. Kale, R. Al-Rfou, A. Siddhant, A. Barua, C. Raffel, mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer, NAACL 2021, 2021, pp. 1-20. J. Devlin, M. Chang, K. Lee, K. Toutanova, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, arXiv, Vol. 2019, No. 10, 2019, pp. 1-20. https://doi.org/10.48550/arXiv.1810.04805. Y. Liu, J. Gu, N. Goyal, X. Li, S. Edunov, M. Ghazvininejad, M. Lewis, L. Zettlemoyer, Multilingual denoising pre-training for neural machine translation, Transactions of the Association for Computational Linguistics, Vol. 8, 2020, pp. 726–742. DOI: 10.1162/tacl_a_00343. H. Le, ChatGPT: Exploring OpenAI’s Language Model, 2023, [Online]. https://hoanhle.github.io/blog/2023/chatgpt/ (Accessed October 17, 2024). M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, L. Zettlemoyer. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 7871– 7880. https://doi.org/10.48550/arXiv.1910.13461 A. Özcift, K. Akarsu, F. Yumuk, C. Söylemez, Advancing Natural Language Processing (NLP) Applications of Morphologically Rich Languages with Bidirectional Encoder Representations from Transformers (BERT): An Empirical Case Study for Turkish, Automatic Control and Computer Sciences, 2021. https://doi.org/10.1080/00051144.2021.19221 50. B. Pengili, D. A. Karras, A detailed comparative analysis towards longer-term traffic loads forecasting with Autoregressive Integrated Moving Average (ARIMA/SARIMA) models to improve transportation services, Financial Engineering, Vol. 3, 2025, pp. 147-171. https://doi.org/10.37394/232032.2025.3.13. T. Keldenich, Recall, Precision, F1 Score: Simple Metric Explanation, Machine Learning, 2021, [Online]. https://insidemachinelearning.com (Accessed April 25, 2024). M. Ulcar, M. Robnik-Sikonja, Sequence-tosequence pretraining for a less-resourced Slovenian language, Frontiers in Artificial Intelligence, vol. 3, 2023. doi: 10.3389/frai.2023.932519. T. T. Ünlü, A. L. Mahamat, M. Turan, Leveraging deep learning for drowning and swimming prevention. WSEAS Transactions on Information Science and Applications, Vol. 22, 2025, pp 234-244. https://doi.org/10.37394/23209.2025.22.20.