KYHTQT.

A Hybrid Approach: Transformer and LSTM Combination for Text Summarization in Vietnamese

Năm XB 2024 Tạp chí / Hội thảo Lecture Notes in Networks and Systems Volume 1205 LNNS Đơn vị CNTT DOI / Link https://doi.org/10.1007/978-3-031-80943-9_9 ↗

Tác giả

Tóm tắt

In natural language processing, text summarization is an essential task, particularly for low-resource languages such as Vietnamese. Traditional techniques like LSTM (long short-term memory) networks demonstrated their potential efficiency in text summarizing;...

Tài liệu tham khảo

[1] Yuskovych-Zhukovska, V., et al.: Application of artificial intelligence in education. Problems and opportunities for sustainable development. Brain (Bacau) 13, 339–356 (2022)

[2] Ellahham, S., Ellahham, N., Simsekler, M.C.E.: Application of artificial intelligence in the health care safety context: opportunities and challenges. Am. J. Med. Qual. 35, 341–348 (2020)

[3] Lee, S.M., Lee, D.: Healthcare wearable devices: an analysis of key factors for continuous use intention. Serv. Bus. 14, 503–531 (2020)

[4] Sestino, A., De Mauro, A.: Leveraging artificial intelligence in business: Implications, applications and methods. Technol Anal Strateg Manag. 34, 16–29 (2022)

[5] Pallathadka, H., Ramirez-Asis, E.H., Loli-Poma, T.P., Kaliyaperumal, K., Ventayen, R.J.M., Naved, M.: Applications of artificial intelligence in business management, e-commerce and finance. Mater Today Proc. 80, 2610–2613 (2023)

[6] Zhang, Y., et al.: Application of artificial intelligence in military: From projects view. In: 2020 6th International Conference on Big Data and Information Analytics (BigDIA), pp. 113–116. IEEE (2020)

[7] Van der Aa, H., Carmona Vargas, J., Leopold, H., Mendling, J., Padró, L.: Challenges and opportunities of applying natural language processing in business process management. In: COLING 2018: The 27th International Conference on Computational Linguistics: Proceedings of the Conference: August 20–26, 2018 Santa Fe, New Mexico, USA, pp. 2791–2801. Association for Computational Linguistics (2018)

[8] Bahja, M.: Natural language processing applications in business. E-Business-higher education and intelligence applications (2020)

[9] Faizal, B., Abraham, S.: NLP based automated business report summarization. In: 2022 International Conference on Innovative Trends in Information Technology (ICITIIT), pp. 1–4. IEEE (2022)

[10] Hegdepatil, P., Davuluri, K.: Business intelligence based novel marketing strategy approach using automatic speech recognition and text summarization. In: 2021 2nd International Conference on Computing and Data Science (CDS), pp. 595–602. IEEE (2021)

[11] Vu, T., Nguyen, D.Q., Nguyen, D.Q., Dras, M., Johnson, M.: VnCoreNLP: A Vietnamese natural language processing toolkit. arXiv preprint arXiv:1801.01331 (2018)

[12] Nguyen-Hoang, T.-A., Nguyen, K., Tran, Q.-V.: TSGVi: a graph-based summarization system for Vietnamese documents. J. Ambient. Intell. Humaniz. Comput. 3, 305–313 (2012)

[13] Baskaran, S., Alagarsamy, S., S, S., Shivam, S.: Text Generation using Long Short-Term Memory. In: 2024 Third International Conference on Intelligent Techniques in Control, Optimization and Signal Processing (INCOS), pp. 1–6 (2024). https://doi.org/10.1109/INCOS59338.2024.10527547

[14] Al-Selwi, S.M., Hassan, M.F., Abdulkadir, S.J., Muneer, A.: LSTM inefficiency in long-term dependencies regression problems. J. Adv. Res. Appl. Sci. Eng. Technol. 30, 16–31 (2023)

[15] Day, M.-Y., Chen, C.-Y.: Artificial intelligence for automatic text summarization. In: 2018 IEEE International Conference on Information Reuse and Integration (IRI), pp. 478–484. IEEE (2018)

[16] Tomer, M., Kumar, M.: Improving text summarization using ensembled approach based on fuzzy with LSTM. Arab. J. Sci. Eng. 45, 10743–10754 (2020)

[17] Song, S., Huang, H., Ruan, T.: Abstractive text summarization using LSTM-CNN based deep learning. Multimed Tools Appl. 78, 857–875 (2019)

[18] See, A., Liu, P.J., Manning, C.D.: Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368 (2017)

[19] Vaswani, A., et al.: Attention is all you need. Adv Neural Inf Process Syst. 30 (2017)

[20] Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

[21] Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving language understanding by generative pre-training (2018)

[22] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners. OpenAI blog. 1, 9 (2019)

[23] Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21, 1–67 (2020)

[24] Mullen, A.L., Benoit, K., Keyes, O., Selivanov, D., Arnold, J.: Fast, consistent tokenization of natural language text. J Open Source Softw. 3, 655 (2018)

[25] Nguyen, L.H., Salopek, A., Zhao, L., Jin, F.: A natural language normalization approach to enhance social media text reasoning. In: 2017 IEEE International Conference on Big Data (Big Data), pp. 2019–2026. IEEE (2017)

[26] Ding, H., et al.: Fewer truncations improve language modeling. arXiv preprint arXiv:2404.10830 (2024)

[27] Dwarampudi, M., Reddy, N.V.: Effects of padding on LSTMs and CNNs. arXiv preprint arXiv:1903.07288 (2019)

Ghi chú

ICTA