Scopus

Next process-activity prediction using Switch-Transformer: approach, visualization, and performance evaluation

Năm XB 2025 Tạp chí / Hội thảo Knowledge and Information Systems Volume 67 (12) DOI / Link https://doi.org/10.1007/s10115-025-02587-z ↗

Tác giả

Tóm tắt

Deep learning architectures extract insights from extensive datasets, thereby enhancing human decision-making with more rational, agile, and precise predictions. Recent research in business process management has increasingly leveraged deep learning to support predictive process monitoring-encompassing tasks such as next-activity prediction, remaining time estimation, and workload forecasting. In this paper, we introduce a novel framework designed to forecast subsequent activities in the temporal sequences of individual process instances. Our method integrates a mixture-of-experts mechanism with the Switch-Transformer neural network architecture, and the model is trained from scratch using historical process enactment logs to estimate the probabilities of upcoming activities. Furthermore, the predicted activities are displayed in a temporal graph that preserves their sequential order for each running process instance. This visualization not only clarifies the predictive process monitoring paradigm and its underlying decision-making workflow but also streamlines risk assessment and analysis in business process monitoring operations. To validate the operational efficacy of our approach, we conducted extensive experiments on both synthetic and real-world process event log datasets commonly employed in previous studies. The results demonstrate that our method not only outperforms existing techniques in predictive accuracy but also offers superior usability and practical applicability in real-world settings.

Tài liệu tham khảo

[1] Aalst W (2018) Spreadsheets for business process management: using process mining to deal with “events’’ rather than “numbers’’. Bus Process Manag J 24(1):105–127. https://doi.org/10.1108/BPMJ-10-2016-0190

[2] Ahn H, Kim KP (2020) A formal approach for discovering work transference networks from workflow logs. Inf Sci 515:1–25. https://doi.org/10.1016/j.ins.2019.11.036

[3] Ellis CA, Rembert AJ, Kim K-H, Wainer J (2006) Beyond workflow mining. In: International conference on business process management, Springer, pp 49–64

[4] Aalst WMP (2009) Process-aware information systems: lessons to be learned from process mining, Springer, Berlin, Heidelberg, pp 1–26. https://doi.org/10.1007/978-3-642-00899-3_1

[5] Di Francescomarino C, Ghidini C (2022) Predictive process monitoring, Springer, vol 448, pp 320–346. https://doi.org/10.1007/978-3-031-08848-3_10

[6] Neu DA, Lahann J, Fettke P (2022) A systematic literature review on state-of-the-art deep learning methods for process prediction. Artif Intell Rev. https://doi.org/10.1007/s10462-021-09960-8

[7] Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

[8] Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324. https://doi.org/10.1109/5.726791

[9] Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2009) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80. https://doi.org/10.1109/TNN.2008.2005605

[10] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 2017(Decem(Nips)):5999–6009

[11] Devlin J, Chang M, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein J, Doran C, Solorio T (eds) Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: Human language technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2–7, 2019, Volume 1 (Long and Short Papers), pp 4171–4186. Association for Computational Linguistics. https://doi.org/10.18653/v1/n19-1423

[12] Fedus W, Zoph B, Shazeer N (2022) Switch transformers: scaling to trillion parameter models with simple and efficient sparsity. J Mach Learn Res 23:1–40 arXiv:2101.03961

[13] IEEE (2016) IEEE standard for extensible event stream (XES) for achieving interoperability in event logs and event streams. IEEE, pp 1849–2016

[14] Sui Q, Wang Y, Liu C, Wang K, Sun B (2025) Attribution-aided nonlinear granger causality discovery method and its industrial application. IEEE Trans Industr Inf. https://doi.org/10.1109/TII.2025.3558343

[15] Sui Q, Wang Y, Liu C, Han M, Yang C (2025) Koopman-constrained hierarchical deep state space model for industrial quality prediction via cloud-edge collaborative framework. IEEE Trans Syst Man Cybernet Syst 55(2):1137–1150. https://doi.org/10.1109/TSMC.2024.3495020

[16] Yang Y, Chen L, Pan L, J H (2022) Image captioning with synergy-gated attention and recurrent fusion LSTM. KSII Trans Internet Inf Syst 16(10):3390–3405. https://doi.org/10.3837/tiis.2022.10.010

[17] Evermann J, Rehse J-R, Fettke P (2017) a deep learning approach for predicting process behaviour at runtime BT: business process management workshops. Business Process Management Workshops, pp 327–338

[18] Tax N, Verenich I, La Rosa M, Dumas M (2017) Predictive business process monitoring with LSTM neural networks. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) 10253 LNCS, pp 477–492. https://doi.org/10.1007/978-3-319-59536-8_30. arXiv:1612.02130

[19] Navarin N, Vincenzi B, Polato M, Sperduti A (2017) LSTM networks for data-aware remaining time prediction of business process instances. In: 2017 IEEE symposium series on computational intelligence (SSCI), pp 1–7

[20] Lin L, Wen L, Wang J (2019) MM-Pred: a deep predictive model for multi-attribute event sequence. In: SIAM international conference on data mining, SDM 2019, pp 118–126. https://doi.org/10.1137/1.9781611975673.14

[21] Camargo M, Dumas M, González-Rojas O (2019) Learning accurate LSTM models of business processes. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) 11675 LNCS, pp 286–302. https://doi.org/10.1007/978-3-030-26619-6_19

[22] Pham DL, Ahn H, Kim KS, Kim KP (2021) Process-aware enterprise social network prediction and experiment using LSTM neural network models. IEEE Access 9:57922–57940. https://doi.org/10.1109/ACCESS.2021.3071789

[23] Mauro N, Appice A, Basile TMA (2019) Activity prediction of business process instances with inception CNN models. In: Alviano M, Greco G, Scarcello F (eds) AI*IA 2019: advances in artificial intelligence. Springer, Cham, pp 348–361

[24] Pasquadibisceglie V, Appice A, Castellano G, Malerba D (2019) Using convolutional neural networks for predictive process analytics. In: 2019 international conference on process mining (ICPM), pp 129–136 . https://doi.org/10.1109/ICPM.2019.00028

[25] Chiorrini A, Diamantini C, Mircoli A, Potena D (2022) Exploiting instance graphs and graph neural networks for next activity prediction, Springer, vol 433 LNBIP, pp 115–126. https://doi.org/10.1007/978-3-030-98581-3_9

[26] Liang T, Pan W, Bao H, Fan X, H L (2023) Bird’s eye view semantic segmentation based on improved transformer for automatic annotation. KSII Trans Internet Inf Syst 17(8):1996–2015. https://doi.org/10.3837/tiis.2023.08.002

[27] Ali W, Zuo W, Ali R, Rahman G, Zuo X, I U (2022) Towards improving causality mining using bert with multi-level feature networks. KSII Trans Internet Inf Syst 16(10):3230–3255. https://doi.org/10.3837/tiis.2022.10.002

[28] Liu B, Lv Z, Zhu N, Chang D, M L (2022) Hot keyword extraction of sci-tech periodicals based on the improved bert model. KSII Trans Internet Inf Syst 16(6):1800–1817. https://doi.org/10.3837/tiis.2022.06.002

[29] Chen H, Fang X, Fang H (2022) Multi-task prediction method of business process based on bert and transfer learning. Knowl-Based Syst 254:109603. https://doi.org/10.1016/j.knosys.2022.109603

[30] Bukhsh ZA, Saeed A, Dijkman RM (2021) ProcessTransformer: predictive business process monitoring with transformer network. Adv Neural Inf Process Syst. arXiv:2104.00721

[31] Beltagy I, Peters ME, Cohan A (2020) Longformer: the long-document transformer. https://arxiv.org/abs/2004.05150

[32] Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2023) Exploring the limits of transfer learning with a unified text-to-text transformer . https://arxiv.org/abs/1910.10683

[33] Oved A, Shlomov S, Zeltyn S, Mashkif N, Yaeli A (2024) SNAP: semantic stories for next activity prediction. https://arxiv.org/abs/2401.15621

[34] Sun X, Yang S, Yuke Ying DY (2023) Next activity prediction of ongoing business processes based on deep learning. Expert Syst. https://doi.org/10.1111/exsy.13421

[35] Nguyen T-H, Kim K-S, Pham D-L, Kim KP (2024) A web-based system for business process discovery: leveraging the SICN-oriented process mining algorithm with django, cytoscape, and graphviz. KSII Trans Internet Inf Syst 18(8):2316–2332. https://doi.org/10.3837/tiis.2024.08.014

[36] Kim K (2012) A model-driven workflow fragmentation framework for collaborative workflow architectures and systems. J Netw Comput Appl 35(1):97–110. https://doi.org/10.1016/j.jnca.2011.03.029

[37] Kim K, Ellis CA (2009) Section II/Chapter VII. An ICN-based workflow model and its advances, handbook of research on BP modeling. IGI Global, ISR, Hershey, Pennsylvania, USA

[38] Kim KP (2024) A structural anomaly-trace detection approach and its implications in process mining and reengineering. IEEE Access 12:171099–171111. https://doi.org/10.1109/ACCESS.2024.3501392

[39] Shazeer N, Mirhoseini A, Maziarz K, Davis A, Le Q, Hinton G, Dean J (2017) Outrageously large neural networks: the sparsely-gated mixture-of-experts layer

[40] Dahouda MK, Joe I (2021) A deep-learned embedding technique for categorical features encoding. IEEE Access 9:114381–114391. https://doi.org/10.1109/ACCESS.2021.3104357

[41] 4TU, C.f.R.D.: BPI Challenges. BPM (2012, 2013, 2014, 2015, 2016, 2017, 2018)

[42] Rama-Maneiro E, Vidal JC, Lama M (2021) Deep learning for predictive business process monitoring: review and benchmark. IEEE Trans Serv Comput 16(1):739–756. https://doi.org/10.1109/TSC.2021.3139807. arXiv:2009.13251

[43] Polato M (2017) Dataset belonging to the help desk log of an Italian company. Univ Padova. https://doi.org/10.4121/uuid:0c60edf1-6f83-4e75-9367-4c63b3e9d5bb

[44] Dongen B (2012) BPI challenge 2012. Eindh Univ Technol. https://doi.org/10.4121/uuid:3926db30-f712-4394-aebc-75976070e91f

[45] Steeman W (2013) BPI challenge 2013, incidents. Ghent Univ. https://doi.org/10.4121/uuid:500573e6-accc-4b0c-9576-aa5468b10cee

[46] Steeman W (2013) BPI challenge 2013, closed problems. Ghent Univ. https://doi.org/10.4121/uuid:c2c3b154-ab26-4b31-a0e8-8f2350ddac11

[47] Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. CoRR abs/1412.6980

[48] Zhang Y, Yang Q (2021) A survey on multi-task learning

← Quay lại danh sách bài báo