Scopus

Cocktail Party Effect with Fine-Tuning Multi-Head Self-Attention

Năm XB 2026 Tạp chí / Hội thảo Advances in Information and Communication Technology: Proceedings of the 4th International Conference, ICTA 2025, Volume 2 Volume 2, pp. 526-534 Đơn vị ICTU DOI / Link https://link.springer.com/chapter/10.1007/978-3-032-18159-6_55 ↗

Tác giả

Trung-Nghia Phung ; Duyen Nguyen Thi ; Duc-Quang Vu

Tóm tắt

In this research, we introduce a fine-tuned multi-head self-attention framework designed to address the cocktail party effect, also known as speech separation. The integration of intra- and inter-segment, along with multi-head self-attention, represents a notable advancement in effectively separating lengthy time-domain speech signals, serving as a baseline architecture. We implement transfer learning methods for each layer of the model with scheduling mechanisms that adjust the learning rate across different datasets. This approach allows the model to be refined and updated using prior knowledge, effectively boosting performance while reducing training time and cost. It is especially useful for adapting existing models to similar tasks or enhancing their effectiveness. The fine-tuned model outperforms non-fine-tuned ones by reusing learned features from earlier training stages. Experimental results indicate that the proposed method achieves better performance than the baseline framework and surpasses existing approaches.

Tài liệu tham khảo

[1] Chen, J.J., Mao, Q., Liu, D.: Dual-path transformer network: direct context-aware modeling for end-to-end monaural speech separation. In: INTERSPEECH (2020)

[2] Garofalo, J., et al.: Continuous speech recognition (CSR-I) Wall Street Journal (WSJ0) news, complete. Linguistic data consortium, Philadelphia (1993)

[3] Hershey, J.R., Chen, Z., Le Roux, J., Watanabe, S.: Deep clustering: discriminative embeddings for segmentation and separation. In: ICASSP, pp. 31–35. IEEE (2016)

[4] Huang, K.P., Wu, Y.K., Lee, H.y.: Improving the transferability of speech separation by meta-learning. CoRR abs/2203.05882 (2022)

[5] Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (Poster) (2015)

[6] Kolbæk, M., et al.: Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks. TASLP 25(10) (2017)

[7] Lam, M.W., et al.: Sandglasset: a light multi-granularity self-attentive network for time-domain speech separation. In: ICASSP, pp. 5759–5763. IEEE (2021)

[8] Le Roux, J., Wisdom, S., Erdogan, H., Hershey, J.R.: SDR–Half-baked or well done? In: ICASSP, pp. 626–630. IEEE (2019)

[9] Luo, Y., Chen, Z., et al.: Dual-path RNN: efficient long sequence modeling for time- domain single-channel speech separation. In: ICASSP, pp. 46–50. IEEE (2020)

[10] Luo, Y., Mesgarani, N.: Conv-TasNet: surpassing ideal time-frequency magnitude masking for speech separation. TASLP 27(8), 1256–1266 (2019)

[11] Mai, TH., Ye, N.-X., Kuan, Y.-W., Lu, P.-Y., Lin, H.-T.: The unexplored potential of vision-language models for generating large-scale complementary-label learning data. In: Wu, X., et al. (eds.) PAKDD 2025. LNCS, vol. 15874, pp. 90–102. Springer, Singapore (2025). https://doi.org/10.1007/978-981-96-8186-0_8

[12] Minh Tan, H., Quang, V.D., et al.: Multi-mask learning and vector training for monaural speech separation. Vietnam J. Comput. Sci. (2025)

[13] Subakan, C., Ravanelli, M., Cornell, S., Bronzi, M., Zhong, J.: Attention is all you need in speech separation. In: ICASSP, pp. 21–25. IEEE (2021)

[14] Subakan, C., Ravanelli, M., Cornell, S., Grondin, F., Bronzi, M.: Exploring self-attention mechanisms for speech separation. TASLP 31, 2169–2180 (2023)

[15] Tan, H.M., Liang, K.W., Wang, J.C.: Discriminative vector learning with application to single channel speech separation. In: ICASSP. IEEE (2023)

[16] Tan, H.M., Vu, DQ., Thi, D.N., Thu, T.P.T.: Voice separation using multi learning on squash-norm embedding matrix and mask. In: Nghia, P.T., Thai, V.D., Thuy, N.T., Son, L.H., Huynh, V.-N. (eds.) ICTA 2023. LNNS, vol. 848, pp. 327–333. Springer, Cham (2024). https://doi.org/10.1007/978-3-031-50818-9_36

[17] Tan, H.M., Vu, D.Q., Wang, J.C.: Selinet: a lightweight model for single channel speech separation. In: ICASSP, pp. 1–5. IEEE (2023)

[18] Tan, H.M., Wang, J.C.: Single channel speech separation using enhanced learning on embedding features. In: GCCE, pp. 430–431. IEEE (2021)

[19] Tan, H.M., Wang, J.C., et al.: Selective mutual learning: an efficient approach for single channel speech separation. In: ICASSP, pp. 3678–3682. IEEE (2022)

[20] Tan, H.M., et al.: Speech separation using augmented-discrimination learning on squash-norm embedding vector and node encoder. IEEE Access 10 (2022)

[21] Vu, D.-Q., Thu, M.T.H.: Smooth balance softmax for long-tailed image classification. In: Nghia, P.T., Thai, V.D., Thuy, N.T., Huynh, V.-N., Van Huan, N. (eds.) ICTA 2024. LNNS, vol. 1205, pp. 323–331. Springer, Cham (2025). https://doi.org/10.1007/978-3-031-80943-9_36

[22] Vu, D.-Q., Van Ha, T., Dang, A., Phung T. Thu, T., Minh Tan, H.: How does data augmentation affect to model performance in long-tailed classification? In: Nguyen, N.T., Huynh, CP., Nguyen, T.T., Le-Khac, N.-A., Nguyen, Q.-V. (eds.) CITA 2024. LNNS, vol. 882, pp. 337–347. Springer, Cham (2024). https://doi.org/10.1007/978-3-031-74127-2_28

[23] Wang, T., Chen, X., Chen, Z., et al.: An adapter based multi-label pre-training for speech separation and enhancement. In: ICASSP, pp. 1–5. IEEE (2023)

[24] Zeghidour, N., Grangier, D.: Wavesplit: end-to-end speech separation by speaker clustering. TASLP 29, 2840–2849 (2021)

Ghi chú

Source ID: 235; Author classification: ICTU - Main Author; First listed author: Ha Minh Tan; Contact author: Duyen Nguyen Thi3,⋆; Total authors: 6; Springer matched from provided ICTA 2025 volumes

← Quay lại danh sách bài báo