KYHTQT.

Deep Learning for Speech Separation: A Comprehensive Review

Năm XB 2024 Tạp chí / Hội thảo Lecture Notes in Networks and Systems Volume 1205 LNNS Đơn vị CNTT DOI / Link https://doi.org/10.1007/978-3-031-80943-9_25 ↗

Tác giả

Trung-Nghia Phung

Tóm tắt

Integrating deep learning in speech separation has revolutionized audio signal processing, impacting fields like speech recognition, audio-visual content creation, telecommunication, hearing aid technologies, etc. In time-frequency domain separation, models leverage amplitude and phase information while the waveform is used for training in the time-domain speech separation model. An emerging approach uses complex values, capturing intricate audio nuances for improved performance. Modern models prioritize speaker-independent separation, adapting to dynamic or unknown speaker identities. Their ability to simultaneously separate multiple speakers is crucial for applications like conference calls, overlapping dialogues, and surveillance. This review explores deep models in speech separation, offering insights into the time domain, and time-frequency domain strategies. We outline the current state of models and state-of-the-art learning methods for audio segmentation.

Tài liệu tham khảo

[1] Chen, J., Mao, Q., et al.: Dual-path transformer network: direct context-aware modeling for end-to-end monaural speech separation. In: InterSpeech (2020)

[2] Hershey, J.R., Chen, Z., Le Roux, J., et al.: Deep clustering: discriminative embeddings for segmentation and separation. In: ICASSP, pp. 31–35. IEEE (2016)

[3] Huang, P.S., Kim, M., et al.: Deep learning for monaural speech separation. In: ICASSP, pp. 1562–1566. IEEE (2014)

[4] Huang, P.S., Kim, M., et al.: Joint optimization of masks and deep recurrent neural networks for monaural source separation. TASLP 23(12), 2136–2147 (2015)

[5] Isik, Y., Roux, J.L., Chen, Z., Watanabe, S., Hershey, J.R.: Single-channel multi-speaker separation using deep clustering. In: Interspeech (2016)

[6] Kolbæk, M., et al.: Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks. IEEE TASLP 25 (2017)

[7] Le Roux, J., Wisdom, S., Erdogan, H., Hershey, J.R.: SDR–half-baked or well done? In: ICASSP, pp. 626–630. IEEE (2019)

[8] Lee, Y.S., Yu, K., et al.: Discriminative training of complex-valued deep recurrent neural network for singing voice separation. In: ACM, pp. 1327–1335 (2017)

[9] Luo, Y., Mesgarani, N.: Tasnet: time-domain audio separation network for real-time, single-channel speech separation. In: ICASSP, pp. 696–700. IEEE (2018)

[10] Luo, Y., Mesgarani, N.: Conv-tasnet: surpassing ideal time-frequency magnitude masking for speech separation. TASLP 27(8), 1256–1266 (2019)

[11] Luo, Y., Chen, Z., Mesgarani, N.: Speaker-independent speech separation with deep attractor network. TASLP 26(4), 787–796 (2018)

[12] Luo, Y., Chen, Z., et al.: Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation. In: ICASSP, pp. 46–50. IEEE (2020)

[13] Lutati, S., Nachmani, E., Wolf, L.: Sepit: approaching a single channel speech separation bound. arXiv preprint arXiv:2205.11801 (2022)

[14] Lutati, S., et al.: Separate and diffuse: using a pretrained diffusion model for improving source separation. arXiv preprint arXiv:2301.10752 (2023)

[15] Nachmani, E., Adi, Y., Wolf, L.: Voice separation with an unknown number of multiple speakers. In: ICML, pp. 7164–7175. PMLR (2020)

[16] Rixen, J., Renz, M.: Sfsrnet: super-resolution for single-channel audio source separation. In: AAAI, vol. 36, pp. 11220–11228 (2022)

[17] Subakan, C., Ravanelli, M., Cornell, S., Bronzi, M., Zhong, J.: Attention is all you need in speech separation. In: ICASSP, pp. 21–25. IEEE (2021)

[18] Tan, H.M., Liang, K.W., Wang, J.C.: Discriminative vector learning with application to single channel speech separation. In: ICASSP, pp. 1–5. IEEE (2023)

[19] Tan, H.M., Vu, D.Q., Thi, D.N., et al.: Voice separation using multi learning on squash-norm embedding matrix and mask. In: ICTA, pp. 327–333. Springer (2023)

[20] Tan, H.M., Vu, D.Q., et al.: Selective mutual learning: an efficient approach for single channel speech separation. In: ICASSP, pp. 3678–3682. IEEE (2022)

[21] Tan, H.M., Vu, D.Q., et al.: Selinet: a lightweight model for single channel speech separation. In: ICASSP, pp. 1–5. IEEE (2023)

[22] Tan, H.M., Wang, J.C.: Single channel speech separation using enhanced learning on embedding features. In: GCCE, pp. 430–431. IEEE (2021)

[23] Tan, H.M., et al.: Speech separation using augmented-discrimination learning on squash-norm embedding vector and node encoder. IEEE Access 10 (2022)

[24] Tzinis, E., et al.: Compute and memory efficient universal sound source separation. J. Signal Process. Syst. 94(2), 245–259 (2022)

[25] Wang, Z.Q., Cornell, S., Choi, S., Lee, Y., Kim, B.Y., Watanabe, S.: TF-gridnet: integrating full-and sub-band modeling for speech separation. TASLP (2023)

[26] Wang, Z.Q., Roux, J.L., et al.: End-to-end speech separation with unfolded iterative phase reconstruction. Interspeech (2018)

[27] Yang, L., Liu, W., et al.: Tfpsnet: time-frequency domain path scanning network for speech separation. In: ICASSP, pp. 6842–6846. IEEE (2022)

[28] Zeghidour, N., Grangier, D.: Wavesplit: end-to-end speech separation by speaker clustering. TASLP 29, 2840–2849 (2021)

[29] Zhao, S., et al.: Mossformer2: combining transformer and RNN-free recurrent network for enhanced time-domain monaural speech separation. In: ICASSP, pp. 10356–10360. IEEE (2024)

Ghi chú

ICTA

← Quay lại danh sách bài báo