Pre-trained Self-Attention Framework: An Efficient Mechanism for Source Separation
Tác giả
Tóm tắt
Tài liệu tham khảo
[1] Beltagy I, Peters ME, Cohan A (2020) Longformer: the long-document transformer
[2] Jing Chen J, Mao Q, Liu D (2020) Dual-path transformer network: direct context-aware modeling for end-to-end monaural speech separation. In: INTERSPEECH
[3] Child R, Gray S, Radford A, Sutskever I (2019) Generating long sequences with sparse transformers
[4] Dang F, Chen H, Zhang P (2022) Dpt-fsnet: dual-path transformer based full-band and sub-band fusion network for speech enhancement. In: ICASSP, pp 6857–6861. IEEE
[5] Dong L, Xu S, Xu B (2018) Speech-transformer: a no-recurrence sequence-to-sequence model for speech recognition. In: ICASSP, pp 5884–5888. IEEE
[6] Gulati A, Qin J, Chiu CC, Parmar N, Zhang Y, Yu J, Han W, Wang S, Zhang Z, Wu Y et al (2020) Conformer: convolution-augmented transformer for speech recognition. Interspeech
[7] Hershey JR, Chen Z, Le Roux J, Watanabe S (2016) Deep clustering: discriminative embeddings for segmentation and separation. In: ICASSP, pp 31–35. IEEE
[8] Huang KP, Wu YK, Lee HY (2022) Improving the transferability of speech separation by meta-learning. CoRR abs/2203.05882
[9] Isik Y, Roux JL, Chen Z, Watanabe S, Hershey JR (2016) Single-channel multi-speaker separation using deep clustering. In: Interspeech, pp 545–549
[10] Kenton JDMWC, Toutanova LK (2019) Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of naacL-HLT, vol 1, p 2
[11] Kitaev N, Kaiser Ł, Levskaya A (2020) Reformer: the efficient transformer. ICLR
[12] Kolbæk M, Yu D, Tan ZH, Jensen J (2017) Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks. TASLP 25(10):1901–1913
[13] Lam MW, Wang J, Su D, Yu D (2021) Sandglasset: a light multi-granularity self-attentive network for time-domain speech separation. In: ICASSP. IEEE
[14] Le Roux J, Wisdom S, Erdogan H, Hershey JR (2019) Sdr–half-baked or well done? In: ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 626–630. IEEE
[15] Liu Y, Gu J, Goyal N, Li X, Edunov S, Ghazvininejad M, Lewis M, Zettlemoyer L (2020) Multilingual denoising pre-training for neural machine translation. TACL 8:726–742
[16] Luo Y, Chen Z, Mesgarani N (2018) Speaker-independent speech separation with deep attractor network. IEEE/ACM Trans Audio Speech Lang Process 26(4):787–796
[17] Luo Y, Chen Z et al (2020) Dual-path rnn: efficient long sequence modeling for time-domain single-channel speech separation. In: ICASSP, pp 46–50. IEEE
[18] Luo Y, Han C, Mesgarani N (2021) Group communication with context codec for lightweight source separation. TASLP 29:1752–1761
[19] Luo Y, Mesgarani N (2018) Tasnet: time-domain audio separation network for real-time, single-channel speech separation. In: ICASSP, pp 696–700. IEEE
[20] Luo Y, Mesgarani N (2019) Conv-tasnet: surpassing ideal time-frequency magnitude masking for speech separation. TASLP 27(8):1256–1266
[21] Phung T, Vu DQ, Mai-Tan H, Nhung LT (2022) Deep models for mispronounce prediction for vietnamese learners of english. In: International conference on future data and security engineering, pp 682–689. Springer
[22] Ranftl R, Bochkovskiy A, Koltun V (2021) Vision transformers for dense prediction. In: ICCV, pp 12179–12188
[23] Rixen J, Renz M (2022) Sfsrnet: super-resolution for single-channel audio source separation. In: AAAI, vol 36, pp 11220–11228
[24] Subakan C, Ravanelli M, Cornell S, Bronzi M, Zhong J (2021) Attention is all you need in speech separation. In: ICASSP, pp 21–25. IEEE
[25] Tan HM, Liang KW, Lee YS, Li CT, Li YH, Wang JC (2022) Speech separation using augmented-discrimination learning on squash-norm embedding vector and node encoder. IEEE Access 10:102048–102063
[26] Tan HM, Liang KW, Wang JC (2023) Discriminative vector learning with application to single channel speech separation. In: ICASSP, pp 1–5. IEEE
[27] Tan HM, Vu DQ, Lee CT, Li YH, Wang JC (2022) Selective mutual learning: an efficient approach for single channel speech separation. In: ICASSP. IEEE
[28] Tan HM, Vu DQ, Thi DN, Thu TPT (2023) Voice separation using multi learning on squash-norm embedding matrix and mask. In: ICTA. Springer
[29] Tan HM, Vu DQ, Wang JC (2023) Selinet: a lightweight model for single channel speech separation. In: ICASSP, pp 1–5. IEEE
[30] Tan HM, Wang JC (2021) Single channel speech separation using enhanced learning on embedding features. In: GCCE, pp 430–431. IEEE
[31] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. NIPS 30
[32] Vu DQ, Le N, Wang JC (2021) Teaching yourself: a self-knowledge distillation approach to action recognition. IEEE Access 9:105711–105723
[33] Vu DQ, Thu TPT (2023) Simultaneous context and motion learning in video prediction. Signal Image Video Process 1–10
[34] Wang S, Li BZ, Khabsa M, Fang H, Ma H (2020) Linformer: self-attention with linear complexity
[35] Wang T, Chen X, Chen Z et al (2023) An adapter based multi-label pre-training for speech separation and enhancement. In: ICASSP, pp 1–5. IEEE
[36] Wang Y, Huang R, Song S, Huang Z, Huang G (2021) Not all images are worth 16x16 words: dynamic transformers for efficient image recognition. NIPS 34:11960–11973
[37] Wang Z, Cornell S, Choi S, Lee Y, Kim B, Watanabe S (2023) Tf-gridnet: integrating full-and sub-band modeling for speech separation. TASLP
[38] Yang L, Liu W, Wang W (2022) Tfpsnet: time-frequency domain path scanning network for speech separation. In: ICASSP. IEEE
[39] Zeghidour N, Grangier D (2021) Wavesplit: end-to-end speech separation by speaker clustering. TASLP 29:2840–2849
[40] Zhao Y, Luo C, Zha ZJ, Zeng W (2021) Multi-scale group transformer for long sequence modeling in speech separation. In: IJCAI, pp 3251–3257