mireabulletinRussian Technological JournalRussian Technological Journal2782-32102500-316XRTU MIREA10.32362/2500-316X-2025-13-3-7-20PVYBDDmireabulletin-1174Research ArticleИНФОРМАЦИОННЫЕ СИСТЕМЫ. ИНФОРМАТИКА. ПРОБЛЕМЫ ИНФОРМАЦИОННОЙ БЕЗОПАСНОСТИINFORMATION SYSTEMS. COMPUTER SCIENCES. ISSUES OF INFORMATION SECURITYМетод конвертации акцента с клонированием голоса в реальном времени на основе неавторегрессионной нейросетевой моделиAccent conversion method with real-time voice cloning based on a nonautoregressive neural network modelhttps://orcid.org/0009-0007-1449-3968НечаевВ. А.NechaevV. A.

Нечаев Владимир Алексеевич, преподаватель-исследователь153003, Россия, Иваново, ул. Рабфаковская, д. 34

Vladimir A. Nechaev, Teacher-Researcher, Ivanovo State Power Engineering University 34, Rabfakovskaya ul., Ivanovo, 153003 Russia

nechaev@gapps.ispu.ruhttps://orcid.org/0000-0003-0231-0750КосяковС. В.KosyakovS. V.

Косяков Сергей Витальевич, д.т.н., профессор, заведующий кафедрой программного обеспечения компьютерных систем 153003, Россия, Иваново, ул. Рабфаковская, д. 34Scopus Author ID 6507182528Researcher ID H-5686-2018

Sergey V. Kosyakov, Dr. Sci. (Eng.), Professor, Head of the Department of Computer Systems Software34, Rabfakovskaya ul., Ivanovo, 153003 Russia)Scopus Author ID 6507182528, ResearcherID H-5686-2018

ksv@ispu.ruФГБОУ ВО «Ивановский государственный энергетический университет имени В.И. Ленина»РоссияIvanovo State Power Engineering UniversityRussian Federation202505062025133720Copyright © Нечаев В.А., Косяков С.В., 20252025Нечаев В.А., Косяков С.В.Nechaev V.A., Kosyakov S.V.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://www.rtj-mirea.ru/jour/article/view/1174

Цели. В настоящее время при разработке моделей для преобразования речи с акцентом в речь без акцента используются архитектуры глубоких нейросетей, а также ансамбли предобученных нейросетей для распознавания и генерации речи. При этом доступ к реализациям таких моделей является ограниченным, что затрудняет их применение, изучение и дальнейшее развитие. Также использование данных моделей ограничено особенностями архитектуры, которая не позволяет гибко менять тембр генерируемой речи и требует накопления контекста, что ведет к увеличению задержки при генерации и делает данные системы непригодными для использования в сценариях коммуникации двух и более людей в реальном времени. В связи с этим актуальной задачей и целью настоящей работы является разработка метода, позволяющего на основе входной речи с акцентом генерировать речь без акцента с минимальными задержками с возможностью сохранения, клонирования и модификации тембра говорящего, что позволит преодолеть ограничения текущих моделей.Методы. Применены методы модификации, обучения и объединения глубоких нейросетей в единую сквозную архитектуру для прямого преобразования речи в речь. Для обучения использованы оригинальные и модифицированные наборы данных из открытых источников.Результаты. Разработан метод конвертации акцента с клонированием голоса в реальном времени на основе неавторегрессионной нейросетевой модели, которая состоит из модулей определения акцента и пола, идентификации говорящего, преобразования речи в фонетическое представление, генерации спектрограммы и декодирования полученной спектрограммы в аудиосигнал. Метод демонстрирует высокое качество конвертации акцента с сохранением оригинального тембра, а также низкие задержки при генерации, приемлемые для использования в сценариях реального времени.Выводы. Апробация разработанного метода подтвердила эффективность предложенной неавторегрессионной нейросетевой архитектуры. Разработанная прикладная нейросетевая модель продемонстрировала возможность работы в информационных системах на английском языке в режиме реального времени.

Objectives. The development of contemporary models for the conversion of accents in foreign languages utilizes deep neural network architectures, as well as ensembles of neural networks for speech recognition and generation. However, restricted access to implementations of such models limits their application, study, and further development. Moreover, the use of these models is limited by their architectural features, which prevents flexible changes from being carried out in the timbre of the generated speech and requires the accumulation of context, leading to increased delays in generation, making these systems unsuitable for use in real-time multiuser communication scenarios. Therefore, the relevant task and aim of this work is the development of a method that generates native-sounding speech based on input accented speech material with minimal delays and the capability to preserve, clone, and modify the timbre of the speaker’s voice.Methods. Methods for modifying, training, and combining deep neural networks into a single end-to-end architecture for direct speech-to-speech conversion are applied. For training, original and modified open-source datasets were used.Results. The work resulted in the development of a real-time accent conversion method with voice cloning based on a non-autoregressive neural network. The model comprises modules for accent and gender detection, speaker identification, speech conversion, spectrogram generation, and decoding the resulting spectrogram into an audio signal. As well as demonstrating high accent conversion quality while maintaining the original timbre, the short generation times of the applied method make it acceptable for use in real-time scenarios.Conclusions. Testing of the developed method confirmed the effectiveness of the proposed non-autoregressive neural network architecture. The developed model demonstrated the ability to work in real-time information systems in English.

конвертация акцентагенерация речираспознавание речиконвертация голосамашинное обучениенейронная сетьaccent conversionspeech synthesistext-to-speechvoice conversionmachine learningneural networkReferencesMcMillin D.C. Outsourcing identities: Call centres and cultural transformation in India. Economic and Political Weekly. 2006;41(3):235–241.McMillin D.C. Outsourcing identities: Call centres and cultural transformation in India. Economic and Political Weekly. 2006;41(3):235–241.Felps D., Bortfeld H., Gutierrez-Osuna R. Foreign accent conversion in computer assisted pronunciation training. Speech Communication. 2009;51(10):920–932. https://doi.org/10.1016/j.specom.2008.11.004Felps D., Bortfeld H., Gutierrez-Osuna R. Foreign accent conversion in computer assisted pronunciation training. Speech Communication. 2009;51(10):920–932. https://doi.org/10.1016/j.specom.2008.11.004Probst K., Ke Y., Eskenazi M. Enhancing foreign language tutors–In search of the golden speaker. Speech Communication. 2002;37(3–4):161–173. https://doi.org/10.1016/S0167-6393(01)00009-7Probst K., Ke Y., Eskenazi M. Enhancing foreign language tutors–In search of the golden speaker. Speech Communication. 2002;37(3–4):161–173. https://doi.org/10.1016/S0167-6393(01)00009-7Türk O., Arslan L. M. Subband based voice conversion. In: 7th International Conference on Spoken Language Processing, ICSLP2002 – INTERSPEECH 2002. Interspeech. 2002. P. 289–292.Türk O., Arslan L. M. Subband based voice conversion. In: 7th International Conference on Spoken Language Processing, ICSLP2002 – INTERSPEECH 2002. Interspeech. 2002. P. 289–292.Biadsy F., Weis, R.J., Moreno P.J., Kanevsky D., Jia Y. Parrotron: An end-to-end speech-to-speech conversion model and its applications to hearing-impaired speech and speech separation. Interspeech. 2019. P. 4115–4119. http://doi.org/10.21437/Interspeech.2019-1789Biadsy F., Weis, R.J., Moreno P.J., Kanevsky D., Jia Y. Parrotron: An end-to-end speech-to-speech conversion model and its applications to hearing-impaired speech and speech separation. Interspeech. 2019. P. 4115–4119. http://doi.org/10.21437/Interspeech.2019-1789Birner B. Why Do Some People Have an Accent? Linguistic Society of America. Washington, DC. 1999. 6 p.Birner B. Why Do Some People Have an Accent? Linguistic Society of America. Washington, DC. 1999. 6 p.Baese-Berk M.M., Morrill T.H. Speaking rate consistency in native and non-native speakers of English. J. Acoust. Soc. Am. 2015;138(3):EL223–EL228. https://doi.org/10.1121/1.4929622Baese-Berk M.M., Morrill T.H. Speaking rate consistency in native and non-native speakers of English. J. Acoust. Soc. Am. 2015;138(3):EL223–EL228. https://doi.org/10.1121/1.4929622Piske T., MacKay I.R.A., Flege J.E. Factors affecting degree of foreign accent in an L2: A review. J. Phonetics. 2001;29(2): 191–215. https://doi.org/10.1006/jpho.2001.0134Piske T., MacKay I.R.A., Flege J.E. Factors affecting degree of foreign accent in an L2: A review. J. Phonetics. 2001;29(2): 191–215. https://doi.org/10.1006/jpho.2001.0134Munro M.J., Derwing T.M. Foreign accent, comprehensibility, and intelligibility in the speech of second language learners. Language Learning. 1995;45(1):73–97. https://doi.org/10.1111/j.1467-1770.1995.tb00963.xMunro M.J., Derwing T.M. Foreign accent, comprehensibility, and intelligibility in the speech of second language learners. Language Learning. 1995;45(1):73–97. https://doi.org/10.1111/j.1467-1770.1995.tb00963.xLev-Ari S., Keysar B. Why don’t we believe non-native speakers? The influence of accent on credibility. J. Exp. Soc. Psychol. 2010;46(6):1093–1096. https://doi.org/10.1016/j.jesp.2010.05.025Lev-Ari S., Keysar B. Why don’t we believe non-native speakers? The influence of accent on credibility. J. Exp. Soc. Psychol. 2010;46(6):1093–1096. https://doi.org/10.1016/j.jesp.2010.05.025Rubin D.L., Smith K.A. Effects of accent, ethnicity, and lecture topic on undergraduates’ perceptions of nonnative Englishspeaking teaching assistants. Int. J. Intercult. Relat. 1990;14(3):337–353. https://doi.org/10.1016/0147-1767(90)90019-SRubin D.L., Smith K.A. Effects of accent, ethnicity, and lecture topic on undergraduates’ perceptions of nonnative Englishspeaking teaching assistants. Int. J. Intercult. Relat. 1990;14(3):337–353. https://doi.org/10.1016/0147-1767(90)90019-SNelson Jr. L.R., Signorella M.L., Botti K.G. Accent, gender, and perceived competence. Hispanic J. Behavior. Sci. 2016;38(2):166–185. https://doi.org/10.1177/0739986316632319Nelson Jr. L.R., Signorella M.L., Botti K.G. Accent, gender, and perceived competence. Hispanic J. Behavior. Sci. 2016;38(2):166–185. https://doi.org/10.1177/0739986316632319Zhao G., Gutierrez-Osuna R. Using phonetic posteriorgram based frame pairing for segmental accent conversion. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2019;27(10):1649–1660. https://doi.org/10.1109/TASLP.2019.2926754Zhao G., Gutierrez-Osuna R. Using phonetic posteriorgram based frame pairing for segmental accent conversion. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2019;27(10):1649–1660. https://doi.org/10.1109/TASLP.2019.2926754Zhao G., Sonsaat S., Levis J., Chukharev-Hudilainen E., Gutierrez-Osuna R. Accent conversion using phonetic posteriorgrams. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2018. P. 5314–5318. https://doi.org/10.1109/ICASSP.2018.8462258Zhao G., Sonsaat S., Levis J., Chukharev-Hudilainen E., Gutierrez-Osuna R. Accent conversion using phonetic posteriorgrams. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2018. P. 5314–5318. https://doi.org/10.1109/ICASSP.2018.8462258Aryal S., Gutierrez-Osuna R. Can voice conversion be used to reduce non-native accents? In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2014. P. 7879–7883. https://doi.org/10.1109/ICASSP.2014.6855134Aryal S., Gutierrez-Osuna R. Can voice conversion be used to reduce non-native accents? In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2014. P. 7879–7883. https://doi.org/10.1109/ICASSP.2014.6855134Ding S., Zhao G., Gutierrez-Osuna R. Accentron: Foreign accent conversion to arbitrary non-native speakers using zero-shot learning. Computer Speech & Language. 2022;72:101302. https://doi.org/10.1016/j.csl.2021.101302Ding S., Zhao G., Gutierrez-Osuna R. Accentron: Foreign accent conversion to arbitrary non-native speakers using zero-shot learning. Computer Speech & Language. 2022;72:101302. https://doi.org/10.1016/j.csl.2021.101302Quamer W., Das A., Levis J., Chukharev-Hudilainen E., Gutierrez-Osuna R. Zero-shot foreign accent conversion without a native reference. Proc. Interspeech. 2022. http://doi.org/10.21437/Interspeech.2022-10664Quamer W., Das A., Levis J., Chukharev-Hudilainen E., Gutierrez-Osuna R. Zero-shot foreign accent conversion without a native reference. Proc. Interspeech. 2022. http://doi.org/10.21437/Interspeech.2022-10664Jin M., Serai P., Wu J., Tjandra A., Manohar V., He Q. Voice-preserving zero-shot multiple accent conversion. In: ICASSP 2023–2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2023. P. 1–5. https://doi.org/10.1109/ICASSP49357.2023.10094737Jin M., Serai P., Wu J., Tjandra A., Manohar V., He Q. Voice-preserving zero-shot multiple accent conversion. In: ICASSP 2023–2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2023. P. 1–5. https://doi.org/10.1109/ICASSP49357.2023.10094737Zhou Y., Wu Z., Zhang M., Tian X., Li H. TTS-guided training for accent conversion without parallel data. IEEE Signal Proc. Lett. 2023;30:533–537. https://doi.org/10.1109/lsp.2023.3270079Zhou Y., Wu Z., Zhang M., Tian X., Li H. TTS-guided training for accent conversion without parallel data. IEEE Signal Proc. Lett. 2023;30:533–537. https://doi.org/10.1109/lsp.2023.3270079Zhao G., Ding S., Gutierrez-Osuna R. Converting foreign accent speech without a reference. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2021;29:2367–2381. https://doi.org/10.1109/TASLP.2021.3060813Zhao G., Ding S., Gutierrez-Osuna R. Converting foreign accent speech without a reference. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2021;29:2367–2381. https://doi.org/10.1109/TASLP.2021.3060813Liu S., Wang D., Cao Y., Sun L., Wu X., Kang S., Wu Z., Liu X., Su D., Yu D., Meng H. End-to-end accent conversion without using native utterances. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2020. P. 6289–6293.Liu S., Wang D., Cao Y., Sun L., Wu X., Kang S., Wu Z., Liu X., Su D., Yu D., Meng H. End-to-end accent conversion without using native utterances. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2020. P. 6289–6293.Zhou X., Zhang M., Zhou Y., Wu Z., Li H. Accented text-to-speech synthesis with limited data. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2024;32:1699–1711. https://doi.org/10.1109/TASLP.2024.3363414Zhou X., Zhang M., Zhou Y., Wu Z., Li H. Accented text-to-speech synthesis with limited data. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2024;32:1699–1711. https://doi.org/10.1109/TASLP.2024.3363414Pinget A.F., Bosker H.R., Quené H., De Jong, N.H. Native speakers’ perceptions of fluency and accent in L2 speech. Language Testing. 2014;31(3):349–365. https://doi.org/10.1177/0265532214526177Pinget A.F., Bosker H.R., Quené H., De Jong, N.H. Native speakers’ perceptions of fluency and accent in L2 speech. Language Testing. 2014;31(3):349–365. https://doi.org/10.1177/0265532214526177Бархударова Е.Л. Методологические проблемы анализа иностранного акцента в русской речи. Вестник Московского университета. Серия 9. Филология. 2012;6:57–70. [Barkhudarova E.L. Methodological Problems in Analyzing Foreign Accents in Russian Speech. Vestnik Moskovskogo universiteta. Seriya 9. Filologiya = Lomonosov Philology J. 2012;6:57–70 (in Russ.).]Бархударова Е.Л. Методологические проблемы анализа иностранного акцента в русской речи. Вестник Московского университета. Серия 9. Филология. 2012;6:57–70. [Barkhudarova E.L. Methodological Problems in Analyzing Foreign Accents in Russian Speech. Vestnik Moskovskogo universiteta. Seriya 9. Filologiya = Lomonosov Philology J. 2012;6:57–70 (in Russ.).]Arik S., Chen J., Peng K., Ping W., Zhou Y. Neural voice cloning with a few samples. Advances in Neural Information Processing Systems (NeurIPS 2018). 2018;31.Arik S., Chen J., Peng K., Ping W., Zhou Y. Neural voice cloning with a few samples. Advances in Neural Information Processing Systems (NeurIPS 2018). 2018;31.Cohen D. Issues in transnet packetized voice communication. In: Proceedings of the fifth Symposium on Data Communications (SIGCOMM’77). 1977. P. 6.10–6.13. https://doi.org/10.1145/800103.803349Cohen D. Issues in transnet packetized voice communication. In: Proceedings of the fifth Symposium on Data Communications (SIGCOMM’77). 1977. P. 6.10–6.13. https://doi.org/10.1145/800103.803349Liang Y.J., Farber N., Girod B. Adaptive playout scheduling and loss concealment for voice communication over IP networks. IEEE Trans. Multimedia. 2003;5(4):532–543. https://doi.org/10.1109/TMM.2003.819095Liang Y.J., Farber N., Girod B. Adaptive playout scheduling and loss concealment for voice communication over IP networks. IEEE Trans. Multimedia. 2003;5(4):532–543. https://doi.org/10.1109/TMM.2003.819095Matzinger T., Pleyer M., Żywiczyński P. Pause Length and Differences in Cognitive State Attribution in Native and Non-Native Speakers. Languages. 2023;8(1):26. http://doi.org/10.3390/languages8010026Matzinger T., Pleyer M., Żywiczyński P. Pause Length and Differences in Cognitive State Attribution in Native and Non-Native Speakers. Languages. 2023;8(1):26. http://doi.org/10.3390/languages8010026Medsker L.R., Jain L. (Eds.). Recurrent Neural Networks. Design and Applications. Boca Raton: CRC Press; 2001. 416 p.Medsker L.R., Jain L. (Eds.). Recurrent Neural Networks. Design and Applications. Boca Raton: CRC Press; 2001. 416 p.Li J., Lavrukhin V., Ginsburg B., Leary R., Kuchaiev O., Cohen J.M., Nguyen H., Gadde R.T. Jasper: An End-to-End Convolutional Neural Acoustic Model. Interspeech 2019. 2019. https://doi.org/10.21437/interspeech.2019-1819Li J., Lavrukhin V., Ginsburg B., Leary R., Kuchaiev O., Cohen J.M., Nguyen H., Gadde R.T. Jasper: An End-to-End Convolutional Neural Acoustic Model. Interspeech 2019. 2019. https://doi.org/10.21437/interspeech.2019-1819Dawalatabad N., Ravanelli M., Grondin F., Thienpondt J., Desplanques B., Na H. ECAPA-TDNN Embeddings for Speaker Diarization. arXiv preprint arXiv:2104.01466. 2021. https://doi.org/10.48550/arXiv.2104.01466Dawalatabad N., Ravanelli M., Grondin F., Thienpondt J., Desplanques B., Na H. ECAPA-TDNN Embeddings for Speaker Diarization. arXiv preprint arXiv:2104.01466. 2021. https://doi.org/10.48550/arXiv.2104.01466Ravanelli M., Bengio Y. Speaker recognition from raw waveform with SincNet. In: 2018 IEEE Spoken Language Technology Workshop (SLT). IEEE; 2018. P. 1021–1028. https://doi.org/10.1109/SLT.2018.8639585Ravanelli M., Bengio Y. Speaker recognition from raw waveform with SincNet. In: 2018 IEEE Spoken Language Technology Workshop (SLT). IEEE; 2018. P. 1021–1028. https://doi.org/10.1109/SLT.2018.8639585Snyder D., Garcia-Romero D., Sell G., Povey D., Khudanpur S. X-vectors: Robust DNN embeddings for speaker recognition. In: 2018 IEEE international Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2018. P. 5329–5333. http://doi.org/10.1109/ICASSP.2018.8461375Snyder D., Garcia-Romero D., Sell G., Povey D., Khudanpur S. X-vectors: Robust DNN embeddings for speaker recognition. In: 2018 IEEE international Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2018. P. 5329–5333. http://doi.org/10.1109/ICASSP.2018.8461375Deng J., Guo J., Xue N., Zafeiriou S. Arcface: Additive angular margin loss for deep face recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE; 2019. P. 4690–4699. https://doi.org/10.1109/CVPR.2019.00482Deng J., Guo J., Xue N., Zafeiriou S. Arcface: Additive angular margin loss for deep face recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE; 2019. P. 4690–4699. https://doi.org/10.1109/CVPR.2019.00482Gulati A., Qin J., Chiu C., Parmar N., Zhang Y., Yu J., Han W., Wang S., Zhang Z., Wu Y., Pang R. Conformer: Convolutionaugmented trans-former for speech recognition. Proc. Interspeech 2020. 2020. P. 5036–5040. https://doi.org/10.21437/interspeech.2020-3015Gulati A., Qin J., Chiu C., Parmar N., Zhang Y., Yu J., Han W., Wang S., Zhang Z., Wu Y., Pang R. Conformer: Convolutionaugmented trans-former for speech recognition. Proc. Interspeech 2020. 2020. P. 5036–5040. https://doi.org/10.21437/interspeech.2020-3015Glorot X., Bengio Y. Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research. 2010. P. 249–256. URL: http://proceedings.mlr.press/v9/glorot10a.htmlGlorot X., Bengio Y. Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research. 2010. P. 249–256. URL: http://proceedings.mlr.press/v9/glorot10a.htmlGu J., Wang Z., Kuen J., Ma L., Shahroudy A., Shuai B., Liu T., Wang X., Wang G., Cai J., Chen T. Recent advances in convolutional neural networks. Pattern Recognition. 2018;77:354–377. https://doi.org/10.1016/j.patcog.2017.10.013Gu J., Wang Z., Kuen J., Ma L., Shahroudy A., Shuai B., Liu T., Wang X., Wang G., Cai J., Chen T. Recent advances in convolutional neural networks. Pattern Recognition. 2018;77:354–377. https://doi.org/10.1016/j.patcog.2017.10.013Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A.N., Kaiser Ł., Polosukhin I. Attention is all you need. Advances in Neural Information Processing Systems. 2017;30:5999–6009. https://doi.org/10.48550/arXiv.1706.03762Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A.N., Kaiser Ł., Polosukhin I. Attention is all you need. Advances in Neural Information Processing Systems. 2017;30:5999–6009. https://doi.org/10.48550/arXiv.1706.03762Ren Y., Ruan Y., Tan X., Qin T., Zhao S., Zhao Z., Liu T.Y. FastSpeech: Fast, robust and controllable text to speech. Advances in Neural Information Processing Systems. 2019;32. https://doi.org/10.48550/arXiv.1905.09263Ren Y., Ruan Y., Tan X., Qin T., Zhao S., Zhao Z., Liu T.Y. FastSpeech: Fast, robust and controllable text to speech. Advances in Neural Information Processing Systems. 2019;32. https://doi.org/10.48550/arXiv.1905.09263Graves A., Fernández S., Gomez F., Schmidhuber J. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd International Conference on Machine Learning. 2006. P. 369–376. https://doi.org/10.1145/1143844.1143891Graves A., Fernández S., Gomez F., Schmidhuber J. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd International Conference on Machine Learning. 2006. P. 369–376. https://doi.org/10.1145/1143844.1143891Ghahremani P., BabaAli B., Povey D., Riedhammer K., Trmal J., Khudanpur S. A pitch extraction algorithm tuned for automatic speech recognition. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2014. P. 2494–2498. http://doi.org/10.1109/ICASSP.2014.6854049Ghahremani P., BabaAli B., Povey D., Riedhammer K., Trmal J., Khudanpur S. A pitch extraction algorithm tuned for automatic speech recognition. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2014. P. 2494–2498. http://doi.org/10.1109/ICASSP.2014.6854049Gerhard D. Pitch Extraction and Fundamental Frequency: History and Current Techniques. Masters Thesis. Regina, SK, Canada: Department of Computer Science, University of Regina; 2003. 23 p.Gerhard D. Pitch Extraction and Fundamental Frequency: History and Current Techniques. Masters Thesis. Regina, SK, Canada: Department of Computer Science, University of Regina; 2003. 23 p.Łańcucki A. Fastpitch: Parallel text-to-speech with pitch prediction. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2021. P. 6588–6592. https://doi.org/10.1109/ICASSP39728.2021.9413889Łańcucki A. Fastpitch: Parallel text-to-speech with pitch prediction. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2021. P. 6588–6592. https://doi.org/10.1109/ICASSP39728.2021.9413889Kong J., Kim J., Bae J. HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis. Advances in Neural Information Processing Systems. 2020;33:17022–17033. http://doi.org/10.48550/arXiv.2010.05646Kong J., Kim J., Bae J. HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis. Advances in Neural Information Processing Systems. 2020;33:17022–17033. http://doi.org/10.48550/arXiv.2010.05646Kominek J., Black A.W. The CMU Arctic speech databases. In: Fifth ISCA Workshop on Speech Synthesis. 2004. P. 223–224.Kominek J., Black A.W. The CMU Arctic speech databases. In: Fifth ISCA Workshop on Speech Synthesis. 2004. P. 223–224.Zhao G., Sonsaat S., Silpachai A., Lucic I., Chukharev-Hudilainen E., Levis J., Gutierrez-Osuna R. L2-ARCTIC: A Nonnative English Speech Corpus. Interspeech 2018. 2018. P. 2783–2787. http://doi.org/10.21437/Interspeech.2018-1110Zhao G., Sonsaat S., Silpachai A., Lucic I., Chukharev-Hudilainen E., Levis J., Gutierrez-Osuna R. L2-ARCTIC: A Nonnative English Speech Corpus. Interspeech 2018. 2018. P. 2783–2787. http://doi.org/10.21437/Interspeech.2018-1110Weinberger S.H., Kunath S.A. The Speech Accent Archive: towards a typology of English accents. In: Corpus-based Studies in Language Use, Language Learning, and Language Documentation. Brill; 2011. P. 265–281. https://doi.org/10.1163/9789401206884_014Weinberger S.H., Kunath S.A. The Speech Accent Archive: towards a typology of English accents. In: Corpus-based Studies in Language Use, Language Learning, and Language Documentation. Brill; 2011. P. 265–281. https://doi.org/10.1163/9789401206884_014Ardila R., Branson M., Davis K., Kohler M., Meyer J., Henretty M., Henretty M., Morais R., Saunders L., Tyers F., Weber G. Common Voice: A Massively-Multilingual Speech Corpus. In: Proceedings of the Twelfth Language Resources and Evaluation Conference. 2020. P. 4218–4222. https://doi.org/10.48550/arXiv.1912.06670Ardila R., Branson M., Davis K., Kohler M., Meyer J., Henretty M., Henretty M., Morais R., Saunders L., Tyers F., Weber G. Common Voice: A Massively-Multilingual Speech Corpus. In: Proceedings of the Twelfth Language Resources and Evaluation Conference. 2020. P. 4218–4222. https://doi.org/10.48550/arXiv.1912.06670Nagrani A., Chung J.S., Zisserman A. Voxceleb: a large-scale speaker identification dataset. Interspeech 2017. 2017. http://doi.org/10.21437/Interspeech.2017-950Nagrani A., Chung J.S., Zisserman A. Voxceleb: a large-scale speaker identification dataset. Interspeech 2017. 2017. http://doi.org/10.21437/Interspeech.2017-950Chung J., Nagrani A., Zisserman A. VoxCeleb2: Deep speaker recognition. Interspeech 2018. 2018. http://doi.org/10.21437/Interspeech.2018-1929Chung J., Nagrani A., Zisserman A. VoxCeleb2: Deep speaker recognition. Interspeech 2018. 2018. http://doi.org/10.21437/Interspeech.2018-1929Panayotov V., Chen G., Povey D., Khudanpur S. Librispeech: an ASR corpus based on public domain audio books. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2015. P. 5206–5210. http://doi.org/10.1109/ICASSP.2015.7178964Panayotov V., Chen G., Povey D., Khudanpur S. Librispeech: an ASR corpus based on public domain audio books. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2015. P. 5206–5210. http://doi.org/10.1109/ICASSP.2015.7178964Veaux C., Yamagishi J., MacDonald K. CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit. University of Edinburgh. The Center for Speech Technology Research (CSTR). 2017. https://doi.org/10.7488/ds/2645Veaux C., Yamagishi J., MacDonald K. CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit. University of Edinburgh. The Center for Speech Technology Research (CSTR). 2017. https://doi.org/10.7488/ds/2645Chen G., Chai S., Wang G., Du J., Zhang W., Weng C., Su D., Povey D., TrmalJ., Zhang J., Jin M., Khudanpur S., Watanabe S., Zhao S., Zou W., Li X., Yao X., Wang Y., Wang Y., You Z., Yan Z. GigaSpeech: An evolving, multi-domain ASR corpus with 10,000 hours of transcribed audio. In: 22nd Annual Conference of the International Speech Communication Association, Interspeech 2021. International Speech Communication Association; 2021. P. 4376–4380. https://doi.org/10.21437/Interspeech.2021-1965Chen G., Chai S., Wang G., Du J., Zhang W., Weng C., Su D., Povey D., TrmalJ., Zhang J., Jin M., Khudanpur S., Watanabe S., Zhao S., Zou W., Li X., Yao X., Wang Y., Wang Y., You Z., Yan Z. GigaSpeech: An evolving, multi-domain ASR corpus with 10,000 hours of transcribed audio. In: 22nd Annual Conference of the International Speech Communication Association, Interspeech 2021. International Speech Communication Association; 2021. P. 4376–4380. https://doi.org/10.21437/Interspeech.2021-1965Bakhturina E., Zhang Y., Ginsburg B. Shallow Fusion of Weighted Finite-State Transducer and Language Model for Text Normalization. Proc. Interspeech 2022. 2022. http://doi.org/10.48550/arXiv.2203.15917Bakhturina E., Zhang Y., Ginsburg B. Shallow Fusion of Weighted Finite-State Transducer and Language Model for Text Normalization. Proc. Interspeech 2022. 2022. http://doi.org/10.48550/arXiv.2203.15917Kudo T., Richardson J. SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2018. P. 66–71. https://doi.org/10.48550/arXiv.1808.06226Kudo T., Richardson J. SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2018. P. 66–71. https://doi.org/10.48550/arXiv.1808.06226Koizumi Y., Zen H., Karita S., Ding Y., Yatabe K., Morioka N., Bacchiani M., Zhang Y., Han W., Bapna A. Libritts-r: A Restored Multi-Speaker Text-to-Speech Corpus. arXiv preprint arXiv:2305.18802. 2023. https://doi.org/10.48550/arXiv.2305.18802Koizumi Y., Zen H., Karita S., Ding Y., Yatabe K., Morioka N., Bacchiani M., Zhang Y., Han W., Bapna A. Libritts-r: A Restored Multi-Speaker Text-to-Speech Corpus. arXiv preprint arXiv:2305.18802. 2023. https://doi.org/10.48550/arXiv.2305.18802Paszke A., Gross S., Massa F., Lerer A., Bradbury J., Chanan G., Killeen T., Lin Z., Gimelshein N., Antiga L., Desmaison A., Kopf A., Yang E., DeVito Z., Raison M., Tejani A., Chilamkurthy S., SteinerB., Fang L., Bai J., Chintala S. PyTorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems. 2019;32:8024–8035.Paszke A., Gross S., Massa F., Lerer A., Bradbury J., Chanan G., Killeen T., Lin Z., Gimelshein N., Antiga L., Desmaison A., Kopf A., Yang E., DeVito Z., Raison M., Tejani A., Chilamkurthy S., SteinerB., Fang L., Bai J., Chintala S. PyTorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems. 2019;32:8024–8035.Kuchaiev O., Li J., Nguyen H., Hrinchuk O., Leary R., Ginsburg B., Kriman S., Beliaev S., Lavrukhin V., Cook J., Castonguay P., Popova M., Huang J., Cohen J. Nemo: a toolkit for building ai applications using neural modules. arXiv preprint arXiv:1909.09577. 2019. https://doi.org/10.48550/arXiv.1909.09577Kuchaiev O., Li J., Nguyen H., Hrinchuk O., Leary R., Ginsburg B., Kriman S., Beliaev S., Lavrukhin V., Cook J., Castonguay P., Popova M., Huang J., Cohen J. Nemo: a toolkit for building ai applications using neural modules. arXiv preprint arXiv:1909.09577. 2019. https://doi.org/10.48550/arXiv.1909.09577Bredin H., Yin R., Coria J.M., Gelly G., Korshunov P., Lavechin M., Fustes D., Titeux H., Bouaziz W., Gill M.P. Pyannote. Audio: neural building blocks for speaker diarization. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2020. P. 7124–7128. https://doi.org/10.1109/ICASSP40776.2020.9052974Bredin H., Yin R., Coria J.M., Gelly G., Korshunov P., Lavechin M., Fustes D., Titeux H., Bouaziz W., Gill M.P. Pyannote. Audio: neural building blocks for speaker diarization. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2020. P. 7124–7128. https://doi.org/10.1109/ICASSP40776.2020.9052974Olatunji T., Afonja T., Yadavalli A., Emezue C.C., Singh S., Dossou B.F., Osuchukwu J., Osei S., Tonja A.L., Etori N., Mbataku C. AfriSpeech-200: Pan-African Accented Speech Dataset for Clinical and General Domain ASR. Transactions of the Association for Computational Linguistics. 2023;11:1669–1685. https://doi.org/10.1162/tacl_a_00627Olatunji T., Afonja T., Yadavalli A., Emezue C.C., Singh S., Dossou B.F., Osuchukwu J., Osei S., Tonja A.L., Etori N., Mbataku C. AfriSpeech-200: Pan-African Accented Speech Dataset for Clinical and General Domain ASR. Transactions of the Association for Computational Linguistics. 2023;11:1669–1685. https://doi.org/10.1162/tacl_a_00627Majumdar S., Balam J., Hrinchuk O., Lavrukhin V., Noroozi V., Ginsburg B. Citrinet: Closing the gap between nonautoregressive and autoregressive end-to-end models for automatic speech recognition. arXiv preprint arXiv:2104.01721.2021. http://doi.org/10.48550/arXiv.2104.01721Majumdar S., Balam J., Hrinchuk O., Lavrukhin V., Noroozi V., Ginsburg B. Citrinet: Closing the gap between nonautoregressive and autoregressive end-to-end models for automatic speech recognition. arXiv preprint arXiv:2104.01721.2021. http://doi.org/10.48550/arXiv.2104.01721Radford A., Kim J.W., Xu T., Brockman G., McLeavey C., Sutskever I. Robust speech recognition via large-scale weak supervision. In: Proceedings of the 40th International Conference on Machine Learning. PMLR 202. 2023. P. 28492–28518. http://doi.org/10.48550/arXiv.2212.04356Radford A., Kim J.W., Xu T., Brockman G., McLeavey C., Sutskever I. Robust speech recognition via large-scale weak supervision. In: Proceedings of the 40th International Conference on Machine Learning. PMLR 202. 2023. P. 28492–28518. http://doi.org/10.48550/arXiv.2212.04356

The authors declare that there are no conflicts of interest present.