{
    "componentChunkName": "component---src-templates-post-tsx",
    "path": "/one-model-many-langs/",
    "result": {"data":{"logo":null,"markdownRemark":{"html":"<h1>One Model, Many Languages: Meta-learning for Multilingual Text-to-Speech</h1>\n<p>Tomáš Nekvinda, Ondřej Dušek<br>\nCharles University<br>\nINTERSPEECH, 2020</p>\n<h2>Reference</h2>\n<ul>\n<li><a href=\"https://arxiv.org/abs/2008.00768\">ArXiv</a></li>\n<li><a href=\"https://github.com/Tomiinek/Multilingual_Text_to_Speech\">Source Code</a></li>\n<li><a href=\"https://tomiinek.github.io/multilingual_speech_samples/\">Demo Webpage</a></li>\n<li><a href=\"https://arxiv.org/abs/1703.10135\">Tacotron</a>, <a href=\"https://arxiv.org/abs/1712.05884\">Tacotron2</a></li>\n<li><a href=\"https://arxiv.org/pdf/1710.08969.pdf\">DC-TTS</a></li>\n<li><a href=\"https://github.com/Kyubyong/css10\">CSS 10 Dataset</a></li>\n<li><a href=\"https://commonvoice.mozilla.org/en/datasets\">Common Voice Dataset</a></li>\n</ul>\n<h2>Summary</h2>\n<ul>\n<li>Multilingual Speech Synthesis</li>\n<li>Meta-learning</li>\n<li>Voice Cloning : Speech in multiple languages with the same voice</li>\n<li>Code switching : Speak two (or more) languages with a single utterance.</li>\n<li>Tacotron2 base architecture</li>\n</ul>\n<h2>Tacotron</h2>\n<img src=\"https://img1.daumcdn.net/thumb/R1280x0/?scode=mtistory2&amp;fname=https%3A%2F%2Fblog.kakaocdn.net%2Fdn%2FwnGyQ%2FbtqDblNauXg%2FJsSXkwgQY1yc3lIHtdgIP0%2Fimg.png\" width=\"700\">\n<ul>\n<li>딥러닝 기반 음성합성의 대표적인 모델</li>\n<li>Attention + Sequence-to-Sequence의 TTS 버전</li>\n<li>Griffin-Lim Vocoder 사용 (빠르지만 성능은 좋지 못함)</li>\n</ul>\n<h2>Tacotron2</h2>\n<img src=\"https://user-images.githubusercontent.com/42150335/94840259-1cfbe900-0453-11eb-8803-cac2ea30b425.png\" width=\"470\">  \n<ul>\n<li>Mel-Prediction Network : Attention based Sequence-to-Sequence Network\n<ul>\n<li>인코더에서 Bi-directional LSTM 적용</li>\n<li>Location Sensitive Attention 적용 (음성 Alignment에 강한 어텐션)</li>\n<li>인코더, 디코더에 Convolution Layer 적용</li>\n</ul>\n</li>\n<li>Stop Token 사용</li>\n<li>Vocoder : WaveNet\n<ul>\n<li>장점 : 상당히 고품질의 음성으로 변환</li>\n<li>단점 : 엄청나게 느림</li>\n</ul>\n</li>\n</ul>\n<h2>Model Architecture</h2>\n<img src=\"https://github.com/Tomiinek/Multilingual_Text_to_Speech/raw/master/_img/generated.png\" width=\"800\">\n<ul>\n<li>Tacotron2 기반의 모델들로 실험 진행</li>\n<li>WaveRNN Vocoder 사용</li>\n</ul>\n<h3>This Paper`s Model: Generated (GEN)</h3>\n<ul>\n<li>\n<p><strong>Parameter Generation Convolutional Encoder</strong></p>\n<ul>\n<li>이 논문에서는 Fully convolutional encoder를 사용 (from DC-TTS)</li>\n<li>Cross-lingual knowledge-sharing을 가능하게 하기 위해 인코더 컨볼루션 레이어의 파라미터를 생성하여 사용</li>\n<li>입력되는 Language ID에 따라 Fully Connected 레이어를 통해 다른 다른 파라미터를 생성</li>\n</ul>\n</li>\n<li>\n<p><strong>Speaker Embedding</strong></p>\n<ul>\n<li>Multi-speaker, Cross-lingual voice cloning을 위해 Speaker Embedding을 사용</li>\n<li>인코더 아웃풋에 Concatenate하여 스펙트로그램 생성시에 반영되도록 함</li>\n</ul>\n</li>\n<li>\n<p><strong>Adversarial Speaker Classifier</strong></p>\n<ul>\n<li>이상적으로 Voice cloning을 위해서는 텍스트(언어)로부터 화자의 정보가 반영되면 안됨</li>\n<li>Speaker Classifier와 나머지 모델(인코더, 디코더)은 forward에서는 독립적이지만,  backpropagation을 진행할 때, 두 loss (L2 of predict spectrogram, cross entropy of predicted speaker ID)가 인코더 파라미터 업데이트에 영향을 미침</li>\n<li>Gradient reversal layer를 통해 인코더가 speaker에 대한 정보를 반영 못하도록 학습</li>\n</ul>\n</li>\n</ul>\n<h3>Baselines: Shared, Separate &#x26; Single</h3>\n<p>※ GEN과 다른점만 비교</p>\n<ul>\n<li><strong>Single (SGL)</strong>\n<ul>\n<li>Monolingual Vanilla Tacotron 2 (Code-switching에 사용 X)</li>\n</ul>\n</li>\n<li><strong>Shared (SHA)</strong>\n<ul>\n<li>GEN과 다르게 Tacotron 2의 인코더 사용 (Multilingual)</li>\n</ul>\n</li>\n<li><strong>Separate (SEP)</strong>\n<ul>\n<li>GEN과 같이 Multiple convolution layer를 사용</li>\n<li>Parameter generation 사용 X</li>\n<li>Adversarial speaker classifier 사용 X</li>\n</ul>\n</li>\n</ul>\n<h2>Dataset</h2>\n<p>10개의 언어로 구성된 CSS10과 Common Voice 데이터셋의 일부를 사용\nCode-switching을 학습하기 위해 multi-speaker 데이터가 필요 (언어와 화자 일치를 없애기 위해)</p>\n<img src=\"https://user-images.githubusercontent.com/42150335/95888064-9680c900-0dbb-11eb-9967-a30b21dbfa80.png\" width=\"600\">  \n<h2>Experiment</h2>\n<p>SGL, SHA, SEP, GEN을 비교했을 때 GEN이 거의 모든 결과에서 우수한 성능을 보임</p>\n<p><img src=\"https://user-images.githubusercontent.com/42150335/95888889-a816a080-0dbc-11eb-81a9-9a2d036f2def.png\" alt=\"image\"></p>\n<img src=\"https://user-images.githubusercontent.com/42150335/95888982-cbd9e680-0dbc-11eb-984f-1524ab3a9f38.png\" width=\"400\">\n<h2>Conclusion</h2>\n<ul>\n<li>본 논문에서 제안하는 모델은 Multilingual Voice cloning, Code-switching에 우수한 성능을 보임</li>\n<li>추후 연구로 어텐션 모듈을 수정하는 것을 생각중이라고 함</li>\n</ul>","htmlAst":{"type":"root","children":[{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"One Model, Many Languages: Meta-learning for Multilingual Text-to-Speech"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Tomáš Nekvinda, Ondřej Dušek"},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\nCharles University"},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\nINTERSPEECH, 2020"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Reference"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://arxiv.org/abs/2008.00768"},"children":[{"type":"text","value":"ArXiv"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://github.com/Tomiinek/Multilingual_Text_to_Speech"},"children":[{"type":"text","value":"Source Code"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://tomiinek.github.io/multilingual_speech_samples/"},"children":[{"type":"text","value":"Demo Webpage"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://arxiv.org/abs/1703.10135"},"children":[{"type":"text","value":"Tacotron"}]},{"type":"text","value":", "},{"type":"element","tagName":"a","properties":{"href":"https://arxiv.org/abs/1712.05884"},"children":[{"type":"text","value":"Tacotron2"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://arxiv.org/pdf/1710.08969.pdf"},"children":[{"type":"text","value":"DC-TTS"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://github.com/Kyubyong/css10"},"children":[{"type":"text","value":"CSS 10 Dataset"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://commonvoice.mozilla.org/en/datasets"},"children":[{"type":"text","value":"Common Voice Dataset"}]}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Summary"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Multilingual Speech Synthesis"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Meta-learning"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Voice Cloning : Speech in multiple languages with the same voice"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Code switching : Speak two (or more) languages with a single utterance."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Tacotron2 base architecture"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Tacotron"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://img1.daumcdn.net/thumb/R1280x0/?scode=mtistory2&fname=https%3A%2F%2Fblog.kakaocdn.net%2Fdn%2FwnGyQ%2FbtqDblNauXg%2FJsSXkwgQY1yc3lIHtdgIP0%2Fimg.png","width":700},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"딥러닝 기반 음성합성의 대표적인 모델"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Attention + Sequence-to-Sequence의 TTS 버전"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Griffin-Lim Vocoder 사용 (빠르지만 성능은 좋지 못함)"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Tacotron2"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/42150335/94840259-1cfbe900-0453-11eb-8803-cac2ea30b425.png","width":470},"children":[]},{"type":"text","value":"  \n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Mel-Prediction Network : Attention based Sequence-to-Sequence Network\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"인코더에서 Bi-directional LSTM 적용"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Location Sensitive Attention 적용 (음성 Alignment에 강한 어텐션)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"인코더, 디코더에 Convolution Layer 적용"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Stop Token 사용"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Vocoder : WaveNet\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"장점 : 상당히 고품질의 음성으로 변환"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"단점 : 엄청나게 느림"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Model Architecture"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://github.com/Tomiinek/Multilingual_Text_to_Speech/raw/master/_img/generated.png","width":800},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Tacotron2 기반의 모델들로 실험 진행"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"WaveRNN Vocoder 사용"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"This Paper`s Model: Generated (GEN)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Parameter Generation Convolutional Encoder"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"이 논문에서는 Fully convolutional encoder를 사용 (from DC-TTS)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Cross-lingual knowledge-sharing을 가능하게 하기 위해 인코더 컨볼루션 레이어의 파라미터를 생성하여 사용"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"입력되는 Language ID에 따라 Fully Connected 레이어를 통해 다른 다른 파라미터를 생성"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Speaker Embedding"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Multi-speaker, Cross-lingual voice cloning을 위해 Speaker Embedding을 사용"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"인코더 아웃풋에 Concatenate하여 스펙트로그램 생성시에 반영되도록 함"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Adversarial Speaker Classifier"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"이상적으로 Voice cloning을 위해서는 텍스트(언어)로부터 화자의 정보가 반영되면 안됨"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Speaker Classifier와 나머지 모델(인코더, 디코더)은 forward에서는 독립적이지만,  backpropagation을 진행할 때, 두 loss (L2 of predict spectrogram, cross entropy of predicted speaker ID)가 인코더 파라미터 업데이트에 영향을 미침"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Gradient reversal layer를 통해 인코더가 speaker에 대한 정보를 반영 못하도록 학습"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Baselines: Shared, Separate & Single"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"※ GEN과 다른점만 비교"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Single (SGL)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Monolingual Vanilla Tacotron 2 (Code-switching에 사용 X)"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Shared (SHA)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"GEN과 다르게 Tacotron 2의 인코더 사용 (Multilingual)"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Separate (SEP)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"GEN과 같이 Multiple convolution layer를 사용"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Parameter generation 사용 X"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Adversarial speaker classifier 사용 X"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Dataset"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"10개의 언어로 구성된 CSS10과 Common Voice 데이터셋의 일부를 사용\nCode-switching을 학습하기 위해 multi-speaker 데이터가 필요 (언어와 화자 일치를 없애기 위해)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/42150335/95888064-9680c900-0dbb-11eb-9967-a30b21dbfa80.png","width":600},"children":[]},{"type":"text","value":"  \n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Experiment"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"SGL, SHA, SEP, GEN을 비교했을 때 GEN이 거의 모든 결과에서 우수한 성능을 보임"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/42150335/95888889-a816a080-0dbc-11eb-81a9-9a2d036f2def.png","alt":"image"},"children":[]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/42150335/95888982-cbd9e680-0dbc-11eb-984f-1524ab3a9f38.png","width":400},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Conclusion"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"본 논문에서 제안하는 모델은 Multilingual Voice cloning, Code-switching에 우수한 성능을 보임"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"추후 연구로 어텐션 모듈을 수정하는 것을 생각중이라고 함"}]},{"type":"text","value":"\n"}]}],"data":{"quirksMode":false}},"excerpt":"One Model, Many Languages: Meta-learning for Multilingual Text-to-Speech Tomáš Nekvinda, Ondřej Dušek Charles University INTERSPEECH, 202…","fields":{"readingTime":{"text":"4 min read"}},"frontmatter":{"title":"Sooftware Speech - One Model, Many Languages: Meta-learning for Multilingual Text-to-Speech Paper Review","userDate":"14 October 2020","date":"2020-10-14T10:00:00.000Z","tags":["speech","tts","paper"],"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#f8f8f8","images":{"fallback":{"src":"/static/deca33714347f50cf9f1b33b2db865ef/7189c/multilingual-tts.png","srcSet":"/static/deca33714347f50cf9f1b33b2db865ef/cefb5/multilingual-tts.png 750w,\n/static/deca33714347f50cf9f1b33b2db865ef/c1615/multilingual-tts.png 1080w,\n/static/deca33714347f50cf9f1b33b2db865ef/7189c/multilingual-tts.png 1280w","sizes":"100vw"},"sources":[{"srcSet":"/static/deca33714347f50cf9f1b33b2db865ef/da87f/multilingual-tts.webp 750w,\n/static/deca33714347f50cf9f1b33b2db865ef/bd382/multilingual-tts.webp 1080w,\n/static/deca33714347f50cf9f1b33b2db865ef/2dc0b/multilingual-tts.webp 1280w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.328125}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"children":[{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#282838","images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/248f9/soohwan.png 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/fd435/soohwan.png 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png 120w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/e7f45/soohwan.webp 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/589ec/soohwan.webp 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/71a38/soohwan.webp 120w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6833333333333333}}]}}]}},"relatedPosts":{"totalCount":20,"edges":[{"node":{"id":"fa9e8cbb-841a-516f-9df6-4be0336b56b0","excerpt":"한국어 Tacotron2 이번 포스팅에서는 Tacotron2 아키텍처로 한국어 TTS 시스템을 만드는 방법에 대해 다루겠습니다. Tacotron2 Tacotron2는 17년 12월 구글이 NATURAL TTS SYNTHESIS BY…","frontmatter":{"title":"Sooftware Speech - 한국어 Tacotron2","date":"2021-10-10T10:00:00.000Z"},"fields":{"readingTime":{"text":"11 min read"},"slug":"/korean_tacotron2/"}}},{"node":{"id":"43c23529-71b1-5d60-8883-a45fbcd55ebd","excerpt":"Textless NLP: Generating expressive speech from raw audio paper / code / pre-train model / blog Name: Generative Spoken Language Model (GSLM…","frontmatter":{"title":"Sooftware NLP - Textless NLP","date":"2021-09-19T10:00:00.000Z"},"fields":{"readingTime":{"text":"4 min read"},"slug":"/Textledd NLP: Generating expressive speech from raw audio/"}}},{"node":{"id":"83c6b4fa-d71e-51d8-90bb-b58bfffc01d0","excerpt":"Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition Yu Zhang et al., 2020 Google Research, Brain Team Reference…","frontmatter":{"title":"Sooftware Speech - Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition Paper Review","date":"2021-03-17T10:00:00.000Z"},"fields":{"readingTime":{"text":"3 min read"},"slug":"/Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition/"}}},{"node":{"id":"894a24ca-fff9-5884-b6da-1c97f2ece7bc","excerpt":"PORORO Text-To-Speech (TTS) 얼마전에 저희 팀에서 공개한 PORORO: Platform Of neuRal mOdels for natuRal language prOcessing 라이브러리에 제가 공들여만든 TTS…","frontmatter":{"title":"PORORO Text-To-Speech (TTS)","date":"2021-02-16T10:00:00.000Z"},"fields":{"readingTime":{"text":"1 min read"},"slug":"/pororo-tts/"}}},{"node":{"id":"b039977c-cecb-50f4-a0dd-fd008731bc99","excerpt":"EMNLP Paper Review: Speech Adaptive Feature Selection for End-to-End Speech Translation (Biao Zhang et al) Incremental Text-to-Speech…","frontmatter":{"title":"Sooftware Speech - EMNLP Paper Review: Speech","date":"2020-12-08T10:00:00.000Z"},"fields":{"readingTime":{"text":"4 min read"},"slug":"/2020 EMNLP Speech Paper Review/"}}}]}},"pageContext":{"slug":"/one-model-many-langs/","prev":{"excerpt":"RoBERTa paper / code Abstract BERT를 제대로 학습시키는 법을 제안 BERT는 엄청난 모델이지만, Original BERT 논문에서 하이퍼파라미터에 대한 실험이 제대로 진행되지 않음 BERT…","frontmatter":{"title":"Sooftware NLP - RoBERTa Paper Review","tags":["nlp","paper"],"date":"2020-10-11T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAASCAYAAABb0P4QAAAACXBIWXMAAAsTAAALEwEAmpwYAAADkUlEQVQ4y62Uy28aVxTG5w9tt5Uqtdv2D2irLqIqqbJIu+hLSjdeVUljx8aY18wwMMM8mGGAwWDAQIIB85gH+AG/iuukirJtj3R0zz3fdz6du7iflCQJ/V6PxWLBPna73X9KaX49I1ss0h0N+T9CSu63DE7PmNlVotst4SomDCNWYUgY7uuEMErEuRL1mjBcv8Pec9aswpjN5gapEnRJPXpM+tfnvNBqHOV1XistUqU+R3Kdw4LO37kKh4UKx6rNy6whekeyz4nWF+eRrPMiY1O2mkiGVSXTUEi7ClkrR6Ujk7UH6G2QvQsU74SzSoZ05ZhT4zUpPYXeypJ3mxQbW2SvjewekTYtDLePZLotjrU2Z0aPjHVJutJB9uaUghty1QmpcotTo8tJqcWR2iRtdDkzL8hXJ4JTcKec6i1OK0Mq7gWSXeug1NeUgg2l4BatuUGtR+JutO+wumD1wN5nH4z2jlJwhxZsKDYStOZaCO81jGr7QTDvrZC9BQVvSc6Zofghf2UDnv5+yKNnBzz97RU//vKCn/5Mcaj2BF5wF8i1UNSqv0L2Y4xq50FQ9iO0ZiK2KjZikTnnmr8yTQ6Oq7zMt4XQodIla0/R23diq/12xXqM1ojeCbaRrFqHjLMk68zJOgvS5oycM0f1I+RahOLHKH6CXIvFUM5dkbGvBfeBE5Jzl+TcCMPtIDlOmU7hS7rFrzmXv2JW+RzN8pADsMwMo8InNEtPBPZW+4x68QmX6he0S99TaIBVOWasfkpN/wPdGyGZlol+9hg9+wxb/plG8Qfy5jm5OmimSTX7Lbp6gFV4RvnkO0ryAa78BLP4nFztjqJews1+g6q+QrN7SMM3U8reALP+BrP+Ft0bUm1c4vht7PpA3O3mW4FV/BF2cIVRG1Lxh1T9Dk5ziOGPMLw+/cEIaXd3y9jUmXgOk5rDUFO4T2LxL28XM7qFDLNWk6nnMNAUlhfnDEoqk7onOJvZhG4+w7J/8fCXN1FMT1HolcpclssMFJXF1YQdsByNOD9N0zcthuUy3XyegW3TVxTeOA7bPWc4JEilmDSDB8G95dxvt2x3O25ub4mThDiO2dvaer0WJnF3fy/w97nebEjWa8Hbc8IoEn0h+LH9LJdLptOp8Mf9wO4d8cOIoojr62uurq6Yz+fc3Nz866XSh+a4j73Z+r5Pq9ViPL566H/EGY/H1Ot1giCg2+2K17wX/Ad7CQ6hmGCGbAAAAABJRU5ErkJggg=="},"images":{"fallback":{"src":"/static/26b5f2f1a6d4d031c6cf36eac285256a/1be58/roberta.png","srcSet":"/static/26b5f2f1a6d4d031c6cf36eac285256a/5dae1/roberta.png 750w,\n/static/26b5f2f1a6d4d031c6cf36eac285256a/35cd7/roberta.png 1080w,\n/static/26b5f2f1a6d4d031c6cf36eac285256a/1be58/roberta.png 1134w","sizes":"100vw"},"sources":[{"srcSet":"/static/26b5f2f1a6d4d031c6cf36eac285256a/76436/roberta.webp 750w,\n/static/26b5f2f1a6d4d031c6cf36eac285256a/ce7b4/roberta.webp 1080w,\n/static/26b5f2f1a6d4d031c6cf36eac285256a/03f03/roberta.webp 1134w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.8835978835978836}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"4 min read"},"layout":"","slug":"/roberta/"}},"next":{"excerpt":"Mac iTerm2 + ZSH 세팅 개발환경에서 가장 중요한 소프트웨어 중 하나는 쉘입니다. 어떤 OS에서 작업하냐에 따라서 어떤 쉘을 쓰는지 등이 달라질텐데요, Mac OS에서 가장 많이 사용되는 iTerm2와 ZSH…","frontmatter":{"title":"Mac iTerm2 + ZSH 세팅","tags":["software","environment"],"date":"2020-12-02T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAYAAAB/Ca1DAAAACXBIWXMAAAsTAAALEwEAmpwYAAABr0lEQVQoz62QaYvTYBSF80fcwKW22bcmTSZJk2Zr0rSjDswI/hxlmO6L/eL810catQjOgIgfHu7lcs55D69wvz+w2+45frtntT9yu1ix2cxZbdbMV1t28wXL5Zr1dsfh65Hd/sB2t+duvuDzl1vu5ksWy3XLaRfGFyGO4+ElGVrfR1R1ZKVLV5J4IxponR6iqNKTdd72FDonujKvOxLPXrzi6fOXPPnJaRdu4oTAD4nqKVE5JS5KqjpiVIwI0gmzMCJNcvLxhGGSk2QlUZwSRAmqbiOrJopmnafwKYrw+gPC5hL/FFiWNE1IWo6Iihk3w5gkCJm9/8DV9Ufq5pKybqgmU7yLCM3oo5sOpu22DwhXSfqjYTUhyCqGWUFVJ8RpjBOk5K5P6AUko5w0H7fNsqJqp2kPsOwBhuViO34bLlxXE7K0IJ++I61njJsZxXiEHwSYbkRou3iWi9m2GLRG2/Fac/f0t5LW0hU1RFlHMFUTWTGQjT6ybqPoNppuoOoWquFgahaKZiPKRmv4HUkx/kDo/RJI2ll4vsnaeX/I/NBdeEj4GI8F/3Pg3/DfA78DuMd7VSe/TtsAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/0804d911b246a6b0f242a087c53d6920/ffc42/iterm_zsh.png","srcSet":"/static/0804d911b246a6b0f242a087c53d6920/ffc42/iterm_zsh.png 673w","sizes":"100vw"},"sources":[{"srcSet":"/static/0804d911b246a6b0f242a087c53d6920/20eb0/iterm_zsh.webp 673w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5557206537890045}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"7 min read"},"layout":"","slug":"/iterm_zsh/"}},"primaryTag":"speech"}},
    "staticQueryHashes": ["3170763342","3229353822"]}