","htmlAst":{"type":"root","children":[{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"RoBERTa"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://arxiv.org/abs/1907.11692"},"children":[{"type":"text","value":"paper"}]},{"type":"text","value":" / "},{"type":"element","tagName":"a","properties":{"href":"https://github.com/pytorch/fairseq/tree/master/examples/roberta"},"children":[{"type":"text","value":"code"}]}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Abstract"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"BERT를 제대로 학습시키는 법을 제안"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"BERT는 엄청난 모델이지만, Original BERT 논문에서 하이퍼파라미터에 대한 실험이 제대로 진행되지 않음"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"BERT를 더 좋은 성능을 내게 하기 위한 replication study."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Background (BERT)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://baekyeongmin.github.io/images/RoBERTa/bert.png","width":700},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"학습 1단계) 많은 양의 unlabeled corpus를 이용한 pre-train"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"학습 2단계) 특정 도메인의 태스크에 집중하여 학습하는 fine-tuning"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"“Attention Is All You Need”에서 제안된 transformer의 encoder를 사용"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Main Idea"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Dynamic Masking"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"기존의 BERT는 학습 전에 데이터에 무작위로 mask를 씌움."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"매 학습 단계에서 똑같은 mask를 보게 됨. (static masking)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"이를 같은 문장을 10번 복사한 뒤 서로 다른 마스크를 씌움으로써 해결하려고 했지만, 이는 크기가 큰 데이터에 대해서 비효율적임."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"RoBERTa는 매 에폭마다 mask를 새로 씌우는 dynamic masking을 사용"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"결과: static masking보다 좋은 성능을 보여줌"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Input Format / Next Sentence Prediction"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"기존 BERT에서는 Next Sentence Prediction(NSP)이라는 과정이 있었음\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"ol","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"두 개의 문장을 이어 붙인다"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"ol","properties":{"start":2},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"두 문장이 문맥상으로 연결된 문장인지를 분류하는 binary classification을 수행"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"RoBERTa는 NSP에 의문을 제기하고, Masked Language Modeling (MLM)만으로 pre-training을 수행"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"NSP를 없앰으로써 두 문장을 이어 붙인 형태의 인풋 형태를 사용할 필요가 없어졌고, RoBERTa는 최대 토큰 수를 넘어가지 않는 선에서 문장을 최대한 이어 붙여서 input을 만들 수 있었음"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"[CLS]가나다라마바사아자카타파하[SEP]오늘 날씨가 좋은걸?[SEP]튜닙은 정말 대단한 회사야!.....[SEP]오늘은 빨리 퇴근하고 싶다."}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"BERT는 짧은 인풋들을 이어붙이는 경우도 있었지만, RoBERTa는 모든 인풋 토큰 수를 최대길이에 가깝게 사용할 수 있었음 (학습 효율면에서 좋음)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"결과: NSP를 없앤 BERT가 기존의 BERT보다 더 나은 성능을 보임."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Batch Size"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"(batch size X step 수)가 일정하게 유지되는 선에서 배치사이즈에 따른 성능 실험\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"batch size가 256에 step이 1M이라면 batch size가 2K에 step이 125K가 되도록. (둘의 곱이 같도록 유지)"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"배치가 클수록 성능이 좋아지는 경향을 보임."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Data"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"데이터가 많을수록 BERT의 성능이 좋아지는 경향을 이전 연구들에서 관찰됐었음."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"기존 BERT는 16GB로 학습했는데, RoBERT는 160GB의 데이터로 학습하였음."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"당연하게도 160GB로 학습한 RoBERTa가 더 좋은 성능을 기록함"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"학습 시간을 길게 하면 할수록 더 좋은 성능을 보였다고함."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Conclusion"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"160GB의 데이터"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Dynamic Masking"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"NSP 밴"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"최대한 문장을 구겨넣은 인풋 ([SEP]으로 분리)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"큰 배치사이즈"}]},{"type":"text","value":"\n"}]}],"data":{"quirksMode":false}},"excerpt":"RoBERTa paper / code Abstract BERT를 제대로 학습시키는 법을 제안 BERT는 엄청난 모델이지만, Original BERT 논문에서 하이퍼파라미터에 대한 실험이 제대로 진행되지 않음 BERT…","fields":{"readingTime":{"text":"4 min read"}},"frontmatter":{"title":"Sooftware NLP - RoBERTa Paper Review","userDate":"11 October 2020","date":"2020-10-11T10:00:00.000Z","tags":["nlp","paper"],"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#f8f8f8","images":{"fallback":{"src":"/static/26b5f2f1a6d4d031c6cf36eac285256a/1be58/roberta.png","srcSet":"/static/26b5f2f1a6d4d031c6cf36eac285256a/5dae1/roberta.png 750w,\n/static/26b5f2f1a6d4d031c6cf36eac285256a/35cd7/roberta.png 1080w,\n/static/26b5f2f1a6d4d031c6cf36eac285256a/1be58/roberta.png 1134w","sizes":"100vw"},"sources":[{"srcSet":"/static/26b5f2f1a6d4d031c6cf36eac285256a/76436/roberta.webp 750w,\n/static/26b5f2f1a6d4d031c6cf36eac285256a/ce7b4/roberta.webp 1080w,\n/static/26b5f2f1a6d4d031c6cf36eac285256a/03f03/roberta.webp 1134w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.8835978835978836}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"children":[{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#282838","images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/248f9/soohwan.png 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/fd435/soohwan.png 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png 120w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/e7f45/soohwan.webp 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/589ec/soohwan.webp 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/71a38/soohwan.webp 120w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6833333333333333}}]}}]}},"relatedPosts":{"totalCount":50,"edges":[{"node":{"id":"6529d72e-80e7-5d61-a672-d66276a3f641","excerpt":"MoE(Mixture of Experts) 기초부터 DeepSeek 혁신까지 작년 이맘때쯤 DeepSeek-V3가 저비용으로 엄청난 성능을 보이면서 화제가 되었습니다. 그 핵심 기술인 MoE(Mixture of Experts…","frontmatter":{"title":"MoE(Mixture of Experts) 기초부터 DeepSeek 혁신까지","date":"2026-01-15T01:11:55.000Z"},"fields":{"readingTime":{"text":"8 min read"},"slug":"/developed-moe/"}}},{"node":{"id":"8b49d3ef-4ce3-568c-9d71-240ff17fc3e0","excerpt":"BERT는 사실 Diffusion 모델이였다?! 최근 굉장히 흥미로운 글을 읽게되어 공유합니다. 원문 : link BERT와 Diffusion이 같은 방식이다?! NLP 연구자들에게 BERT는 너무 익숙한 모델입니다. 201…","frontmatter":{"title":"BERT는 사실 Diffusion 모델이였다?!","date":"2025-10-21T12:00:00.000Z"},"fields":{"readingTime":{"text":"8 min read"},"slug":"/bert_diffusion/"}}},{"node":{"id":"7c7e1676-ea84-58de-970c-ddec9aa10660","excerpt":"RLHF는 수다쟁이를 만든다?! (Does RLHF Breed Verbose Chatterboxes?!) RLHF(Reinforcement Learning from Human Feedback)는 OpenAI의 ChatGPT…","frontmatter":{"title":"RLHF는 수다쟁이를 만든다?! (Does RLHF Breed Verbose Chatterboxes?!)","date":"2024-03-13T01:11:55.000Z"},"fields":{"readingTime":{"text":"7 min read"},"slug":"/rlhf-vervosity/"}}},{"node":{"id":"2b9d3e22-1796-5fab-878d-5941d2e76e9d","excerpt":"LLM Paper Abstract - 2023.12 LLM…","frontmatter":{"title":"LLM Paper Abstract - 2023.12","date":"2024-01-05T10:00:00.000Z"},"fields":{"readingTime":{"text":"5 min read"},"slug":"/llm-abs-202312/"}}},{"node":{"id":"f43fa33b-917b-5c00-8462-937d99592ad7","excerpt":"What it MoE? (Mixture of Experts) 현존 최강 LLM인 GPT-4에서 “MoE (Mixture of Experts)” 방식을 채택하여 사용하고 있다고 알려졌는데요, 최근 AI계의 뜨거운 감자 Mistral AI…","frontmatter":{"title":"What is MoE? (Mixture of Experts)","date":"2023-12-22T01:11:55.000Z"},"fields":{"readingTime":{"text":"9 min read"},"slug":"/moe/"}}}]}},"pageContext":{"slug":"/roberta/","prev":{"excerpt":"Below is just about everything you’ll need to style in the theme. Check the source code to see the many embedded elements within paragraphs…","frontmatter":{"title":"Sooftware NLP - Electra Paper Review","tags":["nlp","paper"],"date":"2020-09-23T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAYAAAAywQxIAAAACXBIWXMAAAsTAAALEwEAmpwYAAABY0lEQVQoz02SacuCUBCF/f+/qO9BGFnaSqVp0UIl7aWtnpdn4MIrXObO3HPObHpVVen7/er3++n1eunz+ej5fOrxeKgsSx2PRw2HQ/N5x7/dbrper7pcLmbhvN9voeVJ0m6302az0Ww202KxUJZlut/vJgwBkeVyqfl8bj6fK4I4CeHweSgnSWKiBBGG2Gg0VKvV1O12dTqdtF6vtd1uNRgM5Pu+6vW6RqOR4eM41nQ6tUo9sqxWKwMj5IQRSNPUKj6fz0akGtqleizFgEMYUbSsZQi0QiUAD4eDtUtbzI47mP1+bxhmyXyLolCe51YEHGsZAR4hAeCOCAIuERjeqQxBtxAOPniX2KNU5sJgmddkMrF7v983Szu8s7D/8V6vp/F4rDAMzcJlJB4OIAAEW62W2u22CWHdaTabiqLI3judjvngiXEnzg5MMAgCE4UImKxYwCzD/VbM0B1iWJbJUvEZxR8WlJe1e/a3LQAAAABJRU5ErkJggg=="},"images":{"fallback":{"src":"/static/7ccdb9951c9d362c8a3548e8c2a87231/59ccb/electra.png","srcSet":"/static/7ccdb9951c9d362c8a3548e8c2a87231/c68af/electra.png 750w,\n/static/7ccdb9951c9d362c8a3548e8c2a87231/87f65/electra.png 1080w,\n/static/7ccdb9951c9d362c8a3548e8c2a87231/d464a/electra.png 1366w,\n/static/7ccdb9951c9d362c8a3548e8c2a87231/59ccb/electra.png 1920w","sizes":"100vw"},"sources":[{"srcSet":"/static/7ccdb9951c9d362c8a3548e8c2a87231/9fb02/electra.webp 750w,\n/static/7ccdb9951c9d362c8a3548e8c2a87231/cd76f/electra.webp 1080w,\n/static/7ccdb9951c9d362c8a3548e8c2a87231/b7397/electra.webp 1366w,\n/static/7ccdb9951c9d362c8a3548e8c2a87231/507b8/electra.webp 1920w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.4479166666666667}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"4 min read"},"layout":"","slug":"/electra/"}},"next":{"excerpt":"One Model, Many Languages: Meta-learning for Multilingual Text-to-Speech Tomáš Nekvinda, Ondřej Dušek Charles University INTERSPEECH, 202…","frontmatter":{"title":"Sooftware Speech - One Model, Many Languages: Meta-learning for Multilingual Text-to-Speech Paper Review","tags":["speech","tts","paper"],"date":"2020-10-14T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAYAAAAIy204AAAACXBIWXMAAAsTAAALEwEAmpwYAAABuklEQVQozyXObU/aUACGYf//55nM/2BcWOMW45iisPqCGoTxOsH2lNa+QM8ptBQOvc8S/PjkTq48R24gaI+feRw90Xnvkm4Ui8mE4LZJ0LJJ2k9Ed/eHvZn7GGOqOFZMxgnjUcR0uiDL1ux220M7KrYFw2DMy1uHv/MBbiRI+gOylo207yhfOij7Dtn8Q/HusFwmrLI93Y7CbglcZ08w/yBT6hPMixxZKILlB3KjKpUr8ihi6wpK4VF6c0ohKF2ByXPSVCL8GC+IEH7IuyMIk4hyV36C6SrlZtTi1+sVV4Nb/oUzosc24ekZSc1CN24Jv9WIT8/AcVnLBf7wmXjaI37rEvqC3MkwRQUHcC2rtvNS2ZOH6n7WrsRyXi06r8jzn6iLS7Y3TbKLS1LrHO147NM5yeUX5PUJ2fUJO+UR1j1yR34+fJg90Rg2aY5tfvcbCOljHJeNdW7SmmVW1g+zrFmm+G4Z43hU0idvfDXL+rGR9WNjypjVQ2py9wBytCrX+s2Z6l6/p2OZ6FJvtYoi7Q5H2ptMtAoCPRsM9Kzf10UqtdnvtAqFHnWfdRq4OpMLnUwjXWYbbYzR/wGH2vvklH7jTwAAAABJRU5ErkJggg=="},"images":{"fallback":{"src":"/static/deca33714347f50cf9f1b33b2db865ef/7189c/multilingual-tts.png","srcSet":"/static/deca33714347f50cf9f1b33b2db865ef/cefb5/multilingual-tts.png 750w,\n/static/deca33714347f50cf9f1b33b2db865ef/c1615/multilingual-tts.png 1080w,\n/static/deca33714347f50cf9f1b33b2db865ef/7189c/multilingual-tts.png 1280w","sizes":"100vw"},"sources":[{"srcSet":"/static/deca33714347f50cf9f1b33b2db865ef/da87f/multilingual-tts.webp 750w,\n/static/deca33714347f50cf9f1b33b2db865ef/bd382/multilingual-tts.webp 1080w,\n/static/deca33714347f50cf9f1b33b2db865ef/2dc0b/multilingual-tts.webp 1280w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.328125}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"4 min read"},"layout":"","slug":"/one-model-many-langs/"}},"primaryTag":"nlp"}},
"staticQueryHashes": ["3170763342","3229353822"]}
![](\"https://baekyeongmin.github.io/images/RoBERTa/bert.png\")
\n