{
    "componentChunkName": "component---src-templates-post-tsx",
    "path": "/longformer/",
    "result": {"data":{"logo":null,"markdownRemark":{"html":"<h1>Longformer: The Long-Document Transformer</h1>\n<ul>\n<li><a href=\"https://arxiv.org/abs/2004.05150\">Paper</a> <a href=\"https://github.com/allenai/longformer\">Code</a></li>\n<li>Iz Beltagy et al.</li>\n</ul>\n<h2>Introduction</h2>\n<ul>\n<li>트랜스포머는 긴 시퀀스는 처리하지 못한다는 한계를 가지고 있음</li>\n<li>이유는 시퀀스 길이에 O(n^2)하게 늘어나는 높은 복잡도 때문</li>\n<li>본 논문은 Attention 이루어지는 복잡도를 낮추는 방법을 제안 (O(n))</li>\n<li>BERT는 512 토큰을 limit으로 가지는데, 본 논문 모델은 4096 토큰까지 가능</li>\n<li><code class=\"language-text\">text8</code>, <code class=\"language-text\">enwik8</code>, <code class=\"language-text\">Wiki-Hop</code>, <code class=\"language-text\">TriviaQA</code>에서 State-Of-The-Art 달성</li>\n</ul>\n<h2>Attention Method</h2>\n<img src=\"https://haebinshin.github.io/public/img/longformer/figure2.png\">  \n<p>본 논문에서는 위 그림과 같은 3가지 어텐션 방식을 제안</p>\n<h3>Sliding window Attention</h3>\n<ul>\n<li>크기가 w인 sliding window 내에서만 attention을 수행하는 방법</li>\n<li>이 방법은 텍스트 길이 n에 대해 O(n x w)의 복잡도를 가짐</li>\n<li>이러한 방식은 레이어가 깊어짐에 따라 receptive field가 넓어지는 CNN과 유사함</li>\n</ul>\n<img src=\"https://haebinshin.github.io/public/img/longformer/receptive_field.png\">  \n<ul>\n<li>예) window size가 2일 때, 레이어가 쌓일수록 w만큼 receptive field가 넓어짐.</li>\n</ul>\n<img src=\"https://haebinshin.github.io/public/img/longformer/text_sliding_window_receptive_field.jpg\">\n<ul>\n<li>l x w의 receptive field size를 가지게 됨.</li>\n<li>각 레이어마다 w의 크기를 다르게 주는 방법이 도움이 될 수도 있음</li>\n</ul>\n<h3>Dilated Sliding Window</h3>\n<ul>\n<li>Sliding window attention보다도 receptive field를 더 넓히기 위해 고안된 방법</li>\n<li>Dilated Convolution에서 착안</li>\n</ul>\n<img src=\"https://haebinshin.github.io/public/img/longformer/dilation_convolution.gif\">  \n<ul>\n<li>dilated 값을 줘서 토큰을 d만큼 건너뛰면서 어텐션하도록 하는 방법</li>\n<li>예) window size가 2이고 dilation size가 2일 때, 아래 그림과 같이 w x d만큼 receptive field가 넓어짐</li>\n</ul>\n<img src=\"https://haebinshin.github.io/public/img/longformer/text_dilated_sliding_window_receptive_field.jpg\">\n<ul>\n<li>l x d x w의 receptive field size를 가지게 됨.</li>\n</ul>\n<h3>Global Attention</h3>\n<ul>\n<li>BERT의 [CLS] 토큰 같은 경우는 전체 컨텍스트를 바라봐야하는데, 위의 2가지 방법만으로는 Finetuning하는 태스크에서는 부족한 부분이 있을 수 있음</li>\n<li>따라서 스페셜 토큰 몇 개에 대해서는 global attention을 수행하도록 함.</li>\n<li>전체 토큰 수에 비해서는 스페셜 토큰은 매우 적기 때문에 복잡도는 여전히 O(n)</li>\n</ul>\n<h3>Linear Projections for Global Attention</h3>\n<ul>\n<li>보통의 트랜스포머의 어텐션은 Q, K, V로 이루어 지는데, sliding window 기반 어텐션과 global 어텐션을 위해 sliding Q, K, V와 global Q, K, V 두 세트로 나눠서 어텐션을 계산하도록 구현</li>\n</ul>\n<h2>Experiments</h2>\n<p>2가지 방식으로 평가를 진행.</p>\n<h3>Autoregressive Language Modeling</h3>\n<ul>\n<li>모델 자체의 임베딩 평가를 위함</li>\n<li>character/token 단위의 language modeling을 수행.</li>\n<li><code class=\"language-text\">text8</code>, <code class=\"language-text\">enwik8</code> 데이터셋에서 SOTA를 달성</li>\n<li>본 태스크는 dilated sliding window attention 사용</li>\n</ul>\n<img src=\"https://haebinshin.github.io/public/img/longformer/table_2_3.png\">\n<h3>Pre-training and Fine-tuning</h3>\n<ul>\n<li>RoBERTa 체크포인트로부터 시작해서 학습</li>\n<li>sliding window attention를 사용</li>\n<li>각 태스크에 따라 스페셜 토큰을 지정하여 global attention을 사용</li>\n<li><code class=\"language-text\">WikiHop</code>과 <code class=\"language-text\">TriviaQA</code>에서 SOTA 달성</li>\n</ul>\n<img src=\"https://haebinshin.github.io/public/img/longformer/table8.png\">","htmlAst":{"type":"root","children":[{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"Longformer: The Long-Document Transformer"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://arxiv.org/abs/2004.05150"},"children":[{"type":"text","value":"Paper"}]},{"type":"text","value":" "},{"type":"element","tagName":"a","properties":{"href":"https://github.com/allenai/longformer"},"children":[{"type":"text","value":"Code"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Iz Beltagy et al."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Introduction"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"트랜스포머는 긴 시퀀스는 처리하지 못한다는 한계를 가지고 있음"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"이유는 시퀀스 길이에 O(n^2)하게 늘어나는 높은 복잡도 때문"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"본 논문은 Attention 이루어지는 복잡도를 낮추는 방법을 제안 (O(n))"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"BERT는 512 토큰을 limit으로 가지는데, 본 논문 모델은 4096 토큰까지 가능"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"text8"}]},{"type":"text","value":", "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"enwik8"}]},{"type":"text","value":", "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Wiki-Hop"}]},{"type":"text","value":", "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"TriviaQA"}]},{"type":"text","value":"에서 State-Of-The-Art 달성"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Attention Method"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://haebinshin.github.io/public/img/longformer/figure2.png"},"children":[]},{"type":"text","value":"  \n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"본 논문에서는 위 그림과 같은 3가지 어텐션 방식을 제안"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Sliding window Attention"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"크기가 w인 sliding window 내에서만 attention을 수행하는 방법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"이 방법은 텍스트 길이 n에 대해 O(n x w)의 복잡도를 가짐"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"이러한 방식은 레이어가 깊어짐에 따라 receptive field가 넓어지는 CNN과 유사함"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://haebinshin.github.io/public/img/longformer/receptive_field.png"},"children":[]},{"type":"text","value":"  \n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"예) window size가 2일 때, 레이어가 쌓일수록 w만큼 receptive field가 넓어짐."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://haebinshin.github.io/public/img/longformer/text_sliding_window_receptive_field.jpg"},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"l x w의 receptive field size를 가지게 됨."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"각 레이어마다 w의 크기를 다르게 주는 방법이 도움이 될 수도 있음"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Dilated Sliding Window"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Sliding window attention보다도 receptive field를 더 넓히기 위해 고안된 방법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Dilated Convolution에서 착안"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://haebinshin.github.io/public/img/longformer/dilation_convolution.gif"},"children":[]},{"type":"text","value":"  \n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"dilated 값을 줘서 토큰을 d만큼 건너뛰면서 어텐션하도록 하는 방법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"예) window size가 2이고 dilation size가 2일 때, 아래 그림과 같이 w x d만큼 receptive field가 넓어짐"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://haebinshin.github.io/public/img/longformer/text_dilated_sliding_window_receptive_field.jpg"},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"l x d x w의 receptive field size를 가지게 됨."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Global Attention"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"BERT의 [CLS] 토큰 같은 경우는 전체 컨텍스트를 바라봐야하는데, 위의 2가지 방법만으로는 Finetuning하는 태스크에서는 부족한 부분이 있을 수 있음"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"따라서 스페셜 토큰 몇 개에 대해서는 global attention을 수행하도록 함."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"전체 토큰 수에 비해서는 스페셜 토큰은 매우 적기 때문에 복잡도는 여전히 O(n)"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Linear Projections for Global Attention"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"보통의 트랜스포머의 어텐션은 Q, K, V로 이루어 지는데, sliding window 기반 어텐션과 global 어텐션을 위해 sliding Q, K, V와 global Q, K, V 두 세트로 나눠서 어텐션을 계산하도록 구현"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Experiments"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"2가지 방식으로 평가를 진행."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Autoregressive Language Modeling"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"모델 자체의 임베딩 평가를 위함"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"character/token 단위의 language modeling을 수행."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"text8"}]},{"type":"text","value":", "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"enwik8"}]},{"type":"text","value":" 데이터셋에서 SOTA를 달성"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"본 태스크는 dilated sliding window attention 사용"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://haebinshin.github.io/public/img/longformer/table_2_3.png"},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Pre-training and Fine-tuning"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"RoBERTa 체크포인트로부터 시작해서 학습"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"sliding window attention를 사용"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"각 태스크에 따라 스페셜 토큰을 지정하여 global attention을 사용"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"WikiHop"}]},{"type":"text","value":"과 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"TriviaQA"}]},{"type":"text","value":"에서 SOTA 달성"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://haebinshin.github.io/public/img/longformer/table8.png"},"children":[]}],"data":{"quirksMode":false}},"excerpt":"Longformer: The Long-Document Transformer Paper Code Iz Beltagy et al. Introduction 트랜스포머는 긴 시퀀스는 처리하지 못한다는 한계를 가지고 있음 이유는 시퀀스 길이에 O(n^…","fields":{"readingTime":{"text":"4 min read"}},"frontmatter":{"title":"Sooftware NLP - Longformer Paper Review","userDate":"6 February 2021","date":"2021-02-06T23:46:37.121Z","tags":["nlp","paper"],"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#f8f8f8","images":{"fallback":{"src":"/static/ef32af94bd92c05cbfebe0cb00c3966e/597e6/longformer.png","srcSet":"/static/ef32af94bd92c05cbfebe0cb00c3966e/597e6/longformer.png 512w","sizes":"100vw"},"sources":[{"srcSet":"/static/ef32af94bd92c05cbfebe0cb00c3966e/3d2a6/longformer.webp 512w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.8046875000000001}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"children":[{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#282838","images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/248f9/soohwan.png 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/fd435/soohwan.png 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png 120w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/e7f45/soohwan.webp 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/589ec/soohwan.webp 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/71a38/soohwan.webp 120w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6833333333333333}}]}}]}},"relatedPosts":{"totalCount":50,"edges":[{"node":{"id":"6529d72e-80e7-5d61-a672-d66276a3f641","excerpt":"MoE(Mixture of Experts) 기초부터 DeepSeek 혁신까지 작년 이맘때쯤 DeepSeek-V3가 저비용으로 엄청난 성능을 보이면서 화제가 되었습니다. 그 핵심 기술인 MoE(Mixture of Experts…","frontmatter":{"title":"MoE(Mixture of Experts) 기초부터 DeepSeek 혁신까지","date":"2026-01-15T01:11:55.000Z"},"fields":{"readingTime":{"text":"8 min read"},"slug":"/developed-moe/"}}},{"node":{"id":"8b49d3ef-4ce3-568c-9d71-240ff17fc3e0","excerpt":"BERT는 사실 Diffusion 모델이였다?! 최근 굉장히 흥미로운 글을 읽게되어 공유합니다. 원문 : link BERT와 Diffusion이 같은 방식이다?! NLP 연구자들에게 BERT는 너무 익숙한 모델입니다. 201…","frontmatter":{"title":"BERT는 사실 Diffusion 모델이였다?!","date":"2025-10-21T12:00:00.000Z"},"fields":{"readingTime":{"text":"8 min read"},"slug":"/bert_diffusion/"}}},{"node":{"id":"7c7e1676-ea84-58de-970c-ddec9aa10660","excerpt":"RLHF는 수다쟁이를 만든다?! (Does RLHF Breed Verbose Chatterboxes?!) RLHF(Reinforcement Learning from Human Feedback)는 OpenAI의 ChatGPT…","frontmatter":{"title":"RLHF는 수다쟁이를 만든다?! (Does RLHF Breed Verbose Chatterboxes?!)","date":"2024-03-13T01:11:55.000Z"},"fields":{"readingTime":{"text":"7 min read"},"slug":"/rlhf-vervosity/"}}},{"node":{"id":"2b9d3e22-1796-5fab-878d-5941d2e76e9d","excerpt":"LLM Paper Abstract - 2023.12 LLM…","frontmatter":{"title":"LLM Paper Abstract - 2023.12","date":"2024-01-05T10:00:00.000Z"},"fields":{"readingTime":{"text":"5 min read"},"slug":"/llm-abs-202312/"}}},{"node":{"id":"f43fa33b-917b-5c00-8462-937d99592ad7","excerpt":"What it MoE? (Mixture of Experts) 현존 최강 LLM인 GPT-4에서 “MoE (Mixture of Experts)” 방식을 채택하여 사용하고 있다고 알려졌는데요, 최근 AI계의 뜨거운 감자 Mistral AI…","frontmatter":{"title":"What is MoE? (Mixture of Experts)","date":"2023-12-22T01:11:55.000Z"},"fields":{"readingTime":{"text":"9 min read"},"slug":"/moe/"}}}]}},"pageContext":{"slug":"/longformer/","prev":{"excerpt":"Computer Architecture Review 오랜만에 컴퓨터 구조에서 배운 내용을 조금 복습해보며 감을 잡기 위함 컴퓨터가 코드를 처리하는 과정 Read Code Assembly 변환 CPU에서 실행 CPU에서 하나의 명령(Ex) Add…","frontmatter":{"title":"Computer Architecture Review","tags":["cs"],"date":"2021-02-05T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAAAsTAAALEwEAmpwYAAACGklEQVQ4y21UCXLbMAz0/7/YVLYsx4oO6+BNguSmgI8kbTmDASGSi2uhQ60VvHwImOdZhIhgjEHTNPjdNLJ3zol9uVzgvcc4ThinCVobuT9NE/q+xwH/WbUU1JJBKYFi5A9IMT4kIBP9uE8PO6WEg6aMyViMymDSRvSgNLaQYHIV2SNh9QG2QOzVR6iUsaeMXCEOSimIMeIw+ghrLahWaGuxa41NKVgfwMXItYJKQUiElLPYiSPi+7nAl4qSCaXWO+AcIlLwEvLtdoPWGh99j33f4Z2DsxbWGKh9x75tcm5CBJWKLeV/ASfPgEEAt8eDEMKrLrxyzjifz3Lef3xgsQ4uFxgqKLinXL8iTHBaSRoMxgdcj2dNWNjBMAzSbbZRyqN7VRpI3wGXXDDMM361Z3Rdh2VZhCKntsXxeIRSCuM4Cp2ssaKZMsZaOO8FjO+/ukwVMCFIrTg1bhBH07y94dJ1ArauK9q2Fa6xQ96zZuFv7ITLwc5/8JDDZuLyg+O1R9t14oibdTqd8P7+LukzsVmzIwZje54mvI0zDgzyt3DozXSTVOKjQdfrvfNcpycQTwrX+rYs8JyZ8V8RPsGkFkQ4DSOMVhJZf71CK/U1GSnBO/uiFTviBs0u3AGfQE9NOWOwHj4l3LYdu7HCvVQqYilwPJIVIpn/A5SF/P33CL8DcnNMotfoaSpYQ5TJuO8TFGWR5xmP4vonwk/QczzzyGBGjgAAAABJRU5ErkJggg=="},"images":{"fallback":{"src":"/static/83debf50782f47536b48f20a7c0c3d46/ac5ac/computer-architecture.png","srcSet":"/static/83debf50782f47536b48f20a7c0c3d46/1d01f/computer-architecture.png 750w,\n/static/83debf50782f47536b48f20a7c0c3d46/ac5ac/computer-architecture.png 798w","sizes":"100vw"},"sources":[{"srcSet":"/static/83debf50782f47536b48f20a7c0c3d46/96ac1/computer-architecture.webp 750w,\n/static/83debf50782f47536b48f20a7c0c3d46/2afea/computer-architecture.webp 798w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.7017543859649122}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"2 min read"},"layout":"","slug":"/computer-ar-review/"}},"next":{"excerpt":"PORORO Text-To-Speech (TTS) 얼마전에 저희 팀에서 공개한 PORORO: Platform Of neuRal mOdels for natuRal language prOcessing 라이브러리에 제가 공들여만든 TTS…","frontmatter":{"title":"PORORO Text-To-Speech (TTS)","tags":["speech","toolkit","record"],"date":"2021-02-16T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAAAsTAAALEwEAmpwYAAADMElEQVQ4y3WTbUyVZRjHr/t+znke4LwhKih4PMiRlx2JDkfXmvDBGMs5yLS5MHEHEOdYwQrFMXrTUEcLXW1y7Ph6WNBxOntxHJMiZa5aWzXXiNl6oWJ8CJZjfelDffm185zp7EP39t917b633/2/711/sQyNZShMLZhKMJQg8v8yDMOuSilbmf6eFJJtKLIMlYEqwTQUHrcLn9eLy+UiOzsbj8eD1+vFsiwbkJOTc/+CNEQ/aMLtEFxpOTVahNI1AWZmZpidnWV6eprx8XHm5uZYWFggHo/T29vL/Pw8yWQSQ6v7ziyHwpelkAJTWG4KeaaQhnssJ9HmXaRXV1cngUCAyclJvpuaIhKJ2PsTE5+ytXGLbSDH1BT5DEIFBuEiA/GbQqEpFDiFfEtjilC4bAmLi4uEw1X2M2KxGKlUyu479rUzN/srifPnWOZ2UrlCU1tisLFYs36lQpqKhYZCoXapUOlV5CuhujxoO6mv22RDkhcvcevWTdaHQ7wzMsIHV1P2eUudny1lQm2xQUWewu9WyLtPCvHNQn+NsLdSUbtciPjzuH4tRahinQ08/FIfidjr1FUX8c3Nyyz8/DXDR/eyY51QX+7gkYCDyGon4dUmcvBRoXOD0FYlbC0RGkM+Ysf6GBp4mbdfbWb0eCep2G7Gzh9guCfIjUQP7w9GOdu/h4bqXMJ+TeUqk7KVTtauMJHBbcKJHcLJ3cKFDuG9XuHHa0389ctp/vnpBbg7zN/fP8NvH+/iUrdwY7SPy6f6uJIYpKGmlLVLhWC+iT/PQVGuA4lHhZEOYeyg8Pkx4c4Z4e5XTfxxO8qfPxzm9y9buXP9Oa4OtdG1rYq3+rs41B3lwLOtVKzKZYkl+LIMPJbGbWmkq0bofUw40iC8uVM4t09zO3WUmS+G+HbiLFOffcj4xZOkriQZvXCGxs319L/yIp98NMaG8EP2H1sOjaElM+Cb/MLjJcIT5cJTIeHphzWJN/Yzeuo42zeW83zrdk4MvMbAkUO0NO+kZ383ba0ttLfvoWRNIBNHnUlLesgl6BZKvUKZL1ODXsGfI3hEcGnBSuf7wSzr/2bbTsq9XhT/AnHU0lj4MouRAAAAAElFTkSuQmCC"},"images":{"fallback":{"src":"/static/f50103e49747efd11b2901a9ce98f113/b444b/tts.png","srcSet":"/static/f50103e49747efd11b2901a9ce98f113/b444b/tts.png 600w","sizes":"100vw"},"sources":[{"srcSet":"/static/f50103e49747efd11b2901a9ce98f113/9ff6b/tts.webp 600w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6666666666666666}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"1 min read"},"layout":"","slug":"/pororo-tts/"}},"primaryTag":"nlp"}},
    "staticQueryHashes": ["3170763342","3229353822"]}