{ "componentChunkName": "component---src-templates-post-tsx", "path": "/wav2vec2/", "result": {"data":{"logo":null,"markdownRemark":{"html":"

wav2vec 2.0 : A Framework for Self-Supervised Learning of Speech Representations

\n

Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, Michael Auli
\nFacebook AI Research (FAIR)
\narXiv (2020.06)

\n

References

\n\n

Summary

\n\n

Abstract

\n\n

Wav2vec

\n\n

Wav2vec 1.0 (previous work)

\n \n\n

VQ-Wav2vec (previous work)

\n \n\n

Vector Quantization

\n \n
    \n
  1. \n

    Z를 선형변환하여 logit을 만듦

    \n
    2. \n
  2. \n

    여기에 Gumbel Softmax와 argmax를 취해 one-hot vector를 만듦

    \n
    3. \n
  3. \n

    이후 Embedding matrix를 내적해 Z^를 만듦

    \n
    4. \n
\n

Wav2vec 2.0

\n \n\n

Model

\n

VQ-Wav2vec와 비교하여 Transformer를 사용했다는 특징이 있음

\n

Feature Encoder

\n\n

Transformer

\n\n

Quantization module

\n\n

Training

\n

BERT의 masked language modeling (MLM)과 유사하게 latent speech의 일부분을 masking하며, 모델은 quantized latent audio representation을 맞추는 방식으로 트레이닝이 진행됨. 이렇게 학습한 후 labeled 된 데이터로 fine-tuning 진행.

\n

Masking

\n\n
1. 전체 오디오 구간 중 6.5%를 랜덤하게 선택  \n  \n2. 선택된 구간부터 10 time-step만큼 masking (masking은 중복될 수 있음)
\n\n

Objective

\n

\"image\"

\n

Lm : Contrastive Loss, Ld : Diversity Loss, Lf : L2 penalty, {alpha, beta} : hyperparameter

\n\n

\"contrastive-loss\"

\n\n

\"diversity-loss\"

\n

Fine-tuning

\n\n

Experiment

\n

Datasets

\n\n

Result

\n\n

data setups of 10 min, 1 hour, 10 hours and the clean 100h subset of Librispeech

\n

\"image\"

\n\n

\"image\"

\n

Conclusion

\n","htmlAst":{"type":"root","children":[{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"wav2vec 2.0 : A Framework for Self-Supervised Learning of Speech Representations"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, Michael Auli"},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\nFacebook AI Research (FAIR)"},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\narXiv (2020.06)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"References"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://arxiv.org/abs/2006.11477"},"children":[{"type":"text","value":"arXiv"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://github.com/pytorch/fairseq/tree/master/examples/wav2vec"},"children":[{"type":"text","value":"source code"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://arxiv.org/abs/1810.04805"},"children":[{"type":"text","value":"BERT"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://arxiv.org/pdf/1904.05862.pdf"},"children":[{"type":"text","value":"Wav2vec"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://arxiv.org/pdf/1910.05453.pdf"},"children":[{"type":"text","value":"VQ-Wav2vec"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://ratsgo.github.io/speechbook/docs/neuralfe/wav2vec"},"children":[{"type":"text","value":"speech book"}]}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Summary"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"BERT in Speech Recognition"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Representation learning with 53,000 hours of unlabeled speech data."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Excellent speech recognition performance with only 40 sentences (10 minutes) of labeled data"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Abstract"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"음성 데이터 자체만으로 학습한 후 Fine-tuning하는 간단한 방법으로 semi-supervised 방법보다 좋은 성능을 냄"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"TIMIT, LibriSpeech 100h 데이터셋에서 State-Of-The-Art (SOTA) 를 달성"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"LibriSpeech 100h은 단 1시간의 데이터만으로 기존 SOTA보다 높은 성능을 보임"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"단 10분의 데이터 (40문장) 으로 LibriSpeech clean 5.7 / noisy 10.1 Word Error Rate (WER) 를 기록"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"LibriSpeech의 전체 학습셋을 사용했을 때 (960h) clean 1.9 / noisy 3.5 WER을 기록 (현재 "},{"type":"element","tagName":"a","properties":{"href":"https://github.com/syhw/wer_are_we"},"children":[{"type":"text","value":"wer-are-we"}]},{"type":"text","value":" SOTA보다 높은 기록)"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Wav2vec"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"현재 어느 정도 수준 이상의 음성인식기를 만들기 위해서는 대량의 데이터가 필요함 (수천 시간)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"세상에는 7,000개 이상의 언어가 존재하는데 모든 언어에 대해 이 정도 수준의 데이터를 구축하기는 어려움"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"영아 (infant) 들은 단순히듣는 것만으로 음성에 대해 학습함"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Wav2vec 1.0 (previous work)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://i.imgur.com/H9X1HiX.png","width":500},"children":[]},{"type":"text","value":" \n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"wav2vec은 크게 "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"encoder"}]},{"type":"text","value":" network "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"f"}]},{"type":"text","value":"와 "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"context"}]},{"type":"text","value":" network "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"g"}]},{"type":"text","value":" 두개의 파트로 구성 (둘 모두 convolution neural network)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"wav2vec은 해당 입력이 Positive인지 Negative인지 이진 분류(Binary Classification)하는 과정에서 학습"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Positive : (C"},{"type":"element","tagName":"sub","properties":{},"children":[{"type":"text","value":"i"}]},{"type":"text","value":", Z"},{"type":"element","tagName":"sub","properties":{},"children":[{"type":"text","value":"i+1"}]},{"type":"text","value":"), Negative : otherwise"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Negative 쌍은 입력 음성의 i번째 context representation C"},{"type":"element","tagName":"sub","properties":{},"children":[{"type":"text","value":"i"}]},{"type":"text","value":"와 다른 음성의 hidden representation들 중 랜덤 추출"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"즉, 2개의 네트워크는 입력 음성의 다음 시퀀스가 무엇일지에 관한 정보를 음성 피처에 녹여내도록 학습"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"VQ-Wav2vec (previous work)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://i.imgur.com/ivviYL1.png","width":500},"children":[]},{"type":"text","value":" \n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Wav2vec 아키텍처 중간에 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Vector Quantization"}]},{"type":"text","value":" 모듈을 추가한 구조"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"VQ 모듈 : continuous representation Z를 discrete representation Z"},{"type":"element","tagName":"sup","properties":{},"children":[{"type":"text","value":"^"}]},{"type":"text","value":"로 변환"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Discretization(이산화)는 discrete한 input을 필요로하는 NLP 알고리즘들을 바로 적용할 수 있다는 장점이 있음"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h4","properties":{},"children":[{"type":"text","value":"Vector Quantization"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://i.imgur.com/y15Qu5Z.png","width":300},"children":[]},{"type":"text","value":" \n"},{"type":"element","tagName":"ol","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Z를 선형변환하여 logit을 만듦"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"여기에 Gumbel Softmax와 argmax를 취해 one-hot vector를 만듦"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"이후 Embedding matrix를 내적해 Z"},{"type":"element","tagName":"sup","properties":{},"children":[{"type":"text","value":"^"}]},{"type":"text","value":"를 만듦"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Wav2vec 2.0"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/42150335/92450554-8a22b280-f1f6-11ea-8f66-0616b29d8c94.png","width":500},"children":[]},{"type":"text","value":" \n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"VQ-wav2vec의 모델을 Transformer로 대체"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Pre-training 이후 labeled data로 fine-tuning (Connectionist Temporal Classification (CTC) loss 사용)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"이전 연구들과의 차이점으로 Filter-Bank와 같은 피쳐추출 과정이 없음"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Model"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"VQ-Wav2vec와 비교하여 Transformer를 사용했다는 특징이 있음"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Feature Encoder"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"N x [Conv1d, Dropout, GroupNorm, GELU]"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Transformer"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Positional Encoding을 conv1d로 대체"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Convoulation Layer 뒷단에 layer normalization 적용"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Quantization module"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Vector Quantization 부분 참고"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Training"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"BERT의 masked language modeling (MLM)과 유사하게 latent speech의 일부분을 masking하며, 모델은 quantized latent audio representation을 맞추는 방식으로 트레이닝이 진행됨. 이렇게 학습한 후 labeled 된 데이터로 fine-tuning 진행."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Masking"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Maksing 방법"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"1. 전체 오디오 구간 중 6.5%를 랜덤하게 선택 \n \n2. 선택된 구간부터 10 time-step만큼 masking (masking은 중복될 수 있음) "}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"전체 오디오 구간 중 약 49% 정도가 masking (평균 14.7 timestep (299ms))"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Objective"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/42150335/93301173-0474b780-f833-11ea-8206-5ab40cf418a5.png","alt":"image"},"children":[]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"L"},{"type":"element","tagName":"sub","properties":{},"children":[{"type":"text","value":"m"}]},{"type":"text","value":" : Contrastive Loss, L"},{"type":"element","tagName":"sub","properties":{},"children":[{"type":"text","value":"d"}]},{"type":"text","value":" : Diversity Loss, L"},{"type":"element","tagName":"sub","properties":{},"children":[{"type":"text","value":"f"}]},{"type":"text","value":" : L2 penalty, {alpha, beta} : hyperparameter"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Contrastive Loss"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/42150335/92514957-c5040500-f24d-11ea-95c3-1183fa1145b2.png","alt":"contrastive-loss"},"children":[]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Diversity Loss"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/42150335/92514997-df3de300-f24d-11ea-835c-f1367aef5ebe.png","alt":"diversity-loss"},"children":[]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Fine-tuning"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"이렇게 Pre-train 된 모델을 ASR 태스크로 Fine-tuning (+ projection layer)"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"29 character token"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"CTC Loss"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"SpecAugment"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Experiment"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Datasets"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Unlabeled data 1 : LibriVox-60k (전처리하여 53.2k 사용) ["},{"type":"element","tagName":"a","properties":{"href":"https://arxiv.org/abs/1912.07875"},"children":[{"type":"text","value":"Reference"}]},{"type":"text","value":"]"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Unlabeled data 2 : LibriSpeech 960h"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"train-10min, train-1h, train-10h, train-100h, train-960h 설정 (LibriSpeech)"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Result"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"WER on the Librispeech dev/test sets when training on the Libri-light low-resource labeled"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"data setups of 10 min, 1 hour, 10 hours and the clean 100h subset of Librispeech"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/42150335/92516234-de0db580-f24f-11ea-88f3-485ee579bfda.png","alt":"image"},"children":[]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"WER on Librispeech when using all labeled data of 960 hours"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/42150335/92516409-262cd800-f250-11ea-8e77-42fdc8761d2c.png","alt":"image"},"children":[]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Conclusion"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"적은 비용으로도 좋은 성능의 음성인식기를 만들 수 있는 연구 방향을 제시함"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Seq2seq 구조 혹은 word-piece 단위로의 변경을 통해 성능 향상 기대"}]},{"type":"text","value":"\n"}]}],"data":{"quirksMode":false}},"excerpt":"wav2vec 2.0 : A Framework for Self-Supervised Learning of Speech Representations Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, Michael…","fields":{"readingTime":{"text":"5 min read"}},"frontmatter":{"title":"Sooftware Speech - Wav2vec 2.0 : A Framework for Self-Supervised Learning of Speech Representations","userDate":"12 September 2020","date":"2020-09-12T10:00:00.000Z","tags":["speech","paper"],"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#f8f8f8","images":{"fallback":{"src":"/static/ef0fb6095c0dcc78d73cfdf24bc9ac11/038c6/wav2vec2.png","srcSet":"/static/ef0fb6095c0dcc78d73cfdf24bc9ac11/e6cc4/wav2vec2.png 750w,\n/static/ef0fb6095c0dcc78d73cfdf24bc9ac11/038c6/wav2vec2.png 842w","sizes":"100vw"},"sources":[{"srcSet":"/static/ef0fb6095c0dcc78d73cfdf24bc9ac11/79bef/wav2vec2.webp 750w,\n/static/ef0fb6095c0dcc78d73cfdf24bc9ac11/daf06/wav2vec2.webp 842w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5106888361045131}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"children":[{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#282838","images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/248f9/soohwan.png 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/fd435/soohwan.png 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png 120w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/e7f45/soohwan.webp 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/589ec/soohwan.webp 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/71a38/soohwan.webp 120w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6833333333333333}}]}}]}},"relatedPosts":{"totalCount":20,"edges":[{"node":{"id":"fa9e8cbb-841a-516f-9df6-4be0336b56b0","excerpt":"한국어 Tacotron2 이번 포스팅에서는 Tacotron2 아키텍처로 한국어 TTS 시스템을 만드는 방법에 대해 다루겠습니다. Tacotron2 Tacotron2는 17년 12월 구글이 NATURAL TTS SYNTHESIS BY…","frontmatter":{"title":"Sooftware Speech - 한국어 Tacotron2","date":"2021-10-10T10:00:00.000Z"},"fields":{"readingTime":{"text":"11 min read"},"slug":"/korean_tacotron2/"}}},{"node":{"id":"43c23529-71b1-5d60-8883-a45fbcd55ebd","excerpt":"Textless NLP: Generating expressive speech from raw audio paper / code / pre-train model / blog Name: Generative Spoken Language Model (GSLM…","frontmatter":{"title":"Sooftware NLP - Textless NLP","date":"2021-09-19T10:00:00.000Z"},"fields":{"readingTime":{"text":"4 min read"},"slug":"/Textledd NLP: Generating expressive speech from raw audio/"}}},{"node":{"id":"83c6b4fa-d71e-51d8-90bb-b58bfffc01d0","excerpt":"Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition Yu Zhang et al., 2020 Google Research, Brain Team Reference…","frontmatter":{"title":"Sooftware Speech - Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition Paper Review","date":"2021-03-17T10:00:00.000Z"},"fields":{"readingTime":{"text":"3 min read"},"slug":"/Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition/"}}},{"node":{"id":"894a24ca-fff9-5884-b6da-1c97f2ece7bc","excerpt":"PORORO Text-To-Speech (TTS) 얼마전에 저희 팀에서 공개한 PORORO: Platform Of neuRal mOdels for natuRal language prOcessing 라이브러리에 제가 공들여만든 TTS…","frontmatter":{"title":"PORORO Text-To-Speech (TTS)","date":"2021-02-16T10:00:00.000Z"},"fields":{"readingTime":{"text":"1 min read"},"slug":"/pororo-tts/"}}},{"node":{"id":"b039977c-cecb-50f4-a0dd-fd008731bc99","excerpt":"EMNLP Paper Review: Speech Adaptive Feature Selection for End-to-End Speech Translation (Biao Zhang et al) Incremental Text-to-Speech…","frontmatter":{"title":"Sooftware Speech - EMNLP Paper Review: Speech","date":"2020-12-08T10:00:00.000Z"},"fields":{"readingTime":{"text":"4 min read"},"slug":"/2020 EMNLP Speech Paper Review/"}}}]}},"pageContext":{"slug":"/wav2vec2/","prev":{"excerpt":"Conformer: Convolution-augmented Transformer for Speech Recognition Anmol Gulati et al. Google Inc. INTERSPEECH, 2020 Reference Conformer…","frontmatter":{"title":"Sooftware Speech - Conformer Paper Review","tags":["speech","paper"],"date":"2020-08-30T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAAAsTAAALEwEAmpwYAAACaklEQVQ4y3WTW2/TQBSE8//feOaJFyRAFQgkQIVQQPRCrzTpLbWTOHEvSRwn8dq79tpef2gdWpoKVlrb2mPNmZk909BaI6VE5zmZ1pTGUBQFcRwzDQLG4zHz+RylFHZVVbXyfrwapqpIoojJwT6jzR/M/SGp1uS5JhIxkyAkyzJs48dgtum1P6DnnHPt92siDVvIpGRxcU7UbpOMxyRKURlDFCeMwwVaZ0j5l+EdoHPp8G79A8/WnvPi7RqLxWIJaDC4QY/W1SlxnhBHMVeBz+7lVr3P/BOyNKOiWgHs93ocdgY0j1w2Wy5CCBplWRInAn/i0r0+QekYY6raU6FShqOAhYhQKl2Ray3odnt82/jAp48v2d35UltQe1hkkur2mMLbhnRKYSCXAdHomOnVIWrukqbZCqAl0u322d14Q/PVEz6/frqUbAtSJixufQLPpcwUpakotCBLhiymDlJcrTC8l9z3ONr7yebXdVqHuyRJQsP6kuuSk70u280WIlS1VzqRiJuAG9dHjCaodFXybDZjMBhwcnZO6/QMISXGGBr2oWRCNBkSXjuUWmIA4Q/xm+t46+8JDvdRWWbR7gGtPLvd01PaW9uM+95ybGy9LFLS8ABx8x2KkLyowBjK3BBMI7RKVwb74VDfeh6t7R28i84S0DK0SSmLAlMW94YnaULvtk9vNCCMp6SPJN8B20udzmYYlueNh7F5GKZQTGn3f3HkHnDpX6Dk/6P38Ltx1+nu0NK2txWLGCHiGsgOtY3fY8l5ntfqLHubpD8erna00+44DsPhkE6nc+/dv9iFYVj/63kel45DFEX8Bpe/KyikSRfWAAAAAElFTkSuQmCC"},"images":{"fallback":{"src":"/static/5ca9b355b1dc4393c360f527478a3ba2/503d5/conformer.png","srcSet":"/static/5ca9b355b1dc4393c360f527478a3ba2/bfaac/conformer.png 750w,\n/static/5ca9b355b1dc4393c360f527478a3ba2/abf2b/conformer.png 1080w,\n/static/5ca9b355b1dc4393c360f527478a3ba2/6c19a/conformer.png 1366w,\n/static/5ca9b355b1dc4393c360f527478a3ba2/503d5/conformer.png 1890w","sizes":"100vw"},"sources":[{"srcSet":"/static/5ca9b355b1dc4393c360f527478a3ba2/1e5e2/conformer.webp 750w,\n/static/5ca9b355b1dc4393c360f527478a3ba2/77f81/conformer.webp 1080w,\n/static/5ca9b355b1dc4393c360f527478a3ba2/b9a60/conformer.webp 1366w,\n/static/5ca9b355b1dc4393c360f527478a3ba2/2837d/conformer.webp 1890w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6888888888888889}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"4 min read"},"layout":"","slug":"/conformer/"}},"next":{"excerpt":"Below is just about everything you’ll need to style in the theme. Check the source code to see the many embedded elements within paragraphs…","frontmatter":{"title":"Sooftware NLP - Electra Paper Review","tags":["nlp","paper"],"date":"2020-09-23T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAYAAAAywQxIAAAACXBIWXMAAAsTAAALEwEAmpwYAAABY0lEQVQoz02SacuCUBCF/f+/qO9BGFnaSqVp0UIl7aWtnpdn4MIrXObO3HPObHpVVen7/er3++n1eunz+ej5fOrxeKgsSx2PRw2HQ/N5x7/dbrper7pcLmbhvN9voeVJ0m6302az0Ww202KxUJZlut/vJgwBkeVyqfl8bj6fK4I4CeHweSgnSWKiBBGG2Gg0VKvV1O12dTqdtF6vtd1uNRgM5Pu+6vW6RqOR4eM41nQ6tUo9sqxWKwMj5IQRSNPUKj6fz0akGtqleizFgEMYUbSsZQi0QiUAD4eDtUtbzI47mP1+bxhmyXyLolCe51YEHGsZAR4hAeCOCAIuERjeqQxBtxAOPniX2KNU5sJgmddkMrF7v983Szu8s7D/8V6vp/F4rDAMzcJlJB4OIAAEW62W2u22CWHdaTabiqLI3judjvngiXEnzg5MMAgCE4UImKxYwCzD/VbM0B1iWJbJUvEZxR8WlJe1e/a3LQAAAABJRU5ErkJggg=="},"images":{"fallback":{"src":"/static/7ccdb9951c9d362c8a3548e8c2a87231/59ccb/electra.png","srcSet":"/static/7ccdb9951c9d362c8a3548e8c2a87231/c68af/electra.png 750w,\n/static/7ccdb9951c9d362c8a3548e8c2a87231/87f65/electra.png 1080w,\n/static/7ccdb9951c9d362c8a3548e8c2a87231/d464a/electra.png 1366w,\n/static/7ccdb9951c9d362c8a3548e8c2a87231/59ccb/electra.png 1920w","sizes":"100vw"},"sources":[{"srcSet":"/static/7ccdb9951c9d362c8a3548e8c2a87231/9fb02/electra.webp 750w,\n/static/7ccdb9951c9d362c8a3548e8c2a87231/cd76f/electra.webp 1080w,\n/static/7ccdb9951c9d362c8a3548e8c2a87231/b7397/electra.webp 1366w,\n/static/7ccdb9951c9d362c8a3548e8c2a87231/507b8/electra.webp 1920w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.4479166666666667}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"4 min read"},"layout":"","slug":"/electra/"}},"primaryTag":"speech"}}, "staticQueryHashes": ["3170763342","3229353822"]}