{
    "componentChunkName": "component---src-templates-post-tsx",
    "path": "/fairseq-hydra/",
    "result": {"data":{"logo":null,"markdownRemark":{"html":"<h2><a href=\"https://github.com/pytorch/fairseq/blob/master/docs/hydra_integration.md\">Fairseq’s Hydra</a></h2>\n<ul>\n<li>Fairseq이 0.10.1로 버젼 업그레이드를 하면서 configuration 관리를 Hydra로 하게됨.</li>\n<li>Fairseq을 실행시키는 command line상에서도, 모델 코드 상에서도 몇 가지 변화가 있음.</li>\n</ul>\n<hr>\n<h3>Creating or migrating components</h3>\n<ul>\n<li>기존 모델 코드는 <code class=\"language-text\">add_args()</code>라는 static method로 config를 관리</li>\n</ul>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token decorator annotation punctuation\">@staticmethod</span>\n<span class=\"token keyword\">def</span> <span class=\"token function\">add_args</span><span class=\"token punctuation\">(</span>parser<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n    <span class=\"token triple-quoted-string string\">\"\"\"Add model-specific arguments to the parser.\"\"\"</span>\n    <span class=\"token comment\"># fmt: off</span>\n    parser<span class=\"token punctuation\">.</span>add_argument<span class=\"token punctuation\">(</span><span class=\"token string\">'--dropout'</span><span class=\"token punctuation\">,</span> <span class=\"token builtin\">type</span><span class=\"token operator\">=</span><span class=\"token builtin\">float</span><span class=\"token punctuation\">,</span> metavar<span class=\"token operator\">=</span><span class=\"token string\">'D'</span><span class=\"token punctuation\">,</span>\n                        <span class=\"token builtin\">help</span><span class=\"token operator\">=</span><span class=\"token string\">'dropout probability'</span><span class=\"token punctuation\">)</span>\n    parser<span class=\"token punctuation\">.</span>add_argument<span class=\"token punctuation\">(</span><span class=\"token string\">'--encoder-embed-dim'</span><span class=\"token punctuation\">,</span> <span class=\"token builtin\">type</span><span class=\"token operator\">=</span><span class=\"token builtin\">int</span><span class=\"token punctuation\">,</span> metavar<span class=\"token operator\">=</span><span class=\"token string\">'N'</span><span class=\"token punctuation\">,</span>\n                        <span class=\"token builtin\">help</span><span class=\"token operator\">=</span><span class=\"token string\">'encoder embedding dimension'</span><span class=\"token punctuation\">)</span>\n    parser<span class=\"token punctuation\">.</span>add_argument<span class=\"token punctuation\">(</span><span class=\"token string\">'--encoder-embed-path'</span><span class=\"token punctuation\">,</span> <span class=\"token builtin\">type</span><span class=\"token operator\">=</span><span class=\"token builtin\">str</span><span class=\"token punctuation\">,</span> metavar<span class=\"token operator\">=</span><span class=\"token string\">'STR'</span><span class=\"token punctuation\">,</span>\n                        <span class=\"token builtin\">help</span><span class=\"token operator\">=</span><span class=\"token string\">'path to pre-trained encoder embedding'</span><span class=\"token punctuation\">)</span></code></pre></div>\n<blockquote>\n<p>이런 방식은 코드를 읽을 때, 어떤 argument들이 사용되는지 한 번에 보기가 어렵다는 단점이 있음.\n→ 내부적으로 사용되는 모듈들을 하나하나 찾아가서 확인해봐야함.\n→ 복잡한 모델일수록 더욱 코드 읽기가 어려워짐.</p>\n</blockquote>\n<ul>\n<li><code class=\"language-text\">@dataclass</code> 데코레이터를 이용하는 class를 이용하는 방법으로 수정 (Tasks &#x26; Models)</li>\n</ul>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token decorator annotation punctuation\">@dataclass</span>\n<span class=\"token keyword\">class</span> <span class=\"token class-name\">Wav2Vec2Config</span><span class=\"token punctuation\">(</span>FairseqDataclass<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n    extractor_mode<span class=\"token punctuation\">:</span> EXTRACTOR_MODE_CHOICES <span class=\"token operator\">=</span> field<span class=\"token punctuation\">(</span>\n        default<span class=\"token operator\">=</span><span class=\"token string\">\"default\"</span><span class=\"token punctuation\">,</span>\n        metadata<span class=\"token operator\">=</span><span class=\"token punctuation\">{</span>\n            <span class=\"token string\">\"help\"</span><span class=\"token punctuation\">:</span> <span class=\"token string\">\"mode for feature extractor. default has a single group norm with d \"</span>\n            <span class=\"token string\">\"groups in the first conv block, whereas layer_norm has layer norms in \"</span>\n            <span class=\"token string\">\"every block (meant to use with normalize=True)\"</span>\n        <span class=\"token punctuation\">}</span><span class=\"token punctuation\">,</span>\n    <span class=\"token punctuation\">)</span>\n    encoder_layers<span class=\"token punctuation\">:</span> <span class=\"token builtin\">int</span> <span class=\"token operator\">=</span> field<span class=\"token punctuation\">(</span>\n        default<span class=\"token operator\">=</span><span class=\"token number\">12</span><span class=\"token punctuation\">,</span> metadata<span class=\"token operator\">=</span><span class=\"token punctuation\">{</span><span class=\"token string\">\"help\"</span><span class=\"token punctuation\">:</span> <span class=\"token string\">\"num encoder layers in the transformer\"</span><span class=\"token punctuation\">}</span>\n    <span class=\"token punctuation\">)</span>\n    encoder_embed_dim<span class=\"token punctuation\">:</span> <span class=\"token builtin\">int</span> <span class=\"token operator\">=</span> field<span class=\"token punctuation\">(</span>\n        default<span class=\"token operator\">=</span><span class=\"token number\">768</span><span class=\"token punctuation\">,</span> metadata<span class=\"token operator\">=</span><span class=\"token punctuation\">{</span><span class=\"token string\">\"help\"</span><span class=\"token punctuation\">:</span> <span class=\"token string\">\"encoder embedding dimension\"</span><span class=\"token punctuation\">}</span>\n    <span class=\"token punctuation\">)</span>\n    encoder_ffn_embed_dim<span class=\"token punctuation\">:</span> <span class=\"token builtin\">int</span> <span class=\"token operator\">=</span> field<span class=\"token punctuation\">(</span>\n        default<span class=\"token operator\">=</span><span class=\"token number\">3072</span><span class=\"token punctuation\">,</span> metadata<span class=\"token operator\">=</span><span class=\"token punctuation\">{</span><span class=\"token string\">\"help\"</span><span class=\"token punctuation\">:</span> <span class=\"token string\">\"encoder embedding dimension for FFN\"</span><span class=\"token punctuation\">}</span>\n    <span class=\"token punctuation\">)</span></code></pre></div>\n<blockquote>\n<p>FairseqDataclass를 상속받는 클래스를 정의해서 사용 → <a href=\"https://github.com/pytorch/fairseq/blob/master/fairseq/models/wav2vec/wav2vec2.py\">Example</a></p>\n</blockquote>\n<ul>\n<li>Task Example</li>\n</ul>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token decorator annotation punctuation\">@dataclass</span>\n<span class=\"token keyword\">class</span> <span class=\"token class-name\">LanguageModelingConfig</span><span class=\"token punctuation\">(</span>FairseqDataclass<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n    data<span class=\"token punctuation\">:</span> Optional<span class=\"token punctuation\">[</span><span class=\"token builtin\">str</span><span class=\"token punctuation\">]</span> <span class=\"token operator\">=</span> field<span class=\"token punctuation\">(</span>\n        default<span class=\"token operator\">=</span><span class=\"token boolean\">None</span><span class=\"token punctuation\">,</span> metadata<span class=\"token operator\">=</span><span class=\"token punctuation\">{</span><span class=\"token string\">\"help\"</span><span class=\"token punctuation\">:</span> <span class=\"token string\">\"path to data directory\"</span><span class=\"token punctuation\">}</span>\n    <span class=\"token punctuation\">)</span>\n    <span class=\"token punctuation\">.</span><span class=\"token punctuation\">.</span><span class=\"token punctuation\">.</span>\n\n<span class=\"token decorator annotation punctuation\">@register_task</span><span class=\"token punctuation\">(</span><span class=\"token string\">\"language_modeling\"</span><span class=\"token punctuation\">,</span> dataclass<span class=\"token operator\">=</span>LanguageModelingConfig<span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">class</span> <span class=\"token class-name\">LanguageModelingTask</span><span class=\"token punctuation\">(</span>LegacyFairseqTask<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n    <span class=\"token punctuation\">.</span><span class=\"token punctuation\">.</span><span class=\"token punctuation\">.</span>\n    <span class=\"token decorator annotation punctuation\">@classmethod</span>\n    <span class=\"token keyword\">def</span> <span class=\"token function\">setup_task</span><span class=\"token punctuation\">(</span>cls<span class=\"token punctuation\">,</span> cfg<span class=\"token punctuation\">:</span> LanguageModelingConfig<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n        <span class=\"token punctuation\">.</span><span class=\"token punctuation\">.</span><span class=\"token punctuation\">.</span></code></pre></div>\n<ul>\n<li>Model Example</li>\n</ul>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token decorator annotation punctuation\">@dataclass</span>\n<span class=\"token keyword\">class</span> <span class=\"token class-name\">TransformerLanguageModelConfig</span><span class=\"token punctuation\">(</span>FairseqDataclass<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n    activation_fn<span class=\"token punctuation\">:</span> ChoiceEnum<span class=\"token punctuation\">(</span>utils<span class=\"token punctuation\">.</span>get_available_activation_fns<span class=\"token punctuation\">(</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span> <span class=\"token operator\">=</span> field<span class=\"token punctuation\">(</span>\n        default<span class=\"token operator\">=</span><span class=\"token string\">\"relu\"</span><span class=\"token punctuation\">,</span> metadata<span class=\"token operator\">=</span><span class=\"token punctuation\">{</span><span class=\"token string\">\"help\"</span><span class=\"token punctuation\">:</span> <span class=\"token string\">\"activation function to use\"</span><span class=\"token punctuation\">}</span>\n    <span class=\"token punctuation\">)</span>\n    dropout<span class=\"token punctuation\">:</span> <span class=\"token builtin\">float</span> <span class=\"token operator\">=</span> field<span class=\"token punctuation\">(</span>default<span class=\"token operator\">=</span><span class=\"token number\">0.1</span><span class=\"token punctuation\">,</span> metadata<span class=\"token operator\">=</span><span class=\"token punctuation\">{</span><span class=\"token string\">\"help\"</span><span class=\"token punctuation\">:</span> <span class=\"token string\">\"dropout probability\"</span><span class=\"token punctuation\">}</span><span class=\"token punctuation\">)</span>\n    <span class=\"token punctuation\">.</span><span class=\"token punctuation\">.</span><span class=\"token punctuation\">.</span>\n\n<span class=\"token decorator annotation punctuation\">@register_model</span><span class=\"token punctuation\">(</span><span class=\"token string\">\"transformer_lm\"</span><span class=\"token punctuation\">,</span> dataclass<span class=\"token operator\">=</span>TransformerLanguageModelConfig<span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">class</span> <span class=\"token class-name\">TransformerLanguageModel</span><span class=\"token punctuation\">(</span>FairseqLanguageModel<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n    <span class=\"token punctuation\">.</span><span class=\"token punctuation\">.</span><span class=\"token punctuation\">.</span>\n    <span class=\"token decorator annotation punctuation\">@classmethod</span>\n    <span class=\"token keyword\">def</span> <span class=\"token function\">build_model</span><span class=\"token punctuation\">(</span>cls<span class=\"token punctuation\">,</span> cfg<span class=\"token punctuation\">:</span> TransformerLanguageModelConfig<span class=\"token punctuation\">,</span> task<span class=\"token punctuation\">:</span> FairseqTask<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n        <span class=\"token punctuation\">.</span><span class=\"token punctuation\">.</span><span class=\"token punctuation\">.</span>\n</code></pre></div>\n<ul>\n<li>기존 방법(<code class=\"language-text\">add_args()</code>)은 여전히 서포트 되지만, 이후 언젠가 deprecated 될 것이라고 함.</li>\n</ul>\n<hr>\n<h3>Training with <code class=\"language-text\">fairseq-hydra-train</code></h3>\n<ul>\n<li>configuraion 관리를 hydra로 하기 때문에 기존에 사용했던 <code class=\"language-text\">fairseq-train</code>이 아닌 <code class=\"language-text\">fairseq-hydra-train</code> training</li>\n<li>Override default values through command line:</li>\n</ul>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">$ fairseq-hydra-train \\\n    distributed_training.distributed_world_size=1 \\\n    dataset.batch_size=2 \\\n    task.data=data-bin \\\n    model=transformer_lm/transformer_lm_gpt \\\n    task=language_modeling \\\n    optimization.max_update=5000</code></pre></div>\n<ul>\n<li>Replace bundled configs with an external config:</li>\n</ul>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">$ fairseq-hydra-train \\\n    --config-dir /path/to/external/configs \\\n    --config-name wiki103</code></pre></div>\n<p>where /path/to/external/configs/wiki103.yaml contains:</p>\n<div class=\"gatsby-highlight\" data-language=\"yaml\"><pre class=\"language-yaml\"><code class=\"language-yaml\"><span class=\"token comment\"># @package _group_</span>\n\n<span class=\"token key atrule\">model</span><span class=\"token punctuation\">:</span>\n  <span class=\"token key atrule\">_name</span><span class=\"token punctuation\">:</span> transformer_lm\n<span class=\"token key atrule\">distributed_training</span><span class=\"token punctuation\">:</span>\n  <span class=\"token key atrule\">distributed_world_size</span><span class=\"token punctuation\">:</span> <span class=\"token number\">1</span>\n<span class=\"token key atrule\">dataset</span><span class=\"token punctuation\">:</span>\n  <span class=\"token key atrule\">batch_size</span><span class=\"token punctuation\">:</span> <span class=\"token number\">2</span>\n<span class=\"token key atrule\">task</span><span class=\"token punctuation\">:</span>\n  <span class=\"token key atrule\">_name</span><span class=\"token punctuation\">:</span> language_modeling\n  <span class=\"token key atrule\">data</span><span class=\"token punctuation\">:</span> /path/to/data\n  <span class=\"token key atrule\">add_bos_token</span><span class=\"token punctuation\">:</span> <span class=\"token boolean important\">false</span>\n  <span class=\"token key atrule\">max_target_positions</span><span class=\"token punctuation\">:</span> <span class=\"token number\">1024</span>\n<span class=\"token key atrule\">optimization</span><span class=\"token punctuation\">:</span>\n  <span class=\"token key atrule\">max_update</span><span class=\"token punctuation\">:</span> <span class=\"token number\">50000</span>\n  <span class=\"token key atrule\">lr</span><span class=\"token punctuation\">:</span> <span class=\"token punctuation\">[</span> <span class=\"token number\">0.25</span> <span class=\"token punctuation\">]</span>\n<span class=\"token key atrule\">criterion</span><span class=\"token punctuation\">:</span> cross_entropy\n<span class=\"token key atrule\">optimizer</span><span class=\"token punctuation\">:</span> adam\n<span class=\"token key atrule\">lr_scheduler</span><span class=\"token punctuation\">:</span>\n  <span class=\"token key atrule\">_name</span><span class=\"token punctuation\">:</span> cosine</code></pre></div>","htmlAst":{"type":"root","children":[{"type":"element","tagName":"h2","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://github.com/pytorch/fairseq/blob/master/docs/hydra_integration.md"},"children":[{"type":"text","value":"Fairseq’s Hydra"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Fairseq이 0.10.1로 버젼 업그레이드를 하면서 configuration 관리를 Hydra로 하게됨."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Fairseq을 실행시키는 command line상에서도, 모델 코드 상에서도 몇 가지 변화가 있음."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"hr","properties":{},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Creating or migrating components"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"기존 모델 코드는 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"add_args()"}]},{"type":"text","value":"라는 static method로 config를 관리"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","decorator","annotation","punctuation"]},"children":[{"type":"text","value":"@staticmethod"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"def"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","function"]},"children":[{"type":"text","value":"add_args"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"parser"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","triple-quoted-string","string"]},"children":[{"type":"text","value":"\"\"\"Add model-specific arguments to the parser.\"\"\""}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# fmt: off"}]},{"type":"text","value":"\n    parser"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"add_argument"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'--dropout'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"type"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"float"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" metavar"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'D'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n                        "},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"help"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'dropout probability'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n    parser"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"add_argument"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'--encoder-embed-dim'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"type"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"int"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" metavar"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'N'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n                        "},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"help"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'encoder embedding dimension'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n    parser"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"add_argument"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'--encoder-embed-path'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"type"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"str"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" metavar"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'STR'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n                        "},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"help"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'path to pre-trained encoder embedding'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"이런 방식은 코드를 읽을 때, 어떤 argument들이 사용되는지 한 번에 보기가 어렵다는 단점이 있음.\n→ 내부적으로 사용되는 모듈들을 하나하나 찾아가서 확인해봐야함.\n→ 복잡한 모델일수록 더욱 코드 읽기가 어려워짐."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"@dataclass"}]},{"type":"text","value":" 데코레이터를 이용하는 class를 이용하는 방법으로 수정 (Tasks & Models)"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","decorator","annotation","punctuation"]},"children":[{"type":"text","value":"@dataclass"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"class"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","class-name"]},"children":[{"type":"text","value":"Wav2Vec2Config"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"FairseqDataclass"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n    extractor_mode"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" EXTRACTOR_MODE_CHOICES "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" field"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"\n        default"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"default\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n        metadata"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"{"}]},{"type":"text","value":"\n            "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"help\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"mode for feature extractor. default has a single group norm with d \""}]},{"type":"text","value":"\n            "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"groups in the first conv block, whereas layer_norm has layer norms in \""}]},{"type":"text","value":"\n            "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"every block (meant to use with normalize=True)\""}]},{"type":"text","value":"\n        "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"}"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n    encoder_layers"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"int"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" field"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"\n        default"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"12"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" metadata"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"{"}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"help\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"num encoder layers in the transformer\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"}"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n    encoder_embed_dim"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"int"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" field"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"\n        default"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"768"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" metadata"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"{"}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"help\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"encoder embedding dimension\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"}"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n    encoder_ffn_embed_dim"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"int"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" field"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"\n        default"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"3072"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" metadata"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"{"}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"help\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"encoder embedding dimension for FFN\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"}"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"FairseqDataclass를 상속받는 클래스를 정의해서 사용 → "},{"type":"element","tagName":"a","properties":{"href":"https://github.com/pytorch/fairseq/blob/master/fairseq/models/wav2vec/wav2vec2.py"},"children":[{"type":"text","value":"Example"}]}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Task Example"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","decorator","annotation","punctuation"]},"children":[{"type":"text","value":"@dataclass"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"class"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","class-name"]},"children":[{"type":"text","value":"LanguageModelingConfig"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"FairseqDataclass"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n    data"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" Optional"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"str"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" field"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"\n        default"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","boolean"]},"children":[{"type":"text","value":"None"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" metadata"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"{"}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"help\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"path to data directory\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"}"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"\n\n"},{"type":"element","tagName":"span","properties":{"className":["token","decorator","annotation","punctuation"]},"children":[{"type":"text","value":"@register_task"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"language_modeling\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" dataclass"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":"LanguageModelingConfig"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"class"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","class-name"]},"children":[{"type":"text","value":"LanguageModelingTask"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"LegacyFairseqTask"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","decorator","annotation","punctuation"]},"children":[{"type":"text","value":"@classmethod"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"def"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","function"]},"children":[{"type":"text","value":"setup_task"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"cls"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" cfg"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" LanguageModelingConfig"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n        "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Model Example"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","decorator","annotation","punctuation"]},"children":[{"type":"text","value":"@dataclass"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"class"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","class-name"]},"children":[{"type":"text","value":"TransformerLanguageModelConfig"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"FairseqDataclass"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n    activation_fn"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" ChoiceEnum"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"utils"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"get_available_activation_fns"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" field"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"\n        default"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"relu\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" metadata"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"{"}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"help\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"activation function to use\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"}"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n    dropout"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"float"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" field"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"default"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" metadata"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"{"}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"help\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"dropout probability\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"}"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"\n\n"},{"type":"element","tagName":"span","properties":{"className":["token","decorator","annotation","punctuation"]},"children":[{"type":"text","value":"@register_model"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"transformer_lm\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" dataclass"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":"TransformerLanguageModelConfig"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"class"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","class-name"]},"children":[{"type":"text","value":"TransformerLanguageModel"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"FairseqLanguageModel"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","decorator","annotation","punctuation"]},"children":[{"type":"text","value":"@classmethod"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"def"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","function"]},"children":[{"type":"text","value":"build_model"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"cls"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" cfg"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" TransformerLanguageModelConfig"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" task"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" FairseqTask"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n        "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"\n"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"기존 방법("},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"add_args()"}]},{"type":"text","value":")은 여전히 서포트 되지만, 이후 언젠가 deprecated 될 것이라고 함."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"hr","properties":{},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Training with "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"fairseq-hydra-train"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"configuraion 관리를 hydra로 하기 때문에 기존에 사용했던 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"fairseq-train"}]},{"type":"text","value":"이 아닌 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"fairseq-hydra-train"}]},{"type":"text","value":" training"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Override default values through command line:"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"$ fairseq-hydra-train \\\n    distributed_training.distributed_world_size=1 \\\n    dataset.batch_size=2 \\\n    task.data=data-bin \\\n    model=transformer_lm/transformer_lm_gpt \\\n    task=language_modeling \\\n    optimization.max_update=5000"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Replace bundled configs with an external config:"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"$ fairseq-hydra-train \\\n    --config-dir /path/to/external/configs \\\n    --config-name wiki103"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"where /path/to/external/configs/wiki103.yaml contains:"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"yaml"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-yaml"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-yaml"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# @package _group_"}]},{"type":"text","value":"\n\n"},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"model"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n  "},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"_name"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" transformer_lm\n"},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"distributed_training"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n  "},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"distributed_world_size"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"dataset"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n  "},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"batch_size"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"task"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n  "},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"_name"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" language_modeling\n  "},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"data"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" /path/to/data\n  "},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"add_bos_token"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","boolean","important"]},"children":[{"type":"text","value":"false"}]},{"type":"text","value":"\n  "},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"max_target_positions"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1024"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"optimization"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n  "},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"max_update"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"50000"}]},{"type":"text","value":"\n  "},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"lr"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.25"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"criterion"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" cross_entropy\n"},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"optimizer"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" adam\n"},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"lr_scheduler"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n  "},{"type":"element","tagName":"span","properties":{"className":["token","key","atrule"]},"children":[{"type":"text","value":"_name"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" cosine"}]}]}]}],"data":{"quirksMode":false}},"excerpt":"Fairseq’s Hydra Fairseq이 0.10.1로 버젼 업그레이드를 하면서 configuration 관리를 Hydra로 하게됨. Fairseq을 실행시키는 command line…","fields":{"readingTime":{"text":"3 min read"}},"frontmatter":{"title":"Sooftware NLP - Fairseq Hydra","userDate":"31 December 2020","date":"2020-12-31T10:00:00.000Z","tags":["toolkit","nlp"],"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#f8f8f8","images":{"fallback":{"src":"/static/1b089752085416a626f169c3547274ad/de522/fairseq.png","srcSet":"/static/1b089752085416a626f169c3547274ad/eb6e7/fairseq.png 750w,\n/static/1b089752085416a626f169c3547274ad/85af9/fairseq.png 1080w,\n/static/1b089752085416a626f169c3547274ad/de522/fairseq.png 1132w","sizes":"100vw"},"sources":[{"srcSet":"/static/1b089752085416a626f169c3547274ad/8ad53/fairseq.webp 750w,\n/static/1b089752085416a626f169c3547274ad/9f48c/fairseq.webp 1080w,\n/static/1b089752085416a626f169c3547274ad/e12ec/fairseq.webp 1132w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.9946996466431096}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"children":[{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#282838","images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/248f9/soohwan.png 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/fd435/soohwan.png 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png 120w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/e7f45/soohwan.webp 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/589ec/soohwan.webp 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/71a38/soohwan.webp 120w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6833333333333333}}]}}]}},"relatedPosts":{"totalCount":25,"edges":[{"node":{"id":"f2a32d9d-b206-5141-8c2d-5440e19caf65","excerpt":"llama.cpp (On device llm inference tool) 최근에 llama.cpp를 사용해봤는데, 상당히 편리하고 미래에 더 많이 쓰일 툴이라는 생각이 들어서 기록해둔다! llama.cpp란? 대표적인 오픈소스 LLM인 Meta…","frontmatter":{"title":"llama.cpp (On device llm inference tool)","date":"2024-09-07T10:00:00.000Z"},"fields":{"readingTime":{"text":"7 min read"},"slug":"/llama-cpp/"}}},{"node":{"id":"fa28b462-1fe2-5d0f-932b-f28e32f1da18","excerpt":"Elastic Search logstash - Nori 토크나이저 설정 이번에 회사에서 검색 기능을 구현하면서 Elastic Search를 다루게 됐다. 이 엔진을 다루면서 삽질을 많이 했는데, 다음에는 하지 않도록 기록용으로 남겨둔다. Elastic…","frontmatter":{"title":"[ELK] Elastic Search logstash - Nori 토크나이저 설정","date":"2024-08-24T10:00:00.000Z"},"fields":{"readingTime":{"text":"4 min read"},"slug":"/elastic-search/"}}},{"node":{"id":"d1f4a9ee-591d-5e95-a930-a1bf17701b0e","excerpt":"단 30줄로 ChatGPT 웹페이지 만들기 (Streamlit chat_message) Streamlit은 파이썬 기반의 오픈소스 웹 UI 라이브러리입니다. 매우 간단한 코드로 손쉽게 웹페이지를 띄울 수 있어서 간단한 데모나 PoC…","frontmatter":{"title":"단 30줄로 ChatGPT 웹페이지 만들기 (Streamlit chat_message)","date":"2024-05-01T10:00:00.000Z"},"fields":{"readingTime":{"text":"9 min read"},"slug":"/streamlit-chatgpt/"}}},{"node":{"id":"4e37b604-dce8-5e09-85e8-337d4f03f0cb","excerpt":"Terabyte(TB) 단위 데이터 셔플링 - terashuf 리눅스에서 TB 단위의 데이터를 line 기준으로 셔플링이 필요할 때가 (가끔) 있다. 직접 코딩해서 쓰기에는 메모리, 속도 등을 신경써야해서 생각보다 큰 작업인데, terashuf…","frontmatter":{"title":"Terabyte(TB) 단위 데이터 셔플링 - terashuf","date":"2024-03-20T10:00:00.000Z"},"fields":{"readingTime":{"text":"1 min read"},"slug":"/terashuf/"}}},{"node":{"id":"ec8327c3-3c4d-51da-9440-8cb74fd1effe","excerpt":"tmux - conf Tmux 설정을 위한 tmux configuraion 기록","frontmatter":{"title":"tmux - conf","date":"2024-02-15T10:00:00.000Z"},"fields":{"readingTime":{"text":"2 min read"},"slug":"/tmux-conf/"}}}]}},"pageContext":{"slug":"/fairseq-hydra/","prev":{"excerpt":"2020년 회고 다사다난했던 2020년이 지나고 어느덧 2021년 새해가 밝았습니다. 🤗 🤗 코로나라는 세계적인 재앙 때문에 생활부터 모든게 많이 달라진 한 해 였습니다. 여태까지…","frontmatter":{"title":"2020년 회고","tags":["record"],"date":"2020-12-31T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAAAsTAAALEwEAmpwYAAADPklEQVQ4yzWT/U/TVxTGv1QCIbQIThYXFSjlbUrZooMC6hwEAkmR1paCLYGCthUoraW8Q6GUFlpaGVJBWsEiaud7XGKWmCzL/rTP8r26H27OzX15znPO8xxpYGCQsTEP6+EYoXAM39QcU/45orFtdpMHLAdCBFbWCUe2CK5tsLS8xuz8MvHEI3Env7979z4jIw6G7feQLBYri0tBtuI7xOI74pMct3f2WFhcZWMzQTgS5yD1jOTjlEi2tBxk/8mRSDQzs4R/eoFJzxRW6xCSzNDlmsDrnWZ+IcBaaJO5bwxWgxHBPLIRF5/r6n6kpKQErfYn+ix32Nl9wkE6I9jL94ODdqS+vjs4nRPY7fcEaGI7KUBlhvJeZhjb+h2j0UxRURHFxcWUlZVRWakh/eyYL//8y+cvf5N9+5HDzAuk0VEnHo9fZNh9nGJv/5BUOiMyZ998JPM8y1HmJbduGamprkWlKuL8+QuoKyqpqFBTr23ANjiEa8zNrzfbkBzOr+XeH5tkfmGF5F6ap0cnvMi+42X2PSev3pI+fI5O14LBaEbX3CIAtdoGVEoVBQUFSJJEIBji2vUbSN1desbHvaLs/9WMbe2wlXjEZvQhiYdJnK4JmpqaSe6nMBhNXLxwEbW6ksLCQnJzc2nU6fj0+S+8Pj+S0dgnypSbLzc59fSY45PXIr764wNv3v8pmJ09W0p1dQ2nFArOnPmOy5e1AjAnJ4eeXgORaIzVUBipra0Dm20Yq20Ys3kA39SssJEsyEEqQ/b1B6y2IfLy8rl0qZ7e3tvcNlnQaKpRKlV0dnbR02Ogu1tPa+t1pJbma7S23uC3m+10delpb+/Eah3G7fYxPbNINLqNydTPuXM/CJDa2jrBVN7n5eejrtQIG9XU1ArlJaPBjH3EwcioE5fLjd3uwOEYZ2LiAe5JH37/vPBXVVWNsI0sgFymLExp6fecLjot+qg4pUChUCD199sE0APfLHPzAQIrYSGOvOTRkpWX2Wo0VSgLlSiVSsrLKwQjvd5AR0e3OCsrK+fKlV+QTOZ+oaLH6/8GuE5oPfp1ttejrIWiwvRXrzYKceR5NZkswoPahp9pamoRDmhs1FFf38B/iywfcoGnX+MAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/c6c1288528036ea9df3176dbafeffabf/b444b/2020.png","srcSet":"/static/c6c1288528036ea9df3176dbafeffabf/b444b/2020.png 600w","sizes":"100vw"},"sources":[{"srcSet":"/static/c6c1288528036ea9df3176dbafeffabf/9ff6b/2020.webp 600w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6666666666666666}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"21 min read"},"layout":"","slug":"/2020/"}},"next":{"excerpt":"Pororo: A Deep Learning based Multilingual Natural Language Processing Library Link: https://github.com/kakaobrain/pororo…","frontmatter":{"title":"Sooftware NLP - Pororo: A Deep Learning based Multilingual Natural Language Processing Library","tags":["nlp","toolkit","record"],"date":"2021-02-02T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAgCAYAAAASYli2AAAACXBIWXMAAAsTAAALEwEAmpwYAAAIfElEQVRIx43Va1CU1x0G8GcvXERaEDBqGKMN8YImQc2ltWkzTROrTZtIjG2jbY1B1CgKCspNqkEEDUY0CCKKVy4RDIqIgkQSCVdBBIRFURQxiAK77LKwwO6+73k6a2diOv3S5+OZc37nOXM+/NHf4QfyXaxNeUOp/Pg4MEtr/8yG08t9I2Lz529Lasfq/IHs7xeOcNizy6x94YqkmxY+0rfGRdbOhKx1U1ELsBcY6VuIJ9HdjwReGqNYuudN2C9LnTdnQ9KtS7Fv8uqBhZQOPU+fmHyuyIggzdNp1ftQ6GdQ1k25L+nmvCJpJ0LWeahkLWDVTYSQGgHegYIEOOS0aWmCP71CTpMJsPbum2wN2btPxpqrwntnmqDBQ4i+ZyRh+IWZfVModJ4DFp3PTKtuMiTdc0qLbgpGunwA821Af+eF9dSCOSWvC/iVWg/GB3D3nmDhFX9ErDr2iTjyzW/FyEMIdkPI3Uqyz87MPifKWo/rZUVU9DEb1sHZMJbMAXT1M2eO3LOXbWDXPTdpauQebspcIZo1zwrbGuVRJL3I4fcoaxcLPp4n5PaZFNpRFupASTvOX+h/BrnXWTXSOR8w1P38S/keyIcKi9wO9t9VC5oUpDSF7F/Eq43rmJIbztaU2ew9NoNdx+eI/jQlLWVjJaFzpKx1rOzdDTADENJfAH2dQ524B5o1kOUuF8HBuaKlyZ+bT26n98bPOHlrHhM2f0DjES+aBzppMslsL8sSfamjxchxtdwXo+T94PELfwgeg4dhP1dCX2vXPdQEmlrdxXDtLNG5bxKL/B0YE7WCx6/r2DVoIdMmUWpPo0kmhTTIYZLt51ZyIBZWXbo9H6W7BWhzFNB+ZaeCvkbdo69VU3vUUTTvfE08uF5E3k0lj9uTmk2knEHjiYlkSxyDo77gyWPHaEvnhQAxcEgtyUY3UqdcSQMg+lQq6BocbhgrwLYItXzjev2TzQ/0ZEfxeg5luXLYUkFDSThNe0fzTm0iHz66Q3NnLvUJEIYrO6Th7pco3UawDbQ0jVKjp97tcNcxsGPXJGv/oEXcud0ssk7l8WZ5Ak1JLrReW0RqPMls0JLqzqGvX6PhyPNiuHIvH5to7dDMJdvhy1KABa8C5YFjgquWKNm7DRaDJpcGicLWUq5aQcN2sOMgmBsIXlwLPjg0l8OP7wjz0PCTLZouCzNOvTMwVGAXpM99JnmozPtlXFzg7dWWafe454iahnh7ebD4bxwseFP0HXmFLTtfZrg3OB+O/Gw22J8zVsgWI21/YxMzvmvm1gBP3b1EldR7FBw579yGjhzA3OW8rKfOk11XDlr78/yE/ts4MWSheFR9gNUbISqDQEMGhKlyCa1WGybxns7CxXGZWaWxDncNaeDtnUreT3bNwbX1UPARYLqB0w2anXxMWmy3W0kxaJZourlXDNctEPr6CFpMelsz8XhQcPXhkjr4FdR/+HkUdSccO1oTxy/mXQDG1pkKchLIBLvGQpcraVm72dxDaeTJoyhs+rAgh0jRTYryNj3/vju3Df45bYj6nsrQkpr9kb6eVfETUHYJChhvTcBAg7uSA64glztlhbo2HYj5qOGLsxUt6SWNLK67JRfUtYszJZXidOpnPLjmV2LUmhMSosppv7nwh9eDD419LywWe3YssOOpV4H+5jG4vNFTcXTxVBz9+PlJxnSHRze3Q/aa96HR5e0AJi4aLypW2on2DWDrBoi/hocJRF+jelMh3ddnveW07ix+HbHHbvrmJCyK3AiYevajfper6vouN9TEjZ1/P1nJgg0KTnl/FfFBspjmF8t9Ye/wX3GfiPHRF4jtDVZElNBxQ95RBBUDIaVqrH4EVUgxvCOzAeo0qE+eBTIdZIO6OMz5XPZaB/GC79oBLDtJBJcIhNcKRDcTsXVUxl21IOqKDdyKtfnA+gLVc+EFmBDy1X9GQM93v0dr+hw0pvoouuNGg7pN6oyY+TMU/zjcgHV5VIRdllRbv6UqvpaKhEZiT6MVMZW0DzmfhbBqIKpe4bExBz9GW7MEveV/xuPv3oVtFGBRPkYtP+irXJ1FhFySEFEisK2MsGH7bxKJzRLiqmi36UIp4tuAY4MYveHMT8B6f+ivL0FXzT/t2m74jYtLmQcsySxAwBli8zdWhF4moiuJxBtEyi0iuUXCzmrahVyoQtIPQDHhHJT3FOzOdlN2ZbjjYabnLw2Fo/utFa5blm5bcV+xKoMIKZYRWiywo5LYf4M4oBFI1kiIq6Y6uKAUu24Cx/vhFPiThp1ZExV3M33Qljl7XO8J9YAuW0EWQayK+UhgVaGsCiuSEVslkFBvaymwr1FCdDntg/LOIbga2NKscA089RQciAY6P4eqcxfwYO+ETzviVdQlwVocP1HC8gwiKJ+IKpURUynbmiGu2oLQSxwVkL0NK08Da86qpoamPwXljcC11C/xcDUUD6KAW9EeO9q3gGWR7rT/5Egf/LMMioBcYuMFKsK+kRBeQkVgnmn8p4emua86DPd1J5XOq9OegvfPL0ZLyvto2vs2+CLQvhJoinQPqg0fF4OoZrfpIceeHbP6qC/8T3UhsJAILuLowNyLWJMDrD2nBIgtSUufgvrqA7ibMA0tn78ETeJb4EJAEwRUfgpMCD2LZZHbgD82IH6FT62nXwKxslCc3Ps7Iyswg9WA8cxkpbHUFf+V1qwV0MRORWPiH1CX7ItbwVBWBUC5P+ZPqpF9KlyOe3mqIQXG5iQnFiV6WeSLoLnEZRcvAXIBVANVv8H/pLM2E7U7ZuPa9hdRFz8X9bGzYMx3VfZ97QH9aY9XLSVq8hrIUgyJStBa7XSeNQCroBiqmoj/K0MV09FfPh/a75c5mstdvx6qcEntLvJeYCl3bjLXePpKVY6wVjgoTQ1v/Hjo30NEdAdoCNSmAAAAAElFTkSuQmCC"},"images":{"fallback":{"src":"/static/7ee72db9d9d5f47d514ac31fc67c7770/1c53b/pororo.png","srcSet":"/static/7ee72db9d9d5f47d514ac31fc67c7770/1c53b/pororo.png 515w","sizes":"100vw"},"sources":[{"srcSet":"/static/7ee72db9d9d5f47d514ac31fc67c7770/92941/pororo.webp 515w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":1.6}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"2 min read"},"layout":"","slug":"/pororo/"}},"primaryTag":"toolkit"}},
    "staticQueryHashes": ["3170763342","3229353822"]}