{
    "componentChunkName": "component---src-templates-post-tsx",
    "path": "/wandb_image/",
    "result": {"data":{"logo":null,"markdownRemark":{"html":"<h1>Wandb (Weights &#x26; Bias) Image Log</h1>\n<p>Wandb 라이브러리는 최근에 가장 편리하면서도 파워풀한 logging 라이브러리입니다.<br>\nNLP에서 많이 쓰이는 PyTorch, PyTorch-Lightning, Huggingface Transformers 등에서도 쉽게 사용이 가능합니다.</p>\n<p><img src=\"https://user-images.githubusercontent.com/42150335/137149370-e8de77b9-83a2-46ea-89ca-d444ba258d80.png\" width=\"800\">.</p>\n<p>보통 이런 logging 라이브러리는 딥러닝 모델 학습시 Loss 혹은 Metric Score를 모니터링할 때 사용합니다.<br>\n여기서 조금 더 모델 학습을 면밀하게 보기 위해 모델의 어텐션 맵을 그려본다거나, 생성한 Mel-Spectrogram을 그려본다거나 하는 등의 logging 방식도 생각해볼 수 있습니다.</p>\n<p>예를 들어 100 스텝마다 모델의 어텐션 맵을 그려보면서 어텐션이 어떻게 학습 되는지를 확인할 수도 있고, 모델이 생성하는 Mel-Spectrogram을 이미지로 보면서 학습 진행 과정을 볼 수도 있습니다.</p>\n<p>실제로 well-made 딥러닝 학습 코드의 경우는 이런 어텐션 맵 등을 학습 중간중간 저장하도록 짜여져 있는 경우가 많습니다.</p>\n<p>그런데 리눅스 서버에서 학습하는 경우는 일일이 노트북으로 이미지를 옮겨서 확인하는 것도 귀찮습니다.</p>\n<p>그렇기 때문에 웹 기반으로 log가 관리되는 wandb의 경우는 이런 이미지를 스텝별로 저장해놓으면 상당히 유용합니다.</p>\n<img src=\"https://user-images.githubusercontent.com/42150335/137151370-5df2fd57-76f3-4351-9166-d98307810b33.png\" width=\"600\">\n<p>위와 같이 모델이 생성한 Mel-Spectrogram, Attention Map 등을 확인해보면 모델이 학습되는 과정을 좀 더 확실하게 알 수 있습니다.</p>\n<h2>Wandb Image Log 찍는 법</h2>\n<p>Wandb로 image log를 찍는건 굉장히 간단합니다.</p>\n<p>pandas 라이브러리의 DataFrame을 이용해서 Confusion Matrix도 찍을 수 있습니다만, 여기서는 Image 찍는 법만 다루겠습니다.</p>\n<p>Image를 찍기 위해서는 matplotlib 라이브러리를 이용해서 이미지를 저장하고 해당 이미지 파일을 넘겨주기만 하면 됩니다.</p>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token keyword\">import</span> wandb\n<span class=\"token keyword\">import</span> matplotlib<span class=\"token punctuation\">.</span>pyplot <span class=\"token keyword\">as</span> plt\n\noutputs<span class=\"token punctuation\">,</span> attention_map <span class=\"token operator\">=</span> model<span class=\"token punctuation\">(</span>inputs<span class=\"token punctuation\">)</span>\n\nplt<span class=\"token punctuation\">.</span>imshow<span class=\"token punctuation\">(</span>attention_map<span class=\"token punctuation\">,</span> aspect<span class=\"token operator\">=</span><span class=\"token string\">'auto'</span><span class=\"token punctuation\">,</span> origin<span class=\"token operator\">=</span><span class=\"token string\">'lower'</span><span class=\"token punctuation\">,</span> interpolation<span class=\"token operator\">=</span><span class=\"token string\">'none'</span><span class=\"token punctuation\">)</span>\nplt<span class=\"token punctuation\">.</span>savefig<span class=\"token punctuation\">(</span><span class=\"token string\">'attention_map.png'</span><span class=\"token punctuation\">,</span> figsize<span class=\"token operator\">=</span><span class=\"token punctuation\">(</span><span class=\"token number\">16</span><span class=\"token punctuation\">,</span> <span class=\"token number\">4</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">)</span>\n\nwandb<span class=\"token punctuation\">.</span>log<span class=\"token punctuation\">(</span><span class=\"token punctuation\">{</span>\n    <span class=\"token string\">\"Attention Map\"</span><span class=\"token punctuation\">:</span> <span class=\"token punctuation\">[</span>\n        wandb<span class=\"token punctuation\">.</span>Image<span class=\"token punctuation\">(</span><span class=\"token string\">'attention_map.png'</span><span class=\"token punctuation\">)</span>\n    <span class=\"token punctuation\">]</span>\n<span class=\"token punctuation\">}</span><span class=\"token punctuation\">)</span></code></pre></div>\n<p>위 코드를 학습하면서 주기적으로 호출해주기만 하면 Attention Map이 어떻게 학습되어 가는지를 확인할 수 있습니다.</p>","htmlAst":{"type":"root","children":[{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"Wandb (Weights & Bias) Image Log"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Wandb 라이브러리는 최근에 가장 편리하면서도 파워풀한 logging 라이브러리입니다."},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\nNLP에서 많이 쓰이는 PyTorch, PyTorch-Lightning, Huggingface Transformers 등에서도 쉽게 사용이 가능합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/42150335/137149370-e8de77b9-83a2-46ea-89ca-d444ba258d80.png","width":800},"children":[]},{"type":"text","value":"."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"보통 이런 logging 라이브러리는 딥러닝 모델 학습시 Loss 혹은 Metric Score를 모니터링할 때 사용합니다."},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\n여기서 조금 더 모델 학습을 면밀하게 보기 위해 모델의 어텐션 맵을 그려본다거나, 생성한 Mel-Spectrogram을 그려본다거나 하는 등의 logging 방식도 생각해볼 수 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"예를 들어 100 스텝마다 모델의 어텐션 맵을 그려보면서 어텐션이 어떻게 학습 되는지를 확인할 수도 있고, 모델이 생성하는 Mel-Spectrogram을 이미지로 보면서 학습 진행 과정을 볼 수도 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"실제로 well-made 딥러닝 학습 코드의 경우는 이런 어텐션 맵 등을 학습 중간중간 저장하도록 짜여져 있는 경우가 많습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"그런데 리눅스 서버에서 학습하는 경우는 일일이 노트북으로 이미지를 옮겨서 확인하는 것도 귀찮습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"그렇기 때문에 웹 기반으로 log가 관리되는 wandb의 경우는 이런 이미지를 스텝별로 저장해놓으면 상당히 유용합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/42150335/137151370-5df2fd57-76f3-4351-9166-d98307810b33.png","width":600},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"위와 같이 모델이 생성한 Mel-Spectrogram, Attention Map 등을 확인해보면 모델이 학습되는 과정을 좀 더 확실하게 알 수 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Wandb Image Log 찍는 법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Wandb로 image log를 찍는건 굉장히 간단합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"pandas 라이브러리의 DataFrame을 이용해서 Confusion Matrix도 찍을 수 있습니다만, 여기서는 Image 찍는 법만 다루겠습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Image를 찍기 위해서는 matplotlib 라이브러리를 이용해서 이미지를 저장하고 해당 이미지 파일을 넘겨주기만 하면 됩니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" wandb\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" matplotlib"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"pyplot "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"as"}]},{"type":"text","value":" plt\n\noutputs"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" attention_map "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" model"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"inputs"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\nplt"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"imshow"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"attention_map"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" aspect"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'auto'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" origin"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'lower'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" interpolation"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'none'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\nplt"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"savefig"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'attention_map.png'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" figsize"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"16"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"4"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\nwandb"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"log"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"{"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"Attention Map\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"text","value":"\n        wandb"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"Image"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'attention_map.png'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"}"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"위 코드를 학습하면서 주기적으로 호출해주기만 하면 Attention Map이 어떻게 학습되어 가는지를 확인할 수 있습니다."}]}],"data":{"quirksMode":false}},"excerpt":"Wandb (Weights & Bias) Image Log Wandb 라이브러리는 최근에 가장 편리하면서도 파워풀한 logging 라이브러리입니다. NLP에서 많이 쓰이는 PyTorch, PyTorch-Lightning, Huggingface…","fields":{"readingTime":{"text":"3 min read"}},"frontmatter":{"title":"Sooftware ML - Wandb Image Log","userDate":"13 October 2021","date":"2021-10-13T22:00:00.000Z","tags":["toolkit","logging"],"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#f8f8f8","images":{"fallback":{"src":"/static/fd6ffa741fe53de299a57e6a8852f68d/2add8/wandb_image.png","srcSet":"/static/fd6ffa741fe53de299a57e6a8852f68d/5a15a/wandb_image.png 750w,\n/static/fd6ffa741fe53de299a57e6a8852f68d/20c21/wandb_image.png 1080w,\n/static/fd6ffa741fe53de299a57e6a8852f68d/397a5/wandb_image.png 1366w,\n/static/fd6ffa741fe53de299a57e6a8852f68d/2add8/wandb_image.png 1920w","sizes":"100vw"},"sources":[{"srcSet":"/static/fd6ffa741fe53de299a57e6a8852f68d/d7d73/wandb_image.webp 750w,\n/static/fd6ffa741fe53de299a57e6a8852f68d/525de/wandb_image.webp 1080w,\n/static/fd6ffa741fe53de299a57e6a8852f68d/f312c/wandb_image.webp 1366w,\n/static/fd6ffa741fe53de299a57e6a8852f68d/42023/wandb_image.webp 1920w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5359375000000001}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"children":[{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#282838","images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/248f9/soohwan.png 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/fd435/soohwan.png 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png 120w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/e7f45/soohwan.webp 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/589ec/soohwan.webp 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/71a38/soohwan.webp 120w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6833333333333333}}]}}]}},"relatedPosts":{"totalCount":25,"edges":[{"node":{"id":"f2a32d9d-b206-5141-8c2d-5440e19caf65","excerpt":"llama.cpp (On device llm inference tool) 최근에 llama.cpp를 사용해봤는데, 상당히 편리하고 미래에 더 많이 쓰일 툴이라는 생각이 들어서 기록해둔다! llama.cpp란? 대표적인 오픈소스 LLM인 Meta…","frontmatter":{"title":"llama.cpp (On device llm inference tool)","date":"2024-09-07T10:00:00.000Z"},"fields":{"readingTime":{"text":"7 min read"},"slug":"/llama-cpp/"}}},{"node":{"id":"fa28b462-1fe2-5d0f-932b-f28e32f1da18","excerpt":"Elastic Search logstash - Nori 토크나이저 설정 이번에 회사에서 검색 기능을 구현하면서 Elastic Search를 다루게 됐다. 이 엔진을 다루면서 삽질을 많이 했는데, 다음에는 하지 않도록 기록용으로 남겨둔다. Elastic…","frontmatter":{"title":"[ELK] Elastic Search logstash - Nori 토크나이저 설정","date":"2024-08-24T10:00:00.000Z"},"fields":{"readingTime":{"text":"4 min read"},"slug":"/elastic-search/"}}},{"node":{"id":"d1f4a9ee-591d-5e95-a930-a1bf17701b0e","excerpt":"단 30줄로 ChatGPT 웹페이지 만들기 (Streamlit chat_message) Streamlit은 파이썬 기반의 오픈소스 웹 UI 라이브러리입니다. 매우 간단한 코드로 손쉽게 웹페이지를 띄울 수 있어서 간단한 데모나 PoC…","frontmatter":{"title":"단 30줄로 ChatGPT 웹페이지 만들기 (Streamlit chat_message)","date":"2024-05-01T10:00:00.000Z"},"fields":{"readingTime":{"text":"9 min read"},"slug":"/streamlit-chatgpt/"}}},{"node":{"id":"4e37b604-dce8-5e09-85e8-337d4f03f0cb","excerpt":"Terabyte(TB) 단위 데이터 셔플링 - terashuf 리눅스에서 TB 단위의 데이터를 line 기준으로 셔플링이 필요할 때가 (가끔) 있다. 직접 코딩해서 쓰기에는 메모리, 속도 등을 신경써야해서 생각보다 큰 작업인데, terashuf…","frontmatter":{"title":"Terabyte(TB) 단위 데이터 셔플링 - terashuf","date":"2024-03-20T10:00:00.000Z"},"fields":{"readingTime":{"text":"1 min read"},"slug":"/terashuf/"}}},{"node":{"id":"ec8327c3-3c4d-51da-9440-8cb74fd1effe","excerpt":"tmux - conf Tmux 설정을 위한 tmux configuraion 기록","frontmatter":{"title":"tmux - conf","date":"2024-02-15T10:00:00.000Z"},"fields":{"readingTime":{"text":"2 min read"},"slug":"/tmux-conf/"}}}]}},"pageContext":{"slug":"/wandb_image/","prev":{"excerpt":"NLP Metrics Confusion Matrix Confusion Matrix는 분류 모델을 평가할때 모델이 얼마나 정밀한지, 얼마나 실용적인 분류를 해냈는지, 얼마나 정확한 분류를 해냈는지에 대한 모든 내용을 포함하고 있습니다. Accuracy…","frontmatter":{"title":"Sooftware NLP - NLP Metrics","tags":["nlp","metric"],"date":"2021-10-13T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAYAAAC0VX7mAAAACXBIWXMAAAsTAAALEwEAmpwYAAACK0lEQVQoz1WRC2+iQBSF+f9/pm12a3RbBBVFXlbrKPhArQIrYEEjgqIzZwNpTfckX87kzs3MmTscY2wPIAOQFjDGSj+fz2kURXmaptfT6XRntVrdOR6PZS1JkjzLMur7/o5jjOX4IcZY6efzGXn+31Yp0zQxn88xGo3uvd/KsmxfHHguNtiXKKVFF8uyjJnjMbPMApNNLKt0ez4rfTGfs4lpsulXfWJZ6Ol6zBVhvsP9vO16veLN6MFQNXRaEqqVCrSuAl1R0RBE6KqObkfG86/fEOsiK3qbYjPmTqfTJU3TIi47Ho847PdIkgRhEEA3VLz1dRg9FYraua/bkohhXykxlBYGhsxGAw2yVI+53W53o5SCMsZm0ylajQbarVZJ31ZAXB2DD6Wkv5RBvB7sjyYSj0ewrCHevCJYVtnB5REuazGXJMnNcRxEUcQ8z0NHkiDwPBRZxtgdYODo0GwZ2kKGbDUw3A7gBQquoYD1+BHxugZ38sQKTzw+5g6Hw/Xh4YESQqiz2dCGIFBNVWlTFOnEfKWfoUyDbYeGgUx3YZduPYla01e6+tulG1+l661KnUC7OYEGMuU/ixmimF3xCa7jlE/mX17KhPGmgWhVQWA/w7WesDEfcXRqCHdKmVSdtdGdSpDGIoxlF0Ovl3G+779fLheS5zkxTZM0BIGI9XrpzvQPidY1EiyrZGtXiDt7JtFHldiLOnl3DNKz26S36BB10hz2V92xPpPkf81nzOjrJYdeAAAAAElFTkSuQmCC"},"images":{"fallback":{"src":"/static/1edcc9c2169958556ac8c9ac51ec28de/a5b23/cm.png","srcSet":"/static/1edcc9c2169958556ac8c9ac51ec28de/ea251/cm.png 750w,\n/static/1edcc9c2169958556ac8c9ac51ec28de/a5b23/cm.png 860w","sizes":"100vw"},"sources":[{"srcSet":"/static/1edcc9c2169958556ac8c9ac51ec28de/678bf/cm.webp 750w,\n/static/1edcc9c2169958556ac8c9ac51ec28de/d78b9/cm.webp 860w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5034883720930232}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"14 min read"},"layout":"","slug":"/metric/"}},"next":{"excerpt":"DeepSpeed Usage…","frontmatter":{"title":"Sooftware NLP - DeepSpeed Usage","tags":["nlp","parallelism","large-scale"],"date":"2021-10-30T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAUCAYAAACNiR0NAAAACXBIWXMAAAsTAAALEwEAmpwYAAACwElEQVQ4y62VyU9TURTGL2NCMIKdoAWKYoVQJqvIFKEoFBlkKkRi4PHoADJLkSGUqARSBpFojHFITAwbjQsToxtjXLh17R/0M+8+KKAQXXRxct87793v3HPO950rUs6NEJ8dIDEnSEIMTCTZhxFWP8IWI0vOHdYfsgKxsaSTAP83yP5/tuMAD4HEZwcPfJpl+hAZQ/q654vLDkg7EvwwoPyoObWNhgG5JtqHSc4d4XT+OEbnFGn5E/I9UdtnVhEW9WjwfcA4zWkelCCmomkueVaobY9wvWeb2o51KltWKfesyNXdtUFdxzpVLWvYLoYQZwYOThltimkQe/kcN25t4+7c4HzlAobCSVJz+sm9PEOWK0R6wTjWsrsYnRPkuEIU14VRRl+gTr7+o8tmlerWNRp6tkh1jCFMCsKkItIDZDkDRJ58ZXTxAx+//GLn+XcWVz/x9t1PfFO77Lz6wdOX33A1PtBLIDKHKKwNc7U9gkjr14ueHSTJ7qewrJuzJQqz42GU2zOEJpYZ9i+hKgvMTi7T1zdPa/MQBodKXecmIkNFaOz29D5CWIZkUxJyAgjLCMUuL+9nHUTUUp6N5BHqcfF53sSdtgp2p3O531fCXI+TkLeYN5EJDKVhhEVBCKuPiuZVitzL+gktKiIzQEZBP03uFqqqOrAUKFKaHncbeU4vlVdaKCztpuvmAJGlJSqaFhFGVWdJoj0oi3nNuynTtrlmkZ03+xCmIMIYRFj8MgNhG8Nceo+8mofUex9T3b5NyoUZhEE94KNsitUnC6qBaVSo79rA07uFp3eTRu/m3vMWDd3r1LSuUlQX5pRjFGFWEJmqHC46bfx7w2GP9TJdgyL5pPm1NKOmBdZ4ZlQQxkHZvKgQTlKKrha9BH9NIKs/KoAo0HGaP0nLR3Rs+8dAOGyxH1+xHrD6FRCb8a9dAb8BUDWKe8aOUkAAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/9b0c4df957330fe4209b1e9f8e708409/67a35/deepspeed.png","srcSet":"/static/9b0c4df957330fe4209b1e9f8e708409/67a35/deepspeed.png 224w","sizes":"100vw"},"sources":[{"srcSet":"/static/9b0c4df957330fe4209b1e9f8e708409/f42a0/deepspeed.webp 224w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":1}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"4 min read"},"layout":"","slug":"/deepspeed/"}},"primaryTag":"toolkit"}},
    "staticQueryHashes": ["3170763342","3229353822"]}