{
    "componentChunkName": "component---src-templates-post-tsx",
    "path": "/bentoml/",
    "result": {"data":{"logo":null,"markdownRemark":{"html":"<h1>BentoML</h1>\n<p>Machine Learning Serving 라이브러리인 BentoML 사용방법에 대해 정리합니다.</p>\n<p><img src=\"https://user-images.githubusercontent.com/54731898/134802579-71c3c3d6-bb96-431a-80fd-ab13595c80d4.png\" alt=\"image\"></p>\n<h2>주요 특징</h2>\n<ul>\n<li>Online / Offline Serving</li>\n<li>Flask 기반 모델보다 100배의 처리량을 가지고, Adaptive Micro Batching 메커니즘을 활용</li>\n<li>모델 관리를 위한 웹 대시보드 존재</li>\n<li>정말 많은 ML 프레임워크를 지원함(transformers, pytorch-lightning도 지원)</li>\n</ul>\n<p>❓ <strong>Online Serving 이란?</strong></p>\n<ul>\n<li>Online Serving은 API 서빙, 실시간 요청에 따른 반응을 합니다.</li>\n<li>Batch 처리가 불가능하고, 동시 여러 요청에 대한 확장 대책이 필요하다.</li>\n</ul>\n<p>❓ <strong>Offline Serving 이란?</strong></p>\n<ul>\n<li>특정 주기로 서빙 하는 것을 말한다.</li>\n<li>Batch로 많은 양을 한꺼번에 처리한다.</li>\n</ul>\n<p>❓ <strong>Adaptive Micro Batching 이란?</strong></p>\n<ul>\n<li>모델 서빙시 개별 추론 요청을 조절할 수 있는 작은 배치 단위로 처리하는 것.</li>\n<li>BentoML은 HTTP 처리 데이터 처리과정까지 Micro batching 지원을 한다.</li>\n<li>최대 배치 사이즈와 인퍼런스의 latency 제한을 설정할 수 있다.</li>\n</ul>\n<p><img src=\"https://user-images.githubusercontent.com/54731898/134803241-893745bf-191b-47ec-9faa-a54c87c71ab7.png\" alt=\"image\"></p>\n<p>그림과 같이 request가 들어올 때 request을 하나씩 처리하는 것이 아니라 여러 개를 한번에 처리하여 응답하고 있는 것을 확인할 수 있다.<br>\n이는 지정한 latency를 넘지 않는 선에서 request들을 합해서 배치 처리한다.<br>\n이후 API로 오는 request는 다음 micro Batch로 받아서 배치간은 비동기식 처리를 진행한다.</p>\n<p><code class=\"language-text\">bentoml.api(mb_max_batch_size=1000, mb_max_latency=10000)</code> 함수를 사용해서 최대 배치 사이즈와 인퍼런스의 latency 제한을 설정할 수 있다.<br>\n위의 값은 디폴트 값이고 latency 단위는 milliseconds이다.</p>\n<h2>설치</h2>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">pip install bentoml</code></pre></div>\n<h2>사용 방법</h2>\n<ol>\n<li>모델 학습</li>\n<li>Prediction Service Class 생성</li>\n<li>Prediction Service에 학습한 모델 저장</li>\n<li>Serving</li>\n<li>Prediction Request</li>\n</ol>\n<h3>1. 모델 학습</h3>\n<p>예시는 transformers 프레임워크를 통해 gpt2 모델을 사용합니다.\n코드를 main.py로 저장</p>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token keyword\">from</span> transformers <span class=\"token keyword\">import</span> AutoModelWithLMHead<span class=\"token punctuation\">,</span> AutoTokenizer\n\nmodel_name <span class=\"token operator\">=</span> <span class=\"token string\">\"gpt2\"</span>\nmodel <span class=\"token operator\">=</span> AutoModelWithLMHead<span class=\"token punctuation\">.</span>from_pretrained<span class=\"token punctuation\">(</span><span class=\"token string\">\"gpt2\"</span><span class=\"token punctuation\">)</span>\ntokenizer <span class=\"token operator\">=</span> AutoTokenizer<span class=\"token punctuation\">.</span>from_pretrained<span class=\"token punctuation\">(</span><span class=\"token string\">\"gpt2\"</span><span class=\"token punctuation\">)</span></code></pre></div>\n<h3>2. Prediction Service Class 생성</h3>\n<p>코드를 gpt.py로 저장</p>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token keyword\">import</span> bentoml\n<span class=\"token keyword\">from</span> bentoml<span class=\"token punctuation\">.</span>adapters <span class=\"token keyword\">import</span> JsonInput\n<span class=\"token keyword\">from</span> bentoml<span class=\"token punctuation\">.</span>frameworks<span class=\"token punctuation\">.</span>transformers <span class=\"token keyword\">import</span> TransformersModelArtifact\n\n\n<span class=\"token decorator annotation punctuation\">@bentoml<span class=\"token punctuation\">.</span>env</span><span class=\"token punctuation\">(</span>pip_packages<span class=\"token operator\">=</span><span class=\"token punctuation\">[</span><span class=\"token string\">\"transformers==3.1.0\"</span><span class=\"token punctuation\">,</span> <span class=\"token string\">\"torch==1.6.0\"</span><span class=\"token punctuation\">]</span><span class=\"token punctuation\">)</span>\n<span class=\"token decorator annotation punctuation\">@bentoml<span class=\"token punctuation\">.</span>artifacts</span><span class=\"token punctuation\">(</span><span class=\"token punctuation\">[</span>TransformersModelArtifact<span class=\"token punctuation\">(</span><span class=\"token string\">\"gptModel\"</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">]</span><span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">class</span> <span class=\"token class-name\">TransformerService</span><span class=\"token punctuation\">(</span>bentoml<span class=\"token punctuation\">.</span>BentoService<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n    <span class=\"token decorator annotation punctuation\">@bentoml<span class=\"token punctuation\">.</span>api</span><span class=\"token punctuation\">(</span><span class=\"token builtin\">input</span><span class=\"token operator\">=</span>JsonInput<span class=\"token punctuation\">(</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">,</span> batch<span class=\"token operator\">=</span><span class=\"token boolean\">False</span><span class=\"token punctuation\">)</span>\n    <span class=\"token keyword\">def</span> <span class=\"token function\">predict</span><span class=\"token punctuation\">(</span>self<span class=\"token punctuation\">,</span> parsed_json<span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n        src_text <span class=\"token operator\">=</span> parsed_json<span class=\"token punctuation\">.</span>get<span class=\"token punctuation\">(</span><span class=\"token string\">\"text\"</span><span class=\"token punctuation\">)</span>\n\n        model <span class=\"token operator\">=</span> self<span class=\"token punctuation\">.</span>artifacts<span class=\"token punctuation\">.</span>gptModel<span class=\"token punctuation\">.</span>get<span class=\"token punctuation\">(</span><span class=\"token string\">\"model\"</span><span class=\"token punctuation\">)</span>\n        tokenizer <span class=\"token operator\">=</span> self<span class=\"token punctuation\">.</span>artifacts<span class=\"token punctuation\">.</span>gptModel<span class=\"token punctuation\">.</span>get<span class=\"token punctuation\">(</span><span class=\"token string\">\"tokenizer\"</span><span class=\"token punctuation\">)</span>\n\n        input_ids <span class=\"token operator\">=</span> tokenizer<span class=\"token punctuation\">.</span>encode<span class=\"token punctuation\">(</span>src_text<span class=\"token punctuation\">,</span> return_tensors<span class=\"token operator\">=</span><span class=\"token string\">\"pt\"</span><span class=\"token punctuation\">)</span>\n\n        output <span class=\"token operator\">=</span> model<span class=\"token punctuation\">.</span>generate<span class=\"token punctuation\">(</span>input_ids<span class=\"token punctuation\">,</span> max_length<span class=\"token operator\">=</span><span class=\"token number\">50</span><span class=\"token punctuation\">)</span>\n        output <span class=\"token operator\">=</span> tokenizer<span class=\"token punctuation\">.</span>decode<span class=\"token punctuation\">(</span>output<span class=\"token punctuation\">[</span><span class=\"token number\">0</span><span class=\"token punctuation\">]</span><span class=\"token punctuation\">,</span> skip_special_tokens<span class=\"token operator\">=</span><span class=\"token boolean\">True</span><span class=\"token punctuation\">)</span>\n\n        <span class=\"token keyword\">return</span> output</code></pre></div>\n<ul>\n<li>BentoService Class를 상속하여 Prediction Service Class를 생성함.</li>\n<li><code class=\"language-text\">@api</code> 데코레이터를 통해 API의 input, output, batch 유무 등을 설정할 수 있음.</li>\n<li><code class=\"language-text\">@artifacts</code> 데코레이터에서는 BentoML에서 미리 만든 Artifact를 사용함. <code class=\"language-text\">transformers</code> 프레임워크는 <code class=\"language-text\">TransformersModelArtifact</code>, <code class=\"language-text\">Scikit-Learn</code> 프레임워크는 <code class=\"language-text\">SklearnModelArtifact</code>처럼 프레임워크마다 있다.</li>\n</ul>\n<p><code class=\"language-text\">TransformersModelArtifact(\"gptModel\")</code> 여기서 ‘gptModel’은 Prediction Service Class에서 부를 이름, 3번 main.py에서도 “gptModel”로 pack하는 것을 볼 수 있다.</p>\n<ul>\n<li><code class=\"language-text\">@env</code> 데코레이터를 통해 환경설정을 한다. <code class=\"language-text\">env(pip_packages=[\"transformers==3.1.0\", \"torch==1.6.0\"])</code>로 패키지 버전을 명시해줄 수도 있고, <code class=\"language-text\">@env(infer_pip_packages=True)</code>로 요구되는 pip dependencies와 버전을 자동적으로 찾게 할 수도 있다. 이를 통해 requirements.txt를 생성해준다. 이외에도 conda/Docker를 활용 할 수도 있다.</li>\n</ul>\n<h3>3. Prediction Service에 학습한 모델 저장</h3>\n<p>1번에 main.py에 코드 추가.</p>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token keyword\">from</span> transformers <span class=\"token keyword\">import</span> AutoModelWithLMHead<span class=\"token punctuation\">,</span> AutoTokenizer\n<span class=\"token keyword\">from</span> gpt <span class=\"token keyword\">import</span> TransformerService\n\n\nmodel_name <span class=\"token operator\">=</span> <span class=\"token string\">\"gpt2\"</span>\nmodel <span class=\"token operator\">=</span> AutoModelWithLMHead<span class=\"token punctuation\">.</span>from_pretrained<span class=\"token punctuation\">(</span><span class=\"token string\">\"gpt2\"</span><span class=\"token punctuation\">)</span>\ntokenizer <span class=\"token operator\">=</span> AutoTokenizer<span class=\"token punctuation\">.</span>from_pretrained<span class=\"token punctuation\">(</span><span class=\"token string\">\"gpt2\"</span><span class=\"token punctuation\">)</span>\n\nservice <span class=\"token operator\">=</span> TransformerService<span class=\"token punctuation\">(</span><span class=\"token punctuation\">)</span>\n\nartifact <span class=\"token operator\">=</span> <span class=\"token punctuation\">{</span><span class=\"token string\">\"model\"</span><span class=\"token punctuation\">:</span> model<span class=\"token punctuation\">,</span> <span class=\"token string\">\"tokenizer\"</span><span class=\"token punctuation\">:</span> tokenizer<span class=\"token punctuation\">}</span>\nservice<span class=\"token punctuation\">.</span>pack<span class=\"token punctuation\">(</span><span class=\"token string\">\"gptModel\"</span><span class=\"token punctuation\">,</span> artifact<span class=\"token punctuation\">)</span>\n\nsaved_path <span class=\"token operator\">=</span> service<span class=\"token punctuation\">.</span>save<span class=\"token punctuation\">(</span><span class=\"token punctuation\">)</span></code></pre></div>\n<ul>\n<li>main.py 스크립트 실행</li>\n</ul>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">python main.py</code></pre></div>\n<ul>\n<li>실행하면 다음과 같은 메세지가 출력됨</li>\n</ul>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">[2021-09-26 17:45:55,819] INFO - BentoService bundle 'TransformerService:20210926174550_16DCF9' saved to: C:\\Users\\sangchun\\bentoml\\repository\\TransformerService\\20210926174550_16DCF9</code></pre></div>\n<p>bentoml 폴더에 가면 logs, repository 폴더가 생성된다.<br>\nrepository로 가면 Prediction Service Class 이름인 TransformerService로 폴더가 있고 안에 Dockerfile, requirements.txt 등이 저장되어있다.</p>\n<br>\n<p><img src=\"https://user-images.githubusercontent.com/54731898/134807019-3104f46d-100f-40ea-9342-5cc50c3c4d6f.png\" alt=\"image\"></p>\n<h2>4. Serving</h2>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">bentoml serve TransformerService:latest</code></pre></div>\n<p><img src=\"https://user-images.githubusercontent.com/54731898/134807518-c774b014-567f-4288-a782-fd3fc41677f9.png\" alt=\"image\"><br>\nlocalhost:5000으로 접근하면 Swagger UI를 확인할 수 있음.</p>\n<h2>5. Prediction Request</h2>\n<ol>\n<li>curl command 사용</li>\n</ol>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">curl -i \\\n  --header \"Content-Type: application/json\" \\\n  --request POST \\\n  --data '{\"text\": \"my name is\"}' \\\n  http://localhost:5000/predict</code></pre></div>\n<ol start=\"2\">\n<li>python requests 라이브러리 사용</li>\n</ol>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token keyword\">import</span> requests\nresponse <span class=\"token operator\">=</span> requests<span class=\"token punctuation\">.</span>post<span class=\"token punctuation\">(</span><span class=\"token string\">\"http://127.0.0.1:5000/predict\"</span><span class=\"token punctuation\">,</span> json<span class=\"token operator\">=</span><span class=\"token punctuation\">{</span><span class=\"token string\">\"text\"</span><span class=\"token punctuation\">:</span> <span class=\"token string\">\"my name is\"</span><span class=\"token punctuation\">}</span><span class=\"token punctuation\">)</span>\n<span class=\"token keyword\">print</span><span class=\"token punctuation\">(</span>response<span class=\"token punctuation\">.</span>text<span class=\"token punctuation\">)</span></code></pre></div>\n<p><img src=\"https://user-images.githubusercontent.com/54731898/134807328-274b1f43-e9b5-499e-9f76-ea01293ba233.png\" alt=\"image\"></p>\n<ol start=\"3\">\n<li>browser에서 직접 Prediction</li>\n</ol>\n<p><img src=\"https://user-images.githubusercontent.com/54731898/134808135-97bdcb96-f315-45a0-9f84-ff14441b71a5.png\" alt=\"image\">\n<img src=\"https://user-images.githubusercontent.com/54731898/134808208-ba93182c-b377-4317-a824-db858aad017e.png\" alt=\"image\"></p>\n<br>\n<br>\n<h2>Yatai 서버</h2>\n<p><strong>BentoML</strong>에서는 model serving api만 제공하는 것이 아니라 BentoML에서 실행되는 각종 모델들을 관리해주는 <strong>Yatai</strong> 서비스도 지원한다.</p>\n<p>Yatai는 Model Management Component로 Repository에 저장된 모델, 배포된 모델을 보여준다.</p>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">bentoml yatai-service-start</code></pre></div>\n<p>Docker로 실행하려면</p>\n<div class=\"gatsby-highlight\" data-language=\"text\"><pre class=\"language-text\"><code class=\"language-text\">docker run \\\n  -v /var/run/docker.sock:/var/run/docker.sock \\\n  -v ~/bentoml:/bentoml \\\n  -p 3000:3000 \\\n  -p 50051:50051 \\\n  bentoml/yatai-service:latest</code></pre></div>\n<p>localhost:3000에 접근하면 저장된 모델을 확인할 수 있다.</p>\n<p><img src=\"https://user-images.githubusercontent.com/54731898/134808309-a2caa51b-0912-4097-b175-1c51fce11391.png\" alt=\"image\"></p>\n<p>Detail을 클릭하면 세부 내용이 나온다.</p>\n<br>\n<p><img src=\"https://user-images.githubusercontent.com/54731898/134807943-c0208f0f-3e45-40f2-9b5c-95cc41611d8a.png\" alt=\"image\"></p>\n<p>이렇게 Yatai를 이용하면 모델을 web UI로 쉽게 관리할 수 있다.</p>","htmlAst":{"type":"root","children":[{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"BentoML"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Machine Learning Serving 라이브러리인 BentoML 사용방법에 대해 정리합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/54731898/134802579-71c3c3d6-bb96-431a-80fd-ab13595c80d4.png","alt":"image"},"children":[]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"주요 특징"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Online / Offline Serving"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Flask 기반 모델보다 100배의 처리량을 가지고, Adaptive Micro Batching 메커니즘을 활용"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"모델 관리를 위한 웹 대시보드 존재"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"정말 많은 ML 프레임워크를 지원함(transformers, pytorch-lightning도 지원)"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"❓ "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Online Serving 이란?"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Online Serving은 API 서빙, 실시간 요청에 따른 반응을 합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Batch 처리가 불가능하고, 동시 여러 요청에 대한 확장 대책이 필요하다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"❓ "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Offline Serving 이란?"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"특정 주기로 서빙 하는 것을 말한다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Batch로 많은 양을 한꺼번에 처리한다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"❓ "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Adaptive Micro Batching 이란?"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"모델 서빙시 개별 추론 요청을 조절할 수 있는 작은 배치 단위로 처리하는 것."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"BentoML은 HTTP 처리 데이터 처리과정까지 Micro batching 지원을 한다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"최대 배치 사이즈와 인퍼런스의 latency 제한을 설정할 수 있다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/54731898/134803241-893745bf-191b-47ec-9faa-a54c87c71ab7.png","alt":"image"},"children":[]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"그림과 같이 request가 들어올 때 request을 하나씩 처리하는 것이 아니라 여러 개를 한번에 처리하여 응답하고 있는 것을 확인할 수 있다."},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\n이는 지정한 latency를 넘지 않는 선에서 request들을 합해서 배치 처리한다."},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\n이후 API로 오는 request는 다음 micro Batch로 받아서 배치간은 비동기식 처리를 진행한다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"bentoml.api(mb_max_batch_size=1000, mb_max_latency=10000)"}]},{"type":"text","value":" 함수를 사용해서 최대 배치 사이즈와 인퍼런스의 latency 제한을 설정할 수 있다."},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\n위의 값은 디폴트 값이고 latency 단위는 milliseconds이다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"설치"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"pip install bentoml"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"사용 방법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ol","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"모델 학습"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Prediction Service Class 생성"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Prediction Service에 학습한 모델 저장"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Serving"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Prediction Request"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"1. 모델 학습"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"예시는 transformers 프레임워크를 통해 gpt2 모델을 사용합니다.\n코드를 main.py로 저장"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" transformers "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" AutoModelWithLMHead"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" AutoTokenizer\n\nmodel_name "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"gpt2\""}]},{"type":"text","value":"\nmodel "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" AutoModelWithLMHead"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"from_pretrained"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"gpt2\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\ntokenizer "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" AutoTokenizer"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"from_pretrained"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"gpt2\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"2. Prediction Service Class 생성"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"코드를 gpt.py로 저장"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" bentoml\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" bentoml"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"adapters "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" JsonInput\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" bentoml"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"frameworks"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"transformers "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" TransformersModelArtifact\n\n\n"},{"type":"element","tagName":"span","properties":{"className":["token","decorator","annotation","punctuation"]},"children":[{"type":"text","value":"@bentoml"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"env"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"pip_packages"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"transformers==3.1.0\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"torch==1.6.0\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","decorator","annotation","punctuation"]},"children":[{"type":"text","value":"@bentoml"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"artifacts"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"text","value":"TransformersModelArtifact"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"gptModel\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"class"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","class-name"]},"children":[{"type":"text","value":"TransformerService"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"bentoml"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"BentoService"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","decorator","annotation","punctuation"]},"children":[{"type":"text","value":"@bentoml"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"api"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"input"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":"JsonInput"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" batch"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","boolean"]},"children":[{"type":"text","value":"False"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n    "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"def"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","function"]},"children":[{"type":"text","value":"predict"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"self"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" parsed_json"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n        src_text "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" parsed_json"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"get"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"text\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\n        model "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" self"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"artifacts"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"gptModel"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"get"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"model\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n        tokenizer "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" self"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"artifacts"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"gptModel"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"get"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"tokenizer\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\n        input_ids "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" tokenizer"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"encode"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"src_text"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" return_tensors"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"pt\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\n        output "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" model"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"generate"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"input_ids"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" max_length"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"50"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n        output "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" tokenizer"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"decode"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"output"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" skip_special_tokens"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","boolean"]},"children":[{"type":"text","value":"True"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\n        "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"return"}]},{"type":"text","value":" output"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"BentoService Class를 상속하여 Prediction Service Class를 생성함."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"@api"}]},{"type":"text","value":" 데코레이터를 통해 API의 input, output, batch 유무 등을 설정할 수 있음."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"@artifacts"}]},{"type":"text","value":" 데코레이터에서는 BentoML에서 미리 만든 Artifact를 사용함. "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"transformers"}]},{"type":"text","value":" 프레임워크는 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"TransformersModelArtifact"}]},{"type":"text","value":", "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Scikit-Learn"}]},{"type":"text","value":" 프레임워크는 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"SklearnModelArtifact"}]},{"type":"text","value":"처럼 프레임워크마다 있다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"TransformersModelArtifact(\"gptModel\")"}]},{"type":"text","value":" 여기서 ‘gptModel’은 Prediction Service Class에서 부를 이름, 3번 main.py에서도 “gptModel”로 pack하는 것을 볼 수 있다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"@env"}]},{"type":"text","value":" 데코레이터를 통해 환경설정을 한다. "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"env(pip_packages=[\"transformers==3.1.0\", \"torch==1.6.0\"])"}]},{"type":"text","value":"로 패키지 버전을 명시해줄 수도 있고, "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"@env(infer_pip_packages=True)"}]},{"type":"text","value":"로 요구되는 pip dependencies와 버전을 자동적으로 찾게 할 수도 있다. 이를 통해 requirements.txt를 생성해준다. 이외에도 conda/Docker를 활용 할 수도 있다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"3. Prediction Service에 학습한 모델 저장"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"1번에 main.py에 코드 추가."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" transformers "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" AutoModelWithLMHead"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" AutoTokenizer\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" gpt "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" TransformerService\n\n\nmodel_name "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"gpt2\""}]},{"type":"text","value":"\nmodel "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" AutoModelWithLMHead"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"from_pretrained"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"gpt2\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\ntokenizer "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" AutoTokenizer"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"from_pretrained"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"gpt2\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\nservice "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" TransformerService"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\nartifact "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"{"}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"model\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" model"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"tokenizer\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" tokenizer"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"}"}]},{"type":"text","value":"\nservice"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"pack"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"gptModel\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" artifact"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\nsaved_path "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" service"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"save"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"main.py 스크립트 실행"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"python main.py"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"실행하면 다음과 같은 메세지가 출력됨"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"[2021-09-26 17:45:55,819] INFO - BentoService bundle 'TransformerService:20210926174550_16DCF9' saved to: C:\\Users\\sangchun\\bentoml\\repository\\TransformerService\\20210926174550_16DCF9"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"bentoml 폴더에 가면 logs, repository 폴더가 생성된다."},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\nrepository로 가면 Prediction Service Class 이름인 TransformerService로 폴더가 있고 안에 Dockerfile, requirements.txt 등이 저장되어있다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/54731898/134807019-3104f46d-100f-40ea-9342-5cc50c3c4d6f.png","alt":"image"},"children":[]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"4. Serving"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"bentoml serve TransformerService:latest"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/54731898/134807518-c774b014-567f-4288-a782-fd3fc41677f9.png","alt":"image"},"children":[]},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\nlocalhost:5000으로 접근하면 Swagger UI를 확인할 수 있음."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"5. Prediction Request"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ol","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"curl command 사용"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"curl -i \\\n  --header \"Content-Type: application/json\" \\\n  --request POST \\\n  --data '{\"text\": \"my name is\"}' \\\n  http://localhost:5000/predict"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ol","properties":{"start":2},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"python requests 라이브러리 사용"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" requests\nresponse "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" requests"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"post"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"http://127.0.0.1:5000/predict\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" json"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"{"}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"text\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"my name is\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"}"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"print"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"response"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"text"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/54731898/134807328-274b1f43-e9b5-499e-9f76-ea01293ba233.png","alt":"image"},"children":[]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ol","properties":{"start":3},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"browser에서 직접 Prediction"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/54731898/134808135-97bdcb96-f315-45a0-9f84-ff14441b71a5.png","alt":"image"},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/54731898/134808208-ba93182c-b377-4317-a824-db858aad017e.png","alt":"image"},"children":[]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Yatai 서버"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"BentoML"}]},{"type":"text","value":"에서는 model serving api만 제공하는 것이 아니라 BentoML에서 실행되는 각종 모델들을 관리해주는 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Yatai"}]},{"type":"text","value":" 서비스도 지원한다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Yatai는 Model Management Component로 Repository에 저장된 모델, 배포된 모델을 보여준다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"bentoml yatai-service-start"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Docker로 실행하려면"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"docker run \\\n  -v /var/run/docker.sock:/var/run/docker.sock \\\n  -v ~/bentoml:/bentoml \\\n  -p 3000:3000 \\\n  -p 50051:50051 \\\n  bentoml/yatai-service:latest"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"localhost:3000에 접근하면 저장된 모델을 확인할 수 있다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/54731898/134808309-a2caa51b-0912-4097-b175-1c51fce11391.png","alt":"image"},"children":[]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Detail을 클릭하면 세부 내용이 나온다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/54731898/134807943-c0208f0f-3e45-40f2-9b5c-95cc41611d8a.png","alt":"image"},"children":[]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"이렇게 Yatai를 이용하면 모델을 web UI로 쉽게 관리할 수 있다."}]}],"data":{"quirksMode":false}},"excerpt":"BentoML Machine Learning Serving 라이브러리인 BentoML 사용방법에 대해 정리합니다. image 주요 특징 Online / Offline Serving Flask 기반 모델보다 100배의 처리량을 가지고, Adaptive…","fields":{"readingTime":{"text":"6 min read"}},"frontmatter":{"title":"Sooftware ML - BentoML","userDate":"28 September 2021","date":"2021-09-28T10:00:00.000Z","tags":["toolkit"],"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#08b8d8","images":{"fallback":{"src":"/static/654765492479aee0d6435ced3ca72f6a/79da5/bentoml.png","srcSet":"/static/654765492479aee0d6435ced3ca72f6a/ad961/bentoml.png 750w,\n/static/654765492479aee0d6435ced3ca72f6a/b9d7b/bentoml.png 1080w,\n/static/654765492479aee0d6435ced3ca72f6a/79da5/bentoml.png 1170w","sizes":"100vw"},"sources":[{"srcSet":"/static/654765492479aee0d6435ced3ca72f6a/32628/bentoml.webp 750w,\n/static/654765492479aee0d6435ced3ca72f6a/82f7c/bentoml.webp 1080w,\n/static/654765492479aee0d6435ced3ca72f6a/0f5ab/bentoml.webp 1170w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":1.0136752136752136}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"children":[{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#282838","images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/248f9/soohwan.png 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/fd435/soohwan.png 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png 120w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/e7f45/soohwan.webp 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/589ec/soohwan.webp 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/71a38/soohwan.webp 120w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6833333333333333}}]}}]}},"relatedPosts":{"totalCount":25,"edges":[{"node":{"id":"f2a32d9d-b206-5141-8c2d-5440e19caf65","excerpt":"llama.cpp (On device llm inference tool) 최근에 llama.cpp를 사용해봤는데, 상당히 편리하고 미래에 더 많이 쓰일 툴이라는 생각이 들어서 기록해둔다! llama.cpp란? 대표적인 오픈소스 LLM인 Meta…","frontmatter":{"title":"llama.cpp (On device llm inference tool)","date":"2024-09-07T10:00:00.000Z"},"fields":{"readingTime":{"text":"7 min read"},"slug":"/llama-cpp/"}}},{"node":{"id":"fa28b462-1fe2-5d0f-932b-f28e32f1da18","excerpt":"Elastic Search logstash - Nori 토크나이저 설정 이번에 회사에서 검색 기능을 구현하면서 Elastic Search를 다루게 됐다. 이 엔진을 다루면서 삽질을 많이 했는데, 다음에는 하지 않도록 기록용으로 남겨둔다. Elastic…","frontmatter":{"title":"[ELK] Elastic Search logstash - Nori 토크나이저 설정","date":"2024-08-24T10:00:00.000Z"},"fields":{"readingTime":{"text":"4 min read"},"slug":"/elastic-search/"}}},{"node":{"id":"d1f4a9ee-591d-5e95-a930-a1bf17701b0e","excerpt":"단 30줄로 ChatGPT 웹페이지 만들기 (Streamlit chat_message) Streamlit은 파이썬 기반의 오픈소스 웹 UI 라이브러리입니다. 매우 간단한 코드로 손쉽게 웹페이지를 띄울 수 있어서 간단한 데모나 PoC…","frontmatter":{"title":"단 30줄로 ChatGPT 웹페이지 만들기 (Streamlit chat_message)","date":"2024-05-01T10:00:00.000Z"},"fields":{"readingTime":{"text":"9 min read"},"slug":"/streamlit-chatgpt/"}}},{"node":{"id":"4e37b604-dce8-5e09-85e8-337d4f03f0cb","excerpt":"Terabyte(TB) 단위 데이터 셔플링 - terashuf 리눅스에서 TB 단위의 데이터를 line 기준으로 셔플링이 필요할 때가 (가끔) 있다. 직접 코딩해서 쓰기에는 메모리, 속도 등을 신경써야해서 생각보다 큰 작업인데, terashuf…","frontmatter":{"title":"Terabyte(TB) 단위 데이터 셔플링 - terashuf","date":"2024-03-20T10:00:00.000Z"},"fields":{"readingTime":{"text":"1 min read"},"slug":"/terashuf/"}}},{"node":{"id":"ec8327c3-3c4d-51da-9440-8cb74fd1effe","excerpt":"tmux - conf Tmux 설정을 위한 tmux configuraion 기록","frontmatter":{"title":"tmux - conf","date":"2024-02-15T10:00:00.000Z"},"fields":{"readingTime":{"text":"2 min read"},"slug":"/tmux-conf/"}}}]}},"pageContext":{"slug":"/bentoml/","prev":{"excerpt":"React 기반 개인 웹페이지 배포하기 이번 글에서는 react…","frontmatter":{"title":"React 기반 개인 웹페이지 배포하기 (gatsby)","tags":["toolkit","web"],"date":"2021-09-22T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAYAAAB/Ca1DAAAACXBIWXMAAAsTAAALEwEAmpwYAAAC6klEQVQozzWSS28TdxTF//bMeMav+IntseM3JrYnDTGx0yR2ksaJDQk2bh0c5UETxSGQgBCF0BD1QVFTIEVCgFiAhIQECxatuqvUfoAuuugH6Lf5VbbUxdG9V7o695yjK+4f3mG1s8y5kQIDNjexYByvJ4SiOJBVB0LSSOhBnh12WbkwgxAydrsb16lB/JkiT9+958ruDRTNicXpRay2GqRCca5e/pxGrU46kcWmubDbPUiyFZNipT6a4ZefbvJwrU7S7eBUbIhC+TwmW4DiZ5d4/v5X0sMlNKcf0Vlf4YcHd7l1bZXS2KfEk1mcNg+a2lPoRJisXMyn+fftMX8/vcek7iNTmuHc1ByOdJnt43e8+PCRxuouQkiIu7e32d9YIRHSSYd1XFYbI4ZBJn2GpfESs5UKP17d4PfH9/nz4S1a2QSnZxrEcwUGgkmad4558Pw1sXIHi55FzBfztM7P8uHNCW9ePqaxVCOXzyKEmePuGr+dHPHzwTX+eHLAX9/vU02F8X4yhSuc6isqNtdZ+vYV3vkNQuN1xHeHexx+9WU/wxtb6xzd3qI+W0FWnJxstfnn2T32W00WiwbXFyawqlacwSguTwhJ1vClctTam+zt7nBzfw9RLxf55uttHh10qZ41OD0YRQ/GMEsOjq50eNTt4A/GEEKj0fqCVDKDWajYbR4kzYHfF6AyMU2rucb1zR3E5qUaY4k0zfocO4s1JjM5goEIimInP5TvLw8bhb695XaHqckKQgg8Az78viBexwDTo2NcmGuzXJ1HlIyzrC1WGQoM0i4Ms16dJu33Y7XY+z/Xy1KSNGTFhmzWUBU7qsVBIhLH4/Ti9YeYToQxvA4uT40i8tEkCxPjHLQv0l0oszJXwUieQe6RSFYkk4oi2/rozT30epOw9KvLrxPVA9hUlXAkgiimk0xmkmxWCizkktRG8owmYhipFFFd71vtKfufrHfIbFKRzCoWxUrQ7cNwe8lEI+QiIf4DVOxddpZZefQAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/ddaa4cb6857e989e32ffc466cf76e8a5/7c984/gatsby.png","srcSet":"/static/ddaa4cb6857e989e32ffc466cf76e8a5/5b584/gatsby.png 750w,\n/static/ddaa4cb6857e989e32ffc466cf76e8a5/c1f05/gatsby.png 1080w,\n/static/ddaa4cb6857e989e32ffc466cf76e8a5/7c984/gatsby.png 1123w","sizes":"100vw"},"sources":[{"srcSet":"/static/ddaa4cb6857e989e32ffc466cf76e8a5/73e0d/gatsby.webp 750w,\n/static/ddaa4cb6857e989e32ffc466cf76e8a5/9fede/gatsby.webp 1080w,\n/static/ddaa4cb6857e989e32ffc466cf76e8a5/d4610/gatsby.webp 1123w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5253784505788067}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"15 min read"},"layout":"","slug":"/react_homepage/"}},"next":{"excerpt":"Uniform Length Batching in PyTorch 전체 토큰 길이가 비슷한 인풋끼리 배치를 이루어주는 방식 그냥 랜덤하게 배치를 묶어주면 길이가 한 데이터를 제외하고는 평균 길이가 10인데 한 데이터 길이가 10…","frontmatter":{"title":"Sooftware NLP - Uniform Length Batching in PyTorch","tags":["nlp"],"date":"2021-09-28T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAYAAAAIy204AAAACXBIWXMAAAsTAAALEwEAmpwYAAABh0lEQVQoz1WR22vUQBSH85f7Ym3dW7K3qFtQQWkfRMF3kVLxoQ8ilH1YhbYpLd1NsptsMrlOLvvJzKrogY/fmTNzZn4zY1RVzeLHA4vFkvn8nvn8jsvLO1zXp5ZXFNktUha0bUtVSU3bNKi+uqqo65p/w/A8gTlJ6Q1LBuOCoV1i2ZKRHXByesHX87dIuf2vabfb/UVFHEWslkuapsFwPcGTXsBhT3A0iDnsx3QtRcqjg4qjfsLZl5i1941o84lE3KC2adudRkWeZYSbQB9guK7AsjPMSYU1lZgTSX9UMRhLrGlJx5Q87rScfT7l+8UBt9fnv322yqvO0iTFc13yPMfw/YSOGdK1Ep4OhHaq3faFHiu6ltDzyu3xq5j3HxNev8l492HN1c8Tws09Ik71+xqrVcT4ecx0ljF5kTJ6lmq1Z3udzva1oZ1p1G36o5SOmWs9fnmN4zzgey5FUWLUdUMUS7ZbSaw0kgSBJAj3uar/IQwloVonJOt1jOP43DiCNMkpy0L/8i/gruyOrih19AAAAABJRU5ErkJggg=="},"images":{"fallback":{"src":"/static/d35fd77fe1abce1948a61534ae0683af/d389c/ulb.png","srcSet":"/static/d35fd77fe1abce1948a61534ae0683af/57129/ulb.png 750w,\n/static/d35fd77fe1abce1948a61534ae0683af/3c97c/ulb.png 1080w,\n/static/d35fd77fe1abce1948a61534ae0683af/6ea65/ulb.png 1366w,\n/static/d35fd77fe1abce1948a61534ae0683af/d389c/ulb.png 1592w","sizes":"100vw"},"sources":[{"srcSet":"/static/d35fd77fe1abce1948a61534ae0683af/86b93/ulb.webp 750w,\n/static/d35fd77fe1abce1948a61534ae0683af/4c98a/ulb.webp 1080w,\n/static/d35fd77fe1abce1948a61534ae0683af/e8f75/ulb.webp 1366w,\n/static/d35fd77fe1abce1948a61534ae0683af/30d2a/ulb.webp 1592w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.3674623115577889}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"6 min read"},"layout":"","slug":"/uniform_length_batching/"}},"primaryTag":"toolkit"}},
    "staticQueryHashes": ["3170763342","3229353822"]}