{
    "componentChunkName": "component---src-templates-post-tsx",
    "path": "/serving_term/",
    "result": {"data":{"logo":null,"markdownRemark":{"html":"<h1>Sooftware Serving - Terminology</h1>\n<p>NLP 모델 서빙 관련해서 계속 마주치는 용어들을 정리하는 페이</p>\n<ul>\n<li><code class=\"language-text\">Load Balancer</code>: 서버에 가해지는 부하(로드)를 분산시켜주는 장치 또는 기술.</li>\n</ul>\n<img width=\"400\" alt=\"image\" src=\"https://user-images.githubusercontent.com/42150335/193868360-da7a4c4c-2a3b-4cd6-ac1b-ee9172451e5b.png\">  \n<ul>\n<li><code class=\"language-text\">Load Balancing - Round Robin Method</code>: 서버에 들어온 요청을 순서대로 돌아가며 배정하는 방식</li>\n<li><code class=\"language-text\">Load Balancing - Weighted Round Robin Method</code>: 각 서버마다 가중치를 매기고 가중치가 높은 서버에 클라이언트 요청을 우선적으로 배분하는 방식</li>\n<li><code class=\"language-text\">Load Balancing - IP Hash Method</code>: 클라이언트의 IP 주소를 특정 서버로 매핑하여 요청을 처리하는 방식</li>\n<li><code class=\"language-text\">Load Balancing - Least Connection Method</code>: 요청이 들어온 시점에 가정 적은 연결상태를 보이는 서버에 우선적으로 트래픽을 배분하는 방식</li>\n<li><code class=\"language-text\">Load Balancing - Least Response Time Method</code>: 서버의 현재 연결 상태와 응답시간을 모두 고려하는 방식. 가장 적은 연결 상태와 가장 짧은 응답시간을 보이는 서버에 우선적으로 배분</li>\n</ul>\n<h3></h3>\n<ul>\n<li><code class=\"language-text\">Health Check</code>: 로드밸런싱 서비스를 제공하는 다수의 서버의 상태를 점검하기 위해 사용되는 기술로, 서버의 상태를 주기적으로 체크하여</li>\n</ul>\n<p>서버의 상태가 통신이 불가능할 경우 서버를 서비스에서 제외하여 서비스를 원활하게 제공하기 위해 이용되는 방법</p>\n<ul>\n<li><code class=\"language-text\">Stress Test</code>:  어플리케이션에 각종 리소스(CPU, RAM, Disk 등)의 허용하는 한도를 넘어서서 비정상적인 높은 부하를 발생시켜보는 테스트.</li>\n</ul>\n<p>한도를 넘어서는 부하가 걸렸을 때의 버그 등을 찾기 위함</p>\n<ul>\n<li><code class=\"language-text\">Load Test</code>: 적절한 부하를 발생시켜 통계적으로 의미있는 수치를 측정하는 테스트. 1. 장시간 서비스 가능 여부를 확인하는 신뢰성 테스트, 2. 성능 테스트 (Performance Test).</li>\n<li><code class=\"language-text\">TPS (Transaction Per Second)</code>: 초당 트랜잭션의 갯수. 서비스 성능의 기준.</li>\n<li><code class=\"language-text\">Saturation Point</code>: TPS가 더 이상 증가하지 않는 시점.</li>\n</ul>\n<img width=\"400\" alt=\"image\" src=\"https://user-images.githubusercontent.com/42150335/193871571-f78af74b-eac1-4381-8dda-e55069a34054.png\">\n<h3></h3>\n<ul>\n<li><code class=\"language-text\">WSGI (Web Server Gateway Interface)</code>: Python 어플리케이션이 웹 서버와 통신하기 위한 인터페이스. 웹 서버에서의 요청을 파싱하여 Python 어플리케이션 쪽으로 넘기는 역할을 수행</li>\n<li><code class=\"language-text\">Uvicorn</code>: uvloop 및 httptools를 사용하는 *ASGI Web Server\n<ul>\n<li><code class=\"language-text\">ASGI (Asynchronous Server Gateway Interface)</code>: 비동기 Python 웹 서버</li>\n<li><code class=\"language-text\">Asyncio</code>:  Asyncio는 async / await 구문을 사용하여 비동기(Asynchronous) 코드를 작성하는 라이브러리. <strong>non-blocking</strong>하게 처리함.</li>\n<li><code class=\"language-text\">uvloop</code>: asyncio를 대체하기 위해 만들어진 라이브러리. Cython으로 작성되었음. 기타 Python 비동기 프레임워크보다 2배 이상 빠름.</li>\n</ul>\n</li>\n<li><code class=\"language-text\">Gunicorn</code>: Python WSGI로 Web Server(Nginx)로부터 요청을 받으면 서버 어플리케이션으로 전달해주는 역할을 수행.</li>\n</ul>\n<img width=\"500\" alt=\"image\" src=\"https://user-images.githubusercontent.com/42150335/193873850-f25635a4-6863-4da9-ad97-7fbcb67c91e6.png\">  \n<ul>\n<li><code class=\"language-text\">NGINX</code>: 가벼우면서도 강력한 프로그램을 목표로 개발된 오픈소스 웹 서버 프로그램. Apache보다 동작이 단순하고, 전달자 역할만 하기 때문에 동시접속 처리에 특화되어 있다.</li>\n</ul>","htmlAst":{"type":"root","children":[{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"Sooftware Serving - Terminology"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"NLP 모델 서빙 관련해서 계속 마주치는 용어들을 정리하는 페이"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Load Balancer"}]},{"type":"text","value":": 서버에 가해지는 부하(로드)를 분산시켜주는 장치 또는 기술."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"width":400,"alt":"image","src":"https://user-images.githubusercontent.com/42150335/193868360-da7a4c4c-2a3b-4cd6-ac1b-ee9172451e5b.png"},"children":[]},{"type":"text","value":"  \n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Load Balancing - Round Robin Method"}]},{"type":"text","value":": 서버에 들어온 요청을 순서대로 돌아가며 배정하는 방식"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Load Balancing - Weighted Round Robin Method"}]},{"type":"text","value":": 각 서버마다 가중치를 매기고 가중치가 높은 서버에 클라이언트 요청을 우선적으로 배분하는 방식"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Load Balancing - IP Hash Method"}]},{"type":"text","value":": 클라이언트의 IP 주소를 특정 서버로 매핑하여 요청을 처리하는 방식"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Load Balancing - Least Connection Method"}]},{"type":"text","value":": 요청이 들어온 시점에 가정 적은 연결상태를 보이는 서버에 우선적으로 트래픽을 배분하는 방식"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Load Balancing - Least Response Time Method"}]},{"type":"text","value":": 서버의 현재 연결 상태와 응답시간을 모두 고려하는 방식. 가장 적은 연결 상태와 가장 짧은 응답시간을 보이는 서버에 우선적으로 배분"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Health Check"}]},{"type":"text","value":": 로드밸런싱 서비스를 제공하는 다수의 서버의 상태를 점검하기 위해 사용되는 기술로, 서버의 상태를 주기적으로 체크하여"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"서버의 상태가 통신이 불가능할 경우 서버를 서비스에서 제외하여 서비스를 원활하게 제공하기 위해 이용되는 방법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Stress Test"}]},{"type":"text","value":":  어플리케이션에 각종 리소스(CPU, RAM, Disk 등)의 허용하는 한도를 넘어서서 비정상적인 높은 부하를 발생시켜보는 테스트."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"한도를 넘어서는 부하가 걸렸을 때의 버그 등을 찾기 위함"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Load Test"}]},{"type":"text","value":": 적절한 부하를 발생시켜 통계적으로 의미있는 수치를 측정하는 테스트. 1. 장시간 서비스 가능 여부를 확인하는 신뢰성 테스트, 2. 성능 테스트 (Performance Test)."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"TPS (Transaction Per Second)"}]},{"type":"text","value":": 초당 트랜잭션의 갯수. 서비스 성능의 기준."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Saturation Point"}]},{"type":"text","value":": TPS가 더 이상 증가하지 않는 시점."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"width":400,"alt":"image","src":"https://user-images.githubusercontent.com/42150335/193871571-f78af74b-eac1-4381-8dda-e55069a34054.png"},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"WSGI (Web Server Gateway Interface)"}]},{"type":"text","value":": Python 어플리케이션이 웹 서버와 통신하기 위한 인터페이스. 웹 서버에서의 요청을 파싱하여 Python 어플리케이션 쪽으로 넘기는 역할을 수행"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Uvicorn"}]},{"type":"text","value":": uvloop 및 httptools를 사용하는 *ASGI Web Server\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"ASGI (Asynchronous Server Gateway Interface)"}]},{"type":"text","value":": 비동기 Python 웹 서버"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Asyncio"}]},{"type":"text","value":":  Asyncio는 async / await 구문을 사용하여 비동기(Asynchronous) 코드를 작성하는 라이브러리. "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"non-blocking"}]},{"type":"text","value":"하게 처리함."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"uvloop"}]},{"type":"text","value":": asyncio를 대체하기 위해 만들어진 라이브러리. Cython으로 작성되었음. 기타 Python 비동기 프레임워크보다 2배 이상 빠름."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Gunicorn"}]},{"type":"text","value":": Python WSGI로 Web Server(Nginx)로부터 요청을 받으면 서버 어플리케이션으로 전달해주는 역할을 수행."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"width":500,"alt":"image","src":"https://user-images.githubusercontent.com/42150335/193873850-f25635a4-6863-4da9-ad97-7fbcb67c91e6.png"},"children":[]},{"type":"text","value":"  \n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"NGINX"}]},{"type":"text","value":": 가벼우면서도 강력한 프로그램을 목표로 개발된 오픈소스 웹 서버 프로그램. Apache보다 동작이 단순하고, 전달자 역할만 하기 때문에 동시접속 처리에 특화되어 있다."}]},{"type":"text","value":"\n"}]}],"data":{"quirksMode":false}},"excerpt":"Sooftware Serving - Terminology NLP…","fields":{"readingTime":{"text":"5 min read"}},"frontmatter":{"title":"Sooftware Serving - Terminology","userDate":"4 October 2022","date":"2022-10-04T10:00:00.000Z","tags":["nlp","serving"],"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#f8f8f8","images":{"fallback":{"src":"/static/84b2e141b3f9a93f47a0cfa786f28c22/3ad85/term.png","srcSet":"/static/84b2e141b3f9a93f47a0cfa786f28c22/912f4/term.png 750w,\n/static/84b2e141b3f9a93f47a0cfa786f28c22/3ad85/term.png 900w","sizes":"100vw"},"sources":[{"srcSet":"/static/84b2e141b3f9a93f47a0cfa786f28c22/c23ad/term.webp 750w,\n/static/84b2e141b3f9a93f47a0cfa786f28c22/fc8e8/term.webp 900w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.7744444444444445}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"children":[{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#282838","images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/248f9/soohwan.png 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/fd435/soohwan.png 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png 120w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/e7f45/soohwan.webp 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/589ec/soohwan.webp 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/71a38/soohwan.webp 120w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6833333333333333}}]}}]}},"relatedPosts":{"totalCount":50,"edges":[{"node":{"id":"6529d72e-80e7-5d61-a672-d66276a3f641","excerpt":"MoE(Mixture of Experts) 기초부터 DeepSeek 혁신까지 작년 이맘때쯤 DeepSeek-V3가 저비용으로 엄청난 성능을 보이면서 화제가 되었습니다. 그 핵심 기술인 MoE(Mixture of Experts…","frontmatter":{"title":"MoE(Mixture of Experts) 기초부터 DeepSeek 혁신까지","date":"2026-01-15T01:11:55.000Z"},"fields":{"readingTime":{"text":"8 min read"},"slug":"/developed-moe/"}}},{"node":{"id":"8b49d3ef-4ce3-568c-9d71-240ff17fc3e0","excerpt":"BERT는 사실 Diffusion 모델이였다?! 최근 굉장히 흥미로운 글을 읽게되어 공유합니다. 원문 : link BERT와 Diffusion이 같은 방식이다?! NLP 연구자들에게 BERT는 너무 익숙한 모델입니다. 201…","frontmatter":{"title":"BERT는 사실 Diffusion 모델이였다?!","date":"2025-10-21T12:00:00.000Z"},"fields":{"readingTime":{"text":"8 min read"},"slug":"/bert_diffusion/"}}},{"node":{"id":"7c7e1676-ea84-58de-970c-ddec9aa10660","excerpt":"RLHF는 수다쟁이를 만든다?! (Does RLHF Breed Verbose Chatterboxes?!) RLHF(Reinforcement Learning from Human Feedback)는 OpenAI의 ChatGPT…","frontmatter":{"title":"RLHF는 수다쟁이를 만든다?! (Does RLHF Breed Verbose Chatterboxes?!)","date":"2024-03-13T01:11:55.000Z"},"fields":{"readingTime":{"text":"7 min read"},"slug":"/rlhf-vervosity/"}}},{"node":{"id":"2b9d3e22-1796-5fab-878d-5941d2e76e9d","excerpt":"LLM Paper Abstract - 2023.12 LLM…","frontmatter":{"title":"LLM Paper Abstract - 2023.12","date":"2024-01-05T10:00:00.000Z"},"fields":{"readingTime":{"text":"5 min read"},"slug":"/llm-abs-202312/"}}},{"node":{"id":"f43fa33b-917b-5c00-8462-937d99592ad7","excerpt":"What it MoE? (Mixture of Experts) 현존 최강 LLM인 GPT-4에서 “MoE (Mixture of Experts)” 방식을 채택하여 사용하고 있다고 알려졌는데요, 최근 AI계의 뜨거운 감자 Mistral AI…","frontmatter":{"title":"What is MoE? (Mixture of Experts)","date":"2023-12-22T01:11:55.000Z"},"fields":{"readingTime":{"text":"9 min read"},"slug":"/moe/"}}}]}},"pageContext":{"slug":"/serving_term/","prev":{"excerpt":"Sooftware Serving - Triton Inference Server Triton Inference Server는 인공지능 모델의 인퍼런스를 도와주는 오픈소스 소프트웨어다. 다양한 프레임워크(TensorRT, TensorFlow…","frontmatter":{"title":"Sooftware Serving - Triton Inference Server","tags":["nlp","serving"],"date":"2022-09-20T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAYAAAB/Ca1DAAAACXBIWXMAAAsTAAALEwEAmpwYAAABOElEQVQoz52MPUtCYRiGr/Oe+zU0aGipoCQhsMKllhoyaC5oi6IthELPSa22oPaIoKWpIRr6CUFDg+BRqGiofkNbv6E4oKL5gXXDxfPwfFzQIX6lDeMHuGH1AggJ533ll8ip93XJ/4RBiyzmV1j1AnJ+hZU/C7duYb+K41chW2LRK1P0ypx5Zd69gIfDNzj6wMnc03/2HnGyJTj5JFV45rTwwmXxlYv8E1fpPCSW4Ou7h0ASruvWkAMRUuuOC4ym8yQ3rtndvOF8+44dwiUQHTIm/Gumo7Deh3EMB0AOyADHgAesuTJY2yrrJRyTlJAUN8ZNKGIWIlEzH4mZ5YFBM2eMmZGUlDQpabirsIm4pJSk8HFC0rSkKUmz1tqEtXZc0lxtP9KPsIG1tlGb6XbfS+jUaO47zdqEP51/VSuGFZPHAAAAAElFTkSuQmCC"},"images":{"fallback":{"src":"/static/2a825b8546ee78dfd77964613068ac58/60936/triton.png","srcSet":"/static/2a825b8546ee78dfd77964613068ac58/60936/triton.png 540w","sizes":"100vw"},"sources":[{"srcSet":"/static/2a825b8546ee78dfd77964613068ac58/1c40e/triton.webp 540w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5629629629629629}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"3 min read"},"layout":"","slug":"/triton/"}},"next":{"excerpt":"Sooftware Serving - Huggingface Optimum 허깅페이스에서 나온 Transformers의 Extension 라이브러리이다. 목적은 모델 학습 및 인퍼런스를 더욱 빠르게 해주기 위한 라이브러리이다. Exporting…","frontmatter":{"title":"Sooftware Serving - Huggingface Optimum","tags":["huggingface","nlp","serving"],"date":"2022-10-05T15:11:55.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAYAAAC0VX7mAAAACXBIWXMAAA7DAAAOwwHHb6hkAAABv0lEQVQoz32Sa2sTURCGj+ZiE0H8ZKC2IFVqQRFL0yab+22zZ87JdlVoKSlJsRhRq4IVxA+VQq0igr/5kd3tLlFKP7zMHmbm2XeYUcYYRIQw+r5PEARRtKMRdhRgvS62X8YONrG6j/WDOGdtqnmGSh5hYjweM5lM2NndRWsPad5DOwtILRvLKSLtVYyxiMR9iRKOSh5mLhkWu9USXlUhrTxuPc+wkcdrZBhWFF5jBWP9qMcYudxhCg0LBxX2TAnfXWTjyV32/GW23SU69TV27CLPurcRt4kxowuoSaWSjwgaAo2P7a5ycqz5cfqSX+fH/Pn5it/nM86+veb7yT5fP7jo9iOM3U4nuxQooqMiad2nX1Z8PupydvqRL5/2eTt7ztHM8OZgPRpbOo8xNoh7rnIYjiH1JXRZcRCs8P6wz1R3mJoW76YOhy8eIpvXkOaDuPYqh9H2hl1kK4eu5nDLit66ityG6j1V9DcUuppFKgVEuxcb/w+Y/sWOkEEdHTqo55D2TUyriHEysVoFpF1E17LorQzi9VJguuX5G5IwhvfXWEZXbqCr1yOn4txBaiW0U0DXFtDOLaS99s8tJnf4F245Z+TmPt+gAAAAAElFTkSuQmCC"},"images":{"fallback":{"src":"/static/d0b8ba0447d1f90942743bf896396659/90018/optimum.png","srcSet":"/static/d0b8ba0447d1f90942743bf896396659/f503b/optimum.png 750w,\n/static/d0b8ba0447d1f90942743bf896396659/e4953/optimum.png 1080w,\n/static/d0b8ba0447d1f90942743bf896396659/c2802/optimum.png 1366w,\n/static/d0b8ba0447d1f90942743bf896396659/90018/optimum.png 1720w","sizes":"100vw"},"sources":[{"srcSet":"/static/d0b8ba0447d1f90942743bf896396659/77272/optimum.webp 750w,\n/static/d0b8ba0447d1f90942743bf896396659/317ed/optimum.webp 1080w,\n/static/d0b8ba0447d1f90942743bf896396659/b5c49/optimum.webp 1366w,\n/static/d0b8ba0447d1f90942743bf896396659/e0b55/optimum.webp 1720w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5232558139534883}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"3 min read"},"layout":"post","slug":"/huggingface_optimum/"}},"primaryTag":"nlp"}},
    "staticQueryHashes": ["3170763342","3229353822"]}