{
    "componentChunkName": "component---src-templates-post-tsx",
    "path": "/neox_injection/",
    "result": {"data":{"logo":null,"markdownRemark":{"html":"<h1>GPT-NeoX - DeepSpeed Inference</h1>\n<p>DeepSpeed Inference를 사용하면 간단하게 모델 추론 성능을 끌어올릴 수 있다.<br>\nTensor Parallel 등의 최적화를 쉽게 제공하는데, <code class=\"language-text\">injection</code>이라는 기능을 사용하면 더욱 빠르게 할 수 있다.</p>\n<p><code class=\"language-text\">replace_with_kernel_inject=True</code>로 넣어주면 Bert, GPT2, GPT-Neo and GPT-J 모델들은 알아서 최적화를 해주는데,\nNeoX는 아직 자동으로 최적화가 안 된다.</p>\n<p>그래서 조금 커스텀하게 다음과 같이 적용해서 사용 가능하다.</p>\n<div class=\"gatsby-highlight\" data-language=\"python\"><pre class=\"language-python\"><code class=\"language-python\"><span class=\"token keyword\">import</span> deepspeed\n<span class=\"token keyword\">import</span> torch\n<span class=\"token keyword\">from</span> transformers <span class=\"token keyword\">import</span> AutoModelForCausalLM<span class=\"token punctuation\">,</span> GPTNeoXLayer\n\n<span class=\"token keyword\">with</span> deepspeed<span class=\"token punctuation\">.</span>OnDevice<span class=\"token punctuation\">(</span>dtype<span class=\"token operator\">=</span>torch<span class=\"token punctuation\">.</span>float16<span class=\"token punctuation\">,</span> device<span class=\"token operator\">=</span><span class=\"token string\">\"cuda\"</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">:</span>\n    model <span class=\"token operator\">=</span> AutoModelForCausalLM<span class=\"token punctuation\">.</span>from_pretrained<span class=\"token punctuation\">(</span><span class=\"token string\">'EleutherAI/polyglot-ko-5.8b'</span><span class=\"token punctuation\">,</span> torch_dtype<span class=\"token operator\">=</span>torch<span class=\"token punctuation\">.</span>float16<span class=\"token punctuation\">,</span> cache_dir<span class=\"token operator\">=</span><span class=\"token string\">'/data/.cache'</span><span class=\"token punctuation\">)</span>\n\nmodel <span class=\"token operator\">=</span> deepspeed<span class=\"token punctuation\">.</span>init_inference<span class=\"token punctuation\">(</span>\n    model<span class=\"token punctuation\">,</span>\n    mp_size<span class=\"token operator\">=</span><span class=\"token number\">2</span><span class=\"token punctuation\">,</span>\n    dtype<span class=\"token operator\">=</span>torch<span class=\"token punctuation\">.</span>float16<span class=\"token punctuation\">,</span>\n    injection_policy<span class=\"token operator\">=</span><span class=\"token punctuation\">{</span>GPTNeoXLayer<span class=\"token punctuation\">:</span> <span class=\"token punctuation\">(</span><span class=\"token string\">'attention.dense'</span><span class=\"token punctuation\">,</span><span class=\"token string\">'mlp.dense_4h_to_h'</span><span class=\"token punctuation\">)</span><span class=\"token punctuation\">}</span>\n<span class=\"token punctuation\">)</span><span class=\"token punctuation\">.</span>module</code></pre></div>","htmlAst":{"type":"root","children":[{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"GPT-NeoX - DeepSpeed Inference"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"DeepSpeed Inference를 사용하면 간단하게 모델 추론 성능을 끌어올릴 수 있다."},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\nTensor Parallel 등의 최적화를 쉽게 제공하는데, "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"injection"}]},{"type":"text","value":"이라는 기능을 사용하면 더욱 빠르게 할 수 있다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"replace_with_kernel_inject=True"}]},{"type":"text","value":"로 넣어주면 Bert, GPT2, GPT-Neo and GPT-J 모델들은 알아서 최적화를 해주는데,\nNeoX는 아직 자동으로 최적화가 안 된다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"그래서 조금 커스텀하게 다음과 같이 적용해서 사용 가능하다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" deepspeed\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" torch\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" transformers "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" AutoModelForCausalLM"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" GPTNeoXLayer\n\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"with"}]},{"type":"text","value":" deepspeed"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"OnDevice"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"dtype"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":"torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"float16"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" device"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"cuda\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n    model "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" AutoModelForCausalLM"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"from_pretrained"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'EleutherAI/polyglot-ko-5.8b'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" torch_dtype"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":"torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"float16"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" cache_dir"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'/data/.cache'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\nmodel "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" deepspeed"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"init_inference"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"\n    model"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n    mp_size"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n    dtype"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":"torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"float16"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n    injection_policy"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"{"}]},{"type":"text","value":"GPTNeoXLayer"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'attention.dense'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'mlp.dense_4h_to_h'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"}"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"module"}]}]}]}],"data":{"quirksMode":false}},"excerpt":"GPT-NeoX - DeepSpeed Inference DeepSpeed Inference를 사용하면 간단하게 모델 추론 성능을 끌어올릴 수 있다. Tensor Parallel…","fields":{"readingTime":{"text":"1 min read"}},"frontmatter":{"title":"GPT-NeoX - DeepSpeed Inference","userDate":"24 May 2023","date":"2023-05-24T10:00:00.000Z","tags":["toolkit","environment"],"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#484848","images":{"fallback":{"src":"/static/27e3ccdfb2446b7df7d68e0f62d4743f/90018/neox.png","srcSet":"/static/27e3ccdfb2446b7df7d68e0f62d4743f/f503b/neox.png 750w,\n/static/27e3ccdfb2446b7df7d68e0f62d4743f/e4953/neox.png 1080w,\n/static/27e3ccdfb2446b7df7d68e0f62d4743f/c2802/neox.png 1366w,\n/static/27e3ccdfb2446b7df7d68e0f62d4743f/90018/neox.png 1720w","sizes":"100vw"},"sources":[{"srcSet":"/static/27e3ccdfb2446b7df7d68e0f62d4743f/77272/neox.webp 750w,\n/static/27e3ccdfb2446b7df7d68e0f62d4743f/317ed/neox.webp 1080w,\n/static/27e3ccdfb2446b7df7d68e0f62d4743f/b5c49/neox.webp 1366w,\n/static/27e3ccdfb2446b7df7d68e0f62d4743f/e0b55/neox.webp 1720w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5232558139534883}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"children":[{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#282838","images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/248f9/soohwan.png 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/fd435/soohwan.png 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png 120w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/e7f45/soohwan.webp 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/589ec/soohwan.webp 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/71a38/soohwan.webp 120w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6833333333333333}}]}}]}},"relatedPosts":{"totalCount":25,"edges":[{"node":{"id":"f2a32d9d-b206-5141-8c2d-5440e19caf65","excerpt":"llama.cpp (On device llm inference tool) 최근에 llama.cpp를 사용해봤는데, 상당히 편리하고 미래에 더 많이 쓰일 툴이라는 생각이 들어서 기록해둔다! llama.cpp란? 대표적인 오픈소스 LLM인 Meta…","frontmatter":{"title":"llama.cpp (On device llm inference tool)","date":"2024-09-07T10:00:00.000Z"},"fields":{"readingTime":{"text":"7 min read"},"slug":"/llama-cpp/"}}},{"node":{"id":"fa28b462-1fe2-5d0f-932b-f28e32f1da18","excerpt":"Elastic Search logstash - Nori 토크나이저 설정 이번에 회사에서 검색 기능을 구현하면서 Elastic Search를 다루게 됐다. 이 엔진을 다루면서 삽질을 많이 했는데, 다음에는 하지 않도록 기록용으로 남겨둔다. Elastic…","frontmatter":{"title":"[ELK] Elastic Search logstash - Nori 토크나이저 설정","date":"2024-08-24T10:00:00.000Z"},"fields":{"readingTime":{"text":"4 min read"},"slug":"/elastic-search/"}}},{"node":{"id":"d1f4a9ee-591d-5e95-a930-a1bf17701b0e","excerpt":"단 30줄로 ChatGPT 웹페이지 만들기 (Streamlit chat_message) Streamlit은 파이썬 기반의 오픈소스 웹 UI 라이브러리입니다. 매우 간단한 코드로 손쉽게 웹페이지를 띄울 수 있어서 간단한 데모나 PoC…","frontmatter":{"title":"단 30줄로 ChatGPT 웹페이지 만들기 (Streamlit chat_message)","date":"2024-05-01T10:00:00.000Z"},"fields":{"readingTime":{"text":"9 min read"},"slug":"/streamlit-chatgpt/"}}},{"node":{"id":"4e37b604-dce8-5e09-85e8-337d4f03f0cb","excerpt":"Terabyte(TB) 단위 데이터 셔플링 - terashuf 리눅스에서 TB 단위의 데이터를 line 기준으로 셔플링이 필요할 때가 (가끔) 있다. 직접 코딩해서 쓰기에는 메모리, 속도 등을 신경써야해서 생각보다 큰 작업인데, terashuf…","frontmatter":{"title":"Terabyte(TB) 단위 데이터 셔플링 - terashuf","date":"2024-03-20T10:00:00.000Z"},"fields":{"readingTime":{"text":"1 min read"},"slug":"/terashuf/"}}},{"node":{"id":"ec8327c3-3c4d-51da-9440-8cb74fd1effe","excerpt":"tmux - conf Tmux 설정을 위한 tmux configuraion 기록","frontmatter":{"title":"tmux - conf","date":"2024-02-15T10:00:00.000Z"},"fields":{"readingTime":{"text":"2 min read"},"slug":"/tmux-conf/"}}}]}},"pageContext":{"slug":"/neox_injection/","prev":{"excerpt":"Sshpass…","frontmatter":{"title":"sshpass","tags":["toolkit"],"date":"2023-05-13T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAYAAAC0VX7mAAAACXBIWXMAAA7DAAAOwwHHb6hkAAAB30lEQVQoz41SPY8SURS9b97M7MA6w9AaqdZsiBbuIgYKwEQni7MaQ4LJJhbSWG5jqAw0ZGzZhp1sAWxDNkIm29GP/ASLtfYPmPAQ6mPeQ9bEj7gvubkfuefkvHsvrVYr1Go1eAceDss+/MfP4Pv+jeypdwjffw7P8xTHcrkEiYWARRZol0AhgXYIRDez/XsuXGdLxYlEAvP5HCSEQCqVUkX9rQH9mwHN18CIgZkMjP3NCIwI8eU71F+WFDaddiG5SHwXsE0bdJfAPxpgxxyU+rcqzjXlH+TKkK9/fqFyKWpNuBCwkzbIIWi3NWgk1WnQH+ngRQ7NXquVIKmMiGH3DiG+qCrCr5/PUSsTtm9tFAoB27H/VHOfQA8JtLXOmcag8zVxed/Cp7M0Xr96gg/HO3j/xoJpOViInzN0HEc1JreTaDQaqB5UcfTiCKW9EnJ7ORSLxf8uyLbtXwo3S8lkMpjNZgiCANFlhPAsxNWXKwyHQ/T7fbTbbQwGA5ycdBEEHfROT1GpVH6boRBwXVcVs9ksRqMRWq0WJpMJOp0Oms0moihCHMfodruYTqcIwxC9Xg/j8RiFQkFhJcc1obwhdTa6DsuylDdN8/o79Xod+XxexbJuGAY458pveiRO3uEPP39hDG45bAAAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/042a7f33c7032b5b6a9ea61d70b17c8a/90018/sshpass.png","srcSet":"/static/042a7f33c7032b5b6a9ea61d70b17c8a/f503b/sshpass.png 750w,\n/static/042a7f33c7032b5b6a9ea61d70b17c8a/e4953/sshpass.png 1080w,\n/static/042a7f33c7032b5b6a9ea61d70b17c8a/c2802/sshpass.png 1366w,\n/static/042a7f33c7032b5b6a9ea61d70b17c8a/90018/sshpass.png 1720w","sizes":"100vw"},"sources":[{"srcSet":"/static/042a7f33c7032b5b6a9ea61d70b17c8a/77272/sshpass.webp 750w,\n/static/042a7f33c7032b5b6a9ea61d70b17c8a/317ed/sshpass.webp 1080w,\n/static/042a7f33c7032b5b6a9ea61d70b17c8a/b5c49/sshpass.webp 1366w,\n/static/042a7f33c7032b5b6a9ea61d70b17c8a/e0b55/sshpass.webp 1720w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5232558139534883}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"1 min read"},"layout":"","slug":"/sshpass/"}},"next":{"excerpt":"LLaMA2 Meta(전 Facebook)가 이번 7월 18일에 LLaMA2를 공개했습니다. 🎉 LLaMA2 관련 내용이 담긴 논문 과 함께 7B, 13B, 70B 모델을 공개했습니다. 이전 LLaMA와 다르게 LLaMA…","frontmatter":{"title":"LLaMA2","tags":["nlp","paper"],"date":"2023-07-21T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAYAAAC0VX7mAAAACXBIWXMAAA7DAAAOwwHHb6hkAAACPUlEQVQoz02SbU8TQRSFJ0QhmihRwFJoC+0CXfpOX2kpLXS3uzM7uy1oFH8BITaGBE0gfhAxYowa//JjdgsJH2Ymmdx5zjlzr/A8D6UU4TkeBWRqEpG0EIkBqeIhpbaD9iRj32V3oNlqDnmUVQhDE696+L5GP2CIe+Ao0NQGb1mun/A867CYs6lUdohvtjAKFfYqSZaKQ0p5k1rXJlVXiIxktSrxtUKpKUeEW+B7NPZdRLzPU+OQQ9vl9fGIRrXCwqsVUkaOdj5JJldlYy3ByNccBYrNlkSkJTv7ipH/AKi1x4JpIeIHtHou48DDcSX9fp8tI01zY4Hy8iz14hbNZiNy4ykVAeZMybO8RHtTl8LXHgd2+G8D5rMWWmuGjsPp6SkXFxe8P3nHl8vP/Lz9wc31V26+XTOZTBhYNkeBJtOcRu/ZipAlQqvVrotIWGRqTlRk2TaTyQduf/3m/NMl//7+4erqivOPE65vvnN2dhbVjANNtTcFVu5ii7Hvsb3rIpJDCh0ZXcowknQx6kM6+3267SaxWIz06hLJjEnfVlHEsBl7gynQbCvG90CzLRHrLuWuJNBetBp9xZOch20NsHod1lMJGnmDTL5FvOwy8lU0LvtWOEKS7O4dMHQU2g1VEjXJcaA4HCpms5LHpuKg16W1k2fbSJFLLfIiVUKkHEodhzdjj8Le9G35PnKoYruKWVNGSi9LMupcWFTqKjzHYq9eYHstxsr8HOmNHDNGmMhioegwsyV5ZEosZ+r4P22nVoKD38TaAAAAAElFTkSuQmCC"},"images":{"fallback":{"src":"/static/314829bce3c3646245a7426ba323c5f5/90018/llama2.png","srcSet":"/static/314829bce3c3646245a7426ba323c5f5/f503b/llama2.png 750w,\n/static/314829bce3c3646245a7426ba323c5f5/e4953/llama2.png 1080w,\n/static/314829bce3c3646245a7426ba323c5f5/c2802/llama2.png 1366w,\n/static/314829bce3c3646245a7426ba323c5f5/90018/llama2.png 1720w","sizes":"100vw"},"sources":[{"srcSet":"/static/314829bce3c3646245a7426ba323c5f5/77272/llama2.webp 750w,\n/static/314829bce3c3646245a7426ba323c5f5/317ed/llama2.webp 1080w,\n/static/314829bce3c3646245a7426ba323c5f5/b5c49/llama2.webp 1366w,\n/static/314829bce3c3646245a7426ba323c5f5/e0b55/llama2.webp 1720w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5232558139534883}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"5 min read"},"layout":"","slug":"/llama2/"}},"primaryTag":"toolkit"}},
    "staticQueryHashes": ["3170763342","3229353822"]}