{ "componentChunkName": "component---src-templates-post-tsx", "path": "/huggingface_optimum/", "result": {"data":{"logo":null,"markdownRemark":{"html":"

Sooftware Serving - Huggingface Optimum

\n

허깅페이스에서 나온 Transformers의 Extension 라이브러리이다. 목적은 모델 학습 및 인퍼런스를 더욱 빠르게 해주기 위한 라이브러리이다.

\n

Exporting Transformers models to ONNX

\n

모델 서빙을 위해서는 많이들 Transformers 모델을 ONNX로 컨버팅하곤 한다.
\nONNX(Open Neural Network Exchange)는 쉽게 말하면, Tensorflow, PyTorch와 같이 서로 다른 딥러닝 프레임워크 환경에서 만들어진 모델들을\n서로 호환해서 사용할 수 있도록 도와주는 오픈소스이다.

\n

원래 Transformers 모델을 ONNX로 모델로 컨버팅하는게 마냥 간단하지만은 않았는데, 이 Optimum 라이브러리로 아래 코드로 쉽게 컨버팅 가능해졌다.\n아마 Optimum을 가장 많이 쓰게되는 이유일 것 같다.

\n
from optimum.onnxruntime import ORTModelForSequenceClassification\n\npretrain_model_name_or_path = \"pretrain_model_name_or_path\"\nsave_directory = \"tmp/onnx/\"\n\n# Load a model from transformers and export it to ONNX\nort_model = ORTModelForSequenceClassification.from_pretrained(pretrain_model_name_or_path, from_transformers=True)\n\n# Save the onnx model\nort_model.save_pretrained(save_directory)
\n

Quantization

\n

Model Quantization도 이 라이브러리로 쉽게 가능하다! Huggingface 만세!

\n
from optimum.onnxruntime.configuration import AutoQuantizationConfig\nfrom optimum.onnxruntime import ORTQuantizer\n\n# Define the quantization methodology\nqconfig = AutoQuantizationConfig.arm64(is_static=False, per_channel=False)\nquantizer = ORTQuantizer.from_pretrained(ort_model)\n\n# Apply dynamic quantization on the model\nquantizer.quantize(save_dir=save_directory, quantization_config=qconfig)
\n

Example of how to load an ONNX Runtime model and generate predictions with Optimum 🤗:

\n
from optimum.onnxruntime import ORTModelForSequenceClassification\nfrom transformers import pipeline, AutoTokenizer\n\nmodel = ORTModelForSequenceClassification.from_pretrained(save_directory, file_name=\"model_quantized.onnx\")\ntokenizer = AutoTokenizer.from_pretrained(save_directory)\n\ncls_pipeline = pipeline(\"text-classification\", model=model, tokenizer=tokenizer)\n\nresults = cls_pipeline(\"I love burritos!\")
\n

Graph Optimization

\n

Graph Optimization을 통해 모델 인퍼런스를 더욱 빠르게도 가능하다.

\n
from optimum.onnxruntime import ORTModelForSequenceClassification, ORTOptimizer\n\nort_model = ORTModelForSequenceClassification.from_pretrained(pretrain_model_name_or_path, from_transformers=True)\noptimizer = ORTOptimizer.from_pretrained(ort_model)\n\n# Optimize the model\noptimizer.optimize(save_dir=save_directory, optimization_config=optimization_config)
\n

이렇게 라이브러리를 통해 간단하게, Quantization, Optimization이 가능해졌다. 정말 가뭄의 단비같은 라이브러리다.

\n

Training

\n

인퍼런스 용도 외에도, ONNX를 이용하여 학습에도 적용 가능하다고 한다. 개인적으로는 Transformers에서 제공하는 Trainer를 잘\n사용하진 않는데, Transformers의 Trainer를 사용하는 코드라면 적용해보고 학습 속도를 비교해보고 싶다.

\n\"image\"\n

Reference

\n","htmlAst":{"type":"root","children":[{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"Sooftware Serving - Huggingface Optimum"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"허깅페이스에서 나온 Transformers의 Extension 라이브러리이다. 목적은 모델 학습 및 인퍼런스를 더욱 빠르게 해주기 위한 라이브러리이다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Exporting Transformers models to ONNX"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"모델 서빙을 위해서는 많이들 Transformers 모델을 ONNX로 컨버팅하곤 한다."},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\nONNX(Open Neural Network Exchange)는 쉽게 말하면, Tensorflow, PyTorch와 같이 서로 다른 딥러닝 프레임워크 환경에서 만들어진 모델들을\n서로 호환해서 사용할 수 있도록 도와주는 오픈소스이다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"원래 Transformers 모델을 ONNX로 모델로 컨버팅하는게 마냥 간단하지만은 않았는데, 이 Optimum 라이브러리로 아래 코드로 쉽게 컨버팅 가능해졌다.\n아마 Optimum을 가장 많이 쓰게되는 이유일 것 같다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" optimum"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"onnxruntime "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" ORTModelForSequenceClassification\n\npretrain_model_name_or_path "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"pretrain_model_name_or_path\""}]},{"type":"text","value":"\nsave_directory "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"tmp/onnx/\""}]},{"type":"text","value":"\n\n"},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# Load a model from transformers and export it to ONNX"}]},{"type":"text","value":"\nort_model "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" ORTModelForSequenceClassification"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"from_pretrained"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"pretrain_model_name_or_path"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" from_transformers"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","boolean"]},"children":[{"type":"text","value":"True"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\n"},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# Save the onnx model"}]},{"type":"text","value":"\nort_model"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"save_pretrained"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"save_directory"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Quantization"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Model Quantization도 이 라이브러리로 쉽게 가능하다! Huggingface 만세!"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" optimum"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"onnxruntime"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"configuration "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" AutoQuantizationConfig\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" optimum"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"onnxruntime "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" ORTQuantizer\n\n"},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# Define the quantization methodology"}]},{"type":"text","value":"\nqconfig "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" AutoQuantizationConfig"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"arm64"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"is_static"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","boolean"]},"children":[{"type":"text","value":"False"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" per_channel"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","boolean"]},"children":[{"type":"text","value":"False"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\nquantizer "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" ORTQuantizer"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"from_pretrained"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"ort_model"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\n"},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# Apply dynamic quantization on the model"}]},{"type":"text","value":"\nquantizer"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"quantize"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"save_dir"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":"save_directory"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" quantization_config"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":"qconfig"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Example of how to load an ONNX Runtime model and generate predictions with Optimum 🤗:"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" optimum"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"onnxruntime "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" ORTModelForSequenceClassification\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" transformers "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" pipeline"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" AutoTokenizer\n\nmodel "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" ORTModelForSequenceClassification"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"from_pretrained"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"save_directory"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" file_name"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"model_quantized.onnx\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\ntokenizer "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" AutoTokenizer"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"from_pretrained"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"save_directory"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\ncls_pipeline "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" pipeline"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"text-classification\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" model"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":"model"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" tokenizer"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":"tokenizer"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\nresults "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" cls_pipeline"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"I love burritos!\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Graph Optimization"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Graph Optimization을 통해 모델 인퍼런스를 더욱 빠르게도 가능하다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" optimum"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"onnxruntime "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" ORTModelForSequenceClassification"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" ORTOptimizer\n\nort_model "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" ORTModelForSequenceClassification"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"from_pretrained"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"pretrain_model_name_or_path"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" from_transformers"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","boolean"]},"children":[{"type":"text","value":"True"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\noptimizer "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" ORTOptimizer"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"from_pretrained"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"ort_model"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\n"},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# Optimize the model"}]},{"type":"text","value":"\noptimizer"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"optimize"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"save_dir"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":"save_directory"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" optimization_config"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":"optimization_config"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"이렇게 라이브러리를 통해 간단하게, Quantization, Optimization이 가능해졌다. 정말 가뭄의 단비같은 라이브러리다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Training"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"인퍼런스 용도 외에도, ONNX를 이용하여 학습에도 적용 가능하다고 한다. 개인적으로는 Transformers에서 제공하는 Trainer를 잘\n사용하진 않는데, Transformers의 Trainer를 사용하는 코드라면 적용해보고 학습 속도를 비교해보고 싶다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"width":440,"alt":"image","src":"https://user-images.githubusercontent.com/42150335/193879406-6b86724b-98de-4bf0-8346-33cd497d82ac.png"},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Reference"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Optimum github: "},{"type":"element","tagName":"a","properties":{"href":"https://github.com/huggingface/optimum"},"children":[{"type":"text","value":"https://github.com/huggingface/optimum"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"ONNX: "},{"type":"element","tagName":"a","properties":{"href":"https://github.com/onnx/onnx"},"children":[{"type":"text","value":"https://github.com/onnx/onnx"}]}]},{"type":"text","value":"\n"}]}],"data":{"quirksMode":false}},"excerpt":"Sooftware Serving - Huggingface Optimum 허깅페이스에서 나온 Transformers의 Extension 라이브러리이다. 목적은 모델 학습 및 인퍼런스를 더욱 빠르게 해주기 위한 라이브러리이다. Exporting…","fields":{"readingTime":{"text":"3 min read"}},"frontmatter":{"title":"Sooftware Serving - Huggingface Optimum","userDate":"5 October 2022","date":"2022-10-05T15:11:55.000Z","tags":["huggingface","nlp","serving"],"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#484848","images":{"fallback":{"src":"/static/d0b8ba0447d1f90942743bf896396659/90018/optimum.png","srcSet":"/static/d0b8ba0447d1f90942743bf896396659/f503b/optimum.png 750w,\n/static/d0b8ba0447d1f90942743bf896396659/e4953/optimum.png 1080w,\n/static/d0b8ba0447d1f90942743bf896396659/c2802/optimum.png 1366w,\n/static/d0b8ba0447d1f90942743bf896396659/90018/optimum.png 1720w","sizes":"100vw"},"sources":[{"srcSet":"/static/d0b8ba0447d1f90942743bf896396659/77272/optimum.webp 750w,\n/static/d0b8ba0447d1f90942743bf896396659/317ed/optimum.webp 1080w,\n/static/d0b8ba0447d1f90942743bf896396659/b5c49/optimum.webp 1366w,\n/static/d0b8ba0447d1f90942743bf896396659/e0b55/optimum.webp 1720w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5232558139534883}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"children":[{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#282838","images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/248f9/soohwan.png 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/fd435/soohwan.png 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png 120w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/e7f45/soohwan.webp 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/589ec/soohwan.webp 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/71a38/soohwan.webp 120w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6833333333333333}}]}}]}},"relatedPosts":{"totalCount":4,"edges":[{"node":{"id":"01bfefe1-36ae-551f-b213-3eaf935c162d","excerpt":"Huggingface PEFT (Parameter-Efficient Fine-Tuning) 허깅페이스에서 나온 PEFT라는 라이브러리인데 LoRA, Prefix Tuning, P-Tuing, Prompt Tuning…","frontmatter":{"title":"Huggingface PEFT (Parameter-Efficient Fine-Tuning)","date":"2023-03-31T15:11:55.000Z"},"fields":{"readingTime":{"text":"2 min read"},"slug":"/peft/"}}},{"node":{"id":"8a6144d6-6780-530b-b695-d92d023c1219","excerpt":"Sooftware Serving - Huggingface Optimum 허깅페이스에서 나온 Transformers의 Extension 라이브러리이다. 목적은 모델 학습 및 인퍼런스를 더욱 빠르게 해주기 위한 라이브러리이다. Exporting…","frontmatter":{"title":"Sooftware Serving - Huggingface Optimum","date":"2022-10-05T15:11:55.000Z"},"fields":{"readingTime":{"text":"3 min read"},"slug":"/huggingface_optimum/"}}},{"node":{"id":"699a7d22-985a-5874-86f8-489f485f930f","excerpt":"이번에 저희 튜닙에서 공들여 만든 TUNiB Electra 모델을 공개했습니다 !! 🎉 🎉 이번 공개에서는 한-영 bilingual 모델과 한국어 모델을 각각 Small/Base 사이즈로 공개했으며, HuggingFace transformers…","frontmatter":{"title":"TUNiB Electra 공개","date":"2021-09-18T15:11:55.000Z"},"fields":{"readingTime":{"text":"2 min read"},"slug":"/tunib_electra/"}}},{"node":{"id":"996d2b57-1627-59bf-ad52-684205e35ca6","excerpt":"최근 NLP 토크나이저를 만드는데 가장 많이 사용되는 라이브러와 실제 사용이 가장 많이 되는 라이브러리로의 변환에 대한 코드를 담고 있습니다. 해당 내용은 버젼에서 수행되었습니다. Train 아래 코드는 wordpiece, char-bpe…","frontmatter":{"title":"Sooftware NLP - Hugging Face Tokenizers","date":"2021-08-11T15:11:55.000Z"},"fields":{"readingTime":{"text":"4 min read"},"slug":"/tokenizers/"}}}]}},"pageContext":{"slug":"/huggingface_optimum/","prev":{"excerpt":"Sooftware Serving - Terminology NLP…","frontmatter":{"title":"Sooftware Serving - Terminology","tags":["nlp","serving"],"date":"2022-10-04T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAPCAYAAADkmO9VAAAACXBIWXMAAAsTAAALEwEAmpwYAAAC4klEQVQ4y2VUOUurURD9XNLHVrCyTSABwUIsFFJZmN8gaqVBUCwUTKGCiYiKuG8k7rgiYhAVNdGoWRRXsFDEFVxIbMQleh5n8r4PfW/g5oZ7556ZOXPmUwDg6+uLG0KhEKamprC6uor29nY5293dxcDAAIqKiqAoyn+rvLxc/GKxGL6/v6Hwh/b+/o6uri7539vbi7OzM/j9ftTV1aG6uhqFhYXQ6XRISEgQIHXnCgaDGqjC7Aj6+fmJiYkJNDc3w+Vy4fX1FQ8PD/B6vWhsbJTzpKSkX9klJyfLXlpaKoAfHx9Q8Nei0Sh6enowODiI6+tr7OzsSICFhQWcnp7CYDDI48TERA1QDWA2m1WYeIaVlZXQ6/VITU3F9PQ0Ojo60NLSgtHRUaysrODm5kYD+QmorpSUFCwtLUlVis1m+3VZVVWF2tpaNDQ0YHh4WHg5Pz9HZmYm0tPTf/mWlZVhdnYWW1tbkoQEzsnJkcvc3FxkZ2ejv78f4+PjqK+vx+XlJR4fH+XB2toauru7UVJSIv4mk0krk9xNTk7Guzw/Pw+j0YiNjQ1Ju6+vD8/Pz5IVQZaXl0VKaid5zgqoAhpB1tfXcXJygra2tnhTeNnU1ITj42Pc39/j7e1Ni+7z+SQ6pbS4uIjt7W38tEgkIokEAgFkZWXFAcPhsMjjX7u4uBCgvb09ET392EQu6pa2ubmJsbExWK1W4VMhsgrGLpGz29tbvLy8QM2eQDxXJ0o1+rOygoICWCyWuGyGhoaEN8qltbVVJGS326XDjEj5jIyMiD45guwmS2QQUkHAjIwMzM3NxQFZCp0paj6gmAnOc/J5d3cnj9kgj8cjXJECTtHh4SGKi4uRl5enNUhhuhUVFdI5p9MpcuHs1tTUyIRcXV3B4XCgs7MTbrdb1HBwcCCB9vf3kZaWJjKjcXwVEjozMyNfGH5Zjo6O8PT0JKQzA3aWgqUmec9yWSor4gzn5+drnDLDP4cIa5VFu31YAAAAAElFTkSuQmCC"},"images":{"fallback":{"src":"/static/84b2e141b3f9a93f47a0cfa786f28c22/3ad85/term.png","srcSet":"/static/84b2e141b3f9a93f47a0cfa786f28c22/912f4/term.png 750w,\n/static/84b2e141b3f9a93f47a0cfa786f28c22/3ad85/term.png 900w","sizes":"100vw"},"sources":[{"srcSet":"/static/84b2e141b3f9a93f47a0cfa786f28c22/c23ad/term.webp 750w,\n/static/84b2e141b3f9a93f47a0cfa786f28c22/fc8e8/term.webp 900w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.7744444444444445}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"5 min read"},"layout":"","slug":"/serving_term/"}},"next":{"excerpt":"개인 운동 기록 - 2022.10.30 한 달하고 3주만의 기록이다. 안타깝게도, 수행능력은 이전 기록과 동일한 것 같다. 그래도 좋은 소식은\n한 달동안 근육량은 유지한채, 지방만 3kg정도 빠졌다. 1년 반동안 멈췄던 운동을 다시 시작하면서…","frontmatter":{"title":"[RECORD] 개인 운동 기록 - 2022.10.30","tags":["record"],"date":"2022-10-30T10:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAYAAACpUE5eAAAACXBIWXMAAAsTAAALEwEAmpwYAAADU0lEQVQ4yx2S7W8aBQDG74MzLV26dusLPShwcIU74Dg47igvpRw9jrfCWgqspe0mKy59S5bodLppph+cftCoycwy4xej1Q/G//Fn5A94fs/vSR5hLZ4mWq6iVF3itSbbg0OePHvJq+9+YDCecHw6wqnVaXbuoxTLOHs9jsZjnj3/lH9u/qR//IhlWUUpVYhVXIRoYZuE2yJZ30Vr7OIcHuPTDEQti7ZT52LUxTWStN069n6PsLHJ3tEJn734nL9+e8eXX7zCp+r4ExmUYgXB7h9idnvorS5avY0zOkUpbOMRg3RPJvz80Skj28KyXXZ6A2L5LYIZi4dnE25+fcu337wmY9fwRuNEdB1BylhIZgHdqaNUHMoHQzYKFWb8YbZqTa6HXa4eHWM4LQzHJb5lE8pYHByf8PvbN/z0/Y9UdrtY1TKqZSLImyWCuolXTeHP5DCaHXoPzxicjumPPmCv0ebs8BCzWEJMZYlk86xrBp3BkD/e/cK/N3/TGR6j5kusRjWElNsi1eyglHeQ8mWMegulUKEzHPHyq9ecf3jBi8kYt+qwEpDxqvp0st3pcnV9zcefPGf0eMJ2exfZyCHIW1WUSo2E00BvdlBLFd5bCZBIxCmnU2jpDF9fPWbcbNCv1ek/GDFsNel2upi1Oq39PS4vn0wL9K0KQri4TbhkozoN0s1dlsIqM8sB1LyNnK+ykbY4Hd7HyW+yld8kbVpsmhb7DYfe0RFRc5Pziwmt4QOKbhMhWt4h/P+Hqi7eeJpbd0VuixKzfpm5YIy5dZmqs0Otvcu9QAzXqjJy9jlv97kejKjldiha26TLDhmnhRAv2oiZHGuaMQWtqTp3ghvMrgbweNcpmAZ+OcbJfpvrfhdbL2GqFtl4jppRoW1WiWk51vQcvnQOQVR0ZsUQ67rJHV8Ev2awFE0w4w1yT4rytNfkqGGTzehcHjTYiCV4f2kdz2oAOZLEH4jh8UnMeoMsRlSEhVCU+eAGYjKDZyXAfEBmQYpx2ycRSGXRtBRPey5mMkE0LLPqD3Hrno/ZZT93RYkFb4h5n4RHDE2zwkJwg1UlNbWa84ZYDCvM+cLMByKIyTQeX5icaaCocRb90tRajOuI8RTeWJKViMpiMMqcT2JOlPgP7T62wL+HivUAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/b946d74ba30a0f569bd677b887e90add/20435/weight_training2.png","srcSet":"/static/b946d74ba30a0f569bd677b887e90add/ea847/weight_training2.png 750w,\n/static/b946d74ba30a0f569bd677b887e90add/4a806/weight_training2.png 1080w,\n/static/b946d74ba30a0f569bd677b887e90add/e2aad/weight_training2.png 1366w,\n/static/b946d74ba30a0f569bd677b887e90add/20435/weight_training2.png 1920w","sizes":"100vw"},"sources":[{"srcSet":"/static/b946d74ba30a0f569bd677b887e90add/57584/weight_training2.webp 750w,\n/static/b946d74ba30a0f569bd677b887e90add/984df/weight_training2.webp 1080w,\n/static/b946d74ba30a0f569bd677b887e90add/4a276/weight_training2.webp 1366w,\n/static/b946d74ba30a0f569bd677b887e90add/9c00f/weight_training2.webp 1920w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6666666666666666}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"2 min read"},"layout":"","slug":"/weight_training2/"}},"primaryTag":"huggingface"}}, "staticQueryHashes": ["3170763342","3229353822"]}