{ "componentChunkName": "component---src-templates-post-tsx", "path": "/metric/", "result": {"data":{"logo":null,"markdownRemark":{"html":"

NLP Metrics

\n

Confusion Matrix

\n\n

Confusion Matrix는 분류 모델을 평가할때 모델이 얼마나 정밀한지, 얼마나 실용적인 분류를 해냈는지, 얼마나 정확한 분류를 해냈는지에 대한 모든 내용을 포함하고 있습니다.

\n

Accuracy, Precision, Recall, F1-Score와 같은 성능 지표를 계산할 수 있습니다.

\n

Accuracy

\n

가장 간단하게 성능을 측정하는 방법인 Accuracy입니다.

\n

ex) Tunib-Eelctra Downstream tasks result

\n\n

위와 같이 다양한 task에서 accuracy가 사용됩니다.

\n

Accuracy는 올바르게 예측된 데이터의 수를 전체 데이터의 수로 나눈 값입니다.

\n\n즉 간단하게 예측값 중 일치한게 몇개인지 확인하는 방법입니다.\n

사용법

\n
>>> import torch\n>>> from torchmetrics.functional import accuracy\n>>> target = torch.tensor([0, 1, 2, 3])\n>>> preds = torch.tensor([0, 2, 1, 3])\n>>> accuracy(preds, target)\ntensor(0.5000)
\n

Recall

\n

accuracy는 데이터에 따라 잘못된 통계를 나타낼 수도 있습니다. 극단적인 예를 들자면, 정답의 비율이 False와 True가 9:1일 경우, 모두 False로 예측해버리면 True가 많지 않기 때문에 매우 높은 accuracy를 얻을 수 있습니다.

\n

이럴 때 사용하는 방법이 바로 Recall입니다.

\n\n

Recall은 실제로 True인 데이터를 모델이 True라고 예측한 데이터의 수입니다. 즉 True에 대한 예측결과값만 계산하는 것입니다.

\n

사용법

\n
>>> from torchmetrics.functional import recall\n>>> preds  = torch.tensor([2, 0, 2, 1])\n>>> target = torch.tensor([1, 1, 2, 0])\n>>> recall(preds, target, average='macro', num_classes=3)\ntensor(0.3333)\n>>> recall(preds, target, average='micro')\ntensor(0.2500)
\n

macro: 각 클래스에 대한 메트릭을 개별적으로 계산하고 클래스 전체의 metric 평균화\nmicro: 모든 샘플 및 클래스에 대해 전역적으로 metric 계산

\n

Precision

\n

역시 Recall도 한계가 있습니다. 정답의 비율이 False와 True가 9:1인 데이터를 다시 예시로 들면 모델이 전부 False로 예측한 것을 반대로 전부 True로 예측했다고 가정해보면 Recall이 1이 되는 것을 확인 할 수 있습니다.

\n

이런 경우 precision을 통해 새로운 지표를 얻을 수 있습니다.

\n

이전에 Recall은 정답이 True인 데이터중에서 모델이 True로 예측한 데이터의 수라면, Precision은 모델이 True라고 예측한 데이터 중 정답이 True인 데이터의 수입니다.

\n\n

Note: Precision과 recall은 서로 trade-off되는 관계입니다.

\n

사용법

\n
>>> from torchmetrics import Precision\n>>> preds  = torch.tensor([2, 0, 2, 1])\n>>> target = torch.tensor([1, 1, 2, 0])\n>>> precision = Precision(average='macro', num_classes=3)\n>>> precision(preds, target)\ntensor(0.1667)\n>>> precision = Precision(average='micro')\n>>> precision(preds, target)\ntensor(0.2500)
\n

F1 Score

\n

ex) HyperCLOVA Experimental Results

\n\n

앞선 설명을 통해 정반대의 성격을 가진 Precision과 Recall을 확인 할 수 있었습니다. 이번엔 정 반대의 성격을 가진 두 지표를 응용한 F1 Score입니다.

\n

F1 Score는 precision과 recall의 조화평균입니다.

\n\n

일반적인 평균이 아닌 조화 평균을 계산하였는데, 그 이유는 precision과 recall이 0에 가까울수록 F1 score도 동일하게 낮은 값을 갖도록 하기 위함입니다.

\n

사용법

\n
>>> from torchmetrics.functional import f1\n>>> target = torch.tensor([0, 1, 2, 0, 1, 2])\n>>> preds = torch.tensor([0, 2, 1, 0, 0, 1])\n>>> f1(preds, target, num_classes=3)\ntensor(0.3333)
\n

EM(Exact Match)

\n

EM은 말그대로 정확하게 전부 일치하는지 측정하는 simple한 metric입니다.

\n

모든 character가 일치하면 EM=1, 그 외의 경우는 모두 EM=0이 됩니다. EM=1인 sample수에 전체 sample수를 나눠 값을 구할 수 있습니다.

\n

BLEU Score(Bilingual Evaluation Understudy Score)

\n

BLEU는 기계 번역의 성능이 얼마나 뛰어난가를 측정하기 위해 사용되는 대표적인 방법으로 측정 기준은 n-gram과 precision에 기반합니다.

\n

n-gram

\n

n-gram은 n개의 연속적인 단어 나열을 의미합니다.

\n

ex) An adorable little boy is spreading smiles

\n
1-gram(unigrams) : an, adorable, little, boy, is, spreading, smiles\n2-gram(bigrams) : an adorable, adorable little, little boy, boy is, is spreading, spreading smiles\n3-gram(trigrams) : an adorable little, adorable little boy, little boy is, boy is spreading, is spreading smiles\n4-grams : an adorable little boy, adorable little boy is, little boy is spreading, boy is spreading smiles
\n

1. 단어 개수 카운트로 측정하기(Unigram Precision)

\n

단순하게 일치하는 단어의 개수로 성능을 측정하는 방법입니다.

\n
\n

ex)\n번역기로 번역된 문장: Candidate, 사람이 직접 번역한 문장: Reference

\n\n

2. Modified Unigram Precision

\n\n
Candidate : the the the the the the the\n\nReference1 : the cat is on the mat\n\nReference2 : there is a cat on the mat
\n

예시의 candidate는 the만 7개나 나오는 말도 안되는 번역이지만 일반 Unigram Precision방법을 사용하면 1이라는 최고의 성능이 나오게 됩니다.

\n\n

따라서 Reference에 등장하는 단어의 max_count를 고려합니다.

\n\n

3. n-gram으로 확장

\n

이번엔 단어의 순서를 고려하기 위해 unigram이 아닌 bigram이상의 n-gram을 고려하는 방법입니다.

\n
Candidate2 : the cat the cat on the mat\n\nReference1 : the cat is on the mat\n\nReference2 : there is a cat on the mat
\n\n

결과적으로 ca2의 바이그램 정밀도는 6분의 4가 됩니다.

\n

이렇게 n-gram을 이용해 성능을 평가하는 방법이 BLEU입니다.

\n

사용법

\n\n
>>> from torchtext.data.metrics import bleu_score\n>>> candidate_corpus = [['My', 'full', 'pytorch', 'test'], ['Another', 'Sentence']]\n>>> references_corpus = [[['My', 'full', 'pytorch', 'test'], ['Completely', 'Different']], [['No', 'Match']]]\n>>> bleu_score(candidate_corpus, references_corpus)\n    0.8408964276313782
\n

ROUGE(Recall-Oriented Understudy for Gisting Evaluation)

\n

ROUGE는 텍스트 요약, 기계번역과 같은 task를 평가하기 위해 사용되는 대표적인 Metric입니다.

\n

이전에 설명 드렸던 BLEU가 n-gram precision에 기반한 지표라면, ROUGE는 이름 그대로 n-gram Recall에 기반해 계산됩니다.

\n

1. ROUGE-N

\n

ROUGE계산시 n-gram에 따라 다르게 계산하는 것이 ROUGE-N기법입니다.

\n

ex) ROUGE-1: unigram

\n\n

ex) ROUGE-2: bigram

\n\n

2. ROUGE-L

\n

가장 긴 Sequence의 recall을 구하며, Sequence는 이어지지 않아도 됩니다.

\n\n

생성된 문장의 예시와 정답문장이 완전히 일치하지는 않지만, 떨어져 있는 Sequence 형태로 정답문장과 일치하기 때문에 1의 ROUGE-L score를 얻을 수 있습니다.

\n

3. ROUGE-W

\n

ROUGE-W는 ROUGE-L의 방법에서 연속적인 매칭(consecutive matches)에 가중치를 주는 방법입니다.

\n\n

ROUGE-L의 관점에서는 Y_1과 Y_2의 결과가 같지만,ROUGE-W의 관점에서는 consecutive matches로 이루어진 예시인 Y1이 더 좋은 결과가 됩니다.

\n

4. ROUGE-S

\n

ROUGE-S는 최대 2칸(bigram) 내에 위치하는 단어 쌍의 recall을 계산합니다.

\n\n

5. ROUGE-SU

\n

ROUGE-SU는 ROUGE-S의 확장된 버전입니다.

\n

아래 예시의 경우 어순을 바꿨을 뿐, 같은 의미를 가진 문장임에도 ROUGE-S가 0이 되어버립니다.

\n
정답문장 : 류현진이 공을 던졌다.\n생성문장 : 던졌다 공을 류현진이
\n

ROUGE-SU는 Unigram을 함께 계산하여 이를 보정해줍니다.

\n
정답문장 : ((류현진,공), (류현진,던졌다), (공,던졌다), 류현진, 공, 던졌다)\n생성문장 : ((던졌다,공), (던졌다,류현진), (공,류현진), 류현진, 공, 던졌다)
\n\n

사용법

\n
>>> targets = \"Is your name John\".split()\n>>> preds = \"My name is John\".split()\n>>> rouge = ROUGEScore()   \n>>> from pprint import pprint\n>>> pprint(rouge(preds, targets))  \n{'rouge1_fmeasure': 0.25,\n 'rouge1_precision': 0.25,\n 'rouge1_recall': 0.25,\n 'rouge2_fmeasure': 0.0,\n 'rouge2_precision': 0.0,\n 'rouge2_recall': 0.0,\n 'rougeL_fmeasure': 0.25,\n 'rougeL_precision': 0.25,\n 'rougeL_recall': 0.25,\n 'rougeLsum_fmeasure': 0.25,\n 'rougeLsum_precision': 0.25,\n 'rougeLsum_recall': 0.25}
\n

CER(문자 오류율)

\n

CER계산은 Levenshtein distance의 개념을 기반으로 합니다.

\n

Levenshtein distance

\n

Levenshtein distance는 두 문자열 시퀀스 간의 차이를 측정하는 거리 측정법입니다.

\n\n

Levenshtein distance는 위와 같이 세 가지의 오류를 고려합니다.

\n

ex) mitten & fitting

\n
m itten → f itten ( m 을 f로 대체)\nfitt e n → fitt i n ( e 를 i로 대체 )\nfittin → fittin g ( 끝에 g 삽입 )
\n

한 문자열을 다른 것으로 변환하려면 최소 3번의 과정의 필요하기 때문에 두 문자열간의 Levenshtein distance는 3입니다.

\n

CER(공식)

\n\n
S = substitutions\nD = deletions\nI = insertions\nN = 전체 문자 수
\n

즉 Levenshtein distance에 전체 문자 수를 나눈 값이 CER입니다.

\n

WER(단어 오류율)

\n

WER은 문단, 문장에서 사용되며 CER의 단위를 character가 아닌 워드 단위로 계산한 값입니다.

\n\n

즉 한 문장을 다른 문장으로 변환하는데 필요한 단어의 Levenshtein distance에 전체 단어수를 나눈 값입니다.

\n

사용법

\n
>>> predictions = [\"this is the prediction\", \"there is an other sample\"]\n>>> references = [\"this is the reference\", \"there is another one\"]\n>>> wer(predictions=predictions, references=references)\ntensor(0.5000)
\n

Pearson correlation coefficient

\n

Pearson correlation coefficient는 두 변수의 선형 상관 관계를 계량화한 것 입니다.

\n

결과값을 -1 ~ 1 사이의 값이며, 서로 비슷할수록 1에 가까워지고, 0일경우 연관x, -1에 가까워 질수록 정반대를 뜻합니다.

\n

Pearson 상관 계수는 이상치의 영향을 많이 받습니다. 따라서 이상치가 존재할 경우 상관 계수의 값이 크게 변경될 수 있으므로 사전에 이상치제거가 필요합니다.

\n

사용법

\n
>>> from torchmetrics.functional import pearson_corrcoef\n>>> target = torch.tensor([3, -0.5, 2, 7])\n>>> preds = torch.tensor([2.5, 0.0, 2, 8])\n>>> pearson_corrcoef(preds, target)\ntensor(0.9849)
\n

Spearman’s correlation coefficient

\n

Spearman’s correlation coefficient는 두 변수의 상관 관계를 계량화한 것 입니다.

\n

Pearson과 동일하게 -1 ~ 1 사이의 값을 가지며, 서로 비슷할수록 1에 가까워지고, 0일경우 연관x, -1에 가까워 질수록 정반대를 뜻합니다.

\n

피어슨 상관 계수와의 큰 차이점은 피어슨 상관 계수는 선형 상관 관계이지만 스피어만 상관계수는 그냥 상관 관계에 대한 값입니다. 따라서 두 변수가 꼭 선형적인 관계를 가질 필요가 없습니다.

\n

사용법

\n
>>> from torchmetrics import SpearmanCorrcoef\n>>> target = torch.tensor([3, -0.5, 2, 7])\n>>> preds = torch.tensor([2.5, 0.0, 2, 8])\n>>> spearman = SpearmanCorrcoef()\n>>> spearman(preds, target)\ntensor(1.0000)
\n

TASK별 metric정리

\n
MNLI (matched or mismatched): Accuracy\nMRPC: Accuracy and F1 score\nQNLI: Accuracy\nQQP: Accuracy and F1 score\nRTE: Accuracy\nSST-2: Accuracy\nSTS-B: Pearson Correlation Coefficient and Spearman's_Rank_Correlation_Coefficient\nWNLI: Accuracy
\n

Reference

\n","htmlAst":{"type":"root","children":[{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"NLP Metrics"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Confusion Matrix"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136820363-585e74ec-3332-43da-abc3-7445bbb1f7c3.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Confusion Matrix는 분류 모델을 평가할때 모델이 얼마나 정밀한지, 얼마나 실용적인 분류를 해냈는지, 얼마나 정확한 분류를 해냈는지에 대한 모든 내용을 포함하고 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Accuracy, Precision, Recall, F1-Score와 같은 성능 지표를 계산할 수 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Accuracy"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"가장 간단하게 성능을 측정하는 방법인 Accuracy입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ex) Tunib-Eelctra Downstream tasks result"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136825941-8a8a5d65-4617-45fe-8337-0d8b8cf7557a.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"위와 같이 다양한 task에서 accuracy가 사용됩니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Accuracy는 올바르게 예측된 데이터의 수를 전체 데이터의 수로 나눈 값입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136826193-11d3a8b7-f5a4-4d9e-8a51-f521de663270.png","width":500},"children":[]},{"type":"text","value":"\n즉 간단하게 예측값 중 일치한게 몇개인지 확인하는 방법입니다.\n"},{"type":"element","tagName":"h4","properties":{},"children":[{"type":"text","value":"사용법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" torch\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" torchmetrics"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"functional "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" accuracy\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" target "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"tensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"3"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" preds "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"tensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"3"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" accuracy"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"preds"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" target"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\ntensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.5000"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Recall"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"accuracy는 데이터에 따라 잘못된 통계를 나타낼 수도 있습니다. 극단적인 예를 들자면, 정답의 비율이 False와 True가 9:1일 경우, 모두 False로 예측해버리면 True가 많지 않기 때문에 매우 높은 accuracy를 얻을 수 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"이럴 때 사용하는 방법이 바로 Recall입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136832571-70189d00-2e3d-4ab6-b74f-40b4b3289e60.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Recall은 실제로 True인 데이터를 모델이 True라고 예측한 데이터의 수입니다. 즉 True에 대한 예측결과값만 계산하는 것입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h4","properties":{},"children":[{"type":"text","value":"사용법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" torchmetrics"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"functional "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" recall\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" preds "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"tensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" target "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"tensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" recall"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"preds"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" target"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" average"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'macro'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" num_classes"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"3"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\ntensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.3333"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" recall"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"preds"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" target"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" average"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'micro'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\ntensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.2500"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"macro"}]},{"type":"text","value":": 각 클래스에 대한 메트릭을 개별적으로 계산하고 클래스 전체의 metric 평균화\n"},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"micro"}]},{"type":"text","value":": 모든 샘플 및 클래스에 대해 전역적으로 metric 계산"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Precision"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"역시 Recall도 한계가 있습니다. 정답의 비율이 False와 True가 9:1인 데이터를 다시 예시로 들면 모델이 전부 False로 예측한 것을 반대로 전부 True로 예측했다고 가정해보면 Recall이 1이 되는 것을 확인 할 수 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"이런 경우 precision을 통해 새로운 지표를 얻을 수 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"이전에 Recall은 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"정답"}]},{"type":"text","value":"이 True인 데이터중에서 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"모델"}]},{"type":"text","value":"이 True로 예측한 데이터의 수라면, Precision은 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"모델"}]},{"type":"text","value":"이 True라고 예측한 데이터 중 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"정답"}]},{"type":"text","value":"이 True인 데이터의 수입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136833856-f031bca2-0014-42ca-936b-14427a46a294.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Note: Precision과 recall은 서로 trade-off되는 관계입니다."}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"사용법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" torchmetrics "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" Precision\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" preds "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"tensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" target "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"tensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" precision "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" Precision"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"average"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'macro'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" num_classes"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"3"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" precision"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"preds"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" target"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\ntensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.1667"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" precision "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" Precision"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"average"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'micro'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" precision"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"preds"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" target"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\ntensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.2500"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"F1 Score"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ex) HyperCLOVA Experimental Results"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136829590-3f31851b-aa42-4921-89a1-0325f7cf253d.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"앞선 설명을 통해 정반대의 성격을 가진 Precision과 Recall을 확인 할 수 있었습니다. 이번엔 정 반대의 성격을 가진 두 지표를 응용한 F1 Score입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"F1 Score는 precision과 recall의 조화평균입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136835253-77b7beeb-7f3e-4cf0-8db7-40c88b3b0097.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"일반적인 평균이 아닌 조화 평균을 계산하였는데, 그 이유는 precision과 recall이 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"0"}]},{"type":"text","value":"에 가까울수록 F1 score도 동일하게 낮은 값을 갖도록 하기 위함입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h4","properties":{},"children":[{"type":"text","value":"사용법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" torchmetrics"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"functional "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" f1\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" target "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"tensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" preds "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"tensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" f1"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"preds"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" target"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" num_classes"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"3"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\ntensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.3333"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"EM(Exact Match)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"EM은 말그대로 정확하게 전부 일치하는지 측정하는 simple한 metric입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"모든 character가 일치하면 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"EM=1"}]},{"type":"text","value":", 그 외의 경우는 모두 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"EM=0"}]},{"type":"text","value":"이 됩니다. EM=1인 sample수에 전체 sample수를 나눠 값을 구할 수 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"BLEU Score(Bilingual Evaluation Understudy Score)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"BLEU는 기계 번역의 성능이 얼마나 뛰어난가를 측정하기 위해 사용되는 대표적인 방법으로 측정 기준은 n-gram과 precision에 기반합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"n-gram"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"n-gram은 n개의 연속적인 단어 나열을 의미합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ex) "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"An adorable little boy is spreading smiles"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"1-gram(unigrams) : an, adorable, little, boy, is, spreading, smiles\n2-gram(bigrams) : an adorable, adorable little, little boy, boy is, is spreading, spreading smiles\n3-gram(trigrams) : an adorable little, adorable little boy, little boy is, boy is spreading, is spreading smiles\n4-grams : an adorable little boy, adorable little boy is, little boy is spreading, boy is spreading smiles"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"1. 단어 개수 카운트로 측정하기(Unigram Precision)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"단순하게 일치하는 단어의 개수로 성능을 측정하는 방법입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ex)\n번역기로 번역된 문장: Candidate, 사람이 직접 번역한 문장: Reference"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136810698-9fcee7a2-33b3-4a18-92a7-c8b0eb5ce40a.png","width":700},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"2. Modified Unigram Precision"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Modified Unigram Precision은 중복되는 단어는 제거하여 측정하는 방법입니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"angular2html"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-angular2html"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-angular2html"]},"children":[{"type":"text","value":"Candidate : the the the the the the the\n\nReference1 : the cat is on the mat\n\nReference2 : there is a cat on the mat"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"예시의 candidate는 the만 7개나 나오는 말도 안되는 번역이지만 일반 Unigram Precision방법을 사용하면 1이라는 최고의 성능이 나오게 됩니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136811641-7380d02f-d983-4c68-87d4-4607c3d00ccf.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"따라서 Reference에 등장하는 단어의 max_count를 고려합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136812473-3df9dfbb-f8df-474c-b160-3457162c3d24.png","width":700},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"3. n-gram으로 확장"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"이번엔 단어의 순서를 고려하기 위해 unigram이 아닌 bigram이상의 n-gram을 고려하는 방법입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"angular2html"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-angular2html"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-angular2html"]},"children":[{"type":"text","value":"Candidate2 : the cat the cat on the mat\n\nReference1 : the cat is on the mat\n\nReference2 : there is a cat on the mat"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136819439-404d8c8c-944b-4475-b107-288b01844fb3.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"결과적으로 ca2의 바이그램 정밀도는 6분의 4가 됩니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"이렇게 n-gram을 이용해 성능을 평가하는 방법이 BLEU입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"사용법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136819917-8cced83d-e9a0-4587-9fdc-51638e6bd6ff.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" torchtext"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"data"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"metrics "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" bleu_score\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" candidate_corpus "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'My'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'full'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'pytorch'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'test'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'Another'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'Sentence'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" references_corpus "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'My'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'full'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'pytorch'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'test'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'Completely'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'Different'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'No'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'Match'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" bleu_score"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"candidate_corpus"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" references_corpus"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.8408964276313782"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"ROUGE(Recall-Oriented Understudy for Gisting Evaluation)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ROUGE는 텍스트 요약, 기계번역과 같은 task를 평가하기 위해 사용되는 대표적인 Metric입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"이전에 설명 드렸던 BLEU가 n-gram precision에 기반한 지표라면, ROUGE는 이름 그대로 n-gram Recall에 기반해 계산됩니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"1. ROUGE-N"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ROUGE계산시 n-gram에 따라 다르게 계산하는 것이 ROUGE-N기법입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ex) ROUGE-1: unigram"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136845207-e98238bc-10f5-4d46-9a7c-c0e8e5ab5dd7.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ex) ROUGE-2: bigram"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136845309-6376c71a-7450-4757-aa31-2ea20378b7db.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"2. ROUGE-L"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"가장 긴 Sequence의 recall을 구하며, Sequence는 이어지지 않아도 됩니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136845529-e57f3563-a40b-494b-b671-8c631d5e0129.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"생성된 문장의 예시와 정답문장이 완전히 일치하지는 않지만, 떨어져 있는 Sequence 형태로 정답문장과 일치하기 때문에 1의 ROUGE-L score를 얻을 수 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"3. ROUGE-W"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ROUGE-W는 ROUGE-L의 방법에서 연속적인 매칭(consecutive matches)에 가중치를 주는 방법입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136846008-e31e76bb-ae8c-4275-a254-c59cc292130f.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ROUGE-L의 관점에서는 Y_1과 Y_2의 결과가 같지만,ROUGE-W의 관점에서는 consecutive matches로 이루어진 예시인 Y1이 더 좋은 결과가 됩니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"4. ROUGE-S"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ROUGE-S는 최대 2칸(bigram) 내에 위치하는 단어 쌍의 recall을 계산합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136846246-741d10f1-5abe-410b-b5f4-f17a59351b42.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"5. ROUGE-SU"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ROUGE-SU는 ROUGE-S의 확장된 버전입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"아래 예시의 경우 어순을 바꿨을 뿐, 같은 의미를 가진 문장임에도 ROUGE-S가 0이 되어버립니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"angular2html"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-angular2html"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-angular2html"]},"children":[{"type":"text","value":"정답문장 : 류현진이 공을 던졌다.\n생성문장 : 던졌다 공을 류현진이"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ROUGE-SU는 Unigram을 함께 계산하여 이를 보정해줍니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"angular2html"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-angular2html"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-angular2html"]},"children":[{"type":"text","value":"정답문장 : ((류현진,공), (류현진,던졌다), (공,던졌다), 류현진, 공, 던졌다)\n생성문장 : ((던졌다,공), (던졌다,류현진), (공,류현진), 류현진, 공, 던졌다)"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136846643-e58feaac-857a-4e58-9ae0-2354f60741f4.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"사용법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" targets "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"Is your name John\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"split"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" preds "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"My name is John\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"split"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" rouge "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" ROUGEScore"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":" \n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" pprint "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" pprint\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" pprint"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"rouge"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"preds"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" targets"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":" \n"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"{"}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'rouge1_fmeasure'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.25"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'rouge1_precision'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.25"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'rouge1_recall'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.25"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'rouge2_fmeasure'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'rouge2_precision'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'rouge2_recall'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'rougeL_fmeasure'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.25"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'rougeL_precision'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.25"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'rougeL_recall'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.25"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'rougeLsum_fmeasure'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.25"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'rougeLsum_precision'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.25"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"'rougeLsum_recall'"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.25"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"}"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"CER(문자 오류율)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"CER계산은 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Levenshtein distance"}]},{"type":"text","value":"의 개념을 기반으로 합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Levenshtein distance"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Levenshtein distance는 두 문자열 시퀀스 간의 차이를 측정하는 거리 측정법입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136838740-658bf758-51b4-41a6-abda-dfc8698694a0.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Levenshtein distance는 위와 같이 세 가지의 오류를 고려합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ex) "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"mitten"}]},{"type":"text","value":" & "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"fitting"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"angular2html"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-angular2html"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-angular2html"]},"children":[{"type":"text","value":"m itten → f itten ( m 을 f로 대체)\nfitt e n → fitt i n ( e 를 i로 대체 )\nfittin → fittin g ( 끝에 g 삽입 )"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"한 문자열을 다른 것으로 변환하려면 최소 3번의 과정의 필요하기 때문에 두 문자열간의 Levenshtein distance는 3입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h4","properties":{},"children":[{"type":"text","value":"CER(공식)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136839223-106671a0-7c42-465d-9ede-68dffad95d4b.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"angular2html"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-angular2html"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-angular2html"]},"children":[{"type":"text","value":"S = substitutions\nD = deletions\nI = insertions\nN = 전체 문자 수"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"즉 Levenshtein distance에 전체 문자 수를 나눈 값이 CER입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"WER(단어 오류율)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"WER은 문단, 문장에서 사용되며 CER의 단위를 character가 아닌 워드 단위로 계산한 값입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/59256704/136839771-6b3bdf81-d8d7-4287-b7f5-e038b864df85.png","width":500},"children":[]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"즉 한 문장을 다른 문장으로 변환하는데 필요한 단어의 Levenshtein distance에 전체 단어수를 나눈 값입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h4","properties":{},"children":[{"type":"text","value":"사용법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" predictions "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"this is the prediction\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"there is an other sample\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" references "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"this is the reference\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"there is another one\""}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" wer"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"predictions"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":"predictions"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" references"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":"references"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\ntensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.5000"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Pearson correlation coefficient"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Pearson correlation coefficient는 두 변수의 선형 상관 관계를 계량화한 것 입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"결과값을 -1 ~ 1 사이의 값이며, 서로 비슷할수록 1에 가까워지고, 0일경우 연관x, -1에 가까워 질수록 정반대를 뜻합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Pearson 상관 계수는 이상치의 영향을 많이 받습니다. 따라서 이상치가 존재할 경우 상관 계수의 값이 크게 변경될 수 있으므로 사전에 이상치제거가 필요합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h4","properties":{},"children":[{"type":"text","value":"사용법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" torchmetrics"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"functional "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" pearson_corrcoef\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" target "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"tensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"3"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"-"}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.5"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"7"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" preds "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"tensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2.5"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"8"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" pearson_corrcoef"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"preds"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" target"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\ntensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.9849"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Spearman’s correlation coefficient"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Spearman’s correlation coefficient는 두 변수의 상관 관계를 계량화한 것 입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Pearson과 동일하게 -1 ~ 1 사이의 값을 가지며, 서로 비슷할수록 1에 가까워지고, 0일경우 연관x, -1에 가까워 질수록 정반대를 뜻합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"피어슨 상관 계수와의 큰 차이점은 피어슨 상관 계수는 선형 상관 관계이지만 스피어만 상관계수는 그냥 상관 관계에 대한 값입니다. 따라서 두 변수가 꼭 선형적인 관계를 가질 필요가 없습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h4","properties":{},"children":[{"type":"text","value":"사용법"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"from"}]},{"type":"text","value":" torchmetrics "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"import"}]},{"type":"text","value":" SpearmanCorrcoef\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" target "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"tensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"3"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"-"}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.5"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"7"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" preds "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"tensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2.5"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.0"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"8"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" spearman "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" SpearmanCorrcoef"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">>"}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":">"}]},{"type":"text","value":" spearman"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"preds"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" target"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\ntensor"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1.0000"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"TASK별 metric정리"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"angular2html"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-angular2html"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-angular2html"]},"children":[{"type":"text","value":"MNLI (matched or mismatched): Accuracy\nMRPC: Accuracy and F1 score\nQNLI: Accuracy\nQQP: Accuracy and F1 score\nRTE: Accuracy\nSST-2: Accuracy\nSTS-B: Pearson Correlation Coefficient and Spearman's_Rank_Correlation_Coefficient\nWNLI: Accuracy"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Reference"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://wikidocs.net/31695"},"children":[{"type":"text","value":"https://wikidocs.net/31695"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://eunsukimme.github.io/ml/2019/10/21/Accuracy-Recall-Precision-F1-score/"},"children":[{"type":"text","value":"https://eunsukimme.github.io/ml/2019/10/21/Accuracy-Recall-Precision-F1-score/"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://torchmetrics.rtfd.io/en/latest/"},"children":[{"type":"text","value":"https://torchmetrics.rtfd.io/en/latest/"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://arxiv.org/pdf/2109.04650.pdf"},"children":[{"type":"text","value":"https://arxiv.org/pdf/2109.04650.pdf"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://supkoon.tistory.com/26"},"children":[{"type":"text","value":"https://supkoon.tistory.com/26"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://github.com/tunib-ai/tunib-electra"},"children":[{"type":"text","value":"https://github.com/tunib-ai/tunib-electra"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://qa.fastforwardlabs.com/no%20answer/null%20threshold/bert/distilbert/exact%20match/f1/robust%20predictions/2020/06/09/Evaluating_BERT_on_SQuAD.html#Exact-Match"},"children":[{"type":"text","value":"https://qa.fastforwardlabs.com/no%20answer/null%20threshold/bert/distilbert/exact%20match/f1/robust%20predictions/2020/06/09/Evaluating_BERT_on_SQuAD.html#Exact-Match"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://lunch-box.tistory.com/109"},"children":[{"type":"text","value":"https://lunch-box.tistory.com/109"}]}]},{"type":"text","value":"\n"}]}],"data":{"quirksMode":false}},"excerpt":"NLP Metrics Confusion Matrix Confusion Matrix는 분류 모델을 평가할때 모델이 얼마나 정밀한지, 얼마나 실용적인 분류를 해냈는지, 얼마나 정확한 분류를 해냈는지에 대한 모든 내용을 포함하고 있습니다. Accuracy…","fields":{"readingTime":{"text":"14 min read"}},"frontmatter":{"title":"Sooftware NLP - NLP Metrics","userDate":"13 October 2021","date":"2021-10-13T10:00:00.000Z","tags":["nlp","metric"],"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#f8f8f8","images":{"fallback":{"src":"/static/1edcc9c2169958556ac8c9ac51ec28de/a5b23/cm.png","srcSet":"/static/1edcc9c2169958556ac8c9ac51ec28de/ea251/cm.png 750w,\n/static/1edcc9c2169958556ac8c9ac51ec28de/a5b23/cm.png 860w","sizes":"100vw"},"sources":[{"srcSet":"/static/1edcc9c2169958556ac8c9ac51ec28de/678bf/cm.webp 750w,\n/static/1edcc9c2169958556ac8c9ac51ec28de/d78b9/cm.webp 860w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5034883720930232}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"children":[{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#282838","images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/248f9/soohwan.png 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/fd435/soohwan.png 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png 120w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/e7f45/soohwan.webp 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/589ec/soohwan.webp 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/71a38/soohwan.webp 120w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6833333333333333}}]}}]}},"relatedPosts":{"totalCount":50,"edges":[{"node":{"id":"6529d72e-80e7-5d61-a672-d66276a3f641","excerpt":"MoE(Mixture of Experts) 기초부터 DeepSeek 혁신까지 작년 이맘때쯤 DeepSeek-V3가 저비용으로 엄청난 성능을 보이면서 화제가 되었습니다. 그 핵심 기술인 MoE(Mixture of Experts…","frontmatter":{"title":"MoE(Mixture of Experts) 기초부터 DeepSeek 혁신까지","date":"2026-01-15T01:11:55.000Z"},"fields":{"readingTime":{"text":"8 min read"},"slug":"/developed-moe/"}}},{"node":{"id":"8b49d3ef-4ce3-568c-9d71-240ff17fc3e0","excerpt":"BERT는 사실 Diffusion 모델이였다?! 최근 굉장히 흥미로운 글을 읽게되어 공유합니다. 원문 : link BERT와 Diffusion이 같은 방식이다?! NLP 연구자들에게 BERT는 너무 익숙한 모델입니다. 201…","frontmatter":{"title":"BERT는 사실 Diffusion 모델이였다?!","date":"2025-10-21T12:00:00.000Z"},"fields":{"readingTime":{"text":"8 min read"},"slug":"/bert_diffusion/"}}},{"node":{"id":"7c7e1676-ea84-58de-970c-ddec9aa10660","excerpt":"RLHF는 수다쟁이를 만든다?! (Does RLHF Breed Verbose Chatterboxes?!) RLHF(Reinforcement Learning from Human Feedback)는 OpenAI의 ChatGPT…","frontmatter":{"title":"RLHF는 수다쟁이를 만든다?! (Does RLHF Breed Verbose Chatterboxes?!)","date":"2024-03-13T01:11:55.000Z"},"fields":{"readingTime":{"text":"7 min read"},"slug":"/rlhf-vervosity/"}}},{"node":{"id":"2b9d3e22-1796-5fab-878d-5941d2e76e9d","excerpt":"LLM Paper Abstract - 2023.12 LLM…","frontmatter":{"title":"LLM Paper Abstract - 2023.12","date":"2024-01-05T10:00:00.000Z"},"fields":{"readingTime":{"text":"5 min read"},"slug":"/llm-abs-202312/"}}},{"node":{"id":"f43fa33b-917b-5c00-8462-937d99592ad7","excerpt":"What it MoE? (Mixture of Experts) 현존 최강 LLM인 GPT-4에서 “MoE (Mixture of Experts)” 방식을 채택하여 사용하고 있다고 알려졌는데요, 최근 AI계의 뜨거운 감자 Mistral AI…","frontmatter":{"title":"What is MoE? (Mixture of Experts)","date":"2023-12-22T01:11:55.000Z"},"fields":{"readingTime":{"text":"9 min read"},"slug":"/moe/"}}}]}},"pageContext":{"slug":"/metric/","prev":{"excerpt":"2021 SSDC 발표 - “TUNiB Electra 개발기” 영광스럽게도 이번에 열리는 SSDC(구 SOSCON) - Samsung Software Developer Conference…","frontmatter":{"title":"2021 SSDC 발표 - \"TUNiB Electra 개발기\"","tags":["presentation"],"date":"2021-10-10T11:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAYAAAC0VX7mAAAACXBIWXMAAAsTAAALEwEAmpwYAAAB7klEQVQoz42S7W4SQRiF9zZN/Osfr8AbMPGXMV6AtVAsidZWqC2x2oo1RkgrqRTK1yJQCrTswi77CezMY3ZqjSZqPMnJeZOZOfPOvEfjT5AgpUAIgZRS8WeNQAy+IXcfELRvUZFFPooxNdlQR7Va9ZTq6Qn1szKlUoGmXmOJ4G+QoFbFPIDqPVjcwcbhbaQziaZoGzsHvNh9z0buA6mtN+x9OqZnXFIftmn3+1iWzdVoxHmvh2dZhMNz7JMjzMJX/IMVouFtYEB96WMIGy1dmrGvh2TLFisFm8QXn+18ifXUU9bSSdKbz0lubJJMrXN50QPnimhQIey/Y75/Fy7u4yL4vJzQ80y046MiuZ0sydUnZLdekn62RibzCl3X1RP9mcFoUMfoN5gOuniWwWJmIbolOHyIcItUcMkFOQq5Q7TVRIJMJovrebiup9SyLHzPIxICEUXIwCU0Ozi1PFY+ifP6EfO9x1jdHPqiSU00MKMznGoardls0el0CMMQ3/dZLBb8G3PAQTAnIMSTXhyJ64HJEC02iM1ms5liEAQ4joNt2+qCWH+ng+N4iEhej/xGpFCqVSoVyuU4MiVarZaq2+020+lUGYzHYwzDUGqaptKYy+XyR1fXOb2ptTgW8Z9NJlNc11Wb4y7/BzdGv+I76pTmcSeDVFoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/1da160f26ed9178318fc117c78f33b63/e2998/ssdc.png","srcSet":"/static/1da160f26ed9178318fc117c78f33b63/719b7/ssdc.png 750w,\n/static/1da160f26ed9178318fc117c78f33b63/e2998/ssdc.png 780w","sizes":"100vw"},"sources":[{"srcSet":"/static/1da160f26ed9178318fc117c78f33b63/bcae1/ssdc.webp 750w,\n/static/1da160f26ed9178318fc117c78f33b63/9e1fb/ssdc.webp 780w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.47692307692307695}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"1 min read"},"layout":"","slug":"/ssdc_2021/"}},"next":{"excerpt":"Wandb (Weights & Bias) Image Log Wandb 라이브러리는 최근에 가장 편리하면서도 파워풀한 logging 라이브러리입니다. NLP에서 많이 쓰이는 PyTorch, PyTorch-Lightning, Huggingface…","frontmatter":{"title":"Sooftware ML - Wandb Image Log","tags":["toolkit","logging"],"date":"2021-10-13T22:00:00.000Z","draft":false,"excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAYAAAB/Ca1DAAAACXBIWXMAAAsTAAALEwEAmpwYAAABuklEQVQoz52Su2sVQRSHp7eUVIKVrYUIdnZ2Vmn8GwwpAio2YuGjkBSJBGKRxAhRLqKFSC4mBDFqTHxdTQiBSxQlUeTi7qI32WQf8/pkdnYXBQvjgbPMnpnzze+cOcIYg3NnFktlVdxaC8619Ges5fccMKX7HKG1/1lqthgZmGT8QoPga+STjUVpw9jcAtfGprkz96YGOvvciRi5McP1S/dpvf5YxIRSulhc7ZvgoDjJYXGKlaX1GpjkOUcvX2Ffbz8nBocw2qBL4KP2Gj2959h/6DTDt2Y9sFL4/u0Hbt98TGPqKVG4VQONNUyvvmRy/gnNdqtoSqWwE4c0Fp8x8fA5K5++eKArQSlFlECQU4I0xvje7WaKaAd2yj131uVYq9natQQxdeeV1gh3myvbJXZTXSrQ/kGMIcslYTch6KZkUhbAai9OMjo/En7GGUpKXLWibvDGd+Zn3vFidpnt7aR+tb+ZV2j/jJVf4WQ6u3i3yYHjZzhy7DzL65tFzN1YJf+rC2088MHqK/pHpzg7dI+NIPTAaj73AqzmKkkNcepH25q9K6uBDialZDNMaX9LiZOcPM8Kdf8D/AXrmDiJkhItEQAAAABJRU5ErkJggg=="},"images":{"fallback":{"src":"/static/fd6ffa741fe53de299a57e6a8852f68d/2add8/wandb_image.png","srcSet":"/static/fd6ffa741fe53de299a57e6a8852f68d/5a15a/wandb_image.png 750w,\n/static/fd6ffa741fe53de299a57e6a8852f68d/20c21/wandb_image.png 1080w,\n/static/fd6ffa741fe53de299a57e6a8852f68d/397a5/wandb_image.png 1366w,\n/static/fd6ffa741fe53de299a57e6a8852f68d/2add8/wandb_image.png 1920w","sizes":"100vw"},"sources":[{"srcSet":"/static/fd6ffa741fe53de299a57e6a8852f68d/d7d73/wandb_image.webp 750w,\n/static/fd6ffa741fe53de299a57e6a8852f68d/525de/wandb_image.webp 1080w,\n/static/fd6ffa741fe53de299a57e6a8852f68d/f312c/wandb_image.webp 1366w,\n/static/fd6ffa741fe53de299a57e6a8852f68d/42023/wandb_image.webp 1920w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.5359375000000001}}},"author":[{"id":"Soohwan Kim","bio":"Co-founder/A.I. engineer at TUNiB.","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"3 min read"},"layout":"","slug":"/wandb_image/"}},"primaryTag":"nlp"}}, "staticQueryHashes": ["3170763342","3229353822"]}