{ "componentChunkName": "component---src-templates-post-tsx", "path": "/kv-cache/", "result": {"data":{"logo":null,"markdownRemark":{"html":"

KV Cache 이해해보기 - 어텐션부터 vLLM까지

\n

최근 팀내에서 발표했던 KV Cache 내용을 글로 정리해봤습니다. AI 엔지니어 분들이 아닌, 백엔드/프론트 엔지니어 분들이나 비개발자 분들도 이해하실 수 있는걸 목표로 나름 고민해서 구성했습니다. 스킵할건 스킵하고, 전체적인 흐름이 어떻게 되는건지에 대한 감을 잡을 수 있게하는걸 목표로 했습니다. 어텐션이나 트랜스포머를 몰라도 읽을 수 있게, 필요한 만큼만 처음부터 쌓아 올려보겠습니다.

\n

“나는 당근마켓의 카키에요.” 를 영어로 번역시키는 걸 예시로 설명을 해보겠습니다.

\n

0. 이런 경험, 있지 않나요?

\n

LLM API를 써봤다면 아래 현상들을 한 번쯤 겪어봤을 겁니다.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
현상설명
1첫 글자만 유독 느리다스트리밍 응답에서 첫 토큰까지는 한참 걸리는데, 그다음부터는 술술 나옴
2앞부분이 같으면 싸고 빨라진다프롬프트 캐싱이 되면 입력 단가가 크게 내려감 (Claude API는 캐시에서 읽은 입력 토큰이 기본 단가의 10%)
3대화가 길수록 느리고 비싸다컨텍스트가 길어지면 토큰 생성 속도도, 서버가 받을 수 있는 동시 요청 수도 줄어듦
\n

이 세 현상은 전부 KV Cache와 연관이 있습니다. 이 포스팅을 다 읽고 나면, 이 물음들에 답할 수 있는걸 목표로 합니다.

\n

순서는 다음과 같습니다.

\n
    \n
  1. LLM은 루프 안의 함수
    2. \n
  2. 모델의 동작구조 살펴보기
    3. \n
  3. 어텐션
    4. \n
  4. Transformer (LLM이 쓰는 디코더와 Self-Attention)
    5. \n
  5. KV Cache
    6. \n
  6. Prefill, Decode와 메모리
    7. \n
\n

1. LLM은 루프 안의 함수

\n

백엔드 관점에서 보면 LLM은 그냥 while 루프 안에서 도는 함수입니다.

\n
prompt = \"영어로 번역하시오. 나는 당근마켓의 카키에요.\"\ntokens = tokenize(prompt)\n\nwhile True:\n    nxt = model(tokens)  # 토큰 하나\n    if nxt == EOS:\n        break\n    tokens.append(nxt)\n    stream(nxt)          # 화면에 출력
\n\n

ChatGPT나 Claude에서 글자가 하나씩 나오는 게 바로 이 루프가 한 바퀴씩 도는 모습입니다. 그래서 입력은 한 칸씩 길어집니다.

\n
\n \n \n \"model\n \n
\n\n
\n

레이어 32개, 벡터 숫자 4,096개 같은 값은 Llama 3 8B를 예시로 든 겁니다. 모델마다 다르고(예: Llama 3 70B는 레이어 80개), 이 글에 나오는 숫자는 따로 언급이 없으면 전부 Llama 3 8B 기준입니다.

\n
\n

정리하면 Transformer는 어텐션과 MLP가 N번 반복되는 구조일 뿐입니다. (디테일하게 들어가면 끝도 없지만, 일단 이 정도만 이해하고 넘어가셔도 무방합니다.)

\n

그리고 중요한 사실 하나. 토큰끼리 정보를 주고받는 곳은 어텐션뿐입니다. MLP는 토큰마다 따로 계산합니다. ‘앞 문맥을 참고한다’는 일은 전부 어텐션에서 일어나고, KV Cache도 바로 이 어텐션에 붙어 있습니다.

\n

3. 어텐션

\n

Attention = 주의, 집중

\n

어텐션은 이름 그대로 어디에 ‘집중’할지 정하는 메커니즘입니다. 무언가를 하나 만들 때마다, 볼 수 있는 것 전체 중에서 지금 필요한 곳을 골라 더 많이 집중합니다.

\n

번역에서 시작된 어텐션

\n

어텐션이 처음 나온 건 2014년 번역 모델입니다. 영어 단어를 하나씩 만들어 가는데, 단어 하나를 만들 때마다 한국어 문장 전체를 다시 보고 지금 필요한 단어에 집중합니다.

\n
\n \n \n \"att\n \n
\n

위는 원조 논문의 실제 어텐션 맵(영어 → 프랑스어)입니다. 밝을수록 많이 본 곳입니다. 대부분 대각선이지만 “European Economic Area” → “zone économique européenne”에서 순서가 뒤집혀 그 부분만 대각선이 꺾여 있습니다.

\n
\n

출처: Bahdanau et al., Neural Machine Translation by Jointly Learning to Align and Translate (2014), Figure 3(a)

\n
\n

음성인식: 집중하는 곳이 곡선처럼 이어진다

\n
\n \n \n \"att\n \n
\n

음성인식에서는 어텐션이 하는 일이 더 눈에 잘 보입니다. 위는 음성 신호, 아래 표의 세로는 출력 글자, 가로는 시간입니다. “how much would a woodchuck chuck”을 받아쓰는 장면입니다.

\n\n

아무도 “이 글자는 이 구간”이라고 알려주지 않았는데도 어텐션이 학습을 통해서 자연스럽게 본인의 역할을 수행해냅니다.

\n
\n

출처: Chan et al., Listen, Attend and Spell (2015), Figure 2

\n
\n

정리: 보는 대상만 바뀌었을 뿐

\n
\n \n \n \"att\n \n
\n

보는 대상만 바뀌었을 뿐, 계산은 똑같습니다. 점수 매기기 → 비율로 바꾸기 → 섞기. 이 계산부터 먼저 보고, LLM이 쓰는 Transformer 디코더는 그다음에 보겠습니다.

\n

4. 어텐션: 연산

\n

Seq2seq에서의 Q, K, V

\n
\n \n \n \"s2s\n \n
\n

어텐션은 Seq2seq(인코더-디코더) 번역 모델에서 제안됐습니다. 인코더(e)가 원문을 단어별 상태로 만들고, 디코더(d)가 단어를 하나 만들 때마다 인코더 상태 전체에 어텐션합니다.

\n\n
\n

처음엔 그냥 디코더 상태, 인코더 상태라고 불렀고, 이걸 Query, Key, Value라는 이름으로 정리한 건 2017년 Transformer 논문입니다.

\n
\n

1단계: 내적으로 관련도 점수 매기기

\n

디코더가 “Kaki”를 만들 차례입니다. 디코더의 현재 상태(Q)와 인코더의 단어별 상태(K)를 내적해서 점수를 매깁니다. (벡터는 숫자 4개로 단순화했습니다)

\n
\n \n \n \"steps\"\n \n
\n\n

√dₖ로 나누는 건 점수가 너무 커지지 않게 하려는 겁니다.

\n

실제로는 Q, K, V에 각각 W를 곱해서 넣는다

\n

Q의 재료는 디코더 상태, K·V의 재료는 인코더 상태라고 했는데, 실제로는 이 재료를 그대로 넣지 않고 학습된 행렬 W를 한 번씩 행렬곱해서 넣습니다.

\n
\n \n \n \"self\n \n
\n\n

W는 학습으로 정해지는 숫자 표

\n

W는 그냥 숫자가 빽빽하게 적힌 표, 행렬입니다. Llama 3 8B 레이어 하나 기준으로 K를 만드는 곱셈을 보면:

\n
\n \n \n \"k\n \n
\n\n

어떤 토큰의 K만 바꾸면 비율이 바뀌고, V만 바꾸면 비율은 그대로인데 섞여 나오는 내용이 바뀝니다.

\n

그리고 이게 KV Cache로 이어집니다. 다음 토큰을 만들 때도 앞 토큰들의 K로 점수를 내고 V를 섞어야 하니까, 둘 다 저장해 둬야 합니다.

\n

5. Transformer: 최신 LLM은 디코더만 쓴다

\n

2017년 Attention Is All You Need 는 RNN 없이 어텐션만 쌓아서 번역 모델을 만들었습니다. 원래는 원문을 읽는 인코더와 글을 한 토큰씩 쓰는 디코더가 있는 구조였는데, 지금의 GPT, Llama 같은 LLM은 디코더만 쌓은 구조입니다.

\n

인코더가 없으니 어텐션이 볼 곳이 달라집니다.

\n

참고할 건 지금까지의 글뿐

\n
\n \n \n \"self\n \n
\n \n \n \"self\n \n
\n

어텐션을 하는 이유는 똑같습니다. 다음 단어(Kaki)를 만들 조건(context)을 모으려고요.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
필요한 정보그래서
Seq2seq 번역기원문 (인코더 출력)원문을 봄 → Cross-Attention
LLM (디코더만)지금까지 나온 토큰 (am 자신 포함)자기 문장을 봄 → Self-Attention
\n

두 경우 모두 “카키에요.”에서 오는 선이 제일 굵습니다. 새로운 기술이라기보다, 정보가 자기 문장 안에만 있으니 거기에 어텐션해서 “Self” 라는 이름이 붙은 겁니다.

\n

자기 문장을 보는 게 왜 필요할까? 뜻은 주변이 정한다

\n
\n \n \n \"q1\n \n
\n

am의 Q 1개로 K 7개와 점수를 매기고, 그 비율대로 V 7개를 섞습니다(am 자신도 포함).

\n
\n \n \n \"q\n \n
\n\n

캐시는 다시 읽히는 것만 저장하니까, K, V만 저장합니다. 이름이 KV Cache인 이유입니다.

\n

토큰이 K, V를 “들고 있다”는 건

\n

“카키에요.”가 레이어에 들어오면 숫자 4,096개짜리 벡터 x입니다. 여기에 W_K를 곱하면 1,024개짜리 K, W_V를 곱하면 V가 나옵니다. 들고 있는 건 이 결과 벡터입니다.

\n
\n \n \n \"per\n \n
\n

어텐션이 있는 레이어마다 캐시가 한 줄씩 있습니다. 레이어마다 들어오는 x도 다르고 W도 다르니까요. MLP는 다른 토큰을 안 보니까 캐시가 없습니다. 토큰이 하나 생기면 32줄 모두에 K, V가 한 칸씩 늘어납니다.

\n
# 실제 메모리 모양 (Llama 3 8B)\nkv_cache[레이어 32][K·V 2][헤드 8][토큰 수 →][128]
\n

토큰 수 축만 계속 늘어납니다.

\n

얼마나 줄어들까?

\n

프롬프트 1,000토큰에 답변 100토큰을 만드는 경우:

\n
\n \n \n \"savings\"\n \n
\n

단, 어텐션에서 캐시를 읽는 비용은 여전히 길이에 비례합니다. 이게 뒤에서 볼 decode 병목의 씨앗입니다.

\n

7. Prefill과 Decode

\n

추론은 두 단계로 나뉜다

\n
\n \n \n \"prefill\"\n \n
\n
    \n
  1. 프롬프트 토큰은 이미 다 주어져 있으니 전부 병렬로 처리합니다. 1,000개면 1,000개를 한 번에요.
    2. \n
  2. 토큰마다 자기 Q로 자기 앞을 보면서, 모든 토큰의 K, V를 32개 레이어에 걸쳐 계산해 캐시를 채웁니다.
    3. \n
  3. 행렬 × 행렬 계산이라 GPU 연산 장치를 꽉 채워 씁니다. 병목은 연산량입니다 (compute-bound).
    4. \n
  4. 그래서 프롬프트가 길수록 TTFT가 길어집니다.
    5. \n
\n

백엔드로 비유하면 대량 INSERT 한 방입니다. RAG로 문서를 잔뜩 붙이면 첫 응답이 느려지는 이유가 이겁니다.

\n

앞부분이 같으면 K, V를 다시 쓴다 (현상 2의 답)

\n
\n \n \n \"batching\"\n \n
\n
    \n
  1. 배칭 = 여러 요청의 Decode를 한 번에 : Decode는 가중치를 읽는 데 시간이 대부분이라, 요청을 묶으면 시간은 거의 그대로인데 처리량이 늘어납니다.
    2. \n
  2. KV Cache는 요청마다 따로 : 요청마다 문장이 달라서 캐시를 나눠 쓸 수 없습니다.
    3. \n
  3. 그래서 한계는 KV Cache : 대화가 길어질수록 캐시 읽기가 가중치 읽기보다 커지고, 한 번에 묶을 수 있는 요청 수도 GPU에 남은 캐시 자리가 정합니다.
    4. \n
\n

8. 메모리 비용

\n

KV Cache는 얼마나 클까?

\n
토큰 1개 = 2 (K, V) × 32 (레이어) × 8 (KV 헤드) × 128 × 2 (바이트) = 128KB
\n
\n \n \n \"kv\n \n
\n\n
\n

bf16 기준. DeepSeek-V3는 레이어당 압축 벡터 512 + 위치용 64 = 576개, 압축이 없다면 헤드 128개 × (K 192 + V 128)로 계산했습니다.

\n
\n

GPU 메모리가 동시 요청 수를 정한다

\n
\n \n \n \"gpu\"\n \n
\n

그래서 LLM 서빙에서 동시 처리량은 연산이 아니라 KV Cache 메모리가 정합니다. 이걸 줄이려고 나온 기법들입니다.

\n\n

예전 서빙: 최대 길이만큼 미리 잡아서 낭비

\n

예전 서빙 시스템은 요청이 들어오면 답이 얼마나 길어질지 모르니까, 최대 길이만큼 연속된 메모리를 통째로 미리 잡아 뒀습니다. 크기를 모르니 일단 최대 크기로 배열을 malloc해 두는 셈입니다.

\n
\n \n \n \"app\n \n
\n\n

MLP가 없으면 토큰들이 다 비슷해진다

\n

어텐션 출력은 V들의 가중평균이라, 섞기만 반복하면 차이가 지워집니다. 물감 일곱 개가 매번 서로를 조금씩 섞어 오면 결국 전부 같은 회갈색이 되는 것처럼요. 순수 어텐션만 쌓으면 층이 깊어질수록 이중 지수적으로 한 벡터로 무너진다는 연구도 있습니다(rank collapse).

\n
\n \n \n \"app\n \n
\n

실제 Transformer가 무너지지 않는 건 두 장치 덕분입니다.

\n\n

MLP는 덩치도 큽니다. Llama 3 8B 레이어 하나에서 어텐션이 약 42M, MLP가 약 176M으로 파라미터의 약 80% 가 MLP입니다.

\n
\n

Dong et al. 2021, Attention is not all you need: pure attention loses rank doubly exponentially with depth

\n
\n

“MLP = 지식 저장소”, 지금은 이렇게 본다

\n
\n \n \n \"app\n \n
\n

처음엔 MLP를 key-value 메모리처럼 보고 “에펠탑 → 파리” 같은 사실이 특정 레이어에 있다고 해석했지만, 고친 사실이 주변으로 퍼지지 않는다는 반론이 나왔습니다. 지금은 지식이 여러 레이어에 흩어져 겹쳐 저장된다(superposition)고 보는 게 주류입니다.

\n

비유하면 MLP는 사전, 어텐션은 사전을 찾아보는 손입니다. 다만 사전은 여러 레이어에 나뉘어 있고, 페이지가 겹쳐 인쇄돼 있습니다. 이 분야는 아직 연구가 활발해서 해석이 계속 바뀌고 있습니다.

\n

더 읽어볼 자료

\n","htmlAst":{"type":"root","children":[{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"KV Cache 이해해보기 - 어텐션부터 vLLM까지"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"최근 팀내에서 발표했던 KV Cache 내용을 글로 정리해봤습니다. AI 엔지니어 분들이 아닌, 백엔드/프론트 엔지니어 분들이나 비개발자 분들도 이해하실 수 있는걸 목표로 나름 고민해서 구성했습니다. 스킵할건 스킵하고, 전체적인 흐름이 어떻게 되는건지에 대한 감을 잡을 수 있게하는걸 목표로 했습니다. 어텐션이나 트랜스포머를 몰라도 읽을 수 있게, 필요한 만큼만 처음부터 쌓아 올려보겠습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"“나는 당근마켓의 카키에요.”"}]},{"type":"text","value":" 를 영어로 번역시키는 걸 예시로 설명을 해보겠습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"0. 이런 경험, 있지 않나요?"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"LLM API를 써봤다면 아래 현상들을 한 번쯤 겪어봤을 겁니다."}]},{"type":"text","value":"\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n"},{"type":"element","tagName":"table","properties":{},"children":[{"type":"element","tagName":"thead","properties":{},"children":[{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"th","properties":{},"children":[]},{"type":"element","tagName":"th","properties":{},"children":[{"type":"text","value":"현상"}]},{"type":"element","tagName":"th","properties":{},"children":[{"type":"text","value":"설명"}]}]}]},{"type":"element","tagName":"tbody","properties":{},"children":[{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"첫 글자만 유독 느리다"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"스트리밍 응답에서 첫 토큰까지는 한참 걸리는데, 그다음부터는 술술 나옴"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"앞부분이 같으면 싸고 빨라진다"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"프롬프트 캐싱이 되면 입력 단가가 크게 내려감 (Claude API는 캐시에서 읽은 입력 토큰이 기본 단가의 10%)"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"3"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"대화가 길수록 느리고 비싸다"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"컨텍스트가 길어지면 토큰 생성 속도도, 서버가 받을 수 있는 동시 요청 수도 줄어듦"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"이 세 현상은 전부 KV Cache와 연관이 있습니다."}]},{"type":"text","value":" 이 포스팅을 다 읽고 나면, 이 물음들에 답할 수 있는걸 목표로 합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"순서는 다음과 같습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ol","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"LLM은 루프 안의 함수"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"모델의 동작구조 살펴보기"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"어텐션"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Transformer (LLM이 쓰는 디코더와 Self-Attention)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"KV Cache"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Prefill, Decode와 메모리"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"1. LLM은 루프 안의 함수"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"백엔드 관점에서 보면 LLM은 그냥 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"while 루프 안에서 도는 함수"}]},{"type":"text","value":"입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"text","value":"prompt "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","string"]},"children":[{"type":"text","value":"\"영어로 번역하시오. 나는 당근마켓의 카키에요.\""}]},{"type":"text","value":"\ntokens "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" tokenize"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"prompt"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\n"},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"while"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","boolean"]},"children":[{"type":"text","value":"True"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n nxt "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" model"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"tokens"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# 토큰 하나"}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"if"}]},{"type":"text","value":" nxt "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"=="}]},{"type":"text","value":" EOS"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"break"}]},{"type":"text","value":"\n tokens"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"append"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"nxt"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n stream"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"nxt"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# 화면에 출력"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"입력은 지금까지의 토큰 전체"}]},{"type":"text","value":" : "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"model()"}]},{"type":"text","value":"은 매번 처음부터 지금까지의 모든 토큰을 받습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"출력은 딱 토큰 하나"}]},{"type":"text","value":" : 다음에 올 토큰 하나만 골라서 돌려줍니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"결과를 붙여서 다시 호출"}]},{"type":"text","value":" : 이 방식을 자기회귀(autoregressive) 생성이라고 부릅니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"ChatGPT나 Claude에서 글자가 하나씩 나오는 게 바로 이 루프가 한 바퀴씩 도는 모습입니다. 그래서 입력은 한 칸씩 길어집니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/d176a7fbadf29a6fcbbe723698dd17da/eaa1b/loop-visual.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 32.2%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAGCAIAAABM9SnKAAAACXBIWXMAAAsTAAALEwEAmpwYAAABK0lEQVQY0y2Nu07DMABF8/8bEwsrIwsjEkICUZBadUFtkyaBNqkdO47fz8R5oAqkO55zbqIYcVpgWJ+/c4KAFlQLWv0UFMO2uTCCRIeEsX77GLPXuCyDYTHYvyVGMm8kbRsEKt5hK7mRHMNadC0jSDKi+TUXvHPPt+HjflqW6PW/rHnn9JWuTyVBQHGiGLmcStY2BAHWNvx6jr136rxz6wfzdBMtj72NwSTOyCE4JSjrWqN4cMZ7yyjRWirFrRbOKu90b5UdxoAK/XIXvYlDiMElg0Cy2tumcKiwTWHBoa+/RngY4d6D1MBcg6OF+cSqmZ0nCWbPJ1ZN9DxKnPj21O7eZbmV5YblG5F+9NlbyFdD+amzlSi3/LgWxWZC6YzTGR1muJtxNqN0otUvAaxJ5hYw6WIAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"loop visual","title":"loop visual","src":"/static/d176a7fbadf29a6fcbbe723698dd17da/f97d7/loop-visual.png","srcSet":["/static/d176a7fbadf29a6fcbbe723698dd17da/0eb09/loop-visual.png 500w","/static/d176a7fbadf29a6fcbbe723698dd17da/1263b/loop-visual.png 1000w","/static/d176a7fbadf29a6fcbbe723698dd17da/f97d7/loop-visual.png 2000w","/static/d176a7fbadf29a6fcbbe723698dd17da/49142/loop-visual.png 3000w","/static/d176a7fbadf29a6fcbbe723698dd17da/eaa1b/loop-visual.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"실제 토큰은 단어보다 잘게 쪼개지지만 단어 단위로 표시했습니다. Kaki 뒤로 from, Karrot이 이어지지만 여기서는 Kaki까지만 따라갑니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"2. 모델의 동작구조 살펴보기"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/958cad32b8620fc8f8624a14ed756908/eaa1b/model-overview.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 37.2%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAAsTAAALEwEAmpwYAAABVElEQVQY0yWQ226cMBQA+f+v6UM/oI0iRe1DLyHJ7oawsAs22Bh8wfbx8SXbppXmdaTRVH5XelsEoyufMvoSIf/D0UY2vwxp3UqtIE4QK4hll6B4SaFEn4KrPmQpFzaxiZQc33MqKd4yrsOhO3ztX+/n4bQvIzmfnFqSXVFOWm1aioy+KhEC7DlhjNE7GwACYonAx/bn8cf3+tvl+mYXQq7deO28EtnML8+PMxnec6w+IlOIMTgPEcFCcBATWNqdHuqnu9912xxU/yzZ4CWPZrHL4HaN3pYUKmckn6kU3MjV7VqwSW3iloIYX9vmrjl9IedHRbuFUaM2tBr14t0e/stWb0PfMjoyOuhNOKP8bm4p8P5wfPh8vP9E32q1zpz0nFzXqQ+K5RgyQkJfRXBgDVgN1qC3GSEjFPSGXeZzPZ+fNtoaPpD2RU69mjrUPCGkYP/e/gPp8YS0HmtuyAAAAABJRU5ErkJggg=='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"model overview","title":"model overview","src":"/static/958cad32b8620fc8f8624a14ed756908/f97d7/model-overview.png","srcSet":["/static/958cad32b8620fc8f8624a14ed756908/0eb09/model-overview.png 500w","/static/958cad32b8620fc8f8624a14ed756908/1263b/model-overview.png 1000w","/static/958cad32b8620fc8f8624a14ed756908/f97d7/model-overview.png 2000w","/static/958cad32b8620fc8f8624a14ed756908/49142/model-overview.png 3000w","/static/958cad32b8620fc8f8624a14ed756908/eaa1b/model-overview.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"토큰 → 벡터 (임베딩)"}]},{"type":"text","value":" : 문장을 토큰으로 쪼개고, 토큰 ID로 거대한 표에서 벡터를 꺼냅니다. 계산이 아니라 조회(Table Lookup)라서 같은 토큰은 언제나 같은 벡터입니다. 실제로는 숫자 4,096개짜리 벡터입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"레이어 × N"}]},{"type":"text","value":" : 그 벡터가 레이어를 N번(Llama 3 8B는 32번) 통과합니다. 레이어 하나는 Transformer 논문 그림의 블록과 같습니다.\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"어텐션"}]},{"type":"text","value":" (= Masked Multi-Head Attention) : 토큰간에 정보를 교환합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"MLP"}]},{"type":"text","value":" (= Feed Forward) : 토큰마다 해당 정보를 더 생각(?)해봅니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"다음 토큰 확률"}]},{"type":"text","value":" : 마지막에 다음 토큰 후보마다 확률이 나오고 ("},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"Kaki 71%"}]},{"type":"text","value":", "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"a 6%"}]},{"type":"text","value":", "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"from 4%"}]},{"type":"text","value":" …), 거기서 하나를 고릅니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"레이어 32개, 벡터 숫자 4,096개 같은 값은 Llama 3 8B를 예시로 든 겁니다. 모델마다 다르고(예: Llama 3 70B는 레이어 80개), 이 글에 나오는 숫자는 따로 언급이 없으면 전부 Llama 3 8B 기준입니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"정리하면 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Transformer는 어텐션과 MLP가 N번 반복되는 구조"}]},{"type":"text","value":"일 뿐입니다. (디테일하게 들어가면 끝도 없지만, 일단 이 정도만 이해하고 넘어가셔도 무방합니다.)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"그리고 중요한 사실 하나. "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"토큰끼리 정보를 주고받는 곳은 어텐션뿐입니다."}]},{"type":"text","value":" MLP는 토큰마다 따로 계산합니다. ‘앞 문맥을 참고한다’는 일은 전부 어텐션에서 일어나고, KV Cache도 바로 이 어텐션에 붙어 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"3. 어텐션"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Attention = 주의, 집중"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"어텐션은 이름 그대로 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"어디에 ‘집중’할지 정하는 메커니즘"}]},{"type":"text","value":"입니다. 무언가를 하나 만들 때마다, 볼 수 있는 것 전체 중에서 지금 필요한 곳을 골라 더 많이 집중합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"번역에서 시작된 어텐션"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"어텐션이 처음 나온 건 2014년 번역 모델입니다. 영어 단어를 하나씩 만들어 가는데, 단어 하나를 만들 때마다 한국어 문장 전체를 다시 보고 지금 필요한 단어에 집중합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 500px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 1640px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/1fd6d77115bf92f14b6880be9b0ad02d/22c1a/att-translate-ko.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 78.6%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAQCAIAAACZeshMAAAACXBIWXMAAAsTAAALEwEAmpwYAAADJklEQVQozy2Ty24cVRCG+z0iIbHKArLMDnkBi7xIBOwjRhaEICVGQomR4AUQSmQgiiwFK0SE2DBgJ/YwvpFk7Bl3u2/T0/fbOae7T/e5FZqJS/+iNr/+qlJ9GqNVhfKalBUuGoIIynGRNQRVqGhIWaGiQgUuMlrhhqCOEsGo6KjkrWBUi3xvu//n0XDQ33q+u93/Z+vZcHfnzfHB3nb/cLi38/df+vhk/9/dw+Fg8HInDgMAUEqBFEowTQkGSoCSoKRSEgCCOKmbBualAn96djo6OhiWWQoAFcGj1/8dHww72oAUC7PkinedEDBYg8c3/Qc99OgmbNySwSnK4sjRQ0dvy1i1uC2jzDMT12A4UbS6MIPoGgHqwcdy+R37syvZ5+/DF+/yk+eiJQqHisQChbwMROoC8hXyeeoIkmuKdwCSVjhNE5gey9e/w2RLjTfl6A8Rn/Pc45nLU4eXoXz1G6z3kp+Ws5+XYb0nTzYXZiW6tkFloaIzYQ64tS/sfWEPRWLxxOGJzWOTF748XIf71/EPn5AfP4X71+WrJ5qcJ6uZa7u2qdqKkYxX+YXqgpNMVLnACe9aufkd9C75Ny4HNy5D75La/P4iGZc5LnNZF6wMOYp4Gc5txksxWGsHD9u9X8T+IzlYU09u17+u1BsrauOWnPTf7gy+5/izqUIRC435kJHBi0A+/QbuLfl3Poy+/gjuLYmjx5xiiQNBIo5C0SANBOMdLfOkrrCoS4YijhNWhqwh8tmquvtBtXqt/vYarC7Jg/X5UInFU4clliCZtrgW1cejKJwBp5JiNReav8jTO7ByJfjyavrVVbj9nho+VG2lUChRpHAsG6TRGudJVJMyT6MknMW+l0a+55ybZ6eBY4yGL2LrNHEmqWeSPE4jPwm8Mo38qdXWWKtJqY/fzFzTMXXXMhxLDzxHH59YxtjUz/TJ2LHNc+PM0CdTx/SntjEeBZ5LK8w7quEiM/Wx55jhbOpPbdfUw5nrmLrnmFPLmLm255hlnvCOtjURbA6TYO2cDsE0wVrJO9Y2vKOsrXnXdLRum4q1jejo2+YCHpALhBZa4PA/SYtQoIFH4aQAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"att translate ko","title":"att translate ko","src":"/static/1fd6d77115bf92f14b6880be9b0ad02d/22c1a/att-translate-ko.png","srcSet":["/static/1fd6d77115bf92f14b6880be9b0ad02d/0eb09/att-translate-ko.png 500w","/static/1fd6d77115bf92f14b6880be9b0ad02d/1263b/att-translate-ko.png 1000w","/static/1fd6d77115bf92f14b6880be9b0ad02d/22c1a/att-translate-ko.png 1640w"],"sizes":"(max-width: 1640px) 100vw, 1640px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"한국어와 영어는 어순이 달라서 집중하는 위치가 이리저리 튑니다. Kaki는 맨 끝 “카키에요.”를 보고, from과 Karrot은 다시 앞쪽 “당근마켓의”로 돌아갑니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"한국어 예시 값은 설명용입니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 450px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 851px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/ccf87301944c0ae01b0994633a6293ac/bd64a/att-translate-paper.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 96.6%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAATCAIAAAAf7rriAAAACXBIWXMAAC4jAAAuIwF4pT92AAACuElEQVQ4y22TzUrrQBzF/zHpNB/U4iJupJZSF1pwobahUrUWSVFcuxR04aIrceEb+AAuRNwIig/go1i8VVBa0vRLTRtqxX6QJpnLvXPv2FrPIhAmvzn/c2YCeETNZtOyLIyxbdv9ft913cHVwVcYXSiVSu12G2Os6/rz8zPZqNvt2rb98fHhuq7jOOQD+GZrWVa1Wu31ehjjWq1WLBZN02y325qmWZZVLpd1XXccp1KpuK4Lo7Z3d3cPDw+5XO7XX2Wz2Vwud39/n8/nHx8fs9lso9EoFAqO48Ag2ev1Li4udnd35+bmpqamgsFgIBAIh8OhUCgajc7OzoZCoZmZmUAgEIlEqtXqP9i27UqlUq/XE4nEycmJqqoAMDY2BiNiGAYAEEKapsFg2larpaqqIAiZTEZRFIQQz/OSJPn9fp7nOY4TRZHjOACYnJwsl8tDY/f7/dXVVQDwer0HBweRSAQhtLW1Jcsy8SQCAEEQdF0HSr6/v7+9va2vr5OB/X6/x+NJJBKnp6fhcJimIDDP88Vi8Q/88vKiaVo+n399fSWwx+MBgOXl5fPz86WlJcr8AJumWSgUnp6eTNNMJpMUvry83NzcBABZlsfHxwn2HSaTk+FXVlYoLMuyz+cDgKOjo52dHUmSKPyVmbZdr9fX1tYAgFRKc6qqenZ2Nj09DQAsy/5QmGEYtVotlUqRtgnJMAzhr6+vt7e36b48z385E/7z85NkZlmWngrLsgihiYmJ4+Pj/f19MrwoikPOzWZT0zQKDzZMdXNzs7e3xzCM1+sdytxqtQzD2NjYILeP/S+O4wRBEEVREASE0O3traIoADDUtm3bGON0Ok2dqSRJEkVRlmVyCouLiwsLC1/Xk/yihmEcHh7Oz88rihKNRmOxGHkqihKLxeLxeCqVisfj6XT66uqq0+n8BiwTKCWrb6YPAAAAAElFTkSuQmCC'); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"att translate paper","title":"att translate paper","src":"/static/ccf87301944c0ae01b0994633a6293ac/bd64a/att-translate-paper.png","srcSet":["/static/ccf87301944c0ae01b0994633a6293ac/0eb09/att-translate-paper.png 500w","/static/ccf87301944c0ae01b0994633a6293ac/bd64a/att-translate-paper.png 851w"],"sizes":"(max-width: 851px) 100vw, 851px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"위는 원조 논문의 실제 어텐션 맵(영어 → 프랑스어)입니다. 밝을수록 많이 본 곳입니다. 대부분 대각선이지만 “European Economic Area” → “zone économique européenne”에서 순서가 뒤집혀 그 부분만 대각선이 꺾여 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"출처: Bahdanau et al., "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Neural Machine Translation by Jointly Learning to Align and Translate"}]},{"type":"text","value":" (2014), Figure 3(a)"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"음성인식: 집중하는 곳이 곡선처럼 이어진다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 500px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 1384px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/6256ef6ec1d2569f9f31f9acb92c6db1/622c8/att-asr.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 87.39999999999999%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAARCAIAAABSJhvpAAAACXBIWXMAAC4jAAAuIwF4pT92AAACPklEQVQ4y4WTS28SURTHJ2miceFCNy5cuHFnjBs3LnVjYoyJxo/gV3Bt3LvWhIVJrVoxilhclFCmEKZCIBCYTgeQ8giFMDjvJ53XPWbmlhGh6j83Nyf33t+59/zPDAEACEHH7JesVmLIxyfT/JTdEZvfpMMCz/TFTEmpvusqH/fFRJ/70BSTPX3r0Pw0MCucSQCAOOGfS2/vGq9v0dn1zOMidT3O3Yvz979mbzdjlw5il59Wnp2PTS9+GRMvPWLDXntjE+vuw10lgA3h5wNyQGwoF17xzOa1Gnu10LshFs+Q9ZuJvTv1z1dedJ6co/izWzLx/phIHq8ljolN59GeGsCe61LMOFPn4vnOLpXsHZXaXYprp/u9/P5RtT0uFYft7QMuw453foSjPd5ujaqDSQBLkqgoEgD47qz4vaLpVmAELAlFawgFgWnoAWyapiRJnu8DAN1oUBQFAI7jeJ7reh6eHTeU5+Hh+b6maQEsh0IIOY4DAKlUiqbpOX8iF+eZxwihExjfHME8z6fT6VXgFFgJhW92XRfz+Xy+1WoBQMScDg+HQ0EQluBarUaS5P9hQRAsy5JlOehZYEQg27YbjQbLskELQmEsigFA10O3Z7MZrhmnx776vk+SZLfbxUkX9Ydhuq5Hz462AcC27VwuJ8uyZVnzTq3Ai4ZF25inaTqbzeLil+Dfz16qeTEoFAoMwyCE/lWzruvaXKqq4kAURVmWy+XyaDQyDCNa1zRNUcK/Cn+uqwIAVVUVRfnbgV+siJyNkBHoywAAAABJRU5ErkJggg=='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"att asr","title":"att asr","src":"/static/6256ef6ec1d2569f9f31f9acb92c6db1/622c8/att-asr.png","srcSet":["/static/6256ef6ec1d2569f9f31f9acb92c6db1/0eb09/att-asr.png 500w","/static/6256ef6ec1d2569f9f31f9acb92c6db1/1263b/att-asr.png 1000w","/static/6256ef6ec1d2569f9f31f9acb92c6db1/622c8/att-asr.png 1384w"],"sizes":"(max-width: 1384px) 100vw, 1384px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"음성인식에서는 어텐션이 하는 일이 더 눈에 잘 보입니다. 위는 음성 신호, 아래 표의 세로는 출력 글자, 가로는 시간입니다. “how much would a woodchuck chuck”을 받아쓰는 장면입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"글자를 하나 쓸 때마다 음성 전체 중 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"지금 발음되는 구간"}]},{"type":"text","value":"에 집중합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"말은 순서대로 이어지니까, 집중하는 위치도 왼쪽 위에서 오른쪽 아래로 곡선처럼 따라 내려갑니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"아무도 “이 글자는 이 구간”이라고 알려주지 않았는데도"}]},{"type":"text","value":" 어텐션이 학습을 통해서 자연스럽게 본인의 역할을 수행해냅니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"출처: Chan et al., "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Listen, Attend and Spell"}]},{"type":"text","value":" (2015), Figure 2"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"정리: 보는 대상만 바뀌었을 뿐"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/77c623bcd2bf65d5c5431e83a6681bd7/eaa1b/att-role.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 36%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAAsTAAALEwEAmpwYAAABWUlEQVQY0yXQy26bQABAUf7/W5pFd1lUTRt1UUtRGtuByDY2BgOGYd7DvF9RlavzBbdQFHiBNJk0ma1kVnLFkOMwrEShuxHYSm6k0OMxXF/ibevbV9NuXb/XfVUQ0N/qylEQFcv/i8Gp7lR9vG6iQDm6nGNOUXW79vnh+PSt+f2Qhm2eK9+9FYIszaVuzvU0DhgCb3W0ahpu5X5HMGQYRW9zCqovm83j/tf3899HWP4Rp03o94VkiCAwj7d73xIEcgpOi5UhAud7305Dl7xLMdihRLufy+5pfvtByufQvvjuX7ESIBkSGAiyaEGMZJJBydAXxfHXCL10ajyo8SiHg5lqB2o1nQuO5+56burj4eO9vdTXy+lQvTMMU3DB6eCMtzo47Z31zsYYck4pxZSTM7JYKWRo4RgicOcUcQLJMnMMJcMrwyvFgiDJiKCILpMSNHkTrIremJV+Asypgsw93XiGAAAAAElFTkSuQmCC'); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"att role","title":"att role","src":"/static/77c623bcd2bf65d5c5431e83a6681bd7/f97d7/att-role.png","srcSet":["/static/77c623bcd2bf65d5c5431e83a6681bd7/0eb09/att-role.png 500w","/static/77c623bcd2bf65d5c5431e83a6681bd7/1263b/att-role.png 1000w","/static/77c623bcd2bf65d5c5431e83a6681bd7/f97d7/att-role.png 2000w","/static/77c623bcd2bf65d5c5431e83a6681bd7/49142/att-role.png 3000w","/static/77c623bcd2bf65d5c5431e83a6681bd7/eaa1b/att-role.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"보는 대상만 바뀌었을 뿐, 계산은 똑같습니다."}]},{"type":"text","value":" 점수 매기기 → 비율로 바꾸기 → 섞기. 이 계산부터 먼저 보고, LLM이 쓰는 Transformer 디코더는 그다음에 보겠습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"4. 어텐션: 연산"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Seq2seq에서의 Q, K, V"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/bfc221af739885613a19b3c1ffed6324/404a1/s2s-qkv.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 57.8%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAAB7CAAAewgFu0HU+AAABwElEQVQoz5WSSY7UQBBFfQ1OxBW4BSdgwaaQaESjFivWSJwCgVCDGBZd3QixYoGgRtvlIZ2DncNDTruq2lIhQUghZ9iOH///yCSEwD768z9n3yAF5FvwbqhDINkDOef433g9u+DF3Xss5t9i7b0fAPtDWZYsszVCNrRti+nTmJOptcZ2He9vvvPq7SVVow8Kk+B9j4jrLE0jMFpjpMRqHQf1zE9l/40o1B3A+ogM3Zh+lOLbDl+LvbF/lWytQ0o18T/pruboj59w118xn79gr2/oLj/QvnlHaNvI4dRSIhHnovwJoHl6gXw4w5ydo549Rz8+R8/OUI+eENJ0wrJ/TACtRasjw356IvBkdni5VAWagCSw1CXO+7igqWfH6Cvd2amHeV5T7EqsceS7gqaWiEqwXm9RUiGVRAhBsavGrCmLiqoS1FlGvfpNd0t2UuQ1u6xksyioSsF2VZKnQ6NRdpQWKHJBlu5Y/MzJ05R14clfPqC5fwe9Xo6AnmRg1KAaQ5FmNLWKdZkXOBdXgvcBKRR1UcUhfY/SDrNdon7McW13ZEjw48Ysq8Uv2tYMIK6beBP/C8Od22w3KCUPPt728A/4fZ4MbkqHuAAAAABJRU5ErkJggg=='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"s2s qkv","title":"s2s qkv","src":"/static/bfc221af739885613a19b3c1ffed6324/f97d7/s2s-qkv.png","srcSet":["/static/bfc221af739885613a19b3c1ffed6324/0eb09/s2s-qkv.png 500w","/static/bfc221af739885613a19b3c1ffed6324/1263b/s2s-qkv.png 1000w","/static/bfc221af739885613a19b3c1ffed6324/f97d7/s2s-qkv.png 2000w","/static/bfc221af739885613a19b3c1ffed6324/404a1/s2s-qkv.png 2540w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"어텐션은 Seq2seq(인코더-디코더) 번역 모델에서 제안됐습니다. 인코더(e)가 원문을 단어별 상태로 만들고, 디코더(d)가 단어를 하나 만들 때마다 인코더 상태 전체에 어텐션합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Q (Query)"}]},{"type":"text","value":" : 디코더의 현재 상태. am 다음 토큰(Kaki)을 만들 차례인 디코더의 벡터(그림의 d₂)로, "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":""}]},{"type":"text","value":", I, am까지 읽은 맥락이 담겨 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"K (Key)"}]},{"type":"text","value":" : 인코더의 단어별 상태. Q와 점수를 매길 대상입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"V (Value)"}]},{"type":"text","value":" : 인코더의 단어별 상태(K와 같은 재료). 비율대로 섞어서 가져올 내용입니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"처음엔 그냥 디코더 상태, 인코더 상태라고 불렀고, 이걸 Query, Key, Value라는 이름으로 정리한 건 2017년 Transformer 논문입니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"1단계: 내적으로 관련도 점수 매기기"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"디코더가 “Kaki”를 만들 차례입니다. 디코더의 현재 상태(Q)와 인코더의 단어별 상태(K)를 내적해서 점수를 매깁니다. (벡터는 숫자 4개로 단순화했습니다)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/9d639699a9395e177fb97d1943066c91/eaa1b/op-dot.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 42.8%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAIAAAC9o5sfAAAACXBIWXMAAAsTAAALEwEAmpwYAAABhUlEQVQozyWR2WrdQBBE9f/flpdAAravLGmultn3rWeRFBRDPzRFdXGKHoxkgmxaEM2JN0JzIhlWDFvFrWJGUP0MURx7IxvEVnIv6VkgDt7IkjxEpwV1WtQccrA5uuQNXucdTZLuEF0OFqLrNYNhxYlW8nPMvn6F7ePWKB2fDv1Jx2el461QY6Oefufjo5LXJZdLzpeYT7l08tnp6xRz18fwfv3l6/eVbVREYaQxSor0qIsTjm9R4qRpEMeP2IKqlrUgW1AtmuHYEKe4leyMVJxqwbxVDRJErwWzSgSrrRbeyAqxQgKnSvIVUoU05GDvs969NIjRaa9Fie7ucNYMwZbkc7DRKoj2bvlqcII/S2o5tByGY13e84im8T2Pwer7at7Idfk+1gVv6D2Pvea7V6cl3hDZ0YYmsr+vVnrJw4amdZnIvklOnVG9QrDqp4ugB6cH5HBf3UjG8K4lU5IZySGFs8GDXZLvJT1hFXrJNQeI/qxwVng46/PYknyDeLXyX3lsvaR/IRf6y/RV8McAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"op dot","title":"op dot","src":"/static/9d639699a9395e177fb97d1943066c91/f97d7/op-dot.png","srcSet":["/static/9d639699a9395e177fb97d1943066c91/0eb09/op-dot.png 500w","/static/9d639699a9395e177fb97d1943066c91/1263b/op-dot.png 1000w","/static/9d639699a9395e177fb97d1943066c91/f97d7/op-dot.png 2000w","/static/9d639699a9395e177fb97d1943066c91/49142/op-dot.png 3000w","/static/9d639699a9395e177fb97d1943066c91/eaa1b/op-dot.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"내적은 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"“같은 방향인 정도”"}]},{"type":"text","value":" 입니다. 비슷한 성분이 많을수록 값이 커져서 관련도 점수로 씁니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"실제 벡터는 숫자 수천 개지만 계산 방식은 똑같습니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"2단계: softmax로 비율 만들기"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"각 점수에 지수함수를 씌우고 전체 합(58.3)으로 나눕니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/a3bcb5079c40dc597757bd3d4bc959ac/eaa1b/op-softmax.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 39.8%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAIAAAB2/0i6AAAACXBIWXMAAAsTAAALEwEAmpwYAAABK0lEQVQY0zWR246kMAxE+f9PXGl7pi80DSHBSXyJncCuIDPSeSjbZaksD5w3WOdtneEiBQd+3tYPpa0p10K10G6ym3TdrHRRCw0puAzeBJVzBh/cxwQ5w/J+KufTrQzrHJapKZtkdo8q+LMcgxvvNxNsyoKRMzRl5eyml3I2QRP08zv6pSorRZpuJj/9Aby7//1DMTRlzlAo1UKFEvilO0xwnd8Yw7mMwO5VC5tgFRw4g3LeTUxQMArGWkgp91v6wXjpZqIUCzgr1EeDYFqmEbw7qnb2i6PZUQvGzc3Tti6f8YkJ/u11b7ZftqY8UIbX4+szPjlHysCYGONFOqom8I/v2/j4Hp93CF4oYQJKYEJHs6EWorTxmZZ/6Z/gMydjhoBx6+UZuE+Vm8p/79rJD8gGs+IAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"op softmax","title":"op softmax","src":"/static/a3bcb5079c40dc597757bd3d4bc959ac/f97d7/op-softmax.png","srcSet":["/static/a3bcb5079c40dc597757bd3d4bc959ac/0eb09/op-softmax.png 500w","/static/a3bcb5079c40dc597757bd3d4bc959ac/1263b/op-softmax.png 1000w","/static/a3bcb5079c40dc597757bd3d4bc959ac/f97d7/op-softmax.png 2000w","/static/a3bcb5079c40dc597757bd3d4bc959ac/49142/op-softmax.png 3000w","/static/a3bcb5079c40dc597757bd3d4bc959ac/eaa1b/op-softmax.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"합이 1이 됩니다"}]},{"type":"text","value":" : 그래서 “얼마나 참고할지” 비율로 바로 쓸 수 있습니다. 음수 점수도 0과 1 사이로 들어옵니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"큰 점수가 더 크게 강조됩니다"}]},{"type":"text","value":" : 지수함수라서 4점과 1점의 차이가 94%와 5%로 벌어집니다. 중요한 토큰에 집중하게 되죠. 그래서 이름이 ‘집중’, 어텐션입니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"3단계: 비율대로 섞기 (가중합)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 600px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/6da3c076aac66455df86f3bcbb10ca95/8e011/op-sum.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 60.8%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAAB7CAAAewgFu0HU+AAABRklEQVQoz5VT246CQAzl/z/KB79CxURcjUaEceeiMNzO5lRmsuCa3W1SKDPt6emFBACGYYjqqgf6vgcGiBTXK655jrnfXMN9Ej7eSVkWUKrEbxJwEj6qqoKva3jvkaYpjDHC0loLawzatkXTNKL0dc7KGZUxPIsMaXRdF/VyuYgTL+u6hnNOwIPSp/F+EsNEEfCZ2aPvupj1fr+LI4FpvwA2jdj07ca4CGiMRlEUwoZ6Pp+x3+8jO6WUlNSGkh8POPssOcS8lDxvbBCyIMt3d2Q3P0+IzrVgyVmWyUCOx+Noa+R5jtPpJMztyGy9XuHjcMBut8NiscDtpqZrw2x80zn0JvSHCQOb4Pe9n+Fssjb/EQJo/SlstdbCnn2cMPzLHxBsMt5sNjK47TbFcrnETamfhzIfzhyUwiFxyn6csKyZc1I+S/4C6BOrwyM8RLgAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"op sum","title":"op sum","src":"/static/6da3c076aac66455df86f3bcbb10ca95/f97d7/op-sum.png","srcSet":["/static/6da3c076aac66455df86f3bcbb10ca95/0eb09/op-sum.png 500w","/static/6da3c076aac66455df86f3bcbb10ca95/1263b/op-sum.png 1000w","/static/6da3c076aac66455df86f3bcbb10ca95/f97d7/op-sum.png 2000w","/static/6da3c076aac66455df86f3bcbb10ca95/8e011/op-sum.png 2125w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"각 토큰의 V에 비율(a)을 곱해 모두 더하면 맥락 벡터(c)가 됩니다. ("},{"type":"element","tagName":"a","properties":{"href":"https://sooftware.github.io/attention/"},"children":[{"type":"text","value":"어텐션 메커니즘"}]},{"type":"text","value":" 글의 그림을 이번 예시로 다시 그렸습니다)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":" 0.94 × [1, 0, 1, 1]\n+ 0.05 × [0, 1, 1, 0]\n+ 0.02 × [0, 2, 0, 0]\n= [0.94, 0.09, 0.99, 0.94]"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"섞은 결과는 거의 “카키에요.” 벡터입니다. 디코더는 이 맥락 벡터를 보고 다음 영어 단어 “Kaki”를 고릅니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"계산을 보이려고 K와 V에 같은 숫자를 썼지만, 실제로는 곧 볼 W 때문에 서로 달라집니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"세 단계를 식 하나로"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 48%"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/b58ff909d4f066962f28d1d646e703db/04189/steps.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 19.8%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAECAYAAACOXx+WAAAACXBIWXMAAC4jAAAuIwF4pT92AAAAmUlEQVQY021Q2wrCUAw7PovIcNhszZyNY4oIfoD//19ydjZv+BBIkxDaJjmSiBREklsB8UL4wn/1aXYRVxG9iFrEKgdbOc4iTuEYwy3EaT6IOIrI/l5EJWIQcRGheBc95mwj2j3JUcnRBY0iKM8GBrk1cuxEW89l21Ju7VTguIXbZvbGolm3rP55RsHCv15g/07OW/bhVmftCWz4ICwy2c1dAAAAAElFTkSuQmCC'); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"steps","title":"steps","src":"/static/b58ff909d4f066962f28d1d646e703db/f97d7/steps.png","srcSet":["/static/b58ff909d4f066962f28d1d646e703db/0eb09/steps.png 500w","/static/b58ff909d4f066962f28d1d646e703db/1263b/steps.png 1000w","/static/b58ff909d4f066962f28d1d646e703db/f97d7/steps.png 2000w","/static/b58ff909d4f066962f28d1d646e703db/04189/steps.png 2306w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"QKᵀ"}]},{"type":"text","value":" = 1단계 점수"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"softmax"}]},{"type":"text","value":" = 2단계 비율"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"× V"}]},{"type":"text","value":" = 3단계 섞기"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"√dₖ"}]},{"type":"text","value":"로 나누는 건 점수가 너무 커지지 않게 하려는 겁니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"실제로는 Q, K, V에 각각 W를 곱해서 넣는다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Q의 재료는 디코더 상태, K·V의 재료는 인코더 상태라고 했는데, 실제로는 이 재료를 그대로 넣지 않고 학습된 행렬 W를 한 번씩 행렬곱해서 넣습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/9948eff76a5de7007dad810d801fbbb5/eaa1b/self-proj.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 33.599999999999994%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAAsTAAALEwEAmpwYAAABa0lEQVQY0zXES0/bMAAA4Pw/DtwqDtkATdqFy078ol1BA8EoCDY2aWMUwUBdW/Jw7DRxXdvxI47zXMSmgpA+fU6plqWihSDmRUYKSY2gheIF9vnsKvNGwhtpcPuXzLrF9NWkXnhOCoMUhRlJBAYKhxbd8eCWBXcMTop4jH4ewl+fk9GQjy+b6KYKr2twXT7fzu8db/KAwke+QBgGNEGRH/l/JuPfYwxBTaMC3utopU6nDZ52xHv1WBPfkYxYI1srFcNKMkSWGMfxHGqxtIrHECDgxzAEgUfSed81TWXbyvZd01bGOTnY+zI8/H4+vDg9+np+fHz6aXiyv7f/8ce3s1JxLagW1EhuJLdaWJ3lkhnFq0LXRjpvB+s7m4Ntd2PLdd+5G9vum/db7mB9bffDjs4IAkEaQ+DPVEb7rnnqu39P/Yu2zB1FE8mwZqmmiVqZK5rkHOccW8UUJzpbCoqNZLmgjc27qlipbWXkfx8CaJsUEf5nAAAAAElFTkSuQmCC'); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"self proj","title":"self proj","src":"/static/9948eff76a5de7007dad810d801fbbb5/f97d7/self-proj.png","srcSet":["/static/9948eff76a5de7007dad810d801fbbb5/0eb09/self-proj.png 500w","/static/9948eff76a5de7007dad810d801fbbb5/1263b/self-proj.png 1000w","/static/9948eff76a5de7007dad810d801fbbb5/f97d7/self-proj.png 2000w","/static/9948eff76a5de7007dad810d801fbbb5/49142/self-proj.png 3000w","/static/9948eff76a5de7007dad810d801fbbb5/eaa1b/self-proj.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"재료가 같아도 K ≠ V"}]},{"type":"text","value":" : K와 V는 같은 인코더 상태에서 나오지만, W_K와 W_V가 달라서 서로 다른 벡터가 됩니다(projection)."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"W는 학습으로 정해진 고정 행렬"}]},{"type":"text","value":" : 레이어마다 W_Q, W_K, W_V가 따로 있고, 추론 중에는 바뀌지 않습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"뒤에 볼 LLM도 재료만 바뀔 뿐 W를 곱하는 건 똑같습니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"W는 학습으로 정해지는 숫자 표"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"W는 그냥 숫자가 빽빽하게 적힌 표, 행렬입니다. Llama 3 8B 레이어 하나 기준으로 K를 만드는 곱셈을 보면:"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/36fd019018db394d59394d3c51b1b68a/eaa1b/w-learn.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 40%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAIAAAB2/0i6AAAACXBIWXMAAAsTAAALEwEAmpwYAAABr0lEQVQY0y3Q22vTUADH8fylgiDTaUWEVWPXenvyXRAfpngrFilSELa2WefKfBlMVym1XbVNmza3ktvJOTn3JBInfPk8/h5+CicojnwIgiQOGUEkiVEcIhByivyN7ZiGbRrr5dxaLWgSC5ZwehmWDCsEAddae67tb5zZxWgyHDimEXouIyj0XNNYOKZhrZcbx8IQJHFUCAtTTpVUUMEZSYqx+WQ4/zWIAaI8ywQNNrZl6AQByXDKScrpP/8nKVYyyWAUzKYTCMKVPjX1C8/zgxDmkrm2pc//BL7HKMUJwglCEGKcXJpKoQiGM8kzwTLJAs/1N7bkVHAqMITWNDLGvj6KjDFxZnm0kv5SBkahvxChqWSCYggIihEICAKoeC5ilOQZA+MT+6zj/NCss0446vP5aWp8z62fuTnIrfPc/62Mh4Pu/pd+r9vvdY8PO4Va++SwfawdHLXqvc/vv7bqR636QeP1/se97qc3WvOt1nzXbrwafNOUZuPDzp3S04eVx1W1ppafVNVHlfKz3bs1dadyv1xRy9UH92q7aml7q7S9detG0e2b169dvfLyxfO/XouWIvWcn2EAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"w learn","title":"w learn","src":"/static/36fd019018db394d59394d3c51b1b68a/f97d7/w-learn.png","srcSet":["/static/36fd019018db394d59394d3c51b1b68a/0eb09/w-learn.png 500w","/static/36fd019018db394d59394d3c51b1b68a/1263b/w-learn.png 1000w","/static/36fd019018db394d59394d3c51b1b68a/f97d7/w-learn.png 2000w","/static/36fd019018db394d59394d3c51b1b68a/49142/w-learn.png 3000w","/static/36fd019018db394d59394d3c51b1b68a/eaa1b/w-learn.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"곱한다는 건 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"x의 숫자 4,096개를 W에 적힌 비율대로 섞어서 새 숫자 1,024개를 만드는 것"}]},{"type":"text","value":"입니다. W_K 안의 숫자만 약 420만 개입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"왜 필요할까?"}]},{"type":"text","value":" W 없이 x를 그대로 쓰면 K와 V가 똑같아집니다. “무엇으로 찾을지”와 “무엇을 건넬지”를 따로 정할 수가 없습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"학습 = W의 숫자를 고치는 과정"}]},{"type":"text","value":" : 다음 토큰을 맞혀 보게 하고, 틀린 만큼 W의 숫자를 조금씩 고칩니다. Llama 3는 이걸 15조 개가 넘는 토큰에 반복했고, 그 결과가 지금의 W입니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"흔히 말하는 “80억 파라미터”의 대부분이 이런 W 행렬(W_Q · W_K · W_V · MLP …)입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"K와 V는 쓰이는 곳이 다르다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"자주 나오는 질문이 “같은 토큰에서 나오는데 K와 V가 왜 두 개나 필요하냐”입니다. 답은 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"쓰이는 곳이 다르다"}]},{"type":"text","value":"는 겁니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/a4e74dcc3c4c4269642c619a94232c41/eaa1b/k-vs-v.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 42.6%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAIAAAC9o5sfAAAACXBIWXMAAAsTAAALEwEAmpwYAAAB3klEQVQozzXHW2vTUADA8XxIH3ywzDEcexIt67RWuxe/heCLIIib7ZJe1oHggzqHg0K3mEtr2zS3k+vJSc7J/SSyqvB7+P8Z5LvQs33XQr6DoFtkCc3TqkiLlBRZXOZpmae0SIssydN4i2TJnTwhzO2tKMpL0/IM4Jq27yOi23ilo5gk641xw0u8sJjdCIoKLAcBG9pu6EHi+gQGmDGB43jI9ZFp/Q8HqyAiJF5u1BtB5EWZF6X5ar2lrHUTeND0Ai9ADHRdXdNiQipKaVnWVVVVtK6rPItlbXX9azaV+NlCWNr60tLXjqEFjh2HJgkcDJkYh2EYJDFOE0LLgtKyqsq6onmKPUO2VN4zZFsVbFVwNcEzpMBaYF+J3CWGBqMARV7PAbSd0LOQ4xMIAnsDQYo9a/pZ+tLTf042P0ah8DX9fRXPvyfzSyx+yxeXFZAYAE0DWiiPUI6jIs5qaidQiayC5nWg1Vi/Q4waG3Wk/9stShyGPTs9H7PDQX80OBtyvQHXGw/Z0aA/GbEX48E5178YcZMhy/VO2E8f/+J6J6cf3k+vr5gXh09eH3fareZx+6hz1Hz5/LDbbnVaT7vtVrfdetZ8/GobB3s7jx4+2N9t7O82DvZ2GvfvvXv75g+zgsdHjyaSlAAAAABJRU5ErkJggg=='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"k vs v","title":"k vs v","src":"/static/a4e74dcc3c4c4269642c619a94232c41/f97d7/k-vs-v.png","srcSet":["/static/a4e74dcc3c4c4269642c619a94232c41/0eb09/k-vs-v.png 500w","/static/a4e74dcc3c4c4269642c619a94232c41/1263b/k-vs-v.png 1000w","/static/a4e74dcc3c4c4269642c619a94232c41/f97d7/k-vs-v.png 2000w","/static/a4e74dcc3c4c4269642c619a94232c41/49142/k-vs-v.png 3000w","/static/a4e74dcc3c4c4269642c619a94232c41/eaa1b/k-vs-v.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"K = 점수를 잘 반영하도록"}]},{"type":"text","value":" : Q와 내적해 점수만 만듭니다. W_K는 점수를 잘 내도록 학습됩니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"V = 결과에 잘 반영되도록"}]},{"type":"text","value":" : 비율대로 섞여 결과가 됩니다. W_V는 결과를 잘 내도록 학습됩니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"어떤 토큰의 K만 바꾸면 비율이 바뀌고, V만 바꾸면 비율은 그대로인데 섞여 나오는 내용이 바뀝니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"그리고 이게 KV Cache로 이어집니다. "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"다음 토큰을 만들 때도 앞 토큰들의 K로 점수를 내고 V를 섞어야 하니까, 둘 다 저장해 둬야 합니다."}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"5. Transformer: 최신 LLM은 디코더만 쓴다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"2017년 "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Attention Is All You Need"}]},{"type":"text","value":" 는 RNN 없이 어텐션만 쌓아서 번역 모델을 만들었습니다. 원래는 원문을 읽는 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"인코더"}]},{"type":"text","value":"와 글을 한 토큰씩 쓰는 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"디코더"}]},{"type":"text","value":"가 있는 구조였는데, 지금의 GPT, Llama 같은 LLM은 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"디코더만 쌓은 구조"}]},{"type":"text","value":"입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"인코더가 없으니 어텐션이 볼 곳이 달라집니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"참고할 건 지금까지의 글뿐"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/bf78fcbe93018b78a399d8b85e8ee1bf/69e0e/self-attn.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 39.6%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAYAAAD5nd/tAAAACXBIWXMAAB7CAAAewgFu0HU+AAABmklEQVQoz02QTY7UMBCFcz3uwJ1gP2uuwIoVG2CBYDFCDLQ6nfQkceLY+evYcWL7Q0mPRpT06klV9Ur1KokRnFvRukNrjVbqYO8Dey/GwD3ikb0ZWbsap0v8Tb924stUUsobv9OWp7QlFwO1mqjVTClH2t4SvMcsgcUF1OCoWnPwvATsCqtb8UPDpkvitpCo3nAqWppu5FL15I1C6pk/hbwvDOG41oeIMwbTK0wnWaYeqxs2a3YbxG0lBk/SDo5aGdrOkpUjeTUi1EyWVjRq/s/yrtuFjugscV0IdiLOPXG1rzOJX8zxl20eCHNPmBT+1rGJE4uuiH4DN8IygOnA9mA76J+h+gXlI1x/gngiroaEqSaOFTuz8yyJ6Rf8tw+HKNqJIs9oRHGgLK6IIqc6PSKeL7RSoJtn5PePzNkPktBdsfWZTZ6JoyAuE+HTO9b3byD9yrYYstMZWQmUqCjyglo01HWLkopZN3idYx/esn1+IPFjg61O2DqFWRFvLZj+xdpIcIZVXnAyw5R/2dT1btkoMPrQHNhry8Q/WMBhpHurzb8AAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"self attn","title":"self attn","src":"/static/bf78fcbe93018b78a399d8b85e8ee1bf/f97d7/self-attn.png","srcSet":["/static/bf78fcbe93018b78a399d8b85e8ee1bf/0eb09/self-attn.png 500w","/static/bf78fcbe93018b78a399d8b85e8ee1bf/1263b/self-attn.png 1000w","/static/bf78fcbe93018b78a399d8b85e8ee1bf/f97d7/self-attn.png 2000w","/static/bf78fcbe93018b78a399d8b85e8ee1bf/69e0e/self-attn.png 2683w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"예전 인코더-디코더"}]},{"type":"text","value":" : 인코더가 원문을 따로 읽어 K, V를 만들고, 디코더(am)에서 나온 Q가 원문을 참고합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"요즘 디코더만"}]},{"type":"text","value":" : 프롬프트, "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":""}]},{"type":"text","value":", 지금까지 생성한 토큰이 전부 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"한 줄"}]},{"type":"text","value":"로 들어갑니다. 참고할 수 있는 게 이 한 줄밖에 없으니, Q, K, V가 전부 같은 줄에서 나옵니다. → "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"이게 Self-Attention입니다."}]}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"목적은 같고, 정보가 있는 곳만 다르다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-row"]},"children":[{"type":"element","tagName":"div","properties":{"className":["image-row-item"],"style":"flex: 0 1 49%"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 1528px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/46251f5bf65bcbed11783f14f7b4cc96/1debb/self-why.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 38.8%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAYAAAD5nd/tAAAACXBIWXMAAB7CAAAewgFu0HU+AAABhElEQVQoz1WQS47TQBRFjVghW4A9MGLEKtgELQYMGWaERHrSKIKk08SxO/6X/3a5qt5BtpNGPOnUq7p6n6vyuMagLUkxLGTlSHzNM6kayav/73NdXo4Lt55RO7x52PGseNhnpKqnGwxPQUk/WuK8JS265X2/izj4iqzsuGQ1ZT3y8zFnd8xpOk2YVBSVxnNOFkHVmjDtSNVAEFWkecsl64jynjDpOAUFwaUkygeirCVSE8HRJ/gTkpSG56Sh6c3q8BbWCcYK8xIzGew0YSezvMU53KSxesQaiwEk3mOj30xu7gURwZuPlxAL4v7hLNgJzHhFw9RDp9b68z3NbkP09Gutg9WhNQbblaB8pPBx+ZqlDFdUsDJr6ozUMey+Yb98wH79iNtvkOoZhhoPEcKH75SHLbQRukxQwSOmyaGNkSZCdIvVA+gWqULEDvDpDdXb13TvXsHde8T0SHXBW/7GWmTsYN4+u8hPUPhwczhrxXnRZqfUMXLYINvPyPYOOf1YhjG2/AUerF9/CfDsnQAAAABJRU5ErkJggg=='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"self why","title":"self why","src":"/static/46251f5bf65bcbed11783f14f7b4cc96/1debb/self-why.png","srcSet":["/static/46251f5bf65bcbed11783f14f7b4cc96/0eb09/self-why.png 500w","/static/46251f5bf65bcbed11783f14f7b4cc96/1263b/self-why.png 1000w","/static/46251f5bf65bcbed11783f14f7b4cc96/1debb/self-why.png 1528w"],"sizes":"(max-width: 1528px) 100vw, 1528px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"element","tagName":"div","properties":{"className":["image-row-item"],"style":"flex: 0 1 49%"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 1582px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/1eb45d8c726227d23b716df7dd8f76d9/d3b5d/self-why-2.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 40.400000000000006%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAYAAAD5nd/tAAAACXBIWXMAAB7CAAAewgFu0HU+AAABYElEQVQoz22Rz47TMBCH826ceAEeigMHTjwDEndOSCCxEtKKXXFABZZWzbZJ1olTb/41sZ35kJNst5UY6SePxjOffmNHnEJAZEnlWf8JWXpEBO8co3fIOE530QlwDrqYXmqjBzdMp8g87PqOOttSqx1DWy/Ac9g5JAC8myG2B3sEb6EzkK6Qv1dw9xV+f4bVp9NcJIv1k4J9bxEXNMy57ZHgrNgg62+IuoO2RDqDtAbim7k/AC/X889v+aTgKkTyE7JfF+/4JPSWoT/SGU3kas1RZww6wRqFPWST3GOO0ztspXHbG+z6GtfVOJPhDymUMZg93N/Cl3fI/S3j6IlMofmxykgfDHXVUB5aVG5YxwWbuEApw5+NIi9bdPHIQ15xLBXSN8jQIW9eUL2KkLcv50/pdUqt9nRFQq8TuiKly/c0KqEJ9XxPpZLpDD1NnuCKeHIohx1y/R7/8TXy/cO0/j/l02b0C4riqQAAAABJRU5ErkJggg=='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"self why 2","title":"self why 2","src":"/static/1eb45d8c726227d23b716df7dd8f76d9/d3b5d/self-why-2.png","srcSet":["/static/1eb45d8c726227d23b716df7dd8f76d9/0eb09/self-why-2.png 500w","/static/1eb45d8c726227d23b716df7dd8f76d9/1263b/self-why-2.png 1000w","/static/1eb45d8c726227d23b716df7dd8f76d9/d3b5d/self-why-2.png 1582w"],"sizes":"(max-width: 1582px) 100vw, 1582px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"어텐션을 하는 이유는 똑같습니다. 다음 단어(Kaki)를 만들 조건(context)을 모으려고요."}]},{"type":"text","value":"\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n"},{"type":"element","tagName":"table","properties":{},"children":[{"type":"element","tagName":"thead","properties":{},"children":[{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"th","properties":{},"children":[]},{"type":"element","tagName":"th","properties":{},"children":[{"type":"text","value":"필요한 정보"}]},{"type":"element","tagName":"th","properties":{},"children":[{"type":"text","value":"그래서"}]}]}]},{"type":"element","tagName":"tbody","properties":{},"children":[{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"Seq2seq 번역기"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"원문 (인코더 출력)"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"원문을 봄 → Cross-Attention"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"LLM (디코더만)"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"지금까지 나온 토큰 (am 자신 포함)"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"자기 문장을 봄 → Self-Attention"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"두 경우 모두 “카키에요.”에서 오는 선이 제일 굵습니다. 새로운 기술이라기보다, "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"정보가 자기 문장 안에만 있으니 거기에 어텐션해서 “Self”"}]},{"type":"text","value":" 라는 이름이 붙은 겁니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"자기 문장을 보는 게 왜 필요할까? 뜻은 주변이 정한다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/2a7610313f41e957a1fc548aa2e787fe/eaa1b/att-why.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 34.400000000000006%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAAsTAAALEwEAmpwYAAABRklEQVQY04WPS0vDQBSF81cFN4JbQXDrTnDlT7Ag2Aq1YoWu1FL7wEolMWlrXgg26eQ9mcx7Iql7hY/DtzmcezWYRkWeZoGPuqeFPe+0ry/OzyxjUSvOcMkJ+gMtj8MwivLZLb08NGbDk+Ojg/29wcNdXUtawX/KMAVxklT6gM9vrKXVuWq1r1qL12ktaLNMseBUcCIYkY1QySmn+BeNYYRhVhGKKOME1UrWSiglJSOCEpYFZeiWgVsBH4VetfWrrS/zQOUbWWy1NNqa+ru7tmzL8Oz1yjTslUWrEiMoOS29hTPqrYY9f9K3Hrvu+N4b99HnhDhT/m1qBBUJCPM0ymIAs0QywjAiCO7KrHDe/Ek/Np6L5Uv6MdzJCOhPcDkSG1MTDDcXMixoI5I3KEGVaN6jwIW+zoK1Ao4EtgJ2k5FTR7ZKv34AxHB8bypYyIYAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"att why","title":"att why","src":"/static/2a7610313f41e957a1fc548aa2e787fe/f97d7/att-why.png","srcSet":["/static/2a7610313f41e957a1fc548aa2e787fe/0eb09/att-why.png 500w","/static/2a7610313f41e957a1fc548aa2e787fe/1263b/att-why.png 1000w","/static/2a7610313f41e957a1fc548aa2e787fe/f97d7/att-why.png 2000w","/static/2a7610313f41e957a1fc548aa2e787fe/49142/att-why.png 3000w","/static/2a7610313f41e957a1fc548aa2e787fe/eaa1b/att-why.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"임베딩 표에서 꺼낸 “카키” 벡터는 두 문장에서 똑같습니다. 이대로는 색깔인지 사람 이름인지 모릅니다. 그래서 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"주변을 보고 벡터를 고쳐 써야 하고"}]},{"type":"text","value":", 이 일을 하는 게 어텐션입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Q는 토큰 1개, K·V는 지금까지 전부"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"헷갈리기 쉬운 부분이라 한 번 정리하고 가겠습니다. am 다음 토큰을 만드는 회차를 보면:"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/7dbde10ec97f6a725f73dab315308487/eaa1b/q1-kvall.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 37.99999999999999%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAIAAAB2/0i6AAAACXBIWXMAAAsTAAALEwEAmpwYAAABuUlEQVQY0yXKz2/SYBzH8f55Xjy5HdhBdpjBSApDMcSDf4GHXfwrPCyRQVmAKjOaeXBCNIGt9CflZ0vbp8/3+/RpaQGDS955JZ/kI2QxpjFzvdCeewEBABYQmC99Crh0AgD0CXDOo+sLPupFcfz4f0wgq9kmjkICvkeBIoRICQY+IGXkIIYEGTC8a4Hah8BLY7bhsOG44SjYCyfN0imJVBemhI9XVHfRWKPlMc0Bc42qA+qKKm60plHE+X6/3+12B7ep8Ln+6XuvI8ktSW62ZKnRvbpq1yW52ZQbDbnRkpuN7sFL6VJq1eW2dCO3v33pfO1cj/4OBPEsXyu9rIjl97VqtVQ8f1UQC2fvXperFbH2plQ7FyvFwtty8cXzk9PccT53lM8dn54cPXv65OPFB0Eb31uGOrU09WE4s81sw4nvmJpiaoqlqxNDTTjbJsxTftn9nju6tQc3VLvbL4dbVxdMfWwb2sTQLF21TS3hiGEwMTXb1GeWMTFUSvx9lgSjH8ZPaf67uxjIi76Mym22GAoYev/zMfSBeBzDJAIgHqOHidRPkyiLWTx/APMPnw53zjhbKtuVkvmTf3cWjoj7TfcTAAAAAElFTkSuQmCC'); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"q1 kvall","title":"q1 kvall","src":"/static/7dbde10ec97f6a725f73dab315308487/f97d7/q1-kvall.png","srcSet":["/static/7dbde10ec97f6a725f73dab315308487/0eb09/q1-kvall.png 500w","/static/7dbde10ec97f6a725f73dab315308487/1263b/q1-kvall.png 1000w","/static/7dbde10ec97f6a725f73dab315308487/f97d7/q1-kvall.png 2000w","/static/7dbde10ec97f6a725f73dab315308487/49142/q1-kvall.png 3000w","/static/7dbde10ec97f6a725f73dab315308487/eaa1b/q1-kvall.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"am의 Q 1개로 K 7개와 점수를 매기고, 그 비율대로 V 7개를 섞습니다(am 자신도 포함)."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/cba5a6079db92e9e48021f2b510a5bc5/a592e/dec-attn.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 52.400000000000006%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAKCAYAAAC0VX7mAAAACXBIWXMAAB7CAAAewgFu0HU+AAAB/UlEQVQoz12SS2sUQRDH9wN69iZ4UPDgTQQPkZisiSaKih/Ckx9BDZ6CEiObGKLLZk10duex8370TE8/fjKzxlULiqrurvpV/aEHANba3i/zzozRCCGo6xrZSpRStG3bu5TyT/758BN7b9/wfu8dZVEwsEvKCnoJ1LovWEQhF55DVVY0TbMEdUAp0Vrz4ukuN65f4/atm8xmDoO/t+pgxlpUt0UjkV2sKlQY9TD5G2SMWfX8Z4OVRIOCpSRr0d25awT03MVAL7urkapFWUurdT9cd/VK9UP6DdX+B4o796i3dijXHiA2txHrQ6q1DepHTxC7z2i2dqjub/ZRrD+kGT6m2dimXh+C7y8FGsugNi11GuOPj7HOHHd8TD6ZEE++Mj85Ih1/4/vRAWIyZTr6SHx6yvnogODkC+HJMWeH+4g0QzVlJ7Pb0CI1/EyLXpaTVAhrSBsIqorawEWUIoGJF5PLhqkX46YJnoBxUlNVBe2PERi1lJylJWfTc9Iox/cj4ijFnQU4zpwwSHFmLllc4HkhSZgRxSVZLiheD0leXoXFGbKukGW+As5mPkGQkyQl4aLAc6PefT9lESQEfkYc5/2AOs8okwzx6i7y+RVMeEFdlTR5vARqbZEtNNIgW0sQCc6dHD8ShEnT30lp+nelLLrKqd0pIvRoG/nPt/kFqnLsTSFSSYUAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"dec attn","title":"dec attn","src":"/static/cba5a6079db92e9e48021f2b510a5bc5/f97d7/dec-attn.png","srcSet":["/static/cba5a6079db92e9e48021f2b510a5bc5/0eb09/dec-attn.png 500w","/static/cba5a6079db92e9e48021f2b510a5bc5/1263b/dec-attn.png 1000w","/static/cba5a6079db92e9e48021f2b510a5bc5/f97d7/dec-attn.png 2000w","/static/cba5a6079db92e9e48021f2b510a5bc5/a592e/dec-attn.png 2645w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"어텐션 모듈은 그대로, 재료만 다르다"}]},{"type":"text","value":" : 번역기에선 K·V를 원문에서, LLM에선 지금까지 쓴 글에서 가져옵니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Q 안에는 앞 맥락이 섞여 있다"}]},{"type":"text","value":" : Q는 am 자리 벡터 하나지만, 앞 레이어를 거치면서 앞 토큰 정보가 이미 섞여 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Q는 매번 1개, K·V는 쌓인다"}]},{"type":"text","value":" : 새 토큰이 올 때마다 Q는 그 토큰에서 1개 나오고, K·V는 한 칸씩 늘어납니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"코드로 보면 모듈 하나, 넣는 재료만 다르다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"python"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-python"]},"children":[{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"class"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","class-name"]},"children":[{"type":"text","value":"Attention"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"nn"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"Module"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"def"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","function"]},"children":[{"type":"text","value":"__init__"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"self"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" d"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","builtin"]},"children":[{"type":"text","value":"super"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"__init__"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n self"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"w_q "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" nn"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"Linear"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"d"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" d"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" bias"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","boolean"]},"children":[{"type":"text","value":"False"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# 학습으로 정해진 가중치"}]},{"type":"text","value":"\n self"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"w_k "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" nn"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"Linear"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"d"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" d"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" bias"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","boolean"]},"children":[{"type":"text","value":"False"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# (추론 땐 고정)"}]},{"type":"text","value":"\n self"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"w_v "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" nn"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"Linear"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"d"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" d"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" bias"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","boolean"]},"children":[{"type":"text","value":"False"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\n "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"def"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","function"]},"children":[{"type":"text","value":"forward"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"self"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" q"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" k"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" v"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":":"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# 벡터 1개, (n, d)"}]},{"type":"text","value":"\n q "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" self"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"w_q"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"q"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# Q: 1개"}]},{"type":"text","value":"\n k "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" self"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"w_k"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"k"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# K: n개"}]},{"type":"text","value":"\n v "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" self"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"w_v"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"v"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# V: n개"}]},{"type":"text","value":"\n score "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" k @ q "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"/"}]},{"type":"text","value":" q"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"shape"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"-"}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"**"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"0.5"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# ① 내적 → 점수 n개"}]},{"type":"text","value":"\n weight "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" torch"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"."}]},{"type":"text","value":"softmax"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"score"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" dim"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"-"}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# ② softmax → 비율"}]},{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["token","keyword"]},"children":[{"type":"text","value":"return"}]},{"type":"text","value":" weight @ v "},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# ③ 가중합 → context vector"}]},{"type":"text","value":"\n\nattn "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" Attention"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"d"},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"4096"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\n"},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# Cross (번역기): K·V 재료 = 원문(인코더 출력)"}]},{"type":"text","value":"\ncontext "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" attn"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"dec_state"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" enc_out"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" enc_out"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]},{"type":"text","value":"\n\n"},{"type":"element","tagName":"span","properties":{"className":["token","comment"]},"children":[{"type":"text","value":"# Self (LLM): K·V 재료 = 지금까지 쓴 글 x (am 포함 7개)"}]},{"type":"text","value":"\ncontext "},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"="}]},{"type":"text","value":" attn"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"("}]},{"type":"text","value":"x"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"["}]},{"type":"element","tagName":"span","properties":{"className":["token","operator"]},"children":[{"type":"text","value":"-"}]},{"type":"element","tagName":"span","properties":{"className":["token","number"]},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":"]"}]},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" x"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":","}]},{"type":"text","value":" x"},{"type":"element","tagName":"span","properties":{"className":["token","punctuation"]},"children":[{"type":"text","value":")"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"forward 안의 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"k"}]},{"type":"text","value":", "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"v"}]},{"type":"text","value":" 두 줄을 기억해 주세요. 이게 바로 KV Cache가 저장하는 값입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"실제 모델은 이걸 레이어마다, 헤드마다 하고, Llama 3 8B는 K, V 크기가 Q보다 작은 GQA를 쓰지만 원리는 같습니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"6. KV Cache"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"캐시가 없으면: 매번 전부 다시 계산"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/4230c7e4d5aca27859ee98bf952ca868/eaa1b/no-cache.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 31.6%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAGCAIAAABM9SnKAAAACXBIWXMAAAsTAAALEwEAmpwYAAABB0lEQVQY013OS07DMBSF4SyWBbEBJsxZAGLGCKYIAW3SxEnr5mEn9rXvtWO7pgiExEM600/nL8wiyCjOSla+jIdG9vtjW/FmO3A2sg1cXcD9NTqfHKSV/q0wSn7hqv3GXd/tjqwc9s3QVRN7g4cbVz0CebuIU3B/sNUzmeXQbJvt87ivBW85Kw/1Zujqvt0NvJO8xttLfLozSFbJ6PEHr2TSSgiz1Z8J3mrU0ipBekYtSQvvKPLXJJoYfHTm93kRnDHLtDrrEDxZhyYFf87pPcfzKQSHCGpWSkqp5AhqRqNzCjmFU/DFSgDL5BDQKGeBrI6eclpzXHP0Hg2CWqZeTr2SAtRMv/AHzxdOg1Dy/AEAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"no cache","title":"no cache","src":"/static/4230c7e4d5aca27859ee98bf952ca868/f97d7/no-cache.png","srcSet":["/static/4230c7e4d5aca27859ee98bf952ca868/0eb09/no-cache.png 500w","/static/4230c7e4d5aca27859ee98bf952ca868/1263b/no-cache.png 1000w","/static/4230c7e4d5aca27859ee98bf952ca868/f97d7/no-cache.png 2000w","/static/4230c7e4d5aca27859ee98bf952ca868/49142/no-cache.png 3000w","/static/4230c7e4d5aca27859ee98bf952ca868/eaa1b/no-cache.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"회차가 늘수록 중복이 계속 늘어납니다. 게다가 이걸 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"레이어 32개 전부에서"}]},{"type":"text","value":" 반복합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"앞 토큰의 K·V는 그대로다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"그럼 다시 계산한 결과가 지난번과 달라질까요? 아닙니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/01750ca017a1de0b93e4abcc247cc883/eaa1b/kv-fixed.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 33%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAAsTAAALEwEAmpwYAAABYUlEQVQY0z2MS0/bQBhF/U/7p7qjazZdAG0BqdlASbExJDQPHD9mPPa8vrHHY8dPTCJUVIIqHV1dnStdq/Id+vlT7Z7Y06twNRu2+TbnTSGbQrYGDnSl+k9r4LA2hbQ68hh9P2IbOyWRpPGuK8fW1Fq0BnIeZwxrQTjeMOzx2Bck0IIcLloDVpUxjgOdw3My34mNa/+yryfPta61SCIP+Su0WQbrR+Qt4mBNwqcEeUCxSCINqVVmcuGjXBHl/iTBbLGY39s3+77cdSWEf9KVk64c7rkaL8vkqUq9HqJWhi33G0BWrYWkuMzoloVFJkZs/1W+t5zfXk06fI+np/Htj9Q5p3cX+Pc36l6+xO6I717pw16sLQNstlwrGVMuIedw+SWZT1xnevb1eGfYIKNBoQHQmOExwwOgVgSdCHsZ9llsNYVKElxpxlKiODIMcU7GRu/7altAbVT9kR+leeefMfAGhIxwhyXntBgAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"kv fixed","title":"kv fixed","src":"/static/01750ca017a1de0b93e4abcc247cc883/f97d7/kv-fixed.png","srcSet":["/static/01750ca017a1de0b93e4abcc247cc883/0eb09/kv-fixed.png 500w","/static/01750ca017a1de0b93e4abcc247cc883/1263b/kv-fixed.png 1000w","/static/01750ca017a1de0b93e4abcc247cc883/f97d7/kv-fixed.png 2000w","/static/01750ca017a1de0b93e4abcc247cc883/49142/kv-fixed.png 3000w","/static/01750ca017a1de0b93e4abcc247cc883/eaa1b/kv-fixed.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"앞 토큰의 K, V는 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"그 토큰과 그 앞만 보고"}]},{"type":"text","value":" 만들어졌습니다. 뒤에 새 토큰이 붙어도 다시 계산할 이유가 없습니다. 모델이 애초에 앞만 보도록 만들어져 있거든요. 그러니까 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"매 회차 새로 계산할 건 새 토큰 하나뿐"}]},{"type":"text","value":"입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"정리하면, "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"바뀌지 않는 값을 매번 다시 계산하고 있었던 겁니다."}]},{"type":"text","value":" 백엔드 개발자라면 바로 떠오르죠. 캐시하면 되잖아요? memoization이랑 똑같습니다. 그게 KV Cache입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"캐시가 있으면: 새 토큰 하나만 계산"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/fc74bacba182ceb274f5d2fcb763f903/404a1/with-cache.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 46%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAJCAYAAAAywQxIAAAACXBIWXMAAB7CAAAewgFu0HU+AAABfElEQVQoz6WRXY7TMBSFszOWwZLYAVtgBYgF8IpAvIymmpb0N43/mzRNnDj2h5wOHZ7hSkfn+so6Pve4SCnxr8gVJ0/wA0PXEHxPwX/UPAes0TS7J/q6ZBo9xTwnQkiMU1yQ+zkmwgxTiIzjzOBH+t7Tth3ONQipqc6C07FCGIcXv2DslkcK14w8bSTPpeTn6syuUuwqQ3msKQ+K9e5EeRCsygPbg+BlV3GqLdtKI3R7tzr1jxiKtgs87xWVdrwcLQdp2JwubIViX1/ZnCWVurGuJNIOrI4CpRzrco9pe1KYSENLTnQRjPG+8h2RKfP8dv6bp2mmt5qbNcssZnc3R4rzIkgW9INH1zUX47hos7BTGpd7bXBa0+RZZm25XD39t8/4j++Zfnx5CP359cL3HiMETlsaa3HKoOoaLSS6lmghMFJjlcJJRdsHuq+f8B/eMX5/FYzhzWEWzojZfwI/jEihUEojaoEQEiUVWmmubUeKaVn19Tqk+MgvC/4G0tmzzLt2j8kAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"with cache","title":"with cache","src":"/static/fc74bacba182ceb274f5d2fcb763f903/f97d7/with-cache.png","srcSet":["/static/fc74bacba182ceb274f5d2fcb763f903/0eb09/with-cache.png 500w","/static/fc74bacba182ceb274f5d2fcb763f903/1263b/with-cache.png 1000w","/static/fc74bacba182ceb274f5d2fcb763f903/f97d7/with-cache.png 2000w","/static/fc74bacba182ceb274f5d2fcb763f903/404a1/with-cache.png 2540w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ol","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"새 토큰 “am” 하나만 Q, K, V를 계산합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"K, V는 캐시 끝에 append합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Q로 캐시 전체를 조회해서 어텐션을 계산합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"레이어 32개 모두 같은 방식입니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"리스트에 append하는 것과 거의 같은 구조입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"그럼 Q는 왜 저장 안 해요?"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"꼭 나오는 질문입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/35673a1512bf025e673a1af6866972e1/eaa1b/q-only.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 28.4%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAGCAIAAABM9SnKAAAACXBIWXMAAAsTAAALEwEAmpwYAAABNUlEQVQY0yXNOU/DMABA4fx5JBYmNmYkipDKzLVBWzXpQaFtrjqHjyS149ixm7gXoFLpG972LI6C4vFS9B+874k7sxuxlhTXjKiSCEpwCmAUpMDPYIST1abmRkujhdnIpi4thdy0c5H0uivvK5xPW0lrRs5URStWSM5YQWh+0tSiUVJLriRvtbCqPElQlhS0CYdt0BsMPrqd261imueC5QkIotBDaUxgXBBoGnXcm8O2Pe6M0dIS6zTwgxC4PJyGK5eAOVw4vEAk9muWsywtICgzqAVtZHlo1dnRaKP46RwBECehoBlmZfV+x/v3tjO8ub5SYEQ/38joBTnP2fgV2U/Yef6Bk9+T8R7NLF0SgjEkMcswSAGLl1ESVRRVBVRwWfl26dvMG1ahU/73Fs93ZLEjC5N5f9jMPvg6w3/zAAAAAElFTkSuQmCC'); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"q only","title":"q only","src":"/static/35673a1512bf025e673a1af6866972e1/f97d7/q-only.png","srcSet":["/static/35673a1512bf025e673a1af6866972e1/0eb09/q-only.png 500w","/static/35673a1512bf025e673a1af6866972e1/1263b/q-only.png 1000w","/static/35673a1512bf025e673a1af6866972e1/f97d7/q-only.png 2000w","/static/35673a1512bf025e673a1af6866972e1/49142/q-only.png 3000w","/static/35673a1512bf025e673a1af6866972e1/eaa1b/q-only.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Q는 자기 자리 계산에 한 번 쓰고 끝납니다."}]},{"type":"text","value":" 입력 토큰도, I도 자기 차례에 Q를 한 번 썼고, 그 뒤로는 다시 쓸 일이 없습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"다음 회차에 필요한 건 새 토큰의 Q 하나"}]},{"type":"text","value":"라서 매번 새로 만들면 됩니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"K, V는 매 회차 앞 토큰 전부가 다시 필요합니다."}]}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"캐시는 다시 읽히는 것만 저장하니까, K, V만 저장합니다. 이름이 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"KV Cache"}]},{"type":"text","value":"인 이유입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"토큰이 K, V를 “들고 있다”는 건"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"“카키에요.”가 레이어에 들어오면 숫자 4,096개짜리 벡터 x입니다. 여기에 W_K를 곱하면 1,024개짜리 K, W_V를 곱하면 V가 나옵니다. 들고 있는 건 이 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"결과 벡터"}]},{"type":"text","value":"입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/65d786bffcf706caae996b7499ab4d6b/eaa1b/kv-what.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 37.6%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAIAAAB2/0i6AAAACXBIWXMAAAsTAAALEwEAmpwYAAABcElEQVQY00XP246bMBAGYN7/gapW7V1vUrWb0N0NSQjhZHCwwQdsxjbYYSuiSit9Gv2/RhppIsmoAzWDCg4eM4T/jHcmzHb17pnBOwiLDX4O3vnn1juIeN/ZSbGBsYEZsJMGC+Bg9HbSYhi6xiixnXPGMKyai+oqB2oxajE6EqThjCcFZK3LGpvjhYoFV5nmtMdVeozLa4LyC+tQczud9rv4188OFVYxq3hktQCtjuWQtqLoxuwuiZhQfhG05bStsvMd3WhbyR7jKsvO78U1acur5sRNMuIEc9kfcHrsi2SokqHCioqeWFDTyLVgzujt7dkIRglGA8GgpTN6sVPkBDEjzdsUkbzj6M5rIe8fdgwgHyBWs80A3E88gFitXI0MIPzEPYxRuftS//7W/Ple7r62Lz8eKPb1wdeHuXxZ6r3fbPnR/P1oXz818UrSSN72Mo/H4imPAb1N1SvUb2N+gPod6q2Oebzg00rTT+QcWP0PTDa4JzTMQWEAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"kv what","title":"kv what","src":"/static/65d786bffcf706caae996b7499ab4d6b/f97d7/kv-what.png","srcSet":["/static/65d786bffcf706caae996b7499ab4d6b/0eb09/kv-what.png 500w","/static/65d786bffcf706caae996b7499ab4d6b/1263b/kv-what.png 1000w","/static/65d786bffcf706caae996b7499ab4d6b/f97d7/kv-what.png 2000w","/static/65d786bffcf706caae996b7499ab4d6b/49142/kv-what.png 3000w","/static/65d786bffcf706caae996b7499ab4d6b/eaa1b/kv-what.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Llama 3 8B 기준입니다. K, V가 1,024개인 건 KV 헤드 8개 × 128이라서입니다. (헤드: 어텐션을 여러 묶음으로 나눠 동시에 하는 단위)"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"캐시는 레이어마다 따로 쌓인다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/3643c52b92501cfda8b0b348107a2b38/eaf97/per-layer.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 57.00000000000001%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAYAAAB/Ca1DAAAACXBIWXMAAB7CAAAewgFu0HU+AAAB1ElEQVQoz3WTi26jMBBF+f//24Zu0iQlT14OBvy2OSucpupKu1c6EgwedGcuFMuysLLKWptZ9aq9ZGxknB3DaHmMBiEN3UNhffw+v7YUP5vUPKPU/H3vrEUrlQ8bYxgnxawM2ji0sTgfSXF9YfrqWCj8PKDmiVF0dLdLZhQt3jmknOjFgLGeSWnk8GBqzuhJ5lqICe9DJsTnpIUfeyYp6e4P9u83jtsbzVVgtePcaN7OHW9VS3l5UO4rNh8V5fbA5tTx67NhexUca4GQFm0jBW7OVmNcCCERQiTG5wgxLXksHxLrqrya8GrGT8OztvL1fHWbHaaxzc0pBVLypOhZUuCfiuaJm/ifCj/UTHJANDWnjwPn/RFR12ilc6r9qBErs0P0/ZO2QUyGfjQMs0Uqm7+CdY9F0hKjFKKVnA5Npq8HgvPcOs3vS8/uItjdJLvjJfOxr9hdBduL4HgfOLUy79C6RJHUM+WhbbhXn9xPVb521mBdRFv/xKecrp4ntBRoG3LduIBdUw6vHWqJmkb6RnLY1hx3Nd19wBnPuVa8VQ3lqftOuSzfed8eKM89m6rNTo/1g354pfzjj/DeZv7S8gUQ1nT7K2Eecs3aiNI+73rWPjv8AxIMT/pacPDFAAAAAElFTkSuQmCC'); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"per layer","title":"per layer","src":"/static/3643c52b92501cfda8b0b348107a2b38/f97d7/per-layer.png","srcSet":["/static/3643c52b92501cfda8b0b348107a2b38/0eb09/per-layer.png 500w","/static/3643c52b92501cfda8b0b348107a2b38/1263b/per-layer.png 1000w","/static/3643c52b92501cfda8b0b348107a2b38/f97d7/per-layer.png 2000w","/static/3643c52b92501cfda8b0b348107a2b38/eaf97/per-layer.png 2592w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"어텐션이 있는 레이어마다 캐시가 한 줄씩 있습니다. 레이어마다 들어오는 x도 다르고 W도 다르니까요. MLP는 다른 토큰을 안 보니까 캐시가 없습니다. 토큰이 하나 생기면 32줄 모두에 K, V가 한 칸씩 늘어납니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"# 실제 메모리 모양 (Llama 3 8B)\nkv_cache[레이어 32][K·V 2][헤드 8][토큰 수 →][128]"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"토큰 수 축만 계속 늘어납니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"얼마나 줄어들까?"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"프롬프트 1,000토큰에 답변 100토큰을 만드는 경우:"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/96fd3e8469fb2007307d7d56b12c0a7e/eaa1b/savings.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 33.800000000000004%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAAsTAAALEwEAmpwYAAABRElEQVQY03WQW0sCURSF/f/vvRU9RBAUJYqCEoQkkpqZeCHM26jjlHPm3Pa5zjlajFZvwXraa38P38qFy3m0DsLlPFzON6tFMBsH0/E6mAFNvFXOCGfkf8lN3garxSRczoa9zvuoz/AWGBacWS29s86ov1dvlbdq58wu1T9wtZRv1O677Ua1lK+W8kZLMXmm/YeochYWTwxa732aKqiWi+XCXeH2+uL89Obq8sjnwmCK4wjHHwooS7ZaApl0o3YlHtTR6EkkkWAkVTDsvbx2mr1Oq1mvtRqPqc50chzHqYaDnnRWeiO1EkKAlKCUkJwAQUaCkUzQRHN6wIRVkMFacglUMMxpIjgVnAiOFRAFVAM1x5YTI/mXt/tM2Hind85kMKMJw4igLfrccIIYQTSJGY4hu8cEbSmOGUaS05032Vq/cVZ9A+rZgIX9TESxAAAAAElFTkSuQmCC'); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"savings","title":"savings","src":"/static/96fd3e8469fb2007307d7d56b12c0a7e/f97d7/savings.png","srcSet":["/static/96fd3e8469fb2007307d7d56b12c0a7e/0eb09/savings.png 500w","/static/96fd3e8469fb2007307d7d56b12c0a7e/1263b/savings.png 1000w","/static/96fd3e8469fb2007307d7d56b12c0a7e/f97d7/savings.png 2000w","/static/96fd3e8469fb2007307d7d56b12c0a7e/49142/savings.png 3000w","/static/96fd3e8469fb2007307d7d56b12c0a7e/eaa1b/savings.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"단, 어텐션에서 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"캐시를 읽는 비용은 여전히 길이에 비례"}]},{"type":"text","value":"합니다. 이게 뒤에서 볼 decode 병목의 씨앗입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"7. Prefill과 Decode"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"추론은 두 단계로 나뉜다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/37ab0ad8ae48b22b0d69d341c0feaf23/eaa1b/two-phases.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 34.400000000000006%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAAsTAAALEwEAmpwYAAABc0lEQVQY02XNzWoTURgG4LkAd63SK+nSFrGFIlkEegHuBYlTmyZxEYqg0E1t1fYS3OtK6A24NykMSZNOkkkmmfMzc/7P+WZGJ3RXePg27/vyeQ5NcjqHNUcqQOcpoZgwQhjBjy5hGGeCcdDCG71+Qt5uFf5TeLdZ+pt5Y6P88Kxx/HO/OTrs9Oqt21qzf+D36u1+vdWvNfuv3vf2Gn9//I7KUnmD1va4uzPuPo8/7sqzF+zzbvHl5emnX2/Oh61vwfHXwL8IGudB5/qufTU8+T48uhz4l4ObP3GZS4+iGC8jtJhJRrXiSmRGcRRHjCQZWckUiRRxikALpwWYitMPPIsmhZVFDmB17nThDFhVOFOAsVotEqmkLnNbpVY7I8EosCq3qhpPh0F4N0iWMcVJRnFGUDQJo0moJNOSj0f30/B+Fo55SilaLaIpWjclz8BIj6UkJYlVHKwCI52RVgurBVhlFVeM8hSzFFeRFm5dgLX/n/8BZjVjaINuFOsAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"two phases","title":"two phases","src":"/static/37ab0ad8ae48b22b0d69d341c0feaf23/f97d7/two-phases.png","srcSet":["/static/37ab0ad8ae48b22b0d69d341c0feaf23/0eb09/two-phases.png 500w","/static/37ab0ad8ae48b22b0d69d341c0feaf23/1263b/two-phases.png 1000w","/static/37ab0ad8ae48b22b0d69d341c0feaf23/f97d7/two-phases.png 2000w","/static/37ab0ad8ae48b22b0d69d341c0feaf23/49142/two-phases.png 3000w","/static/37ab0ad8ae48b22b0d69d341c0feaf23/eaa1b/two-phases.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Prefill"}]},{"type":"text","value":" : 프롬프트 전체를 한 번에 처리해서 KV Cache를 통째로 채우고 첫 토큰을 만듭니다. 여기까지 걸린 시간이 TTFT(Time To First Token)입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Decode"}]},{"type":"text","value":" : 토큰을 하나씩 만들면서 KV Cache에 한 칸씩 추가합니다. 앞에서 본 루프가 이 부분이고, 토큰 하나에 걸리는 시간이 TPOT입니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"서빙 대시보드에서 이 두 지표를 따로 보는 이유가 여기 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Prefill: 프롬프트를 통째로 한 번에 (현상 1의 답)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/bc7653728a7227c35b2654182baad848/eaa1b/prefill.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 30.599999999999998%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAGCAIAAABM9SnKAAAACXBIWXMAAAsTAAALEwEAmpwYAAABQUlEQVQY00WOOUvEQACF87/sbcTSysp/YGdnZWOj4NGIhbUgoojgwrK9unF12cvNsZvMTOZKYjJJJtdkJWshfHy85vGeVklRZfG/pch+GPUAxxAsTGs+JcjlBFHkUg9gsESOHXJSZKKUiVbr9+ruQHXP1MOhejpSi/eAEXM2DhimyAVLi6xrFLlsbQwdhiFcWj5FWt05Uccb6mZPnW6qi+1m9OL7zJh8+QSFnLQwj2PIMWRe+wXDpU9RyPB6uXe5Ot9a3e6vrnaa69163E3S1LENBByKEfHgH1kqZJYkIkpEJFORZ0lVSK3ypvHHM+132OtjOuqpEEruhu7Um38Sc8jsEbPHxBzm3CkDUAawiXETeSpCtWBaVZZ5WWVSZnlelKWqigRMIqMv7EFs6sli0AZLD77fQqMfmXpDZi14ogLnF95MPb3nmGEUAAAAAElFTkSuQmCC'); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"prefill","title":"prefill","src":"/static/bc7653728a7227c35b2654182baad848/f97d7/prefill.png","srcSet":["/static/bc7653728a7227c35b2654182baad848/0eb09/prefill.png 500w","/static/bc7653728a7227c35b2654182baad848/1263b/prefill.png 1000w","/static/bc7653728a7227c35b2654182baad848/f97d7/prefill.png 2000w","/static/bc7653728a7227c35b2654182baad848/49142/prefill.png 3000w","/static/bc7653728a7227c35b2654182baad848/eaa1b/prefill.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ol","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"프롬프트 토큰은 이미 다 주어져 있으니 전부 병렬로 처리합니다. 1,000개면 1,000개를 한 번에요."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"토큰마다 자기 Q로 자기 앞을 보면서, 모든 토큰의 K, V를 32개 레이어에 걸쳐 계산해 캐시를 채웁니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"행렬 × 행렬 계산이라 GPU 연산 장치를 꽉 채워 씁니다. "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"병목은 연산량"}]},{"type":"text","value":"입니다 (compute-bound)."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"그래서 프롬프트가 길수록 TTFT가 길어집니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"백엔드로 비유하면 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"대량 INSERT 한 방"}]},{"type":"text","value":"입니다. RAG로 문서를 잔뜩 붙이면 첫 응답이 느려지는 이유가 이겁니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"앞부분이 같으면 K, V를 다시 쓴다 (현상 2의 답)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/bfb92bc50d7caa97aaa3cfdd101e0265/eaa1b/prefix-cache.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 30.8%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAGCAIAAABM9SnKAAAACXBIWXMAAAsTAAALEwEAmpwYAAAA+0lEQVQY032M206EMABE+f8/UhL3URPQCOx6YwXZlkKhpVxaWnoBw/q6MTmZnIeZ8YauYR37zPskpaAae4oxAnV5YQQ7LY0S/+C1NUSoeYzaQ1C9ZwzBn/PXWwULJUajhJb82ptvITwjueAcog5jQgnt2oo2qCdYDN3IyMRaIycrp5vpiZGBIgN5CvKUD9Qu8u9YK+GWecwimb+s6GTLxJaJK4+7wMTC2OJ0HyNQ1CWo4KXFiGA0T4MS08xHpyX9eCqCO/T6AEK/CO5B6F9CHzz7OD6Y8uQ5LTdnNqc3Z1arV7tsTu/izGrUDI78O7Aw0SAyV3aBsYWRa86/ft5M9/8oWxoAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"prefix cache","title":"prefix cache","src":"/static/bfb92bc50d7caa97aaa3cfdd101e0265/f97d7/prefix-cache.png","srcSet":["/static/bfb92bc50d7caa97aaa3cfdd101e0265/0eb09/prefix-cache.png 500w","/static/bfb92bc50d7caa97aaa3cfdd101e0265/1263b/prefix-cache.png 1000w","/static/bfb92bc50d7caa97aaa3cfdd101e0265/f97d7/prefix-cache.png 2000w","/static/bfb92bc50d7caa97aaa3cfdd101e0265/49142/prefix-cache.png 3000w","/static/bfb92bc50d7caa97aaa3cfdd101e0265/eaa1b/prefix-cache.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"왜 꼭 앞부분이어야 할까?"}]},{"type":"text","value":" 토큰의 K, V는 그 토큰과 그 앞만 보고 만들어집니다. 앞이 같으면 K, V도 같습니다. 반대로 앞이 한 글자라도 다르면 그 뒤는 전부 달라집니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"서버는 자주 쓰는 앞부분의 K, V를 요청이 끝난 뒤에도 한동안 보관합니다(Claude API는 기본 5분). 공통 앞부분의 prefill을 건너뛰니까 첫 토큰이 빨라지고, Claude API에서는 캐시에서 읽은 입력 토큰이 기본 단가의 10%입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"실제 Claude API는 캐시할 지점을 "},{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"cache_control"}]},{"type":"text","value":"로 지정하고, 일정 길이(모델에 따라 1,024토큰 등) 이상이어야 캐시됩니다. 캐시에 처음 쓸 때는 기본 단가의 1.25배입니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Decode: 한 번에 한 토큰씩"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"토큰 1개를 만들려고 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"모델 가중치 전체(16GB)"}]},{"type":"text","value":" 와 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"이 요청의 KV Cache 전체"}]},{"type":"text","value":"를 매번 전부 읽습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"행렬 × 벡터라 연산은 금방 끝나지만, 매 스텝 메모리에서 가중치와 캐시를 다 읽어와야 합니다. "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"병목은 메모리 대역폭"}]},{"type":"text","value":"입니다 (memory-bound)."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/c08ae73fde339bc95790561712e51a71/eaa1b/decode.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 37.4%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAAsTAAALEwEAmpwYAAABdklEQVQY0yXKzUrcUBjG8VySV+FOdOFgGRCloCJaEcVCL6BQwbvwAkRGVy220EURdHRwPjoznUSjIcnkJDmZnJyv93yUjA+/xbP4O8q9pW+DFCVJ+KYl+/3z+9H+zpfjg8+He7sf1zebq5vNxvqH1bnGRnOtsbJ09u1rjmJGCsf8Oa/6v6J4Gry4RuDWzWhx63L50/Xy/tXidmuhcXFy+mPcaz893NUe7zsPd+5o0Ou0Q//ZAQUAUgmqJLfA0px1Xdb3WM+l3X+kPSR+xKzVxmpbz1irrQajBAjqEJxMhoPx317/6XE87DOCLZRGFFYzaxXDweuo8zzsut02n6VGUKOkUdJq0MAdWuLA9wLfiwIfxWESh68vnjcZo2msAcosyUK/FrglCkmOqrIgRc7ITEnugKgEI5wSDdwaZTXUDBgNBjirygJnOEN5hgSnktN5XEpeKckcgtMineI0zpMoTyKM4mx+inT6TgmqJdWCKl7VBH0HvPoPxolqO807fgAAAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"decode","title":"decode","src":"/static/c08ae73fde339bc95790561712e51a71/f97d7/decode.png","srcSet":["/static/c08ae73fde339bc95790561712e51a71/0eb09/decode.png 500w","/static/c08ae73fde339bc95790561712e51a71/1263b/decode.png 1000w","/static/c08ae73fde339bc95790561712e51a71/f97d7/decode.png 2000w","/static/c08ae73fde339bc95790561712e51a71/49142/decode.png 3000w","/static/c08ae73fde339bc95790561712e51a71/eaa1b/decode.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"백엔드로 비유하면 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"매번 테이블 풀스캔을 하는 단건 쿼리를 루프로 돌리는 것"}]},{"type":"text","value":"과 비슷합니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"배칭과 KV Cache는 함께 쓴다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"“KV Cache를 쓰면 배칭은 안 쓰냐”는 질문도 자주 나오는데, 둘은 같이 씁니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/901e642a7a0a50c8d47c35ab1af76089/eaa1b/batching.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 36.199999999999996%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAAsTAAALEwEAmpwYAAABtklEQVQY0wGrAVT+ALm1smxnZHp1cnx3c3x3c396dn56dnh0cHdyb3Vxb356ePXy7+rn5OLg3d/c2uHe29zZ1unm4/r39Pr49ACloZ4+OTVQS0dMR0VbV1VnY2BnYV5OSUVPSUNTS0JjWlL08e3v6+jp5ODn49/u6+ft6ujt6ujy8O37+fcA+vf19fLx9vTy/fjx+PPq+vfx8/Hx+fbz6enxqLbZtMDh9/X28fDx6+vv6urv7/D1/Pr5//z5/vz5/Pn2AP789f//9//+9pyw4WeG0r7M6///8///8cPQ7CdZzjRi0ePm8e7t7uPk69/h6OHj6fLx8fXz8vXz8vr49gDY3e6suNnQ1+xXftgSSsyBnOB8l9plhdSVrOQyYNAzYtHj5O377uHu49rx5dv06d7+8+r/9Ov+8un98+wAqLrmXoHXlKrfkKfdXoDSpLbgeJXaZYfXorPfbYvWcZLe7OXl7NG83MSz4Mi25My58djE8dfE9dvH/OXUAP789//+9Pz27vr17/747/r17v758P/47/n07vz37///+Pz07fbj1vXj1vbk1vXj1fXj1fPi1Pjm1/zt4hZqSkt/nGMdAAAAAElFTkSuQmCC'); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"batching","title":"batching","src":"/static/901e642a7a0a50c8d47c35ab1af76089/f97d7/batching.png","srcSet":["/static/901e642a7a0a50c8d47c35ab1af76089/0eb09/batching.png 500w","/static/901e642a7a0a50c8d47c35ab1af76089/1263b/batching.png 1000w","/static/901e642a7a0a50c8d47c35ab1af76089/f97d7/batching.png 2000w","/static/901e642a7a0a50c8d47c35ab1af76089/49142/batching.png 3000w","/static/901e642a7a0a50c8d47c35ab1af76089/eaa1b/batching.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ol","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"배칭 = 여러 요청의 Decode를 한 번에"}]},{"type":"text","value":" : Decode는 가중치를 읽는 데 시간이 대부분이라, 요청을 묶으면 시간은 거의 그대로인데 처리량이 늘어납니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"KV Cache는 요청마다 따로"}]},{"type":"text","value":" : 요청마다 문장이 달라서 캐시를 나눠 쓸 수 없습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"그래서 한계는 KV Cache"}]},{"type":"text","value":" : 대화가 길어질수록 캐시 읽기가 가중치 읽기보다 커지고, 한 번에 묶을 수 있는 요청 수도 GPU에 남은 캐시 자리가 정합니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"8. 메모리 비용"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"KV Cache는 얼마나 클까?"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["gatsby-highlight"],"dataLanguage":"text"},"children":[{"type":"element","tagName":"pre","properties":{"className":["language-text"]},"children":[{"type":"element","tagName":"code","properties":{"className":["language-text"]},"children":[{"type":"text","value":"토큰 1개 = 2 (K, V) × 32 (레이어) × 8 (KV 헤드) × 128 × 2 (바이트) = 128KB"}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/da1a1ec30a96e4d5b1f4c5971829d74a/404a1/kv-howmuch.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 41.2%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAYAAAD5nd/tAAAACXBIWXMAAB7CAAAewgFu0HU+AAABrElEQVQoz52SzWsTQRjGg4gIKqWHotCCReq3VSyFFqogCp5E8BLoQc+C4EUE/zaJxkbbgtaPJLU1Yr+EdDfZbZPdzXZndmb2J5kkogcvPvAODzPw4314Jsc/lGUZ/6NcmircRgNjDFJKXNclDEP7KFJDK0zZ3HHY3HasT1WGMZrat3WWlxYpvipQelPk62oVpRQ5pTO+Oz4ZZrBbf2DbERRWAt6WQ0pfQgof2uz6GpUm3JiZZmToGKMjw5wcPs7khQl8z+sB1+oeQh4gEkmzpYgTDVlqgcVPIYvViHeViNcfA+q+QiYdbs3NcmbsFDNTVzk3Psb1y+fxfb8HXO8CRUwnTqhsHOC3JZkW7DiChc8hy6sRS9XI+roniaM2t2/Ocfb0KPfu3uHSxDhTkxfZ6wGh5u7/jtw9Tb+PrV3By/dtSpWYhXJsfb0fefraFU4cOWTjDh09bOFes9krpbb1E6WVheg4wGhtfWM/pbwhqKw1KXfnR8JeoNFK8uL5Mx7O58k/uM+j+TxPnzwmCAJyUgoajkMqpYXELR/Thw/KUZ0WKm7/dTdQt9k/v80vGewrDDraj04AAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"kv howmuch","title":"kv howmuch","src":"/static/da1a1ec30a96e4d5b1f4c5971829d74a/f97d7/kv-howmuch.png","srcSet":["/static/da1a1ec30a96e4d5b1f4c5971829d74a/0eb09/kv-howmuch.png 500w","/static/da1a1ec30a96e4d5b1f4c5971829d74a/1263b/kv-howmuch.png 1000w","/static/da1a1ec30a96e4d5b1f4c5971829d74a/f97d7/kv-howmuch.png 2000w","/static/da1a1ec30a96e4d5b1f4c5971829d74a/404a1/kv-howmuch.png 2540w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"별것 아닌 것 같지만 쌓이면 다릅니다. 8K 토큰 요청 하나면 1GB, 128K 토큰짜리 긴 요청이면 16GB입니다. "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Llama 3 8B 모델 가중치 전체가 fp16으로 16GB"}]},{"type":"text","value":"니까, 긴 요청 하나의 캐시가 모델 하나만 한 겁니다. 그리고 동시에 요청이 여러 개 들어오면 요청마다 따로 생깁니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"모델이 크면 캐시도 클까? 구조가 정한다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/0c038f0d8e498a910859a27daa0e97ed/eaa1b/kv-models.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 32%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAGCAIAAABM9SnKAAAACXBIWXMAAAsTAAALEwEAmpwYAAABL0lEQVQY0xXG206DMBgAYN7/cYzGCy+8UGN0yzRz043TCi2U0hMtbX8K2zQm38WXaMFkTyO4GdwS3eQtxchbJbqGEtRUBbjhHP0yOYrROAjNO8Upp0RQkjirGSXBmXOEGNwyBaMEOGsHyVnLWjxIfo5hBid66owajTJaOKO0YIlsT9n2rfrZdOU+8DoIzE7fI6s0ydv8Cx0+VVOAaoIk6PChmkLgTJKc12lXZUmQ7VAfdH0Edvo17XVooEcXTRaJAytttZvbwy8vrrywaDvT478uncjetXkyGoXKHFenBldlnnJGL8s8gQfw1yUO2Sp/vsXrh+PTTfl6n73cdZ+PEW2W6uPapwk4IxlVvLNaTmEEbyc/Tt6CszM4j/fq+K6ztUpXMv0P1FvAX5Hszrz4A+zGR0JD+D5DAAAAAElFTkSuQmCC'); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"kv models","title":"kv models","src":"/static/0c038f0d8e498a910859a27daa0e97ed/f97d7/kv-models.png","srcSet":["/static/0c038f0d8e498a910859a27daa0e97ed/0eb09/kv-models.png 500w","/static/0c038f0d8e498a910859a27daa0e97ed/1263b/kv-models.png 1000w","/static/0c038f0d8e498a910859a27daa0e97ed/f97d7/kv-models.png 2000w","/static/0c038f0d8e498a910859a27daa0e97ed/49142/kv-models.png 3000w","/static/0c038f0d8e498a910859a27daa0e97ed/eaa1b/kv-models.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"파라미터 수가 아니라 구조가 정합니다."}]},{"type":"text","value":" 토큰당 캐시 = 레이어 수 × 레이어마다 저장하는 K·V 크기입니다. 70B는 레이어가 80개라 8B의 2.5배입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"DeepSeek는 K·V를 압축해서 저장합니다 (MLA)."}]},{"type":"text","value":" K·V를 그대로 두지 않고 작은 벡터 하나로 줄여 저장했다가, 쓸 때 W로 다시 펼칩니다. 671B인데 토큰당 캐시는 8B보다 작습니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"bf16 기준. DeepSeek-V3는 레이어당 압축 벡터 512 + 위치용 64 = 576개, 압축이 없다면 헤드 128개 × (K 192 + V 128)로 계산했습니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"GPU 메모리가 동시 요청 수를 정한다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/b06d8d77a25a346e95af691996dd2570/eaa1b/gpu.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 41.8%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAIAAAB2/0i6AAAACXBIWXMAAAsTAAALEwEAmpwYAAABjElEQVQY002NTWsTURSG598ICi60rkqFCq7ctOIvKAil0nVrBRFx3V3Xghs3LoQuXFT6sRFiEqwTk9pANKVDJnPv3I+ZczN37rekaYrw8HJ4XnhP1G58+/jh/f7nTwdf9s+7P77H468NdNicctREx7Ns4ZMWPmnjo9a1P2hk8TmJ3r7euXfn1pPHywt3b+++21l703zwvLO0ES+ux8ubnZXt3rNXZ6vbvZWt7tOXvUebncX1+OGL+P7a6dbeIOIEXVz8HY0SnGMoKGOACOAZeTlG5WXKR6jErEY54Py6QjlwDpEWlP7pQtK35NLXEIIPwYRgZyiW0sFPSH5bnv7vQ7DOqsgIptBQZUOR9DVQa7TVtdXK6tpoVTMk04HMhhVObvxVKqNkVImSEsI5C8F7Z7xRTtdzpLPae+ed887eeKul1dKoKuIk+3Xa7vc6nCCKx4zgCRRTymICvIJCQiEKSrJUlBwKOgEenAneOqMiLYUoWAV8AlyUrBLcajmfr525Yv5teijpjfJGWSX/AdUho0kkMpuDAAAAAElFTkSuQmCC'); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"gpu","title":"gpu","src":"/static/b06d8d77a25a346e95af691996dd2570/f97d7/gpu.png","srcSet":["/static/b06d8d77a25a346e95af691996dd2570/0eb09/gpu.png 500w","/static/b06d8d77a25a346e95af691996dd2570/1263b/gpu.png 1000w","/static/b06d8d77a25a346e95af691996dd2570/f97d7/gpu.png 2000w","/static/b06d8d77a25a346e95af691996dd2570/49142/gpu.png 3000w","/static/b06d8d77a25a346e95af691996dd2570/eaa1b/gpu.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"그래서 LLM 서빙에서 동시 처리량은 연산이 아니라 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"KV Cache 메모리"}]},{"type":"text","value":"가 정합니다. 이걸 줄이려고 나온 기법들입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"GQA"}]},{"type":"text","value":" : 여러 Q 헤드가 K, V 헤드를 나눠 씁니다. (Llama 3 8B는 Q 32개, KV 8개 → 이미 1/4)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"KV 양자화"}]},{"type":"text","value":" : fp8로 저장하면 절반"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"PagedAttention"}]},{"type":"text","value":" : 빈 공간 낭비 줄이기"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"예전 서빙: 최대 길이만큼 미리 잡아서 낭비"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"예전 서빙 시스템은 요청이 들어오면 답이 얼마나 길어질지 모르니까, "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"최대 길이만큼 연속된 메모리를 통째로 미리 잡아"}]},{"type":"text","value":" 뒀습니다. 크기를 모르니 일단 최대 크기로 배열을 malloc해 두는 셈입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/069a4ec1f9b91015f74f46acda16abaf/eaa1b/paged-why.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 31.8%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAGCAIAAABM9SnKAAAACXBIWXMAAAsTAAALEwEAmpwYAAABMUlEQVQY0y2Py0rDQBhG83q+jaBLn8GCio+gO0HtSurCjStBi9XG2tg0aZJJMveZf25JqrQIZ/kdOF8kphN0NyLpJyqycr1sirTIsuvx/Go8v3/aSEYkrXuri3Yz+XgGQSRBvEWKth5kFB4v4PTAplPBMKtzJ1pK2NFZcjj6Pj5fNlVJ0Xpr9NtqdnJ7KUhFqgyXKas3VtKorMokXS/jj6/3V6OENyo42HZcvdyY2YMH0QfbORiC+90OvTe9t3tM5yBStGFNwdqKYaQ5URw7kBY0q0tc5YrhzkHvTXDaO72X/9nJIFmyiEmLvNFGc2/0ECwq8+UiVoIFBxbkEJxR/GcR56vEgnSwqxs6F1nFaYsYqRmuKa6tFp0zgrZG8f1OWpC9Nw4Exw3DNSfN7p3VfbB/2/9H2j635R0AAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"paged why","title":"paged why","src":"/static/069a4ec1f9b91015f74f46acda16abaf/f97d7/paged-why.png","srcSet":["/static/069a4ec1f9b91015f74f46acda16abaf/0eb09/paged-why.png 500w","/static/069a4ec1f9b91015f74f46acda16abaf/1263b/paged-why.png 1000w","/static/069a4ec1f9b91015f74f46acda16abaf/f97d7/paged-why.png 2000w","/static/069a4ec1f9b91015f74f46acda16abaf/49142/paged-why.png 3000w","/static/069a4ec1f9b91015f74f46acda16abaf/eaa1b/paged-why.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ol","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"예약 낭비"}]},{"type":"text","value":" : 최대 길이만큼 미리 잡았는데 끝까지 안 쓰면 그대로 남습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"내부 단편화"}]},{"type":"text","value":" : 짧게 끝난 요청은 잡아 둔 덩어리 안에 빈자리가 남습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"외부 단편화"}]},{"type":"text","value":" : 요청마다 덩어리 크기가 달라서 사이사이 틈이 생기고, 새 요청이 못 들어갑니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"vLLM 논문이 측정해 보니 기존 시스템에서 KV Cache 메모리 중 실제 토큰이 차지한 비율은 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"20~38%"}]},{"type":"text","value":" 밖에 안 됐습니다. vLLM은 이걸 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"96%"}]},{"type":"text","value":" 까지 올렸습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"PagedAttention: KV Cache를 OS 페이지처럼"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"vLLM의 해법은 OS의 가상 메모리에서 가져왔습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/2caf33dbb61a2f7248f44087a0547019/eaa1b/paged.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 40.8%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAIAAAB2/0i6AAAACXBIWXMAAAsTAAALEwEAmpwYAAABkUlEQVQY00XPuZLTMACA4TwZBQX3DGcDWZLsZtnkKSgoeAMKCt4GKqChYEicOLYsS7Is+ZB1S06YMLPDzN9+xT8RDNfFrmMkWOWNNEpEp6PTTomuRqyCLauiM97b6PXoZLQq3DbxeKM331xPg9OiZbgEQ8eDkVrKJOMYsxoDw0sPf3WUpqVUUkV3iyPZhORrEMxIoYbOWdW3XA+tFALgvuFtTwvDClf+7ioMIFdC/MdjtYvZ95Ptj9E5NTgtrRKjN6do+hr6Bh7zn7ZKa0YDz0/Fj5Pm0dtg5BmboUMgPSR/aoJGb4M9D9cE5WmCIcj3245Vo9NKtDDb57tNvk9k30ZrztiZgVFCcckp6XjdMDoGyynBJagJIiXgdeWN8lpWqKAEEQQbVumhP45h0lCCQEbKAhUZAgdcpLjIst2WQID/RcpzME9hnhaHPUh3eZpk+y3FcPLl86fXL5+tr+bX84vV4uLdm1ery9lyPl0v56ur2fV8erN4u5xN18vFzeXs+eMHTx/de/Hk4f27dz5+eP8XaZ+b62W0Rd8AAAAASUVORK5CYII='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"paged","title":"paged","src":"/static/2caf33dbb61a2f7248f44087a0547019/f97d7/paged.png","srcSet":["/static/2caf33dbb61a2f7248f44087a0547019/0eb09/paged.png 500w","/static/2caf33dbb61a2f7248f44087a0547019/1263b/paged.png 1000w","/static/2caf33dbb61a2f7248f44087a0547019/f97d7/paged.png 2000w","/static/2caf33dbb61a2f7248f44087a0547019/49142/paged.png 3000w","/static/2caf33dbb61a2f7248f44087a0547019/eaa1b/paged.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ol","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"16토큰씩 블록으로"}]},{"type":"text","value":" : 미리 잡지 않고, 토큰이 16개 찰 때마다 빈 블록을 하나씩 받습니다. 낭비는 마지막 블록 하나뿐입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"블록 테이블 = 페이지 테이블"}]},{"type":"text","value":" : 블록이 메모리 여기저기 흩어져도 테이블이 위치를 기억합니다. 어텐션은 테이블을 따라가며 K·V를 읽습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"같은 앞부분은 블록 공유"}]},{"type":"text","value":" : 같은 프롬프트로 여러 답을 만들면 그 블록을 같이 가리키고, 고쳐야 할 때만 복사합니다(copy-on-write)."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"이 아이디어로 만든 서빙 엔진이 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"vLLM"}]},{"type":"text","value":"입니다. 같은 GPU에 요청을 더 많이 묶을 수 있게 되면서 당시 다른 서빙 시스템보다 처리량이 2~4배 높았고, 지금은 이 방식이 서빙 엔진의 사실상 표준이 됐습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Kwon et al. (UC Berkeley), "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Efficient Memory Management for Large Language Model Serving with PagedAttention"}]},{"type":"text","value":", SOSP 2023"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"9. 처음 본 세 현상, 이제 답할 수 있다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"현상 1. 첫 글자만 유독 느리다"}]},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\n첫 토큰은 프롬프트 전체를 레이어 32개에 통과시켜야 나옵니다(Prefill). 그다음부터는 KV Cache 덕분에 새 토큰 하나만 계산해서 술술 나옵니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"현상 2. 앞부분이 같으면 싸고 빨라진다"}]},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\n토큰의 K, V는 그 토큰과 그 앞만 보고 만들어집니다. 앞이 같으면 K, V도 같으니까, 서버가 보관해 둔 K, V를 꺼내 쓰고 그 부분의 Prefill을 건너뜁니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"현상 3. 대화가 길수록 느리고 비싸다"}]},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\nDecode는 토큰 하나마다 이 요청의 캐시 전체를 읽습니다. 캐시는 토큰 하나에 128KB씩 쌓여서, GPU 한 장이 동시에 받을 수 있는 요청 수도 줄어듭니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"정리"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ol","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"LLM은 토큰을 하나씩 만드는 루프다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"어텐션은 점수 → 비율 → 섞기로 “어디에 집중할지” 정한다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"LLM은 같은 줄 안에서 어텐션한다. Q는 지금 토큰의 쿼리, K는 얼마나 볼지 정하는 재료, V는 실제로 건네지는 내용."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"앞 토큰의 K, V는 바뀌지 않는데 매 회차 다시 필요하고, Q는 한 번 쓰고 끝난다. 그래서 K, V만 저장한다 = "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"KV Cache"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"Prefill은 한 번에 채우고(연산이 병목), Decode는 한 칸씩 늘리며 캐시 전체를 읽는다(메모리가 병목)."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"비용은 GPU 메모리. 토큰 하나에 128KB(Llama 3 8B)라서, 길수록 동시에 받을 요청 수가 줄어든다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"부록: MLP는 왜 캐시가 없을까?"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"어텐션은 주고받고, MLP는 각자 처리한다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"어텐션은 am 자리의 출력을 만들 때 7개 토큰 전부의 K·V를 섞어 옵니다. MLP는 am 벡터 하나만 넣습니다. 그래서 아래 두 코드의 결과가 같습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/4281426d614c6b8aed4e688808769dbf/eaa1b/app-mlp-role.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 34.199999999999996%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAAsTAAALEwEAmpwYAAABoUlEQVQY0z3KzU/TUAAA8P4VJB4MuGkZNq4kQMLJs4kHryae/H8MHIAQElwCWGgszosH48eFI5Ou2+q6bl0Lk71+vL62r69du3W0Bg8mv+OPmiVxlsZJHAMLAxsHOCYkMW0MEXFc4vkR8iJCJlGURtFknt7n/yjLdufTJIrTPojVW9Ia+qLqXnahqKKGAhs9+KsH1T/h0IwBiou7aT6fFv/kWUqNwQgjx3SAZCiS0e1cq1eaLGq/JV2RdKVl9CS9Kw7kRr+t3PSJDwPPwZ4TIJsELvX2zevqypPnm+sbLLPBMuxyiaWX1p5VVhm6ulxerz5dZSrsCs3Q5Up5iS4t0qXFyuNHDx8svHr5gvr+9ctx7ZD/cMQd1YQzrv7x7BPPfRb4usALpyf84S63v3V6sH2y947b3zqv7Qrvd85re/zB9rc6RyVRaIPb8eh6ZGjYR0WeFXmWz2d32bQo8nQsW82frnwBxB+ufFGYnQJ0CtC+hwbUhATYg6Hvhr7ruw5yTA9aHrSQY05CLwEq1puh0SI37UCXfE30NdHTxGDYnNmDvwJ8Qd0ixQ1VAAAAAElFTkSuQmCC'); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"app mlp role","title":"app mlp role","src":"/static/4281426d614c6b8aed4e688808769dbf/f97d7/app-mlp-role.png","srcSet":["/static/4281426d614c6b8aed4e688808769dbf/0eb09/app-mlp-role.png 500w","/static/4281426d614c6b8aed4e688808769dbf/1263b/app-mlp-role.png 1000w","/static/4281426d614c6b8aed4e688808769dbf/f97d7/app-mlp-role.png 2000w","/static/4281426d614c6b8aed4e688808769dbf/49142/app-mlp-role.png 3000w","/static/4281426d614c6b8aed4e688808769dbf/eaa1b/app-mlp-role.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"그럼 MLP는 맥락을 모를까?"}]},{"type":"text","value":" 아닙니다. 어텐션 다음에 오니까, 앞 토큰 정보가 이미 섞인 am 벡터를 받아서 가공합니다. 회의에서 듣고(어텐션) 자리로 돌아와 혼자 정리하는(MLP) 셈입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"그래서 MLP는 캐시할 게 없습니다."}]},{"type":"text","value":" 이전 토큰의 MLP 결과는 다시 쓸 일이 없습니다. 매번 이전 토큰이 필요한 건 어텐션의 K, V뿐입니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"MLP가 없으면 토큰들이 다 비슷해진다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"어텐션 출력은 V들의 가중평균이라, 섞기만 반복하면 차이가 지워집니다. 물감 일곱 개가 매번 서로를 조금씩 섞어 오면 결국 전부 같은 회갈색이 되는 것처럼요. 순수 어텐션만 쌓으면 층이 깊어질수록 이중 지수적으로 한 벡터로 무너진다는 연구도 있습니다(rank collapse)."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/949e4f925cf996bd9abd317aaa63268b/eaa1b/app-mlp-why.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 38.4%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAIAAAB2/0i6AAAACXBIWXMAAAsTAAALEwEAmpwYAAABj0lEQVQY003M3U7bMBgG4NzSDnbCtCEETNoGYxDRrioSN7MDTrggJMQEQ0Eb3Rg/TdL8tI0TO7ab0DbUATeOjRIEQnr06nsP3k/DCNAY3ZI4SUhZ8HJ+/0LO7ycpQeEwJbF4YMUrgueC55rZue7Zvy3bnIBhyjKlpJRCPUsoti0ToYixWV5jbDa7yzjnpeDa/8Nz0zz4eXKSXHRuYlCNxVyVhSoLWTzwFIoxVBlWGalyiusbq2ksMqr9/XV62Tn+9+cssm9ANFRKyXpZEZxeHo27x7l3dmefMtfIHYM5xswxctcokKk51xe+deV2rwLXQkFfqVKKotpXb0RB/ZJ6cuSXI08mvkr6sqYSX06g5trdvmt7PdPrmb5jBb4DBh4YeARFBAY4GmIICAoxBBAMoqD/BAy8lGLt66fV3dZ2S99obK7vNPV2Y6vd2Npp6o3Ntdb6SmtteXvjS7upf9e/fV5dWv6wsLL47uPS+4W3b/b3fmgYBjSOauEIhQSBCgQ0DslThQDXdXo7ejFJKcvGjzgZkqv34RfFAAAAAElFTkSuQmCC'); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"app mlp why","title":"app mlp why","src":"/static/949e4f925cf996bd9abd317aaa63268b/f97d7/app-mlp-why.png","srcSet":["/static/949e4f925cf996bd9abd317aaa63268b/0eb09/app-mlp-why.png 500w","/static/949e4f925cf996bd9abd317aaa63268b/1263b/app-mlp-why.png 1000w","/static/949e4f925cf996bd9abd317aaa63268b/f97d7/app-mlp-why.png 2000w","/static/949e4f925cf996bd9abd317aaa63268b/49142/app-mlp-why.png 3000w","/static/949e4f925cf996bd9abd317aaa63268b/eaa1b/app-mlp-why.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"실제 Transformer가 무너지지 않는 건 두 장치 덕분입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"잔차 연결 (Add)"}]},{"type":"text","value":" : 어텐션 결과에 원래 내 벡터를 다시 더합니다. 섞어 온 색 위에 내 색을 다시 칠하는 셈입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"MLP"}]},{"type":"text","value":" : 토큰마다 따로 비선형 변환을 해서 섞인 색을 각자 다른 방향으로 다시 틀어 줍니다."}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"MLP는 덩치도 큽니다. Llama 3 8B 레이어 하나에서 어텐션이 약 42M, MLP가 약 176M으로 "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"파라미터의 약 80%"}]},{"type":"text","value":" 가 MLP입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"blockquote","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Dong et al. 2021, "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Attention is not all you need: pure attention loses rank doubly exponentially with depth"}]}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"“MLP = 지식 저장소”, 지금은 이렇게 본다"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"div","properties":{"className":["image-sized"],"style":"width: 700px"},"children":[{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-wrapper"],"style":"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 2000px; "},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"a","properties":{"className":["gatsby-resp-image-link"],"href":"/static/8fce227e751f3a76faf5b32318068c07/eaa1b/app-mlp-knowledge.png","style":"display: block","target":"_blank","rel":["noopener"]},"children":[{"type":"text","value":"\n "},{"type":"element","tagName":"span","properties":{"className":["gatsby-resp-image-background-image"],"style":"padding-bottom: 34.400000000000006%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAHCAIAAACHqfpvAAAACXBIWXMAAAsTAAALEwEAmpwYAAABlklEQVQY0x3MyU7bQACAYb9kL731AWiFY5IRiqlUDpyrnipVLBFZnDQBCajEqWoPbVUk1oQkBmM7wfZ47BlPvIzjFQS3T//h5xbEwQjCxxnFqEiXecLyhGEEXWi+Os4TVmTLBD86kzM4/p9ZcoHuclsuXI2jGM11VVfvkWVgB3qvC2OmqYoMzblrW8Sx8zxLLFn/d/Lw98S++omHv8noV2kOucDDxLYIskJKmE/jYFGmMfMxC0jgOVkclCl7ypOMzJP5ZTy7To1hCcclvC1dhZvcj89HF3eaPJzejKY3U3UShAtMIpdE1GeERsSLiBeGQWCjBXJ876WE2AspDbmxpfxRrhRqKtRQfUuPkZ+xa5mMtEiD6YO5VGGG/CcaFedT/0KmOsoVc2l4pRsW3I/To53mdqPX2O3sNaRmo7vf7fdarV670293Bm3poC0NpO7hoN/vvOBA6h42W9+/be8fH59yX798BvwHYe39WoUHwjoQgMBX66Bar66KNf7jurABKmKNr9d4scZvgMqmCAC/8u7tm61P4jPpDVXlcCZGWwAAAABJRU5ErkJggg=='); background-size: cover; display: block;"},"children":[]},{"type":"text","value":"\n "},{"type":"element","tagName":"img","properties":{"className":["gatsby-resp-image-image"],"alt":"app mlp knowledge","title":"app mlp knowledge","src":"/static/8fce227e751f3a76faf5b32318068c07/f97d7/app-mlp-knowledge.png","srcSet":["/static/8fce227e751f3a76faf5b32318068c07/0eb09/app-mlp-knowledge.png 500w","/static/8fce227e751f3a76faf5b32318068c07/1263b/app-mlp-knowledge.png 1000w","/static/8fce227e751f3a76faf5b32318068c07/f97d7/app-mlp-knowledge.png 2000w","/static/8fce227e751f3a76faf5b32318068c07/49142/app-mlp-knowledge.png 3000w","/static/8fce227e751f3a76faf5b32318068c07/eaa1b/app-mlp-knowledge.png 3456w"],"sizes":"(max-width: 2000px) 100vw, 2000px","style":"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;","loading":"lazy","decoding":"async"},"children":[]},{"type":"text","value":"\n "}]},{"type":"text","value":"\n "}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"처음엔 MLP를 key-value 메모리처럼 보고 “에펠탑 → 파리” 같은 사실이 특정 레이어에 있다고 해석했지만, 고친 사실이 주변으로 퍼지지 않는다는 반론이 나왔습니다. 지금은 지식이 여러 레이어에 흩어져 겹쳐 저장된다(superposition)고 보는 게 주류입니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"비유하면 MLP는 사전, 어텐션은 사전을 찾아보는 손입니다. 다만 사전은 여러 레이어에 나뉘어 있고, 페이지가 겹쳐 인쇄돼 있습니다. 이 분야는 아직 연구가 활발해서 해석이 계속 바뀌고 있습니다."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"더 읽어볼 자료"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://sooftware.github.io/attention/"},"children":[{"type":"text","value":"어텐션 메커니즘 · Sooftware 블로그"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://jalammar.github.io/illustrated-transformer/"},"children":[{"type":"text","value":"The Illustrated Transformer · Jay Alammar"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://arxiv.org/abs/1706.03762"},"children":[{"type":"text","value":"Attention Is All You Need (2017)"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"element","tagName":"a","properties":{"href":"https://arxiv.org/abs/2309.06180"},"children":[{"type":"text","value":"Efficient Memory Management for LLM Serving with PagedAttention (2023)"}]}]},{"type":"text","value":"\n"}]}],"data":{"quirksMode":false}},"excerpt":"KV Cache 이해해보기 - 어텐션부터 vLLM까지 최근 팀내에서 발표했던 KV Cache 내용을 글로 정리해봤습니다. AI…","fields":{"readingTime":{"text":"41 min read"}},"frontmatter":{"title":"KV Cache 이해해보기 - 어텐션부터 vLLM까지","userDate":"25 October 2026","date":"2026-10-25T10:00:00.000Z","tags":["nlp","llm"],"series":"AI 개념","excerpt":null,"image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#f8f8f8","images":{"fallback":{"src":"/static/fc74bacba182ceb274f5d2fcb763f903/e620f/with-cache.png","srcSet":"/static/fc74bacba182ceb274f5d2fcb763f903/5c54e/with-cache.png 750w,\n/static/fc74bacba182ceb274f5d2fcb763f903/e35fe/with-cache.png 1080w,\n/static/fc74bacba182ceb274f5d2fcb763f903/8b343/with-cache.png 1366w,\n/static/fc74bacba182ceb274f5d2fcb763f903/e620f/with-cache.png 1920w","sizes":"100vw"},"sources":[{"srcSet":"/static/fc74bacba182ceb274f5d2fcb763f903/c4ba1/with-cache.webp 750w,\n/static/fc74bacba182ceb274f5d2fcb763f903/a761e/with-cache.webp 1080w,\n/static/fc74bacba182ceb274f5d2fcb763f903/6a9e9/with-cache.webp 1366w,\n/static/fc74bacba182ceb274f5d2fcb763f903/f16e6/with-cache.webp 1920w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.4598958333333334}}},"author":[{"id":"Soohwan Kim","bio":"ML Engineer at Karrot (당근).","avatar":{"children":[{"gatsbyImageData":{"layout":"fullWidth","backgroundColor":"#282838","images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/248f9/soohwan.png 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/fd435/soohwan.png 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/0d6f4/soohwan.png 120w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/e7f45/soohwan.webp 40w,\n/static/a9e6b445142b247ee4cfa66155398bb2/589ec/soohwan.webp 80w,\n/static/a9e6b445142b247ee4cfa66155398bb2/71a38/soohwan.webp 120w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6833333333333333}}]}}]}},"relatedPosts":{"totalCount":51,"edges":[{"node":{"id":"e5a017aa-767e-5ef7-ba58-2473728f1465","excerpt":"KV Cache 이해해보기 - 어텐션부터 vLLM까지 최근 팀내에서 발표했던 KV Cache 내용을 글로 정리해봤습니다. AI…","frontmatter":{"title":"KV Cache 이해해보기 - 어텐션부터 vLLM까지","date":"2026-10-25T10:00:00.000Z"},"fields":{"readingTime":{"text":"41 min read"},"slug":"/kv-cache/"}}},{"node":{"id":"6529d72e-80e7-5d61-a672-d66276a3f641","excerpt":"MoE(Mixture of Experts) 기초부터 DeepSeek 혁신까지 작년 이맘때쯤 DeepSeek-V3가 저비용으로 엄청난 성능을 보이면서 화제가 되었습니다. 그 핵심 기술인 MoE(Mixture of Experts…","frontmatter":{"title":"MoE(Mixture of Experts) 기초부터 DeepSeek 혁신까지","date":"2026-01-15T01:11:55.000Z"},"fields":{"readingTime":{"text":"8 min read"},"slug":"/developed-moe/"}}},{"node":{"id":"8b49d3ef-4ce3-568c-9d71-240ff17fc3e0","excerpt":"BERT는 사실 Diffusion 모델이였다?! 최근 굉장히 흥미로운 글을 읽게되어 공유합니다. 원문 : link BERT와 Diffusion이 같은 방식이다?! NLP 연구자들에게 BERT는 너무 익숙한 모델입니다. 201…","frontmatter":{"title":"BERT는 사실 Diffusion 모델이였다?!","date":"2025-10-21T12:00:00.000Z"},"fields":{"readingTime":{"text":"8 min read"},"slug":"/bert_diffusion/"}}},{"node":{"id":"7c7e1676-ea84-58de-970c-ddec9aa10660","excerpt":"RLHF는 수다쟁이를 만든다?! (Does RLHF Breed Verbose Chatterboxes?!) RLHF(Reinforcement Learning from Human Feedback)는 OpenAI의 ChatGPT…","frontmatter":{"title":"RLHF는 수다쟁이를 만든다?! (Does RLHF Breed Verbose Chatterboxes?!)","date":"2024-03-13T01:11:55.000Z"},"fields":{"readingTime":{"text":"7 min read"},"slug":"/rlhf-vervosity/"}}},{"node":{"id":"2b9d3e22-1796-5fab-878d-5941d2e76e9d","excerpt":"LLM Paper Abstract - 2023.12 LLM…","frontmatter":{"title":"LLM Paper Abstract - 2023.12","date":"2024-01-05T10:00:00.000Z"},"fields":{"readingTime":{"text":"5 min read"},"slug":"/llm-abs-202312/"}}}]}},"pageContext":{"slug":"/kv-cache/","prev":{"excerpt":"옅어지는 경계에서, AI 엔지니어로서의 나의 무기가 뭘까? 요즘 일하면서 자주 드는 생각이 있습니다. 직군 사이의 경계가 점점 옅어지고 있다는 생각입니다. 몇 년 전에는 AI…","frontmatter":{"title":"AI 엔지니어로서의 나의 무기","tags":[],"date":"2026-10-17T10:00:00.000Z","draft":false,"excerpt":null,"series":"회고","image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAANCAIAAAAmMtkJAAAACXBIWXMAAAsTAAALEwEAmpwYAAACyklEQVQozz3S2U8TURQH4PkLVcKDUdFEEiLRkBiMCggBEQXKomUpBVkLtggWSgOU3bKVGsoa0EpLgdIp7Uw7A23n3jt3mTFQNTkPv7PkPH0cw5BiSFXAMGAEZltGoMawrt2UTjSKGPl/AKiq0JvMMQx1CjQCMEYqglgFGoYaRZeSKCViYjwqxHgE0owijSBKAMFAI1AniGHAaRTFZPA7mtk/S+2FlSOBXAFCMbyUxEj4xOtZE2K8nBQYgcex0MG53x8NBOOhZCbBMORSkPhOFevMjsm62GC2uTyHQZEoEK17PSAjr6+tHP061HVdVpIhKez6sTCyMDa54ZrxLYUTYc69Hzc7dh6/7ij4YL9TUNlstgQFNcBLUzOzjIC1lUX7xLfIpbh8sGGZHy2uLbtVlH/3aV7TQMuPk11uYU8wDG0UvbPee/7pTaWhuLh8bnnL4xfc2ycfjR2dXZ3Tu97hHU/X+ODtBw8LnjzLLczPKcx72Vk7sjHFrf6SGqybPc6t4RGHrlP7yFhpueFMhB19oxUVNeOTznmv+xIka+oM76rqhAve6XS0fe2utrWbXFZucV8wO7ZNw99H7Q7/zwNL/2BNrXEvwLeYesKnx+kr2bPjPeSPyquqzZ2fz89DE86xeZ+72tbeMjnIbQflSffu0MBQb1dvaUnZw0eFraa+iCCXvG2ur280GlvfNxoi4nn9x085ufeLX7xqamzttlis7mlPwMfJVxl/VFlY256YXZ1b9a1s/uQFiU9mjD3f+nos/b2Wtq6Os+jJof/gi91mGf5is4+61pemNr3+8BGnU8QoIpRipgNVVQnUNawiJS6KoVDgNBQUEhcEK7pGEQEpKEOaQSSNcUYjiMtyY1jRsELVjJKW02lZSsQScV6IR8X4RfwiIieFdFpKpSSCFKYCdq0QUKxwWc/XL264ahhmd//mN87J35C1/b/+AP41fWZwHHV3AAAAAElFTkSuQmCC"},"images":{"fallback":{"src":"/static/dcb85e0dfcc118ec251365c7f1d6226b/6e90d/as-an-ai-engineer-2.png","srcSet":"/static/dcb85e0dfcc118ec251365c7f1d6226b/ea847/as-an-ai-engineer-2.png 750w,\n/static/dcb85e0dfcc118ec251365c7f1d6226b/4a806/as-an-ai-engineer-2.png 1080w,\n/static/dcb85e0dfcc118ec251365c7f1d6226b/e2aad/as-an-ai-engineer-2.png 1366w,\n/static/dcb85e0dfcc118ec251365c7f1d6226b/6e90d/as-an-ai-engineer-2.png 1536w","sizes":"100vw"},"sources":[{"srcSet":"/static/dcb85e0dfcc118ec251365c7f1d6226b/57584/as-an-ai-engineer-2.webp 750w,\n/static/dcb85e0dfcc118ec251365c7f1d6226b/984df/as-an-ai-engineer-2.webp 1080w,\n/static/dcb85e0dfcc118ec251365c7f1d6226b/4a276/as-an-ai-engineer-2.webp 1366w,\n/static/dcb85e0dfcc118ec251365c7f1d6226b/7db9c/as-an-ai-engineer-2.webp 1536w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6666666666666666}}},"author":[{"id":"Soohwan Kim","bio":"ML Engineer at Karrot (당근).","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"21 min read"},"layout":"","slug":"/as-an-ai-engineer/"}},"next":{"excerpt":"Welcome to sooftware;s Blog 안녕하세요! sooftware blog…","frontmatter":{"title":"Welcome to sooftware.io","tags":["welcome"],"date":"2033-12-10T10:00:00.000Z","draft":false,"excerpt":null,"series":"회고","image":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsTAAALEwEAmpwYAAACmUlEQVQozx3M+08ScQAA8PvTqrV+0LVWy8fSoS2nbKmbZq5wlFqGEJaOaRAuU7GSgIE85KEgcQjyOgrE483xnNz0FEEed9+7Vtvn5w8E58GvHLBj4AADVgxYMsCUBobkP/oE0MaAOgZUKFCiQBkh95MthQXZs9htVue+0Qy5i8BdAocF4CwARw4cYJQVo/YylDkDjClgSFD6/4suDpQopUgyUmtK7Yqj4ZDGYIeCOB04pZxYzZGuwplr3yntLTOuEu0s0o4C4ywxtiwwpcBuCviKJIIzqsOU1R+vXp2tWRJQjGgFio2tvaB0xyVWwWtG/6rOu2U9/mJEFuV2ic5nilaCOINdUi2SIlqMN5R2IDGb98QbiED1ZhWvAX8a96RwX5aAY2U4WnanLtzJc2cMtyCZUKnWoGiKBIXzep6oBwKh8HH0PB+l8AREEEWSpEiaIQFTJZkayVy2mIsmfdUEpSp90WCIqyZWrp0ULvP5YhxNHh0G/MfJRgULeTRQCoswNINXmp5iayd+vY1ey8LVjXBV6idWHNnPcPYTXBDCp6bf6XK+GAolY2hWrzU4TZvfZLPQD806EkXsufryUU4UJETImThcEUXq/H20n/28jz3Zw34pUSnMRqXo/dLczPwCf6mnk8XjPPvj0UPcmT6RhGMySy0miVKrWFzdFMu2tTp1xGUVcLn9vQNt7Z0rc9NTw6O3brTfvtneda+jo+2uiDN6hiLQ1Kuh6Wm2fPNNJQuLFwUDTwZ6WX3L/HcBm9UgWx9hPep+8HCI1f/h9ZSYxxXPvTBLeQcbSz8/zvh3FVDb/TsjI6wF/pjzu1A++1TI7uYNdq1MDCrmORr+pE04rns7vCscD8sFCfVCXCVAtnj2Nf7X2QnR2OO/5QntTUY9L1sAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/7e07b3dee7a4598e97e04f770da6082f/56785/welcome-post-2.png","srcSet":"/static/7e07b3dee7a4598e97e04f770da6082f/0dee1/welcome-post-2.png 750w,\n/static/7e07b3dee7a4598e97e04f770da6082f/8beaa/welcome-post-2.png 1080w,\n/static/7e07b3dee7a4598e97e04f770da6082f/a4198/welcome-post-2.png 1366w,\n/static/7e07b3dee7a4598e97e04f770da6082f/56785/welcome-post-2.png 1672w","sizes":"100vw"},"sources":[{"srcSet":"/static/7e07b3dee7a4598e97e04f770da6082f/a66aa/welcome-post-2.webp 750w,\n/static/7e07b3dee7a4598e97e04f770da6082f/65dd5/welcome-post-2.webp 1080w,\n/static/7e07b3dee7a4598e97e04f770da6082f/f9724/welcome-post-2.webp 1366w,\n/static/7e07b3dee7a4598e97e04f770da6082f/52a32/welcome-post-2.webp 1672w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.562799043062201}}},"author":[{"id":"Soohwan Kim","bio":"ML Engineer at Karrot (당근).","avatar":{"childImageSharp":{"gatsbyImageData":{"layout":"fullWidth","placeholder":{"fallback":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAABYlAAAWJQFJUiTwAAADjElEQVQ4y23Oy48TdQDA8TExEaTttvSBfdHddru7bWc6bWd+8+i0u23n0SdlBRQXZGHDgiiLDwgqXjCIJEaC8cLBmHgwUU9EYzRqQuLJi/4NJB6NF+8evgbOHr7XT76SM5ix7s4QnRHnnm9yZzvBTIswN8JcP5bC67ZoWi7CdtFsn5bloZo+dSNAMYbIIqCmD6mJIVXdR3J6I7r9IQ3T59pZlW+uxzndCbPdC/HZXoq3TikowsfqBOjOmFZ7SrM9p2kfRbXn1O0jKNbsSbJ5BMnu9Gl3+qiiz4OPSvz0QRLHFGwOZH6+neLHT5YRbRerM0J0JrTas/8FZXNKzZgg2es+ZtvDdbs8+iHJl+9lWVP7OG2br26k+PvXJCfmFqoYIpwhTXtMwxpTN0coxhjFmCKbE2rGmKoYPQY9qi2fyzsq//4e5sNXi5TlLqbd5t6lPH/9kuDTG2usqgGi7dO0hzSsIXUzQDaGKGKMLEZUxZCKCJB02+PUpMH9vQwP7ybYnSxyfNrjwumA3dEiD94v8cfnK7xzWsbpujQtH9X0qBs+iuEjC5+a7lPVPSqai9SyelyYFbjkh5hqC3TkKOutLON2nkYpgtuMcT6IszdPslo8SLlcpmUH1A0XRQyQhUtNuFQ1l4o2QFI1h3OTVXbcg+grIfSVBRqlEGopgrYSRi0e4KiIse1mScSeopA/iLADmoZL3Rgg631q+oCq1qeq95FOzhQefpzn7eMpBmqcoBVj04pzwkky0eNMRYIdL8F3twusLe2nslrE6g5pGX1U8/FhD1nvUW1tUJK7SH9+n+DR1zF2vQSvH0lz88wiV45V2J1WuHqsxLXNAnuzNJdfWmK5/BylUgFhb6DZAxR9g7rpIos+Wttja2uE9M9vYb69k+DqfJErWzLOuoVqdKlpDo4jODms4zlFckuHObx4iHwuRi6fYqmYY201x/LqMrnFDNl8gRvvukhNdT9394rcOlNFMxrUhYFmW9imTEsp0mhWKC2nWSokyWSiFA7HSKcXiEb34fayyLVniUQk8rmn2drKIqlqiNcuVrh2ymQS1Nh8sc/GoMEbl7u8eb5D1yqTzkRJJsOkkiGymQVShyLE4yHWKnEq1RjpTIhmI8r2y3mkyloI38uha1n6bpVXrkw5+4LD/S/2uHnrIopaJhZ9hmQqwqHkAXJP8BDhyD46ZoWdMxOGcx/PNfECk/8AbxXdRjRliPoAAAAASUVORK5CYII="},"images":{"fallback":{"src":"/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png","srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/34f77/soohwan.png 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/a94f6/soohwan.png 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7cf1f/soohwan.png 1148w","sizes":"100vw"},"sources":[{"srcSet":"/static/a9e6b445142b247ee4cfa66155398bb2/38420/soohwan.webp 750w,\n/static/a9e6b445142b247ee4cfa66155398bb2/7470d/soohwan.webp 1080w,\n/static/a9e6b445142b247ee4cfa66155398bb2/b5ef6/soohwan.webp 1148w","type":"image/webp","sizes":"100vw"}]},"width":1,"height":0.6829268292682927}}}}]},"fields":{"readingTime":{"text":"8 min read"},"layout":"","slug":"/welcome-post/"}},"primaryTag":"nlp"}}, "staticQueryHashes": ["3170763342","3229353822"]}