Posts tagged "GQA"

KV Cache Memory Math: Calculating Exactly How Much VRAM You Need

The exact formula for KV cache memory and worked examples for every major model architecture. Calculate your GPU requirements precisely.

Multi-Query Attention and Grouped-Query Attention: Reducing KV Cache by 8× at the Architecture Level

Standard multi-head attention uses separate K and V for each head. MQA and GQA share them — reducing KV cache dramatically with minimal quality loss.