ByteBell - AI for Engineering TeamsSkip to main content

Posts tagged with "GQA"

2 posts found

Featured image for article: KV Cache Memory Math: Calculating Exactly How Much VRAM You Need
Mar 31, 2026 KV cache VRAM calculation GPU memory GQA model deployment inference optimization

KV Cache Memory Math: Calculating Exactly How Much VRAM You Need

The exact formula for KV cache memory and worked examples for every major model architecture. Calculate your GPU requirements precisely.

Featured image for article: Multi-Query Attention and Grouped-Query Attention: Reducing KV Cache by 8× at the Architecture Level
Mar 31, 2026 MQA GQA multi-query attention grouped-query attention KV cache reduction Llama 3

Multi-Query Attention and Grouped-Query Attention: Reducing KV Cache by 8× at the Architecture Level

Standard multi-head attention uses separate K and V for each head. MQA and GQA share them — reducing KV cache dramatically with minimal quality loss.