Ring Attention and Sequence Parallelism: Distributing Context Across GPUs
When a single GPU can't hold the KV cache, you distribute the sequence across multiple GPUs. Here's how ring attention enables million-token contexts.
// tag
Posts tagged "DeepSpeed"
1 post found