The journal

How the context layer is built.

Benchmarks, retrieval architecture and the engineering behind indexing a codebase once and serving it to every tool that asks.

All 109 Customers 20 Engineering 89

61–70 of 89 · page 7 of 9

Mar 31, 2026 RoPErotary position embeddingsmathematical derivation

Rotary Position Embeddings: The Full Mathematical Derivation

RoPE is used by virtually every modern LLM. Here's the complete derivation from first principles, proof of the relative position property, and NTK-aware scaling.

Mar 31, 2026 scaling lawscontext lengthChinchilla

Scaling Laws for Context Length: How Performance Changes with Window Size

Similar to Kaplan's scaling laws for model size, there are scaling laws for context length. Performance doesn't scale linearly — here's the math.

Mar 31, 2026 context window sizemodel comparisonAI performance

Small vs. Large Context Windows: What's the Actual Difference for Users?

4K vs 32K vs 128K vs 200K vs 1M — what do these numbers actually mean for your experience? Bigger isn't always better.

Mar 31, 2026 softmaxattention dilutioncontext rot

Softmax Attention and the Dilution Problem: The Math Behind Context Rot

As context grows, softmax normalizes attention weights so each relevant token gets less attention. This mathematical property is why AI accuracy drops with length.

Mar 31, 2026 sparse attentionLongformerLinformer

Sparse Attention Architectures: Breaking O(n²) with O(n√n), O(n·log n), and O(n)

Multiple research approaches attack the quadratic bottleneck: Longformer, Reformer, Linformer, and linear attention. Here's the math behind each one.

Mar 31, 2026 tokenstokenizationBPE

Tokens Explained: Why Your AI Counts Words Differently Than You Do

AI doesn't read words — it reads tokens. Learn how tokenization works, why code costs more than prose, and how to calculate your token usage.

Mar 31, 2026 transformerattention mechanismquery key value

How the Transformer Attention Mechanism Actually Works (No Math Required)

The attention mechanism is the beating heart of every LLM. Here's how it decides which parts of your conversation matter most — explained with analogies before equations.

Mar 31, 2026 file uploaddocument tokenizationClaude

What Happens Inside the Context Window When You Upload a File to Claude

When you upload a PDF or code file, it gets converted to tokens and placed directly into the context window — consuming the same limited space as your messages.

Mar 31, 2026 AI memorystateless systemsRAG

Why Does My AI Forget Everything Between Sessions?

AI models are stateless. Each conversation is a fresh start. There's no database storing what you said yesterday. Here's why — and what it means for you.

Mar 31, 2026 RAG retrieval augmented generationlong context

Why RAG Exists: Context Windows Are Too Small for Your Data

If context windows were infinite, free, and didn't degrade, we wouldn't need RAG. Here's why retrieval-augmented generation exists and when to use it.