Glossary
Abstract interpretation. Running a program on over-approximating abstract values to compute guaranteed bounds. Veer’s planner is one. Ch. 7
Affine / linear type. A value that may be used at most once (affine) or exactly once (linear). Veer contexts are linear. Ch. 7
Attention. The transformer operation where each token mixes the values of earlier tokens, weighted by query–key similarity. Ch. 2
Block (page). Fixed-size unit of KV-cache allocation in paged attention, set by a model’s block: in Veer. Ch. 3
Block scaling / microscaling. Giving each small group of elements its own scale factor (MXFP4: 32 elements, NVFP4: 16). Ch. 5
BPE. Byte-pair encoding, the common tokenizer algorithm. Every token is at least one byte. Ch. 1
Constrained decoding. Masking tokens during generation so the output matches a grammar or JSON schema. as json<T>, as choice[...]. Ch. 4
Context. In Veer, a linear handle Ctx<M> to a sequence of tokens and its KV cache on model M.
Continuous batching. Letting requests join and leave a decode batch at every step. Ch. 1
Decode. Generating output tokens one at a time. Memory-bandwidth-bound. Ch. 1
Disaggregation. Running prefill and decode on separate GPU pools and transferring the KV cache between them. Ch. 3
E2M1, E4M3, E5M2, E8M0. Minifloat formats named by exponent/mantissa bits. E2M1 is the FP4 element format, E4M3/E5M2 are FP8, E8M0 is the power-of-two MX scale. Ch. 5
Fork. Splitting one context into two that share all existing cache. Tutorial 2
GQA. Grouped-query attention: several query heads share one KV head, shrinking the cache. Ch. 2
γ_K. Ku/(1 − Ku): the rounding-error growth factor of a length-K sum. Ch. 6
KV cache. Stored key and value vectors for every past token in every layer. Ch. 2
Lipschitz constant. The most a function can stretch distances. It bounds how much input error grows. Ch. 6
Prefill. Processing all prompt tokens in parallel to fill the KV cache. Compute-bound. Ch. 1
Prefix caching / RadixAttention. Reusing the KV cache of shared prompt prefixes across requests. Ch. 3
RoPE. Rotary position embeddings. They make cached keys depend on absolute position, so mid-context edits invalidate the tail. Ch. 2
Soundness. A guarantee that holds for every possible execution, not just the ones you tested. Ch. 7
Span. A labelled region of a context (#[name]) that splice can replace. Tutorial 4
Speculative decoding. A draft model proposes tokens that the target model verifies in parallel. Veer’s speculate gives the transactional cache semantics it needs. Ch. 3
Splice. Replacing a labelled span inside a context. Later tokens are re-prefilled (or position-corrected). Tutorial 4
TTFT / TPOT. Time to first token and time per output token, the latency metrics of prefill and decode.
Unit roundoff (u). The maximum relative error of rounding one number to a format: 2^-(mantissa bits + 1). Ch. 5