Precision types: choosing a 4-bit format
The same tiny MLP with weights in NVFP4, INT8, MXFP4, and overflow-aware MXFP4. The types carry proven worst-case error bounds, and @accuracy contracts turn them into compile-time checks.
File: examples/quant_mlp.veer (included in every release download)
// The precision layer: number formats and error bounds live in the types.// `veer types` prints each function's inferred bound; @accuracy turns it into// a compile-time contract. Bounds are sound worst cases (every input in range,// every rounding adversarial), so they are larger than typical measured error.
// NVFP4 weights: worst-case relative error ≈ 0.20.@accuracy(rel <= 0.25)fn mlp_nvfp4(x: Tensor<[1, 64], f32, abs<=1.0>, w: Tensor<[64, 32], f32, abs<=0.5>) -> Tensor<[1, 32], f32> { return (quant(x, bf16) @ quant(w, nvfp4)) |> silu}
// INT8 weights: ≈ 0.009 worst case.@accuracy(rel <= 0.01)fn mlp_int8(x: Tensor<[1, 64], f32, abs<=1.0>, w: Tensor<[64, 32], f32, abs<=0.5>) -> Tensor<[1, 32], f32> { return (quant(x, bf16) @ quant(w, int8)) |> silu}
// MXFP4 uses power-of-two (E8M0) block scales, which waste up to half of the E2M1// range, so its worst case is 0.25 x block max: about 0.28 here, enough to fail// the 0.25 contract that NVFP4 (E4M3 scales, 16-wide blocks) meets.// Try adding `@accuracy(rel <= 0.25)` below: the compiler rejects it (V0301).// OCP scaling clamps the largest values; overflow-aware scaling (mxfp4_oas) never// clamps. Same worst case, but OAS avoids the systematic bias on the biggest values.fn mlp_mxfp4(x: Tensor<[1, 64], f32, abs<=1.0>, w: Tensor<[64, 32], f32, abs<=0.5>) -> Tensor<[1, 32], f32> { return (quant(x, bf16) @ quant(w, mxfp4)) |> silu}
fn mlp_mxfp4_oas(x: Tensor<[1, 64], f32, abs<=1.0>, w: Tensor<[64, 32], f32, abs<=0.5>) -> Tensor<[1, 32], f32> { return (quant(x, bf16) @ quant(w, mxfp4_oas)) |> silu}
fn main() { let x = rand_uniform([1, 64], -1.0, 1.0, 7) let w = rand_uniform([64, 32], -0.5, 0.5, 42) let a = mlp_nvfp4(x, w) let b = mlp_int8(x, w) let c = mlp_mxfp4(x, w) let d = mlp_mxfp4_oas(x, w) report(a) report(b) report(c) report(d)}Inferred types
Section titled “Inferred types”$ veer types examples/quant_mlp.veerfn mlp_nvfp4(x: Tensor<[1, 64], f32, abs<=1, err<=0>, w: Tensor<[64, 32], f32, abs<=0.5, err<=0>) -> Tensor<[1, 32], f32, abs<=32, err<=6.3955>fn mlp_int8(x: Tensor<[1, 64], f32, abs<=1, err<=0>, w: Tensor<[64, 32], f32, abs<=0.5, err<=0>) -> Tensor<[1, 32], f32, abs<=32, err<=0.2768>fn mlp_mxfp4(x: Tensor<[1, 64], f32, abs<=1, err<=0>, w: Tensor<[64, 32], f32, abs<=0.5, err<=0>) -> Tensor<[1, 32], f32, abs<=32, err<=8.9721>fn mlp_mxfp4_oas(x: Tensor<[1, 64], f32, abs<=1, err<=0>, w: Tensor<[64, 32], f32, abs<=0.5, err<=0>) -> Tensor<[1, 32], f32, abs<=32, err<=8.9721>fn main() -> ()Simulated run
Section titled “Simulated run”$ veer run examples/quant_mlp.veer --statsreport `a`: shape [1, 32] fmt f32 | max|x| 2.018 (static ≤ 32) | error 0.2624 (static ≤ 6.3955) ✓ within boundreport `b`: shape [1, 32] fmt f32 | max|x| 2.018 (static ≤ 32) | error 0.0113 (static ≤ 0.2768) ✓ within boundreport `c`: shape [1, 32] fmt f32 | max|x| 2.018 (static ≤ 32) | error 0.5459 (static ≤ 8.9721) ✓ within boundreport `d`: shape [1, 32] fmt f32 | max|x| 2.018 (static ≤ 32) | error 0.2478 (static ≤ 8.9721) ✓ within bound
── engine: sim ── peak KV 0 B planned ≤ 0 B ✓ prefill tokens 0 planned ≤ 0 ✓ decode tokens 0 planned ≤ 0 ✓ reused tokens 0 gen calls 0veer typesprints each function’s inferred bound. NVFP4 is about 0.20 relative, MXFP4 about 0.28, INT8 about 0.009.- Adding
@accuracy(rel <= 0.25)tomlp_mxfp4is a compile error (V0301): power-of-two block scales cost accuracy that E4M3 scales don’t. reportshows measured error next to the bound. The bound is a guarantee over all inputs in range, so it’s larger than typical error.