Generics

Generic Code and Performance

Rust's generic system is designed around the zero-cost abstraction principle: generic code compiles to the same—or better—machine code as hand-written type-specific code, with no runtime overhead for type erasure or boxing. At the same time, how you structure generic code directly affects instruction cache locality, binary size, and compile times.

Why Generic Code Runs at Native Speed

When the compiler monomorphizes a generic function for a concrete type T, three optimizations happen automatically during LLVM code generation:

  1. Unboxed inline storage: Vec<Point<f32>> stores [x, y, x, y, ...] flat in a single contiguous heap allocation rather than an array of heap pointers to boxed objects.
  2. Inlining of trait methods: Calls such as a + b or item.cmp(&other) inside a generic loop are resolved statically to the concrete type's implementation and inlined directly into the loop body.
  3. Auto-vectorization (SIMD): Once small trait methods (Add, Copy, Ord) are inlined into a tight loop over a contiguous slice, LLVM can vectorize the loop using CPU SIMD instructions.
use std::ops::{AddAssign, Mul};

fn dot_product<T>(xs: &[T], ys: &[T]) -> T
where
    T: Copy + Default + Mul<Output = T> + AddAssign,
{
    assert_eq!(xs.len(), ys.len());
    let mut acc = T::default();
    for (&x, &y) in xs.iter().zip(ys.iter()) {
        acc += x * y;
    }
    acc
}

fn main() {
    let a = [1.0_f64, 2.0, 3.0, 4.0];
    let b = [10.0_f64, 20.0, 30.0, 40.0];

    println!("dot_product(f64) = {}", dot_product(&a, &b));

    let ia = [1_i32, 2, 3, 4];
    let ib = [10_i32, 20, 30, 40];
    println!("dot_product(i32) = {}", dot_product(&ia, &ib));
}

dot_product(f64) = 300
dot_product(i32) = 300

Memory Layout: Contiguous Generic Storage vs. Boxed Indirection

Contiguous Generic Storage: Vec<Pair<u32>>
Stack Header                Single Contiguous Heap Allocation
+-------------------+       +-------+-------+-------+-------+-------+-------+
| ptr | len=3 | cap | ----> | a0 b0 | a1 b1 | a2 b2 | (Sequential L1 cache) |
+-------------------+       +-------+-------+-------+-------+-------+-------+

Pointer-Erased Storage (Java/Python or Vec<Box<dyn Any>>)
Stack Header                Array of Pointers       Scattered Heap Objects
+-------------------+       +-----+-----+-----+     +-------+   +-------+
| ptr | len=3 | cap | ----> | p0  | p1  | p2  | --> | a0 b0 |   | a1 b1 |
+-------------------+       +-----+-----+-----+     +-------+   +-------+

Instruction Cache Pressure and Binary Bloat

While monomorphization maximizes speed for hot inner loops, instantiating large generic functions across dozens of concrete types can increase binary size and evict hot instructions from the CPU's L1 instruction cache (i-cache):

Scenario Performance Impact Recommended Technique
Tight numerical or iterator loop (map, filter, fold) Inlining + SIMD yields major speedup Keep generic (impl Iterator<Item = T>)
Large I/O, parsing, or logging function taking impl AsRef<str> Conversion is tiny; body is huge and I/O-bound Extract a non-generic inner function taking &str
Cold error-formatting path inside a generic function Inflates every monomorphized caller Mark error helper #[cold] and #[inline(never)]

Outlining Cold Paths from Generic Functions

When a generic function contains a verbose error-reporting or fallback branch, extracting that branch into a non-generic (or #[cold]) helper keeps the monomorphized hot path compact so LLVM is more likely to inline it:

#[cold]
#[inline(never)]
fn panic_index_out_of_bounds(index: usize, len: usize) -> ! {
    panic!("index {index} out of bounds for slice of length {len}");
}

pub fn get_checked_or_panic<T>(slice: &[T], index: usize) -> &T {
    match slice.get(index) {
        Some(value) => value,
        None => panic_index_out_of_bounds(index, slice.len()),
    }
}

fn main() {
    let items = ["alpha", "beta", "gamma"];
    println!("Item 1: {}", get_checked_or_panic(&items, 1));
}

Item 1: beta