Rust Performance Engineering

Inlining

Function inlining replaces a function call instruction with a copy of the called function's body at the call site. In Rust, inlining is the foundational catalyst optimization: once a small getter, closure, or trait method is inlined into its caller, LLVM can propagate constants, eliminate dead branches, remove redundant bounds checks, and vectorize the surrounding loop.

Without Cross-Crate Inlining:
  Crate A calls `crate_b::is_ascii_digit(ch)`
  ---> Must spill registers, emit `call` instruction, and jump to `crate_b`!

With `#[inline]` on `crate_b::is_ascii_digit`:
  `rustc` embeds the MIR of `is_ascii_digit` into `libcrate_b.rlib` metadata!
  ---> Crate A inlines `ch.wrapping_sub(b'0') < 10` directly into its register loop!

How Cross-Crate Inlining Works in Rust (#[inline])

Within the same crate, LLVM can automatically inline non-generic functions without any attribute annotations.

However, across different crates (when Crate A calls a function in Crate B, and Link-Time Optimization lto is not enabled), rustc only embeds a function's machine-code/MIR body into Crate B's .rlib metadata if the function is:

  1. Generic (fn foo<T>(...)), or
  2. Annotated with #[inline] (or #[inline(always)]).

If a small public non-generic function in a library crate is not marked #[inline], downstream crates compiling without LTO cannot inline it at all!

Attribute Meaning When to Use
(None) Compiler decides within the current crate; not inlinable across crates (unless generic or LTO is enabled). Default for medium/large functions (> 10–20 lines).
#[inline] Hints to the compiler to inline the function and exports its MIR into .rlib metadata for cross-crate inlining. Small, hot public methods and constructors in library crates (getters, math helpers, iterator .next()).
#[inline(always)] Strongly forces the compiler to inline the function at every call site (even in debug builds). Tiny 1–2 line wrappers (such as monomorphization-thinning thunks) or SIMD intrinsic helpers.
#[inline(never)] Forbids the compiler from inlining the function. Keeping slow error/fallback paths out of the caller's stack frame, or isolating a function in CPU profiler flamegraphs.
#[cold] Marks a function as unlikely to be called (e.g., error formatting or panic paths). Tells LLVM to optimize the caller's non-cold branch prediction and place the cold function in a separate code section!

Keeping Fast Paths Fast by Splitting Cold Error Paths (#[cold])

When a tiny, hot function contains a large error-formatting or logging block (if unlikely_error { format!(...); log!(...); }), the large slow path can make the function look "too big to inline" to LLVM's cost model—preventing the 2-line fast path from being inlined!

Splitting the slow path into a separate #[cold] #[inline(never)] helper makes the fast-path function tiny and trivially inlinable:

#[derive(Debug, PartialEq, Eq)]
pub struct CapacityError {
    pub message: String,
}

/// Fast-path function: tiny body is trivially inlined into callers!
#[inline]
pub fn check_capacity(requested: usize, limit: usize) -> Result<usize, CapacityError> {
    if requested <= limit {
        Ok(limit - requested)
    } else {
        Err(make_capacity_error(requested, limit))
    }
}

/// Cold error path: kept out of the caller's instruction cache and stack frame!
#[cold]
#[inline(never)]
fn make_capacity_error(requested: usize, limit: usize) -> CapacityError {
    CapacityError {
        message: format!(
            "requested capacity ({requested} bytes) exceeds maximum limit ({limit} bytes)"
        ),
    }
}

fn main() {
    assert_eq!(check_capacity(128, 1024), Ok(896));
    let err = check_capacity(2048, 1024).unwrap_err();
    println!("Cold error path message: {}", err.message);
}

Cold error path message: requested capacity (2048 bytes) exceeds maximum limit (1024 bytes)