Rust's Compilation Pipeline — Rust Lecture Notes

When rustc compiles a crate from .rs source files into a native executable or library, it lowers the program through a sequence of intermediate representations (IRs). Each representation strips away surface syntax and makes control flow, types, lifetimes, and machine operations progressively more explicit.

End-to-End Pipeline Diagram

  .rs Source Files (UTF-8 text)
         |
         |  1. Lexing & Parsing
         v
  Token Stream & Unexpanded AST
         |
         |  2. Macro Expansion & Name Resolution (`macro_rules!`, proc macros, `#[cfg]`)
         v
  Expanded AST (Abstract Syntax Tree)
         |
         |  3. AST Lowering & Desugaring (`for` -> `loop`, `?` -> `match`, `async` -> state machine)
         v
  HIR (High-Level Intermediate Representation)
         |
         |  4. Type Inference, Trait Resolution & Type Checking (`THIR` -> `MIR` construction)
         v
  MIR (Mid-Level Intermediate Representation — Control-Flow Graph)
         |
         |  5. Borrow Checking, Move/Init Analysis, Drop Elaboration & Const Evaluation
         v
  Optimized & Monomorphized MIR (Generic `<T>` stamped out into concrete types)
         |
         |  6. Codegen Lowering (`rustc_codegen_llvm`)
         v
  LLVM IR (.ll / .bc)
         |
         |  7. LLVM Optimization Passes & Target Instruction Selection
         v
  Object Files (.o) + Static Archives (.rlib)
         |
         |  8. Linking (`cc` / `ld` / `lld`)
         v
  Native Binary Executable or Library

Summary of Compilation Stages

Stage	Representation	Key Responsibilities	Typical Errors Caught Here
1. Lexing & Parsing	Token Stream / AST	Converts UTF-8 characters into tokens and builds a syntax tree	Unclosed delimiters, invalid tokens, missing semicolons
2. Expansion & Name Resolution	Expanded AST	Evaluates `#[cfg(...)]`, expands declarative and procedural macros, resolves imports (`use`)	Unknown macros, unresolved `use` paths (`E0432`), recursive macro limits
3. Lowering to HIR	HIR (High-Level IR)	Desugars surface syntax (`for`, `while`, `if let`, `?`, `async`) into core constructs	Invalid item placements, lifetime elision mismatches
4. Type & Trait Checking	HIR / THIR (Typed HIR)	Hindley-Milner local type inference, method lookup, trait bound verification, exhaustiveness checking	Type mismatches (`E0308`), missing trait implementations (`E0277`), non-exhaustive `match` (`E0004`)
5. MIR Analysis & Borrow Check	MIR (Mid-Level IR)	Builds a basic-block control-flow graph; runs borrow checker, definite initialization, and inserts `Drop` calls	Use after move (`E0382`), conflicting borrows (`E0502`), uninitialized bindings (`E0381`), dangling references (`E0597`)
6. Monomorphization & Codegen	LLVM IR	Instantiates generic functions for concrete types and translates MIR into LLVM instructions	Linker symbol collisions, target feature mismatches
7–8. LLVM & Linking	`.o` -> Native Binary	Runs hardware optimizations (inlining, SIMD) and links with `std` and system libraries	Missing external C libraries (`linking with cc failed`)

What Happens During Desugaring (AST to HIR)

Surface Rust provides ergonomic syntax like for loops, the ? error-propagation operator, and compound assignments (+=). Before type checking and borrow checking even begin, rustc desugars these constructs into simpler primitives in HIR so the rest of the compiler only needs to reason about a small core grammar:

for x in iter { body } becomes a loop calling IntoIterator::into_iter(iter) and matching Iterator::next(&mut iter).
while cond { body } becomes loop { if cond { body } else { break; } }.
expr? becomes a match expression that unwraps Ok(val) / Some(val) or returns early on Err(e) / None.

Why MIR Is the Heart of Rust's Safety Checks

Prior to 2016, rustc performed borrow checking directly on the AST, which made control-flow analysis complex and limited lifetimes to strict lexical scopes { ... }.

Today, rustc lowers typed HIR into MIR (Mid-Level Intermediate Representation)—a simplified control-flow graph made of basic blocks (sequences of simple assignments with no nested expressions) connected by explicit jumps (goto, switchInt, return, drop):

MIR Basic Block Structure for `if cond { a } else { b }`:

        +-----------------------+
        | bb0:                  |
        |   switchInt(cond)     |
        +----+-------------+----+
             | true        | false
             v             v
      +------------+ +------------+
      | bb1:       | | bb2:       |
      |   _0 = a   | |   _0 = b   |
      |   goto bb3 | |   goto bb3 |
      +-----+------+ +-----+------+
            |              |
            +------+-------+
                   v
            +--------------+
            | bb3:         |
            |   return     |
            +--------------+

Operating on this control-flow graph allows rustc to:

Track Non-Lexical Lifetimes (NLL)—ending a borrow at the exact basic-block statement where the reference is last used rather than waiting for the closing brace }.
Prove definite initialization across complex if/else, match, and loop branches.
Insert exact drop flags and Drop::drop calls only on paths where a variable was initialized and not yet moved.
Execute const functions and const blocks at compile time using the built-in MIR interpreter.

Query-Driven Incremental Compilation

Unlike traditional C++ compilers that process a translation unit strictly top-to-bottom in fixed passes, rustc uses a demand-driven query system. When you run cargo check or cargo build, each analysis step (type_of(item), mir_borrowck(fn), optimized_mir(fn)) is a cached query whose result is hashed on disk inside target/debug/incremental/.

If you edit the body of one function without changing its signature, rustc re-runs the HIR and MIR queries only for that modified function and reuses the cached query results for the rest of the crate.