Pinning and Async State Machines

The primary motivation for Pin<P> in the Rust standard library is supporting zero-allocation async / .await state machines. When an async fn or async { ... } block holds a local reference across an .await suspension point, the compiler transforms the block into a self-referential state-machine enum that implements std::future::Future:

pub trait Future {
    type Output;
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

Why async Blocks Become Self-Referential Across .await

Consider an async block that declares a local array buf and holds a slice reference slice = &buf[..] across an .await point:

async fn example() -> u8 {
    let buf = [10_u8, 20, 30, 40];
    let slice: &[u8] = &buf[1..3];
    std::future::ready(()).await;
    slice[0] + slice[1]
}

Because example() can be suspended at .await and resumed later by an executor, all local variables live across the .await point (buf and slice) must be stored inside the returned Future state machine struct on the stack or heap rather than in an ephemeral CPU call frame:

Compiler-Generated Future State Machine (Paused at `.await`):
+-------------------------------------------------------------------+
| State::Suspended1 {                                               |
|     buf:   [10, 20, 30, 40],  <---+ (Stored at offset +4)         |
|     slice: (ptr, len = 2),    ----+ (`ptr` points into `self.buf`)|
|     child_fut: Ready(()),                                         |
| }                                                                 |
+-------------------------------------------------------------------+

1. Before the first Future::poll call: The future is in State::Unresumed; slice has not been created yet, so the unpolled future can be moved freely (into tokio::spawn, Box::pin, or pin!).
2. During the first Future::poll(self: Pin<&mut Self>, cx) call: The state machine initializes buf and sets slice to point to self.buf's memory address. Because poll requires self: Pin<&mut Self>, the future is now pinned and guaranteed never to move before completion or Drop!

Polling a Pinned Future Manually

Attempting to call .poll() on an unpinned async block fails because async blocks are !Unpin. Pinning the future on the stack with std::pin::pin! (or on the heap with Box::pin) satisfies self: Pin<&mut Self>:

use std::future::Future;
use std::pin::pin;
use std::task::{Context, Poll, Waker};

async fn sum_across_await() -> i32 {
    let numbers = [10, 20, 30];
    let borrowed_slice: &[i32] = &numbers[..];
    // Yield once through an immediately ready future while `borrowed_slice`
    // references `numbers` inside the state machine:
    std::future::ready(()).await;
    borrowed_slice.iter().sum()
}

fn main() {
    // Before polling, we can move the unpinned future freely:
    let unpinned_fut = sum_across_await();
    let moved_fut = unpinned_fut;

    // Pin the future on the stack before calling `.poll()`:
    let mut pinned_fut = pin!(moved_fut);
    let mut cx = Context::from_waker(Waker::noop());

    match pinned_fut.as_mut().poll(&mut cx) {
        Poll::Ready(total) => println!("future resolved to: {}", total),
        Poll::Pending => println!("future still pending"),
    }
}

future resolved to: 60

Polling a Generic Future Stored in a Struct (F: Future + Unpin vs. Pin<Box<dyn Future>>)

When writing a combinator or stream wrapper that polls an inner future inside &mut self, you have two choices:

1. Require F: Future + Unpin so Pin::new(&mut self.fut).poll(cx) works without pinning the outer struct (callers pass Box::pin(async { ... }) or &mut pinned_fut, both of which implement Future + Unpin).
2. Keep F: Future without Unpin and project structural pinning from self: Pin<&mut Self> to Pin<&mut F>.