Message Passing

Message Passing vs. Shared State

Message passing (mpsc channels / actor tasks) and shared state (Arc<Mutex<T>>, RwLock<T>, Atomic*) are dual concurrency primitives: you can implement a channel using a Mutex<VecDeque<T>> + Condvar, and you can implement shared state using an actor thread that owns T and processes commands over a channel. Choosing between them depends on data ownership topology, latency vs. decoupling, and contention patterns.

Side-by-Side Architectural Comparison

Dimension Message Passing (mpsc / Actor Model) Shared State (Arc<Mutex<T>> / RwLock<T> / Atomic*)
Data ownership Single owner at all times (transfers ownership or routes to 1 actor thread) Shared ownership across $N$ threads simultaneously
Locking & Deadlocks No multi-lock ordering needed; lock-free enqueue on unbounded channels Requires disciplined lock ordering if acquiring $>1$ lock
Cancellation & Shutdown Built-in: dropping all Senders (or Receiver) propagates EOF automatically Manual: requires a separate AtomicBool or Condvar shutdown flag
Read Latency Higher (requires sending a request message + context switch to actor thread + awaiting reply) Lowest (RwLock::read or Mutex::lock reads in place on the calling thread in nanoseconds)
Memory Overhead Each queued message occupies buffer memory until processed Constant memory (single in-place instance of T)
Panic Isolation If actor panics, channel returns SendError / RecvError cleanly If writer panics while holding lock, Mutex / RwLock becomes poisoned 
When to Choose Message Passing vs. Shared State:

  Use Message Passing When:
    - Work flows through stages (Producer -> Worker -> Sink)
    - One dedicated thread owns an external resource (Socket, SQLite conn, UI loop)
    - You want automatic backpressure (`sync_channel`) and EOF shutdown cascading

  Use Shared State When:
    - Many threads need low-latency in-place reads/updates on a shared cache or index
    - State is a simple counter/flag (`AtomicU64`, `AtomicBool`) or write-once config (`OnceLock`)
    - Copying or context-switching to an actor thread per lookup is too expensive

Hybrid Architecture: Combining Both in Real Systems

Production Rust systems routinely combine both models rather than treating them as mutually exclusive:

  • Shared state (Arc<RwLock<RoutingTable>>, OnceLock<Config>, AtomicU64) is used on the hot request path so any worker thread can read configuration, routing rules, and increment counters in nanoseconds without waiting on an actor queue.
  • Message passing (mpsc::sync_channel) is used to offload asynchronous background jobs, batch disk writes, and coordinate graceful shutdown across worker threads:
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::mpsc;
use std::thread;

struct AppTelemetry {
    // 1. Shared-state atomic counter for zero-wait inline reads/increments:
    total_requests: AtomicU64,
    // 2. Message-passing channel for offloading slow audit-log formatting:
    audit_tx: mpsc::Sender<String>,
}

fn main() {
    let (audit_tx, audit_rx) = mpsc::channel::<String>();
    let telemetry = AppTelemetry {
        total_requests: AtomicU64::new(0),
        audit_tx,
    };

    // Background audit-log consumer thread:
    let logger_thread = thread::spawn(move || {
        let mut logged = Vec::new();
        for entry in audit_rx {
            logged.push(entry);
        }
        logged
    });

    thread::scope(|s| {
        for worker in 0..3 {
            let tel = &telemetry;
            s.spawn(move || {
                let seq = tel.total_requests.fetch_add(1, Ordering::Relaxed) + 1;
                tel.audit_tx
                    .send(format!("worker {worker} handled request (seq={seq})"))
                    .unwrap();
            });
        }
    });

    // Drop `telemetry` (and its `audit_tx`) so `logger_thread` finishes:
    let final_count = telemetry.total_requests.load(Ordering::Relaxed);
    drop(telemetry);

    let logs = logger_thread.join().unwrap();
    println!("total_requests (atomic): {}", final_count);
    println!("audit logs collected:    {} entries", logs.len());
}

total_requests (atomic): 3
audit logs collected:    3 entries