Buffered Input

Buffered input wraps an unbuffered reader (std::fs::File or std::net::TcpStream) in an in-memory byte array (typically 8 KiB) so that small, frequent read operations are served directly from user-space RAM instead of making an expensive operating-system kernel call (read syscall) per byte or per line.

Unbuffered vs. Buffered Read Cost

Every OS system call (read(fd, ...) on Unix, ReadFile on Windows) switches the CPU from user mode (Ring 3) to kernel mode (Ring 0) and back, costing several hundred nanoseconds. Reading a 1 MiB file 1 byte at a time via an unbuffered File executes 1,048,576 system calls, whereas reading it through an 8 KiB BufReader<File> executes only 128 system calls—often 100x to 500x faster!

Unbuffered `File::read(&mut [u8; 4])` (1 OS Syscall per Small Read):
  [App] ---syscall---> [OS Kernel] (copies 4 bytes)
  [App] ---syscall---> [OS Kernel] (copies 4 bytes)
  [App] ---syscall---> [OS Kernel] (copies 4 bytes)

Buffered `BufReader<File>` (1 OS Syscall Fills 8,192-Byte RAM Buffer):
  [App] <--memcpy 4B-- [BufReader: 8 KiB RAM Buffer] <---(1 syscall per 8 KiB)--- [OS Kernel]
  [App] <--memcpy 4B-- [BufReader: 8 KiB RAM Buffer] (Served in nanoseconds from RAM!)

When to Use Buffered Input vs. Unbuffered Input

Access Pattern	Use `BufReader<R>`?	Why
Line-by-line or token-by-token parsing (`read_line`, `lines`, `split`)	Yes (Required)	Scanning for `\n` byte-by-byte without a buffer would issue 1 syscall per character!
Many small `read_exact` calls (binary headers, integers, msgpack/bincode)	Yes	Amortizes thousands of 2–8 byte field reads into a single `8 KiB` syscall
Reading an entire file into memory at once (`fs::read`, `fs::read_to_string`)	No	`fs::read` queries the file size and reads directly into the target `Vec<u8>` without double-buffering
Reading into a large caller-managed buffer (`>= 8 KiB` chunks)	No	Calling `file.read(&mut large_buf)` directly avoids an extra `memcpy` through `BufReader`

Inspecting `BufReader`'s Internal Buffer State

BufReader exposes its internal capacity (.capacity()), currently buffered unread slice (.buffer()), and underlying reader reference (.get_ref()):

use std::io::{self, BufReader, Cursor, Read};

fn main() -> io::Result<()> {
    let raw_data = b"HEADER:1234567890_PAYLOAD_BYTES";
    let mut reader = BufReader::with_capacity(16, Cursor::new(&raw_data[..]));

    println!(
        "initial -> capacity: {} B, buffered: {} B",
        reader.capacity(),
        reader.buffer().len()
    );

    // Reading 7 bytes triggers a 16-byte fill from the underlying source,
    // returning 7 bytes to `header` and leaving 9 unread bytes in `reader.buffer()`!
    let mut header = [0_u8; 7];
    reader.read_exact(&mut header)?;

    println!(
        "after 7B read -> header: {:?}, still buffered in RAM: {:?} ({} B)",
        std::str::from_utf8(&header).unwrap(),
        std::str::from_utf8(reader.buffer()).unwrap(),
        reader.buffer().len()
    );

    Ok(())
}

initial -> capacity: 16 B, buffered: 0 B
after 7B read -> header: "HEADER:", still buffered in RAM: "123456789" (9 B)

Unbuffered vs. Buffered Read Cost

When to Use Buffered Input vs. Unbuffered Input

Inspecting BufReader's Internal Buffer State

Inspecting `BufReader`'s Internal Buffer State