Standard Input and Output

BufRead

std::io::BufRead is a subtrait of std::io::Read (pub trait BufRead: Read) for input streams that maintain an internal buffer of unread bytes. Because a BufRead implementation can peek at its internal buffer (fill_buf) before advancing its read cursor (consume), it unlocks delimiter-based parsing methods (read_line, read_until, lines, split) that are impossible to implement efficiently on an unbuffered Read.

The Two Required Methods of BufRead: fill_buf and consume

pub trait BufRead: Read {
    fn fill_buf(&mut self) -> Result<&[u8]>;
    fn consume(&mut self, amt: usize);

    // Provided default methods:
    fn has_data_left(&mut self) -> Result<bool> { ... }
    fn read_until(&mut self, byte: u8, buf: &mut Vec<u8>) -> Result<usize> { ... }
    fn skip_until(&mut self, byte: u8) -> Result<usize> { ... }
    fn read_line(&mut self, buf: &mut String) -> Result<usize> { ... }
    fn split(self, byte: u8) -> Split<Self> where Self: Sized { ... }
    fn lines(self) -> Lines<Self> where Self: Sized { ... }
}

How `fill_buf()` and `consume(amt)` Enable Zero-Copy Lookahead:

  Internal Buffer:  [ 'G', 'E', 'T', ' ', '/', 'a', 'p', 'i' ]
                      ^~~~~~~~~~~~~~~~^
  1. `let available: &[u8] = reader.fill_buf()?;`
     (Returns a slice of all unread bytes currently in buffer; does NOT advance cursor!)

  2. Inspect `available` in place (e.g. find `' '` at index 3).

  3. `reader.consume(4);`
     (Advances the internal cursor past `'G', 'E', 'T', ' '` without copying bytes!)

Types That Implement BufRead Out of the Box

Type Why It Implements BufRead
BufReader<R> (where R: Read) Wraps any File, TcpStream, or ChildStdout in an 8 KiB heap buffer
StdinLock<'_> (io::stdin().lock()) Exposes the global Stdin internal BufReader directly
&[u8] (Byte slice!) A byte slice is already an in-memory buffer! Calling fill_buf() returns the slice itself, and consume(n) advances &self[n..]!
Cursor<T> (where T: AsRef<[u8]>) Wraps Vec<u8>, &[u8], or String with a seekable position cursor

Zero-Copy Parsing Using fill_buf, consume, and read_until

use std::io::{self, BufRead};

fn main() -> io::Result<()> {
    // A plain byte slice `&[u8]` implements `BufRead` directly!
    let mut stream: &[u8] = b"TOKEN_A|TOKEN_B|REMAINDER";

    // 1. Read up to delimiter `b'|'` into a byte vector:
    let mut first_token = Vec::new();
    stream.read_until(b'|', &mut first_token)?;
    println!(
        "read_until '|': {:?}",
        std::str::from_utf8(&first_token).unwrap()
    );

    // 2. Skip past the next `b'|'` without allocating a buffer (`skip_until`, Rust 1.80+):
    let skipped = stream.skip_until(b'|')?;
    println!("skipped {} bytes up to second '|'", skipped);

    // 3. Peek at the remaining buffer with `fill_buf()` and `consume()`:
    let peeked = stream.fill_buf()?;
    println!("peeked remainder: {:?}", std::str::from_utf8(peeked).unwrap());
    let len = peeked.len();
    stream.consume(len);

    println!("has_data_left after consume: {}", stream.has_data_left()?);
    Ok(())
}

read_until '|': "TOKEN_A|"
skipped 8 bytes up to second '|'
peeked remainder: "REMAINDER"
has_data_left after consume: false