File System Operations

Reading Files

Rust provides three primary patterns for reading files, depending on the file's size and structure: whole-file reading (fs::read / fs::read_to_string), buffered streaming (BufReader::new(file)), and random-access / fixed-chunk reading (Read::read_exact + Seek).

Choosing the Right File Reading Strategy

How large is the file and how is it structured?
  |
  +-- Fits comfortably in RAM & needed all at once (configs, JSON, source files):
  |     +-- Text (UTF-8) -------------> `std::fs::read_to_string(path)`
  |     +-- Binary bytes -------------> `std::fs::read(path)`
  |
  +-- Line-oriented or large streaming file (logs, CSV, multi-GB datasets):
  |     +-----------------------------> `BufReader::new(File::open(path)?)` + `read_line`
  |
  +-- Indexed binary file (SQLite pages, ZIP central directory, media headers):
        +-----------------------------> `File::open(path)?` + `Seek::seek` + `read_exact`
Pattern Memory Footprint OS Syscalls Best Use Case
fs::read_to_string(path) $O(\text{file size})$ (1 allocation) Minimal (open, fstat, read, close) Config files, templates, small inputs
fs::read(path) $O(\text{file size})$ (1 allocation) Minimal Images, WASM modules, binary blobs
BufReader::new(file) + read_line $O(\text{longest line})$ (8 KiB buffer) 1 syscall per 8 KiB Multi-gigabyte log files and line streams
file.seek(...) + file.read_exact(...) $O(\text{record size})$ lseek + read per record Reading a specific byte offset without loading the whole file

Random-Access Reading with Seek and read_exact

Every std::fs::File implements std::io::Seek (seek(SeekFrom::Start(pos)), seek(SeekFrom::End(offset)), seek(SeekFrom::Current(delta)), rewind(), and stream_position()). This allows jumping directly to a footer or indexed record inside a large binary file without reading the preceding bytes:

use std::fs::{self, File};
use std::io::{self, Read, Seek, SeekFrom, Write};

fn main() -> io::Result<()> {
    let path = std::env::temp_dir().join("rust_ch50_seek_demo.bin");

    // Write a 16-byte binary file where the last 4 bytes store a u32 footer magic:
    {
        let mut f = File::create(&path)?;
        f.write_all(b"PAYLOAD_1234")?;
        f.write_all(&0xCAFE_BABE_u32.to_be_bytes())?;
    }

    let mut file = File::open(&path)?;

    // Jump directly to 4 bytes before the end of the file (`SeekFrom::End(-4)`):
    let footer_pos = file.seek(SeekFrom::End(-4))?;
    let mut magic_bytes = [0_u8; 4];
    file.read_exact(&mut magic_bytes)?;
    let magic = u32::from_be_bytes(magic_bytes);

    // Rewind back to byte 0 (`file.rewind()`) to read the first 7 bytes:
    file.rewind()?;
    let mut prefix = [0_u8; 7];
    file.read_exact(&mut prefix)?;

    fs::remove_file(&path)?;
    println!(
        "footer at offset {}: {:#010x}, prefix: {:?}",
        footer_pos,
        magic,
        std::str::from_utf8(&prefix).unwrap()
    );
    Ok(())
}

footer at offset 12: 0xcafebabe, prefix: "PAYLOAD"