Basic Network Protocols — Rust Lecture Notes

Because TCP provides an unstructured byte stream without built-in message boundaries, two writes (write_all(b"msg1") and write_all(b"msg2")) may arrive at the receiver merged into a single read() call (b"msg1msg2") or split across multiple read() calls (b"ms", b"g1m", b"sg2"). An application-layer network protocol defines the framing rules that allow a receiver to reconstruct exact message boundaries from a byte stream.

Sender calls:   write("HELLO")  then  write("WORLD")
TCP Stream:     [ H | E | L | L | O | W | O | R | L | D ] (No boundaries!)

Solution 1 — Delimiter Framing (Text Protocols: HTTP/1.1 headers, Redis RESP, SMTP):
  [ H | E | L | L | O | \n | W | O | R | L | D | \n ]

Solution 2 — Length-Prefixed Framing (Binary Protocols: HTTP/2, gRPC, PostgreSQL):
  [ 0x00, 0x00, 0x00, 0x05 | H | E | L | L | O | 0x00, 0x00, 0x00, 0x05 | W | O | R | L | D ]
  |<-- u32 big-endian = 5 ->|<--- 5 bytes ---->|

Comparing Common Framing Strategies

Framing Strategy	Mechanism	Advantages	Pitfalls to Guard Against
Length-Prefixed	Fixed-size big-endian length header (e.g., `u32::to_be_bytes`) followed by `N` payload bytes.	Handles arbitrary binary data; exact buffer allocation via `read_exact`.	Always cap maximum allowed `N` before allocating `vec![0u8; N]` to prevent OOM attacks.
Delimiter-Terminated	Messages end with `\n` or `\r\n` read via `BufRead::read_line` or `read_until`.	Human-readable; easy to debug with `nc` / `telnet`.	Payload cannot contain raw unescaped delimiter; must bound line length via `.take(MAX)`.
Close-Delimited	Sender transmits payload and closes write half (`Shutdown::Write`).	Zero framing overhead.	Only supports a single message per direction per connection.

Implementing a Length-Prefixed Binary Frame Protocol

Network protocols standardize on big-endian (network byte order) integers via .to_be_bytes() and from_be_bytes(), combined with Read::read_exact and Write::write_all:

use std::io::{self, ErrorKind, Read, Write};
use std::net::{TcpListener, TcpStream};

const MAX_FRAME_SIZE: usize = 64 * 1024; // 64 KiB safety limit

fn write_frame(mut writer: impl Write, payload: &[u8]) -> io::Result<()> {
    let len = u32::try_from(payload.len())
        .map_err(|_| io::Error::new(ErrorKind::InvalidInput, "frame exceeds u32::MAX"))?;
    writer.write_all(&len.to_be_bytes())?;
    writer.write_all(payload)?;
    writer.flush()
}

fn read_frame(mut reader: impl Read) -> io::Result<Option<Vec<u8>>> {
    let mut len_bytes = [0u8; 4];
    match reader.read_exact(&mut len_bytes) {
        Ok(()) => {}
        Err(e) if e.kind() == ErrorKind::UnexpectedEof => return Ok(None),
        Err(e) => return Err(e),
    }

    let len = u32::from_be_bytes(len_bytes) as usize;
    if len > MAX_FRAME_SIZE {
        return Err(io::Error::new(
            ErrorKind::InvalidData,
            format!("frame length {len} exceeds limit {MAX_FRAME_SIZE}"),
        ));
    }

    let mut payload = vec![0u8; len];
    reader.read_exact(&mut payload)?;
    Ok(Some(payload))
}

fn main() -> io::Result<()> {
    let listener = TcpListener::bind("127.0.0.1:0")?;
    let addr = listener.local_addr()?;

    let mut client = TcpStream::connect(addr)?;
    let (mut server, _) = listener.accept()?;

    // Send two distinct binary frames back-to-back over the same TCP stream
    write_frame(&mut client, b"First Frame")?;
    write_frame(&mut client, b"Second Frame with \n newline inside")?;
    drop(client);

    // Server reconstructs exact frame boundaries despite TCP coalescing
    while let Some(frame) = read_frame(&mut server)? {
        println!("Decoded frame: {:?}", String::from_utf8_lossy(&frame));
    }

    Ok(())
}

Decoded frame: "First Frame"
Decoded frame: "Second Frame with \n newline inside"