Strings and Text Processing

String Slicing and UTF-8 Boundaries

Slicing a String or &str with a byte range (&s[start..end]) borrows a contiguous &str subslice in $O(1)$ time, provided both start and end lie on valid UTF-8 character boundaries and within 0..=s.len().

What Is a UTF-8 Character Boundary?

A byte index i in a string s is a valid UTF-8 character boundary if and only if:

  • i == 0 (the start of the string),
  • i == s.len() (one past the final byte), or
  • s.as_bytes()[i] is the first byte of a UTF-8 code point (i.e., not a UTF-8 continuation byte matching 0b10xxxxxx, or 0x80..=0xBF).
String: "aé" (3 bytes: [0x61, 0xC3, 0xA9])

Byte index:  0       1       2       3
             │  'a'  │     'é'       │
             ▼       ▼       ✕       ▼
Boundary?   Yes     Yes     NO!     Yes
                           (mid-'é')

Panicking vs. Non-Panicking String Slicing

If you slice a string with &s[start..end] where start or end splits the middle of a multi-byte character (or is out of bounds), the program panics at runtime. To slice fallibly without panicking, use s.get(start..end), which returns Option<&str> (None if either endpoint is out of bounds or not on a character boundary):

Expression Invalid Boundary / Out-of-Bounds Behavior Return Type
&s[start..end] Panics at runtime &str
s.get(start..end) Returns None Option<&str>
s.split_at_checked(mid) Returns None Option<(&str, &str)>
s.is_char_boundary(idx) Returns false bool
fn main() {
    let word = "café"; // 'c' (0..1), 'a' (1..2), 'f' (2..3), 'é' (3..5)

    // Valid character boundaries:
    println!("&word[0..3] = {:?}", &word[0..3]);
    println!("&word[3..5] = {:?}", &word[3..5]);

    // Byte offset 4 is in the middle of 'é' (bytes 3..5):
    println!("is_char_boundary(3): {}", word.is_char_boundary(3));
    println!("is_char_boundary(4): {}", word.is_char_boundary(4));

    // Safe fallible slicing with `.get()` and `.split_at_checked()`:
    println!("word.get(0..4):            {:?}", word.get(0..4));
    println!("word.get(0..5):            {:?}", word.get(0..5));
    println!("word.split_at_checked(4):  {:?}", word.split_at_checked(4));
}
&word[0..3] = "caf"
&word[3..5] = "é"
is_char_boundary(3): true
is_char_boundary(4): false
word.get(0..4):            None
word.get(0..5):            Some("café")
word.split_at_checked(4):  None

Finding the Nearest Valid UTF-8 Boundary (floor_char_boundary and ceil_char_boundary)

When truncating a string to fit a fixed byte budget (such as a 64-byte database column or packet field), s.floor_char_boundary(max_bytes) and s.ceil_char_boundary(index) adjust an arbitrary byte index down or up to the nearest valid UTF-8 character boundary:

fn truncate_to_byte_budget(s: &str, max_bytes: usize) -> &str {
    let end = s.floor_char_boundary(max_bytes);
    &s[..end]
}

fn main() {
    let text = "Hello 🦀!"; // "Hello " is 6 bytes, '🦀' is bytes 6..10
    // A budget of 8 bytes lands inside '🦀', so `floor_char_boundary(8)` rounds down to 6:
    println!("Truncated to <= 8 bytes: {:?}", truncate_to_byte_budget(text, 8));
}
Truncated to <= 8 bytes: "Hello "