Decoder panics with 'attempt to subtract with overflow' (overflow-checks) when an instruction straddles a 4 GiB address boundary — deterministic reproducer for #383

[justadudewithtime]

Summary

The Decoder panics with attempt to subtract with overflow under any profile with overflow-checks = true (the default for dev/test) when one instruction's byte span straddles a 4 GiB virtual-address boundary inside a large buffer. Reopening #383 with the mechanism and a deterministic reproducer — that report was closed not-reproducible because the panic depends only on where the allocator places the buffer, not on the input bytes.

Root cause

iced-x86 1.21.0, src/decoder.rs (current published layout):

• line 1421: let instr_len = self.data_ptr as u32 - data_ptr as u32;
• line 1468 (the IS_INVALID twin): same expression

data_ptr is a usize holding a raw address. Truncating both operands to u32 and subtracting is exact under wrapping semantics (the true difference is at most MAX_INSTRUCTION_LENGTH, so mod-2^32 arithmetic returns it), but it panics under checked arithmetic whenever instr_start sits below a multiple of 2^32 and data_ptr ends above it:

instr_start low32 = 0xFFFF_FFFB
data_ptr    low32 = 0x0000_0002
0x2u32 - 0xFFFF_FFFBu32  ->  attempt to subtract with overflow

Why it looks non-deterministic

Only the mapping's address layout matters. A 178 MiB read-only mmap straddles a 4 GiB boundary on roughly 4% of fresh placements (Linux x86-64, ASLR), so a test suite that decodes out of large mmaps hits it sporadically — a loop over small heap-allocated inputs can never hit it, which is also why fuzzing cannot reach it: an input would need a > 4 GiB backing allocation to place bytes across a boundary.

Reproducer (deterministic, safe Rust, no dependencies beyond iced-x86)

Allocate consecutive 64 MiB zeroed Vec<u8>s (the allocator serves each with its own mapping); among 65 live mappings one straddles a 4 GiB boundary by pigeonhole. Write a multi-byte instruction so it ends exactly at the boundary, decode it, and compare against the same bytes decoded in an ordinary small buffer:

use iced_x86::{Decoder, DecoderOptions, Instruction};

#[test]
fn straddle() {
    const CHUNK: usize = 64 << 20;
    let mut keep: Vec<Vec<u8>> = Vec::new();
    for _ in 0..80 {
        let mut chunk = vec![0u8; CHUNK];
        let base = chunk.as_ptr() as usize;
        let off = 0x1_0000_0000usize.wrapping_sub(base & 0xFFFF_FFFF);
        if !(15..CHUNK - 16).contains(&off) { keep.push(chunk); continue; }
        // `mov rax,[rip+0x10]`, 7 bytes: starts 6 below the boundary,
        // last byte is the first byte above it
        chunk[off - 6..off + 1].copy_from_slice(&[0x48, 0x8B, 0x05, 0x10, 0x00, 0x00, 0x00]);
        let mut d = Decoder::try_with_ip(64, &chunk[off - 14..off + 23], 0x1400_1000, DecoderOptions::NONE).unwrap();
        d.set_position(8).unwrap();
        let mut insn = Instruction::default();
        d.decode_out(&mut insn);
        assert_eq!(insn.len(), 7); // panics inside decoder.rs:1421 before the fix
        return;
    }
    panic!("allocator defeated the premise");
}

Run with plain cargo test (dev profile, overflow-checks on). On 1.21.0 it panics:

thread 'straddle' panicked at ~/.cargo/registry/src/.../iced-x86-1.21.0/src/decoder.rs:1421:25:
attempt to subtract with overflow

Suggested fix

The operands are ordered (data_ptr >= instr_start in both places, per the constructor's own verification comment), so a usize subtraction cannot wrap:

let instr_len = self.data_ptr.wrapping_sub(data_ptr) as u32;

(or (self.data_ptr - data_ptr) as u32 — the checked form is fine if the invariant holds on every path). Same change at both sites. This also keeps debug_assertions builds clean.

Environment

• iced-x86 1.21.0 (features decoder, std; observed with more features enabled as well), rustc stable, Linux x86-64
• Originally observed in production-shaped runs decoding out of a 178 MiB mmap of a PE file: sporadic, ~4% of process launches, always at decoder.rs:1421:25

Happy to send a PR if the maintainers prefer.