Range starting with `Bound::Excluded(&vec![])` causes `BadValSize` error (aka `MDB_BAD_VALSIZE`)

[jasonaowen]

Trying to take a range starting with Bound::Excluded(&vec![]) used to work, but no longer does.

Example code that worked in v0.11:

[dependencies]
heed = { version = "0.11", default-features = false, features = ["lmdb"] }
mktemp = "0.5.1"

fn main() {
    let temp_file = mktemp::Temp::new_dir().unwrap();
    let env = heed::EnvOpenOptions::new().open(temp_file).unwrap();
    let db: heed::UntypedDatabase = env.create_database(None).unwrap();

    let txn = env.read_txn().unwrap();
    let iter = db
        .range(
            &txn,
            &(
                std::ops::Bound::Excluded::<&[u8]>(&vec![]),
                std::ops::Bound::Unbounded::<&[u8]>,
            ),
        )
        .unwrap();
    for _ in iter {
        assert!(false, "This is unreachable");
    }
}

And example code for v0.22, showing the error:

[dependencies]
heed = { version = "0.22", default-features = false }
mktemp = "0.5.1"

use heed::types::Bytes;

fn main() {
    let temp_file = mktemp::Temp::new_dir().unwrap();
    let env = unsafe { heed::EnvOpenOptions::new().open(temp_file).unwrap() };
    let mut wtxn = env.write_txn().unwrap();
    let db: heed::Database<Bytes, Bytes> = env.create_database(&mut wtxn, None).unwrap();

    let txn = env.read_txn().unwrap();
    let iter = db
        .range(
            &txn,
            &(
                std::ops::Bound::Excluded::<&[u8]>(&vec![]),
                std::ops::Bound::Unbounded::<&[u8]>,
            ),
        )
        .unwrap();
    for i in iter {
        println!("This should be unreachable, but is an error: {i:?}");
        if let Err(heed::Error::Mdb(e)) = i {
            assert_eq!(e, heed::MdbError::BadValSize);
        } else {
            assert!(false, "This is unreachable");
        }
    }
}

This appears to be related to PR #212. In particular, this removed code:

heed/heed/src/iterator/range.rs

Lines 35 to 37 in 26bf088

	Bound::Excluded(start) => {
	advance_key(start);
	let result = cursor.move_on_key_greater_than_or_equal_to(start);

heed/heed/src/iterator/mod.rs

Lines 9 to 11 in 26bf088

	fn advance_key(bytes: &mut Vec<u8>) {
	match bytes.last_mut() {
	Some(&mut 255) | None => bytes.push(0),

Passing an empty Vec<u8> in a Bound::Excluded was explicitly handled, but no longer is. Should the old behavior still work?

[Kerollmops]

Hey @jasonaowen 👋

I am wondering what we should do about this because LMDB does not support empty keys by design, so it doesn't make sense to exclude them from a range. I am thinking about two ways of fixing this problem:

• Either on your side, you redefine your bounds to something like Bound::Unbounded.
• Or on Meilisearch's side, we handle that explicitly and redefine bounds too.

In your case, I would say that Bound::Unbounded on both sides would give the same result and would be valid. Am I right?

I tend to prefer the former solution, as it lets people get what they would receive from LMDB itself without hiding its behavior. We would benefit from an update to some part of the documentation, such as adding a downside section on the main page. Feel free to open a PR to improve the documentation.

Have a nice day 🌵