Improve all simple paths multi-target performance

[lazyhope]

• I ran rustfmt locally
• I have added the tests to cover my changes. (reusing previous test)
• I have updated the documentation accordingly. (not necessary)
• I have read the CONTRIBUTING document.

This PR continues the work from #1488.

Now that multiple-target support for all_simple_paths has been added to petgraph (see all_simple_path_multi and petgraph/petgraph#865), this PR simply update the underlying implementation from rustworkx_core::connectivity::all_simple_paths_multiple_targets to petgraph::algo::all_simple_paths_multi.

This avoids the need to flatten DictMap into a Vec and results in better performance and cleaner code.

[coveralls]

Pull Request Test Coverage Report for Build 19041961653

Details

• 20 of 20 (100.0%) changed or added relevant lines in 1 file are covered.
• 2 unchanged lines in 1 file lost coverage.
• Overall coverage decreased (-0.01%) to 94.175%

---

Files with Coverage Reduction	New Missed Lines	%
rustworkx-core/src/connectivity/all_simple_paths.rs	2	97.14%

Totals
Change from base Build 18781234584:	-0.01%
Covered Lines:	18268
Relevant Lines:	19398

---

💛 - Coveralls

[lazyhope]

Here is a temporary benchmark's output, where you can see an improvement when comparing new and prev:

all_simple_paths_complete_graph/new (petgraph+foldhash)/complete-8
                        time:   [1.0261 ms 1.0277 ms 1.0316 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe
all_simple_paths_complete_graph/prev (rustworkx+foldhash)/complete-8
                        time:   [1.3590 ms 1.3621 ms 1.3698 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
all_simple_paths_complete_graph/new (petgraph+foldhash)/complete-9
                        time:   [8.8866 ms 8.9201 ms 8.9442 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe
all_simple_paths_complete_graph/prev (rustworkx+foldhash)/complete-9
                        time:   [10.011 ms 10.032 ms 10.051 ms]
Found 2 outliers among 10 measurements (20.00%)
  2 (20.00%) high severe
all_simple_paths_complete_graph/new (petgraph+foldhash)/complete-10
                        time:   [77.236 ms 77.866 ms 78.708 ms]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
Benchmarking all_simple_paths_complete_graph/prev (rustworkx+foldhash)/complete-10: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 5.6s or enable flat sampling.
all_simple_paths_complete_graph/prev (rustworkx+foldhash)/complete-10
                        time:   [98.948 ms 101.27 ms 103.94 ms]
Benchmarking all_simple_paths_complete_graph/new (petgraph+foldhash)/complete-11: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 7.3s.
all_simple_paths_complete_graph/new (petgraph+foldhash)/complete-11
                        time:   [708.96 ms 716.83 ms 724.43 ms]
Benchmarking all_simple_paths_complete_graph/prev (rustworkx+foldhash)/complete-11: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 10.6s.
all_simple_paths_complete_graph/prev (rustworkx+foldhash)/complete-11
                        time:   [1.0221 s 1.0333 s 1.0427 s]

all_simple_paths_path_graph/new (petgraph+foldhash)/path-20
                        time:   [3.0907 µs 3.1152 µs 3.1476 µs]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
all_simple_paths_path_graph/prev (rustworkx+foldhash)/path-20
                        time:   [3.9991 µs 4.0371 µs 4.0739 µs]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
all_simple_paths_path_graph/new (petgraph+foldhash)/path-40
                        time:   [4.3007 µs 4.3472 µs 4.4044 µs]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe
all_simple_paths_path_graph/prev (rustworkx+foldhash)/path-40
                        time:   [5.8010 µs 5.8332 µs 5.8575 µs]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
all_simple_paths_path_graph/new (petgraph+foldhash)/path-60
                        time:   [5.5484 µs 5.6040 µs 5.6418 µs]
all_simple_paths_path_graph/prev (rustworkx+foldhash)/path-60
                        time:   [7.4919 µs 7.5250 µs 7.5524 µs]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

all_simple_paths_random_graph/new (petgraph+foldhash)/0.2-12
                        time:   [4.6690 µs 4.6867 µs 4.7145 µs]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe
all_simple_paths_random_graph/prev (rustworkx+foldhash)/0.2-12
                        time:   [6.3301 µs 6.3785 µs 6.4115 µs]
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe
all_simple_paths_random_graph/new (petgraph+foldhash)/0.5-12
                        time:   [12.681 ms 12.729 ms 12.817 ms]
Found 2 outliers among 10 measurements (20.00%)
  1 (10.00%) high mild
  1 (10.00%) high severe
all_simple_paths_random_graph/prev (rustworkx+foldhash)/0.5-12
                        time:   [18.220 ms 18.563 ms 19.203 ms]
Benchmarking all_simple_paths_random_graph/new (petgraph+foldhash)/0.8-12: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 10.0s.
all_simple_paths_random_graph/new (petgraph+foldhash)/0.8-12
                        time:   [1.0109 s 1.0191 s 1.0284 s]
Benchmarking all_simple_paths_random_graph/prev (rustworkx+foldhash)/0.8-12: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 14.6s.
all_simple_paths_random_graph/prev (rustworkx+foldhash)/0.8-12
                        time:   [1.3872 s 1.4034 s 1.4204 s]

The benchmark code for reference:

use criterion::{black_box, criterion_group, criterion_main, BenchmarkId, Criterion};
use foldhash::fast::RandomState;
use hashbrown::HashSet as HashbrownHashSet;
use petgraph::algo;
use petgraph::graph::{NodeIndex, UnGraph};
use rand::prelude::*;
use rustworkx_core::connectivity::all_simple_paths::all_simple_paths_multiple_targets;

fn new(
    graph: &UnGraph<(), ()>,
    from: NodeIndex,
    to: &HashbrownHashSet<NodeIndex, RandomState>,
    min_intermediate_nodes: usize,
    max_intermediate_nodes: Option<usize>,
) -> Vec<Vec<usize>> {
    algo::all_simple_paths_multi::<Vec<_>, _, RandomState>(
        graph,
        from,
        to,
        min_intermediate_nodes,
        max_intermediate_nodes,
    )
    .map(|v: Vec<NodeIndex>| v.into_iter().map(|i| i.index()).collect())
    .collect()
}

fn prev(
    graph: &UnGraph<(), ()>,
    from: NodeIndex,
    to: &HashbrownHashSet<NodeIndex, RandomState>,
    min_intermediate_nodes: usize,
    max_intermediate_nodes: Option<usize>,
) -> Vec<Vec<usize>> {
    all_simple_paths_multiple_targets(graph, from, to, min_intermediate_nodes, max_intermediate_nodes)
        .into_values()
        .flatten()
        .map(|path| path.into_iter().map(|node| node.index()).collect())
        .collect()
}

fn benchmark_runner(c: &mut Criterion) {
    // --- Complete Graph Benchmark (Dense) ---
    let mut group_complete = c.benchmark_group("all_simple_paths_complete_graph");
    group_complete.sample_size(10);

    for n in [8, 9, 10, 11].iter() {
        let mut graph = UnGraph::<(), ()>::new_undirected();
        let nodes: Vec<_> = (0..*n).map(|_| graph.add_node(())).collect();
        for i in 0..*n {
            for j in (i + 1)..*n {
                graph.add_edge(nodes[i], nodes[j], ());
            }
        }
        let from = nodes[0];
        let to_nodes = [nodes[n - 2], nodes[n - 1]];
        let to_set: HashbrownHashSet<_, RandomState> = to_nodes.iter().cloned().collect();
        let cutoff = Some(n - 2);

        run_benchmarks(&mut group_complete, "complete", *n, graph, from, to_set, cutoff);
    }
    group_complete.finish();

    // --- Path Graph Benchmark (Sparse) ---
    let mut group_path = c.benchmark_group("all_simple_paths_path_graph");
    group_path.sample_size(10);

    for n in [20, 40, 60].iter() {
        let mut graph = UnGraph::<(), ()>::new_undirected();
        let nodes: Vec<_> = (0..*n).map(|_| graph.add_node(())).collect();
        for i in 0..(n - 1) {
            graph.add_edge(nodes[i], nodes[i + 1], ());
        }
        let from = nodes[0];
        let to_nodes = [nodes[n - 2], nodes[n - 1]];
        let to_set: HashbrownHashSet<_, RandomState> = to_nodes.iter().cloned().collect();
        let cutoff = Some(n - 2);

        run_benchmarks(&mut group_path, "path", *n, graph, from, to_set, cutoff);
    }
    group_path.finish();

    // --- Random Graph Benchmark (Variable Density) ---
    let mut group_random = c.benchmark_group("all_simple_paths_random_graph");
    group_random.sample_size(10);
    let n = 12; // Fixed size for random graphs

    for p in [0.2, 0.5, 0.8].iter() {
        let mut graph = UnGraph::<(), ()>::new_undirected();
        let nodes: Vec<_> = (0..n).map(|_| graph.add_node(())).collect();
        let mut rng = StdRng::seed_from_u64(42);
        for i in 0..n {
            for j in (i + 1)..n {
                if rng.gen_bool(*p) {
                    graph.add_edge(nodes[i], nodes[j], ());
                }
            }
        }
        let from = nodes[0];
        let to_nodes = [nodes[n - 2], nodes[n - 1]];
        let to_set: HashbrownHashSet<_, RandomState> = to_nodes.iter().cloned().collect();
        let cutoff = Some(n-2);

        run_benchmarks(&mut group_random, &p.to_string(), n, graph, from, to_set, cutoff);
    }
    group_random.finish();
}

fn run_benchmarks(
    group: &mut criterion::BenchmarkGroup<criterion::measurement::WallTime>,
    graph_type: &str,
    n: usize,
    graph: UnGraph<(), ()>,
    from: NodeIndex,
    to_set: HashbrownHashSet<NodeIndex, RandomState>,
    cutoff: Option<usize>,
) {
    let param = format!("{}-{}", graph_type, n);
    group.bench_with_input(
        BenchmarkId::new("new (petgraph+foldhash)", &param),
        &param,
        |b, _| {
            b.iter(|| {
                new(
                    black_box(&graph),
                    black_box(from),
                    black_box(&to_set),
                    black_box(0),
                    black_box(cutoff),
                )
            })
        },
    );

    group.bench_with_input(
        BenchmarkId::new("prev (rustworkx+foldhash)", &param),
        &param,
        |b, _| {
            b.iter(|| {
                prev(
                    black_box(&graph),
                    black_box(from),
                    black_box(&to_set),
                    black_box(0),
                    black_box(cutoff),
                )
            })
        },
    );
}

criterion_group!(benches, benchmark_runner);
criterion_main!(benches);

[IvanIsCoding]

Firstly, thanks for the contribution. It seems you ported #577 to petgraph/petgraph#865 and made the optimizations.

It just needs some minor changes, but this should be good to go.

[lazyhope]

Firstly, thanks for the contribution. It seems you ported #577 to petgraph/petgraph#865 and made the optimizations.

It just needs some minor changes, but this should be good to go.

Thanks for the comments and they are now all resolved!

[jakelishman]

This PR unfortunately ties Rustworkx to now having implicit dependencies on the hashbrown version matching both petgraph's version and PyO3's, which means that even though recent PyO3s can support hashbrown 0.16, Rustworkx can't be updated to relax the upper-bound pin on hashbrown. It's also not possible to compile Rustworkx with anything other than hashbrown in the 0.15 series, due to the petgraph pin on it being 0.15.x, so the lower bound of 0.13 is invalidated, but that was actually already the case - PyO3 0.26's hashbrown feature required >=0.15.

Is there any way the performance improvements in this PR can be kept without transitively pinning the required version of hashbrown further?¹

Footnotes

1. I would really like Rustworkx to relax its upper bound on hashbrown or get hashbrown out of its public APIs entirely because the required version matching from it effectively breaks Rust-space Dependabot from us on Qiskit - it'll try to bump hashbrown on any Dependabot PR in a way that's incompatible because of Rustworkx, so they all need manual intervention right now. ↩

[jakelishman]

Answer: yes - #1552.