`clean_crowns` over-cleans due to transitive union-find clustering

[CiSong10]

Dear developer,

Thank you for the updates of the clean_crowns function and the post_clean function. I was excited to test it but unfortunately, I ran into a significant over-cleaning issue I'd like to flag.

The screenshot shows the result on one of my datasets. Yellow polygons are the original uncleaned crowns, red are the crowns after clean_crowns, and blue are crowns added back by post_clean. As you see, there are still huge gaps even after post_clean converges.

Root cause

I believe this is due to the "transitivity problem" of the union-find cluster strategy in clean_crowns. Consider three crowns A, B, C arranged in a line:

• A overlaps B, B overlaps C, while A and C does not overlap.
• A and B are merged, B and C are merged. A, B, C will be in one cluster.
• Say A has the highest confidence score, C will be removed even though it does not overlap A at all.

Proposed fix

Rather than keeping one crown per cluster, I'd suggest a greedy NMS (non-maximum suppression) approach that resolves conflicts locally and directly. Crowns are sorted by confidence score descending; a crown is kept if it doesn't directly conflict with any already-kept crown, and only its direct conflicts are suppressed.

def clean_crowns():
    # 1. Filter
    ...
    # 2. Spatial join for candidate pairs
    ...
    # 3. Build a conflict graph: edges between crowns that truly conflict:
    # (high IoU OR containment).
    from collections import defaultdict
    conflicts = defaultdict(set)  # crown_idx -> set of conflicting crown_idxs

    for _, row in tqdm(
        join.iterrows(),
        ...
    ):
        i = row.name
        j = row["index_right"]
        if i >= j:
            continue  # Process each pair once

        geom_i = crowns.at[i, "geometry"]
        geom_j = crowns.at[j, "geometry"]
        intersection_area = geom_i.intersection(geom_j).area

        # IoU check
        iou_val = calc_iou(geom_i, geom_j)
        is_conflict = iou_val > iou_threshold

        # Containment check
        if not is_conflict and containment_threshold is not None:
            min_area = min(geom_i.area, geom_j.area)
            if min_area > 0 and (intersection_area / min_area) > containment_threshold:
                is_conflict = True

        if is_conflict:
            conflicts[i].add(j)
            conflicts[j].add(i)

    # 4. Greedy NMS: sort by confidence descending, 
    #  keep a crown if it doesn't conflict with any already-kept crown. 
    sorted_indices = crowns[field].sort_values(ascending=False).index.tolist()
    kept = set()
    removed = set()

    for i in sorted_indices:
        if i in removed:
            continue
        kept.add(i)
        # Only remove direct conflicts, not transitive ones
        for j in conflicts[i]:
            removed.add(j)

    cleaned_crowns = crowns.loc[sorted(kept)].copy()
    return gpd.GeoDataFrame(cleaned_crowns, crs=crowns.crs).reset_index(drop=True)

The result is shown below. I think this gives a much better balance between retaining high-confidence predictions and maintaining coverage of the tree canopy. If doing it right, it isn't even necessary to run post_clean, or only one interation of post_clean is enough.

In #192, I also explored a slightly different strategy for the containment case: comparing the average confidence score of the smaller (contained) crown against the larger (container) crown before deciding which to keep. This produces slightly different results from the greedy NMS approach above, while a single-shot NMS is easier and more straightforward. I'd love to hear your thoughts on this.

Thank you for you attention to this issue.

[PatBall1]

Thanks for this excellent analysis and proposed fix. You're right that the union-find approach causes transitive over-cleaning. We've replaced it with greedy NMS as you suggested:

• Crowns are sorted by confidence descending
• A crown is kept only if it has no direct conflict (IoU or containment) with any already-kept crown
• Only direct conflicts are suppressed — non-overlapping crowns are never transitively removed

The fix also uses make_valid() to repair invalid geometries rather than discarding them.

This change is on the local branch and will be pushed shortly.

[CiSong10]

Thank you!

[PatBall1]

@CiSong10 for context, would you please tell me why you start with so many overlapping predictions? Have you used multitemporal imagery or just substantial buffers?

[PatBall1]

Evaluation: Union-Find vs Greedy NMS for clean_crowns

We ran a landscape-level comparison of the two cleaning strategies to assess the impact of switching from union-find clustering to greedy NMS. The evaluation protocol is saved as a repeatable script (data/eval_landscape_nms.py) for re-running on future model upgrades.

Methodology

• Model: Pretrained flexible model (250312_flexi.pth, R-101-FPN backbone)
• Sites: Paracou UAV 20210406 (5 cm GSD, 44 tiles) and Danum (10 cm GSD, 197 tiles)
• Tiling: 100 m tiles with 30 m buffer
• Evaluation: Post-stitch, post-clean predictions evaluated within GT-dense 200 m windows (4 per site). One match per GT crown (highest IoU prediction), counted as TP if IoU >= 0.5.
• Default thresholds: iou_threshold=0.7, containment_threshold=0.85

Results

Paracou (4396 GT crowns, 3371 stitched predictions):

Strategy	Cleaned	Mean P	Mean R	Mean F1
Union-Find	1935	0.540	0.356	0.423
Greedy NMS	1941	0.540	0.356	0.423

Both strategies produce identical results at default thresholds on Paracou. The conservative thresholds prevent transitive clustering from having any effect.

Danum (833 GT crowns, 11877 stitched predictions):

Strategy	Cleaned	Mean P	Mean R	Mean F1
Union-Find	4086	0.630	0.696	0.658
Greedy NMS	4496	0.609	0.740	0.665

On Danum (denser, noisier predictions with 11.9k pre-clean crowns), greedy NMS retains ~10% more crowns, yielding +4.4% recall and +0.7% F1 at the cost of -2.1% precision.

Interpretation

The transitive merging issue only manifests when predictions are dense enough to form multi-way conflict chains. At conservative default thresholds, the two strategies converge (Paracou). When predictions are denser (Danum has 3.5x more pre-clean crowns per GT crown), NMS preserves valid crowns that union-find removes transitively.

For the multitemporal overlapping scenario @CiSong10 described, where multiple prediction epochs produce many overlapping candidates per real crown, the advantage of NMS would be larger since union-find's transitive closure would aggressively collapse large clusters even when not all crowns within them directly conflict.

Implementation

clean_crowns now uses greedy NMS by default. The API is unchanged -- existing code calling clean_crowns(crowns, iou_threshold, confidence=...) works without modification.

Visual comparison PNGs (3-panel: GT / Union-Find / Greedy NMS with crown outlines on imagery) are available in the repo at data/nms_landscape_visuals/.

[PatBall1]

Visual Evidence

Paracou Window 0 — Both strategies identical at default thresholds (173 crowns each):

Danum Window 0 — NMS retains 136 vs UF's 127 crowns (F1: 0.664 vs 0.672):

Danum Window 3 — Strongest NMS advantage (F1: 0.694 vs 0.663, recall: 0.87 vs 0.78):

Panels: Left = Ground Truth (white), Middle = Union-Find (blue), Right = Greedy NMS (cyan). Magenta dashed outlines on the NMS panel highlight crowns that NMS retains but union-find removes.

Precision tradeoff

Looking at the Danum visuals, some of the extra crowns retained by NMS appear to be small fragments or noise that union-find was correctly removing through transitive clustering. This is reflected in the metrics: NMS gains recall at the cost of precision (-2.1% on Danum).

At the current default thresholds (iou_threshold=0.7, containment_threshold=0.85), this tradeoff is modest. Users dealing with heavy overlaps (multitemporal stacking) may benefit from greedy NMS, while users who prefer fewer false positives can increase the confidence threshold via clean_crowns(crowns, confidence=0.3).

[CiSong10]

@CiSong10 for context, would you please tell me why you start with so many overlapping predictions? Have you used multitemporal imagery or just substantial buffers?

Hi, thank you for taking the time to look into this.

I almost always get this many overlapping predictions when using Detectree2.
The imagery I am working with is RGB only (not multitemporal), at 3-inch (~7.6 cm) resolution. Tiling is 100 m tiles with 30 m buffer as recommended. The model is finetuned based on 250312_flexi.pth.

I'll look at the model prediction by the pre-trained model only, to see if my finetuning caused too much overlapping.

One difference I noticed is that, in your “Ground Truth” examples, the entire green area is not fully covered by tree crown annotations. In contrast, I have been trying to annotate crowns across almost the entire canopy area to the best of my judgment. The reason is that my project requires something close to a complete tree inventory within the project area.

Could this difference in annotation strategy contribute to the overlapping predictions? In particular, is there a reason you did not cover all vegetated areas with crown annotations, such as you consider those places as low vegetation or small trees rather than individual crowns?

[PatBall1]

Updated evaluation: Union-Find + post_clean vs Greedy NMS

We expanded the comparison to include post_clean (the gap-filling step described in the tutorials). This changes the picture significantly.

Danum (833 GT, 11877 stitched predictions) — all four strategies:

Strategy	Crowns	Mean P	Mean R	Mean F1
Union-Find	4086	0.630	0.696	0.658
UF + post_clean	4359	0.615	0.728	0.663
Greedy NMS	4496	0.609	0.740	0.665
NMS + post_clean	4480	0.613	0.738	0.666

Key findings:

1. UF + post_clean (F1=0.663) nearly matches greedy NMS alone (F1=0.665). The gap-filling step recovers most of the valid crowns that NMS retains, but with better precision (0.615 vs 0.609).

2. NMS + post_clean actually reduces crown count (4496→4480) because the re-cleaning step in post_clean removes some of the noisy crowns NMS had retained.

3. All four strategies converge within ±0.008 F1. The differences are within noise for this dataset.

4. Of the 41 extra crowns NMS retains over UF (in the evaluation windows), only 16 (39%) are true positives — the other 25 are noise/fragments that UF was correctly removing.

Conclusion

We are reverting clean_crowns to union-find as the default for backward compatibility and because UF + post_clean already achieves comparable results to NMS with better precision. Greedy NMS is now available as an option via method="nms":

# Default (union-find, recommended with post_clean):
clean = clean_crowns(crowns, 0.6, confidence=0.5)
clean = post_clean(crowns, clean)

# Alternative (greedy NMS):
clean = clean_crowns(crowns, 0.6, confidence=0.5, method="nms")

@CiSong10 — thanks for the context on your setup. Since you're getting very dense predictions with standard tiling parameters, it's worth checking whether tuning the clean_crowns parameters helps for your dataset. The defaults (IoU=0.7, confidence=0.2) may not be optimal when the model produces many overlapping candidates:

1. Have you tried adjusting the iou_threshold and confidence parameters? Lowering iou_threshold (e.g. 0.5) would merge more overlapping predictions, while raising confidence (e.g. 0.4) would filter out weaker duplicates earlier.

2. Are you using post_clean after clean_crowns? This iterative gap-filling step is designed to recover valid crowns removed during cleaning, and is particularly effective for dense canopies.

3. You can also try the new method="nms" option if you find that union-find is removing crowns you want to keep — though our testing suggests the difference is marginal when post_clean is used.

[CiSong10]

Hi, thank you for the detailed suggestions.

Upon reviewing my code, I realized that I made a big mistake that I misunderstood the tile_width & tile_height parameters in tile_data() as the core size, and set them to 40 instead of 100. Correcting this reduces overlapping to a normal level.

With that said, there're still a lot of transitive overlapping crowns in my prediction . post_clean failed to fill the gaps, I am not sure why.

Below are screenshots from my test using default settings: clean_crowns(iou_threshold=0.5, area_threshold=2, confidence=0.2,containment_threshold=0.85), post_clean(iou_threshold=0.3).

In the left image, a big chunk of uncleaned crown predictions (in yellow) are not added back afterpost_clean converges. In the right image, since the default post_clean iou_threshold is 0.3, lower than the default clean iou_threshold 0.7, Thus, post_clean actually removes some predicted crowns.

[PatBall1]

@CiSong10 sorry for the delay in getting back to you, I've had other things on. I'll look into this now. I appreciate the investigations and contributions!

[PatBall1]

@CiSong10 would you mind sharing some of your data and pipeline so I can replicate the issue?