Tong Wang¹², Yaolei Qi¹, Siwen Wang², Imran Razzak², Guanyu Yang¹✉, Yutong Xie²✉
¹ Southeast University, China
² Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), UAE
✉ Corresponding Author
- [Feb, 2026] CMSA-Net is currently under review.
- [Feb, 2026] Released prediction results for both PVT-V2-B2 and Res2Net50 backbones.
Video polyp segmentation (VPS) is an important task in computer-aided colonoscopy, as it helps doctors accurately locate and track polyps during examinations. However, VPS remains challenging because polyps often look similar to surrounding mucosa, leading to weak semantic discrimination. In addition, large changes in polyp position and scale across video frames make stable and accurate segmentation difficult.
To address these challenges, we propose a robust VPS framework named CMSA-Net. The proposed network introduces a Causal Multi-scale Aggregation (CMA) module to effectively gather semantic information from multiple historical frames at different scales. By using causal attention, CMA ensures that temporal feature propagation follows strict time order, which helps reduce noise and improve feature reliability.
Furthermore, we design a Dynamic Multi-source Reference (DMR) strategy that adaptively selects informative and reliable reference frames based on semantic separability and prediction confidence. This strategy provides strong multi-frame guidance while keeping the model efficient for real-time inference.
Extensive experiments on the SUN-SEG dataset demonstrate that CMSA-Net achieves state-of-the-art performance, offering a favorable balance between segmentation accuracy and real-time clinical applicability.
Figure: Framework Overview.
Figure: Qualitative comparison of CMSA-Net against other SOTA methods.
Complete quantitative results on the SUN-SEG benchmark. Bold indicates the best performance.
| Method | Publication | Backbone | Sα↑ | Eφ↑ | Fβ↑ | Dice↑ | IoU↑ | MAE↓ |
|---|---|---|---|---|---|---|---|---|
| COSNet | TPAMI'19 | - | 84.5 | 83.6 | 72.7 | 73.0 | 64.8 | 3.4 |
| PCSA | AAAI'20 | - | 85.2 | 83.5 | 68.1 | 70.9 | 60.4 | 3.9 |
| 2/3D | MICCAI'20 | - | 89.5 | 90.9 | 81.9 | 82.9 | 75.6 | 2.1 |
| PraNet | MICCAI'20 | R2-50 | 91.8 | 94.2 | 87.7 | 88.3 | 82.5 | 2.0 |
| ACSNet | MICCAI'20 | R-34 | 92.0 | 94.2 | 87.4 | 88.2 | 82.8 | 1.7 |
| SANet | MICCAI'21 | R2-50 | 91.6 | 93.3 | 86.6 | 87.2 | 82.0 | 1.8 |
| SEPNet | TCSVT'24 | PVTv2-B2 | 93.1 | 96.2 | 88.3 | 89.6 | 83.4 | 1.7 |
| PNS | MICCAI'21 | R2-50 | 90.6 | 91.0 | 83.6 | 84.1 | 78.3 | 2.0 |
| PNS+ | MIR'22 | R2-50 | 91.7 | 92.5 | 84.8 | 85.5 | 78.7 | 2.1 |
| MAST | Arxiv'24 | PVTv2-B2 | 92.5 | 96.2 | 87.8 | 89.3 | 82.7 | 1.6 |
| SALI | MICCAI'24 | PVTv2-B2 | 90.2 | 93.2 | 84.9 | 85.8 | 78.9 | 2.4 |
| SALI | MICCAI'24 | PVTv2-B5 | 90.7 | 93.7 | 85.1 | 86.2 | 79.6 | 2.2 |
| STDDNet | ICCV'25 | R2-50 | 93.5 | 96.0 | 89.7 | 90.5 | 85.0 | 1.5 |
| STDDNet | ICCV'25 | PVTv2-B2 | 94.1 | 96.9 | 90.5 | 91.5 | 86.1 | 1.4 |
| Ours | - | R2-50 | 94.5 | 97.3 | 90.5 | 91.9 | 86.5 | 1.2 |
| Ours | - | PVTv2-B2 | 95.1 | 97.5 | 91.6 | 92.6 | 87.6 | 1.1 |
| Method | Publication | Backbone | Sα↑ | Eφ↑ | Fβ↑ | Dice↑ | IoU↑ | MAE↓ |
|---|---|---|---|---|---|---|---|---|
| COSNet | TPAMI'19 | - | 65.4 | 60.0 | 43.1 | 42.3 | 34.2 | 7.3 |
| PCSA | AAAI'20 | - | 68.0 | 66.0 | 45.1 | 45.0 | 35.3 | 7.8 |
| 2/3D | MICCAI'20 | - | 78.6 | 77.7 | 65.2 | 65.6 | 57.0 | 4.4 |
| PraNet | MICCAI'20 | R2-50 | 78.1 | 78.8 | 66.3 | 66.5 | 58.2 | 5.2 |
| ACSNet | MICCAI'20 | R-34 | 77.2 | 76.6 | 63.0 | 63.8 | 56.4 | 4.6 |
| SANet | MICCAI'21 | R2-50 | 75.0 | 72.8 | 59.0 | 59.3 | 52.4 | 5.2 |
| SEPNet | TCSVT'24 | PVTv2-B2 | 82.9 | 88.3 | 73.5 | 75.1 | 66.6 | 4.2 |
| PNS | MICCAI'21 | R2-50 | 76.7 | 74.4 | 61.6 | 61.8 | 54.5 | 4.8 |
| PNS+ | MIR'22 | R2-50 | 80.6 | 79.8 | 67.6 | 67.8 | 59.1 | 4.4 |
| MAST | Arxiv'24 | PVTv2-B2 | 83.2 | 89.4 | 74.9 | 77.0 | 67.4 | 3.7 |
| SALI | MICCAI'24 | PVTv2-B2 | 73.1 | 75.2 | 58.7 | 59.2 | 50.2 | 6.3 |
| SALI | MICCAI'24 | PVTv2-B5 | 77.1 | 82.1 | 64.6 | 65.6 | 56.8 | 5.5 |
| STDDNet | ICCV'25 | R2-50 | 81.7 | 83.0 | 72.1 | 72.4 | 64.3 | 3.7 |
| STDDNet | ICCV'25 | PVTv2-B2 | 86.0 | 90.3 | 78.6 | 80.1 | 72.4 | 3.4 |
| Ours | - | R2-50 | 84.4 | 90.1 | 75.3 | 77.5 | 69.2 | 3.5 |
| Ours | - | PVTv2-B2 | 86.7 | 90.3 | 79.3 | 80.3 | 72.6 | 2.9 |
| Method | Publication | Backbone | Sα↑ | Eφ↑ | Fβ↑ | Dice↑ | IoU↑ | MAE↓ |
|---|---|---|---|---|---|---|---|---|
| COSNet | TPAMI'19 | - | 78.5 | 77.2 | 62.6 | 63.3 | 54.1 | 4.6 |
| PCSA | AAAI'20 | - | 77.2 | 75.9 | 56.6 | 58.5 | 47.9 | 5.7 |
| 2/3D | MICCAI'20 | - | 84.9 | 86.9 | 75.3 | 76.4 | 67.1 | 3.5 |
| PraNet | MICCAI'20 | R2-50 | 88.4 | 91.9 | 83.1 | 83.9 | 76.6 | 3.1 |
| ACSNet | MICCAI'20 | R-34 | 87.2 | 91.0 | 80.6 | 82.0 | 74.8 | 3.6 |
| SANet | MICCAI'21 | R2-50 | 87.4 | 90.5 | 81.0 | 82.0 | 74.8 | 3.3 |
| SEPNet | TCSVT'24 | PVTv2-B2 | 89.4 | 94.0 | 83.5 | 85.7 | 77.6 | 3.4 |
| PNS | MICCAI'21 | R2-50 | 87.0 | 89.2 | 78.7 | 79.6 | 72.1 | 3.3 |
| PNS+ | MIR'22 | R2-50 | 88.7 | 90.2 | 80.6 | 81.3 | 72.8 | 3.0 |
| MAST | Arxiv'24 | PVTv2-B2 | 89.2 | 94.2 | 83.2 | 85.3 | 76.7 | 2.6 |
| SALI | MICCAI'24 | PVTv2-B2 | 86.8 | 90.9 | 79.9 | 81.0 | 72.6 | 3.4 |
| SALI | MICCAI'24 | PVTv2-B5 | 86.6 | 91.0 | 79.7 | 81.0 | 72.9 | 3.8 |
| STDDNet | ICCV'25 | R2-50 | 91.3 | 95.2 | 86.9 | 88.1 | 81.0 | 2.3 |
| STDDNet | ICCV'25 | PVTv2-B2 | 91.1 | 95.0 | 86.0 | 87.8 | 80.6 | 2.8 |
| Ours | - | R2-50 | 92.7 | 96.1 | 87.1 | 89.8 | 83.0 | 1.8 |
| Ours | - | PVTv2-B2 | 92.3 | 95.6 | 87.1 | 88.9 | 82.1 | 1.9 |
| Method | Publication | Backbone | Sα↑ | Eφ↑ | Fβ↑ | Dice↑ | IoU↑ | MAE↓ |
|---|---|---|---|---|---|---|---|---|
| COSNet | TPAMI'19 | - | 67.0 | 62.7 | 44.3 | 43.8 | 35.3 | 7.0 |
| PCSA | AAAI'20 | - | 68.2 | 66.0 | 44.2 | 45.0 | 35.1 | 8.0 |
| 2/3D | MICCAI'20 | - | 78.6 | 77.5 | 63.4 | 64.4 | 55.8 | 4.4 |
| PraNet | MICCAI'20 | R2-50 | 78.7 | 80.2 | 66.7 | 67.5 | 58.7 | 5.3 |
| ACSNet | MICCAI'20 | R-34 | 76.2 | 77.6 | 61.0 | 62.4 | 54.7 | 5.3 |
| SANet | MICCAI'21 | R2-50 | 75.3 | 73.6 | 59.0 | 59.5 | 52.7 | 5.5 |
| SEPNet | TCSVT'24 | PVTv2-B2 | 84.7 | 89.5 | 74.5 | 77.4 | 68.4 | 3.9 |
| PNS | MICCAI'21 | R2-50 | 76.7 | 75.5 | 60.9 | 61.5 | 53.9 | 5.0 |
| PNS+ | MIR'22 | R2-50 | 79.8 | 79.3 | 65.4 | 66.1 | 57.1 | 5.0 |
| MAST | Arxiv'24 | PVTv2-B2 | 85.6 | 91.3 | 77.2 | 79.9 | 70.8 | 3.1 |
| SALI | MICCAI'24 | PVTv2-B2 | 72.8 | 75.9 | 56.9 | 57.9 | 48.7 | 6.8 |
| SALI | MICCAI'24 | PVTv2-B5 | 76.5 | 81.3 | 62.0 | 63.6 | 54.7 | 5.7 |
| STDDNet | ICCV'25 | R2-50 | 83.4 | 85.6 | 74.1 | 75.0 | 67.3 | 3.7 |
| STDDNet | ICCV'25 | PVTv2-B2 | 86.3 | 90.2 | 78.1 | 80.2 | 72.2 | 3.5 |
| Ours | - | R2-50 | 85.1 | 89.8 | 75.0 | 78.0 | 69.5 | 3.6 |
| Ours | - | PVTv2-B2 | 87.3 | 91.0 | 79.6 | 81.3 | 73.7 | 2.9 |
All scores are in percentage (%).
As the paper is currently under review, the source code is not yet publicly available. However, we provide the full prediction maps for comparison and evaluation purposes:
- PVT-V2-B2 Predictions: Download via OneDrive
- Res2Net50 Predictions: Download via OneDrive
If you find this work useful, please consider citing:
@article{wang2026cmsanet,
title={CMSA-Net: Causal Multi-scale Aggregation with Adaptive Multi-source Reference for Video Polyp Segmentation},
author={Tong Wang and Yaolei Qi and Siwen Wang and Imran Razzak and Guanyu Yang and Yutong Xie},
journal={arXiv preprint arXiv:2602.22821},
year={2026}
}