MuM: Multi-View Masked Image Modeling for 3D Vision

Chalmers University of Technology; Linköping University; University of Amsterdam

David Nordström, Johan Edstedt, Fredrik Kahl, Georg Bökman

Updates

• [July 8, 2026] Added a more detailed description on the feed-forward reconstruction experiment, found here.
• [June 14, 2026] Experimental eval code released on the eval-dev branch. Code will be merged after further cleanup. Currently just the lightweight matching eval (more coming). This is the main metric used for monitoring our training and guiding our architectural choices. In our experiment, this evaluation strongly correlates with downstream performance.
• [May 1, 2026] Experimental training code released on the train-dev branch. Code will be merged after further cleanup.
• [March 4, 2026] Accepted to CVPR26.
• [November 22, 2025] Initial release of inference code.

Overview

MuM is a Transformer (ViT-L) that extracts highly geometric features from an image. It includes an encoder-decoder structure and for most purposes the decoder is discarded. We show in the paper that MuM beats DINOv3 and CroCo v2 on multi-view tasks such as feedforward 3D reconstruction, matching, and relative pose estimation.

Encoder

The easiest way to use our trained model is to extract patch features solely through the encoder. We use a similar API as DINOv3, e.g.,

from mum import mum_vitl16
from mum.utils import transform_image

model = mum_vitl16(pretrained=True).cuda().eval()
images = transform_image("assets/wma1.jpg").unsqueeze(0).cuda()
feats = model.forward_features(images)['x_norm_patchtokens'] # B, P, 1024

Encoder + Decoder

The full model (including decoder) performs normalized pixel reconstruction in its forward pass. It can also be easily finetuned for any multi-view task. Here is an example use-case:

from mum import mum_vitl16_decoderb
from mum.utils import transform_image
from mum.utils.viz import qualitative_evaluation
import torch

IMAGE_LIST = ["assets/wma1.jpg", "assets/wma2.jpg", "assets/wma3.jpg"]
model = mum_vitl16_decoderb(pretrained=True).cuda().eval()
images = torch.stack([transform_image(image_path, size=(256, 256)) for image_path in IMAGE_LIST]).unsqueeze(0).cuda()
# Two example usages:
# (1) Compute reconstruction loss.
loss, pred, mask = model(images, mask_ratio=0.75) # loss should be ~0.23
# (2) Pixel reconstruction.
qualitative_evaluation(model, images, "assets/reconstruction.png")

This should give something like:

Pretrain

See train-dev branch. Code will be merged after further cleanup.

Evaluation

[COMING]: We aim to release all our evaluations in a separate branch.

Setup/Install

Install via pip (requires Python 3.10+):

# Core package (torch and torchvision required)
pip install mum

# With visualization support (includes matplotlib)
pip install mum[viz]

For this release (solely inference code), you need a modern torch and torchvision version. I use 2.6.0+cu118 and 0.21.0+cu118, respectively, together with Python 3.11.9. Using this, I can replicate the reported evaluations (to be released shortly), but any new version should work fine.

To install from source:

git clone https://github.com/davnords/MuM.git
cd MuM
pip install -e .          # core package
pip install -e ".[viz]"   # with visualization support

Checklist

• Provide basic feature extraction inference code
• Release the pre-training code
• Provide data processing scripts and annotation files
• Release all evaluations

License

Code is MIT licensed. Code for layers is directly descending from DINOv3. DINOv3 has a custom license, see DINOv3.

Acknowledgement

Our code is largely inspired by VGGT, DINOv3, CroCo, and MAE.

BibTeX

If you find our work useful, please consider citing our paper!

@inproceedings{nordstrom2026mum,
  title={MuM: Multi-View Masked Image Modeling for 3D Vision}, 
  author={David Nordström and Johan Edstedt and Fredrik Kahl and Georg Bökman},
  booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2026}
}