The MR_Drawing application was developed for use in cognitive science experiments involving mixed reality (MR) and neuroanatomy. The application is designed for the HoloLens 2 and provides a digital overlay of anatomical structures directly onto paper drawings, enhancing the participant’s spatial visualization of the structures.
In the MR condition, participants can interact with both 2D and 3D representations of anatomical structures, helping them reproduce instructor drawings. The application offers several key features:
- Digital Overlays: Participants can view anatomical structures projected directly onto their drawings. A floating menu on the left allows for overlay adjustments, such as changing the structure or modifying the transparency.
- QR Code Tracking: A QR code system placed on the paper ensures that the digital overlays follow the movements of the paper accurately, providing a seamless experience.
- 3D Holograms: In an advanced version, participants can access 3D holograms of anatomical structures. These holograms can be manipulated by grabbing, moving, and resizing, providing a three-dimensional representation that complements the 2D drawings.
- Task: Participants follow a video tutorial and reproduce the instructor's drawings. The MR conditions provide digital assistance, while the control condition involves no digital overlays. Participants are not allowed to interact directly with the video, but can take notes as needed.
Language: The application’s written instructions are provided in French.
You can find a video demonstration on https://youtu.be/bN2Br_fCvEU.
Follow this Microsoft guide to deploy the app on HoloLens by using Visual Studio.
- Start the application on the HoloLens 2 device.
- Choose the version : with our without 3D brain model
- Place the paper with QR codes in the camera's view to activate the tracking system.
- Use the floating menu to adjust and switch the anatomical structure overlays.
- In the 3D model version, user can interact with the 3D model by hands
- Development & Research: Matisse Poupard - CATIE and Flowers Inria Bordeaux.
- Scientific Supervision: H. Sauzéon, A. Tricot, F. Larrue, D. Liguoro, M. Bertrand.
- Cerebral Arterial System: Created by Doctor Jana (purchased model under Standard License).
- Poupard, M., Larrue, F., Bertrand, M., Liguoro, D., Sauzéon, H., & Tricot, A. (2025). Enhancing Anatomy Learning Through Mixed Reality-Supported Drawing: Investigating Learning Performance, Cognitive Load, and Intrinsic Motivation. https://hal.science/hal-05050491
- Matisse Poupard, Florian Larrue, Martin Bertrand, Dominique Liguoro, André Tricot, Hélène Sauzéon. Guided Drawing in Medical Education: Evaluating AR and MR Systems for Anatomy Learning. STE 2024 - 4th International Conference on Science and Technology Education, Oct 2024, Porto, Portugal. ⟨hal-04816181⟩
For inquiries or collaborations, please contact poupard.matisse@gmail.com.