SkinCaRe: A Multimodal Dermatology Dataset with Medical Captions and Chain-of-Thought Reasoning

SkinCaRe bridges the gap between what models see and how they reason. It combines

• SkinCAP — dermatologist-authored, observation-first captions
• SkinCoT — clinician-certified, hierarchical Chain-of-Thought (CoT) diagnostic narratives

Total size: 7,041 dermatologic cases.

🧭 Overview

Most dermatology datasets offer only class labels, limiting transparency and clinical utility. SkinCaRe provides both descriptive captions and structured reasoning, enabling models to describe findings and explain diagnoses.

📦 Dataset Components

📝 SkinCAP (Captioning)

• 4,000 images from Fitzpatrick 17k and DDI
• Captions cover: anatomic site, primary/secondary morphology, color, distribution, surface changes
• Format
  • Images: .png
  • Metadata: .csv with id, disease, caption_zh, caption_en, source links

🧠 SkinCoT (Reasoning)

• 3,041 image–text pairs, clinician-reviewed on six axes: Accuracy, Safety, Medical Groundedness, Clinical Coverage, Reasoning Coherence, Description Precision
• Structure
• Images by category: SkinCoT/images/<disease_class>/<image>.jpg
• CoT (English): SkinCoT/EN/<disease_class>/<image>.jpg.txt
• CoT (Chinese): SkinCoT/ZH/<disease_class>/<image>.jpg.txt

Examples

• Image: SkinCoT/images/Urticaria Hives/dermagraphism-32.jpg
  CoT-EN: SkinCoT/EN/Urticaria Hives/dermagraphism-32.jpg.txt
  CoT-ZH: SkinCoT/ZH/Urticaria Hives/dermagraphism-32.jpg.txt

🔓 Access & License

• Repository: https://huggingface.co/datasets/yuhos16/SkinCaRe
• License: CC-BY-4.0
• Modalities: Image + Text (PNG, JPEG, CSV, TXT)
• Updates to images/metadata will be noted on the dataset page.

🧪 Suggested Uses

• Train/evaluate dermatology VLMs for captioning + reasoning
• Research on medical explainability and trustworthy AI

📚 Citation

• If you find SkinCaRe helpful for your research, please consider citing:

@misc{shen2025skincaremultimodaldermatologydataset,
      title={SkinCaRe: A Multimodal Dermatology Dataset Annotated with Medical Caption and Chain-of-Thought Reasoning}, 
      author={Yuhao Shen and Liyuan Sun and Yan Xu and Wenbin Liu and Shuping Zhang and Shawn Afvari and Zhongyi Han and Jiaoyan Song and Yongzhi Ji and Tao Lu and Xiaonan He and Xin Gao and Juexiao Zhou},
      year={2025},
      eprint={2405.18004},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2405.18004}, 
}