Awesome-Model-Reuse-Papers

This is the paper list of "A Unifying Perspective on Model Reuse: From Small to Large Pre-Trained Models" (IJCAI 2025). If you have a relevant paper not included in the library, please raise new pull requests. If you use any content of this repo for your work, please cite the following bib entry:

@inproceedings{zhou2025unifying,
  title={A Unifying Perspective on Model Reuse: From Small to Large Pre-Trained Models},
  author={Da-Wei Zhou and Han-Jia Ye},
  booktitle={IJCAI},
  year={2025}
}

Model Resuse

Model Selection

Semantic/rule-based methods

Paper Title	Year	Conference/Journal
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face	2024	NeurIPS
Visual Programming: Compositional visual reasoning without training	2023	CVPR
Taskonomy: Disentangling Task Transfer Learning	2018	CVPR
Learnware: on the future of machine learning	2016	Frontiers of Computer Science

Metric-based methods

Paper Title	Year	Conference/Journal
Bridge the Modality and Capability Gaps in Vision-Language Model Selection	2024	NeurIPS
LOVM: Language-Only Vision Model Selection	2023	NeurIPS
Transferability Estimation Using Bhattacharyya Class Separability	2022	CVPR
Not All Models Are Equal: Predicting Model Transferability in a Self-challenging Fisher Space	2022	ECCV
Transferability Estimation Based On Principal Gradient Expectation	2022	arXiv
LogME: Practical Assessment of Pre-trained Models for Transfer Learning	2021	ICML
Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs	2022	JMLR
PACTran: PAC-Bayesian Metrics for Estimating the Transferability of Pretrained Models to Classification Tasks	2022	ECCV
Ranking Neural Checkpoints	2021	CVPR
A linearized framework and a new benchmark for model selection for fine-tuning	2021	arXiv
LEEP: A New Measure to Evaluate Transferability of Learned Representations	2020	ICML
Deep Model Transferability from Attribution Maps	2019	NeurIPS
Transferability and Hardness of Supervised Classification Tasks	2019	ICCV
An Information-Theoretic Approach to Transferability in Task Transfer Learning	2019	ICIP

Learning-based methods

Paper Title	Year	Conference/Journal
GraphRouter: A Graph-based Router for LLM Selections	2025	ICLR
Leeroo Orchestrator: Elevating LLMs Performance Through Model Integration	2024	arXiv
Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models	2024	NAACL
MODEL SPIDER: Learning to Rank Pre-Trained Models Efficiently	2023	NeurIPS
Pre-Trained Model Reusability Evaluation for Small-Data Transfer Learning	2022	NeurIPS
Task2Vec: Task Embedding for Meta-Learning	2019	ICCV
Adaptive Mixtures of Local Experts	1991	Neural Computation

Model Adaptation

Adapt PTMs for target data preparation

Paper Title	Year	Conference/Journal
Model Reuse with Reduced Kernel Mean Embedding Specification	2023	TKDE
Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA	2023	arXiv
Visual Classification via Description from Large Language Models	2023	ICLR
What Does a Platypus Look Like? Generating Customized Prompts for Zero-Shot Image Classification	2023	ICCV
Forward Compatible Training for Large-Scale Embedding Retrieval Systems	2022	CVPR
Heterogeneous Model Reuse via Optimizing Multiparty Multiclass Margin	2019	ICML
Deep Label Distribution Learning with Label Ambiguity	2017	TIP
Learning Using Privileged Information: Similarity Control and Knowledge Transfer	2015	JMLR
A new learning paradigm: Learning using privileged information	2009	NN

Adapt PTMs for target model training

Paper Title	Year	Conference/Journal
MiniLLM: Knowledge Distillation of Large Language Models	2024	ICLR
Generalized Knowledge Distillation via Relationship Matching	2023	TPAMI
Explicit Inductive Bias for Transfer Learning with Convolutional Networks	2018	ICML
FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformer	2023	AAAI
Visual Instruction Tuning	2023	NeurIPS
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models	2023	ICML
LoRA: Low-Rank Adaptation of Large Language Models	2022	ICLR
Robust fine-tuning of zero-shot models	2022	CVPR
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models	2022	ACL
Deep Model Reassembly	2022	NeurIPS
Visual Prompt Tuning	2022	ECCV
Heterogeneous Few-Shot Model Rectification with Semantic Mapping	2021	TPAMI
Parameter-Efficient Transfer Learning with Diff Pruning	2021	ACL/IJCNLP
Prefix-Tuning: Optimizing Continuous Prompts for Generation	2021	ACL/IJCNLP
Rethinking the Hyperparameters for Fine-tuning	2020	ICLR
Catastrophic Forgetting Meets Negative Transfer: Batch Spectral Shrinkage for Safe Transfer Learning	2019	NeurIPS
Towards Safe Weakly Supervised Learning	2019	TPAMI
Parameter-Efficient Transfer Learning for NLP	2019	ICML
Relational Knowledge Distillation	2019	CVPR
Spectral Normalization for Generative Adversarial Networks	2018	ICLR
Explicit Inductive Bias for Transfer Learning with Convolutional Networks	2018	ICML
Spectral Normalization for Generative Adversarial Networks	2018	ICLR
A Gift From Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning	2017	CVPR
Deep Learning for Fixed Model Reuse	2017	AAAI
Deep Residual Learning for Image Recognition	2016	CVPR
Distilling the Knowledge in a Neural Network	2015	arXiv
FitNets: Hints for Thin Deep Nets	2015	ICLR
How transferable are features in deep neural networks?	2014	NIPS
Extracting Symbolic Rules from Trained Neural Network Ensembles	2003	AI Communications

Adapt PTMs for target model inference

Paper Title	Year	Conference/Journal
Git Re-Basin: Merging Models modulo Permutation Symmetries	2023	ICLR
REPAIR: REnormalizing Permuted Activations for Interpolation Repair	2023	ICLR
ZipIt! Merging Models from Different Tasks without Training	2023	CVPR
Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learning	2023	CVPR
Head2Toe: Utilizing Intermediate Representations for Better Transfer Learning	2022	ICML
Merging Models with Fisher-Weighted Averaging	2022	NeurIPS
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time	2022	ICML
Model Fusion via Optimal Transport	2020	NeurIPS
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks	2020	NeurIPS
Language Models are Few-Shot Learners	2020	NeurIPS
Distance-Based Image Classification: Generalizing to New Classes at Near-Zero Cost	2013	TPAMI

Other Topics

Model Assembly

Paper Title	Year	Conference/Journal
Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks	2024	NeurIPS
Modular Deep Learning	2024	TMLR
Learngene: From Open-World to Your Learning Task	2022	AAAI
Deep Model Reassembly	2022	NeurIPS

Model Representation Learning

Paper Title	Year	Conference/Journal
Towards Scalable and Versatile Weight Space Learning	2024	ICML
Universal Neural Functionals	2024	NeurIPS
Model Zoos: A Dataset of Diverse Populations of Neural Network Models	2022	NeurIPS
Predicting Neural Network Accuracy from Weights	2020	arXiv

Model Collaboration

Paper Title	Year	Conference/Journal
Tool Learning with Foundation Models	2024	ACM Computing Surveys

Model Compression

Paper Title	Year	Conference/Journal
Adaptive Quantization for Deep Neural Network	2018	AAAI

Model Repair and Editing

Paper Title	Year	Conference/Journal
Zero-Shot Model Diagnosis	2023	CVPR
Memory-Based Model Editing at Scale	2022	ICML

Managing LLMs

Paper Title	Year	Conference/Journal
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face	2024	NeurIPS