Exploring Robustness & Explanation Agreement in Deep-Learning Models Via Gradient-based Saliency Methods and Cosine Similarity

This repo accompanies the paper "Do Models Think Similarly? Exploring Robustness & Explanation Agreement in Deep-Learning Models" written by Skyler Wu '24 for Professors Xiao-li Meng and Adel Daoud's STAT 288: Deep Statistics: AI and Earth Observations for Sustainable Development.

Supplementary Figures

Within the figures folder, the quantitative subfolder contains dataset-specific aggregate figures showing the mean SmoothGrad cosine similarity heatmaps across both training and testing splits, and across difficulty of examples (easiest, hardest, or overall). The qualitative subfolder contains specific examples of SmoothGrad saliency maps and associated figures for each dataset. For example, the subfolder CIFAR10_class=0_split=test_diff-q=0.25 stores a class 0 image from the CIFAR-10 dataset's testing split. The q=0.25 tag means that the test accuracy on this specific image (i.e., proportion of total models that classified this image correctly) was in the 25th percentile of unique test accuracies for this dataset. Within this CIFAR10_class=0_split=test_diff-q=0.25 subfolder,

• img.png shows the original image.
• kde.png shows the kernel density estimation curves of the pairwise cosine similarities of various models' saliency maps.
• maps.png shows sample SmoothGrad saliency maps from various models and random-initializations on this particular image.
• tsne.png shows the 2-component T-SNE clustering separation of all saliency maps generated for this image by all of our models, color-coded by model architecture variant.

Auditing and Reproducibility

• The logs folder contains the full train/test accuracies of each of our model variant random initializations on each individual data point in our three datasets. We compress this folder to respect GitHub's file size restrictions.
• The models folder contains final-epoch model weights for each of our model architectures and random initializations for each of our dataset (450 apiece for MNIST and FashionMNIST, 300 for CIFAR-10), as well as a logs.csv file that records train/test loss and train/test accuracy across epochs.
• The notebooks folder includes some notebooks that were used to process our results, including some deprecated notebooks for prototyping and debugging. Of particular importance are the analyzer_{MNIST, FashionMNIST, CIFAR10}.ipynb notebooks used to generate the files in the logs folder described above. The quantitative_visualizer_main.ipynb notebook was used to generate the figures in the quantitative folder described earlier above.
• The scripts folder contains the raw .sh and .py scripts used to run jobs at scale on the cluster. Within the helpers folder, gradients.py contains the SmoothGrad saliency map implementation from pytorch-smoothgrad, resnet.py contains the ResNet models specifically designed for CIFAR-10 from the pytorch_resnet_cifar10 library, and utils.py contains various data-loading helper functions as well as the MLP and CNN model designs, which we gratefully acknowledge inspiration from Brandon Rohrer's Parameter Efficiency Leaderboard, scs-gallery and sharpened-cosine-similarity repositories. In-line citations with the appropriate URLs to each repository are also provided in all code files in the helpers folder. The model_training folder contains the pipelines to train all 1200 models tested (MLP, CNN, and ResNet). If you would like to reproduce our results, please make sure all runner files and the files in helpers are all in the same directory, ensure your SLURM partitions, directories, credentials, and environments have been toggled appropriately, and then call bash cifar10_cnn_main_runscript_driver.sh, for example, to start the model training process. Analogous scripts can be found in the model_training folder for training the other model architectures. The saliency_maps folder contains a saliency_map_generator_main_runscript_driver.sh with supporting files that you can run as a set of SLURM jobs to generate saliency maps from our models and datasets. Finally, the qualitative_figures folder contains the scripts necessary to generate sample saliency maps, T-SNE representations, KDE plots, and the original images themselves for select images as described earlier.

To run all scripts, please ensure that all scripts and helper files are in the same directory, and please make sure to configure your SLURM partitions, directories, credentials, and environments appropriately before submitting SLURM jobs via our (...)_runscript_driver.sh files. Please run these jobs at your own risk, noting that nontrivial high-performance computing power is required.

Acknowledgements

We gratefully acknowledge the use of the pytorch-smoothgrad implementation of the SmoothGrad saliency map method. Candidate models for convolutional neural network (CNN) architectures were inspired by various models from Brandon Rohrer's curated Parameter Efficiency Leaderboard, sharpened-cosine-similarity and scs-gallery repositories. We chose to base our CNN variants off of Rohrer's repositories because these architectures were designed to be parameter-efficient and thus require less compute power. It should be noted that the architectures showcased in Rohrer's repositories are based on the Sharpened Cosine Similarity (SCS) layer, which is an alternative to traditional convolutional layers. However, previous work by Wu et al. suggests that it is not the layer type (SCS or convolution) that induces increased parameter efficiency, but rather the choice of the layers design itself (e.g., size of the layers, ordering of the layers, etc.).

Thus, for this project, we borrow parameter-efficient architectures from Rohrer's repositories, but then replace the SCS layers with traditional convolutional layers, with corresponding traditional ReLU activation functions. Please see the file scripts/helpers/utils.py for the specific designs of the MLP and CNN models that we worked with for this study, with in-line citations to the relevant repository files whenever relevant. For our ResNet family models, we gratefully acknowledge the use of ResNet implementations specifically designed for CIFAR-10 from the pytorch_resnet_cifar10 library by Idelbayev. The MLP models were designed completely from scratch by me, via simple trial-and-error.