Feature visualization is a powerful technique for improving the interpretability of deep neural networks. Activation maximization is particularly popular for visualizing the learned features of a network unit. Through gradient ascent, this approach generates human-interpretable images that reveal what the neural network is effectively looking for in the input. For an introduction to feature visualization, visit: https://distill.pub/2017/feature-visualization/
A detailed report on the project, which includes a literature review on feature visualization, an overview of the visualization process, and experimental results with detailed discussions, can be found at: https://www.academia.edu/127100598/Feature_Visualization_of_Deep_Neural_Networks_MMI727_Term_Project_Report
Short explanation of code files:
-
activationMaximization.ipynb: Implements a pipeline for generating a visualization that highlights features associated with a specific ImageNet class using a Residual Network (ResNet50) and a Vision Transformer (ViT-B16) model. It incorporates image augmentations during optimization to improve generalization.
-
activationMaximization_multipleIndexes.ipynb: An update on the previous one and performs activation maximization to generate visualizations for multiple class indexes with two deep learning models (ResNet18 and ViT-B/16) in a single pipeline. It uses early stopping to halt optimization if the loss does not improve after a specified number of iterations, and saves the best visualizations for each class. The code includes random augmentations and L2 regularization to enhance the diversity of the generated images.
-
crossEvaluation.ipynb: Implements a cross-evaluation strategy for comparing two deep learning models (ResNet18 and ViT-B/16) on a dataset of images. The dataset includes images generated by activation maximization, both with and without augmentations.
Example visualization using ResNet18 for class index 852 (tennis ball):
