What is it about?
Image two neural networks that solve the same task, like identifying objects in images. How similar are these networks? This paper surveys the many ways researchers have been using to assess the similarity of neural networks, focusing on two main approaches: * Representational similarity: is the data transformed and represented similarly in intermediate steps? * Functional similarity: how similar is the input-output behavior?
Featured Image
Photo by Edge2Edge Media on Unsplash
Why is it important?
We collect, categorize, and explain over 50 existing methods for both representational and functional similarity, providing a unified view of what was previously a scattered field. Our work guides researchers in application of the surveyed methods and highlights potential pitfalls. Ultimately, better understanding of model similarity can lead to better understanding of neural networks in general, helping improve them in many aspects.
Perspectives
We were surprised by the large number of different ways researchers measured similarity of neural networks. These methods were often proposed in an ad-hoc manner without much evaluation making it difficult to chose an appropriate method for a specific application. This also inspired follow-up work, like our ReSi benchmark, that shows that picking the right method is indeed very context-dependent.
Max Klabunde
Universitat Passau
Read the Original
This page is a summary of: Similarity of Neural Network Models: A Survey of Functional and Representational Measures, ACM Computing Surveys, April 2025, ACM (Association for Computing Machinery),
DOI: 10.1145/3728458.
You can read the full text:
Resources
Contributors
The following have contributed to this page
