Explainability for Large Language Models: A Survey

What is it about?

We provide a comprehensive overview of existing interpretability techniques for medium-sized models and large language models respectively. In the part of medium-sized models, interpretability methodologies are introduced in two classes: local and global explanation. Unlike local explanation is less meaningful to LLMs, global explanation is still a major tool. Besides, we introduce some interpretability approaches that are specifically applying to LLMs.

Featured Image

Photo by Andrew Neel on Unsplash

Photo by Andrew Neel on Unsplash

Why is it important?

In the era of LLMs, it is a quick start for practitioners to learn what tools they can use to interpret their learning models. We also provide some ideas on why some techniques applicable to medium-sized models are no long useful with LLMs.