Confronted with Data: Challenges and Paths Toward Methodological Transition

What is it about?

The process I focus on in this paper is the methodological transition, particularly in moving from introspective to data-driven methods. The paper examines the types of evidence acting as tenets of introspective, corpus-based, and psycholinguistic methods and reflects on when the three types act as complementary evidence in studying a phenomenon. To start with, introspective methods are usually foundational and serve the purpose of providing operational definitions and frameworks using a limited number of examples. Introspective methods are based on a methodology, but their description is usually implicit and not well defined, almost known by means of implicit convention. For instance, we do not know why an example, used as the backbone data for introspection, is such a cognitively powerful defining tool and why, despite its strengths, definitions based on examples morph once confronted with data. To focus more on the concept of “examples as a defining tool,” I do two things: (1) I define examples and how they work from a cognitive perspective (really, how does an example instantiate or give you access to a whole concept?) and (2) I collect a sample of examples from the literature on a specific phenomenon and show how they all converge in properties. Following this, I move to the corpus-based method, which, together with any data-driven method, is important for the advancement of theory on any phenomenon (otherwise introspection-based descriptions only, the literature on a phenomenon can fossilize and fade away). First, I define the process of retrieval, show how it is a point of confrontation with messy data, and how it is subject to precision and recall. Then, I examine the methodological implications of retrieval procedures, especially on the definition of the studied phenomenon. I look at eight corpus retrieval strategies of the same metaphor to expose the reader to different retrieval methods and also show how introspective definitions are morphing across these strategies. I also show how the process of corpus cleansing and annotation is the epitome of an ontological confrontation. I end this study with reflections on the need for psycholinguistic evidence and the centrality of intuition. Particularly, I state that “In introspective frameworks, we rely on the intuition of linguists to choose data. In corpus-based models, we rely partly on the intuition of the linguist in creating classifications, definitions, and sampling techniques to access representative data, and partly on the accessed usage data in representing different aspects of the studied phenomenon. What is missing in both models, however, is the intuition of the native speaker and the validation of the psychological reality of these aspects, information on the selection of forms, identification of cues, and explanations of observed constraints, or at least validation of the constraints and the explanations provided in introspective frameworks.”

Featured Image

Why is it important?

A research journey typically starts with research questions and ends with questions about research. The specific question emerging from my PhD and motivating this paper is: What happens when you study a phenomenon using a data-driven approach? Usually, you start with a “clean” definition and then go through a massive process of confrontation with newly collected data. This is the process I focus on in this paper: methodological transition. This article is the outcome of the frustrations I experienced in my doctoral work while transitioning from operational definitions to corpus-based analysis. Part of the reason I wrote this paper is to normalize the challenges researchers face when moving from one method to another. There are sufficient descriptions of methods, but transitions are usually overlooked in the literature. This paper also aligns with my commitment to clarifying research processes more explicitly and making them visible as part of the scholarly conversation.

Perspectives