More on validity

Maxwell's typology and the nature of evidence

I want to focus a bit more on Maxwell’s typology of validity. He has three primary types of validity: descriptive validity, interpretive validity, and theoretical validity, which I will consider in turn.

Descriptive validity

Descriptive validity relates to the factual accurateness of some account. Everyone would agree that I should at least attempt to accurately transcribe an interview with a participant. If Alice said that some event made her sad, it would be malpractice to record, code, and theorize from her saying that the event made her happy. As such, descriptive validity pertains to data and evidence (more on that below). There are some minor difficulties with this account, but I think they are easily accounted for and, on the whole, the idea is intuitive. One problem might be that different observers may have contradictory descriptions of the same event. If a man raises his voice, did he shout at someone or was he, instead, trying to get their attention? But this example offers the solution. What may appear to be a straightforward description often contains implicit interpretation. Here, the description would simply be that the man increased the volume of his voice. The nature of that increase in volume is a matter of interpretive validity.

Image of actual atoms taken from an electron microscope

The second problem is that what is that some things that are theoretical in some contexts are descriptive in another. Some Greek philosophers theorized that atoms existed. So what was once an inferential theoretical claim has now, with new methods, become directly observable and thus has shifted into the domain of descriptive validity. And that gets to the last nuance on descriptive validity: it pertains to observables. I can hear the man raise his voice. I can’t actually see whether he is shouting or trying to get my attention. The former is a matter of description of an observable fact; the latter is an inference.

Interpretive validity

Descriptive validity relates observable facts, whereas interpretive validity relates to unobservable processes—especially (for qualitative researchers) mental processes. I can’t actually see or feel what my participants are thinking. I have to infer that. Sometimes, those inferences are easy: Alice tells me she was sad, so I can infer that she was sad. Sometimes, I have to make inferences based on other factors such as body language, tone of voice, and so on. I think of interpretive validity, then, also in terms of accuracy: how accurately does my interpretive claim reflect or capture what my participant was really thinking or feeling? In chaplaincy, I was taught to check my assumptions. If I thought a patient seemed sad, I could just remark on her behavior and invite her to correct me. I was (unbeknownst to me) ensuring the interpretive validity of my assessment. Member checking provides another way to assess interpretive validity.

Theoretical validity

Whereas both descriptive validity and interpretive validity are in some sense concerned with accuracy, theoretical validity is more concerned with notions like appropriateness and explanatory power. This validity is in some ways akin to focused coding or theoretical coding. Here the researcher is asking about real causal (social) mechanisms driving the observed data. What theory best accounts for and could predict the data? The popular metaphor of building a puzzle is helpful here. Whatever image emerges, the important question is how well the pieces fit together.

Critical realism is helpful here. I’ve already argued why I think any attention to accuracy and validity as such require a precommitment to at least some form of ontological realism, but theoretical validity says that we are trying to accurately describe at least some sort of real causal processes driving what we observe. But this theory is highly constructed. Interpretive validity notes that what I perceive as shouting, another researcher might perceive as attempting to get attention. Theoretical validity moves beyond this subjective interpretation by seeking causal explanations for patterns in the data. As such, theoretical claims will always be heavily theory laden, and this should be thought of as a feature, not a bug. When checking theoretical validity, I am bringing my sense of the world, my unique perspective, and offering it as a way to understand what I take to be really “out there.” It is possible, perhaps highly probable, that I’m mistaken. It is here that the research community can consider my evidence and findings and offer alternative explanations. I, as a researcher, should seriously consider those alternatives as much as possible. That turns out to be good science—thinking through other possibilities and trying to see if they better account for my data. This method is actually closer to Popper’s falsificationism than standard null hypothesis testing (NHST). In NHST, you try to falsify the null, never your own hypothesis! But following Maxwell, we can look at our own theories and challenge their validity directly, which can lead to much more robust findings, even if they are contextually and perspectivally bound.

Data, evidence, and validity

A final thought: data can be evidence for the validity of a claim. There’s a lot of nuance in that statement. Data is just what is observed; it isn’t evidence until it is put to use with respect to some claim. A man raising his voice is only evidence of a particular claim (“men are more likely to shout when angry”) when that datapoint is considered in the context of that claim. Claims, then, are theoretically situated. That means that data is really only interesting as evidence in a theoretical context.

We can take this observation and thereby dismiss all science and research as theory-laden and subjective; or, we can hereby recognize the importance of and interrelationship between all three levels of validity. A constructivist epistemology allows me to take seriously both the subjectivity of my participants’ experience and my own subjectivity. By extension, this lets me recognize that all of my theoretical claims are bound by subjective context and perspective. But the real processes about which we are theorizing (which is what makes the entire enterprise interesting!)—assuming, of course, they are real—can push back on or theories, giving real feedback, and help us refine or even reject our models as invalid.