Fundamental Limitations of the Randomized Controlled Trial Method

What is it about?

The Randomized Controlled Trial (RCT) method is often used by researchers to investigate whether or not an intervention has specific effects in a particular group of people. It is considered the golden standard, as the method can reduce bias to a low level. However, there are many examples in nutritional science (e.g. using probiotics) where RCTs given varying results. Although this is often attributed to the trial setup or the intervention, this review shows that the RCT method as such has inherent presuppositions (implicit assumptions) and that if these are not true, the method cannot provide valid conclusions. Three specific presuppositions are identified. First, the RCT method presupposes that the groups in a RCT (e.g. the active intervention and the placebo group) are uniform, or, in other words, that the groups are exchangeable without changing the results. Second, it is presumed that the effects of the intervention are independent of any uncontrolled factors. And as a third presupposition, the active intervention and placebo are assumed to be well-defined. Using probiotics as example, the review shows that these three presuppositions are often invalid. That is, the effect of probiotics, being live microorganisms, is not independent of the host and their microbiome. Moreover, although the probiotic intervention often is well-defined in terms of exact strain(s) and dose being used, the therapeutic effects of probiotics are typically mediated by metabolites produced by the probiotics when it arrives in the gut. And since these depend on the local circumstances, it is questionable whether the intervention can be considered well-defined. Finally, the two aforementioned points are also linked to the fact that each person's microbiome is as unique as a fingerprint. That means, that in many cases the groups are not exchangeable. The review concludes that in some cases, where the effects of probiotics may be less dependent on the local circumstances and microbiome, RCTs may provide valid results, but in others it may be needed to use other means to investigate the effects of probiotics. Several options are listed.

Featured Image

Photo by National Institute of Allergy and Infectious Diseases on Unsplash

Photo by National Institute of Allergy and Infectious Diseases on Unsplash

Why is it important?

The review shows that typical explanations about why some RCTs with probiotics give varying results may not be sufficient. This may not be due to absence or presence of actual effectiveness, but, as shown, can also be due to the fact that the inherent presuppositions behind the RCT method may not be true. When these presuppositions are not true, no valid conclusion (neither positive , not negative) can be drawn. This is important, since the RCT method is considered the golden standard, without considering the fundamental presuppositions behind it. Moreover, the fact that these implicit presuppositions exist and may not be true in all cases, could also apply to other nutritional studies.

Perspectives