Does question order affect alcohol outcome measurement? Protocol for a trial on ORBITAL COS

What is it about?

The ORBITAL core outcome set (COS) established an internationally agreed minimum standard of ten outcomes to be measured in alcohol brief intervention trials, organised into four clusters: consumption, alcohol-related problems, health and wellbeing, and motivation to change. A practical question for researchers implementing the COS is whether the order in which these four clusters are presented to participants affects how they respond, a phenomenon known as question order bias. This protocol paper describes a pre-registered, double-blind factorial randomised trial with 24 arms, each representing a different ordering of the four clusters. Participants are adults searching online for help with their drinking, who are randomised equally across all 24 arms and complete the full COS questionnaire. The primary analyses will estimate the effect of cluster order on participant responses and examine patterns of questionnaire abandonment by arm.

Featured Image

Why is it important?

Even a well-designed, internationally validated core outcome set can generate misleading data if question order systematically influences how participants answer. Establishing whether order effects exist in the ORBITAL COS is a necessary quality assurance step before the set can be confidently recommended for widespread adoption in trials and evaluations. The 24-arm factorial design is a methodologically rigorous and efficient approach to testing all possible orderings simultaneously, producing evidence that directly informs guidance on how the COS should be administered. Pre-registration and publication of the protocol in advance supports transparency and reduces the risk of undisclosed analytical decisions, setting a high methodological standard for this type of measurement validation work.