A new measure of consensus (dissent) for qualitative scales

What is it about?

A new metric of consensus for Likert-type scales is presented. The statistic provides the level of agreement for any given number of response options as the percentage of consensus among respondents. A geometric framework is used to analytically derive a positional indicator. The statistic is obtained as the relative weight of the distance from the point containing the proportions of observations that fall in each category to the centre of a regular polygon with as many vertices as categories, which corresponds to the point of maximum dissent. In order to assess the performance of the proposed metric of consensus, an iterated forecasting experiment is conducted to test whether the inclusion of the degree of agreement in households’ expectations improves out-of-sample forecast accuracy of the unemployment rate in seven European countries and the Euro Area. Additionally, a simulation experiment is designed in order to compare the sampling distribution of the proposed metric of consensus (dissent) to the index of qualitative variation (IQV). Starting from the simulation of 100,000 aggregated response combinations for five categories, we group the shares of the extreme reply options in order to compute the consensus metrics for 3 and 5 categories, and the inverse of the IQV for both scenarios.

Featured Image

Photo by Paweł Czerwiński on Unsplash

Photo by Paweł Czerwiński on Unsplash

Why is it important?

The paper derives a measure of positional agreement that has a straightforward interpretation, as it provides the percentage of agreement among the respondents. The study also provides evidence that the level of consensus among households contains useful information to predict unemployment rates, showing the potential of agreement metrics to track the evolution of economic variables. In the simulation experiment, we find that the proposed metric for three-response alternatives shows a higher level of granularity and dispersion than the one for five-response categories. When comparing the proposed metric to the inverse of the IQV, we find that extreme values and standard deviations are quite similar between both statistics, but the interquartile range, which is obtained as the difference between upper and lower quartiles, is higher for the consensus indicators, especially for three categories. We also observe that the consensus metrics yield higher mean values. This notion is further confirmed when comparing the boxplots for both measures. Finally, we use a scatter plot to depict the type of relationship between both metrics, observing a non-linear relationship between them.

Perspectives