Estimating party positions on immigration: Assessing the reliability and validity of different methods

  • Didier Ruedin, Laura Morales
  • Party Politics, June 2017, SAGE Publications
  • DOI: 10.1177/1354068817713122

We assess different methods to identify party positions on immigration

Photo by Perry Grone on Unsplash

Photo by Perry Grone on Unsplash

What is it about?

In this article, we systematically assess different methods that can be used to identify party positions on immigration. We focus on immigration because this policy domain does not necessarily overlap with left–right, and because its salience and issue complexity vary across time and space. To assess the validity and reliability of different methods, we use 283 party manifestos (main parties in 8 Western European countries over 20 years) and different methods to obtain party positions: manual sentence-by-sentence coding using a conventional codebook, manual coding using checklists, automated coding using Wordscores, Wordfish and keywords. In addition, we use expert surveys and scores from the Comparative Manifesto Project (CMP). We find that the positions from experts, manual sentence-by-sentence coding, and manual checklist coding are consistent. By contrast, we find poor or inconsistent results with the CMP, Wordscores, Wordfish and the dictionary approach. An often-neglected method offers resource efficiency with no loss in validity or reliability: manual coding using checklists.

Why is it important?

Immigration increasingly transpires as an important policy domain to structure contemporary politics. With so many different methods, it can be hard to choose one. With no stakes in the game, we set out to compare different approaches. Automated methods like Wordscores or Wordfish cannot (yet) replace manual coding for all purposes.


Didier Ruedin
Universite de Neuchatel

We needed party positions on immigration and how they develop over time. The Chapel Hill time series started too late for our purpose, so we started to look into party manifestos. There are many methods that draw on party manifestos to deliver party positions.

Read Publication

The following have contributed to this page: Didier Ruedin