Combining historical data. The population of Belgian cities from 1880 to 1970.

What is it about?

In order to get historical data on the size and growth of cities, we are often forced to choose between quality and availability: the higher the quality of the source, the less frequent it is collected. Census data for example is highly qualitative, but in the case of Belgium only collected every ten years. A more readily available source of data are the population records (deaths, births and migration) that are continuously collected by each individual city. However, this information often contains many errors. Changes in administrative boundaries of cities and mergers only further complicate matters. Our paper shows how these sources can be combined in an automated statistical way, that takes into account the reliability of each source, differences in the type of data (growth vs. levels), and qualitative information such as administrative changes. We showcase our technique by computing the yearly population data for 2600 Belgian cities from 1880 to 1970.

Featured Image

Why is it important?

With the continuing digitization of historical data sources, historians are presented with the tantalizing possibility of combining ever larger datasets in their research. However, even though they now are relieved from the time-consuming business of putting in data by hand, the digital datasets they can work with present their own problems. Combining these increasingly large datasets requires newly developed statistical methods, which are not yet readily available. In this article, we present one such method, a state-space model, that succeeds in merging the sources in a thoughtful way, taking into account the precise characteristics and flaws of each source.