Reproducibility of social science research using differential privacy

What is it about?

To help preserve privacy, the US Census Bureau, Meta, and other prominent organizations have added statistical randomness, or noise, to statistics about people before public release. This technique has triggered heated debate among social scientists about the potentially deleterious impacts of random noise on scientific findings. Replicating 93 social science studies involving regression analyses of aggregate statistics, we find that, under levels of noise commonly used in industry, the impacts of differential privacy on reproducibility are limited. They are smaller than impacts due to typical levels of data error, and smaller than rates of disparity typically observed in replication studies. Our study estimates key trade-offs that could inform policymakers and data curators designing the next generation of privacy-preserving technologies.

Featured Image