I’ve always treated the Eurobarometer surveys as something to dip into occasionally. They regularly cover public opinion in the member states of the EU, with candidate nations like Serbia and Turkey also taking part. Several have dealt with various aspects of education and training, or other issues in which I’m interested such as civic participation, and I’ve cited their results.
Now, though, I wished I’d checked the technical details a bit more thoroughly before quoting the findings. Two German social scientists have gone over the methods used in the surveys, and their findings make uncomfortable reading. Martin Höpner and Bojan Jurczyk set out what they call ten ‘good rules of public opinion survey research’, all of which seem to me broadly aligned with good practice in survey design. They then check in detail selected examples of Eurobarometer surveys, and conclude that they are so poorly designed as to blur the line between research and propaganda.
More specifically, they accuse Eurobarometer of using
incomprehensible, hypothetical, and knowledge-inadequate questions, unbalanced response options, insinuation and leading questions, context effects, and the strategic removal of questions that led to critical responses in previous Eurobarometer waves.
Has this bias been unintended, a simple result of accident or drift? The authors of this study believe not, and conclude with a stark warning that ‘survey manipulation’ simply intensifies the gap between citizens and elites. Eurobarometer is an arm of the European Commission and if Höpner and Jurczyk are even half right, then its value to the research community, as well as the wider public, has been compromised.