Can we trust the Eurobarometer surveys?

eurobarom

From Eurobarometer 75, one of the reports analysed by Hoepner & Jurczyk

I’ve always treated the Eurobarometer surveys as something to dip into occasionally. They regularly cover public opinion in the member states of the EU, with candidate nations like Serbia and Turkey also taking part. Several have dealt with various aspects of education and training, or other issues in which I’m interested such as civic participation, and I’ve cited their results.

Now, though, I wished I’d checked the technical details a bit more thoroughly before quoting the findings. Two German social scientists have gone over the methods used in the surveys, and their findings make uncomfortable reading. Martin Höpner and Bojan Jurczyk set out what they call ten ‘good rules of public opinion survey research’, all of which seem to me broadly aligned with good practice in survey design. They then check in detail selected examples of Eurobarometer surveys, and conclude that they are so poorly designed as to blur the line between research and propaganda.

More specifically, they accuse Eurobarometer of using

incomprehensible, hypothetical, and knowledge-inadequate questions, unbalanced response options, insinuation and leading questions, context effects, and the strategic removal of questions that led to critical responses in previous Eurobarometer waves.
I find their analysis pretty compelling. They give detailed examples of questions that seem to lead respondents directly to express views that are favourable to the European Union. Note that we are not talking here about the way that others – the media, for instance – report the findings, but rather about the very design of the survey questionnaires themselves.
New Picture (2)

From Hoepner & Jurczyk 2015

Has this bias been unintended, a simple result of accident or drift? The authors of this study believe not, and conclude with a stark warning that ‘survey manipulation’ simply intensifies the gap between citizens and elites. Eurobarometer is an arm of the European Commission and if Höpner and Jurczyk are even half right, then its value to the research community, as well as the wider public, has been compromised.

Advertisements

3 thoughts on “Can we trust the Eurobarometer surveys?

  1. Thank you for posting this! It was an useful reference in my article manuscript and feeds into my argumentation on EC’s monopolising the quantitative data production and blurring lines between evaluation/follow ups and research in the area of student progression.

  2. John, thanks for pointing this out. Noteworthy indeed. If only the pro-Europeans would do more to acknowledge the problems that need to be tackled; the last MEP contribution I saw was written as if all was fine with its purpose and institutions….

  3. Pingback: The Disease of Externally Funded Projects in Social Sciences | Polidigwerkschdaeddle

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s