Why Surveys? A Critique of the Tools Used to Judge Us All
About the author: Christopher Hutton is a freelance journalist from Bloomington who writes on technology, religion, and the ideas of the day. He currently writes for Christ and Pop Culture and Paste Magazine. He is also the Social Media Manager/Intern for Rivendell Sanctuary, a new education program designed to provide a truly thorough education. Follow Christopher’s blog at http://liter8.net.
Every day I seem to see a new study or statistic being used to prove a point about something. Beginning in the 1980’s, the religious survey organization known as the Barna Group provided data point after data point in order to reveal truths of our culture and how to interact with it.
Most recently, Generations Radio host Kevin Swanson teamed up with Brian Ray in order to record the social and spiritual conditions of the millennial generation in a survey. (You can find the survey at gen2survey.com.)
Now, to gather research is not bad. In fact, it’s the very first thing that self-proclaimed culture-reclaimers should do.
But are they doing it effectively?
The problem with surveys is that they rely on the human language to present and record human behavior. Number concepts are different than word concepts because their definitions are clearer. If you say “five”, then people understand you are discussing a quantity. However, if you say the word “religion”, then the topic becomes fuzzier. Cultural and personal circumstances often cause words to be understood in different ways.
This causes companies to create a lot of ideologically inaccurate surveys. Consider a survey by the Barna Group which seemed to state that “only 4 percent of Christians have a biblical worldview”. This is a big claim, and can seem scary to those fighting for a biblical worldview.
But what does Barna mean by a biblical worldview? This isn’t obvious at first glance. Historically, the concept of a Biblical Worldview has historically had only a few tenets which almost all people who argue for it would agree with (Inerrancy of the Bible, existence of God, Personhood of Christ, etc.), but everything else is flexible. So, was Barna adding political elements to their definition? Were they adding debatable theological ideas into the biblical worldview? It’s hard to know from how a group like Focus on the Family used the Barna Group’s statistics.
Kevin Swanson’s latest study is another clear example of this. If you look at the language used in the study, it does focus on his particular form of Christianity, which emphasizes extreme forms of Conservative Christian theology, as well as an instinctual anti-government bent, an emphasis on the family relationship, and limited options for explaining one’s relationship with God, family, and the Church.
This kind of ideological shifting is dangerous, because it causes the otherwise objective data to be skewed and misbalanced. It will misrepresent its survey-takers. It will also skew the facts.
So, be wary of surveys. Mark Twain once famously stated that “There are three kinds of lies; lies, damned lies and statistics.” While quantitative data is helpful, it is dangerous if it is not true. So track statistics, check the facts, and never let them skew your take on a topic.