Tortured Polling Logic
By Eastan McNeal on May 1, 2009 at 10:00 PM in Christianity, Current Affairs, Religion
On CNN today we see: Survey: Support for terror suspect torture differs among the faithful.
I suspect there may be some pushing involved in that Pew poll, as there may be a coordinated push to break down the cohesiveness of what the democrat party believes is the foundation of the republican party, White evangelical Protestants.
Years ago I worked with Dr. Marvin Kottke, a research professor at the University of Connecticut. Dr. Kottke adopted a couple of booklets I had written on survey statistical sampling methods and analysis presentation for his classes. We all understood the importance of how you select who you survey and how you phrase and order the questions. From him we learned the importance of preventing your desire for a pre-conceived outcome from influencing any of the process going into the survey, collecting the sample and analyzing the results.
Kottke lobbied the Council of American Research Organizations (CASRO) to include “in the box” guidance for researchers. He believed that no study should be released without a boxed description explaining how the survey / poll was conducted. You have seen them. Xyz research interviewed nnn people between this date and that date and the margin of error is x percent with a confidence interval of y. More importantly, he argued, was to state WHO commissioned the study and WHY.
My booklets were written to accompany a computer program I had just released that allowed organizations to tabulate their own surveys. I feared that if people put garbage into my statistical analysis program that they would get garbage out or that they would misinterpret the raw data and present leading findings. If the user of my software was challenged on their methodology they could easily blame the new software so, to shield myself from that end, I offered a free education through these little books. What I did not realize or expect was that people were sometimes using my guides as instructions on how to pre and post doctor the polls to further their agenda.
I was attending a National Ski Areas Association conference in Nashville when I heard Dr. Kottke warn an audience: “Do not tell your researchers what you expect to find with the study you are hiring them to conduct.” No matter how honest we are we fear our desire to satisfy the client, and we don’t really have accurate formulas for calculating out of your quantitative results, our qualitative bias.
I am seeing more and more surveys released without the “box” of information telling me about the research methodology. A more troubling omission is the who and why. Who paid Pew to conduct a study on torture that included questions about religious faith? Why did the client do that? What were they expecting to find?
If the survey would have shown that black agnostic eggheads supported torture more, would the survey results have been released?
That, by the way, is a good question. Most surveys are buried because the results did not support the client’s desired narrative. If the survey showed that religious and non-religious people felt the same way then who gave the researchers the suggestion to break out whites from blacks? And if that did not create a measurable enough difference, why was the evangelical group broken out, from the religious breakout, and why was that the only group breakout published by CNN?
Without answers to the above questions, I doubt the sincerity of the poll, the pollsters, the media that would publish the selected results and, most certainly, the mysterious client who commissioned the poll.
By the way, the information that should have been in the box:
Data from a Pew Research Center survey conducted April 14-21, 2009, under the direction of Princeton Survey Research Associates, among 742 American adults. Other religious groups are not reported due to small sample sizes.
Question wording: Do you think the use of torture against suspected terrorists in order to gain important information can often be justified, sometimes be justified, rarely be justified, or never be justified?
Pew
Pew’s own analysis of this poll showed that of the 742 respondents 188 were republican. They state that a number that low dropped the margin of error to 8% from 4% for the total sample. Also, using Pew’s own data from other studies, the number of people in the U.S. claiming to be evangelical is 19%, including black evangelicals. So the sample in this poll should have included at most 140 evangelical respondents and the margin of error should be stated as a number greater than 8%. These are assumptions because some of these sub-numbers are not published. The charts released indicate they surveyed 174 WEPs
The press is being biased and misleading in their treatment of the study. The above link goes to US News with a headline: Most Evangelicals and Catholics Condone Torture in Some Instances, when the headline actually could have read: Americans Split on Torture. According to the raw numbers 49% of the TOTAL sample says that torture can often or sometimes be justified. 47% say that torture can rarely or never be justified.
What was the intended goal when these cross-tabs were created and released to the public? Was this pure research or do some of you Christians out there now feel less likely to publicly embrace your religion?
by Ronald Scott
* More novels by 
* 
(
)
- You'll have access to 18 million movies, TV shows, songs, magazines, and books, as well as ultra-fast Web browsing with Amazon Silk -- faster than iPad. Full e-mail and document access and a keyboard. Fire can transmit what is on your screen onto a connected TV or monitor without an accessory. It runs Flash (iPad doesn't). 
Pingback: Bible Thumping Torture Lovers? : NO QUARTER