The Patriot Act, ‘support’ for public policies and the construction of democratic control
Not long ago, I argued that how poll questions are often framed, and more important, how they are interpreted in the media, worked to reinforce the status quo, specifically on the issue of mass surveillance.
I’ve since ran across an article (h/t Chris Bowers) that addresses this issue and sheds some important light on my point: Samuel J. Best and Monika L. McDermott, Measuring Opinions vs. Non-Opinions – The Case of the USA Patriot Act (pdf). They investigate whether pollsters are manufacturing opinions on subjects where they don’t exist, in response to the pressure to add public opinion to political debates. In essence, they argue that respondents do not know what the Patriot Act (a complex piece of legislation) does, but use clues from the wording of questions to make up for that ignorance. So what appears to be actual opinions about the law (which for the record, shows very different levels of support depending on the question wording) is simply an artifact.
As they note:
Granted, the question wording used to tap Patriot Act preferences differed from one organization to the next, but research has shown that on familiar issues (e.g. vote choice questions) wording variations do not yield substantial differences in the distribution of responses.
This comparison is important, but I would draw a different conclusion from it. Vote choice isn’t like anything else. Or perhaps a better way of putting it is that how pollsters ask about vote choice is not like anything else. We don’t take some element of a candidate’s platform or background and include it as a cue to the respondent to give them hints about what the candidate stands for (“Do you support experienced, 6 term incumbent Smith or challenger X, whose campaigned in favor of term limits?”) A parallel here would be to ask simply “Do you support the Patriot Act?” although it’s likely this would only tell us that people don’t know what it is. Candidates for office have at least some incentive to make sure voters know about them; it’s not clear anyone has a similar incentive for a complex law, at least as a general matter.
Another important thing to note is that what Best and McDermott are talking about is a specific law, “the Patriot Act” and not policy, or a complex bundle of policies. In fact, their evidence suggests that people do have opinions in relation to policy. In their survey experiment, they use four different explanations of what the Patriot Act is in the question. Interestingly, none are completely accurate reflections of this very complex law. Indeed, it’s hard to imagine how you could convey everything the law does (or for that matter, how to convey provisions whose purpose or effect are contested, but we’ll leave that for another day).
The results in Table 2 demonstrate that the wording variations had a statistically significant effect on the distribution of support.
It appears that the more the Patriot Act is portrayed as invading Americans’ personal lives – going from investigating third party financial records to snooping around people‟s homes – the more support for the legislation dwindles. When it is defined in general terms, though, without mentioning any provision, respondents seemingly interpret is as a less invasive measure, supporting the legislation in large numbers.
That’s not a non-opinion – at least not in relation to the policy question. It’s a perfectly reasonable approach to the problem. Indeed, it’s one embedded in the Fourth Amendment itself.
As for the issue of non-opinions as it relates to the law itself, there is strong evidence. In addition to the above, there’s this:
The proportion expressing no-opinion about the Patriot Act nearly doubled from 24 percent to 41 percent, when a no-opinion option was explicitly offered. Those gravitating to the no-opinion option were nearly entirely drawn from among those seemingly supporting the legislation.
This is why I’m skeptical of claims that X% of people “support” something that are often made. The immediate questions that should be asked are 1) did they have the option of saying I Don’t Know and 2) they might not have enough information to make a judgment.
These results raise an important and frequently reoccurring question in the world of opinion polling: If any question content can influence otherwise uninformed respondents, how can pollsters avoid manufacturing opinion?
The authors don’t answer this question directly, but I think the answer is implicit. Pollsters will do better to ask people about particular policies, rather than support for particular pieces of legislation. To the extent they do the latter, they should not include any partial explanations and make sure to offer the no opinion option.
That said, I think Chris Bowers drew the wrong conclusion from this research. Here’s how he applied it in the present context.
These two questions from the June 9-10 CBS poll on the NSA surveillance program do an excellent job of summing up why it might be impossible to accurately gauge public opinion on this topic (emphasis added):
“In order to reduce the threat of terrorism, do you approve or disapprove of federal government agencies collecting phone records of ordinary Americans?”
“In order to reduce the threat of terrorism, do you approve or disapprove of federal government agencies collecting phone records of Americans that the government suspects of terrorist activity?”
If you say the NSA is collecting phone records of ordinary Americans, then public opinion is solidly against what the NSA is doing. But if you just change “ordinary Americans” to “Americans the government suspects of terrorist activity,” then the public is overwhelmingly in favor.
Chris sees this as question wording effects. But the more likely answer is substantive. People were more supportive of searches of suspects than of everyone. ‘Ordinary Americans’ can be read in different ways–those who are suspects or everyone. If you read this as ‘ordinary Americans who are suspected of terrorist activity’ then question effects would make sense. But especially given the findings of Best and McDermott, it makes more sense to conclude that these two questions are understood as addressing different things–mass surveillance versus suspicion-based surveillance. These raise different legal, ethical and moral issues. What looks initially like non-opinions turns out to be perfectly reasonable .
On another note, we should repeal the Patriot Act, which does little to prevent terrorism, but much to violate civil liberties.