Project Research: Round Two
There has long been concern that the way journalists cover potentially divisive topics could do more to push people apart than bring them closer to mutual understanding. Journalists tend to treat issues in a dichotomous fashion and present them in sharp contrast, often emphasizing conflict and including perspectives from those with strong views on one side or the other. Left out of the media conversation are more moderate views and voices that seek common ground.
Religion and science represent two ways of knowing that are often portrayed as being at loggerheads in the media. The storyline is that the two are opposing worldviews, one undercutting the other and inherently in conflict. But there are other ways of looking at the relationships. Some see the two as involving separate spheres, with religion addressing the spiritual and science the physical. Others, still, see the two as complementary ways of looking at the world, with each informing the other and jointly helping humanity to achieve knowledge and wisdom. When journalists cover points of contact between science and religion, we can see, then, there are several ways of looking at the relationship between the two that they could draw upon.
With the support from the John Templeton Foundation, we developed an online experiment to examine how different ways journalists might present the relationship between religion and science might affect the way people with strong views on the issue react to the media and view differences in opinion between each other and the public. The experiment tested two features of content, viewpoint representation and tone. We considered three approaches to representing the range viewpoints: a “dualist” approach with two strongly opposing sides that view religion and science as incompatible, a middle-view approach with perspectives that portray religion and science as not in conflict, and a “full-spectrum” approach that merged the dualist and middle-view approaches. For tone, we considered a high-conflict tone that used war and sports metaphors to emphasize conflict between the two, and a low-conflict tone that used a more muted language.
Past research has shown that people with strong views tend to see balanced media content as biased against them or hostile to their views—a phenomenon known as the hostile media effect. But what happens when we change the array of viewpoints represented or the tone of the coverage? Other research has shown that with political issues, presenting an issue in terms of conflict can lead to a greater perception of conflict in public opinion and a perception that different sides are further apart in how they look at an issue. But is this due to the language used, or the distribution of viewpoints expressed? Our goal in this study is examine how changing the way journalists present potentially divisive topics might affect way people with different viewpoints in a debate view each other and the media itself.
Below, we provide an overview of those who responded to our survey, the methods we are using to address these questions and show how different groups responded to key variables designed to answer the above questions.
In order to find people with varying views on science and religion, we recruited participants for our study from two large national organizations in addition to a representative sample of Americans. The two groups include the Freedom From Religion Foundation (FFRF) and BioLogos. The FFRF describes itself as an “association of freethinkers,” including atheists, agnostics and skeptics, with a purpose of “protecting the constitutional principle of separation of state and church” and to “educate the public on matters relating to nontheism.”
BioLogos is an organization that in its words, “invites the church and the world to see the harmony between science and biblical faith as we present an evolutionary understanding of God’s creation.” The groups invited members to participate through their organizational email lists.
A total of 3194 participated from FFRF and 659 participated from BioLogos. A national sample of Americans was obtained through Survey Sampling International, a respected survey firm often used for experimental and marketing research. In order to ensure we had enough people interested in the topic of religion, we drew participants from three populations, those with high interest in science (439), those with high interest in religion (438), and a representative sample (813).
Key variables of interest for us were the extent to which people aligned with a spiritual/religious worldview and with a scientific worldview.
We described the spiritual/religious worldview in a prompt as the belief that “spiritual or divine forces are at work in the universe, and that religious and spiritual teachings and explanations are needed to fully understand the origins of the universe, life, and human consciousness.” We described the scientific worldview as the belief that “only natural forces, as opposed to spiritual or divine forces, operate in the universe, and that studying nature through science is the best way to understand the origins of the universe, life, and human consciousness.” Response options for each were (1) “not at all aligned,” (2) “slightly aligned,” (3) “moderately aligned” (4) “strongly aligned,” and (5) “very strongly aligned.”
The strength of spiritual/religious worldview differed by groups. BioLogos reported the highest alignment with the spiritual/religious worldview (mean = 4.26). In contrast, the members of Freedom From Religion Foundation reported the lowest alignment (mean = 1.11). The representative sample from SSI were nearly in the middle (M = 3.19). For scientific worldview, the pattern was reversed, but the groups did not differ to as great a degree. See table 1 for more detail.
Respondents participated in our experiment by visiting an online survey hosted through Qualtrics. After answering some questions about their interest in various topics and questions about their worldviews, participants were randomly assigned to read one of six news articles about the relationship between science and religion. The stories were roughly the same length, just short of 700 words, and were all branded as Reuters news articles. These news stories varied on two dimensions: the range of viewpoints presented in the article and the journalist’s tone when describing the conflict between the two worldviews.
Range of viewpoints. There were three versions of how the news story approached the relationship between science and religion. This was accomplished by quoting sources with differing views on the topic. Though we made small edits to the quotes, the content was gathered from real-world material to ensure that the news story would be as realistic as possible. The pool of nearly 30 quotes was evaluated in a preliminary test by a pool of respondents who rated the extent to which the arguments favored science, religion, or fell somewhere in the middle. The sources behind many of these quotes were important figures on either side of the debate (e.g. Francis Collins of the National Institute of Health, minister Billy Graham and physicist Stephen Hawking).
The first version of the news story in our experiment presented the issue in dualist terms. This version juxtaposed two solid arguments for the religious worldview against two solid arguments for the scientific worldview. The second version of the news story presented the issue in terms of compromise. This version only featured quotes from sources that believe there is middle ground between science and religion. The final version of the story presented an array of views on the issue, so both strong arguments on each “side” and appeals to compromise were included.
Conflict language. There were two versions of conflict language used in the articles. The high-conflict version capitalized on every opportunity to hype up the tension between science and religion, whereas the low-conflict version opted for flat language that avoids fanning the flames of controversy. For example, the headline for the high-conflict version claims that new research “reignites the age-old battle” between science and religion. Contrast that with the low-conflict version, which states that new research “raises questions” about the roles of science and religion. There were 18 instances where the articles differed between high and low conflict languages. Many of these instances were simple word substitutions (e.g. differ versus clash) or short phrase substitutions (e.g. an online dispute versus a firestorm of controversy).
Order and quote content. To be sure any observed effects were due to the distribution of the opinions (dualist, compromise or full array) rather than the content of any one quote, we created six versions of the article for each level of our manipulations. These six versions pulled from a pool of possible quotes and arranged them in a randomized order. For example, every dualist condition featured two pro-science quotes, but those two quotes were chosen from a pool of five possible pro-science quotes. In half of the dualist conditions, the pro-science quote appeared first and in others, the pro-religion quote appeared first. This resulted in a total of 36 version of the stimulus article. For analysis, the responses to each version were combined into groups representing the tone (2 groups) and viewpoint range in the article they viewed (3 groups). A sample of the stimulus is presented in the appendix.
After reading the stimulus, half of the respondents completed a questionnaire in which they were asked rate the article on various criteria and to answer a number of questions related to how they perceive the relationship between science and religion and how they thought others perceive it. The other half were directed to a separate study. We report findings below only for those who were directed to the full survey for the primary study.
The three key outcome variables we for which we describe initial findings are the following:
Article content bias: This question asked respondents “Would you say that the content in this article was strictly neutral, or was it biased in favor of one side or the other?” Response options ranged from (-5) “strongly biased in favor of the scientific worldview” to (5) “strongly biased in favor of the spiritual/religious worldview,” with a midpoint of (0) “strictly neutral.”
Personal view of science/religion conflict: This question asked respondents, “To what extent do you see the relationship between science and religion as one of conflict?” Response options on the 5-point scale ranged from “not at all” to “a great extent.”
Perceived public’s view of science/religion conflict: This question asked, “How much conflict would you say an average American sees in the relationship between science and religion?” Response options on the 5-point scale ranged from “no conflict at all” to “an extreme amount of conflict.”
In the graphs presented below, we show for each of these variables how they differ by group (FFRF, BioLogos and General Population) and by experimental condition (range of viewpoints X tone) . For ease of presentation, we have collapsed the three SSI groups for the general population group. This decision has the effect of cancelling out any difference between those high in science or religious interest. However, because these groups varied on the worldview question less dramatically than FFRF and BioLogos, we believe any differences on the outcome variables of interest would be minimal within the SSI sample.
We will briefly describe some of the general patterns we observed and show them through a series of graphs. Because our analysis remains in the early stage we do not report any statistical tests nor make any formal claims as to what the data show.
Figure 1 shows perceptions of content bias. A clear pattern in the data is that the the amount of conflict differed by group membership. Regardless of the experimental manipulation, the BioLogos members tended to see a bias in favor of the scientific worldview, but less so in the middle-view condition compared to the full-spectrum or dualism conditions. The effect of conflict was less clear among BioLogos members, but seemed to reduce perceived bias for some conditions and increase it for others. FFRF members tended to see bias in favor of the spiritual/religious worldview in the compromise condition, but little bias in the other conditions. High conflict appeared to heighten views among FFRM that the middleview condition is biased. Those in the SSI sample showed no clear evidence they though the content was biased.
Figure 2. shows the extent to which respondents viewed science and religion to be in conflict. Biologos members, on average, saw less conflict than those in the SSI general public sample, who in turn, saw less conflict than those from the FFRF. There was no clear effect of the exponential conditions on perceptions of conflict.
Figure 3 shows the extent to which respondents thought people in the general public viewed science and religion as being in conflict. BioLogos members reported a marginally greater perception of conflict among the general public than the other groups, and the use of high conflict language appeared to slightly increase this view.
This study sought to see whether changes in the way a potentially divisive topic is presented affects views of the media and the extent to which people perceive the level of conflict surrounding the issue. Our test case was science and religion, which can be presented in a variety of ways. A clear pattern was that groups with strong views on science and religion tended to see bias against them in media coverage and the range of viewpoints presented can affect the extent to which bias is seen. We also found media presentation had little effect on the extent to which people personally saw the issue as one of conflict, but mild effects for at least one group on the extent to which they thought the general American public saw conflict.
As these are only preliminary analyses, we urge caution in relying on the results, particularly those where the differences are slight. Some limitations are inherent in the way we analyzed the results. Firstly, we organized people by group, which ignores the differences in worldview within groups, particularly in the general population sample. Another way to approach this problem would be to group people by their worldview to see how the experimental manipulations affect responses among those who are more toward the spiritual/religious and scientific poles and those who see the two as compatible. Finally this report only highlighted a few of the variables that we measured, and we have many more that deal with different dimensions of polarization, group identity and views toward the media to analyze as we seek to answer our questions about the role of media practices in influencing the way people on the several sides of an issue view each other, the public, the media and the nature of the issue itself.
(Last Updated: August 8, 2014)