Supporting Statement - Part B revisesd

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

In order to address the research questions for this project, we propose survey-based experiments. The experiments will be administered using an Internet-based survey panel. We propose formal experiments that involve variations in question wording and question format. These experiments will be done with 3,500 U.S. adults age 18 and over from an Internet survey panel. The experiments will be designed to directly address the research questions regarding how question wording, content, and format influence response distributions and how respondents’ acceptance of scientific findings is related to the accuracy of their answers to questions about those findings; the results of this study are not intended to produce reliable estimates of scientific knowledge among the U.S. adult population. The questions that will be used in these experiments are based on the findings from expert reviews and cognitive interviews previously undertaken using NSF’s generic clearance.

Overview of KnowledgePanel

This project will be using the KnowledgePanel, created by Knowledge Networks (KN) – a Gfk company. The KnowledgePanel is a probability-based online panel of survey respondents. Panel members are recruited from a frame of residential addresses that covers approximately 97% of U.S. households, and includes persons living in cell phone only households. Households without Internet access are provided a laptop computer and free Internet service so they may also participate in taking online surveys. (Approximately 27% of the U.S. population does not use the Internet or e-mail, and this varies by demographic group, according to Fox and Vitak, 2008.) KnowledgePanel currently consists of about 50,000 adult members (ages 18 and older.)

The response rate for an Internet panel is the product of several factors including components such as the recruitment rate, retention rate, and survey specific response rate. Therefore it is not surprising for cumulative response rates to be quite low (e.g. 5-10%) for Internet panels.

Nonetheless, despite a low response rate, this research will provide useful data for addressing the important questions that animate it. First, data from the research will not be presented as reliable point estimates of scientific knowledge for the U.S. adult population. Rather, the data will be reported as methodological research to understand how and how much features of survey questions on factual knowledge of science influence answers to those questions. The data will enable publication of a methodological sidebar in SEI that will acquaint SEI’s policy-oriented users with consequential methodological considerations that affect accurate interpretation of data on factual knowledge of science. Second, to assess the effect of changes in question wording or response format on answers to the science knowledge questions, it is most important to obtain a diverse and minimally biased sample in terms of demographics such as sex, race, age, and education. Thus, this study is a significant improvement on the representativeness of published literature in this area, which has been conducted on small quota samples or samples from small geographic areas (Sturgis et al., 2008; Mondak and Davis, 2001). Third, the cost of more rigorous sampling methods with substantially higher response rates to ensure representativeness is prohibitive and would preclude performing a study of this kind. For example, data collection alone for 15 minutes per respondent on the General Social Survey (GSS) would cost about $900,000, approximately seven times the cost of data collection using the KnowledgePanel.

B1. Respondent Universe and Sampling Methods

B1a. Sample Universe

The potential respondent universe for the survey is the total number of non-institutionalized, English-speaking adults in the United States. The sampling points, respondent universe, and planned sample are shown below.

Number of sampling points: 114,761,359 households¹

Estimated size of respondent universe: 235,435,530²

Planned sample size: 3,500

B1b. Sampling

Address-Based Sample (ABS) Recruitment Methodology for KnowledgePanel

Recruitment to KnowledgePanel was first conducted in 1999 using random-digit dial (RDD) methods. In 2009, KN initiated the use of an address-based sample (ABS) frame to replace the RDD telephone recruitment. Panel members were recruited using mail-based techniques, with telephone-based refusal conversion. ABS involves probability-based sampling of addresses from the U.S. Postal Service’s Computerized Delivery Sequence File (CDSF). The address sampling from the CDSF is done without replacement. Addresses with matched telephone numbers from the former RDD recruitment samples (limited to the previous five years) are also removed to eliminate duplication.

For all new panel members, demographic information such as gender, age, race/ethnicity, income, education, and for Latino members, language proficiency are collected in an online “profile” survey. This information is used to determine eligibility for specific studies and eliminates the need for gathering basic demographic information on each panel survey. After this survey is completed, the panel member is regarded as active and ready to be sampled for research studies.

Once panel members are profiled, they become “active” for selection for specific studies. Samples are drawn from among active members using a probability proportional to size (PPS) weighted sampling approach.

Knowledge Panel Survey Sampling

Knowledge Networks (KN) will randomly sample from the Knowledge Panel. This sample is suitable for the proposed study (despite the low response rate of individuals to enroll in the panel and lack of a nonresponse bias study) as we are more interested in internal than external validity.

B1c. Weighting

A KnowledgePanel sample begins as an equal probability sample with several enhancements incorporated to improve efficiency. Since any alteration in the sampling process is a deviation from an equal probability selection design, statistical weighting adjustments are made to the data to offset known selection deviations. These adjustments are incorporated in a base weight for each panel member.

There are also several sources of survey error that are an inherent part of any survey process, such as non-coverage and non-response due to panel recruitment methods and to inevitable panel attrition. These sources of sampling and non-sampling error are addressed using a post-stratification weighting procedure to balance the panel on a variety of geographic and demographic dimensions. An enhancement to this adjustment is the inclusion of weights to balance language proficiency among the Hispanic portion of the sample and to balance the proportion of households with and without Internet access. Using primarily U.S. Census Bureau data sources, the weighted panel is thus raked to be statistically representative of the nation’s adult population.

B1d. Expected Response Rate

Metrics for calculating response rates for Internet panel surveys are complicated (Callegaro and DiSogra, 2008). The two major factors driving the overall response rate for the KnowledgePanel are the panel recruitment rate and the survey specific completion rate. The panel recruitment rate refers to the percentage of people who join the panel after being randomly chosen for recruitment. Approximately one in seven people who are recruited decide to join the panel. We expect 60-65% of panel members who are chosen to respond to this survey to complete it. This suggests that we might expect an overall response rate around 8-9%.

As explained above, it is important to note that the purpose of the experiment is not to produce national point estimates. Rather we are interested in understanding methodologically the factors that influence response to the science knowledge questions. In spite of the low response rate, we are confident that the KnowledgePanel can provide a diverse, minimally biased sample for methodological research (Mutz, 2011) in a cost-effective manner.

B2. Procedures for Information Collection

Experimental design

Respondents will be randomly assigned to treatment groups within the experimental design. The specific experiments are described below. The questionnaire is shown in Appendix B.

Research topic 1: Question wording and format experiments for science knowledge questions.

The first experiment, involving questions in Section A of the questionnaire, will address question wording and format of the traditional science knowledge questions. One question variation in the experiment will investigate the extent to which questions testing knowledge of human evolution differ from responses to questions about the theory of evolution that do not involve specific reference to humans. One of the themes that emerged from discussions with experts in the sociology of religion was that religious individuals, particularly those from evangelical or fundamentalist backgrounds, are more likely to reject aspects of evolution that are related to humans than to other species such as plants or micro-organisms. In particular, religious fundamentalists are likely to reject the notion of humans being descended from other species. For example, the existing evolution item asks if the statement “Human beings, as we know them today, developed from earlier species of animals” is true. Religious fundamentalists do not necessarily object to the notion of species changing over time, but may object to the notion of humans evolving from “earlier species of animals.” This may be understood as being in conflict with the literal Biblical account of creation, in which God created everything at once “according to its kind.”

Another question wording variation in the experiment will investigate the best way to describe the scientific account of the origins of the universe. The existing item describes the “big bang” account of the origins of the universe as a “huge explosion.” A different way to cast this account would be to highlight the continuous expansion of the universe. We include one version in the experiment that refers only to expansion of the universe and another version that describes the galaxies moving away from each other.

We are also including question wording variations for two other items. First, we include two versions of the item referring to whether the father or the mother determines a baby’s sex. Some experts who reviewed the questionnaire indicated that this was not exactly correct. Rather it is a chromosome from the father that determines the sex of the baby. This rewording also seemed to make the most sense to participants of the cognitive test also. Second, we include two versions of the forced choice laser item. Our concern with the forced choice item was that simply offering the alternatives of sound waves and light waves may make the question qualitatively easier than the true/false version. Hence we added a version that mentions “some other way” rather than “light waves.”

The different versions for each of the questions mentioned above are shown below in Table 1.

Table 1. Question wording variations.
Format	Original Wording	Alternative Wording 1	Alternative Wording 2
Evolution
True/False (A9)	Human beings, as we know them today, descended from earlier species of animals.	Human beings, as we know them today, descended from earlier species.	Elephants, as we know them today, descended from earlier species of animals.
Forced choice (AA9)	Did human beings, as we know them today, descend from earlier species of animals or did human beings not descend from earlier species of animals?	Did human beings, as we know them today, descend from earlier species or did human beings not descend from earlier species?	Did elephants, as we know them today, descend from earlier species of animals or did elephants not descend from earlier species of animals?
Origin of the universe
True/False (A7)	The universe began with a huge explosion.	The universe has been expanding ever since it began.	The galaxies in the universe have been moving away from each other ever since the universe began.
Forced choice (AA7)	Did the universe begin with a huge explosion or did it begin some other way?	Has the universe been expanding, shrinking, or staying about the same ever since it began?	Have the galaxies in the universe been moving away from each other, moving closer to each other, or staying in about the same place ever since the universe began?
Determination of baby’s sex
True/False (A3)	It is the father’s genes that decide whether the baby is a boy or a girl.	It is a chromosome from the father’s genes that decides whether the baby is a boy or a girl.
Forced choice (AA3)	What decides whether a baby is a boy or a girl? Is it the father’s genes, the mother genes, or both?	What decides whether a baby is a boy or a girl? Is it a chromosome from the father’s genes, from the mother genes, or both?
Lasers
True/False (A4)	Lasers work by focusing sound waves.
Forced choice (AA4)	Do lasers work by focusing sound waves or light waves?	Do lasers work by focusing sound waves or do they work in some other way?

We will also manipulate the response format of the ten basic science knowledge questions. We propose manipulating both the form in which the answer is given (true/false vs. forced choice) and also the extent to which “don’t know” responses are discouraged. One concern with the true/false format is acquiescence bias. That is, respondents may have a tendency to agree (i.e., choose true) with statements that they are presented regardless of the content of the statements (Krosnick, 1999). Another concern is that it may be easier for respondents to guess the correct answer in the true/false format given that there are always only two options. Rewording the items into a forced choice format may help minimize this tendency (Harris and Changas, 1994).

There are different views on the use of don’t know response options for knowledge questions. One view is that “don’t know” responses accurately reflect the views of someone who has very little information on a topic (Sturgis, Allum, and Smith, 2008). Others argue that encouraging “don’t know” responses reduces validity, either by encouraging “satisficing” or by confounding level of knowledge with willingness to guess in the face of some uncertainty (Krosnick, 2002; Mondak, 2001).

The first experiment then includes three factors: question format (True/False vs. Forced Choice), acceptance of don’t know responses (encouraged, accepted, discouraged), and question wording (original vs. alternative). The number of question wording alternatives varies slightly. In some cases there is an original wording versus two alternatives and in other cases there is an original wording and one alternative. Table 2 illustrates the twelve conditions in experiment 1 assuming an original and alternative wording for illustrative purposes. Respondents will be randomly assigned to these conditions.

Table 2. General summary of experiment 1 with two question alternatives.

Treatment	N	Question format	Acceptance of don’t know	Question wording
1	291	True/False	Encouraged	Original
2	291	True/False	Encouraged	Alternative
3	291	True/False	Accepted	Original
4	291	True/False	Accepted	Alternative
5	291	True/False	Discouraged	Original
6	291	True/False	Discouraged	Alternative
7	291	Forced Choice	Encouraged	Original
8	291	Forced Choice	Encouraged	Alternative
9	291	Forced Choice	Accepted	Original
10	291	Forced Choice	Accepted	Alternative
11	291	Forced Choice	Discouraged	Original
12	291	Forced Choice	Discouraged	Alternative

Analysis plan for experiment 1. We will first compare the average number of correct answers across conditions of the question format and don’t know experiments independently. The primary dependent variable in the analysis of the question format experiment is the average number of items that the respondents answer correctly in the true/false and forced choice conditions. We will also compare the average number of correct responses across the three don’t know conditions. In addition, we will compare the average number of don’t know responses across the three conditions. The power analyses shown below in Table 3 demonstrate that we need sample sizes of approximately 100 in each condition to detect a difference between two means. For planning purposes we use a standard deviation of 2.5 for the mean and assume two sided tests for the differences in the mean using a type I error rate (alpha) of .05. The standard deviation was obtained by computing the mean number of don’t know responses across the existing science knowledge questions on the 2010 GSS.

Table 3. Sample size needed to compare differences in means in experiment 1 (Standard Deviation = 2.5, alpha = .05, two-sided test).

Mean difference		Power
	.7	.8	.9
1	79	100	133
2	21	26	34
3	10	12	16

Additional power analyses revealed that we would need sample sizes of approximately 217 in each don’t know by response format condition to detect an interaction between acceptance of don’t know responses and response format. For example, encouraging don’t know responses may discourage guessing in the true/false condition. Therefore we will want to analyze the data to see if encouraging don’t know responses will have a bigger effect on the true/false questions than the forced choice questions.

We will also look at differences in the percent of the evolution and origin of the universe items that are answered correctly. For example, a higher percentage of respondents should answer the alternative version of the evolution item correctly compared to the original version, because the alternative version should be less likely to evoke the religious views of the survey respondents. The power analysis in Table 4 shows that samples sizes of around 1,565 in each question wording condition would be adequate to detect a difference of 5 percent. Much smaller sample sizes are needed to detect differences of 10 percent or more.

Table 4. Sample size needed to compare differences in percent correct (Assumes proportion of approximately .5, alpha = .05, two-sided test).

Mean difference	Power
	.7	.8	.9
.05	1231	1565	2095
.10	305	388	519
.20	134	170	227

Research topic 2: Question context and question format experiments for measuring knowledge of evolution.

Another theme that emerged from the discussions with experts was that it is important to measure an individual’s understanding of evolution and the individual’s acceptance of the theory separately, so that the relationship between the two can be studied. There were several aspects of evolution that the experts thought that people should understand. Although many topics of understanding were discussed, there were a few core concepts that everyone seemed to agree on. These include how species evolve over time, common ancestry, the inheritance of traits, and the idea that change can occur either slowly or quickly.

Fortunately several resources are available to measure knowledge and acceptance. Items may be adapted from sources such as the American Association for the Advancement of Science’s Project 2061 website.³ The item below is about common ancestry; the correct response is underlined.

Which of the following statements is TRUE about the evolution of plants and animals?

A. All plants and all animals share a common ancestor with each other.

B. All plants share a common ancestor, but all animals do not share a common ancestor.

C. All animals share a common ancestor, but all plants do not share a common ancestor.

D. No plants share a common ancestor with each other, no animals share a common ancestor with each other, and no plants share a common ancestor with any animals.

In Section B of the questionnaire, we have adapted these items to be more easily administered in the context of a survey of the general population where respondents are likely to spend less effort answering the question. For example, we present the respondents with a true/false statement like the following:

According to the theory of evolution, all plants and animals, including humans, came from the same life form that lived a long time ago.

We have also created forced choice versions of the items such as the following:

Which statement best describes what the [biologists believe/ the theory of evolution says]?

All plants and animals, including humans, evolved from a single life form that lived a long time ago.

There is no single life form that plants, animals, and humans evolved from.

Hence, we are continuing the response format and don’t know acceptance experiment from experiment 1 in this experiment also. In addition, we are varying the context in which the questions about evolution are asked. In one condition the questions begin with “According to biologists…” and in the other condition the questions begin with “According to the theory of evolution…” This factor allows us to understanding if asking these questions in the context of the theory of evolution may lead some respondents to reject some of the items on religious grounds. Table 6 shows the conditions in experiment 2.

Table 6. General summary of experiment 2.

Treatment	N	Question format	Acceptance of don’t know	Question context
1	291	True/False	Encouraged	Modern biology
2	291	True/False	Encouraged	Theory of evolution
3	291	True/False	Accepted	Modern biology
4	291	True/False	Accepted	Theory of evolution
5	291	True/False	Discouraged	Modern biology
6	291	True/False	Discouraged	Theory of evolution
7	291	Forced Choice	Encouraged	Modern biology
8	291	Forced Choice	Encouraged	Theory of evolution
9	291	Forced Choice	Accepted	Modern biology
10	291	Forced Choice	Accepted	Theory of evolution
11	291	Forced Choice	Discouraged	Modern biology
12	291	Forced Choice	Discouraged	Theory of evolution

These treatment groups are randomized across the participants of research topic 1.

Analysis plan for experiment 2. Once again we will compare the average number of correct and don’t know responses across the conditions as in experiment 1. The power requirements for experiment 2 are similar to experiment 1.

Research topic 3: Measurement of the acceptance of evolution.

Distinct from an individual’s knowledge of evolution is his or her acceptance of various aspects of the theory. One may perfectly understand the theory, but for religious or other reasons choose to reject different aspects of it. The theory may also be rejected because someone thinks that the scientific evidence supporting it is weak. Hence, the acceptance of the theory may partially explain one’s response to items regarding different aspects of the theory of evolution. This makes measurement of the acceptance of evolution important.

Section C of the questionnaire will be administered to all participants. It includes questions to understand whether the respondents accept or reject the theory of evolution and potential reasons for acceptance or rejection of the theory. We have designed these questions to provide an understanding of how individuals who hold specific views answer the science knowledge questions. For example, respondents may reject the theory because they believe the theory is not supported by scientific evidence and others may reject the theory because it conflicts with their religious views. Even within the subset of respondents who evaluate the theory of evolution in light of religious beliefs, there are different points of view. For example, some may object to the theory because the account conflicts with the age of the Earth as understood from scripture. Others may assess the theory based on views about whether acceptance is compatible with the idea that evolution is guided by a divine creator. We will analyze the data by comparing how these different groups answer the different versions of the evolution questions as discussed below in Research Topic 4.

Research topic 4: Assessing validity of alternative question versions.

The inclusion of the knowledge (Section B of the questionnaire) and acceptance (Section C of the questionnaire) items allows for correlational analyses that help us to understand the validity of the experimental items about evolution in Section A of the questionnaire. The correlation of the experimental items with the knowledge and acceptance measures can be thought of as validity coefficients. One would expect that better knowledge items (i.e., items that separate beliefs from knowledge) would have a stronger correlation with knowledge and variables related to knowledge and a weaker correlation with acceptance. This set of analyses will involve nonexperimental comparisons of how respondents with different levels of knowledge of the theory of evolution and acceptance of the theory answer the evolution items. Below is an example of the type of analyses that we would conduct with respect to knowledge of evolution. We may begin by dividing respondents into quartiles according to their level of knowledge about evolution and analyze crosstabs like the one below.

Table 7. Example bivariate analysis between knowledge and answers to the evolution items.

Human beings, as we know them today, descended from earlier species of animals.	Knowledge Quartile
	1	2	3	4
True	P11	P12	P13	P14
False	P21	P22	P23	P24

Elephants, as we know them today, descended from earlier species of animals.	Knowledge Quartile
	1	2	3	4
True	P11	P12	P13	P14
False	P21	P22	P23	P24

The main hypothesis in this type of analysis would be that the relationship between knowledge and answers to the survey item about elephants would be stronger than the relationship between knowledge and the survey item about humans. This would be the case since the human evolution question is more contaminated with beliefs. We can calculate measures of association appropriate for categorical variables (e.g. Gamma) to test this hypothesis. We can also use logistic regression analyses to test for interactions. For example, these models would include the answers to the evolution item as the dependent variable (0=false; 1=true) and the knowledge quartiles as predictors. We could interact the knowledge terms with question version to detect whether the relationship between knowledge and answers to the evolution items is stronger for the question asking about elephants. We can also add terms for the response format of the questions (true/false versus forced choice) in these models.

Similar analyses would be conducted with the acceptance measures. We would begin by examining whether answers to the evolution items vary by simple acceptance of the theory. Then we would look at whether answers to the evolution items varied by specific beliefs.

We can analyze the validity of the different versions of the origins of the universe items by examining their correlations with level of education and number of science courses taken. We can also examine how overall scores on the existing NSF basic science knowledge battery vary by the format of those items and how well score on the battery are predicted by level of education and the number of science courses taken. In these analyses, we may drop the evolution and origin of the universe items and look at overall scores without these two items to avoid contamination from religious beliefs.

B3. Methods to Maximize Response

Response Rates

As a member of AAPOR, Knowledge Networks (KN) follows the AAPOR standards for response rate reporting. However, the AAPOR standards were not established for Web panels. KN’s Chief Statistician, Charles DiSogra, together with Mario Callegaro published an article in Public Opinion Quarterly to provide a standard for Web panel response rate calculations (Callegaro and DiSogra, 2008). For Internet panels based on probability samples, the cumulative response rate can be calculated as shown below.

(Recruitment Rate) X (Profile Rate) X (Completion Rate)

Survey Frequency and Burden

To minimize panel attrition, KN surveys are usually kept short (from 5 to 20 minutes in duration). For surveys requiring more survey minutes, participation is rewarded with a variety of incentives (for example, small cash awards).

Further, steps are taken to ensure that panel members are not overburdened with survey requests. The primary sampling rule is to assign no more than one survey per week to members. This level of survey frequency helps to keep panelists engaged as part of the panel yet does not overburden panel members. On average, most panelists participate in about two surveys a month. This is closer to four per month for panel segments that may be in higher demand depending on the studies being fielded.

B4. Test of Procedures or Methods

In the spring of 2013 cognitive testing was conducted to refine the survey instrument. Cognitive testing was cleared under NSF’s generic clearance package. A total of 40 cognitive interviews were completed. The subjects were recruited primarily from the Washington, DC, Frederick, MD Raleigh, NC, and Tulsa, Oklahoma areas. Subjects were recruited through advertisements in local newspapers, the Internet website Craig’s List, and fliers in local churches. Westat screened respondents by demographics such as age and education and also according to their interpretation of the Bible as the literal word of God since these are the individuals who have the most problems with questions on evolution. Revisions were made to the questionnaire following the interviews.

B5. Contact Information

Westat is overseeing the design and development of the questionnaire and analysis of the data. Dr. Roger Tourangeau (301-294-2828), Vice President, will oversee the project. The data collection contractor is Gfk. Mr. Joseph Garrett (703-830-0613) will monitor the data collection. The staff and management of NSF’s National Center for Science and Engineering Statistics monitor all aspects of the data collection and survey design. The NSF contact person is Dr. Robert Bell (703-292-4977).

References

Callegaro, Mario and Charles DiSogra. 2008. “Computing Response Metrics for Online Panels.” Public Opinion Quarterly 72: 1008-32.

Fox, Susanna and Jessica Vitak. 2008. “Degrees of Access.” Presentation at the Ovarian Cancer National Alliance.

Fricker, Scott, Mirtal Galesic, Roger Tourangeau, and Ting Yan. 2005. “An Experimental Comparison of Web and Telephone Surveys.” Public Opinion Quarterly 69: 370-92.

Harris, Diane K. and Paul S. Changas. 1994. “Revision of Palmore’s Second Facts on Aging Quiz from a True-False to a Multiple-Choice Format.” Educational Gerontology 20: 741-754.

Krosnick, Jon A. 1999. “Survey Research.” Annual Review of Psychology 50: 537-67.

Krosnick, Jon A. 2002. “The Causes of No-Opinion Responses to Attitude Items in Surveys: They Are Rarely What They Appear to Be.” Pp. 87-100 in Robert M. Groves, Don Dillman, et al., eds., Survey Non-Response. New York: Wiley.

Mondak, Jeffrey T. 2001. “Developing Valid Knowledge Scales.” American Journal of Political Science 45: 224-238.

Mondak, Jeffrey T. and Belinda Creel Davis. “Asked and Answered: Knowledge Levels When We Will Not Take “Don’t Know” for an Answer.” Political Behavior 23: 199-224.

Mutz, Diana C. 2011. “Population-Based Survey Experiments.” Princeton, NJ: Princeton University Press.

Sturgis, Patrick, Nick Allum, and Patten Smith. 2008. “An experiment on the Measurement of Political Knowledge in Surveys.” Public Opinion Quarterly 85: 90-102.

Appendix A: Knowledge Networks Informed Consent

Knowledge Networks Privacy Statement

Knowledge Networks Panel Members have been invited to take part in a major national research effort. Members may complete surveys to provide feedback and opinions on a range of political, lifestyle, advertising and other questions, and may contribute other types of data along with other Panel Members. Active Panel Members receive certain benefits from Knowledge Networks in exchange for participating in surveys.

Our Privacy Commitments

Knowledge Networks respects the privacy of every Panel Member. This Privacy Statement outlines the information Knowledge Networks will collect and how we will use that information. This Statement will also tell you how you can verify the accuracy of your Personal Information (defined below) submitted to Knowledge Networks and how you can request that we delete or update your Personal Information.

We've developed our privacy policy from industry guidelines and standards, and local, national, and international laws and requirements. All privacy practices and methods described in this policy apply only insofar as permitted by the applicable standards, laws and requirements. In addition, as a member of the Council of American Survey Research Organizations (“CASRO”), the professional trade association of survey research companies, Knowledge Networks adheres to the mandated CASRO Code of Standards and Ethics for Survey Research. You may visit CASRO, and view the CASRO Code, at www.casro.org.

Thanks again for placing your trust in Knowledge Networks.

Knowledge Networks is a licensee of the TRUSTe Privacy Program. TRUSTe is an independent, non-profit organization whose mission is to build user’s trust and confidence in the Internet by promoting the use of fair information practices. This privacy statement covers the site www.knowledgenetworks.com. Because this Web site wants to demonstrate its commitment to your privacy, it has agreed to disclose its information practices and have its privacy practices reviewed for compliance by TRUSTe.

If you have questions or concerns regarding this statement, you should first contact Privacy Compliance Coordinator at [email protected]. If you do not receive acknowledgement of your inquiry or your inquiry has not been satisfactorily addressed, you should contact TRUSTe at http://www.truste.org/consumers/watchdog_complaint.php. TRUSTe will then serve as a liaison with us to resolve your concerns.

The TRUSTe program covers only information that is collected through this Web site, and does not cover information that may be collected through software downloaded from this site.

Information Collected and How We Use It:

What We Collect

A) Personal and Demographic Information

During the Knowledge Networks recruitment process, Knowledge Networks collects personal information such as your name, address, email address and the names and ages of members of your household (“Personal Information”) via telephone. In addition, Knowledge Networks frequently asks Panel Members for characteristics (such as health problems or consumer habits), attributes, and demographic information (such as age, income, gender) (collectively, “Demographic Information”) via online surveys.

You are in control of the Personal Information and Demographic Information that you provide to Knowledge Networks. We rely on you to provide updates to your Personal Information and Demographic Information. You can:

• Ask for a copy of your Personal Information or Demographic Information.

• Ask for your Personal Information or Demographic Information to be corrected or updated.

• Ask us to remove your Personal Information or Demographic Information from your records.

To do this, please contact us using the links provided later in this statement. Knowledge Networks will use reasonable efforts to provide the requested information to you.

B) Operating Information

Knowledge Networks may, in the course of interacting with Panel Members, gather other types of information from Panel Members (“Operating Information”). For example, as a Knowledge Networks Panel Member, you will receive surveys from Knowledge Networks on a regular basis. These surveys will ask questions about your interests, needs, and attitudes. We will receive your responses to these surveys. If we offer other features, services, or programs in which you explicitly agree to participate, then Knowledge Networks will receive information from those features, services, or programs.

C) Log Files

As is true of most Web sites, we gather certain information automatically and store it in log files. This information includes Internet protocol (IP) addresses, browser type, Internet service provider (ISP), referring/exit pages, operating system, data/time stamp, and clickstream data.

We use this information, which does not identify individual users, to analyze trends, to administer the site, to track users’ movements around the site, and to gather demographic information about our base as a whole.

We do not link this automatically-collected data to personally identifiable information.

D) Cookies

A cookie is a small text file that is stored on a user’s computer for record-keeping purposes. We use cookies on this site. We do not link the information we store in cookies to any personally identifiable information you submit while on our site.

We use persistent cookies. A persistent cookie remains on your hard drive for an extended period of time. You can remove persistent cookies by following directions provided in your Internet browser’s “help” file.

If you reject cookies, you may still use our site, but your ability to use some areas of our site, such as contests or surveys, will be limited.

How We Use Your Information:

A) Personal and Demographic Information

Personal Information may is used to set up e-mail accounts for each household member who is 13 years of age or older (if you do not have Internet access at the time of recruitment), to communicate with you and your household members, and to assist you with questions that you may have about the Knowledge Networks Panel. It may also be used for panel recruitment, contest entry processing or delivery of free gifts to members of the Knowledge Networks Panel, and delivery of our quarterly newsletter, which you are automatically enrolled to receive via email. (To opt out of the newsletter, please click here: http://members.knowledgenetworks.com/Profile.aspx.)

Personal Information concerning location or address is also used to ensure that our panel accurately represents the country’s population as a whole.

Panel Members are asked for Demographic Information in order to pre-qualify members or households for surveys that target specific groups. We also use this information to ensure that our panel accurately represents the country’s population as a whole.

Knowledge Networks uses Personal Information and Demographic Information solely in the conduct of its research business. Personal Information or Demographic Information may be combined with information collected about you by Knowledge Networks or third parties with your express permission, with information that is collected about you from public records, or with information that Knowledge Networks may acquire from third parties that have a legal right to provide such information to Knowledge Networks.

B) Operating Information

Knowledge Networks uses Operating Information solely in the conduct of its research business. As is the case with Personal Information and Demographic Information, Operating Information may be combined with information collected about you by third parties with your express permission, with information that is collected about you from public records, or with information that Knowledge Networks may acquire from third parties that have a legal right to provide such information to Knowledge Networks.

Data Sharing and Transfer

A) How We May Share Information

Personal Information regarding Panel Members will never be shared with any third parties without your express permission. We reserve the right, however, to disclose your personally identifiable information as required by law and when we believe that disclosure is necessary to protect our rights and/or comply with a judicial proceeding, court order, or legal process served on our Web site. In addition, occasionally Knowledge Networks will share contact information, such as name and address, with third parties with whom we have partnered to provide specific services to Knowledge Networks, or services on behalf of Knowledge Networks, such as panel recruitment, contest entry processing or delivery of free gifts to Knowledge Networks Panel Members. These partners have agreed not to share or resell this data and they are contractually obligated not to use any personally identifiable information except for the purpose of providing these services, unless you enter into a relationship with them that would directly allow them to do so.

Knowledge Networks collects and shares Demographic Information and Operating Information with its research clients in an anonymous form. Our research clients will never receive Personal Information of Panel Members, nor will they be able to identify Panel Members, without your express permission.

Knowledge Networks is an expanding business, and like other companies, we sometimes acquire or divest business units. As part of such transfers, we may convey the business assets of the particular business unit, including Personal Information, Demographic Information, or Operating Information of Panel Members. In that eventuality, you would be notified via email.

B) Data Transfers

Your Personal Information, Demographic Information, and Operating Information will generally be stored in our Knowledge Networks databases, which are located in the United States. For easier processing of e-mail communications, contests, sweepstakes, or other marketing purposes, however, your Personal Information, Demographic Information, or Operating Information may be sent, usually on a temporary basis, to countries outside the United States or the European Union. Knowledge Networks data protection standards are the same, regardless of where your information is stored.

Knowledge Networks adheres to United States/European Union Safe Harbor principles. If you feel that Safe Harbor privacy principles may have been violated by Knowledge Networks, you may contact Knowledge Networks or CASRO, 3 Upper Devon, Port Jefferson, NY 11777 (email: [email protected]) (telephone: 631-928-6954) with your complaint. Knowledge Networks is a member of CASRO.

Children’s Privacy

Knowledge Networks believes that it's especially important to protect children's privacy online and encourages parents and guardians to spend time online with their children to participate and monitor their Internet activity.

Knowledge Networks complies with all applicable national and international children's privacy regulations.

We do not permit children who are under 13 years of age to become Knowledge Networks Panel Members. We do not collect any Personal Information from children under 13 years of age. As part of the registration process for new Panel Members, we collect from the new Panel Member the names of each individual in the Panel Member’s household, which may include the names of children who are under 13 years of age. Occasionally we may send a survey to a Panel Member who is a parent or guardian of a child under the age of 13 that asks that Panel Member to have his or her child who is under 13 answer the survey. We take reasonable steps to ensure parental consent to such procedure by sending the survey to the parent or guardian’s password-protected email address. The information collected in response to such surveys is not combined with identifying information about the child. In every case such survey does not collect Personal Information about the child. We comply with the Children’s Online Privacy Protection Act of 1998.

If a Panel Member has provided us with Personal Information about a child in the Panel Member’s household who is under the age of 13, a parent or guardian of that child may contact us at the email address or mailing address listed at the bottom of this Privacy Statement if he or she would like this information deleted from our records. We will use reasonable efforts to delete the child's information from our databases.

If you would like to contact us about your or your child's Personal Information, or to find out how you can have your child's Personal Information removed from our database, please see the “Access To Your Information and Complaints” section later in this document.

Data Security and Responsibility

Knowledge Networks is committed to keeping the data you provide us secure and will take reasonable precautions to protect your Personal Information from loss, misuse or alteration. Vendors, contractors, or partners of Knowledge Networks who have access to your Personal Information in connection with providing services for Knowledge Networks are contractually required to keep the information confidential and are not permitted to use this information for any other purpose than to carry out the services they are performing for Knowledge Networks.

Knowledge Networks also safeguards Personal Information, Demographic Information and Operating Information from unauthorized access. Most Operating Information is maintained in databases that are separate from those containing Personal Information and Demographic Information. Only authorized Knowledge Networks employees or agents carrying out permitted business functions are allowed to access these databases. In addition, each employee of Knowledge Networks is required to sign a confidentiality agreement requiring him or her to keep confidential all Personal Information of Panel Members. Employees who violate the confidentiality agreement are subject to disciplinary actions, including termination when appropriate.

Access to Your Information and Complaints

A) Accessing, Correcting, Updating and Preventing Use of Your Personal Information or Demographic Information

If you have submitted Personal or Demographic Information to Knowledge Networks through the Knowledge Networks Panel or otherwise, or if someone else has otherwise submitted your Personal or Demographic Information to Knowledge Networks, you can:

1) Access, Correct, Update Your Personal or Demographic Information, by clicking here, emailing your request to: [email protected], or by sending us a letter to the address listed below. Knowledge Networks will use reasonable efforts to supply you with the information you requested to access and to correct any factual inaccuracies in this information.

2) Prevent Further Use of Your Personal or Demographic Information, by emailing your request to: [email protected]. Alternatively, you may call us at 1-800-782-6899 or send us a letter to the address listed below. In each case, we will then use reasonable efforts to prevent further use of your Personal Information or Demographic Information in our files.

B) Complaints

Knowledge Networks is committed to working with consumers to obtain a fair and rapid resolution of any complaints or disputes about privacy. Please send us your questions or comments regarding our privacy practices by emailing us at: [email protected], or send us a letter to:

Privacy Compliance Coordinator

Knowledge Networks, Inc.

Ashley Business Park, Bldg. G

570 South Avenue East

Cranford, NJ 07016

Knowledge Networks will be happy to respond to your questions and comments.

Complaints may also be addressed to CASRO by emailing it at [email protected] , by telephoning it at (631) 928-6954, or by writing to:

CASRO

3 Upper Devon

Port Jefferson, NY 11777

Opting Out

At any time, you may discontinue your participation in the Knowledge Networks Panel by sending us an email ([email protected]) or by calling us at 1-800-782-6899. Parents or legal guardians of any member of the Knowledge Networks Panel who is under the age of eighteen may discontinue the participation of such child in the same manner. Please note that opting-out of the Knowledge Network Panel will not automatically cause us to delete Personal, Demographic, or Operating Information about you (or your child) that we have previously collected. We will do so, however, at your request, as set forth above.

Other Sites

Knowledge Networks is not responsible for the privacy practices or the content of other Websites. We recommend that you carefully read the privacy policies of each site you visit.

Contact Us

You may contact us at:

Knowledge Networks, Inc.

P.O. Box 40

Cranford, NJ 07016
USA

1-800-782-6899

[email protected]

Your Acceptance of Our Privacy Practices

By agreeing to become a Knowledge Networks Panel Member or participating in a Knowledge Networks’ service or program, you signify your acceptance of the terms and conditions of this Privacy Statement. Knowledge Networks may make changes to this Privacy Statement from time to time. We will post changes to our Privacy Statement on the Knowledge Networks Panel Member Web Site, so from time to time please check the Privacy Statement posted there. In addition, we will alert you to significant changes to this Privacy Statement by providing you with written, electronic, or other notice from Knowledge Networks.

Appendix B: Questionnaire

True/False Versions of the Questions

[DON’T KNOW ENCOURAGED] Next, you will see a few short questions like those you might see on a television game show. For each statement, please select true or false. If you don't know or aren't sure, select don’t know and go to the next question.

[DON’T KNOW ACCEPTED] Next, you will see a few short questions like those you might see on a television game show. For each statement, please select true or false. If you don't know or aren't sure, just skip to the next question.

[DON’T KNOW DISCOURAGED] Next, you will see a few short questions like those you might see on a television game show. For each statement, please select true or false. If you don't know or aren't sure, just take your best guess.

 

A1. The center of the Earth is much hotter than the surface of the Earth.

 

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

A2. All radioactivity is man-made.

 

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

A3. It is the father’s genes that decide whether the baby is a boy or a girl.

 

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

A3_Alternative. It is a chromosome from the father’s genes that decides whether the baby is a boy or a girl.

 

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

A4. Lasers work by focusing sound waves.

 

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

 

A5. Electrons are smaller than atoms.

 

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

 

A6. Antibiotics kill both viruses and bacteria.

 

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

A7. The universe began with a huge explosion.

 

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

A7_ALTERNATIVE 1: The universe has been expanding ever since it began.

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

A7_ALTERNATIVE 2: The galaxies in the universe have been moving away from each other ever since the universe began.

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

A8. The continents on Earth have been slowly moving for millions of years.

 

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

A9. Human beings, as we know them today, descended from earlier species of animals.

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

A9_ALTERNATIVE 1. Human beings, as we know them today, descended from earlier species.

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

A9_ALTERNATIVE 2. Elephants, as we know them today, descended from earlier species of animals.

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

A10. The Earth goes around the Sun.

 

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

[ASK IF A10 = TRUE] A11. It takes one month for the Earth to go around the Sun.

 

True

False

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

Forced Choice Versions of the Questions

[DON’T KNOW ENCOURAGED] Next, you will see a few short questions like those you might see on a television game show. For each question , please select the answer that comes closer to your point of view. If you don't know or aren't sure, select don’t know and go to the next question.

[DON’T KNOW ACCEPTED] Next, you will see a few short questions like those you might see on a television game show. For each question , please select the answer that comes closer to your point of view. If you don't know or aren't sure, just skip to the next question.

[DON’T KNOW DISCOURAGED] Next, you will see a few short questions like those you might see on a television game show. For each question, please select the answer that comes closer to your point of view. If you don't know or aren't sure, just take your best guess.

AA1. Which is hotter – the center of the Earth or the surface of the Earth?

 

Center of the Earth

Surface of the Earth

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

 

AA2. Is all radioactivity man-made or does some occur naturally? {ROTATE ORDER OF RESPONSE OPTIONS IN EXPERIMENT}

 

All man-made

Some natural

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

 

AA3. What decides whether a baby is a boy or a girl? Is it the father’s genes, the mother genes, or both? {ROTATE ORDER OF RESPONSE OPTIONS IN EXPERIMENT}

 

The father’s genes

The mother’s genes

Both the mother and the father’s genes

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

AA3_Alternative. What decides whether a baby is a boy or a girl? Is it a chromosome from the father’s genes, from the mother genes, or both? {ROTATE ORDER OF RESPONSE OPTIONS IN EXPERIMENT}

 

Chromosome from the father’s genes

Chromosome from the mother’s genes

Chromosome from both the mother and the father’s genes

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

AA4. Do lasers work by focusing sound waves or light waves? {ROTATE ORDER OF RESPONSE OPTIONS IN EXPERIMENT}

 

Sound waves

Light waves

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

AA4_ALTERNATIVE. Do lasers work by focusing sound waves or do they work in some other way?

 

Sound waves

Some other way

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

AA5. Are electrons smaller, larger, or the same size as atoms?

 

Smaller

Larger

The same size

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

AA6. Do antibiotics kill bacteria, viruses, or both? {ROTATE ORDER OF RESPONSE OPTIONS IN EXPERIMENT}

 

Bacteria only

Viruses only

Both

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

AA7. Did the universe begin with a huge explosion or did it begin some other way?

 

With a huge explosion

Some other way

[Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

AA7_Alternative 1. Has the universe been expanding, shrinking, or has it stayed about the same ever since it began?

 

Expanding

Shrinking

Stayed the same

[Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

AA7_Alternative 2. Have the galaxies in the universe been moving away from each other, moving closer to each other, or have they stayed in about the same place ever since the universe began?

 

Galaxies have been moving away from each other

Galaxies have been getting closer to each other

[Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

AA8. Have the continents on Earth been slowly moving for millions of years or have they always been where they are now? {ROTATE ORDER OF RESPONSE OPTIONS IN EXPERIMENT}

Have been slowly moving

Have always been where they are now

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

AA9. Did human beings, as we know them today, descend from earlier species of animals or did humans not descend from earlier species of animals?

Humans descended from earlier species of animals

Humans did not descent from earlier species of animals

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

AA9_ALTERNATIVE 1. Did human beings, as we know them today, descend from earlier species or did humans not descend from earlier species?

Humans descended from earlier species

Humans did not descent from earlier species

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

AA9_ALTERNATIVE 2. Did elephants, as we know them today, descend from earlier species of animals or did elephants not descend from earlier species of animals?

Elephants descended from earlier species of animals

There has been elephants as long as there have been animals

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

AA10. Does the Earth go around the Sun, or does the Sun go around the Earth? {ROTATE ORDER OF RESPONSE OPTIONS IN EXPERIMENT}

 

Earth goes around the Sun

Sun goes around the Earth

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

[ASK IF AA10 = EARTH GOES AROUND THE SUN] AA11. How long does it take for the Earth to go around the Sun: one day, one month, or one year?

 

One day

One month

One year

Don’t know [FOR DON’T KNOW ENCOURAGED CONDITION ONLY]

[FOR DON’T KNOW DISCOURAGED: IF NO ANSWER IS ENTERED AND RESPONDENT TRIES TO ADVANCE, ASK: “Please take your best guess”]  

B1_INTRO1. The next questions are about what biologists believe. Please indicate whether each of the following is True or False according to what biologists believe.

B1_INTRO2. The next questions are about the theory of evolution. Please indicate whether each of the following is True or False according to the theory of evolution.

{ROTATE ORDER OF A-I} {CONTINUE DK EXPERIMENT}

True/False Format

	True	False
A. According to [biologists, the theory of evolution], all members within a species are equally able to find food and reproduce.
B. According to [biologists, the theory of evolution], even species that look different have many of the same genes.
C. According to [biologists, the theory of evolution], bacteria can inherit genes that make them resistant to drugs.
D. According to [biologists, the theory of evolution], all plants and animals, including humans, came from a single life form that lived a long time ago.
E. According to [biologists, the theory of evolution], traits that an animal develops during its lifetime will be passed on to its offspring.
F. According to [biologists, the theory of evolution], all dogs and cats came from a single life form that lived a long time ago.
G. According to [biologists, the theory of evolution], a plant can develop a new trait during its lifetime, like resistance to a pest.
H. According to [biologists, the theory of evolution], lizards with traits that help them avoid predators are more likely to survive and reproduce than other lizards.
I. According to [biologists, the theory of evolution], a population of birds that lives in two different places may slowly adapt to those places and over many generations become two different species.

Forced choice format

A. Which statement best describes what [biologists believe/the theory of evolution says]?

1. All members within a species are equally able to find food and reproduce.

2. Some individual members of the same species are better able to find food and reproduce than others.

B. Which statement best describes what [biologists believe/the theory of evolution says]?

1. Species that look different share many of the same genes with each other.

2. Species that look different share few or no genes with each other.

C. Which statement best describes what [biologists believe/the theory of evolution says]?

1. Bacteria can inherit genes that make them resistant to drugs.

2. Bacteria cannot inherit genes that make them resistant to drugs.

D. Which statement best describes what [biologists believe/the theory of evolution says]?

1. All plants and animals, including humans, came from the same life form that lived a long time ago.

2. There is no single life form that plants, animals and humans all came from.

Biologists often think about traits that a life form develops during its lifetime (such as big muscles) versus traits an animal inherits from its ancestors (such as blue eyes).

E. Which statement best describes what [biologists believe/the theory of evolution says]?

1. Animals pass along traits that they develop during their lifetime to their offspring.

2. Animals pass along traits that they inherit to their offspring.

F. Which statement best describes what [biologists believe/the theory of evolution says]?

1. All dogs and cats came from a single life form that lived a long time ago.

2. There is no single life form that dogs and cats evolved from.

G. Which statement best describes what [biologists believe/the theory of evolution says]?

1. A plant can develop a trait, like resistance to a pest, during its lifetime.

2. A plant cannot develop a trait like resistance to a pest, but can inherit this trait.

H. Which statement best describes what [biologists believe/the theory of evolution says]?

1. Lizards with traits that help them avoid predators are more likely to survive and reproduce than other lizards.

2. All lizards are equally likely to survive and reproduce.

I. Which statement best describes what [biologists believe/the theory of evolution says]?

1. A population of birds that lives in two different places may slowly adapt to those places and over many generations become two different species.

2. A population of birds that lives in two different places may slowly adapt to those places, but they will always be the same species.

Section C Introduction. Now we would like to ask you a few questions about your view about the theory of evolution.

C1. Do you, personally, accept the theory of evolution, reject the theory of evolution or do you not have an opinion?

1. Accept – GO TO C3

2. Reject

3. Do not have an opinion – GO TO C3

C2. How important are each of the following factors in shaping your beliefs about the theory of evolution?

	Very Important	Somewhat important	Not too important	Not at all important
Quality of the scientific evidence
Conflict or consistency with your religious beliefs?
Your education or what you learned in school
Your family’s beliefs

C3. Please type in your own words why you [accept/reject] the theory of evolution?

_________________________________________________________________

C4. Which of the following statements comes closest to your views on the origin and development of life?

1. God created all life but has not guided the process of evolution.

2. God created all life and continues to guide the process of evolution.

3. God created all life and all species as they are today.

4. God had no role in the process of evolution.

C5. Which statement comes closer to your own view?

1. The universe is probably about 10,000 years old.

2. The universe is probably billions of years old.

C6. Which statement comes closer to your own view?

1. Life as we know it most likely developed through random natural processes.

2. Life as we know it most likely developed through the guidance of an intelligent power.

C7. How informed would you say you are about the theory of evolution?

1. Very informed

2. Somewhat informed

3. Not too informed

4. Not informed at all

C8. What is the highest level of education that you have completed?

1. None, or grade 1-8

2. High school incomplete (grades 9-11)

3. High school graduate (grade 12)

4. GED

5. Business, technical, or vocational school other than high school

6. Some college, no 4-year degree

7. College graduate, (B.S., B.A., other 4-yr. Degree)

8. Master's degree, Ph.D..; law medical or other professional degree

CONTINUE IF R HAS HIGH SCHOOL DIPLOMA OR ABOVE EDUCATION, OTHERWISE SKIP TO CLOSING

C9. Have you ever taken any college-level science courses?

1. Yes

2. No

C9a. (IF YES TO C9) How many college-level science courses have you taken?

______ Number of courses

C10. Now think about the courses you took in high school. What was the highest level of math you completed in high school?

1. No math in high school

2. General math, business, or vocational math

3. Pre-algebra

4. One year of algebra

5. Two years of algebra

6. Geometry (plane or solid or both)

7. Trigonometry/Linear Programming/Analysis

8. Pre-Calculus

9. Calculus

10. Statistics

11. Other (Specify ___________________________________)

C11. Did you take a high school biology course?

1. Yes

2. No

C12. Did you take a high school chemistry course?

1. Yes

2. No

C13. Did you take a high school physics course?

1. Yes

2. No

Closing: Thank you for taking the time to complete this survey.

1 http://quickfacts.census.gov/qfd/states/00000.html

2 http://quickfacts.census.gov/qfd/states/00000.html

3 See http://assessment.aaas.org/topics/EN#/ for examples.

31