SS Part B July 18 2011

formative research FOR and community uptake of the Eagle Books and youth books for American Indians and Alaska Natives

New

Supporting Statement: Part B

July 18, 2011

Submitted by:

Division of Diabetes Translation

National Center for Chronic Disease Prevention and Health Promotion

Centers for Disease Control and Prevention

Atlanta, Georgia

Lemyra DeBruyn, PhD

Telephone: 505.232.9906

Email: [email protected]

Dawn Satterfield, PhD

Telephone: 770.488.5285

Email: [email protected]

Table of Contents

Part B: Statistical Methods

B1. Respondent Universe and Sampling Methods

B2. Procedures for the Collection of Information

B3. Methods to Maximize Response and Deal with Nonresponse

B4. Tests of Procedures or Methods to be Undertaken

B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Attachments

A-1. Authorizing Legislation: PHSA

A-2. Authorizing Legislation: WIC

B-1. Federal Register Notice

B-2. Summary of Public Comments and CDC Response

C-1. Community Representatives Interview Guide

C-2. Community Representatives Information Sheet

C-3. Community Representatives Consent Form

D-1. School Administrators Interview Guide

D-2. School Administrators Information Sheet

D-3. School Administrators Consent Form

E-1. Teachers Discussion Guide

E-2. Teachers Information Sheet

E-3. Teachers Consent Form

F-1. Parents Discussion Guide

F-2. Parents Information Sheet

F-3. Parents Consent Form

G-1. Children Grades K-1 Discussion Guide

G-2. Children Grades K-4 Parental Consent

G-3. Children Grades K-4 Youth Assent

H-1. Children Grades 2-3-4 Discussion Guide

H-2. Children Grades K-4 Parental Consent

H-3. Children Grades K-4 Youth Assent

I-1. Middle Schooler Youth Books Discussion Guide

I-2. Middle Schooler Parent Information Sheet

I-3. Middle Schooler Parental Consent

I-4. Middle Schooler Youth Assent

J. IRB Approval

K. Site Visits Scheduling Worksheet

L. Observation Checklist

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

The potential respondent universe consists of the six AI/AN communities that received communication and outreach support from NDWP (i.e., “high dose communities”) and four communities that independently adopted and used the Eagle Books without external NDWP support (i.e., “independent uptake” communities). Two of the sites that received onsite communication and outreach support from NDWP (i.e., the Confederated Tribes of Umatilla in Pendleton, OR and the Pueblo communities in Albuquerque, NM) were not selected because they did not have onsite, ongoing Eagle Books promotion. These communities only received support for a more short-term communication campaign and marketing “launch” of the Eagle Books. All four of the “independent uptake” communities have been selected for data collection.

The four “independent uptake” communities were identified primarily through informal technical assistance and consultation activities provided by NDWP and Lorelei DeCora (Eagle Books Talking Circles health educator). Specifically, NDWP and Ms. DeCora have received requests from these tribal communities seeking feedback or guidance on their efforts to use the Eagle Books to prevent diabetes. These communities have also requested large amounts of Eagle Books materials (over 1000 sets of books).

The following eight tribal communities have been selected for site visit:

“High Dose” Communities:

• Lake Superior Band of Chippewa Indians (Keweenaw Bay, MI)

• Ho-Chunk Tribe (Winnebago, Nebraska)

• Seminole Tribe (Ahfachkee Day School, Seminole, FL)

• Lower Kuskokwim Schools; Rocky Mountain Schools; schools served by the Dena’ina Health Clinic (Alaska)

“Independent Uptake” Communities:

• The Chickasaw Nation (Ada, Oklahoma)

• Yankton Sioux (Marty, South Dakota)

• Yakama Indian Nation (Toppenish, Washington)

• Menominee Tribe of Wisconsin (Keshena, Wisconsin)

Respondents will be adults ages 18 and over and children in grades K through 4 and middle schoolers (approximate ages 5 through 13) who are residents of and/or work in these communities. Each year, we expect to collect data from 108 respondents, 104 of whom are participants in focus groups or discussion groups (24 individuals in each of the four sites plus an additional 4 children in two sites each).

Given that this project is using qualitative research methods, purposeful sampling techniques are being used to identify individual respondents. In general, respondents will be selected based on their knowledge of and experience with the Eagle Books.

To facilitate this proccess, NDWP has established contacts at each of the eight selected communities who will assist with the scheduling of interviews and focus groups. Criteria will be sent to each local scheduling facilitator to ensure respondents’ eligibility to participate (see Attachment K). Because we are visiting communities where the Eagle Books have been promoted and used, we expect that each respondent will have had some minimal level of exposure to and awareness of the Eagle Books. However, local scheduling facilitators will be instructed to confirm respondents’ experience with the books, using the scheduling worksheet provided (see Attachment K). Specifically, community representatives, school administrators, and teachers must have experience using the Eagle Books as part of their work. Parents must have some exposure to the books, either having read them or at the very least seen the books. Children must attend schools that have been using the Eagle Books in the classroom.

Scheduling facilitators will submit copies of the completed scheduling worksheets to Westat, the evaluation contractor. The worksheet will not contain any identifying information for any respondents. Westat will review the worksheet to ensure respondents meet established criteria for participating in the site visit. Westat has also designated a site visit coordinator to work with facilitators and arrange all site visit logistics, including scheduling.

2. Procedures for the Collection of Information

Information will be collected using qualitative, case study methodology in eight selected AI/AN communities that currently use the Eagle Books: four communities that received onsite communication and outreach support from NDWP and NIH-supported contractors promoting the DETS curriculum and the accompanying Eagle Books (i.e., “high dose” communities) and four communities that independently adopted and used the Eagle Books without external NDWP support (i.e. “independent uptake” communities).

Data collection will involve focus groups and in-depth interveiws conducted during site visits to eight selected AI/AN communities over a two-year period. Each site visit will involve the following data collection activities:

• Interviews with up to three community health representatives (e.g., health department representatives, community health workers, tribal council members, representatives from the Indian Health Service, etc.).

• Interviews with one school administrator (e.g., school principals) from a local elementary school

• One discussion (focus) group with K-4 teachers from a local elementary school

• Two discussion (focus) groups with parents: one group with parents of younger children (kindergarten and first grade) one group with parents of older children (second, third, and fourth graders).

• Two discussion (focus) groups with children: one group with younger children (kindergarten and first grade) and one group with older children (second, third, and fourth graders).

• Observational tours of the community (e.g., health clinics, recreation centers, tribal offices, school building and grounds, local businesses, etc.).

• Two discussion (focus groups) with middle schoolers (conducted in only four of the eight communities) to gather feedback on draft versions of Youth Books 2 and 3.

Because this data collection uses purposeful sampling techniques and qualitative data collection methods, the findings will not be statistically generalizable to the larger population of American Indians and Alaska Natives. However, we believe that the qualitative data collection techniques we will use provide several advantages. Focus groups, for example, capitalize on the interaction within a group to elicit “real life” responses from participants. The structured group process frequently provides very rich, detailed data as the opinions heard from one participant may stimulate the responses of others. In addition, a skilled moderator can follow up on or probe certain tangents or views that emerge but could not be planned for or anticipated in the design of the discussion guide. This unique aspect of focus group methodology often yields new information and/or additional nuances on existing information that could not be gathered using more quantitative techniques. The group dynamic fostered by this approach is also extremely helpful in eliciting information on the prevailing attitudes, experiences, opinions, and values of the target audience and provides the opportunity to explore new, emerging, or divergent opinions to increase understanding of an audience.

Both focus groups and interviews will also allow us to identify potential communication concepts, strategies, and approaches for effectively reaching AI/AN audiences, in their own words. Specifically, understanding current sources of information on diabetes prevention and how AI/AN audiences conceptualize or articulate issues related to diabetes can serve as the basis for future promotion and outreach strategies, as well as materials and products.

Information will be collected by staff from Westat (the evaluation contractor) as well as two consultants on the project, Dr. Carolee Dodge-Francis and Lorelei DeCora. Information will be collected using in-person interviews and focus groups. Two to three-day site visits will be conducted to four selected AI/AN communities each year, for the remaining two years of the project. Ideally, we would like to visit all eight communities as soon as possible. However, given the fact that much of the data collection involves schools (teachers, administrators, and school children), we can only visit sites during the school year. In an effort to reduce burden on communities, we will also need to avoid conducting site visits during the first month schools are in session, the winter holidays, spring break, and spring testing. Thus, we anticipate that on average, we will visit four sites per year, to allow for maximum participation and reduce burden on communities.

3. Methods to Maximize Response Rates and Deal with Nonresponse

NDWP will employ several strategies to maximize responses. First, reminder calls will be used to maximize attendance at interviews and focus groups. Second, every effort will be made to conduct all interviews and focus groups at times and locations convenient for respondents. For example, the early evening hours (after work) are often most convenient for collecting information from parents and children. In our experience, employees (e.g., community representatives or school staff) often prefer times before work, during lunch, or immediately after work. In addition, these audiences often prefer interviews conducted at or near their place of employment. Every effort will also be made to identify community settings, such as local churches, community centers, or schools for focus groups with parents and children as these locations are typically convenient and familiar. Scheduling facilitators in each community will assist with determining the times and locations of interviews and focus groups, to minimize respondent burden.

Monetary payment will also be used to maximize response rates and minimize barriers to participation. Parents will receive $70 for participating in the discussion (focus) groups. This amount has been the standard and usual level of reimbursement for audiences in similar CDC funded activities. This amount was also determined based on 1) the time commitment and effort asked of parents in relation to the average hourly rate of US workers; 2) the cost of childcare for children not participating in the focus groups; and 3) the fact that data will be collected from parents as well as their children. Further, the $70 will also help defray transportation costs associated with attending the focus groups. The AI/AN communities that will be visited are located in remote areas. In many cases, residents travel from 10 to 20 miles to reach a town center or other community resources.

Parents of middle schoolers participating in the discussion groups to test the draft Youth Books will receive $35. This amount was determined based on the time commitment and effort asked of parents in relation to the average hourly rate of US workers. The $35 is also intended to help defray transportation costs associated with bringing their children to the focus groups. Children participating in the groups will receive a free set of Eagle Books materials and promotional products.

School administrators and teachers, as state or local government employees, are typically not allowed to receive incentive payments, gifts, or honoraria for activities considered to be part of their official duties or job. Thus, at this time we do not plan to provide any payment or gift directly to these respondents. We do plan to conduct interviews and focus groups on school premises, preferably before school begins, during lunch, or after students are dismissed, so as to minimize transportation burden for school staff. NDWP will also provide participating schools with a set of Eagle Books materials and promotional products.

Given that community health representatives are often employed by local, state, or Federal government agencies, we anticipate that these respondents will likely be ineligible for any payments or gifts for their participation. Thus, NDWP will provide respondents with free copies of Eagle Books materials and promotional products. In the event that a community representative is employed in the private sector, a $70 cash payment will be provided.

4. Tests of Procedures or Methods to be Undertaken

Although the data collection instruments have not been formally pilot tested, each has been reviewed by consultants to the project: Dr. Carolee Dodge-Francis, L. Carson Henderson, and Dr. Lynn Aho. Dr. Dodge-Francis is Executive Director of the American Indian Research and Education Center at the University of Nevada, Las Vegas and currently trains teachers in implementing the DETS curriculum in Nevada and surrounding states. L. Carson Henderson is Associate Professor of Research and Project Coordinator at the American Indian Diabetes Prevention Center at the University of Oklahoma Health Sciences Center. Dr. Lynn Aho is a researcher at the Keweenaw Bay Ojibwa Community College. Dr. Dodge-Francis, Ms. Henderson, and Dr. Aho have reviewed the instruments to ensure that the content, wording, format, overall flow/ordering of questions, and timing are appropriate.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Data collection and analysis will be performed by Westat staff, as well as Dr. Carolee Dodge-Francis and Lorelei Decora, project consultants. Neither Dr. Carolee Dodge-Francis nor Lorelei DeCora will conduct site visits in communities in which they have participated in any Eagle Books promotion or outreach activities.

As the research and evaluation contractor, Westat will analyze interview and focus group responses, as well as observational data, across the eight communities visited. Findings will be examined to identify trends and patterns as well as differences in overall responses and patterns and differences by type of respondent. Westat will use a notes-based analysis process similar to that recommended by Krueger (1994) and Patton (2002) for the analysis of qualitative data:

• Content analysis will involve the identification, labeling, and categorization of data. This analysis reviews records from data collection sources (notes, top line reports, observations, etc.) to identify trends across respondent categories or groups.

• Thematic analysis will determine if certain themes emerge from patterns identified in the content analysis. These themes may be articulated directly by the participants or identified by the study team. In addition to assisting with the identification of patterns and themes in the data, content and thematic analyses also allow large amounts of collected data to be reduced or distilled into a more easily studied and understood format.

  • Logical analysis yields patterns of difference, as well as similarities that may emerge from cross-classifying the data. This step is helpful in identifying patterns or themes that are not immediately obvious, while also taking into consideration the outlier opinions.

Westat will also use text analysis software, Atlas Ti, to facilitate data analysis. Atlas Ti allows users to electronically label selected text using a set of user-defined codes or labels representing certain themes, concepts, or findings of interest. Once selected text from a transcript is coded, Atlas Ti allows users to conduct various analyses and comparisons of coded responses across or between focus group transcripts. Westat staff have received training on Atlas Ti and have used the text analysis software to analyze data from over 100 focus groups and interviews.

Westat will also develop a codebook that will be used to code the focus group and interview data (notes). To ensure the reliability and validity of the coding process, each data set will be assigned a primary coder, whose coding will be reviewed by a secondary coder. A team of two coders will serve as primary coders on half of the data sets and secondary coder on the other half.

Any discrepancies in the coding of data will be resolved using a consensus approach, with input from the Westat project director, as needed.

11