9-17-07 memo

9/5/07Health Resources and Services Administration (HRSA) response to questions from OMB on the Bright Futures ICR

1. Since obesity and overweight are disproportionately affecting minorities, particularly non-English speaking minorities, would suggest translating the materials (at least into Spanish). Otherwise, the study should clearly state this study limitation. Supporting statement should also clarify that "cultural competency" of the materials cannot be
   assessed.

Response: One of this study’s limitations is that the cultural competency of the materials is not and cannot be assessed. At the time of the release of the contract to conduct an intermediate assessment, Spanish language materials did not exist and the award made was for the evaluation of the English language materials only. The HRSA Office of Women’s Health is in the process of creating Spanish language versions of the materials.

OMB: OK

2. The $1000 incentive appears rather high. Please provide an itemized list of the costs that each site will likely incur as part of this study and why the $1000 incentive payment is therefore justified.

Response: The $1,000 per site is an incentive payment to help ensure participation. It was not intended to fully reimburse all costs but rather to acknowledge time and effort required on the part of each of the 6 sites participating in the assessment. The sites will incur direct expenses that go beyond the normal scope of providing services in their clinic. These expenses include using their administrative and support staff to determine patient eligibility for the materials and the study, recruit clients into the study, handle informed consent, distribute the Bright Futures for Women’s Health and Wellness (BFWHW) materials and assessment form and collect the assessment form (estimated to be more than $6,000 in the OMB Burden Table). There is also a cost associated with the health care providers using the tool in their clinical encounter; this cost was estimated in the OMB Burden Table at more than $21,000.

OMB: For studies at different agencies (Education, for example) where the administrative burden is similar, the incentives have been in the range of $200-$250. This amount seems reasonable for this study as well. Is HRSA amenable to offering an incentive in this range?

HRSA: The sites being recruited for this assessment (faith-based, school-based and Federally-qualified health centers) are likely to be serving the uninsured and are in communities that lack the resources to join in this activity. HRSA believes the $1000 per site incentive is justified because of the nature of the health services provided by the sites, which are different than services provided under contract by the Department of Education.

OMB: Actually, ED studies can be very labor intensive and often occur in underserved areas (e.g. inner city areas where teachers are expected to implement a completely new curriculum) over a sustained period of time (e.g. a school year).

Given the study limitations (and hence the ultimate utility of the data collected), and given the comparable burdens, OMB is still very uncomfortable with a $1000 incentive. How about $300-$350? That’s our “final offer,” so to speak.

HRSA – We understand the concerns with the $1,000 incentive, and will instead reduce the offer to $350 per site.

3. Has the study received IRB approval yet?  

Response: The study received the approval of the Centers for Disease Control and Prevention, National Center for Health Statistics, Ethical Review Board approval in June 2007. The statement of approval for CDC Protocol #2007-25 is attached.

OMB: OK

4. How will non-response bias be analyzed? What is the expected response rate and what is this estimate based on?

Response: To capture information about the numbers of patients who choose to participate and those who choose not to participate, the health care clinic staff at the 6 sites will keep logs counting the number of eligible patients recruited and the number that consent to participate. The usable responses divided by the total number of eligible women patients will provide the percent response rate and the number of eligible women who were asked to participate but did not complete the survey divided by the total number of eligible women will provide the non-response rate.

OMB: Does this response mean that HRSA does not have an expected response rate? Is HRSA also saying that it is not possible given the data collection methods to analyze for non-response bias? Please clarify.

HRSA: The expected response rate for this assessment is 85%.  This estimate is based upon the methods that have been proposed for purposes of increasing response rate, including 6th grade reading level, training at each site among front office staff, buy-in from site administrators and staff, and the experience of the project contractor in conducting similar studies. There are no plans to evaluate non-response bias beyond a simple assessment of response rates obtained by adult or adolescent status, due to the fact that the project will not have any personal identifiers on potential respondents, and the small number of sites in this assessment (six potential sites).

OMB: Please add this to the list of limitations that will be acknowledged when reporting study results.

HRSA: We agree, and will add this to the list of limitations when reporting the study results.

In addition to the use of a purposive sample, please list the other study limitations that will be disclosed along with the publication of the results. For example, the sample is skewed in the sense that providers know they are being assessed and are therefore
likely to use the materials more than in a non-study setting; the sites are volunteering to participate and therefore are interested in using the materials and improving women's health and wellness, etc.

Response: In addition to the purposive sample and the limitations offered above, other study limitations include the following:

• the use of a convenience sample at each of the sites;

• the small number of sites (e.g., two school based, one worksite, one faith-based, and two Federally-qualified health centers);

• the use of materials that are English-language only;

• the inability to assess cultural competency;

• a post-test only design that does not permit the measurement of change in intentions, behavior, attitudes, or knowledge;

• the immediacy of measurement that does not permit assessment of behavior change;

• the one-time data collection period with no subsequent follow-up: and

• no control group.

Despite these limitations, it is believed that the results of this study will be of value to other sites, particularly HRSA grantees that are interested in using the BFWHW materials. The study results will also provide useful information to the HRSA Office of Women’s Health for future BFWHW programming.

OMB: OK

5. Given that the data collection period could last for 4 or more months, how will you ensure that patients don't participate twice (e.g. if they came to the clinic more than once in the 4 month period)?

Response: The materials are to be distributed to women who are having a wellness or health maintenance visit, which typically happens once a year. While we do not anticipate duplication of participation to be an issue, staff training at the sites will address this concern. Administrative staff will ask whether the woman was given the BFWHW materials, including the assessment form, at her last appointment. If she was offered the assessment, she will not be offered the assessment form again.

OMB: OK

6. Questionnaires:

a. On the young women survey, it is very important to clarify in each question that the questions in Section II relate to their attitudes and beliefs PRIOR to the doctor visit, since the questions are being asked AFTER the visit. In this regard, the question wording for the adult woman survey was more clear and should be used for the young woman survey.

Response: The words “Before today’s visit” will be added to each of the questions in Section II of the young women’s survey.

OMB: OK. Please follow the same format used in the adult woman survey (e.g. the word “before” is capitalized for emphasis, etc.).

    b.    For question 11, given that the clinic staff will hand the materials to the patients when they sign in for their appointment, how useful is it to ask whether the patient has seen or heard of the Bright Future materials? Won't all of them say yes?

Response: Question 11 on the young woman assessment form reads: “11. Had you heard of or seen the “My Bright Future: Physical Activity and Healthy Eating Guide for Young Women” and wallet card before today?” This is unlikely, because none of the sites have used the BFWHW materials before this project.

OMB: For clarity, please move “before today” up to the front of the sentence (i.e. “BEFORE TODAY, had you heard of…”).

    c.    If not all will say yes, would suggest some kind of skip pattern for the patients who say "no" or "don't know" for question #11 (i.e. questions 12A-B and 13i would not apply to these people).

Response: The time frame for Question 11 is before today’s visit, in other words, before the day of the appointment. Questions 12A-B for adult women and Question 13i for adolescent young women concern today’s visit, specifically the time immediately prior to seeing the clinician. Because of the different time frames, we do not believe a skip pattern is needed.

OMB: For clarity, move “in this visit today” up to the front of the sentence (i.e. “In your visit today, …”)

    d.    How will the responses to question 9 of the adult woman survey be used in the analysis? Why did you decide that these questions were relevant for adult women but not for young women?

Response: Question 9 addresses health locus of control. The responses will be used to describe the relationship between clinical experiences and attitudes, knowledge, intentions, and the perceived usefulness of the tool. While these questions are appropriate for adult women, they had not been used with patients as young as some of those who will be completing the young women’s assessment form.

OMB: Does this mean these questions have been tested and validated on adult women but not on young women? If it is important to assess the relationship between experiences and attitudes/knowledge/intentions/etc for adult women, it would seem to be important to assess this for young women as well. Can the questions be modified to be suitable for a younger audience?

HRSA: The questions have been tested and validated with adult women; to our knowledge; similar validation has not taken place among young women. We concur this is an important assessment for both age groups. We propose using the following questions on the Young Women’s Assessment Form, which are from the Parcel-Meyers Children’s Health Locus of Control instrument.

Parcel, G.S., & Meyer, M.P. Development of an instrument to measure children’s health locus of control. Health Education Monographs, 1978, 6, 149-159.

Adult Women’s Version

How strongly do you agree or disagree with the following statements? (please check one response for each statement)

	Strongly Disagree	Moderately Disagree	Disagree	Agree	Moderately Agree	Strongly Agree
a. I am responsible for my health.	�	‚	ƒ	„	…	†
b. If I take care of myself, I can avoid getting sick.	�	‚	ƒ	„	…	†
c. Good health is mostly due to luck.	�	‚	ƒ	„	…	†
d. Doing what my doctor tells me to will make me well.	�	‚	ƒ	„	…	†
e. There are so many strange diseases around that you never know how or when you might get one.	�	‚	ƒ	„	…	†
f. When people get sick it is because they are careless.	�	‚	ƒ	„	…	†

Young Women’s Version

How strongly do you agree or disagree with the following statements? (please check one response for each statement)

	Strongly Disagree	Moderately Disagree	Disagree	Agree	Moderately Agree	Strongly Agree
a. I can do things to keep from getting sick.	�	‚	ƒ	„	…	†
b. If I get sick, it is because getting sick just happens.	�	‚	ƒ	„	…	†
c. Good health comes from being lucky.	�	‚	ƒ	„	…	†
d. Doing what my doctor or nurse tells me to will make me well.	�	‚	ƒ	„	…	†
e. When I am sick I can do things to get better.	�	‚	ƒ	„	…	†
f. I can do things to fight illness.	�	‚	ƒ	„	…	†

OMB: Great. Thanks for adding this.

HRSA: Thank you.

5