Supporting Statement A

Health Resources and Services Administration

SUPPORTING STATEMENT

Constituent Feedback to Oral Health Education Messages and Materials

A. Justification

1. Circumstances of Information Collection

The Health Resources and Services Administration (HRSA) currently has approval under the generic clearance, Office of Management and Budget (OMB) Control No. 0915-0212, to conduct customer satisfaction surveys and focus groups. This collection of information helps fulfill the requirements of:

1. Executive Order 12862, “Setting Customer Service Standards,” which directs Agencies to continually reform their management practices and operations to provide service to the public that matches or exceeds the best service available in the private sector.

This is a request for OMB approval of a voluntary customer satisfaction data collection endeavor under HRSA’s generic clearance, which includes oral and written responses collected via focus groups, cognitive interviews, and online discussion board forums.

HRSA’s Office of Planning, Analysis and Evaluation (OPAE) is responsible for identifying public health priority areas and coordinating cross-agency initiatives to address those priorities, as a means for optimizing the overall impact of HRSA programs and the constituents in which they serve.

HRSA has identified improving oral health literacy as a priority area for the agency. Oral health is fundamental to our overall health, yet profound disparities exist in the United States.^1,2,3

Oral health literacy is the degree of a person’s capacity to obtain, process, and understand basic oral health information and services needed to make appropriate oral health decisions.⁴

To address this priority area, OPAE and their Contractor are seeking to garner feedback on existing oral health education materials and messages with its constituent populations. This includes: HRSA health center patients, parents and caregivers of young children, people with HIV, and people residing in rural areas. The oral health messages and materials, in their current form, are not appropriately tailored to HRSA’s constituents. They do not address the unique characteristics and experiences of constituents’ cultures, nor do they consider concepts, words, and numeracy for people with low oral health literacy.

In line with Executive Order 12862, which directs agencies to “provide significant services directly to the public” to “survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services,” HRSA OPAE and their Contractor seek to test the existing oral health education materials and messages with its constituents to adequately determine their level of satisfaction with the current messages and materials, how they perceive and understand the messages in their current form, how well they resonate, and how well they increase constituents’ willingness to engage in good oral health behaviors. HRSA OPAE and their Contractor will revise its messages and materials based on customer feedback - to better tailor them to HRSA’s niche constituent groups. HRSA OPAE will also use customer feedback to inform effective future oral health education outreach strategies and tactics among its constituents.

2. Purpose and Use of the Information

The purpose of this information collection request is to assess the understanding, clarity, resonance, and potential behavioral impact of existing oral health education messages and materials. The information gleaned from this data collection effort will be used to revise messages and materials - specifically to tailor them to HRSA’s constituent groups. Customer feedback received will also inform effective oral health education outreach strategies and tactics for reaching HRSA constituents with the revised messages and materials. The Contractor will protect the integrity of the data collected, provide opportunities for honest feedback, and maintain the confidentiality of participants.

This information collection request contains six types of customer feedback:

• Focus Group Participant Recruitment Screener (see Attachment 1)

• Cognitive Interview Participant Recruitment Screener (see Attachment 2)

• Online Discussion Board Forum Recruitment Screener (see Attachment 3)

• Focus Group Evaluation Protocols (see Attachment 4)

• Cognitive Interview Evaluation Protocols (see Attachment 5)

• Online Discussion Board Forum Evaluation Protocols (see Attachment 6)

A professional recruitment vendor will utilize the recruitment screeners (Attachments 1 - 3) to determine whether interested participants qualify to participate in the project. An experienced moderator will use the evaluation protocols (Attachments 4 - 6) to ask participants for their feedback on the existing oral health education messages and materials. Completion of all information collection requests relevant to this project is voluntary.

3. Use of Improved Information Technology

HRSA and its Contractor will use a video conferencing platform to conduct the cognitive interviews and focus groups remotely. The online discussion board forums will be conducted via an online platform. Because the intent is to capture qualitative feedback with open-ended questions, the most efficient and least burdensome approach is through oral (i.e., cognitive interviews and focus groups) and written interviews (i.e., online discussion board forums).

The online discussion board forums serve as an opportunity to include participants who are uncomfortable participating in oral interviews (e.g., if they lack confidence in their English-speaking or verbal articulation abilities), but still want to provide feedback via written responses. All responses will be recorded to ensure full interviews are captured.

4. Efforts to Avoid Duplication

This information is not available through any other source and is not currently being collected elsewhere.

5. Involvement of Small Entities

No small businesses will be involved in this proposed information collection.

6. Consequences if Information Collected Less Frequently

This information collection request is for a one-time project that will help evaluate HRSA’s existing oral health education messages and materials. If this data collection effort is not conducted, HRSA OPAE will not know whether its messages are effective in motivating the agency’s constituents to practice good oral health behaviors, and the agency will not have the opportunity to improve its messages based on HRSA constituent representative participant feedback.

There are no legal obstacles to reduce the burden.

7. Consistency with the Guidelines in 5 CFR 1320.5(d)(2)

These data collection efforts will be implemented in a manner fully consistent with 5 CFR 1320. 5(d)(2).

8. Consultation Outside the Agency

The notice required in 5 CFR 1320.8(d) was published in the Federal Register on November 13, 2017, (Vol. 82, No. 217, pages 52308-52309). No public comments were received.

9. Remuneration of Respondents

A $75 monetary token of appreciation will be given to participants to thank them for their participation. Our experience in collecting qualitative data indicates that offering nonmonetary incentives or an incentive that is below the commonly accepted rate will result in increased costs that exceed the amount saved on a reduced incentive. The consequences of an insufficient incentive include the following:

• Increased time and cost of participant recruitment

• Increased likelihood of “no-shows” (which may result in methodologically unsound focus groups, cognitive interviews, and online discussion forums, with small numbers of participants)

• Increased probability that a discussion may need to be cancelled or postponed due to insufficient numbers recruited by the scheduled date of the data collection effort, which not only incurs additional costs, but also puts additional burden on the recruited participants who have to reschedule their participation.

Our proposed incentive amount will help ensure that participants honor their commitment of participating in the data collection effort. Our incentive was chosen based on (1) an estimated cost related to childcare for 2 hours (e.g., 30 minutes for setting-up, logging-in, and resolving any technology issues, plus the 90-minute discussion), which is approximately $35.46;⁵ (2) potential transportation costs, particularly because this project seeks participants from rural areas who may not have an internet connection at home - thus they will need to commute to work or a local library to participate; and (3) any loss of income to participants (e.g., if a participant has to take time off work to participate). The proposed amount is comparable to what has been the level of reimbursement for target audiences in similar government-funded activities. Parents of young children are often more difficult to recruit than more general audiences and the incentive needs to be enough to help the participants cover outside childcare costs if needed. Offering tokens of appreciation is considered necessary to also recruit minorities and historically underrepresented groups into data collection efforts. As noted above, we expect that lower or nonmonetary incentives will necessitate over-recruitment by higher percentages and result in longer recruiting time as well as higher overall project costs.

Research has shown improved participation in qualitative interviews with adults in the general United States population when given a monetary incentive in comparison to a nonmonetary incentive or no incentive.⁶ The importance of monetary compensation for focus group participation has been discussed by Krueger and Casey (2014), who indicate that offering minimal levels of monetary compensation can help ensure that sufficient numbers of participants will attend, thereby yielding more useful research results.⁷ Further, in a meta-analysis of 38 experiments and quasi-experiments, Church (1993) found that providing cash incentives for participation was far more effective than nonmonetary gifts in generating survey response, and prepaid monetary incentives yielded an average increase of 19.1 percentage points over comparison groups.⁸ When applied in a reasonable manner, incentives are not an unjust inducement and are an approach that acknowledges respondents for their participation and treats them justly and with respect by recognizing and acknowledging the effort they expend to participate.⁹ Finally, the importance of monetary incentives has been corroborated in experiences related to the National Adult Literacy Survey by Berlin and colleagues (1992).¹⁰

10. Assurance of Confidentiality

Participation is fully voluntary, and the privacy of individuals will be protected at all phases of the project, from initial screening and recruitment to reporting and beyond.

11. Questions of a Sensitive Nature

No questions of a sensitive nature will be asked as part of this proposed data collection activity.

12. Estimates of Annualized Hour Burden

Respondents:

A professional recruitment vendor will seek to recruit the following participants for this project:

• Up to 30 cognitive interview participants (n=30)

• Up to 60 focus group participants (n=60) [6 focus groups of up to 10 participants in each]

• Up to 18 online discussion board forum participants [3 online discussion board forums of up to 6 participants in each]

Audience segments will include low health literacy participants representing three groups:

1. Parents and caregivers of young children (from 6 months to 3½ years of age)

2. People with HIV

3. Adults aged 65 and older

All segments will include representation from constituents who seek services at HRSA health center clinics and/or who reside in rural areas. The Contractor will also seek to recruit a mix of participants across gender, age, race/ethnicity, geographic location, and geographic type.

Exhibit 12. A - Annual respondent burden estimates:

Type of Collection	Number of Respondents	Responses per Respondent	Total Responses	Hours per Respondent	Total Burden Hours	Wage Rate	Total Hour Cost
Recruitment Screener	200	1	200	0.1	20	$25.72	$514.40
Evaluation Protocol	108	1	108	1.5	162	$25.72	$4166.64
Total	308		308		182		$4681.04

This information collection request contains two types of customer feedback: Participant Recruitment Screener (see Attachments 1 - 3) and Evaluation Protocol (see Attachments 4 - 6). The annual burden estimate table summarizes the number of respondents per information collection request.

Planned frequency of information collection:

This is a one-time project.

13. Estimates of Annualized Cost Burden to Respondents

HRSA anticipates the total annualized cost to respondents to be $4681.04 (Exhibit 12.A).

No capital or start-up costs are associated with this information collection request. The total annualized cost estimate is related to the time for participants to complete the screening questionnaire and provide an oral or written response to the evaluation protocol. This annualized cost to respondents is based on the national occupational wage estimate in the United States from the 2019 Bureau of Labor Statistics report on Wage Estimates.¹¹ The wage is then multiplied by the estimated total respondent hours for each form.

14. Estimates of Annualized Cost to the Government

Costs to the federal government fall into the following categories:

• Cost for overseeing Contractor

• Costs of contractual support for planning, implementation, analysis, and reporting

Exhibit 14. A - Annual Cost to Government Estimates:

Type of Cost	Description of Services	Annual Cost
Oversight of Contractors (GS-13 Government Program Analyst - 10%)	Federal staff time to oversee contractors who administer information collection request	$10630.88
Cost of Contractual Support	Time and effort for contractors to recruit, analyze, and report on information collection request	$299,882

HRSA anticipates the average annual cost for the federal government will include personnel costs for contractual oversight. This will include a federal program analyst at Grade 13 Step 4 ($51.11 hourly rate) (Office of Personnel Management, 2019) for 208 hours.

The total cost to the federal government for this activity is $310,512.88 (Exhibit 14.A).

15. Change in Burden

Not Applicable. This is a new activity under HRSA’s generic clearance and is included in the total burden currently approved under OMB Control No. 0915-0212.

16. Plans for Analysis and Timetable of Key Activities

The information collection requests will take place from December 2020 through May 2021. The Contractor will plan, implement, and report findings from the discussions in narrative form. A final topline report with actionable recommendations will be submitted to HRSA following the conclusion of all discussions. Findings will be used for internal improvement and are not generalizable to the general public.

17. Exemption for Display of Expiration Date

No exemption is being requested. The expiration date will be displayed.

18. Certifications

This information collection activity will comply with the requirements in 5 CFR 1320.9.

1 Centers for Disease Control and Prevention (CDC) (2020). Oral Health Basics. Retrieved from https://www.cdc.gov/oralhealth/basics/index. html.

2 U. S. Health Resources and Services Administration (HRSA), National Advisory Committee on Rural Health and Human Services (2018). Improving Oral Health Care Services in Rural America: Policy Brief and Recommendations. Retrieved from https://www.hrsa.gov/sites/default/files/hrsa/advisory-committees/rural/publications/2018-Oral-Health-Policy-Brief.pdf (PDF).

3 Sheiham, A. (2005). Oral Health, General Health and Quality of Life. Bulletin of the World Health Organization. Retrieved from https://www.who.int/bulletin/volumes/83/9/editorial30905html/en/.

4 Healthy People 2020. Health Literacy. Retrieved from https://www.healthypeople.gov/2020/topics-objectives/topic/social-determinants-health/interventions-resources/health-literacy.

5 Assumes an hourly rate of $17.73 per hour for a professional babysitter: https://blog.urbansitter.com/2020-babysitting-rates-childcare-costs-study/.

6Kelly, B., Margolis, M., McCormack, L., LeBaron, P.A., & Chowdhury, D. (2017). What Affects People’s Willingness to Participate in Qualitative Research? An Experimental Comparison of Five Incentives. Field Methods, 29(4), 333–350. https://doi.org/10. 1177/1525822X17698958

7 Krueger, R.A.& M.A. Casey. (2014). Focus groups: A practical guide for applied research. (5th ed). Thousand Oaks, CA: Sage Publications, Inc.

8 Church, A.H. (1993). Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public Opinion Quarterly, 57, 62-79.

9 Halpen, S.D., Karlawish, J.H., Casarett, D., Berlin, J. A., & Asch, D.A. (2004). Empirical assessment of whether moderate payments are undue or unjust inducements for participation in clinical trials. Archives of Internal Medicine, 164(7), 801-803.

10 Berlin, M., L. Mohadjer, J. Waksberg, A. Kolstad, I. Kirsch, D. Rock, & K Yamamoto. An experiment in monetary incentives. American Statistical Association, Proceedings of Survey Research Methods Section; Alexandria, VA: 1992. pp. 393–398.

11 Bureau of Labor Statistics (2030). May 2019 National Occupational Employment and Wage Estimates, 11-9151 Social and Community Service Managers. Retrieved from https://www.bls.gov/oes/current/oesnat.htm#11-0000

8