Constituent Engagement Related to Use of AI
Administration for Children and Families Generic for Engagement Efforts
0970 – 0630
Supporting Statement Part A
Justification
July 2024
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
A1. Necessity for the Data Collection
The Department of Health and Human Services (HHS) Administration for Children and Families (ACF) Office of Planning, Research, and Evaluation proposes to collect information to better understand human services field constituents’ wants and needs related to use of artificial intelligence (AI), particularly relating to the administration of public benefits and services.
The Executive Order (EO) on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (EO 14110) requires ACF to (1) identify and prioritize what guidance the human services field seeks on the use of AI when administering public benefits and services, and (2) provide the human service perspective to a strategic plan for AI use in HHS. In addition, more broadly, ACF has been thinking about using AI internally as well as how ACF grantees might use AI in their operations. Some states and state-level advocacy groups have reached out to ACF for guidance about using AI.
Hearing the perspective of those affected by, experienced in, interested in, or potentially interested in using AI in ACF programs and similar programs is vital to ensure ACF is responsive to the needs of those it serves and that resources are, and programming is appropriate, useful, and relevant for audiences.
There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.
A2. Purpose of Survey and Data Collection Procedures
Overview of Purpose and Use
The primary purpose of this information collection is to obtain rapid ACF constituent input and feedback on prioritization efforts related to:
ACF’s position and response to HHS-level mandates on AI, including the following requirements from EO 14110:
Identifying and prioritizing what guidance and support the human services field wants and needs related to use of AI, particularly relating to administration of public benefits and services per E.O 14110 Section 7.2(b)(i);
Providing human service perspectives to an HHS strategic plan on AI, per E.O 14110 Section 8 (b)(i);
Informing ACF's AI strategy and ongoing constituent engagement channels.
The gathered information will be used to develop an internal memo synthesizing findings from constituent engagement. This memo will be used to inform ACF’s AI strategy and help determine which topics to issue guidance on the use of AI to ACF grantees.
Gathering information from individuals with diverse experiences and perspectives to inform ACF policies and programs.
Making opportunities for ongoing, two-way collaborative and actionable communications between ACF and its state, local and/or Tribal partners, program participants, communities served or affected by ACF programs, and or others experienced with or interested in ACF programs or similar programs.
Exploring emerging areas
Informing program implementation
Informing the development and dissemination of resources and products developed under ACF studies, regulatory activities, guidance, outreach and/or training activities.
Overview of Information Collections
Information Collection Title |
Respondent, Content, Purpose of Collection |
Mode and Duration |
Instrument 1: Focus groups with state leaders and others |
Respondents: State leaders, local leaders, and frontline service providers from a range of human services programs.
Content: The virtual focus groups will be based on scenarios. Focus group facilitators will read a scenario in which AI could be used in human services programs. Facilitator will ask respondents to reflect on the scenario and facilitate dialogue among respondents. During registration, respondents will be asked to share several demographic characteristics to understand the diversity across respondents and to establish similar groups for focus group breakouts.
Purpose: To gather constituent input and feedback on potential benefits and challenges associated with use of AI in human services programs. |
Mode: Virtual focus groups with online registration (6 total)
Duration: 60 minutes |
Instrument 2: Focus groups with public benefit participants |
Respondents: Enrollees in a diverse set of public benefit programs.
Content: The virtual focus groups will be based on scenarios. Focus group facilitators will read a scenario in which AI could be used in human services programs. Facilitator will ask respondents to reflect on the scenario and facilitate dialogue among respondents.
Purpose: To gather constituent input and feedback on potential benefits and challenges associated with use of AI in human services programs. |
Mode: Virtual focus groups (2 total)
Duration: 60 minutes |
Instrument 3: Survey of AI opinions—participants |
Respondents: Participants in a diverse set of public benefit programs.
Content: Multiple choice questions that present short scenarios in which AI could be used in human services and ask respondents to select which response option they agree with.
Purpose: To gather constituent input and feedback from a broad group on potential benefits and challenges associated with use of AI in human services programs. Demographic information will be collected to ensure we are reaching a diverse group, and to see how people of different backgrounds respond (with acknowledgement that the sample is not representative). |
Mode: Online survey
Duration: 5 minutes |
Instrument 4: Survey of AI opinions—nonparticipants |
Respondents: State leaders, local leaders, and frontline service providers from a range of human services programs.
Content: Several demographic and characteristics items, including professional role, plus multiple choice questions that ask about respondents’ current and planned use of AI; resources and guidance needed from ACF; and reasons for not using AI.
Purpose: To gather constituent input and feedback from a broad group on use, challenges, and resource needs for using AI in human services programs. Demographic information will be collected to ensure we are reaching a diverse group, and to see how people of different backgrounds respond (with acknowledgement that the sample is not representative). |
Mode: Online survey
Duration: 5 minutes |
Instrument 5: Screener for participants |
Respondents: Enrollees in a diverse set of public benefit programs.
Content: Demographic items and self-reports of participation in public benefits programs.
Purpose: For a recruiting panel to identify potential public benefit participants to participate in focus groups and the survey of AI opinions. |
Mode: Online survey
Duration: 2 minutes |
Processes for Information Collection
Instrument 1: Focus groups with state leaders and others: ACF will leverage existing partnerships by working with relevant associations of governors, mayors, and human services to identify potential focus group participants. In addition, a contractor will provide support. The contractor will develop an outreach email for distribution to potential virtual focus group participants inviting them to the session. The email will include a link to register for a focus group and ask participants to provide contact information. Instrument 1 includes the registration questions, which will be provided within the registration form in Zoom.gov. Appendix 1 includes the outreach email for the focus groups. The contractor will track registration and provide reminders to individuals prior to the scheduled focus group. If registration is low, ACF will ask partners to reshare the outreach email to encourage additional registrants. Each focus group will be held virtually using Zoom.gov.
Instrument 2: Focus groups with public benefit participants: ACF’s contractor will use a national recruiting panel. The panel will administer the screener for participants (Instrument 5) to its members. The panel will provide the contractor with the data from the screener and the contractor will determine focus group participants to ensure diversity. Each focus group will be held virtually using meeting technology provided by the recruiting panel.
Instrument 3: Survey of AI opinions—participants: ACF’s contractor will use a recruiting panel to obtain responses from public benefit participants. The recruiting panel will administer the screener (Instrument 5) to people who may be eligible to participate in the study. ACF’s contractor will review the results of the screener to ensure demographic diversity in respondents. The recruiting panel will distribute a survey link to those screener respondents who are eligible to participate in the survey. Upon clicking the link, respondents will complete the survey using ACF’s Qualtrics platform.
Instrument 4: Survey of AI opinions—nonparticipants: ACF and its contractor will publicize the survey by circulating a link to it through existing newsletters distributed by ACF and professional associations to which respondents might belong. Interested respondents will click the link to complete the survey using ACF’s Qualtrics platform. Appendix 1 includes the outreach email for the survey.
Instrument 5: Screener for participants: The recruiting panel will administer the screener to people who may be eligible to participate in either the focus groups with public benefit participants (Instrument 2) or the survey of AI opinions, from the perspective of public benefit participants (Instrument 3).
A3. Improved Information Technology to Reduce Burden
Participating in this information collection will not require any in-person activities or printing of any materials. The study team will use information technology as appropriate to reduce the burden on respondents. The contractor will conduct the virtual focus groups with state leaders and others via Zoom.gov and with public benefit participants via meeting technology provided by the recruitment panel. Both technologies can be accessed on a computer, phone, or tablet. When scheduling the virtual focus group, the contractor will email respondents an overview of the discussion topics.
Focus groups will be transcribed and/or detailed notes will be taken. With participant consent, information in the transcripts may be revisited for secondary analysis at a future date, with a goal of reducing burden on the group of respondents.
The surveys and screener will be online and accessible by computer, smartphone, or tablet.
A4. Efforts to Identify Duplication
The proposed virtual focus groups and surveys do not duplicate other data collection efforts. The information the contractor will collect through the focus groups and surveys is not available through other existing data sources.
A5. Involvement of Small Organizations
Small business or other small entities may be involved in these efforts, but ACF will minimize the burden on them based on needs, for example, by scheduling data collection at times convenient for respondents.
A6. Consequences of Less Frequent Data Collection
This is a one-time data collection.
A7. Special Circumstances
There are no special circumstances for the proposed data collection efforts.
A8. Federal Register Notice and Consultation
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection request for a new umbrella generic clearance. The notice was published on December 11, 2023, (88 FR 85890), and provided a sixty-day period for public comment. ACF did not receive any comments on the first notice. A second notice was published, allowing a thirty-day period for public comment (89 FR 12352), in conjunction with submission of the request to OMB. ACF did not receive any comments on the second notice.
No consultations have taken place with experts outside of the project team.
A9. Tokens of Appreciation for Respondents
It is extremely important to provide those with lived experience, experts, staff, and others providing their feedback for these efforts with equitable compensation or tokens of appreciation for participation. As noted in a 2022 report by the Office of the Assistant Secretary for Planning and Evaluation1 this “helps ensure a diverse population with varied views can participate.”
As such, we plan to provide honoraria to public benefit participants who participate in focus groups and surveys, as described in section A13.
A10. Privacy of Respondents
Personally Identifiable Information (PII):
Information collected will include the following personally identifiable information (PII):
Instrument 1: During registration, respondents will provide their name and email address to facilitate tracking registration and sending reminders. During the focus group discussion, respondents will share their first name to help facilitators manage the group.
Assurances of Privacy:
Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data and that their participation is voluntary.
Data Security and Monitoring:
As specified in the contract, the contractor shall protect respondents’ privacy to the extent permitted by law and will comply with all federal and departmental regulations for private information. The contractor has developed a data security plan that assesses all protections of respondents’ personally identifiable information. The contractor will ensure all employees receive training on data privacy issues and comply with all requirements. All Mathematica staff must sign an agreement to (1) maintain the privacy of any information from individuals, businesses, organizations, or families participating in any projects conducted by Mathematica; (2) complete online security awareness training when they are hired; and (3) participate in a refresher training annually.
As specified in the evaluator’s contract, the contractor will use encryption compliant with the Federal Information Processing Standard (Security Requirements for Cryptographic Module, as amended) to protect all sensitive information during storage and transmission. The contractor will securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the standard. The contractor will incorporate the standard into its property management and control system and establish a procedure to account for all laptop and desktop computers and other mobile devices and portable media that store or process sensitive information. The contractor will secure any data stored electronically in accordance with the most current National Institute of Standards and Technology requirements and other applicable federal and departmental regulations. In addition, the contractor’s data safety and monitoring plan includes strategies for minimizing to the extent possible including sensitive information on paper records and for protecting any paper records, field notes, or other documents that contain sensitive information to ensure secure storage and limits on access.
No information will be given to anyone outside the study team and ACF.
A11. Sensitive Questions
The proposed information collection does not request any sensitive information.
A12. Estimation of Information Collection Burden
Burden Estimates
Respondents will participate in data collection only once. Our burden estimates were derived based on the following.
Focus groups: (Instrument 1) We assume the registration questions included within Instrument 1 will take respondents 1 minute to complete. We expect to hold 6, 60-minute focus groups with state leaders, local leaders, and frontline staff of human services programs. We anticipate 20 participants per focus group, for a total of 120 burden hours. (Instrument 2) We expect to hold 2, 60-minute focus groups with public benefit participants. We anticipate 10 participants per focus group, for a total of 20 burden hours. Focus groups are time-limited and will not extend beyond 60 minutes.
Survey of AI opinions—participants (Instrument 3): We expect to survey up to 100 public benefit participants. Each survey is expected to take up to 5 minutes, for a total of 8.33 burden hours. This estimate is based on internal pretesting by Contractor staff.
Survey of AI opinions—nonparticipants (Instrument 4): We expect to survey up to 300 state leaders, local leaders, and frontline staff of human services programs. Each survey is expected to take up to 5 minutes, for a total of 25 burden hours. This estimate is based on internal pretesting by Contractor staff.
Screener (Instrument 5): We expect about 280 people who are potential public benefit participants will take the screener (80 will take the screener to achieve the 20 focus group participants and 200 will take the screener to achieve the 100 survey completes). Each screener is expected to take up to 2 minutes, for a total of 9.33 burden hours. This estimate is based on internal pretesting by Contractor staff.
Cost Estimates
The cost to respondents who are state leaders and others (Instrument 1 and Instrument 3) was calculated using the Bureau of Labor Statistics (BLS) job code for Community and Social Service Specialists [21-1099] and wage data from May 2023, which is $25.97 per hour. To account for fringe benefits and overhead the rate was multiplied by two which is $51.94. Registration questions for the focus groups with state leaders and others are included in the burden estimate for Instrument 1.
The cost to respondents who are public benefit participants (Instruments 2, 3, and 4) was based on the federal minimum wage of $7.25 from the U.S. Bureau of Labor Statistics, Occupational Employment Statistics (OES), 2023. This wage estimate is appropriate for employment and salary levels for adults who are receiving public benefits, which includes individuals with relatively high proportions of unemployment or who may be working part-time minimum wage jobs.
https://www.bls.gov/oes/current/oes_stru.htm
Instrument |
Total Number of Respondents |
Total Number of Responses Per Respondent |
Average Burden Hours Per Response |
Total Burden Hours |
Average Hourly Wage |
Total Annual Cost |
Instrument 1: Focus groups with state leaders and others |
120 |
1 |
1 |
120 |
$51.94 |
$6,232.80 |
Instrument 2: Focus groups with public benefit participants |
20 |
1 |
1 |
20 |
$7.25 |
$145.00 |
Instrument 3: Survey of AI opinions participants |
100 |
1 |
.083 |
8 |
$7.25 |
$58.00 |
Instrument 4: Survey of AI opinions – nonparticipants |
300 |
1 |
.083 |
25 |
$51.94 |
$1,298.50 |
Instrument 5: Screener for participants |
280 |
1 |
.033 |
9 |
$7.25 |
$65.25 |
Total Burden and Cost Estimates: |
182 |
|
$7,799.55 |
|||
A13. Cost Burden to Respondents or Record Keepers
Directly engaging the communities ACF serves and including these individuals in ACF research is in line with the following priorities of the current Administration and HHS:
⦁ Advancing Racial Equity and Support for Underserved Communities Through the Federal Government (EO 13985)
⦁ Further Advancing Racial Equity and Support for Underserved Communities Through the Federal Government
⦁ Presidential Memorandum on Restoring Trust in Government through Scientific Integrity and Evidence-Based Policy Making
⦁ ACF Evaluation Policy
⦁ HHS Strategic Plan FY 2022-2026,
⦁ ASPE’s Methods and Emerging Strategies to Engage People with Lived Experience (2021)
⦁ ASPE’s Recruiting Individuals with Lived Experience (2022)
Consistent with the guidance documents referenced, and to ensure involvement with a variety of people with diverse experiences and perspectives with public benefits programs, we propose to offer public benefit participants an honorarium for their time spent providing their expertise and experience. Specifically, we propose to offer public benefit participants who participate in focus groups $50. We do not intend to provide honoraria to other respondent types because the focus groups are about their work and they are providing their expertise as part of their normal work duties. The panel that will recruit public benefit participants requires that we offer an honorarium of 100 points, which is equivalent to $1, to public benefit participants who complete the short surveys.
Equitable compensation is in line with leading practices for ethical engagement of those with lived expertise and advancing equity for populations who have been historically underserved (as noted in section A1, advancing equity is a priority, as highlighted in the referenced EOs in that section). Providing equitable compensation recognizes the value of the time provided by participants, helps to remove barriers to participation, and affirms that the contributions from those with lived experience are as valuable as those from other experts.
As noted in the 2022 report by ASPE this “helps ensure a diverse population with varied views can participate.” Additionally, in an earlier report it was noted that “Providing [those with lived experience] with compensation commensurate with the rates that other experts—i.e., experts engaged based on their expertise as practitioners or researchers, rather than lived experience—receive helped recognize the valuable and unique expertise that people with lived experience lend, which promoted meaningful engagement.” The report goes on to specify that not doing so could result in “unintended consequences…. when lived experience engagements have scarce resources and experts are undercompensated, which can undermine, disregard, and/or marginalize people with lived experience.”
A14. Estimate of Cost to the Federal Government
The total cost for the data collection activities under this current request will be $75,000.
A15. Change in Burden
This is for an individual information collection under the umbrella generic clearance for ACF engagement activities (0970-0630).
A16. Plan and Time Schedule for Information Collection, Tabulation and Publication
All data collections will take place from following OMB approval for about four months. The information collected under this request is meant to inform ACF through an internal memo. The primary purpose of the internal memo is to inform ACF planning and future guidance to ACF grant recipients. In addition, this information may be incorporated into documents or presentations that are made public such as through conference presentations, websites, or social media. Audiences for such products would include federal leadership and staff, grant recipients, program participants, local implementing agencies, and training and technical assistance providers.
A17. Reasons Not to Display OMB Expiration Date
All instruments will display the expiration date for OMB approval.
A18. Exceptions to Certification for Paperwork Reduction Act Submissions
No exceptions are necessary for this information collection.
Attachments
Instrument 1: Focus groups with state leaders and others
Instrument 2: Focus groups with public benefit participants
Instrument 4: Survey of AI opinions—nonparticipants
Instrument 5: Screener for participants
Appendix 1: Outreach materials
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OPRE OMB Clearance Manual |
| Author | DHHS |
| File Modified | 0000-00-00 |
| File Created | 2025-02-17 |