PHLF ICR_SSA_09192023_10242023_clean

The mission of this fellowship is to train and provide experiential learning to current students and early career professionals in public health law and policy. The purpose of this data collection is to assess the quality and value of the Public Health Law Fellowship Program (PHL Fellowship) through the collection of information from two key stakeholder groups: host site supervisors and fellows.

Intended use of the resulting data:

Information will be used to inform program improvements and document evidence of outcomes and impact to inform future decision-making.

Methods to be used to collect:

The method that will be used to collect data will be through seven online data collection instruments, one focus group instrument, and one interview instrument.

The subpopulation to be studied:

The annual subpopulation of respondents will consist of 70 PHL Fellowship fellows/alumni and 40 external host site supervisors. (Respondents will consist of a total of 210 fellows and 120 external host site supervisors at the end of the three-year funding period.)

How the data will be analyzed:

Descriptive and inferential statistics (where appropriate) will be used to analyze quantitative data. Qualitative analysis will be conducted on open-ended responses.

Section A. JUSTIFICATION

Circumstances Making the Collection of Information Necessary Background

Background

The Centers for Disease Control and Prevention (CDC) is requesting approval for a new Information Collection Request. This information collection is authorized by Section 301 of the Public Health Service Act (42 U.S.C. 241) (See Att. A-1_Public Health Service Act). CDC works to protect America from health, safety, and security threats, both foreign and in the U.S.¹ CDC strives to fulfill this mission, in part, through a competent and capable public health workforce. One mechanism for developing the public health workforce is through training programs like the Public Health Law Fellowship Program (PHL Fellowship), administered by CDC’s Office of Public Health Law Services (OPHLS). OPHLS advances the use of law as a public health tool by building legal epidemiology capacity and workforce development to public health practitioners, policymakers, and attorneys in state, tribal, local, and territorial (STLT) jurisdictions.

The mission of the PHL Fellowship is to train and provide experiential learning to current students and early career professionals in public health law and policy.² The PHL Fellowship targets current graduate students and law students, as well as recent graduates of graduate and law programs with a demonstrated interest in public health law. Each year, an expected cohort of up to 70 fellows will be enrolled in the PHL Fellowship (totaling 210 fellows by the end of the three-year approval period). PHL fellows will complete a minimum one semester assignment in a host site (e.g., a state, tribal, local, or territorial health department or nonprofit organization, or placement internally in CDC). Host sites design their fellows' assignments to meet their organization's unique needs while also providing on-the-job experience that prepares fellows for future careers in public health. At host sites, fellows are mentored by members of the public health workforce (referred to as “host site supervisors”). It is the goal of the fellowship that following participation in the program, alumni will seek employment within the public health law system (e.g., federal, state, tribal, local, or territorial health agencies, or non-governmental organizations), focusing on health equity and/or emergency response.

The fellowship was created pursuant to American Rescue Plan (see Attachment A-2) funding to expand on CDC’s Public Health Law Program’s intern/extern program. There were no prior efforts to systematically assess the intern/extern program necessitating the creation of an evaluation plan for the PHL Fellowship. This is a new information request. Evaluation priorities focus on continuously monitoring program processes and activities to improve the program’s quality and documenting program outcomes to demonstrate impact and inform decision-making about future program direction. To assess the Fellowship’s processes and short-term outcomes, we request 3 years of approval.

Purpose and Use of the Information Collection

The purpose of this data collection is to collect information from five distinct respondent groups: Public Health Law Fellowship Prospective Fellows, Public Health Law Fellowship Current Fellows, Public Health Law Fellowship Alumni, Public Health Law Fellowship Prospective Host Sites, and Public Health Law Fellowship Host Site Supervisors to: a) monitor program processes and activities to improve the program’s quality, and b) document program outcomes to demonstrate impact and inform decision making about future program direction. A description of the purpose of the information collected for each survey is below.

The Public Health Law Fellowship — Application (see attachments E-1 and E-2) will gather information about the fellow’s education, interest in public health law/policy, experience with public health law/policy, and demographics. The application contains 28 items. The application will be available electronically and the link will be posted on the PHL Fellowship’s webpage. The application will consist of both closed-ended and open-ended questions. Data collected will assess the suitability of each prospective fellow for the PHL Fellowship program.

The Public Health Law Fellowship — Welcome Survey (see attachments F-1 and F-2) will gather information about the fellow’s current knowledge of public health law topics and expectations for the fellowship experience. The survey contains 32 items and will be administered once for each fellow at the beginning of their fellowship. Surveys will be administered electronically; a link to the survey will be provided in an email invitation. The survey will consist of both closed-ended and open-ended questions. Data collected will assess each fellow’s current knowledge of material covered through the fellowship’s curriculum and topical expectations for the program (e.g., health equity, emergency preparedness and response).

The Public Health Law Fellowship — Focus Group (see attachment G-1) will gather qualitative information about the fellows’ experience in the fellowship. The instrument contains 8 items and will be administered at the conclusion of the fellows’ term. Some items may have follow-up questions as appropriate. The questions will ask about experiential components of the fellowship, such as the successes and challenges of the fellows’ relationship with their host site, their workload, and which trainings were most beneficial. The information gathered in the focus groups will provide explanatory data that the End of Program Survey is not designed to collect.

One focus group will be conducted at the end of each semester (fall, summer, spring). Thus, three focus groups will be held per year. All fellows within each cohort (up to 30 fellows per semester cohort) will be emailed an invitation to participate in the focus group for their cohort. Fellows will be asked to respond to a doodle poll with their availability to voluntarily participate in the focus group. The day and time which has the largest number of fellows who are available will be selected. If more than 10 fellows have indicated availability for that day and time, to adhere to best practices for focus groups, random sampling will be used to select 10 participants. Up to 30 fellows per year will participate in focus groups. The focus groups will be held in Teams.

The Public Health Law Fellowship — End of Program Survey (see attachments H-1 and H-2) will gather information on the fellow’s experience in the fellowship, knowledge gained through the fellowship, content areas on which the fellow gained experience, and relationship with the host site. The survey contains 53 items and will be administered once for each fellow at the conclusion of their fellowship. This survey will consist of both closed-ended and open-ended questions. The survey will contain logic to direct fellows to additional questions as relevant for the content areas on which they worked. For example, if a fellow responds that they did not work on any projects related to health equity, follow up questions will ask whether they were offered an opportunity and declined and if they would have liked an opportunity to work on health equity. Surveys will be administered electronically; a link to the survey web site will be automatically sent via email at the end of the fellowship. Data collected will assess fellows’ perspectives of the PHL Fellowship’s value to the fellow, growth in knowledge of covered curriculum, topic areas on which the fellow worked, and overall satisfaction with the PHL Fellowship.

The Public Health Law Fellowship — Follow-up Survey (see attachments I-1 and I-2) will gather information on the status of alumni careers, career progression, and what factors have influenced the alumni’s career path. The survey contains 11 items and will be administered 6 months after the fellow graduates if the fellow was a student at the time of their fellowship, or 6 months after the fellow completes the fellowship if the fellow graduated prior to beginning the fellowship. Surveys will be administered electronically; a link to the survey web site will be provided in the email invitation. This survey will consist of both closed-ended and open-ended questions. Skip logic and branching methods will be used to direct alumni to questions that are relevant to their current position status. For example, if an alumnus is currently employed, they will be directed to additional questions about their employment status. The data collected from this survey will assess program outcomes, specifically documenting how many alumni are retained as members of the public health workforce, where alumni are employed, what topical and functional public health areas alumni support (e.g., chronic disease, infectious disease, survey, communications), and to what extent alumni support the capabilities of public health agencies at the federal, state, territorial, local, tribal, and non-governmental organizational levels. If alumni are participating in educational programs or other endeavors, the survey will capture their activities.

PHL Fellowship — Host Site Application (see attachments J-1 and J-2) will gather information on the host site’s readiness to host a fellow, prior experience with public health law/policy, and proposed project. The application consists of 36 items. The application link will be available on the Fellowship’s webpage and may be electronically distributed through a listserv to those who have opted in to receive information about hosting a fellow. The application will consist of both closed-ended and open-ended questions. The information collected will assess suitability of host site for the PHL Fellowship. Questions indicate eligibility for the program, prior knowledge and experience with public health law/policy, and proposed projects.

The Public Health Law Fellowship — Host Site Welcome Survey (see attachments K-1 and K-2) will gather information on the host’s expectations for the fellowship, and perceptions of the usefulness of law and legal epidemiology as instruments to positively impact public health. The survey contains 17 items and will be administered once for each host site at the beginning of the fellowship. Surveys will be administered electronically; a link to the survey web site will be sent via email at the start of the fellowship. Questions indicate anticipated outcomes from the fellow’s work, and Likert ratings of the value of law and related research as a tool to affect public health. Data collected will assess changes to the project and oversight of the fellow since the application period, as well as the hosts’ perception of the usefulness of legal epidemiology and public health law.

The Public Health Law Fellowship — Host Site End-of-Program Survey (see attachments L-1 and L-2) will gather information on the host’s experience in the fellowship, satisfaction with the logistics of the program, progress on projects, and perceived value of the fellowship. The survey contains 68 items and will be administered once for each fellow hosted at the conclusion of the fellowship. Questions indicate actual outcomes from the fellow’s work, and Likert ratings of the value of law and related research as a tool to affect public health.

Surveys will be administered electronically; a link to the survey web site will be sent via email at the end of the fellowship. Data collected will assess hosts’ perspectives of the PHL Fellowship’s value to the host and fellow, knowledge gained, progress on projects, and overall satisfaction with the PHL Fellowship.

The Public Health Law Fellowship — Host Site Supervisor Interview (see attachment M-1) will gather information on host site supervisors’ perceptions of the Fellowship’s value to their agencies and their suggestions to improve the program. As the Fellowship is a semester program, the interview will be administered once every semester, to capture the perspectives of new supervisors as fellows cycle in and out of the program. Interviews will be administered via MS Teams. The interview contains 8 open-ended questions with follow-up items to be asked as appropriate. Data collected will gather suggestions for improvement to ensure the program is most effective in facilitating a meaningful host site experience (and overall PHL Fellowship experience) for all involved.

These data collections will be instrumental in helping PHL Fellowship staff learn about important stakeholder perspectives that describe the quality, impact, and value of this fellowship. Data will also inform program improvements such as refining the host site selection and matching process. Collection of this information moving forward will continue to meet these purposes and allow for longitudinal evaluation of the PHL Fellowship, giving program leadership opportunities to see how the PHL Fellowship influences alumni career progression and contributions to public health over time. The results of these surveys may be published in peer reviewed journals and/or in non-scientific publications such as practice reports and/or fact sheets.

Use of Improved Information Technology and Burden Reduction

Survey data will be collected via web-based surveys allowing respondents to complete and submit their responses electronically. Interviews and focus groups will take place via web conferencing platforms such as MS Teams and will be recorded and transcribed. These methods were chosen to reduce the overall burden on respondents. The information collection instruments were designed to collect the minimum information necessary for the purposes of this project.

Efforts to Identify Duplication and Use of Similar Information

Information concerning PHL Fellowship host site supervisors and fellow perspectives and activities has not been collected previously since this is a significant expansion of the intern/externship to the extent that it requires a new program evaluation plan. The purpose of this information collection is to assess the new and active PHL Fellowship program requiring on-going learning of program processes and outcomes to continuously improve quality and demonstrate value to inform program decision making. PHL Fellowship data collection is not being administered through CDC’s Fellowship Management System (FMS) (OMB No. 0920-0765, exp. date 3/31/2026), and thus there is no duplication of data collection effort. Data collection efforts include program-specific focus groups, in-depth interviews, and an alumni survey, all of which distinguish this data collection from FMS data collection. Additionally, PHL Fellowship staff considered FMS as an option for quantitative data collection early in the planning process. However, FMS was unavailable to new projects at the time. As an alternative staff decided to administer quantitative data collection through REDCap.

Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this information collection.

Consequences of Collecting the Information Less Frequently

There are no legal obstacles to reduce the burden. The consequences of not collecting this information would be:

Failure to systematically collect information to document evidence of the effectiveness, value, and impact of the PHL Fellowship program.
Limited guidance to the program on how to adjust and strengthen the PHL Fellowship program.

Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances with this Information Collection Request. This request fully complies with the regulation 5 CFR 1320.5 and will be voluntary.

Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A 60-day Notice was published in the Federal Register on March 24, 2023; Vol. 88, No. 57, pp. 19849-19850 (see attachment B). CDC received one public comment (see attachment B-1). No changes were made to the information collection plan as a result of the public comment process. There were no additional efforts to consult outside the agency.

Explanation of Any Payment or Gift to Respondents

CDC will not provide payments or gifts to respondents.

Protection of the Privacy and Confidentiality of Information Provided by Respondents

The Privacy Office reviewed the project and determined that the Privacy Act does not apply (Attachment C).

Data will be kept private to the extent allowed by law. Responses will be kept in REDCap Main which requires Secure Access Management System (SAMS) authentication and credentials to enter or access data. Only evaluation staff will have access to view or download personally identifiable information, however all fellowship staff will have access to de-identified data. The evaluation lead and evaluation fellow will be the only staff with access to view or download PII for all stages of data collection, management, and analysis. PII from the application may be disclosed to fellowship management staff who need access to interview, place, and extend offers to fellows, as is customary and expected for information submitted in applications. Evaluation staff will download applicant data and share with the fellowship management staff via CDC network Sharefile link sent in an encrypted email.

Institutional Review Board (IRB) and Justification for Sensitive Questions

This data collection is not research involving human subjects; therefore, CDC IRB approval is not required for this project (see attachment D). Some instruments ask about race and disability, which can be considered sensitive information. Because the funding for this program requires efforts to increase diversity, equity, inclusion, and accessibility in the public health workforce, we must have information about fellows’ race and disability status to determine if our efforts are resulting in an increase of racial diversity and accessibility for those who identify as having a disability.

Estimates of Annualized Burden Hours and Costs

All instruments were pilot tested:

PHL Fellow Application (see attachment E-1): The estimate for burden hours is based on pilot response completed by program staff. Pilot responses took 5–7 minutes to complete. For the purposes of estimating burden, the upper limit of 7 minutes was used.

PHL Fellow Welcome Survey (see attachment F-1): The estimate for burden hours is based on pilot response completed by program staff. Pilot responses took 3–6 minutes to complete. For the purposes of estimating burden, the upper limit of 6 minutes was used.

PHL Fellow Focus Group (see attachment G-1): The estimate for burden hours is based on time allocated to complete the focus group. Focus groups will be scheduled for one-hour increments. A subpopulation of 30 fellows will be randomly selected to participate.

PHL Fellow End-of-Program Survey (see attachment H-1): The estimate for burden hours is based on pilot responses by staff. Pilot responses took 4–7 minutes to complete. For the purposes of estimating burden, the upper limit of 7 minutes was used.

PHL Fellowship Alumni Survey (see attachment I-1): The estimate for burden hours is based on pilot responses by staff. Pilot responses took 2–10 minutes to complete. For the purposes of estimating burden, the upper limit of 10 minutes was used.

PHL Fellowship Host Site Application (see attachment J-1): The estimate for burden hours is based on pilot response completed by program staff. Pilot responses took 9–21 minutes to complete. For the purposes of estimating burden, the upper limit of 21 minutes was used.

PHL Fellowship Host Site Welcome Survey (see attachment K-1): The estimate for burden hours is based on pilot responses completed by program staff. Pilot responses took 2–5 minutes to complete. For purposes of estimating burden, the upper limit of 5 minutes was used.

PHL Fellowship Host Site End-of-Program Survey (see attachment L-1) The estimate for burden hours is based on pilot response completed by program staff. Pilot responses took 7–12 minutes to complete. For the purposes of estimating burden, the upper limit of 12 minutes is used.

PHL Fellowship Host Site Supervisor Interview (see attachment M-1): The estimate for burden hours is based on time allocated to complete the interview. Interviews will be scheduled for one-hour increments.

Table A shows the estimated burden for respondents by type and form.

Table A: Estimated Annualized Burden Hours

Type of Respondent	Form Name	No. of Respondents	No. of Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
PHL Fellowship Applicants	PHL Fellow Application	200	1	7/60	23
PHL Fellowship Participants	PHL Fellow Welcome Survey	70	1	6/60	7
PHL Fellowship Participants	PHL Fellow End-of-Program Survey	70	1	7/60	8
PHL Fellowship Participants	PHL Fellow Focus Group	30	1	60/60	30
PHL Fellowship Alumni	PHL Fellowship Alumni Survey	70	1	10/60	12
PHL Fellowship Host Site Applicants	PHL Fellowship Host Site Application	50	1	21/60	18
PHL Fellowship Host Site Supervisors	PHL Fellowship Host Site Welcome Survey	40	1	5/60	3
PHL Fellowship Host Site Supervisors	PHL Fellowship Host Site End-of-Program Survey	40	1	12/60	8
PHL Fellowship Host Site Supervisors	PHL Fellowship Host Site Supervisor Interview	40	1	60/60	40
	Totals				149

Estimates for the average hourly wage for hosts and alumni are based on the US Department of Labor (DOL) National Occupational and Wage Estimates for life, physical, and social science occupations.3 Based on DOL data, an average hourly wage of $40.21 is estimated for all host respondents. Estimates for the average hourly wage for fellows are based on the average hourly payment provided to fellows, $24.00/hour. Table B shows estimated cost information.

Table B. Estimated Annualized Burden Costs

Type of Respondent	Form Name	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
PHL Fellowship Applicants	PHL Fellow Application	23	$24.00	$552
PHL Fellowship Participants	PHL Fellow Welcome Survey	7	$24.00	$168
PHL Fellowship Participants	PHL Fellow End-of-Program Survey	8	$24.00	$192
PHL Fellowship Participants	PHL Fellow Focus Group	30	$24.00	$720
PHL Fellowship Alumni	PHL Fellowship Alumni Survey	12	$24.00	$288
PHL Fellowship Host Site Applicants	PHL Fellowship Host Site Application	18	$40.21	$724
PHL Fellowship Host Site Supervisors	PHL Fellowship Host Site Welcome Survey	3	$40.21	$121
PHL Fellowship Host Site Supervisors	PHL Fellowship Host Site End-of-Program Survey	8	$40.21	$322
PHL Fellowship Host Site Supervisors	PHL Fellowship Host Site Supervisor Interview	40	$40.21	$1608
Totals				$4695

Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There is no cost to respondents other than their time to complete the data collection. There is no additional cost burden to respondents.

Annualized Cost to the Government

The government costs include personnel costs for federal staff involved in project oversight, data collection, analysis, and reporting. The total cost to the federal government is $5525.

Table C: Estimated Annualized Cost to the Federal Government

Staff	Average Hours per Collection	Average Hourly Rate	Average Cost
GS-14, step 1 Public Health Analyst	110	$50.23	$5525
Total			$5525

Explanation for Program Changes or Adjustments

This is a new information collection request.

Plans for Tabulation and Publication and Project Time Schedule

The following schedule will be followed for each data collection instrument.

Project Time Schedule

Activity	Time Schedule
Email invitation sent to respondents	Upon approval in accordance with data collection plan
Data Collection	Upon approval in accordance with data collection plan
Analyses	Approximately within 3 months of close of data collection
Report Developed	Approximately within 6 months of close of data collection: communicate to leadership, program, or stakeholders about results and recommendations for improvement or actions
Publication of Report	The project team will determine if this step is appropriate based on data analysis. If appropriate, findings will be submitted 6 months or more from close of data collection.

Reason(s) Display of OMB Expiration Date is Inappropriate

Display of the OMB Expiration Date is appropriate for this information collection.

Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

LIST OF REFERENCES – Section A

Centers for Disease Control and Prevention. About CDC: Mission, Role, and Pledge. Accessed 02.06.2023; http://www.cdc.gov/about/organization/mission.htm

Centers for Disease Control and Prevention. CDC Public Health Law Educational Opportunities. Accessed 06.08.23; https://www.cdc.gov/phlp/externship.html

Bureau of Labor Statistics. May 2022 National Occupational Employment and Wage Estimates United States for Life, Physical, and Social Science Occupations. Accessed 06.08.2023; http://www.bls.gov/oes/current/oes_nat.htm

Page 14 of 14

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	CDC User
File Modified	0000-00-00
File Created	2023-11-01