Supporting Statement Part A - revised June 2015

Supporting Statement Part A - revised June 2015.docx

Fatherhood and Marriage Local Evaluation and Cross-Site Data Collection

OMB: 0970-0460

Document [docx]
Download: docx | pdf



U.S. Department of Health and Human Services


Administration for Children and Families


Office of Planning, Research, and Evaluation


Aerospace 7th Floor West

901 D Street, SW

Washington DC 20447


Federal Project Officer:

Seth Chamberlain



OMB Supporting Statement for the Fatherhood and Marriage Local Evaluation and Cross-Site Data Collection

Part A: Justification

March 23, 2015











LIST OF ATTACHMENTS



ATTACHMENT A: LEGISLATIVE AUTHORITY


ATTACHMENT B: 60-DAY FEDERAL REGISTER NOTICE


ATTACHMENT C: SUMMARY OF PUBLIC COMMENTS AND ACF RESPONSES


ATTACHMENT D: CONSENT STATEMENT FOR HM AND RF FOCUS GROUP PROGRAM PARTICIPANTS


ATTACHMENT E: RECOMMENDED LANGUAGE FOR GRANTEE CONSENT FORMS


ATTACHMENT F: MATHEMATICA EMPLOYEE CONFIDENTIALITY PLEDGE


ATTACHMENT G: ASSUMPTIONS FOR CALCULATION OF BURDEN ESTIMATES IN TABLE A.4


ATTACHMENT H: FOCUS GROUP RECRUITMENT LETTER AND PHONE SCRIPT

A1. Circumstances making the Data Collection Necessary

This information collection request (ICR) is for clearance to collect information for the Fatherhood and Marriage Local Evaluation (FaMLE) and Cross-Site Project. The FaMLE Cross-Site Project will gather information from the next round of healthy marriage (HM) and responsible fatherhood (RF) grantees; the funding is authorized under Sec. 811 (b) Healthy Marriage Promotion and Promoting Responsible Fatherhood Grants of the Claims Resolution Act of 2010, Pub. L. No. 111-291, 124 Stat. 3064 (Dec. 8, 2010). A copy of the legislative authority is included as Attachment A. The HM and RF grants are expected to be awarded in fall 2015.

The project is being undertaken by the U.S. Department of Health and Human Services, Administration for Children and Families (ACF), and is being implemented by Mathematica Policy Research.

A. Background

Healthy marriage and responsible fatherhood (HMRF) programs have been undergoing a transformation in the past few decades. At first a new approach for serving vulnerable families, such programs have become an established presence in many communities, with connections to other agencies and a growing number of families served. Responsible Fatherhood (RF) programs began in the 1990s with such efforts as the Young Unwed Parents program, Parents’ Fair Share, and Partners for Fragile Families. In the early 2000s, ACF announced the Healthy Marriage Initiative, which provided funding to federal grantees through existing legislative authorities to add marriage education to their service offerings. This effort coincided with findings from the longitudinal Fragile Families and Child Well-being Study that suggested the period around a child’s birth could be an opportunity for intervening with unmarried couples, who typically were romantically involved and interested in marriage (McLanahan et al. 2001).

The Deficit Reduction Act of 2005 created the HMRF grant program, which authorized $150 million over five years to support program activities aimed at promoting and sustaining healthy marriages, fostering economic stability, and promoting responsible parenting. The Claims Resolution Act of 2010 re-authorized this grant program, and three-year grants totaling $150 million were awarded in September 2011 (they were subsequently extended for a fourth year, i.e. through September 2015). The Office of Family Assistance (OFA), Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) administers the HMRF grant program and, to date, 347 HMRF grants have been awarded (some agencies have received successive grants).

As the reach and variety of HMRF programs grow, so too does interest in their operations and effectiveness. Some research results have accumulated—largely from a few multisite, federally funded evaluations. For example:

  • The Building Strong Families project (BSF) and the Supporting Healthy Marriages (SHM) evaluations employed rigorous research designs to evaluate Healthy Marriage (HM) programs aimed at unmarried parenting couples (BSF) and married parents (SHM). These evaluations focused on a small number of programs with the capacity to recruit large samples and meet other "evaluability" criteria.

  • Also a rigorous experimental design, the Parents and Children Together (PACT) Evaluation is assessing the implementation and effectiveness of four RF and two HM programs chosen from among the 2011 cohort of grantees.1

Strong evidence still is limited, however, and is derived mostly from a narrow range of programs selected more for their suitability for such studies than their representativeness of HMRF programs in general. According to an extensive review of HM and RF program evaluations conducted by Mathematica Policy Research (Avellar 2011, 2012), local evaluations carried out on grant-funded HMRF programs often are small, and most do not use rigorous methods. About 20 percent of RF and HM programs that have been the subject of research were assessed with designs that were moderately or well designed to detect the effects of the program. Most other studies have reported participant outcomes but were unable to determine if programs or other factors caused observed changes or differences, due to evaluation design (Avellar 2011, 2012). Policy makers need to know more about what works to guide future funding decisions, and program operators need to understand what makes for an effective program to guide future programming.

B. Overview of the project

In anticipation of the next round of HMRF grant funding expected to be awarded in September 2015, ACF contracted with Mathematica Policy Research to conduct the FaMLE Cross-Site project, with the dual goals of supporting quality and consistent collection of performance measures data, and fostering strong local evaluations by the next round of HMRF grantees. More specifically, Mathematica will: (1) collect information on the design and implementation of grantees’ programs; and (2) develop data collection tools that grantees can use to collect information on program participation and participant outcomes for use in reporting performance and conducting evaluations.

The project will rely on two complementary sources of data collection: (1) data collection by the contractor for cross-site analysis of program design and implementation, hereafter referred to as DCI (Data collected by the Contractor Itself); and (2) data collection by the grantees themselves for performance reporting and cross-site analysis, hereafter referred to as DCS (Data Collected for Cross-Site analyses).

1. DCI

The DCI effort will focus on program design and implementation. Information for this effort will be collected via document review, a series of stakeholder interviews with grantee staff and partner organizations, and focus groups with clients. All document reviews will include materials readily available to ACF without imposing burden on any participants.

DCI will occur in four stages (Figure 1). The first two stages will be focused on a broad group of grantees and will document program design, while stages three and four, focused on a narrower group of grantees, will document program implementation. Stage one will consist of document reviews examining the grant applications of all (approximately 120) HMRF grantees, followed by telephone interviews in stage two probing a subset of approximately 60 grantees for additional detail on program design. In stage three, further document review of performance reports will focus on a subset of about 20 grantees within the group of 60 contacted during stage two. The documents for review at this stage will be those that grantees must submit to ACF as part of performance monitoring. In stage four, telephone interviews and site visits will probe for additional detail on program implementation among these 20 grantees, adding depth to the implementation information gathered during stage three.

Table A.1. Overview of stages with increasingly focused subsets of grantees

Stage

Purpose

Data Source

Number of programs

1

Collect basic program design information

Grant applications

120

2

Confirm and further explore program design information with a subset of well-designed programs

Telephone interviews

60

3

Review basic program implementation information

Semiannual reports

20

4

Confirm and further explore program implementation

Telephone interviews, site visits, client focus groups

20



2. DCS

The DCS effort will focus on performance data to be collected by the next round of HMRF grantees, which will be used by ACF to monitor program performance and to conduct cross-site analyses. As a condition of their grant award, all HMRF grantees will be required to collect information on:

  1. Program applicant characteristics

  2. Program operations

  3. Enrollment and participation in program services

  4. Participant outcomes, pre- and post.

Grantees will provide these data by using a management information system (MIS) called Information, Family Outcomes, Reporting, and Management (or simply, nFORM) to be developed and managed by ACF (specifically, by the contractor for this project).

C. Data Collection Activities Requiring Clearance

This ICR includes eight data collection activities and two data reporting activities:

  • DCI: Data Collection. Three DCI activities will be carried out by contractor staff to collect information on the design and implementation of grantees’ programs.

  • DCS: Data Collection. The remaining three are DCS data collection activities, which will be conducted by the grantees themselves to document program applicant characteristics, program operations, services received, and outcomes for program participants.

  • DCS: Performance Reporting. Grantees will be required to submit two semi-annual reports and two quarterly reports to OFA each grant year; these reports will draw upon a subset of the DCS data described above.

These data collection and reporting activities are described along with the instruments that will be used for each in Section A.2.



A2. Purpose and Use of the Information Collection

The information to be obtained through the FaMLE Cross-Site project is critical to understanding the broad array of RF and HM programs to be funded—the services provided, the context in which they operate, the nature and extent of participation, and the outcomes for program participants. Information will be used to report performance to OFA and for cross-site analyses. The goal is to better understand the design, operations, and benefits of HMRF programs, thereby informing decisions about future government investments in HMRF programming.

1. DCI: Data Collection

The information collected by the contractor for DCI will help clarify the processes and contextual factors associated with program design and implementation, and will supplement gaps in information about these topics that will be gathered via document review. To achieve these purposes, we propose collecting the following information.

  • Program design. A topic guide will be used to conduct semi-structured telephone interviews with lead program staff at the approximately 60 grantees selected for stage two DCI data collection. The semi-structured Instrument DCI-1, Grantee Staff Topic Guide on Program Design, will focus on information about program design, including clarifying information from the grant application, perceived strengths and weaknesses, and the array of services grantees intend to offer. This will help the project clearly document aspects of strong program design and analyze these across sites to provide information for ACF and other programs to use when designing future programs.

  • Implementation. A topic guide will be used to conduct semi-structured interviews, by phone or in person, with a range of program staff from the approximately 20 grantees selected in stage three to learn about program implementation. The semi-structured Instrument DCI-2, Grantee Staff Topic Guide on Implementation, will focus on information about program implementation. The interviews will focus on the timing, duration, and frequency of program activities and services the grantee delivered; enrollment and participation; characteristics of clients who enroll and participate; actual staffing, and successes and challenges. The interview data will allow the project to gain a deeper understanding about how a subset of grantees with strong program design actually implemented and operated their program. This information will help ACF and other programs identify practical examples, promising practices, and lessons learned when implementing future programs.

  • Program participants. A topic guide will be used to conduct four (in-person) focus groups with participants at each of the 20 stage-three grantees to learn about their program experiences. Instrument DCI-3, the Program Participant Focus Group Topic Guide, will be used to explore and document program participants’ perspectives on their motivation for enrolling in the program, and the availability, quality, and value of program services. Of particular interest will be participants’ level of satisfaction with the program and their assessment of the knowledge and skills gained as a result of program participation. This information will help ACF and programs learn which aspects of RFHM programs are of greatest interest and value to clients.



2. DCS: Data Collection

The information collected by the HMRF grantees for DCS will be used: (1) in analyses of program design and implementation; (2) in cross-site analyses of outcomes across all grantees and impacts among a subset of grantees conducting experimental or quasi-experimental evaluations; and (3) for reporting program performance to OFA. DCS data may also be used for special topics reports as requested by ACF. The specific use and purpose of each of the DCS information collection activities is described below.2

  • Applicant characteristics. All HMRF grantees (an estimated 120) will collect and enter information about fathers and couples applying to the program. HMRF grantees will collect information on demographic characteristics (e.g., gender, age); financial well-being (e.g., employment status); family status (e.g., marital and parenting status); health and well-being (e.g., psychological distress); and how the program applicant heard about the program and reasons for enrolling. This information will be collected from all program applicants at program intake using Applicant Characteristics, Instrument DCS-1. Information from DCS-1, will be used in cross-site descriptive analyses of who applies to and who eventually participates in HMRF programs, and for reporting participant characteristics to OFA in the semi-annual reports. We estimate that approximately 35 HMRF grantees will conduct experimental or strong quasi-experimental local evaluations. Applicant characteristics may also be used by local evaluators and for the cross-site analysis of impacts as covariates in a regression model to increase the precision of the impact estimates, to check that the characteristics of program and control group members are on average similar at baseline, and to predict program participation in analyses of the impacts among those who actually participate in the program(s).

  • Program operations. All HMRF grantees will collect and enter data on: (1) strategies used to market to and recruit fathers and couples into their programs (such as the amount and types of mass marketing strategies; recruitment methods; and the number of FTE staff dedicated to marketing, outreach, and recruitment); (2) practices to monitor quality (such as staff training, staff supervision, and program observations); (3) staff qualifications (including the proportion of staff with various levels of educational attainment, training and years of experience), and (4) implementation challenges (such as staff turnover and recruitment challenges) using Program Operations, Instrument DCS-2. Grantees will enter this information quarterly and it will be used for reporting performance to OFA semi-annually and in cross-site descriptive analyses of program design and implementation.

  • Service receipt. All HMRF grantees will collect and enter data on program services offered and fathers’ and couples’ participation in these services using Service Receipt, Instrument DCS-3 (wireframes which show the information grantees will enter on service receipt). Attendance in program activities will be documented immediately (for example, from workshop sign-in sheets) and will be entered into nFORM at least once a week. HM grantees will track participation at both the individual and couple levels. This information will be used for reporting performance to OFA in the quarterly and semi-annual reports and in cross-site descriptive analyses of program implementation.

  • Self-Administered Questionnaires (SAQs) upon program entry (pre-test) and program exit (post-test). All HMRF grantees will ask participants to complete a SAQ assessing information at program entry (at the first workshop attended) and at program exit (the last core program activity), or one month post-exit if the program is structured to last less than one month. HMRF grantees will collect information in five outcome domains: (1) parenting, co-parenting, and fatherhood; (2) economic stability; (3) healthy marriage and relationships; and (4) personal development; and (5) program perceptions. Within these five domains, there are 21 outcome constructs, as shown in Table A.2. Twelve of these constructs are common to both RF and HM grantees given their common program components, another four constructs (pertaining to healthy relationships and marriage) are specific to HM grantees, and another five constructs (pertaining to parenting, co-parenting, and fatherhood) are unique to RF grantees.

The English version of the pre- and post-test SAQs for the two types of RF grantees are included in this ICR as Instruments DCS-4HM and DCS-4-RF. There are two versions of the pre- and post-program SAQs for HM grantees: one for adult populations (DCS-4HM.1 and DCS-4HM.2), and one for youth populations (DCS-4HM.3 and DCS-4HM.4). There are also two versions of the pre- and post-test SAQs for RF grantees: one for fathers residing in the community (DCS-4RF.1 and DCS-4RF.2, and one for incarcerated fathers (DCS-4RF.3 and DCS-4RF.4. These instruments will also be translated into Spanish.

Outcomes data collected through the pre- and post-tests (Instrument DCS-4HM and DCS-4RF) will be used for reporting performance to OFA semi-annually and in cross-site descriptive analyses. For the estimated 35 HMRF grantees conducting experimental or strong quasi-experimental evaluations, outcomes data for both program and control/comparison groups may be used by local evaluators and the cross-site analysis to calculate the average impacts of program participation. Measuring many of the same variables at pre-test and post-test will increase the precision of estimate impacts. Outcomes data may also be used for a special topics report as requested by ACF.

Table A.2. Unique and Overlapping Constructs for RF and HM Grantees, by Outcome Domain


Outcomes by Grantee Type

Outcome Domain

HM Grantees Only

Both HM and RF Grantees

RF Grantees Only

Parenting/

Co-parenting

None

  • Parenting attitudes

  • Parenting skills/behavior

  • Parenting alliance


  • Engagement with child

  • Amount of contact with child

  • Attempts to connect with child

  • Responsibility for child’s financial support

  • Knowledge of child support

Economic Stability

None

  • Ability to manage money

  • Human capital development

  • Employment

None

Healthy Marriage/ Relationship

  • Relationship stability

  • Relationship quality

  • Infidelity

  • Attitudes toward sex (youth only)

  • Attitudes toward marriage

  • Communication & conflict management

None

Personal Development

None

  • Psychological well-being

  • Social support (HM youth only)

None

Program Perceptions

None

  • Perceived helpfulness

  • Other thoughts

None


3. DCS: Performance Reporting

Grantees will be required to report on their performance using the data they collect through the DCS task. A description of and the purpose of each of the two kinds of performance reports is described below.

  • Semi-annual performance progress report (PPR, Instruments DCS-5HM and DCS-5RF). The semi-annual PPR will include quantitative descriptive information on program applicants and quantitative information on a subset of DCS performance measures pertaining to program operations (staff training and supervision; marketing, outreach, and recruitment; and implementation challenges); program enrollment; and program participation. The semi-annual report will also include narrative descriptions of the grantee’s major activities and accomplishments; implementation challenges and steps taken to address these challenges; program successes; and emerging promising practices. The template for use by HM grantees is included in this ICR as Instrument DCS-5HM, and the template for use by RF grantees is included in this ICR as Instrument DCS-5RF.

Grantees are required by ACF’s Office of Grants Management (OGM) to submit a PPR twice during each grant year (in October and April), reporting on the programmatic activities conducted by the grantee in the prior six months and activities planned for the next six months. This report meets OGM reporting requirements and will be used by grantees to self-monitor semi-annual performance and by the Office of Family Assistance (OFA) to monitor and manage these grants.

  • Quarterly Performance Report (QPR, Instruments DCS-6HM and DCS-6RF). Grantees will also report on a subset of the (quantitative) performance measures reported in the semi-annual performance report—namely, staff training and supervision; program enrollment; program participation; and implementation challenges–on a quarterly basis. Grantees will be required to submit these two quarterly reports (in January and July) to provide an interim view of performance using a subset of the performance measures reported in the semi-annual PPR. OFA and the grantees will use these quarterly reports to assess progress and identify areas for improvement.

The DCS performance reporting will be facilitated by nFORM, which will automatically pre-populate the PPR or QPR with required performance measures from the data grantees collect and enter into nFORM.

A3. Use of Improved Information Technology and burden reduction

The FaMLE Cross-Site Evaluation will use the following technology to collect information for the DCI and DCS study components.

1. DCI: Data Collection

Audio recording, with respondent permission, will be used to facilitate interviewer-participant dialogue and interaction without distraction of extensive note taking and to increase accuracy of documentation of all points raised during the focus group discussions with clients.

2. DCS: Data Collection

As part of the FaMLE Cross-Site project, the contractor will develop and operate an MIS, called nFORM, which grantees will use to enter performance measures data (applicant characteristics, program operations, enrollment and participation, and participant outcomes). The web-based MIS will have a user-friendly interface accessible to authorized users from any computer with internet access, allowing for ease of data entry without purchasing or installing additional software or changing the configuration of their computers. All data will be housed on secure servers behind the contractors’ firewall, thereby maintaining data security. Each grantee will be able to view and report only data for its own program. The nFORM system is an adaptation of a similar MIS that the contractor designed for HMRF grantees participating in the PACT Evaluation. This web-based MIS will reduce grantees’ reporting burden by providing a convenient and simple method for submitting data electronically.

Program participants will use computers or tablets to self-administer the pre- and post-tests in nFORM. This method presents several advantages over interviewer-administered surveys. It will ensure greater privacy, and respondents will be more likely to avoid socially desirable responses, particularly with sensitive questions (Turner et al. 1998; Tourangeau and Smith 1996). It will also reduce burden for grantee staff who would otherwise need to administer the surveys. To address possible literacy limitations, respondents will have the option to wear headphones and listen to a recording of the questions, known as Audio Computer-Assisted Self-Interview (A-CASI).

3. DCS: Performance Reporting

nFORM will allow grantees to generate the required quantitative performance measures for quarterly reporting to OFA with only the touch of a button, thereby minimizing grantee burden while maximizing cross-site consistency and quality of performance data.

A4. Efforts to Identify Duplication and Use of Similar Information

There are no other sources of information that would allow us to assess the design, implementation, and outcomes of all ACF-funded RF and HM programs for the upcoming round of grant funding. We do propose to use measures that have successfully been used in prior studies involving similar populations and programs. No superfluous or unnecessary information is being requested of program staff or participants from the upcoming round of HMRF grants. We will ensure that we do not collect information that is available elsewhere. None of the instruments will ask for information that can be reliably obtained through other sources.

The local evaluations will allow grantees to propose and answer a research question of interest that will be specific to their program design and will inform their program and service provision as well as the responsible fatherhood and healthy marriage field. The grantee will use measures of their choice to assess their outcomes of interest and these measures may or may not draw upon the proposed performance measures. In contrast, the proposed cross-site evaluation component of the ACF project will conduct cross-site analyses across all grantees and if possible, will examine average impacts of program participation across a subset of the grantees conducting experimental or quasi-experimental local evaluations. The ACF cross-site analyses will draw only on the proposed performance measures.



A5. Impact on Small Businesses or Other Small Entities

The potential exists for data collection activities to affect small entities associated with the grantee. HMRF grantee partners and direct service providers will be included as part of DCI interviews. Additionally, HMRF grantees may conduct local evaluations led by local evaluators; if so, they may task the local evaluator with the collection of some or all of the DCS performance measures data. Proposed data collection efforts are designed to minimize the burden on all organizations involved, including small businesses and entities, by collecting only critical information.

A6. Consequences of Not Collecting Information or Collecting Information Less Frequently

The purpose of each information collection instrument included in this submission is described in Item A2, above. Not collecting information using these instruments would limit the government’s ability to document the performance of its grantees and to assess the extent to which these federal grants are successful in achieving their purpose. Furthermore, the FaMLE Cross-Site Project provides a valuable opportunity for OFA, practitioners, and researchers to gain empirical knowledge about the design and implementation of a broad range of HMRF programs and the characteristics of and outcomes for program participants.

Specifically, without the information collected through grantee staff interviews and participant focus groups, the FaMLE Cross-site Project would have to rely entirely on implementation information reported by a single source: the HMRF grantee semi-annual progress reports. Thus, the study would lack the broader perspectives of other staff and program participants, and we would be severely hampered in our understanding of how HMRF grantees design and implement their programs, critical program challenges and successes, and what leads applicants to participate fully in program services.

In addition, without collecting information on applicant characteristics (Instrument DCS-1), program operations (Instrument DCS-2), service receipt (Instrument DCS-3), and participant outcomes (collected through SAQs, Instrument DCS-4), HMRF grantees would not be able to report on the required performance measures, and the cross-site evaluation would be unable to link participant outcomes to various implementation factors (such as levels or combinations of specific services received).

If service receipt data were collected less frequently, providers would have to store service data or try to recall it weeks or months after delivery. Less frequent data collection would also reduce our ability to identify and address data quality issues, such as missing data and data entry errors, in a timely way. Finally, if participant outcomes were not collected at both program entry and program exit (or one month after program exit), we would not be able to assess changes in outcomes pre- and post-program participation, which is required for grantees to report performance to OFA.

A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the general information collection guidelines of 5 CFR 1320.5(d) (2). No special circumstances apply to the proposed data collection.

A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

In accordance with the Paperwork Reduction Act of 1995, the public was given an opportunity to review and comment through the 60-day Federal Register Notice, published on November 6, 2014, (Vol. 79, No. 215, pp. 65973- 65974). A copy of this notice is attached as Attachment B. The notice provided 60 days for public comment.

ACF received 57 requests for the measures and 28 emails with comments during the 60-day period, most containing multiple comments. Responses were submitted by current grantees and researchers. There was some support expressed for the measures and ACF’s intent to increase our understanding of Healthy Marriage and Responsible Fatherhood programs. Comments were related to literacy levels, survey length, appropriateness of questions, the youth survey, case management expectations, mode of administration, and quarterly reporting. Attachment C provides a summary of the comments received during the 60-Day period, and ACF’s responses. This summary has been provided to the public as part of the 30-Day Federal Register Notice.

A9. Explanation of Any GiftS to Respondents

In stage four of DCI data collection, we propose to provide a $25 gift card for each focus group participant in order to reduce anticipated nonresponse bias and to offset participation costs. We expect the focus groups to be approximately 90 minutes long. Research has shown that such tokens of appreciation are effective at increasing response rates for populations similar to participants in HMRF programs—people with lower educational level (Berlin et al. 1992) and low-income and nonwhite populations (James and Bolstein 1990).

A10. Assurance of Privacy Provided to Respondents

Respondents to Instrument DCI-3 (Program Participant Focus Group Topic Guide) will be informed that the identifying information they provide will be kept private. Participants will be given a hard copy of the consent statement for their records (Attachment D contains the consent statement for RF and HM program participants). All consent forms that are given to focus group participants will include assurances that the research team will protect their privacy to the fullest extent possible under the law. At the beginning of each focus group, the data collectors will state that the information provided by the respondent will be kept private and that the results of the study will be presented in aggregate form only. All focus group respondents will be provided with the informed consent form before the interviews are conducted. The consent form will explain the purpose of the evaluation, the duration of the interviews, and any benefits, risks, or discomfort involved.

For DCS data collection, grantees will be responsible for obtaining the necessary IRB approvals for their data collection and submission to the FaMLE Cross-Site Project.

In all data collection and performance reporting efforts, ACF will take the following specific measures to protect respondents’ privacy:

  • Adopt strict security measures and web security best practices to protect data collected through the project MIS, called nFORM. Data entered into nFORM will be housed on secure servers that conform to the requirements of the HHS Information Security Program Policy. This MIS will employ strict security measures and web security best practices to ensure the data will be submitted, stored, maintained, and disseminated securely and safely. Strict security measures will be employed to protect the privacy of participant information stored in the system including data authentication, monitoring, auditing, and encryption. Specific security procedures include, but are not limited to:

  • All data will be encrypted in transit (using TLS protocol backward compatible to SSL).

  • Data will be encrypted at rest and reside behind firewalls.

  • nFORM users will be able to access the system only within the scope of their assigned roles and responsibilities:

    • Only authorized contractor staff will have access to the securely-held individual-level data.

    • Other FaMLE Cross-Site Project staff will have access only to auto-generated reports that provide aggregated information only.

    • Only authorized staff at each grantee will be able to view all individual-level data for their participants. Other staff will have access to auto-generated reports that provide aggregated information only.

  • Authorized research staff will be assigned a password only with permission from the study director. Each user will have a unique user id/password combination.

  • Security procedures will be integrated into the design, implementation, and day-to-day operations of the portal.

  • To further ensure data security, project personnel are required to adhere to strict standards, receive periodic security training, and sign security agreements as a condition of employment.

  • The nFORM system will develop and implement standard procedures for assigning identification numbers to all participant-level data. Case- and individual-level numbers will be content-free. For example, they will not include special codes to indicate enrollment dates, participant location, gender, age, or other characteristics. Data extracts from nFORM, which may not be secured, will include IDs and not PII.

  • Protecting personally identifiable information (PII). The Office of Planning, Research and Evaluation is in the process of publishing a Privacy Impact Assessment (PIA) to ensure that information handling conforms with applicable legal, regulatory, and policy requirements regarding privacy; determine the risks of collecting and maintaining PII; assists in identifying protections and alternative processes for handling PII to mitigate potential privacy risks; and communicates an information system’s privacy practices to the public. This PIA, titled ACF Research and Evaluation Studies, will be available online through the Department of Health and Human Services.

Regarding this ICR, grantees and local evaluators will be collecting data from participants, including PII, such as name and contact information. The data will be stored in nFORM, which will hosted and maintained by Mathematica. Each grantee will be required to obtain IRB approval for their data collection. The IRBs will be responsible for reviewing and approving the procedures that grantees have in place for protecting PII. Mathematica has developed recommended language for the consent form that explains that data, including PII, will be shared with Mathematica (as required to host nFORM and for the cross-site analyses)(Attachment E). Mathematica will secure a memorandum of understanding with each grantee to share data, and also will prepare guidelines for grantees and local evaluators for protecting PII. Only contractor staff responsible for ensuring data quality will have access to PII; research staff will have access only to the de-identified data. Limiting the number of contractor staff with access to PII will reduce the risk of disclosure.

  • Training cross-site evaluation interviewers in privacy procedures. All site visit interviewers will be knowledgeable about privacy procedures and will be prepared to describe them in detail or to answer any related questions raised by respondents. During the introduction to each interview, site visit informants will be told that none of the information they provide will be used for monitoring or accountability purposes and that the results of the study will be presented in aggregate form only.

In addition to these study-specific procedures, the contractor has extensive corporate administrative and security systems to prevent the unauthorized release of personal records, including state-of-the-art hardware and software for encryption that meets federal standards, other methods of data protection (e.g., requirements for regular password updating), and physical security that includes limited key card access and locked data storage areas.

Finally, the contractor requires every employee to sign a pledge to protect the privacy of data and respondent identity, and breaking that pledge is grounds for immediate dismissal and possible legal action. A copy of that pledge is provided as Attachment F.

In accordance with the requirements of the Privacy Act of 1974, as amended (5 U.S.C. 552a), ACF/OPRE established system of records titled: 09-80-0361 OPRE Research and Evaluation Project Records, HHS/ACF/OPRE. A Federal Register Notice (80 FR 17893) announced the system.

A11. Justification for Sensitive Questions

There are no sensitive questions in the protocols for the DCI data collection.

For the DCS data collection, some of the items that grantees will be required to collect may be considered sensitive questions. Some sensitive questions are necessary when a key project goal is the development of performance measures and when the programs involved are designed to affect personal relationships and employment. Grantees will be responsible for obtaining the necessary IRB approvals for their data collection, including the necessary consent procedures. Table A.3 lists these topics and the justifications for including them.

Table A.3. Sensitive Topics and Justification for Inclusion

Sensitive topic

Relevant instrument(s)

Justification

Attitudes about sex

DCS-4HM (youth questionnaire only)

Healthy marriage and relationship programs for youth in high school aim to prevent nonmarital childbearing by educating youth on the disadvantages that most children face when they are born outside of marriage. Attitudes and intentions regarding engaging in sex are strong predictors of subsequent behavior (Buhi and Goodson 2007), in particular, sexually active teens are more likely to cohabit as young adults (Raley et al. 2007). These questions were adapted from the Toledo Adolescent Relationships Study, the PREP evaluation, and from Connections: Dating and Emotions (Kay Reed, Dibble Institute).

Infidelity

DCS-4HM

Infidelity has been found to be a major obstacle to marriage for unwed parents (Edin and Kefalas 2005). The curricula used by the HMRF programs addresses this in different ways, including discussing the importance of fidelity and trust in building healthy relationships and marriage. Several large surveys have included similar questions concerning infidelity, such as the Study of Marital Instability Over the Life Course, the Louisiana Fragile Families Study, and the Baseline Survey of Family Experiences and Attitudes in Florida. These questions were also used in the Building Strong Families 15- and 36-month follow-up surveys and had low nonresponse rates (Wood et al. 2010).

Psychological distress

DCS-4HM, DCS-4RF

Psychological distress is likely to affect key RFHM goals—improved parenting, employment, and relationship quality—and thus may be an important mediator of program outcomes. Symptoms of parental depression and anxiety have been shown to have adverse consequences for child outcomes (Downey and Coyne 1990, Gelfand and Teti 1990). To measure psychological distress, we will use the K-6, a brief but highly reliable and valid measure frequently used in government health surveys in the U.S. and Canada and by the World Health Organization (Kessler et al. 2002).

Harsh discipline

DCS-4HM, DCS-4RF

A measure of harsh disciplinary practices will enable us to determine whether the HMRF programs’ emphasis on conflict management and parenting skills leads to a reduction in the use of harsh discipline techniques among participants. These items were adapted from the Supporting Healthy Marriages evaluation, where they were successfully used with a population of low-income married couples with children (Lundquist et al. 2014).

Criminal history

DCS-4HM, DCS-4RF

Recent research suggests that a history of incarceration and involvement with the criminal justice system may be fairly common among men in the RFHM target population (Zaveri et al. 2014; Pearson et al. 2011). Incarceration has major negative effects on child and family well-being, including reducing the financial support and other types of support adults can provide to their partners, children, and families, thus, documenting the incidence is important. Further, because fatherhood programs encourage men to become more responsible, we want to explore whether the programs had any effect on criminal involvement. Similar questions have been included in other large national studies, such as the Fragile Families and Child Wellbeing Study, the National Job Corps Study, the Building Strong Families Study, and the Parents and Children Together evaluation. In the Building Strong Families survey (the most recent completed study cited), nonresponse was less than 1 percent for these items (Wood et al. 2010).

Income

DCS-1

A key goal of RF and some HM programs is to improve participants’ economic stability. The outcomes of an individual employed when he/she enters the program may be very different than those of an individual who enters without employment. The applicant characteristics survey asks whether the respondent is currently working and, if so, the income they have earned in the past 30 days. Questions on earnings are asked on many surveys including the Building Strong Families survey (Wood et al. 2010). In this survey, only 0.4 percent of mothers and 0.1 percent of fathers did not respond to the earnings questions.



A12. Estimates of Annualized Burden Hours and Costs

Table A.4 provides the estimated annual reporting burden calculations for DCI (contractor data collection) and DCS (grantee data collection). Estimates are broken out separately as burden for HMRF program applicants and participants, and burden for HMRF grantee staff. The total annual burden for program applicants/participants (including application intake, participant pre- and post-tests, and participant focus groups) is estimated to be 180,090 hours, and the associated annual burden cost is estimated to be $1,610,705.

For all cost calculations, we estimate the average hourly wage for program directors and managers to be the average hourly wage of “Social and Community Services Manager” ($31.60), that of grantee staff to be the average hourly wage of “Social Workers” ($26.95), that of data entry specialists to be the average hourly wage of “Data Entry and Information Processing Workers” ($13.60), taken from the U.S. Bureau of Labor Statistics, Occupational Employment Statistics, 2013. The average hourly wage of HMRF program clients is estimated from the average hourly earnings ($4.92) of study participants in the Building Strong Families Study (Wood et al. 2010). These average hourly earnings are lower than minimum wage because many study participants were not working. We expect that to also be the case for the grantee clients.

The estimates in Table 4 are per year of the grant. The burden and costs for DCI data collection are annualized over three years, meaning the total number of respondents has been divided by three. Information for DCI data collection at stages one and three will impose no additional burden on respondents, as it will be extracted from grant applications the HMRF grantees will have prepared and submitted previously (stage 1) and document review of PPRs and QPRs (stage 3).

Table A.4. Estimates of Burden and Costs for the FaMLE Cross-site

Activity, by Respondent

Annual Number of Respondents

Number of Responses per Respondenta

Average Burden per Response (hours)

Total Annual Burden Hours

Average Hourly Wage

Total Annualized Cost

DCI (data collection by contractor)


DCI-1: Topic Guide on Program Design

20

1

1

20

$31.60

$632


DCI-2: Topic Guide on Program Implementation

100

1

1

100

$31.60

$3,160


DCI-3: Focus group Protocol

267

1

1.50

401

$4.92

$1,972


DCS (data collection by grantees)


DCS-1: Applicant Characteristics








Program applicants

137,125

1

0.25

34,281

$ 4.92

$168,663


Program staff

360

381

0.10

13,716

$26.95

$369,646


DCS-2: Grantee Program Operations

120

1

0.75

90

$13.60

$ 1,224


DCS-3: Service Receipt in MIS

120

678

0.50

40,680

$13.60

$553,248


DCS 4: SAQ Pre-Test and Post-Test








Program participants (pre-test)

111,675

1

0.42

46,904

$ 4.92

$230,767


Program participants (post-test)

90,130

1

0.42

37,855

$ 4.92

$186,246


Program staff (entry from paper)

12

1,412

0.30

5,083

$13.60

$ 69,129


DCS-5: Semi-annual Progress Report

120

2

3

720

$31.60

$ 22,752


DCS-6: Quarterly Performance Report

120

2

1

240

$13.60

$ 3,264










Total




180,090


$1,610,705


a Total number of responses per respondent is rounded to the nearest whole number.



The process for generating these burden and cost estimates, including assumptions regarding the number of respondents and periodicity of data collection, is described in Attachment G. We have maintained the same number of instruments and estimated time for completion, but we have made substantial changes in burden over those proposed in the 60-day Federal Register Notice in terms of the number of respondents. We also have increased the number of proposed focus groups, and adjusted burden for the semi-annual and quarterly progress reports to account for additional narrative detail grantees will be required to submit. The changes include:

  • Added burden for grantees to collect applicant characteristics and pre- and post-test data from control or comparison group members. We estimated that approximately 35 grantees will propose evaluation designs that include a control or comparison group. We had not included estimates for any comparison or control groups members in the 60-day Federal Register Notice. As the requirements and recommendations for the next round of grantees has been determined, there has been an increasing emphasis on conducting local evaluations, necessitating the addition of burden for comparison or control groups.

  • Increased burden for HM grantees serving youth, for the following reasons:

    • HM programs serving youth typically serve more individuals than HM programs serving couples. We had previously assumed that 60 HM grantees would serve, on average, about 300 couples year. In light of recent data showing the populations served by current grantees, we have revised our assumptions. We now estimate that 30 grantees will serve couples (on average about 300 couples per year), and 30 grantees will serve youth (about 2,200 youth per year).

    • Response rates for programs serving youth are likely to be higher than for other types of grantees, since the students will take the tests during regular class hours. Since most HM respondents will be youth, we increased our assumptions for HM response rates to 85% for the pre-test and 70% for the post-test (earlier assumptions were 75% and 60%, respectively).

  • We increased the number of DCI focus groups from one to four focus groups for each of 20 grantees, and adjusted burden estimates accordingly.

  • We increased the estimated burden for completing the semi-annual progress reports and quarterly progress reports to reflect the narrative that grantees will need to provide as context and detail for the quantitative measures. We had not previously included time to complete the narrative as the templates for the reports were still incomplete and did not yet include these sections.

A13. Estimates of Other Total Cost Burden to Respondents and Record Keepers

These information collection activities do not place any additional costs on respondents or record keepers other than those described above.

A14. Cost to the Federal Government

If all core and optional service components are exercised over the seven-year project period, the total value of the FaMLE Cross-site to the federal government for information collection, design, development, tests, printing forms, editing, coding, tabulation, analysis, publication of results, salaries, and travel is $2,871,792, and the annualized cost to the federal government is $410,256. Broken down by task, the cost for data collection by the contractor (DCI) is $535,747 (3751 hours), the cost for data collection by the grantees themselves for performance reporting and cross-site analysis (DCS) is $1,172,340 (7157 hours), the cost for the cross-site analysis is $667,334 (4028 hours), and the cost for dissemination is $496,371 (2819 hours).

A15. Explanation for Program Changes or Adjustments

This is a new data collection. There are no changes or adjustments.

A16. Plans for Tabulation and Publication and Project Time Schedule

A. Plans for tabulation

For data collected through both DCI and DCS, we will follow standard protocols for cleaning data, constructing variables that address the project’s purposes and research questions, and computing descriptive statistics. Additional plans for tabulation of DCI and DCS data are described below.

1. DCI

The contractor will use standard qualitative procedures to analyze and summarize information from telephone interviews and interviews conducted using the semi-structured interview topic guide (for program staff), and the focus group discussion guide. Analysis will involve organization, coding, triangulation, and theme identification. For each qualitative data collection activity, standardized templates will be used to organize and document the information and then code this documentation. Coded text will be searched to gauge consistency and triangulate across respondents and data sources. This process will reduce large volumes of qualitative data to a manageable number of topics/themes/categories (Yin 1994; Coffey and Atkinson 1996) which can then be analyzed to address the study’s research questions.

2. DCS

Grantees will be able to produce quarterly tabulations of the information below as necessary for reporting performance to ACF. The contractor will tabulate data as requested by ACF for conducting cross-site analyses. To achieve these purposes, we anticipate conducting the descriptive analyses presented below.

  • Applicant characteristics (Instrument DCS-1). Summary statistics within and across all HMRF grantees on applicant characteristics—for all program applicants, and for those who ultimately participate in program services. For HM grantees, some applicant characteristics will be calculated at the couple level. For grantees conducting impact studies, t-tests will assess baseline equivalence in applicant characteristics between program and control/comparison groups.

  • Enrollment (Instrument DCS-3). For example, the number enrolled in the program in the previous quarter, and the total number enrolled in the program since the beginning of the grant year. For HM grantees, enrollment will be calculated at both the individual and couple levels. For grantees conducting impact studies, numbers enrolled in the control/comparison group will also be tracked.

  • Program participation (Instrument DCS-3). For example, the proportion of enrollees who attend a core workshop within two months of enrollment, and the average number of hours of services received by program participants. For HM grantees, program participation will be calculated at both the individual and couple levels.

  • Program operations (Instrument DCS-2):

    • Marketing, outreach, and recruitment. Summary statistics within and across all HMRF grantees on the marketing strategies, recruitment methods, and referral sources used.

    • Staff Characteristics. Summary statistics on the proportion of program staff with various levels of education and experience.

    • Quality assurance and monitoring. Summary statistics within and across all HMRF grantees on measures of staff training, supervision, and observation of program services.

    • Implementation challenges. Summary statistics within and across all HMRF grantees on the degree to which potential implementation challenges have been a problem.

  • Participant outcomes (Instrument DCS-4). Summary statistics within and across all HMRF grantees on outcomes collected at program entry and exit (or one-month post program exit, for programs shorter than one month). For HM grantees, some outcomes will be calculated at the couple level.

  • Program impacts (Instrument DCS-4). For the estimated 35 HMRF grantees conducting experimental or strong quasi-experimental evaluations, we will also estimate impacts of the programs. We anticipate that these grantees will provide our research team with individual-level data on participants. This structure makes it feasible to identify program effects using ordinary least squares (OLS) (for linear outcomes) and probit regressions (for binary outcomes), such as

and

.

Yig represents some outcome of interest for individual i served by grantee g, Treat indicates an individual’s treatment status, X represents demographic controls such as age and race, the δg variables are grantee-fixed effects, and Φ indicates the standard normal distribution. The key coefficient from the OLS regressions is β, or the change in an outcome associated with treatment status. (We will propose reporting marginal effects, interpretable as the change in associated with a change in X for the average individual, when we use probit specifications).

B. Time schedule and publications

The schedule for FaMLE Cross-site data collection and reporting is shown below in Table A.5.

Table A.5. Schedule for the FaMLE Cross-site Project

Activity*

Date

Grantee applications and awards

March 31, 2015 (est.) – September 30, 2015

DCI data collection


Program design data collection

Summer 2016

Program implementation data collection

Summer 2018-Summer 2019

Participant focus groups

Summer 2018-Summer 2019

DCS data collection*


Applicant Characteristics

On-going, Spring 2016 – September 2020

Program Operations

Spring 2016, updated quarterly through September 2020

Service delivery data

On-going, Spring 2016 – September 2020

Pre-test instruments

Post-test instruments

On-going, Spring 2016 – September 2020

On-going, Spring 2016 – September 2020

Quarterly Performance Report

Semi-Annual Program Performance Report (PPR)

Summer 2016, updated quarterly through Winter 2020

Fall 2016, updated semi-annually through January 2020

Reports


Program design report

Fall 2017

Program implementation report

Fall 2018

Program outcome report

Spring 2020**

Note: Collection beyond the three years of OMB approval contingent upon continued approval.

*Although grants are expected to be awarded September 30, 2015, grantees will have a six-month planning period and thus will begin data collection March 1, 2016.

**We expect to apply for an extension.



In addition to the planned reports described above, the FaMLE Cross-Site Project provides opportunities for analyzing and disseminating additional information through special topics reports and research or issue briefs on an as-requested basis. We will also provide a restricted use data file for others to replicate and extend our analysis.

A17. Reason(s) Display of OMB Expiration Date Is Inappropriate

All instruments will display the expiration date for OMB approval.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

REFERENCES

Avellar, S., A. Clarkwest, M. R. Dion, S. Asheer, K. Borradaile, M. Hague Angus, T. Novak, J. Redline, H. Zaveri, and M. Zukiewicz. “Catalog of Research: Programs for Low-Income Couples.” OPRE Report # 2012-09. Report submitted to the Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Washington, DC: Mathematica Policy Research, May 2012.

Avellar, S., M. R. Dion, A. Clarkwest, H. Zaveri, S. Asheer, K. Borradaile, M. Hague Angus, T. Novak, J. Redline, and M. Zukiewicz. “Catalog of Research: Programs for Low-Income Fathers.” OPRE Report # 2011-20. Report submitted to the Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. Washington, DC: Mathematica Policy Research, December 2011.

Berlin, Martha, Leyla Mohadjer, Joseph Waksberg, Andrew Kolstad, Irwin Kirsch, D. Rock, and Kentaro Yamamoto 1992 An experiment in monetary incentives. Pp. 393-398 in Proceedings of the Section on Survey Research Methods. Alexandria, VA: American Statistical Association.

Buhi, E.R. and P. Goodson. “Predictors of adolescent sexual behavior and intention: a theory-guided systematic review.” Journal of Adolescent Health, 40(1): 4-21, 2007.

Coffey, A., B. Holbrook and P. Atkinson (1996) 'Qualitative Data Analysis: Technologies and Representations', Sociological Research Online, vol. 1, no. 1, <http://www.socresonline.org.uk/ 1/1/4.html>.

Dion, M. R., B. Devaney, M. Ford, H. Hill, S. McConnell, and P. Winston. “Helping Low-Income Families Build Strong and Healthy Marriages: A Conceptual Framework for Interventions.” Washington, DC: Mathematica Policy Research, January 2003.

Downey, G. and J.C. Coyne. “Children of depressed parents: An integrative review.” Psychological Bulletin, July, 108(1): 50-76, 1990.

Edin, Kathryn and Maria Kefalas. Promises I Can Keep: Why Poor Women Put Motherhood Before Marriage. Berkley and Los Angeles, CA: University of California Press, 2005.

Gelfand, D.M. and D.M. Teti. The effects of maternal depression on children. Clinical Psychology Review, 10, 329-353, 1990.

James, Jeannine M., and Richard Bolstein.1990 The effect of monetary incentives and follow-up mailings on the response rate and response quality in mail surveys. Public Opinion Quarterly 54:346-361.

Kessler, R., G. Andrews, L. Colpe, E. Hiripi, D. Mroczek, S. Norman, E. Walters, and A. Zaslavsky. “Short screening scales to monitor population prevalences and trends in non-specific psychological distress.” Psychological Medicine, 32, 959-976, 2002.

Lundquist, Erika, JoAnn Hsueh, Amy Lowenstein, Kristen Faucetta, Daniel Gubits, Charles Michalopoulos, and Virginia Knox. A Family-Strengthening Program for Low-Income Families: Final Impacts from the Supporting Healthy Marriage Evaluation. OPRE Report 2014-09A. Washington, DC: Office of Planning Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2014.

McLanahan, S., I. Garfinkel, N. Reichman, J. Teitler, M Carlson, and C. A. Audigier. “The National Report: The Fragile Families and Child Wellbeing Study Baseline Report.” Princeton, NJ: The Center for Research on Child Wellbeing, August 2001.

Raley, R. K.; Crissey, S.; & Muller, C. (2007). Of sex and romance: Late adolescent relationships and young adult union formation. Journal of Marriage and Family, 69(5), 1210-1226.

Tourangeau, R., & Smith, T W. (1996). Asking sensitive questions: The impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60, 275-304.

Turner, C., Ku, L., Rogers, S. M., Lindberg, L. D., Pleck, J. H., & Sonenstein, F. L. (1998). Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science, 280, 867-873.

Wood, Robert G., Sheena McConnell, Quinn Moore, Andrew Clarkwest, and JoAnn Hsueh. “Strengthening Unmarried Parents’ Relationships: The Early Impacts of Building Strong Families.” Princeton, NJ: Mathematica Policy Research, May 2010.

Yin, R. (1994). Case study research: Design and methods (2nd ed.). Thousand Oaks, CA: Sage Publishing.

Zaveri, Heather, P. Holcomb, R. Dion, D. Friend, and R. Selekman. “Fathers’ motivations for enrolling and engaging in responsible fatherhood programs: Insights from the Parents and Children Together (PACT) evaluation.” Presentation for the Welfare Research and Evaluation Conference, May 2014.






ATTACHMENT A


LEGISLATIVE AUTHORITY


ATTACHMENT B


60-DAY FEDERAL REGISTER NOTICE


APPENDIX C


SUMMARY OF PUBLIC COMMENTS AND ACF RESPONSES


APPENDIX D


CONSENT STATEMENT FOR HM AND RF FOCUS GROUP PARTICIPANTS


ATTACHMENT E


RECOMMENDED LANGUAGE FOR GRANTEE CONSENT FORMS


ATTACHMENT F


MATHEMATICA EMPLOYEE CONFIDENTIALITY PLEDGE


ATTACHMENT G


ASSUMPTIONS FOR CALCULATION OF BURDEN ESTIMATES IN TABLE A.4

ATTACHMENT H

FOCUS GROUP RECRUITMENT LETTER AND

TELEPHONE REMINDER SCRIPT

1 Other evaluations of HMRF programs include the Community Healthy Marriage Initiative, which assessed the implementation and outcomes of a community-wide approach to strengthening relationships; and the Ex-Prisoner Reentry Strategies Study, which assesses program implementation.

2 In addition to the activities described, ACF may conduct limited additional analyses on special topics, e.g. marketing and recruitment strategies associated with greater enrollment and participation.



File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorFelita Buckner
File Modified0000-00-00
File Created2021-01-24

© 2024 OMB.report | Privacy Policy